Chunks

To check the checksums regularly on all media in the RAID, you can use the line

btrfs scrub start path | device

in a cron job. In the case of a read or checksum error, the data is automatically read from another copy and newly created in the faulty replica on the fly for RAID 1 and RAID 10.

Btrfs reserves storage space for data and metadata in fixed units known as block groups. Block groups in turn consist of one or more chunks (depending on the RAID level). These chunks are usually 1GB for data and 256MB for metadata. One exception is the first metadata chunk, which mkfs.btrfs creates 1GB in size, assuming there is enough space. Intermediate sizes are used for block devices of just a few gigabytes.

This means a total of 18GB of disk space is already assigned on the filesystem created previously – 16x1GB user data, 2x1GB metadata (with the duplicate) – with a small quota for the special system chunks (where the B-tree that manages all the chunks resides). For filesystems of less than 1GB, Btrfs automatically uses special mixed chunks that contain both user data and metadata to better exploit the meager space.

The metadata on this filesystem does not occupy 80MB yet, but the filesystem also has not seen any production use. Once chunks were assigned to a data type, they were (until now) not automatically released until no longer needed. When you create new data, this process can lead to a filesystem that has insufficient reserve capacity, according to df -h, which responds earlier than expected with an error message that it has run out of free space (except in the metadata areas). These are the notorious ENOSPC errors reported by many Btrfs users that were incorrectly generated much more frequently in the past.

In Btrfs, keep an eye on the

btrfs fi show

and

btrfs fi df mountpoint

output to know how much space has been used and to discover potentially free space on a filesystem. The good old df -h is only of limited value in Btrfs, especially when it comes to reliable statements about available space.

Make Room!

If you need to create space by recycling chunks, you have only one option, and that is launching the btrfs balance command. Unfortunately, you cannot determine in advance how much space can be recovered, and you have no option to simulate the action, but at least this command can be applied to a filesystem on the fly (Figure 4):

btrfs balance start /mnt &

Figure 4: Balancing can free up unneeded but assigned chunks. Running btrfs fi df and btrfs fi show – before and after balancing – show that the metadata chunk size has been reduced to 256MB; thus, 1.5GB (2x750MB) of space has been made available.

Your mileage for balancing the filesystem will vary depending on the fill level and system performance. All data is completely read and written to new chunks, so for a successful balance, you still need a bit of free space, at least enough to copy the first selected block group, including enough space for the metadata created during the COW process.

Only with the Current Kernel

If the load caused by running balance in the background is worrisome, you can cancel the action; for this example, you would need to issue a

btrfs balance stop /mnt

to do so. Note that all the commands for balance except start generate an error on Ubuntu 12.04. To fix this, you need a more recent kernel. Newer kernels as of version 3.3 also let you stop the balance temporarily and continue later, as well as display the current progress for the process.

An involuntary interruption to a balance action (e.g., a power outage) is not critical, and the action will continue automatically the next time the filesystem is mounted (as of kernel 3.3, the mount option skip_balance prevents this).

In production operations with Btrfs, you need to allow extra space for metadata when dimensioning a new filesystem. How much you need to allow is probably something learned by experience. However, many small files will cause a large volume of metadata because the data is stored directly in the metadata areas (also empty files). Snapshots are also very demanding in this respect.