Explaining OverlayFS – What it Does and How it Works

Posted by: Thom Denholm

Union file systems are a creative solution to allow a virtual merge of multiple folders, while keeping their actual contents separate. The Overlay file system (OverlayFS) is one example of these, though it is more of a mounting mechanism than a file system.

Brought into the Linux kernel mainline with version 3.18, OverlayFS allows you to overlay the contents (both files and directories) of one directory onto another. The source directories can be on different volumes and can even be different file systems, which creates an interesting mechanism for allowing temporary modification of read-only files and folders.

The simplest case (image below) involves two directories, each containing files and folders. We can think of them as “upper” and “lower,” with the rest of Linux and applications positioned above that. The “lower” directory is read-only. File access through the OverlayFS retrieves data from the “upper” directory first, and then defaults to the “lower” directory if a file doesn’t exist.

Linux OverlayFS allows you to overlay the contents of one directory onto another

Note that the two original directories (“upper” and “lower”) are still directly accessible to the Linux kernel, but this access could be limited by the application.

Modifications to files in the “upper” directory will take place as usual. Any modification to a file from the “lower” folder will create a copy in the upper directory, and that file will be the one modified. This leaves the base files untouched and available through direct access to the “lower” folder.

Interestingly, a second task could copy modified files from the “upper” folder to the “lower” when modifications are complete. In this way, an OverlayFS setup could simulate some of the functionality of transaction points within the Reliance Nitro file system.

A file removed from the OverlayFS directory would directly remove a file from the “upper” directory, and simulate that removal from the “lower” directory by creating what is called a “whiteout” file. This file exists only within the OverlayFS directory, without physically appearing in either the “upper” or “lower” directories. When the OverlayFS is dismounted, this state information will be lost, so care should be taken to reflect any necessary changes to the “lower” directory.

A subdirectory can also be deleted from the “lower” directory, which creates what is known as an “opaque directory” in the OverlayFS directory. Behind the scenes, OverlayFS uses the “trusted” extended attribute class or namespace to record “whiteouts” and “opaque directories.” Linux file systems that support the “trusted” namespace can be used for either, and Reliance Nitro is among that set.

Have you used OverlayFS in your projects? Drop me a line or comment below.

Discover Reliance Nitro

Comments (3)

  1. Olga Levy:
    May 21, 2018 at 02:33 AM

    How can I implement a fitering mechanism so that certain files will be written in lower dir?

  2. Thom Denholm:
    May 22, 2018 at 10:00 AM

    Offline changes are allowed to either the upper or lower file systems, but the "lower" directory is meant to be read-only while the OverlayFS is mounted. Per the documentation, if the underlying filesystem is changed, the behavior of the overlay is undefined.

    See https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt

  3. ranran:
    Oct 10, 2018 at 02:59 PM

    Hello Thom,

    Thank you for the tutorial.
    Maybe you can help with a problem I have ?
    I have an issue related to overlay filesystem, and I don't understand how to overcome it. :cry:
    I hope someone can help...

    I have vncserver (tigervnc), and in startup I see that vncserver service has started for all users (root+user) have vncserver (tigervnc).

    Recently we added overlayfs using init script as done here: https://blockdev.io/read-only-rpi/

    As you can see in above link, the overlay is above rootfs (rootfs is readonly).

    At first, it seemed to me that all works the same (as without overlay).

    But today I noticed a strange thing: With overlay rootfs, only the root's vncserver service has started, not the user's vncserver. I have no idea why.

    I actually assume that EVERYTHING should be the same whether I use overlay or not, including all the started services.

    What can be the reason for this change in behavior of started services per user with overlay ?

    Thanks for any idea,

Add a Comment

Allowed tags: <b><i><br>Add a new comment: