Recently we ran benchmarks comparing ext3 and ext4 to Reliance Nitro on eMMC and Linux. Here's what we learned.
IOzone Test - Reliance Nitro compared to ext3/4 on eMMC
Operating System: Linux 2.6.39 with eMMC enhancements Reference Platform: OMAP 4430 PandaBoard Memory: Socketed eMMC via SD adapter Linux SanDisk 4GB iNAND with 3 partitions
- 64 MB boot partition
- 512 MB root file system
- 3.4 GB test partition
Testing was run on three different PandaBoards: two Rev A3 boards and one Rev A2 board in two different configurations that reflect distinctly different operating environments that Linux file systems may experience. Each test configuration was run against each file system twice on each target board, for a total of six tests per file system/configuration. The six sets of test results for each file system/configuration were then averaged together, graphed and analyzed. The configurations were as follows: Configuration 1 - Low RAM, fsyncIn this configuration the target system had little available RAM, and the test issued the fsync command to the file system to force data to be written out to the media. This measures the performance of the file system when only small amounts of data can be cached and the data must be written out to the media. The goal of this configuration is to measure the file system overhead.
- Designed to show hardware throughput
- Embedded Device or High Reliability Device Use Case
- Boot Arguments: root=/dev/mmcblk0p2 rw rootwait mem=64M console=ttyO2,115200n8 init=/sbin/init earlyprintk
- Test Arguments: iozone -Raze -+w 1 -q 16m -g 128m
Configuration 2 - High RAM, no fsyncThis test configuration had large amounts of available RAM and did include the fsync command. This measures the performance of the file system when large amounts of data can be cached and the data need not be written to the disk; therefore measuring the file system interaction with the cache.
- Reflects performance through internal caches
- Closer to desktop or data center use case
- Boot arguments: root=/dev/mmcblk0p2 rw rootwait mem=463M console=ttyO2,115200n8 init=/sbin/init earlyprintk
- Test Arguments: iozone -Raz -+w 1 -q 16m -g 128m
- Benchmarks not a normal use case
- IOzone creates a single file, then deletes it before creating the next
- Tests were run on freshly formatted disk
- Test runs were single threaded
- IOzone can test multithreaded
- Reliance Nitro is protecting user data; ext3/ext4 journals only the file metadata
It is important to note that IOzone and the tests run are for benchmarking purposes and do not attempt to measure real world use case performance. IOzone creates a single file, and deletes it before creating the next, and the test was run against a freshly formatted disk. Both test configurations used were single-threaded, while this is the default "automatic" behavior for IOzone, it is unlikely for a real world use case. Target systems have many files that are accessed in different ways, often all at once. Additionally, IOzone measures performance and in no way compares the data at risk for a given file system. For all tests run, Reliance Nitro is protecting user data and file data via transaction points while ext3 and ext4 are only protecting file metadata via the journal.
Performance data was generated using IOzone v3.397. The test results are graphed in a manner that allows side-by-side comparison of each file system's performance in each test case. The test executes the test cases against file sizes that are powers of 2 from 64 KB - 128 MB, and record sizes that are also powers of 2 from 4 KB - 16 MB. The vertical axis of the graphs shows the performance measure during the test, in units of kilobytes per second (KB/s). The horizontal axis shows the file sizes, and the data points between the file sizes represent the record sizes. For example, the first data point (immediately above the 64 KB mark) is for 64 KB files with 4 KB records, the next is for 64 KB files with 8 KB records, and so on.
In both test configurations, the read performance of all three file systems is largely comparable. With the sequential read test, all three file systems appear to be operating outside the cache at 32 MB file size and greater which is the point where the files are too large to be cached. Ext3 and ext4 are operating within the cache on 8 MB and 16 MB file sizes. All three file systems are within the cache below 8 MB. A likely reason for Reliance Nitro operating outside the cache at smaller file sizes than ext3 and ext4 is that Reliance Nitro allocates more RAM for private structures therefore starving the cache sooner.
Sequential writes are comparable in Configuration 1, in Configuration 2 ext3 and ext4 perform better due to more optimized integration with cache. With random writes in Configuration 1, Reliance Nitro outperforms at larger file sizes perhaps due to the extent based design, this could also explain ext4's advantage over ext3 since it is also extent based. In Configuration 2, ext3 and ext4 perform random writes faster due to better integration with the cache. IOzone Testing Conclusions
- Read performance of all three file systems is comparable
- Configuration 1: sequential writes are comparable, random writes Reliance Nitro outperforms at larger file sizes
- Configuration 2: ext3 and ext4 perform better on sequential and random writes due to Linux cache integration
Linux has a very cache-centric I/O architecture that, with proper support in the file system, leverages available RAM to avoid doing actual I/O to the storage media. The media used in this test is capable of reading at about 25 MB/s and writing at about 11 MB/s. Measured performance in excess of those constraints indicates that some or all of the data is being cached. In Configuration 1, all three file systems tested perform comparably on reads and sequential writes. For random writes Reliance Nitro outperforms ext3 and ext4 and 16 MB file sized and above likely due to the extent based design of Reliance Nitro. In Configuration 2, there is ample available RAM to cache the test files and the test is not forcing the data to be written to the disk with fflush/fsync, so the read and write performance of all three file systems consistently exceeds the disk's capabilities. Ext3 and ext4 perform the same in almost all test cases, but Reliance Nitro performs slower in write cases. This is because Reliance Nitro integration with the Linux page cache is not optimized, resulting in a less efficient cached performance. While optimal use of the Linux cache is valuable in terms of performance; cache utilization and its costs and risks must be considered for the individual use case. Using the cache is RAM intensive meaning it can add both BOM cost in the form of extra memory needed as well as additional power consumption cost. Cached data also adds a reliability risk. When data is cached, it is not flushed to the media so it is vulnerable in the event of power loss. Reliance Nitro can dynamically mitigate this risk with transaction points, although further cache optimization is required to boost performance for some use cases.