At Datalight, we frequently find ourselves helping customers on what we call 'rescue missions' – when a device is failing in the field and the design team is under pressure to quickly resolve a data corruption or data loss issue. Many times, the failure happens because data didn't get to the media, usually because a cache or other performance optimization has delayed those slow flash writes. In our recent presentation, we examined reliability on Linux with a focus on when the data is on the media.
Last week, we recorded a web seminar of the talk Datalight gave at Embedded World 2018 on techniques for effective power interruption testing. There were some excellent questions at that session, and this blog post answers a few of them in more detail.
Recently a failed update once again emphasized the importance of testing all aspects of product updates before going live with them. This can be the best strategy to avoid angry users who quickly take to social media to express their frustrations.
Data corruption happens when the media or the file system fail in some way. While this is bad enough, there can be serious security complications afterwards. Our blog post looks at the problem and the solution.
In another demonstration of an update failure (something that seems to happen monthly), the company LockState pushed out a firmware update for the wrong model earlier this month. LockState manufactures door locking systems that can be remotely managed, and many of the affected devices were installed in AirBnB locations, in partnership with Host Assist.
More and more embedded devices are gaining the ability to connect – to each other, to private networks, and to the public cloud. This increasing connectivity is creating a fresh ease of delivering software updates to all kinds of devices that have been deployed to the field. While no one would argue that the ability to provide timely security updates is a bad thing, we need to be careful that this ubiquitous updatability doesn’t tempt us to ship a product before it’s really ready. And more importantly, that we plan for a failsafe way to manage updates when they do happen.
Embedded devices today are performing more frequent updates, including monthly security updates for Android devices. One concern for any update is the potential for failure, with the worst case failure leaving the device a useless brick.
Recently, William Lamie published a mythbusting piece that examined the use of an RTOS in IoT devices on Electronic Design. His insightful comments apply to most device designs complex enough to use a microprocessor. Which, these days, is pretty much all of them. We’ve run into a lot of the same myths around one crucial component of the RTOS – the file system.
Perhaps the most widely held belief about write caching is that it makes a system unreliable. Something along the lines of "Data written to a cache instead of to the media will be lost on an unexpected power interruption, leaving the system in a damaged state and rendering data useless." There are a variety of options busting this myth.
Today automotive OEMs—and by extension, their suppliers—are being held to a standard that demands absolute answers whenever a component or system fails. The safeguards built in and care taken in developing these systems is high and failures are a rare occurrence, only small fractions of a percentage.
One of the key differentiators for Datalight's Reliance Nitro file system is the runtime flexibility. Not only does this file system provide more reliability options than any other file system on the market, they can all be changed on the fly through a simple API. To demonstrate just how easy this was, we created an intern project to do just that.
The Mars Opportunity Rover was in the news again this week, as NASA mission engineers try to overcome what they refer to as an increasingly troubling bout of rover "amnesia". In September of 2014, the team reformatted the flash memory.
Datalight developers know the danger of power interruption on embedded devices, and we also know the FAT file system well. In my 10 years as a software engineer on ROM-DOS, the internals of our FAT implementation were of daily interest.
Datalight's FlashFX Tera is quite similar in structure to its predecessor, FlashFX Pro, but there have been some features that have evolved from our experience in working with NAND flash over the years that are reflected in some of the new features you will now find in our FlashFX Tera product, and the Error Policy Manager is one of them.
On Linux, the Memory Technology Device (MTD) subsystem is designed to be a generic interface to memory devices, primarily Flash media. The integrated hardware driver handles the storage formats used on the media, and then MTD provides simple routines for block read, write and erase. MTD does not contain any bad block handling or wear leveling routines, so the use of MTD alone is not recommended on NAND flash media. Instead, developers are asked to use a full Flash File System on Linux, such as YAFFS, JFFS2 and UBIFS.
In an earlier blog post titled "Managed NAND Performance: It's All About Use Case," we referred to an article measuring SD media, specifically sequential write performance. Speed was measured using a camera to shoot continuous pictures. Photographers and other users of SD media have another concern - reliability.
Datalight's Reliance Nitro and journaling file systems,such as ext4, are designed to recover from unexpected power interruption. These kinds of "post mortem" recoveries typically consist of determining which files are in which states, and restoring them to the proper working state. Methods like these are fine for recovering from a power failure, but what about a media failure?
If you've been following this blog, you've probably noticed a lot of discussion and analysis around eMMC. We've written about the reasons we are so excited about eMMC, but also why the Write Amplification issues caused by eMMC parts are a problem that needs more attention by the industry.
The challenge - making ext4 just as reliable as Datalight's Reliance Nitro file system, within limitations of the POSIX specification. Unlike most real world embedded designs, performance and media lifetime are not a consideration for this exercise.
If you're on LinkedIn, check out the Realtime Embedded Engineering Group for an interesting and often lively discussion of the issues facing our community. We particularly enjoyed reading the recent thread about the drivers included by hardware vendors being less than optimal for most flash parts. The consensus can be summed up in one blogger's statement;
"What many silicon vendors refer to as a 'driver' is nothing more than the code left over by their inhouse hardware development team. This code typically exercises just a small subset of the device capabilities (or whatever they were working on last) and doesn't even come close to meeting the definition (or spirit) of a general purpose device driver."
We couldn't have said it better ourselves. A lot of our time in developing our FlashFX family of flash memory drivers is spent…
According to a recent Nielsen/Netratings report, 89.4% of us connect our gadgets to a Windows host machine, underscoring how crucial it is for manufacturers to consider file system compatibility with the Windows desktop.
With the release of our new file system this week, Reliance Nitro, we asked our Account Managers what they liked most about our new product. Their answers of course included reliability and high performance. Wes Johns and Phillip Allison were so excited they decided to make a video watch the youtube video
We're totally psyched about Reliance Nitro, our newest file system (yes, we're file geeks), and we're always on the lookout for opportunities to show off the performance and reliability attributes it adds to Windows Mobile and Windows CE. When we discovered the relatively new Beagle Board, it occurred to us that a small, low-cost platform might be just the thing to demonstrate Nitro's amazing benefits. As you've probably heard, the Beagle is making waves with its low cost (around $150) and diminutive size. It uses an OMAP 3530 processor and 256MB of NAND. Though they are most commonly used with Linux, we lucked out in having a partner (MPC Data) who has already developed a Windows CE BSP for it. After a few phone calls, the wizards at MPC Data were able to develop a slick video playback demo app, and presto, the Reliance Nitro Beagle Demo was born! Amateur videographers that we are (ok, REALLY amateur), we recently videotaped…
Everyone knows that NAND has challenges: from factory bad blocks and spontaneous bit failures to endurance limits, etc. That's why a few years ago managed NAND (NAND flash plus an integrated controller) seemed to be the answer, offering the density of raw NAND, while mitigating many of its inherent limitations.
There are two possible configurations in how boot code might be stored on a device: 1.) Boot code is stored in raw flash (no file system) and directly accessed from bootloader, 2.) Boot code is stored on a Reliance formatted flash volume
Demonstrating how system software work in a visual manner is an interesting problem, especially in embedded space. There is no UI or visual effects to WOW the audience. To evaluate the value system software components bring to an embedded design, the customer usually needs to configure our software on his embedded development board.
Continuing the conversation started in Flash File Systems and JFFS2 blog posts, this post talks about a YAFFS, another Linux flash file system alternative. YAFFS (Yet Another Flash File System) was designed to solve some of the performance issues suffered by JFFS2 on NAND flash.
Linux has been slowly but surely establishing itself as the predominant OS in the embedded industry. ABI research report suggested that 23% of Smartphones will be based on Linux by 2013. High-profile industry support from Android and the LiMo foundation has put the spotlight back on embedded Linux.