Subscribe to Datalight's Blog


 

File System Sculpting

For many embedded operating systems, the default file system is the only choice available. Sometimes a few customization options exist, and usually only at mount time. When using Linux as an embedded OS, there are a few more customization options and the source is available, but it is rarely easy to understand and modify. Reliance Nitro is an embedded file system that can be sculpted to fit where you need it.

Datalight has a strong history of customization. Our ROM-DOS operating system was designed to offer a variety of choices to the device designer. FAT32 and Long Filename Support were both optionally available, together or separate. Internal files, buffers and other structures could also be customized at will – without losing complete compatibility with MS-DOS.

This same strength is available in both our FlashFX and Reliance families of products. The latter offers many options for customization, from cache, buffers and debug mode to filename and pathname length and character sets. Our reliability through fully configurable Transaction Points is unlike any other file system in the industry.

One recent customer came to us with a request for our fantastic reliability – in under 60k of RAM. Datalight’s Reliance Nitro code was trimmed down to remove multiple threads, multiple volumes and unnecessary API calls. International character support and the idea of a current working directory were also removed. The end result provided perfect reliability in just 55k.

These and many other customization options are available to our customers, either on their own (with the friendly assistance of Datalight Support) or through a Datalight services project. When you take a look at our source code, you will find the Operating System and Block Device interfaces well documented and flexible. The Reliance Nitro source is available for free evaluation in your design today.

Learn More About Datalight Services

Thom Denholm | January 8, 2014 | Datalight Products, Flash File System | Leave a comment

FlashFX Tera Feature Highlight: The Error Policy Manager

Datalight’s FlashFX Tera is quite similar in structure to its predecessor, FlashFX Pro, but there have been some features that have evolved from our experience in working with NAND flash over the years that are reflected in some of the new features you will now find in our FlashFX Tera product, and the Error Policy Manager is one of them.

FlashFX Pro was originally developed in the day where NAND flash was SLC (Single-Level Cell), and required 2-bit detect, single bit correct ECCs. The error rates on these types of flash devices were much lower when compared to today’s newer NAND flash memory – both newer, lower lithography SLC flash designs that require higher levels of ECCs than their predecessors, and the introduction of MLC (Multi-Level Cell) NAND flash devices, in which multi-bit errors can be seen as part of normal device operation.

When single bit errors occurred in FlashFX Pro, these errors were eventually “scrubbed” over time as part of wear leveling. With newer, more error-prone NAND flash, a more active role needs to be taken.

The usual recommendation for scrubbing a block on NAND flash is to read the data, correct any bit errors using Error Correcting Codes (ECC), and then write the corrected data to an unused block. The existing block is then marked for erasure and reuse. Erasure clears any charge leakage or disturbance, which can make the cell appear to be inconsistently programmed.

In modern MLC NAND flash, errors occur more frequently. Partial programming and associated pages can also cause errors to occur. The severity of an error (and the recommended response) can also vary among manufacturers. FlashFX Tera needed some feature which would allow customization of how these different levels of errors were handled for different devices.

FlashFX Tera introduced the Error Policy Manager, a layer which can react to bit errors during a read and decide (based on the configured Error Policy) whether or not a page has reached the threshold to require action “soon” or “now”. Other functions that the Error Policy Manager can employ in addition to scrubbing include Relocate the Data, Retire the Block and Abandon Operations.

Error Policies can be specified for any number of bits, from zero to the maximum supported by the media and NAND controller. This flexibility allows the device designer to completely control how their software works with media from any NAND flash vendor. Error handling requirements can easily be translated from the design document into the software stack.

Datalight’s Error Policy Manager works equally well with new higher ECC requirement SLC NAND flash. At present, the Error Policy Manager requires the Datalight VBF layer be present on the media. Correctable errors in any NAND flash exempted from this coverage must currently be handled by the OS read routines.

Learn more about FlashFX Tera

Thom Denholm | December 13, 2013 | Datalight Products, Extended Flash Life, Flash Memory, Flash Memory Manager, Reliability | Leave a comment

2013 Embedded Tech Favorites

Wrapping up another busy development year at Datalight, it occurs to us that as much as we love helping our customers with software to handle their critical data needs on things like smart meters, medical infusion pumps, factory controllers, automobile infotainment systems and first responder radios, there are some cool embedded technologies we’d like to have in our personal Christmas stockings in 2013.

Here is a list of some of our favorite uses of embedded tech for consumers in 2013:

Misfit Shine ($119.95): The recent push for devices to help us track our activity and improve our fitness has created a flood of embedded technology hitting the market helping us accomplish all our wellness goals. Started in 2011 by the creators of a blood glucose monitoring system that works with iOS devices, Misfit Wearables is not just focused on fitness but wants their devices to become a part of your wardrobe and a must have accessory. It’s easy to see on Kerri McConnell, VP of Sales and Marketing, “I love mine. It’s the most elegant tracker I’ve ever used and combines sophisticated function with understated form”. The Shine, launched in 2013, can not only be worn as the traditional bracelet, but also as a broach, necklace or even attached to clothing with a hidden magnet.

Chromecast ($35.00): This July, Google hit the market with Chromecast, turning any TV into a streaming device, making Chromecast a huge hit on the market and an interesting choice for users who needed to update their technology at home. As TV and movie watching turns to the cloud, transferring that data to your older TV sets hasn’t always been easy. Rachel Householder, Senior Customer Care Administrator, uses hers to stream music, “We’ll create a YouTube playlist and use Chromecast to broadcast it over our home theater system during parties giving us a visual as well as sound. It’s a great addition to the festivities!” Chromecast allows users to communicate from multiple platforms, and now using multiple services like Hulu, VEVO and Redbox Instant. Chromecast is compatible with WiFi-enabled Android 2.3+ smartphones and tablets; iOS 6.0+ iPhones, iPads, and iPods; Chrome for Mac® and Chrome for Windows®; and Chromebook Pixel. Shipping for less than $35 dollars from some providers, Chromecast is also a very affordable gadget that embraces all that embedded technology can provide.

TiVo Roamio ($199.99 to $599.99): When you can’t be there to watch your favorite TV shows (or in Tony’s case, the Seahawks games), the TiVo Roamio has you covered! TiVo has created a device released in August of 2013 that now allows you to have your shows and recordings wherever you go. Currently supporting three different models, the TiVo Roamio has all sort of embedded functionality for the user, connecting to Wi-Fi, built in music servers, photos and home movies. The TiVo Roamio allows you to really have your TV shows anywhere so you don’t even have to think about missing your favorite shows and episodes! To quote our Customer Support and Quality Services Architect, Tony Questad, “It’s really slick!” With models supporting up to 3TB of data, consistent embedded memory management is a must!

iSmartAlarm ($199 and up): Speaking of things funded by Indiegogo, this company that recently raised money through the crowd-funding site is helping to revolutionize the home security market, moving it from a service based (~$ 30 / month) model where the hardware is largely incidental to a hardware- and smartphone app-centric solution. With the devices from iSmartAlarm installed on doors, windows, appliances and even pets the user’s smart phone can be notified if an unexpected event occurs then get a view of what’s happening using installed camera and take some simple actions with their smartphone. Datalight CEO, Roy Sherrill has this on his Christmas list.

Google Glass (pricing not yet available): You can’t talk about embedded tech for the consumer in 2013 without talking about the immersive experience that is Google Glass. Google had a great year promoting embedded technology and Google Glass is a great indicator of that. Google Glass is a wearable headset that communicates with your voice and gestures to capture all that you see around and in front of you. Google glass can then transmit or store that data to allow for your access later. The interactive aspect of Google Glass allows you to experience an amplified version of your life, such as directions on a walking path, recipe instructions while you cook and your hands are dirty and even added content when you watch movies, see music concerts or go to museums. Google is currently testing all of the technology on their “explorers” with plans to release the technology as early as Q1 2014. Although we don’t have complete specs on the Google glass, we do know that they are running 12 GB of usable memory, synced with Google cloud storage and 16 GB of Flash memory total. It’s amazing to know that embedded technology has allowed us to put all that content in something so small!

Don’t forget about all the ways embedded technology can help shape our lives in other ways. Diagnostic medical equipment, onboard electronics for automobiles, and manufacturing equipment all use embedded technology, too, creating a smarter and more efficient world that we live in.

Embedded technology is sure to change the shape of the tech market well into and beyond 2014. What are you most excited for next year? What couldn’t you live without in 2013? Let us know in the comments!

Rachel Householder | December 12, 2013 | Flash Industry Info | Leave a comment

ARM Tech Con 2013 – Enabling the Internet of Things

I recently attended the ARM Tech Con show which is an ARM ecosystems event. Internet of Things (IoT) was the main theme.   Simon Segars (ARM CEO) keynote was all about Internet of Things (IoT) and ARM sponsored an Economist magazine report to explore the level at which IoT is gaining traction in the marketplace.  One of the unique things he displayed is a cow health monitor that is ingested by the cow and communicates wirelessly to a neck worn device that relays the info to the internet cloud.   The Internet cloud then provides the farmer and veterinarian’s data on health, and nutrition.

IoT is a kind of shorthand for capturing “small data” from the edge of the network and sending it to the Cloud where that small piece of data can be aggregated with other data to make intelligent decisions. One example of this would be using a sensor to track the amount of fuel oil in a tank for one’s home then combining that with data about the home’s usage patterns and local weather forecasts to determine when to schedule a refill that optimizes time and routing for delivery folks while ensuring the homeowner never runs out of fuel.

The good news is that there appears to be a very LOW entry cost to build IoT prototypes. Prototyping an IoT sensor and making it operational can be done with readily available low-cost boards that easily connect together: from a MCU board, to a sensor and a wireless Zigbee or Wifi connection. Development tools are available for free from companies like Atmel. Boards can cost in the $10 – $50 range. Low end boards are programmed in C.  Higher end MCU boards can run Java for app development by the xxx,xxx Java programmers out there. Below is a picture of a board and the metal case – the size of an Altoid’s tin — that Freescale gave away.  It usually retails for just $13!   It connects to PC for downloading and debugging via USB.

ARM-tech-con-2

Untitled

Another item that caught my eye was a small display at the ARM booth that showed power harvesting that did not use any batteries.  A solar cell was hooked up to a capacitor that stored the energy and was connected to an MCU.    Considering IoT devices that never needed a new battery opens possibilities for even more remote deployments.

Daniel Kraft, a doctor, entrepreneur and president of BioniQ, gave a keynote on IoT and medicine.   He showed many products that monitor the body at all times (wearable IoT devices) and report back to your doctor or yourself. One of his quotes that stuck in my head was “OnStar knows more about the health of your modern car than doctors know about most of their patients”. He went on to say that medicine is virtually the same today as it was as it was in 1990 when he graduated from med school, but it need not be with all the new devices collecting patient data in real time.

ARM has an initiatives call “mbed” to make developing IOT devices easier.  Flashing a new program into the system is a drag-and-drop onto a USB mass storage device (The development board).

Freescale is coming out with a wearable computer development kit that includes a MCU and an i.Mx6 on a board the size of a stick of gum.

Conclusion:

The IoT space, while not new, is still in its infancy. Harnessing the creativity of embedded engineers to build tiny devices and connecting them all together will undoubtedly produce applications and uses that we can’t even imagine today. I’m excited by the prospect and the promise.

RoySherrill | November 13, 2013 | Customer Industries | Leave a comment

Is the Linux MTD good enough for MLC NAND flash?

On Linux, the Memory Technology Device (MTD) subsystem is designed to be a generic interface to memory devices, primarily Flash media. The integrated hardware driver handles the storage formats used on the media, and then MTD provides simple routines for block read, write and erase.

MTD does not contain any bad block handling or wear leveling routines, so the use of MTD alone is not recommended on NAND flash media. Instead, developers are asked to use a full Flash File System on Linux, such as YAFFS, JFFS2 and UBIFS.

Unfortunately, neither the low level driver portions of MTD nor the commonly available Flash File Systems have kept up with modern MLC NAND Flash media, which requires special handling in several ways. Among these are:

  • Shrinking dies have reduced costs but increased the error rate, resulting in higher requirements of Error Correcting Codes (ECC). A larger amount of storage is also required for the Error correcting data. Most ECCs are handled with hardware now, and this means either MTD or the low level driver must communicate with the hardware ECC controller located in the processor or on the board.
  • Persistent bit errors, often caused by a Read Disturb, need special handling to correct. Flash vendors recommend a procedure called scrubbing. A related Program Disturb error can cause errors in a subsequent page of the media if vendor programming guidelines are not followed. Flash vendor guidelines vary from manufacturer to manufacturer. The Linux MTD subsystem is generic, and does not contain any vendor-specific routines.
  • With both SLC and MLC NAND, all data writes to a given erase block should be sequential. The MTD subsystem by itself cannot enforce this policy, but a Flash File System can. None of the available Flash File Systems handle this for MLC.

Another problem worth mentioning, though more of a hardware problem, is power interruption while writing to a flash block. This is problematic for all NAND flash, but especially so for MLC. Pages on the media are coupled with other pages, so when power is interrupted during one block write, two (or more) pages may be corrupted. This significant problem has drawn some interest in the programming world, but unfortunately for Linux and MTD, the proposed solutions are all patented.

By itself, MTD is not sufficient to handle MLC NAND flash. It does provide the necessary operations and information to the upper level Flash File Systems, but none of them have been architected to support MLC flash media. The MTD subsystem doesn’t contain any vendor specific code, treating all NAND flash the same. The UBIFS documentation puts it this way: “Good FTL must be a rather complex piece of software.” We tend to agree.

Datalight Flash management software works closely with flash media vendors, providing far more than a simple read, write and erase driver. FlashFX Tera’s bad block management, wear leveling and scrubbing algorithms are a key part of getting the best performance and endurance out of MLC NAND flash media.

Learn more about FlashFX Tera

Thom Denholm | October 23, 2013 | Flash Memory, Flash Memory Manager | Leave a comment

eMMC TRIM new to Android?

An August 1st blog post on Tom’s Hardware reports that the new Android 4.3 NOW has support for the eMMC TRIM command. How odd that the Android OS that’s on 80% of smart phones sold last quarter does not support eMMC TRIM for old deleted data, a feature which has been in the JEDEC specification since version 4.4 which was released in March of 2009.

The symptom reported is that the Nexus 7 is fast after first purchased, but slows down considerably as time goes on. This is because as more and more data is downloaded and later deleted, is not TRIMed from the eMMC device, the eMMC device stills manages that old-deleted data along with the live active data. The reported fix is to run a utility (fstrim) when the device is idle for a long period of time. This utility performs TRIMs on areas of the media that the file system identifies as non-allocated, in periods of long system idle.

So why did Android have the file system take care of this bit of housekeeping itself? Choosing a more flash-aware file system like Reliance Nitro would have not only negated the need for this utility it would make the process transparent for the user. Was this just a mis-configuration that testing did not reveal?

TRIM is important not just for performance but also for extending the life of the Flash memory. An eMMC has to perform garbage collection and maintain it thinks is still needed. If the file system identifies some data as un-need in the future, (TRIM) then there is less data to be moved in Garbage collection, which improves both performance and reduces the Flash wear, thus increasing the endurance of the flash and life of the product. Given that eMMC has been the primary storage for mobile phones since about 2010 and Android has been gaining market share since about 2009, this seems quite surprising. Reliance Nitro has supported TRIM/Discard since its first version. It just makes you wonder if there are other oversights in the Android storage stack.

RoySherrill | September 27, 2013 | Android | Leave a comment

SD Reliability – Best Practices

In an earlier blog post titled Managed NAND Performance: It’s All About Use Case, we referred to an article measuring SD media, specifically sequential write performance. Speed was measured using a camera to shoot continuous pictures. Photographers and other users of SD media have another concern – reliability.

A short survey of photographer websites finds that physical damage causes the most problems with SD cards. This isn’t so much because the internal NAND flash has been damaged, but as a result of the access pins and internal connections failing. This is less of a concern for devices that use SD media as their internal storage, such as medical or industrial equipment. SD makes a lot of sense in those cases, as it can be replaced or upgraded.

Newer SD cards use newer NAND flash, and there are reports of problems with data retention. A good wear leveling solution in the firmware can help with this, as the blocks are worn evenly and rewritten more frequently. Firmware is not common between vendors, so buying a name brand can result in name brand firmware under the hood.

Another suggestion given on the photographer websites is to format the media instead of erasing files. This is most likely done to improve performance for large sequential file allocations. Another benefit is that the format only rewrites the FAT and root directory on the card, where individual file deletes will affect those locations and any subdirectory metadata. On an SD card, a format performs less writes than a series of deletes.

While cameras will likely continue to use the FAT file system for exchangeability, other devices using SD as storage media are not limited to this file system. Datalight’s Reliance Nitro is an excellent fit for SD storage where data is critical and product life is long. An installable Windows driver is available if the SD card needs to be removed and accessed via a Windows-based desktop.

The one thing most SD manufacturers emphasize is that their cards will last for a long time. The average camera application writes large files, often sequentially, and these stay in place until downloaded and then deleted. This results in a very low count of writes and erases to the NAND media. With proper physical care, an SD card used in this fashion can last ten years or more. It is likely to be replaced for capacity reasons before then – the average size of an SD card in 2003 was 256 MB.

While the camera use case is particularly friendly to SD media, one that isn’t is database applications – which are becoming more prevalent in multi-function devices like handheld terminals and smartphones. These applications generate many small writes which can greatly reduce the life of the flash media. For optimum SD card life these small writes should be grouped into fewer, larger writes. Of course, for reliability reasons, maintaining the order of the writes is critical. Datalight has addressed this need with our newest product, FlashFXe, which enhances Reliance Nitro to bring faster writes, power-efficiency and improved flash endurance while keeping the best reliability for the application.

Read more about FlashFXe

Thom Denholm | September 6, 2013 | Consumer Other, Flash File System, Industrial, Medical | Leave a comment

FMS 2013 Exhibition

The Exhibition Floor has grown a bit from previous years, and the traffic in the space reflects that. This year’s biggest booth award goes to Smart Modular. Smart Modular Booth

 

Congratulations to vendor Micron whose 16nm NAND flash wins the Best of Show award for Most Innovative Flash Memory Technology.

Best of Show award presentation

Micron wins for Most Innovative Flash Memory Technology

The exhibition floor is open one more time today for just a few hours. After you visit there, I encourage you to stick around and sit in on the Software session, chaired by yours truly. Todays topics will include Optimizing F lash with Software Built Specifically for Solid State; The Modern Age of Software Defined Storage; Storage Virtualization Technology for SSD and Flash Memory; and All I/O is Random I/O.

Thom Denholm | August 15, 2013 | Flash Industry Info, Flash Memory | Leave a comment

Flash Memory Summit 2013

Flash Memory Summit continues to grow, the organizers are expecting record attendance levels.
FMS-Registration

Brian Berg hosted a great session on Flash Memory Based Architectures. This technical discussion covered NAND Lithography trends, PHY Units for advanced design, and Solving the Power Cycling Challenge. The nearly full audience was eager for more.
FMS-BergAudienceFMS-Berg

The lead-off keynote saw Jason Taylor explain just how Flash is used at Facebook. It is amazing to hear that in just a few years they have gone from Hard Drives to Flash Cached Hard Drives and now to pure Flash SSDs for storage. No rotating media!
FMS-Facebook

The Beer and Pizza chat with the experts on Tuesday evening had an attendance of over 400 people, and plenty of tables and topics. I co-hosted the Software table with Amit of Skyera. Each company does software, but each at different ends of the spectrum, if SSDs are right in the middle.

SSDs were less of a direct focus at the show, with more attention paid to the underlying Flash technology once again. This is the reason Datalight attends the show, and I for one was happy to hear about breakthroughs in latency, throughput and security.

Speaking of breakthroughs, the exhibition floor opens today at noon. I expect to see some 3D NAND, and Smart Modular has a large booth this year.

Thom Denholm | August 14, 2013 | Flash Industry Info, Flash Memory | Leave a comment

What is Industrial Grade eMMC?

eMMC has seen strong adoption and become the storage of choice for consumer devices such as smartphones, e-readers and tablets. These small devices all run on battery power and require high-density storage with low power consumption — at a low cost. Offered for less than $0.60 per GB and in a wide array of sizes — 4GB to 128GB in a single package — consumer-grade eMMC provides this price-performance-power combination. Built to a JEDEC standard, eMMC is now produced by all the major storage vendors including: Toshiba, Samsung, SanDisk, and Micron.

Newly emerging industrial-grade eMMC seeks to leverage this well-defined standard to satisfy a different set of demands. While consumer storage is focused first on cost and size almost to the exclusion of other requirements, industrial applications have a greater need for endurance and reliability with cost much lower on the priority list. Low power consumption – an escalating priority for the consumer market — is also a growing concern for some industrial uses. Many applications within the industrial market require a wider operating temperature range, greater environmental tolerances, additional data security protection, and power fail recovery features.

The new breed of industrialized eMMC responds to these requirements and is gaining consideration for embedded applications in areas such as automotive, medical, aerospace, and other commercial applications. While the industrial marketplace demands a much greater ruggedness than the consumer market, it is also not as cost-sensitive. Industrial devices are much more durable and often mission-critical, products that may be in use for ten or more years recording important data day in and day out. At least one Datalight customer expects their products to operate for twenty years in the field with zero failures.

Industrial grade devices often need to operate in harsh environments that include extended operating temperature ranges — extremes that would cause consumer-grade eMMC to fail. Components must be specified in consideration of the particular need. Automotive and aerospace uses, for example, have a broader temperature demands than medical environments. The table below shows typical temperature ranges for different grades of electrical parts.

Grade

Low Temperature

High Temperature

Consumer

0°C

70°C

Industrial

-40°C

85°C

Mil/Aero

-55°C

175°C

 

The higher reliability of industrial eMMC comes with a higher price, which is justified by the longer payback period and business-critical performance. The chart below shows the general relationship of cost and reliability for the consumer and industrial eMMC memory.

industrial-emmc

So how do eMMC manufacturers achieve the endurance necessary to meet the industrial standard? Unlike other industrial solid-statue storage – eMMC can be configured to operate in one of two modes: Single-Level Cell (SLC) at one bit per flash cell – sometimes called “enhanced mode” or Multi-Level Cell (MLC) at two bits per flash cell – referred to as “standard mode”. While enhanced mode has half the number of storage bits, it lasts about 20 times as long as standard mode. Stated conversely, standard mode provides twice as much storage, but its endurance is reduced by a factor of 20. In more specific terms, standard mode may fail at about 3,000 P/E cycles, while enhanced endures for about 60,000 P/E cycles.

In addition to the existing consumer eMMC standard packaging of 153 or 169 ball packages, JEDEC has recently standardized on a 100-ball industrial-grade packaging. This new Industrial packaging provides reduced cost of manufacturing with simplified PCB trace/space designs, fewer number of balls, and fewer number of PCB layers. Industrial eMMC devices are available today from Micron, Kingston, and Memoright. Greenliant and Smart Modular are also entering the industrial eMMC market.

Industrial storage requires increased reliability and endurance. The hardware is only half of the solution.   The software driving the storage is the other half, and has an equal or greater effect on reliability and endurance.   Are you leveraging all the tools available to you to meet the endurance and reliability requirements?    To learn more about solutions for managing eMMC devices, see our FlashFXe page.

RoySherrill | July 23, 2013 | Flash Memory, Flash Memory Manager | 1 Comment