Follow up (Part A) to A Category Called General with some discussion about the expected lifespan of flash storage.
Transcript available
This is pragmatic follow-up part A for episode 7 a category called general I'm Ben Alexander my co-host is John Shiggy. Thanks Ben in this follow-up for episode 7. I asked or rather I lamented the fact that I couldn't find any detailed information about the shelf life of flash drives and Michael Ortley once again sort of. to the rescue there and sent through a link to a How Stuff Works article, link is in the show notes for this follow-up episode. And this article suggested a five to ten year shelf life. So thanks very much for Mike for sending that in. In addition to that, I did some more digging and found some more other interesting links that I've also included. And in particular, there's an Ars Technica comment thread, and in particular, the comment by Peter N, that's the only information I've got on him, suggested 10 years, which sort of aligns with the article that Mike sent me as well. And the issue seems to be, as I may have, may not have alluded to, is the difficulty in accelerated life testing, because normally, If you're trying to do ALT on a product to determine its long-term lifespan, normally you're focusing on temperature cycling. And the problem is that not all wear out mechanisms are driven by thermal cycling. A lot of them are, but not all of them. And sometimes it's difficult to know with any degree of certainty. So, there's another method that's sometimes referred to as a heat soak whereby you run the equipment at an elevated temperature for a significant period of time. And what you do is you make an estimate of how many hours it would ordinarily experience at the higher temperature on a given day. And statistically, you can then project if it fails after 100 hours, that's the equivalent of 10,000 hours or some equivalent math. Obviously, it depends on the the technology, but yeah, that might be a way of testing it. Because the problem is that because the charge is stored, the way the charge is stored in the flash disk, eventually it's going to leak out, eventually you're going to lose data. And there's no question about that. It's just a matter of how long. And there are a couple of ideas that I thought were worth mentioning that I was reading up about. And one of the ideas is that if you are going to use a flash disk, every time you write to a flash disk, in essence, you are damaging its ability to contain the charge. So, every time you reflash it, every time you update the data in there. And the way they get around that is they build in additional capacity into the flash disks. So, they say, okay, well, I'm going to account for, I I don't know, a 5% loss. And therefore I'll rate my drive at, you know, I'll oversize my drive by 5%. I'm not sure what the exact figure is, but I'm sure it varies from manufacturer to manufacturer. But the idea is that they shift the data around to ensure that everything gets an equal amount of wear. And then they flag bad parts that don't go back. And so kind of like flagging bad sectors, that kind of idea. And eventually you'll get to a point where everything's had so much wear that it can't reliably hold data. And that's the problem with Flash. So, and it's different depending upon if it's MLC or SLC, and I don't want to go into too much depth into that, but just take my word for it, that there's a different technology used for different kinds of flash, not all flash is created equal. Again, might do a show about that later. But anyway, the point is that if you are trying to store data on a flash disk for long term shelf storage, your best bet is to go for either a new or near new flash drive that has had minimal rewrites done to it. when I say rewrites, I should say just flash writes to it. So in other words, I would be buying a flash drive, writing the data I intended to store long term on it, take the drive out, put on the shelf and not touch it again. There's also a suggestion that because the charge will leak out over time, it may be advantageous to do a data refresh every two to three years to ensure the data integrity. So in such a case, you might get it out of storage. And when I say do a complete refresh, I mean literally completely read it and then completely rewrite it. So it's not just enough to read the data. It needs to be flashed and then flashed back, if you know what I mean. So, and that should then reinforce those bits such that they would then maintain integrity for theoretically for longer. But again, very conflicting reports on to how often you should do that. And the truth is that the technology is just- Is very new. We've got a lot of history on hard drives. We've got even more with floppy disks and flexible magnetic media. We've got plenty with optical disks, although Blu-ray is still relatively young in, you know, those sorts of terms, but similar enough technology to CDR and DVD-R, that we can make intelligent assumptions based on our knowledge of their lifespans. So I'm sure the case will come out in the next three or four years as flash drive capacities increase, we reach a point where we can say, well, data is still present after seven, eight, nine years. Therefore, using this technology, we expect a maximum lifespan of, say, 15 years. And the numbers should increase. How much they increase to, I don't know. But I would not expect Flash to outlast a Blu-ray disc, for example. I wouldn't expect Flash to outlast even some hard drives, maybe. And certainly not- I don't expect the Flash would last as long as a 35-year lifespan tape, as we discussed in the show. So, in any case, it's interesting. And it- Personally, I wouldn't be using them for long-term data storage until we have more information about how their- What their longevity is, their shelf life is. Well, and there's the expense still, right, at this point. Well, absolutely. I mean, you can't store large amounts of data and, you know, you've got to ask the question, why would you in the first place? But I just wanted to make sure that we covered it off as an option, because there are still cases. Like, for example, we had a project that I did when I was working for a large consultancy. And when we did the project at the end of it, the client specifically asked for a flash drive with all of the project data on it. And fair enough, it was, you know, about four or five gigabytes. But then, you know, a 16 gig flash drive is like five bucks. It's not a lot of money, if that. Depending on where you go. They're at the checkout aisle at the grocery store. Well, yeah. Yes, you're right. They are. They're everywhere. They're everywhere now. So, we did that for them and we sort of said, oh, you know, here you go, and that was to be for their long-term storage. Now, we're going to keep that on file, keep a copy on their own servers, a live copy, essentially, that they can access when they had to. And this was going to be their backup. And that's just what they wanted to do. So I don't suggest that individuals are going to back up their entire collection of photos, of gigabytes of photos to these things, because it's just too expensive. But, you know, someone may have a library that's of a reasonable size, have a flash disk lying around and think they're covered. And I guess in that case, I'd just advise caution. That's all. Right. Well, and there's a certain, you know, for primarily text documents, you could, I think most people could get everything they reasonably need into a single one of those. And there's a there is a reassuring aspect to their little keychain like size. That's true. That's true. So in any case, thank you, Mike, for that feedback. Much appreciated. We're- And yeah, I don't have too much more else to say about that. [BLANK_AUDIO]
