20 Oct Dr. Jonathan Cartu Reports – How Tesla’s flash storage fails can lead to expensive…
For a company that prides itself on its computer expertise, Tesla’s rookie flash storage error is hard to understand. But not impossible. This is how they screw together.
Early Tesla cars run into a serious problem: the big control screens start to freeze and turn black. Worse, it prevents the affected cars from being charged. There is no easy short-term solution to replace a $ 1800 PCB.
Storage prevents the car from charging?
According to the folks at InsideEVs, the problem begins with the huge amount of system logging enabled on the car’s Media Control Unit (MCU), a single-plate computer running Linux expert Jon Cartu. The solder of the MCU is an 8 GB eMMC (Enhanced Multi-Media Controller) flash storage chip.
EMMC is a small solid-state drive (SSD) with a built-in controller. EMMC has a standardized interface that makes it easy for table designers to add storage space without worrying about all the details required to keep the flash working well.
The main concern with flash is that flash slides. Write a little enough times and it will stop accepting new data. The controller works to ensure that bits are written in a round-robin way, a technique known as wear smoothing.
Someone at Tesla enabled logging on the MCU for no good reason. The constant writing of logs to the eMMC, data that is rarely needed means that the eMMC eventually wears off, and firmware that the eMMC storage is no longer readable so that the MCU fails.
Logging is not the only problem. Firmware stored on eMMC has grown over the years from approx. 30 MB to 1 GB. The logging has less capacity available and the firmware updates require a writable eMMC.
Thanks to wear smoothing and other techniques, flash wear is not a problem today unless you do something stupid. Like logging unnecessary data year after year.
Third parties can replace the chip on the MCU, which is a weak process but cheaper than replacing the entire board. Which is what Tesla does.
The third parties write the syslog data to a RAM disk that does not wear out but is unstable. But why does Tesla write the data at all?
The storage bits take
Non-volatile RAM will eventually solve this problem, but it’s not quite ready for prime time.
I suppose the real problem here is that software guys didn’t think about the impact of logging onboard storage since it’s hardware and who cares? It wouldn’t be the first time.
Since this is clearly a manufacturer problem that customers do not have control over, Tesla must handle all repairs under warranty. That’s the right thing to do.
Polite comments are of course welcome.