- What they looked for and what they found
- Violations of MISRA (and NASA) standards
- Variables
- Misleading code
- Recursion
- And so?
- Media
- Investigation report
A story about the fact that the software is penetrating more and more in our daily life. However, with the comfort and usefulness come new dangers. Now we deal with the bugs not only sitting at the computer, but driving on a road.
- People: Hey, Toyota, we counted that 89 people died from 2000 to 2010 because of your screwed up your electronics and software.
- Toyota: Yes, but these are people to blame, they confuse the pedals.
- People: Houston, we have a problem.
- NASA: Wait a little, we'll sort it out. We'll need 10 months and 3 million dollars.
- People: Here, take it.
- Toyota: 3 million isn't enough, here's some more cash.
(10 months later)
- NASA: Hey, Toyota, we found a couple of bugs in your code, namely 7134 MISRA standards violations, recursion, 740-string long function and 9000 global variables.
- Toyota: We have our own standards. Have you guys, been to the Moon?
- NASA (publicly): Toyota is not to blame.
- (Toyota Shares went up by 4.6%)
- People: What was that?
(3 years later)
- Two American testers (whose grandfathers died at Pearl Harbor): No bugs you say? What if we find them?
The National Highways Traffic Safety Administration (NHTSA), have evaluated that in the 10 year period from 2000 till 2010, in accidents caused by defective electronics, 89 people died and 57 people were seriously injured.
Toyota denies its fault, and states, based on their own research, that it's all because of "sticking" accelerator pedals, and a design flaw that enabled accelerator pedals to become trapped by floor mats, but recalls nearly 8 million vehicles around the world because of these two defects.
Still, there are more complaints coming.
We recommend that those with a nervous disposition not watch this video.
NHTSA started their own investigation, asking NASA to help.
During the 10-month investigation, NASA specialists claimed that the software does not comply with MISRAstandards (Motor Industry Software Reliability Association), and contains 7134 violations. Toyota responded, saying that they have their own standards.
December 20 2010, Toyota rejects all the accusations, but pays 16 billion dollars in pre-trial actions, releases software updates for some car models, and recalls 5.5 million vehicles.
After the announcement of the results of NASA's research, Toyota shares on the Tokyo Stock Exchange went up by 4.6%.
In the year 2013, an action is filed in Oklahoma Court in regards to an accident in 2007, involving two girls in a 2005 Toyota Camry. One of them died, the other spent five months in a hospital with injuries to the back and head. Toyota has not admitted its guilt. They said that the cause of the accident was the driver confusing the gas pedal and the brakes; when she realized her mistake and started braking — it was too late.
Two engineers started the investigation. Michael Barr and Philip Koopman. It took them 20 months to review 280 000 lines of code and write a 800-page long report. Each.
The address was kept in secret. The hotel room, where the engineers worked was guarded 24 hours a day - security ensured that nobody was bringing in or taking out any papers. All the phones and internet connections were disabled.
Toyota recalled more than 10 million vehicles worldwide. Still, they have never admitted their guilt.
According to Michael Barr, their report was classified as secret. The same thing was done with the contract which gave them access to Toyota's source code. Barr recommends Googling the transcript of the hearing material.
- TRANSCRIPT OF MORNING TRIAL PROCEEDINGS FROM THE 14TH OF OCTOBER, 2013
- BOOKOUT V. TOYOTA 2005 Camry L4 Software Analysis
- KILLER APPS Embedded Software's Greatest Hit Jobs
Here is where the analysts worked:
Here is the report they wrote:
What they looked for and what they found
The main program in the dock is the electronic throttle control system (ETCS).
NASA experts scanned the chips with x-rays.
Cosmic rays were also considered as a possible cause of errors.
They checked the C code:
And then they finally got at the code.
Violations of MISRA (and NASA) standards
According to estimates, every 30 MISRA standard violations lead to one "serious bug".
- In MISRA-C:1998 - the list contains 127 rules (93 mandatory and 34 advisory)
- In MISRA-C:2004 141 rules (121 mandatory and 20 advisory). The rules are divided into 21 categories.
- In MISRA-C:2012 there are 143 rules (each of them can be checked by a static code analyzer) and 16 directives (whose compliance is more open to interpretation, or relates to process or procedural matters). The rules are divided into mandatory, required, and advisory; can be applied to individual units or the entire system. The rules are divided into Decidable and Undecidable.
Toyota took only 11 rules from MISRA.
NASA analysis tools were able to check 35 MISRA rules, and 14 of them were violated.
The source - NASA report, appendix A: Software, page 28]
Total: 7134 (NASA estimation), or 81 514 (according to Michael Barra's estimations).
10 rules of NASA
The Power of Ten - 10 Rules for Writing Safety Critical Code
- Restrict to simple control flow constructs.
- Give all loops a fixed upper-bound.
- Do not use dynamic memory allocation after initialization.
- Limit functions to no more than 60 lines of text.
- Use minimally two assertions per function on average.
- Declare data objects at the smallest possible level of scope.
- Check the return value of non-void functions, and check the validity of function parameters.
- Limit the use of the preprocessor to file inclusion and simple macros.
- Limit the use of pointers. Use no more than two levels of dereferencing per expression.
- Compile with all warnings enabled, and use one or more source code analyzers.
[The source - spinroot.com/p10]
The function length is limited to 60-75 code strings, after removing empty strings and comments. More than 200 functions in Camry05 code exceeded the specified length. One of the functions was 740 strings long.
Variables
31 names were declared several times in different scopes. The most frequent name is sts_flags1, which appeared in 57 different scopes.
This is worth a closer look.
Misleading code
A graph of the flow control of a simple program.
Cyclomatic complexity of the program above 50 - an indicator that the program cannot be tested.
In the ETCS-code Toyota has:
- 67 functions with complexity above 50
- The complexity of Throttle angle function = 146; 1300 code strings without the plan for unit testing.
Recursion
Programmers used the recursion in the Toyota code, every issue related to its usage led to the restart of the processor (CPU reset).
And so?
The amount of shitty code, on which the lives of people depends on, gets bigger. The example of the Toyota company, shows that the system developers can screw the code on an elementary level, not to mention, on the level of accepting the ethical decisions of the artificial intelligence. Although the main trouble is not that there are errors, but the fact that the owners hinder their process of finding and fixing the issues. These people are powerful enough to push on NASA.
"Applications programming is a race between software engineers, who strive to produce idiot-proof programs, and the universe which strives to produce bigger idiots. So far the Universe is winning."
- Rick Cook, writer
Media
- U.S. Department of Transportation Releases Results from NHTSA-NASA Study of Unintended Acceleration in Toyota Vehicles (February 8, 2011)
- Toyota's runaway-car worries may not stop at floor mats
- NHTSA-NASA Study of Unintended Acceleration in Toyota Vehicles
- Analysis of Toyota ETCS-1 System Hardware and Software
- Toyota Recall Timeline
Investigation report
An exhaustive presentation of Philippe Kupmana's:
NASA Report on Toyota Unintended Acceleration Investigation
NHTSA Report on Toyota Unintended Acceleration Investigation
Four years before that
Wherever I'm going, I'll be there to apply the formula. I'll keep the secret intact.
It's simple arithmetic.
It's a story problem.
If a new car built by my company leaves Chicago traveling west at 60 miles per hour, and the rear differential locks up, and the car crashes and burns with everyone trapped inside, does my company initiate a recall?
You take the population of vehicles in the field (A) and multiply it by the probable rate of failure (B), then multiply the result by the average cost of an out-of-court settlement (C). A times B times C equals X. This is what it will cost if we don't initiate a recall.
If X is greater than the cost of a recall, we recall the cars and no one gets hurt.
If X is less than the cost of a recall, then we don't recall.
- Chuck Palahniuk "Fight club", 1996
- How often do such accidents happen?
- You won't believe it.
- Which company do you work for?
- Oh, it's a very large one.
- "Fight club", film, 1999.
This article was originally published (in Russian) on habrahabr.ru. The original and translated versions were posted on our blog with the permission of the author.
No comments:
Post a Comment