Shuttle launches are difficult I think, because of course, there is only one chance to get it right. Reading these two articles made me realize how difficult it is to predict failure in any kind of engineering endeavor. The only way to see if a shuttle has been built perfectly is to actually launch the the thing, but the engineers at NASA cannot do that. They can only predict failures beforehand the best they can. Even though the individual parts of the Challenger has been tested for various conditions hundreds of times, the O-ring was still found faulty. The excessive compression on the O-ring was only discovered in the post-accident tests.
This reminds me a lot of building and launching websites. Unlike space shuttles, launching faulty websites do not result in death. But there are significant dangers of poorly built websites such as identity theft or unintentionally exposing sensitive information. Security holes and data bottlenecks must be predicted before launch, but it is extremely rare to predict all the flaws of your website.
If there are any problems with a popular website, it is found rather quickly after launch. The different combination of clicks and text submissions of millions of users expose all the flaws of the website.
There is no way to build a perfectly secure and perfectly scaled website without actually launching it, and predicting all of the problems is extremely difficult as it is hard to predict how users will use the website.
Just like websites, it must be extremely difficult for the engineers at NASA to have predicted all the factors for failure on launch day. Engineering fields where faulty launches do not end in catastrophic results are fortunate as they can improve their product incrementally.
I enjoyed reading these articles, and both presented some ideas I had not heard before concerning the failure of the Challenger. The Dumar/Waring article described the what was really a "witch hunt" that followed the failure of the space shuttle. I thought one part of the article, which described an engineer's objections to certain legal proceedings was especially insightful. The engineer remarks something to the effect that engineers don't think like lawyers, they think like engineers. I think this points to the idea that legal proceedings which involve very specific and complex scientific and mathematical principles can be difficult to handle simply because of communication problems between the parties involved. Lawyers may not understand the specifics of the science and math involved, and so, may not make completely informed decisions.
ReplyDeleteThe Fischbeck article also presented some insightful ideas. Risk is something unavoidable in most any endeavor. Like John mentions, web design and launch carries with it a certain level of risk, although the consequences may be less severe. And I agree that often times, "bugs" in software, websites, and other technology are not effectively eliminated until after launch of a so-called "beta version." And, like the Fischbeck article makes clear, when consequences of failure are high, risk must be assessed and managed prior to launch. I think it is effective, although extremely tedious, as the article mentions, to assess individual risk of failure of parts (i.e., the O-ring). In doing this, one can calculate the overall risk of failure as a function of risk of individual parts, the Fischbeck article explains. Assuming that one was able to find all possible risks and calculate accurate probabilities of such failures occurring, it seems this would be an effective way to manage potential risk. However, it seems unlikely that one would actually be able to predict all possible sources of potential risk and even more unlikely that one would be able to calculate accurate probabilities of failures from such risks occurring.
I agree John that risk analysis has varying importance depending on the resulting factors of the risk, and in the Pate-Cornell and Fischbeck article they make it clear that this is an important part of Probabilistic Risk Analysis. The first way of looking that the risk consultants used involved identifying the individual risk of each tile failing, and then the significance of each individual tile's failure, noting that 15% of the tiles accounted for 85% of the risk.
ReplyDeleteThough I enjoyed reading the articles to some degree, I was rather upset with some of the things that I read. NASA rejected using Probabilistic Risk Analysis from the time of the moon landings all the way until the Challenger disaster because they didn't like the bad press that came with it. Bad press would've led to lack of funding, so NASA winds up being far too dependent on its public image. Consequently, they rush to meet deadlines and operating with unwarranted optimism, which, as can be seen clearly in the image on page 15 of the Pate-Cornell and Fischbeck article, can potentially lead to the destruction of the shuttle and loss of life. Even if you're an astronaut, a combination of bureaucratic maneuvering and rushing to meet deadlines increases risk, and they apply an essentially monetary value to your life. As much as I hate this, just like the risk associated with space flight, it is unavoidable.
Zach,
DeleteI also found myself disgusted that some of our nations top engineers and scientists would ignore probabilistic modeling for something as dangerous as space exploration for fear of "bad press." Such behavior is not only ethically unsound (we are talking about human lives here) but just plain old bad scientific rigor. Seeing as in order to 'prove' something in modern science one need not only perform an experiment but have sufficient statistical evidence to suggest the experiment is truly observing what is claimed (in particle physics the standard is 5 standard deviation confidence interval or 99.99994% certainty in a result before it can be considered 'proven').
This lack of statistical modeling (and disregard or negligence of safety considerations) makes me think of the iconic (well... to russians anyway) image of Vladamir Komarov's open casket funeral. He was the first cosmonaut to go into outerspace more than once and died on re-entry in a shoddy capsule (it had a parachute malfunction). If you are unfamiliar you can read a really good version of the story here:
http://www.npr.org/blogs/krulwich/2011/05/02/134597833/cosmonaut-crashed-into-earth-crying-in-rage
and see the open casket funeral:
http://all-that-is-interesting.com/open-casket-funeral-soviet-cosmonaut-vladimir-komarov
The point of which was to remind Russian officials that their actions can cost lives. I wonder if the proceedings would have played out differently if the American public, or at least the presidential investigative committee, had to see the charred remains (if they even existed) of the Challenger astronauts. There might have been less pointing of fingers and a more serious look at how safety procedures could be improved.
Honestly though, i found these readings pretty illustrative of a rampant problem within our government (and frankly most democracies): when we elect people by popular vote and not necessarily credentials, are they really able to make the most informed decisions? Michael Heinz I think hit it square on the nose pointing out that lawyers don't think like engineers and vice versa. Would our country fare better if people running special committees were themselves experts in the field (such as astro-engineering for space shuttles or climatologists and ecologists for committees on global warming)?