Perils of computer simulation of complex systems
Story submitted by Ricky Seltzer
John F. McGowan, Ph.D., writing on math-blog, describes the various ways in which breakthrough science can be misunderstood and miscalculated even by top-flight computer simulation. (One example of breakthrough science, of course, would be climate modeling).
Another important aspect, is that common-mode errors, where different teams rely on the same erroneous data, or repeat the same error, are common. Different teams have been observed to make the same or equivalent errors in constructing software. There were some experiments in the previous century which assigned the same task to different, competing teams of software developers. It was found that some errors appeared in many or all of the teams.
Here is one reference from Lawrence C Paulson:
“Redundancy in software means having several different teams code the same functions. This has been shown to improve reliability, but the improvement is much less than would be expected if the failure behaviours were independent. This suggests that different teams make similar coding errors or fail to consider similar unlikely cases.”
Short summary:
Story title:
Software bugs, physics errors, typos, and so on all can combine to lead good people astray. Read the whole thing.