I agree that data need not be at the 99% confidence level to be useful. I would like to point out that data valid at the 50% confidence level is no better than a coin toss. Most decision makers choose a language on much better criteria than that. What I said was "statistically valid." It is so easy to "lie" with statistics, why bother using invalid data to make our case. Often the decision has nothing to do with, "What language produces systems that cost less or are more reliable?" (IMO Ada). The language decision is often made based on the cost of the compiler and tools, the personal preferences of the current staff, or the availability of trained staff in the region. In these cases, unfortunately, Ada looses. Unfortunately, I have seen managers who ignore statistically valid data; which demonstrates that Ada systems require fewer man-hours to produce (after about the 3rd system), are more reliable, and run equally fast; and choose C++ or Java because it's what the trade rags are pushing. Rush Kester charter member Team Ada -----Original Message----- From: Tom Moran [mailto:[log in to unmask]] Sent: Wednesday, March 15, 2000 10:07 AM To: [log in to unmask] Subject: Re: Our Lack of Hard Data is a Disgrace. >I can assure you that this is not easy. The SEL has been collecting human >effort data, program size data, error rates, and many other metrics for over >two decades. >... >What you're left with is a situation where only many experiments over time >can be used to arrive at statistically valid results. We'll take results at the 50% confidence level. ;) Seriously, it may take hundreds of years to get enough data to prove, at the 99% level and without a shadow of a doubt, that X is true. But even knowing that X is more likely true than not, is *useful* information. While we wait for certainty, people *are* making decisions that are more likely bad than good, based on their conclusions, statistically valid at *very* low confidence, from single circumstance anecdotes.