At 04:49 PM 12/26/98 -0800, Robert C. Leif, Ph.D. wrote:
>Fortunately, our universe is restricted to software development.
>Unfortunately, I do NOT believe that a true double blind crossover study is
>even conceivable. This would require the same project being developed in
>both Ada an another language. However, I believe that it is impossible
>because there is no straight-forward way to organize an experiment where
>neither the monitor (teacher) nor the student (user) know which programming
>language they are using.
Okay, I have to put in my own two cents as a statistician. ;-) Neither
traditional single or double blind studies are possible because in any case
the patient (programmer) knows what language he is writing in. But it is
fairly easy to do a study where neither placebo or Hawthorne effects occur.
The important thing is not to have the same programmer, or programming
team writing the software in two languages--each sample has to approach the
problem as new. Basically this has occured at several places, probably the
most conclusive was at SUNY Albany.
My "study of studies" synthesis of what I have seen is that Ada (83) is
about five times more productive from design through integration and test
than C or Fortran. But putting on my Operations Research hat instead, the
interesting--and totally obvious in hindsight--conclusion is one that I
have stated here many times. The major difference is that in Ada, coding
is done means both that and that the software is very nearly complete.
(Testing remains, and usually some GUI tweaking.) In other languages, it
is not unusual for 90% or more of the source lines to be changed after
"coding is done." On one (Fortran 77) project I remember vividly because
we had the numbers, there were 560 KSLOC of changes to a (final) 300+ KSLOC
project that was just over 200 KSLOC when "coding was done." On the other
hand, I worked one Ada program where there were 17 lines changed between
handover to integration and test and six months after fielding. (And over
half of those changes were clarification of error messages or correcting
typos in same.)
I know I'm preaching to the choir, but it took a lot of getting used to
fifteen years ago. We were converting some tools from Multics to the DPS6,
and needed to translate them from PL/I to Ada. Even though we built
several translation tools to help in the process, most of the "hand work"
time was spent dealing with cases, error or otherwise, that the PL/I
programmer hadn't considered. Most of those changes were backfit into the
PL/I, and they became "just as good" as the Ada versions. Our conclusion
was that, in other languages it can be up to ten times as expensive to turn
out product quality code than a one-off kludge, but in Ada, the difference
is under 50%.
Robert I. Eachus
function Message (Text: in Clever_Ideas) return Better_Ideas is...