To: Alan & Carmel Brain et al.
From: Bob Leif, Ph.D.
There are two types of studies prospective and retrospective. Prospective
studies are really definitive. Prospective studies are expensive; however,
they can be definitive. Retrospective studies almost always are less useful.
However, since they often only consist of data mining, they can be much less
expensive and more comprehensive, include more data. Below, I will explain
how the "Year 2000 Bug" should provide very good data for a retrospective
The actual problem is that the subject will know which language is being
used. In medical studies, the patients and doctors do not know what is in
the pill or injection. There has been some work with undergraduates to
justify the choice of a language for introductory courses. However, the
significance of this could be questioned as to its relevancy to the work of
mature software engineers. In the past, the US Department of Defense used to
select planes by having two or more competitors build flying prototypes. It
actually would pay to do this when comparing languages. However, one must
then be very careful about defining what the results mean. Results would
include total labor, defects, and maintenance costs. These will be convolved
with the abilities of the software engineers unless one has two projects and
has the teams switch languages. If the users chose the language, Ada would
almost always win because of the Ada culture. Parenthetically, I have always
argued that Ada was an excellent personnel management tool. The hackers see
her and quit!
One interesting test would involve finding a group of programmers who knew a
language, such as FORTRAN, COBOL, or BASIC but who had no experience with
both C and Pascal nor any of their descendants. A test of their capacity to
decipher programs in C++ or Java versus Ada would be most informative.
Another measure would be to define a software project and make it a contest
for real money and fame. This should attract the best in the field. Then one
could employ metrics, such as errors per 1,000 lines of code, statements per
function point or feature point, and/or my simple minded efficiency formula:
Efficiency = Output/Input;
Lines of source code is by itself a useless metric for OO languages with
inheritance. I believe that the ratio of lines of code written to the linked
code (output) is what counts.
Linked Lines of Source code or text equals the total number of semicolons
which would be actually used in the linked Executable if
1) all loop structures remained intact (no unrolling);
2) all instantiations of generics are treated as the equivalent of the
source text which would have been created without the use of the generic;
3) all instances of inherited subprograms of tagged types are treated as the
equivalent of the source text which would have been created without the use
of the tagged type.
Retrospectives studies are possible. There one determines with appropriately
matched cases what the costs were. It should be emphasized that with a
significant number of cases, something between 10 and 100, the criteria can
be reasonably broad. A universe of 100 males between the ages of 40 and 60
years is very often sufficient when there is a significant effect. However,
this data should NOT be used for females or males of other age ranges. They
would require another study. I might note that if one found a real benefit
for the males of a limited age range, that would provide enough data to make
it worthwhile to test on another group or in the case of a lethal disease
where nothing worked to have a reasonable probability of obtaining a
humanitarian exemption from the US FDA.
The US DoD should have enough data from C++, C, and Ada to permit a useful
study. In fact, the Year 2000 Bug" is perfect for this. There should be data
on the cost by language per project of detecting and eliminating these
design errors. I must now EMPHASIZE that a simple minded study will probably
produce incorrect data. Cases where the cost was not significant will NOT be
reported!!! Thus, there may be very significant under reporting for Ada.
Yours, Bob Leif
> -----Original Message-----
> From: Team Ada: Ada Advocacy Issues (83 & 95)
> [mailto:[log in to unmask]]On Behalf Of Alan E & Carmel J Brain
> Sent: Thursday, December 17, 1998 11:34 PM
> To: [log in to unmask]
> Subject: Re: Language Efficiency was RE: Choose Ada flyer
> Robert C. Leif, Ph.D. wrote:
> > To: Wesley Groleau et al.
> > From: Bob Leif, Ph.D.
> > Subject: Language Efficiency
> > I might add a comment, which I hope does not hurt anyone's
> feelings. Having
> > spent almost 40 years doing science, analytical cytology and cancer
> > research, I am totally horrified at the level of many computer science
> > studies. Most of what I read would be totally unacceptable in
> the scientific
> > literature.
> Hallelujah, Brother! I couldn't agree more.
> OK, so what metrics should we use? How do we design the experiment? I
> would have thought the (US) DOD for one would be fascinated by getting
> some hard date vis-a-vis comparative language capabilities. The FP
> database in Melbourne seems to show that the single biggest contrivuting
> factor to productivity is language.
> I've tried to get the ball rolling several times here (I have a mere
> BSc, and such a topic would be a LOVERLY PhD thesis..) but no joy. Yet
> it's one of those dream areas, where there's b***** all in the way of
> literature, most of it is very soft, yet there's a potential huge
> practical economic benefit in the short term, ie Funding SHOULD be easy
> to get.
> Experimental design is another matter.
> It should also either a) Prove what we knew all along or b) Shake our
> complacency and kill some long-cherished beliefs. Probably a bit of
> both. Might be an idea to get some Adaphobes along to review things
> too, as otherwise we may lose objectivity (and hence usefulness).
> Over to you. Can we do something about this?
> [log in to unmask] <> <> How doth the little Crocodile
> | Alan & Carmel Brain| xxxxx Improve his shining tail?
> | Canberra Australia | xxxxxHxHxxxxxx _MMMMMMMMM_MMMMMMMMM
> [log in to unmask] o OO*O^^^^O*OO o oo oo oo oo
> By pulling MAERKLIN Wagons, in 1/220 Scale