A couple of weeks ago, I promised to write up my impressions of the
Public Briefing on the NRC study and post it to Team Ada (I am also
posting this to the ARA Exec list). I apologize for taking so long
to get this written--my wife was in the hospital last week (she's OK
now), and I had to go pick her up Friday after the briefing. I was
out of the office on travel yesterday, so hopefully this will still be
At the end of my message is a copy of the text that Richard Riehle
already sent out that contains the summary of the NRC report. My
message will focus on the information that was presented at the
briefing, and my impressions. There were other Teamers at the meeting
(Brad Balfour, Mike Feldman, Karl Nyberg, David Hoos, Bruce Bachus) who
may weigh in with confirming or contradictory input.
This is a long message, but necessarily so.
Hope this is of use to all.
NRC Ada Study Public Briefing--1NOV96
If I had to choose a single word to describe my impressions of the
briefing and the study, it would be "appalled." I am appalled not
so much at WHAT the briefing included; rather, I am appalled at HOW
the information is being presented.
The committee's recommendations can be distilled into four major
1. Use Ada for warfighting software.
2. Drop the requirement for Ada for all other software.
3. Invest $15 million/year in Ada infrastructure or drop Ada altogether.
4. Integrate the Ada decision process into an overall Software
Engineering Plan Review (SEPR) process.
These are the four main points that will be picked up by the media and
widely publicized throughout the software industry and government agencies.
The first 40+ presentation slides go through all of the justifications and
rationale for these bottom line recommendations. On slide #44 is the
most important statement contained in the entire briefing:
"Recommendations of this study are based on a mix of inadequate data,
anecdotal evidence, and expert judgment;"
I am willing to bet that this statement does NOT get picked up in any
article that appears in the media. This statement puts the value of
the entire study in question--how can a "study" based on such criteria
be used in any valid manner to drive critical DoD software decisions?
Of the 3 criteria (inadequate data, anecdotal evidence, and expert
judgment), I believe the most valuable is "expert judgment," yet, in
a study which addresses warfighting software, NOT A SINGLE MEMBER OF
THE COMMITTEE IS/WAS A MEMBER OF THE MILITARY. In other words, there
was not one warfighter on a committee which was commissioned to study
warfighting software. Thus, the "expert judgment" did not include
the expertise really needed to ensure value in the recommendations.
Slide #4 noted: "COTS, commercial trends make Ada less competitive
for most non-warfighting applications." The justification given for
this statement was that when a COTS product is updated, it requires
the user to update all the Ada bindings to it, which is a costly and
time-consuming process. This is somewhat baffling--I expect that most
every Teamer has gone through an awful experience trying to install and
use a new COTS release, even without having to update Ada bindings.
Using the same criteria as the committee (inadequate data, etc.), I would
consider this item a "wash"--the disruption of moving to the latest release
of a COTS product (which is often required to ensure ongoing support) is
often much greater than any Ada-related impact.
Slide #5 addresses the recommendation for the $15 million/year investment
"at the margin" for Ada infrastructure. Although my degree is in accounting/
finance, I don't know what "at the margin" means. I think it refers to
continuing the funding for AJPO (or something equivalent) to provide some
ongoing funding for R&D efforts related to Ada tools, bindings, validation,
Ada information services, etc. Assuming this is the case, the $15 million
is less than half of previous funding levels for the AJPO, so it is even
less than "a drop in the bucket." When you consider the amount of money
that is spent by commercial software companies in just sales, marketing,
and advertising, $15 million is a laughable amount of money--Microsoft
spends about $23 million every 48 HOURS throughout the year on just
sales, marketing, and advertising.
When asked about how much money the DoD should spend on actually buying
Ada products and services (as opposed to funding R&D efforts), the panel
had no answer. From an Ada industry perspective, the amount of money
that will be allocated to actually buying something is far more important
than a paltry figure that will be used for R&D activities. Experience
has shown that around 20% of such funds are consumed by the administration
of the projects funded.
Perhaps more importantly, this $15 million figure will be largely
interpreted as being the TOTAL amount of money that the DoD should expend
on Ada--not exactly a major market opportunity.
There are glaring and dangerous assumptions/declarations in the study.
For example, logistics is specifically excluded from warfighting software.
This is a huge and deadly mistake, and one which would have been caught
if any knowledgeable warfighter had been on the committee. As the
history of warfare has shown, logistics and logistics support are
absolutely critical for warfighting--Desert Storm was successful primarily
because of the tremendous logistics effort that was mounted. As a former
warfighter, I know firsthand what can happen if an embarkation process
is fouled up--critical supplies (ordnance, medical, communications, etc.)
are delayed or misrouted, with resultant loss of life.
Similarly, simulation software is not included as a part of the warfighting
domain. With the rapidly increasing reliance on advanced, complex
simulation software, the DoD can ill afford to rely on marginal COTS
Finally, the committee shows a serious lack of knowledge/insight into the
workings of the military community, when it lumps financial systems in
with non-critical software. Again, as a former military person, I learned
from firsthand experience the importance of morale in military operations.
The best way to ensure a huge negative effect on morale is to have something
go wrong with the pay of your organization--I have seen top notch fighting
units basically rendered ineffective due to foul ups in the pay system--it's
hard to focus on your mission when you are worried that your family can't
buy groceries or pay the rent due to a mixup in your allotment back home.
My point here is that it is almost impossible to draw a line between those
software systems that are essential to warfighting and those which are
clearly not essential. The effect of the NRC recommendations will be to
encourage the use of non-Ada software and tools for DoD applications.
There is also a more subtle "hit" against Ada that exists throughout the
committee's report--the assumption/assertion that Ada is non-COTS. Quite
the contrary is true: Ada products and tools, are, by definition, COTS.
There are no mainstream Ada products which are owned by the government.
Ergo, if a new application is to be developed, Ada should be considered
as COTS just like any other language. Yet, the committee infers throughout
its report that the DoD has been solely responsible for the ongoing
investment in Ada products and tools. This is simply not true--all of the
mainstream Ada products were developed using private/commerical funds, with
the attendant entrepreneurial risks.
I was stunned at the content and justification for slide #16, entitled:
Ada Today: Overall Software Demand. This slide purports to show that
the demand for Ada in the overall software market is only 3%, as compared
to C (23%), C++ (23%), COBOL (11%), Visual Basic (13%), Java (4%), and
Power Builder (10%). The slide does NOT tell the reader that these
statistics are based on the committee's review of one issue of the
Washington Post newspaper and one issue of the LA Times, back in April
and March of 1996. They read through the classified ads of just those 2
issues, and counted the number of ads for each language.
In the conclusions slide (#63), there is a statement that "Further study
is not necessary." The clear implication is that the committee did such
a thorough job that the DoD can rely on its findings without further work.
What is NOT stated is the verbal explanation by the committee that the
"no further study" was a requirement levied on them by Mr. Paige. When
you consider their assertion that the recommendations are based on
inadequate data, anecdotal evidence, etc., the "no further study" claim
On page 25 of the complete report, there is a table showing the comparitive
advantages/disadvantages of the integration of COTS software vs. custom
software development. There are 7 advantages listed under COTS, vs. 5
advantages for custom work. There are 13 disadvantages cited for COTS, vs.
only 5 for custom software. This comparison flies in the face of the
overall recommendations to move away from custom work to COTS integration.
The report also includes a comparison of the technical features of Ada
(both Ada 83 and Ada 95) with other languages, as well as the results of
studies that show the error rates of Ada development vs. other langauges,
source lines of code per function point, and cost per SLOC. Ada does well
in all of these comparisons.
There is really nothing new in the NRC study--you can go back and look
at previous study/review efforts performed by the Defense Science Board,
the GAO, and Congressional Staff studies, and find most, if not all, of
the comments and recommendations made by the NRC committee. There seems
to be a "need" for the DoD to periodically conduct a study on its Ada-
related policies, which invariably result in negative press for Ada and
another round of the same poorly implemented guidelines and recommendations.
It is widely acknowledged that most program managers do not have the
necessary training/experience to make knowlegeable software-related
decisions. It has long been known that DoD's software problems are not
technology/language related; rather, they are management related.
The Ada policy has failed in the DoD--not because of deficiencies with Ada,
but with deficiencies in the implementation of the policy. I strongly
disagree with the committee's assertion that the criteria for driving DoD's
software policy for Ada is significantly different now than it was in
the early 1980s--if anything, there is even more of a need for a strong
Ada policy today than there was 15-20 years ago. The DoD's mission is
substantially different from the "mission" of commercial software firms--
the COTS products companies have the mission to make money, period.
They often shy away from standards and stable, well-designed technologies,
to increase their competitive advantage and their profits. The DoD can
ill afford to rely on such a mission orientation to support its warfighters.
I participated in a group briefing with the Joint Chiefs of Staff about 3
years ago, in which we had to address the issue of whether the DoD should
back away from its Ada requirement. The bottom line for that briefing was
the answer to 2 questions:
1. Does the DoD require well-engineered software to perform its mission?
2. Does Ada support the development/support of well-engineered software
better than other languages?
If the answer to either question is "No," then the DoD should back off from
its Ada requirement. To my knowledge, the answer to both questions is
still a resounding, "Yes," and I see no reason to move away from the DoD's
Ada requirement. If anything, it should be made stronger, and implemented
better than it has over the last 10 years.
The NRC study is a public relations/publicity disaster for the Ada community.
The committee's recommendations will be widely publicized and interpreted
as being a valid justification for the DoD to drop its Ada requirement.
The disclaimers regarding the inadequate data, etc., will not make it into
print. As a result of the publicitly, many DoD projects, especially in the
areas of simulation, logistics, C4I, and operations support, will relook
and perhaps rescind their plans to use Ada. These actions will have a
very bad business impact on Ada vendors, and will put pressure on the Ada
vendor community to get out of the Ada business and into more lucrative
The intermediate to long-term effect of the study will be a return to the
proliferation of software languages that existed in the DoD before the
advent of Ada. There may not be 400+ languages, but there will be a few
hundred different dialects/versions of languages, such as C, C++, etc.
This will put our warfighters at risk, and ensure a continuing rapid
increase in the cost of DoD software--software costs will consume an ever-
increasing amount of the DoD's annual budget.
The Software Engineering Plan Review will not amount to anything of value,
if the DoD even tries to implement it. There will not be enough of a
commitment to fund the training required to make it viable.
There are 2 action items that come to mind for the Ada community to consider:
1. Send Emmett Paige an e-mail, registering your disbelief that such a
report is being made public with so many glaring holes and discrepancies.
Ask him to ensure that the caveats stated by the committee are included
in DoD public statements and press releases. Ask him what effect the
report will have on DoD policy, and when we might see such results.
2. More importantly, it is imperative that key Members of Congress be
approached and briefed on the issues and the deficiencies of the report.
The only way to effectively combat the negative effects of the NRC study
is through the Congress. Keep in mind that in previous years, when the
DoD put in a budget that zeroed out the Ada line item, Congress put it back
in and increased it, due to the efforts of Ada proponents. The DoD does
not report to us ("the people"); rather, they report to the Congress.
Congress DOES report to us, so we either work the system as it is designed,
or we give up on Ada being a major part of future DoD systems.
One thing that is definitely NOT true: Ada will NOT succeed on its own
merits--it needs promotion, visibility, backing, and funding.
[This is the report summary previously posted by Richard Riehle.]
Ada and Beyond: Software Policies for the Department of
What should the Department of Defense do about the Ada
programming language? Despite the languages technical
capabilities, it has not been widely used outside of
military and other safety-critical applications, as was
hoped when the language was developed in the 1970s. The
Defense Departments policy on programming languages, which
requires the use of Ada for all new software development,
has resulted in many conflicts and has not been implemented
evenly. Meanwhile, commercial software applications have
grown rapidly, and do not generally use Ada.
The Committee on the Past and Present Contexts for the Use
of Ada in the Department of Defense was created by the
National Research Councils Computer Science and
Telecommunications Board to review the current policy, and
to examine the role of Ada in software development. The
study was requested by the Office of the Assistant secretary
of Defense for Command, Control, Communications, and
Intelligence. Released today, Ada and Beyond: Software
Policies for the Department of Defense presents the
committees findings and recommendations.
The committee concluded the vigorous support of Ada would
benefit warfighting systems, and recommends that the
Department of Defense should continue to use and promote Ada
in such systems. However, the committee found significant
problems with the two primary components of the Defense
Departments current strategy for Ada. First, the current
programming language policy requires the use of Ada for all
new defense software, which the committee finds to be overly
broad in scope. Second, the Defense Departments plan to
discontinue investments in both Ada technology and user-
community support will weaken the Ada infrastructure, and
will work against any requirement to use Ada in the future.
In summary, Ada and Beyond: Software Policies for the
Department of Defense concludes that the Defense Department
should take the following steps regarding Ada:
1. Continue to require Ada for its warfighting software and
drop the Ada requirement for its other software. In
commercially dominated areas, using Ada is generally less
cost-effective than using other languages. Requiring the
use of Ada in these areas would place the Defense Department
at a competitive disadvantage. In warfighting application
areas, however, Adas technical capabilities for real-time,
high assurance custom software are generally superior to
those of other languages. The Defense Departments
investments in Ada to date have created a set of Ada-based
production factors that give its system advantages in these
2. Provide roughly $15 million per year for Ada
infrastructure support, or drop the requirement to use Ada
entirely. The commercial marketplace will not sustain a
robust Ada infrastructure. However, a relatively modest
($15 million per year) investment at the margin by the
Defense Department would be sufficient to sustain a robust
Ada infrastructure for warfighting applications. The
Defense Departments inventory of more than 50 million lines
of warfighting software must either be maintained in Ada or
converted to another programming language. The recommended
investment is modest with this maintenance burden.
3. Make programming language decisions in the context of a
Software Engineering Plan Review process. The choice of
programming language is one component of the software-
engineering process, and should be considered in the broader
context of decisions regarding architecture, software reuse,
and software process management.
In the course of this study, the committee also concluded
that the currently available data are insufficient, on their
own, to accurately determine the impact of programming
language choice on the outcome of defense programs.
Committee briefings also highlighted the difficulty that
program managers have in finding data on which they can make
informed decisions. Based on the limitations of
availability data, the committee has made an additional
recommendation that the Defense Department institute a
corporate effort to collect software metrics to guide future
policy and management decisions.
The committee developed its recommendations, presented in
detail in Ada and Beyond: Software Policies for the
Department of Defense, in light of changes in the
environment for military software development. A decade
ago, the policy preference for Ada had some merit. Most
software was custom made, and Ada had a good track record in
delivering custom software with higher quality and lower
life-cycle costs. However, custom Ada solutions are no
longer competitive in many application areas, due to several
trends. First, software solutions increasingly depend on
commercial-off-the-shelf, or COTS, software, which provides
much of an applications information infrastructure,
including operating system, database management, networking,
user interface, and distributed processing functions. Much
of this software is written in programming languages other
than Ada, that often do not have readily available
interfaces to programs written in Ada.
Second, software for many application areas is increasingly
developed in so-called product-line architectures, which
enable software assets to be reused across families of
applications. These product-line solutions are driven by
strongly coupled production factors, including software
components, processes and methods, human resources, and
expertise in particular domains. In warfighting
applications areas such as weapon control and electronic
warfare, there is little commercial development, and the
Defense Department has established a strong community of
warfighting software developers whose production factors are
oriented to Ada. However, for the numerous defense
applications in which the market is dominated by commercial
solutions, such as finance and logistics, production factors
have been built around programming languages other than Ada
putting solutions at a disadvantage.
Several other factors have strongly influenced the
committees findings and recommendations. One is the
Defense Departments increasing emphasis on information
dominance. According to Secretary of Defense William Perry,
the Defense Departments warfighting strategy sustains and
builds on ... the application of information technology to
gain great military leverage to continue to give us [an]
unfair competitive advantage. A second factor is that the
Defense Department now has more than 50 million lines of
operational weapon systems software written in Ada, with a
great deal more under development. Most of this software is
in critical warfighting applications areas. There are no
quick and cheap ways to translate this software into other
languages. Polices and investment strategies that weaken
Ada support for this software are very risky.
A third factor is that, while language proliferation has
decreased, "polylingualism" is here to stay. One goal of
developing Ada was to reduce the proliferation of
programming languages used in defense systems, estimated to
be approximately 450 in the 1970s. The number of languages
used in defense systems has indeed decreased. The use of
machine and assembly languages has diminished, and the
number of third-generation languages in use has been
reduced. However, there has been a rapid increase in
development of fourth-generation languages by the commercial
sector, and increased use of these languages by defense
programs. A final factor is that choosing a programming
language is one of several key software engineering
decisions. The requirement to use Ada and the waiver
process both isolate programming language decisions from
other key software engineering decisions. Modern software
engineering considerations include the choices of computer
and software architectures, COTS components, and milestone
schedules. This arrangement creates an incentive for
defense programs to make decisions that are not optimal for
the Defense Department as an organization. Future
programming language decisions need to be part of an
integrated software engineering process.
This report will be available electronically on the World
Wide Web in mid-November at <www2.nas.edu/cstbweb>. Paper
copies will be made available in December.
Computer Science and Telecommunications Board
National Research Council
210 Constitution Avenue, N.W., Washington, D.C. 20418
Internet: [log in to unmask]
World Wide Web: WWW2.NAS.EDU/CTSBWEB
Milton Harris BLDG.RM 560
2001 Wisconsin Avenue, NW
Washington, D.C. 20007
(Off Whitehaven Street)
Committee on the Past and Present Contexts for the Use of
Ada in the Department of Defense
Barry Boehm, Chair
University of Southern California
Florida State University
Silicon Graphics, Inc.
University of California at Berkeley
Boeing Defense and Space Group
J. Elliot B. Moss
University of Massachusetts at Amherst
Rational Software Corporation
Carnegie Mellon University
S. Tucker Taft
Electronic Data Systems Corporation
Interactive Development Environments, Inc.
Barbara Liskov, Special Advisor
Massachusetts Institute of Technology
Study Director, CSTB
This report will be available electronically on the World
Wide Web in mid-November at <www2.nas.edu/cstbweb>. Paper
copies will be made available in December.