<-- Back to books

Defense Science Board: Software production

Fred Brooks

Original report here

1. Statutes, regulations, and cultural norms that get in the way of deploying software to the field quickly weaken our national security and expose our nation to risk.

2. Faster reduces risk because it demands focus on the critical functionality rather than over-specification or bloated requirements. It also means we can identify trouble earlier and take faster corrective action, which reduces cost, time, and risk. Faster leads to increased reliability: the more quickly software/code is in the hands of users, the more quickly feedback can focus on efforts to deploy greater capability. Faster gives us a tactical advantage on the battlefield by allowing operation and response inside our adversaries’ observe–orient– decide–act (OODA) loops. Faster is more secure.

3. Talented software developers and acquisition personnel with software experience are often put in jobs that do not allow them to make use of those talents, particularly in the military where rotating job assignments may not recognize and reward the importance of software development experience.

4. Engineers should be protected as far as possible from meetings and allowed to spend their time thinking about problems and writing code. This means getting concise value driven specs up front.

5. The lives of those who defend our nation ultimately depend on DoD’s ability to redefine its approach to delivering combat-critical software to the field.

6. Current practice in DoD programs is that each individual program builds its own infrastructure for computing, development, testing, and deployment, and there is little ability to build richer development and testing capabilities that are possible by making use of common infrastructure. Instead, we need to create, scale, and optimize an enterprise-level architecture and supporting infrastructure that enables creation and initial fielding of software within six months and continuous delivery of improvements on a three-month cycle.

7. Few fields have so large a gap between best current practice and average current practice.

8. The big problems are not technical. In spite of the substantial technical development needed in requirements setting, metrics and measures, tools, etc., the Task Force is convinced that today’s major problems with military software development are not technical problems, but management problems. Hence we call for no new initiatives in the development of the technology, some modest shift of focus in the technology efforts under way, but major re-examination and change of attitudes, policies, and practices concerning software acquisition.

9. For major new software builds, we recommend that competitive level-of-effort contracts be routinely let for determining specifications and preparing an early prototype (Rec. #26). The work of specification is so crucial, and yet its fraction of total cost is so small, that we believe duplication in this phase will save money in total program cost, and surely save time. After a converged-specification has been prepared and validated by prototyping, a single competitively-bid contract for construction is appropriate.

10. Misjudgements in requirements badly hurt effectiveness, cost, and schedule. Such misjudgements abound. Most common is the specification of over-rich function, whose bad effects on size and performance become evident only late in the design cycle. Another common error is the misimagination of how user interfaces should work.

11. The systems built today are just too complex for the mind of man to foresee all the ramifications purely by the exercise of the analytic imagination.

12. Custom-built software has historically been notoriously bad in the respects of ‘cheapest, fastest, surest’, without market pressures to fix it.

13. Even if the best off-the-shelf product is not ultimately used, adopting it for pilot use will help radically in setting specifications for the custom product.

14. Given the regulatory and financial structure of DoD contracting outlined above, a vendor can participate in substantial DoD software contracting only if it:

• has staying power to average over crests and troughs in contracting business,

• has a management superstructure to cope with the regulatory overhead, including a Program Control Management System, and specialized accounting to meet DoD STD 7000.2,

• has an infrastructure of technical specialists to deal with configuration management per regulation, documentation per regulation, acceptance testing, etc.

• therefore has a critical mass of skilled people so that it can carry several contracts at once, both to smooth crests and troughs and to pay for the administrative and technical superstructure necessary to cope with the regulatory overhead.

  As a result, we estimate there to be only about two dozen houses that are regularly available to participate in the development of substantial mission-critical software systems, and this number grows slowly. On the contrary, several vendors are seriously considering leaving the DoD business entirely, and refusing to bid in the future.

  The net result of this small stable of vendors in a time of exponential growth of work is sure to be higher bids and longer schedules. It is in DoD’s interests, and the nation’s interests, for DoD to make itself an attractive customer. We believe the net costs to the nation of weapon-system software will be lower if it does. Present practices are penny-wise and pound-foolish in many petty ways.

15. To those technical arguments we would add an economic one: the proposed DoD Supplement 27.4 is, intentionally or otherwise, grabby in spirit and effect. No fair-minded person would consider it to propose equitable treatment among vendors, or between DoD and vendors. Its lack of clarity and clumsiness of drafting will occasion litigation. The net effect will be to further shrink the vendor pool, at great cost to DoD and the taxpayer.

16. Although some parts of the recent Draft DOD-STD-2167A appear to encourage modern methods, the draft as a whole continues to reinforce exactly the document-driven, specify-then-build approach that we believe causes so many of DoD’s software problems.