Turing Lecture Paper
On Building Systems That Will Fail
Fernando J. Corbató
It is an honor and a pleasure to accept the Alan Turing Award. My own work
has been on computer systems and that will be my theme. It is the essence
of systems that they are integrating efforts, requiring broad knowledge
of the problem area to be addressed, and that the detailed knowledge required
is rarely held by one person. Thus the work of systems is usually done by
teams and so it is in my case too. Hence I am accepting this award on behalf
of the many whom I have worked with as much as for myself. It is not practical
to name them all, so I will not. Nevertheless I would like to give special
mention to Marjorie Daggett and Bob Daley for their parts in the birth of
CTSS and to Bob Fano and the late Ted Glaser for their critical contributions
to the development of the Multics System.
Introduction
Let me turn now to the title of this talk: On Building Systems That Will
Fail. Of course the title I chose was a teaser. Some of the alternate titles
I came up with and discarded were: “On Building Messy Systems,”
but it seemed too frivolous and suggests there is no systematic approach;
“On Mastering System Complexity” sounded like I have all the answers.
The title that came closest, “On Building Systems that are likely to
have Failures” did not have the nuance of inevitability that I wanted
to suggest.
What I am really trying to address is the class of systems that for want
of a better phrase, I will call “ambitious systems.” It almost
goes without saying that ambitious systems never quite work as expected.
Things usually go wrong and sometimes in dramatic ways. And this leads me
to my main thesis, namely, that the question to ask when designing such
systems is not: “if something will go wrong, but when
will it?”
Some Examples
Now ambitious systems that fail are really much more common than we may
realize. In fact in some circumstances we strive for it revelling in the
excitement of the unexpected. For example let me remind you of our national
sport of football. The whole object of the game is for each team to play
at the limit of its abilities. Besides the sheer physical skills required,
one has the strategic intricacies, the ability to audibilize, and the quickness
to react to the unexpected, which are all a deep part of the game. Of course,
occasionally one team approaches perfection, all the plays work, and the
game becomes dull.
Another example of a system too ambitious for perfection, which I will not
dwell on because of its painful immediacy, is military warfare. The same
elements are there with opposing sides having to constantly improvise and
deal with the unexpected. In fact we get from the military that wonderful
acronym, SNAFU, which is politely translated as “situation normal,
all fouled up.” And if any of you are still doubtful, consider how
rapidly the phrases: “precision bombing” and “surgical strikes”
are replaced by “the fog of war” and “casualties from friendly
fire” as soon as hostilities begin.
On a somewhat more whimsical note, let me offer Boston driving as an example
of systems that will fail. Automobile traffic is an excellent case
of distributed control with a common set of protocols called traffic regulations.
The Boston area is notorious for the free interpretations which drivers
make of these pesky regulations and perhaps the epitome of it occurs in
the arena of the traffic rotary. A case can be made for rotaries. They are
efficient. There is no need to wait for sluggish traffic signals. They are
direct. And they offer great opportunities for creative improvisation, thereby
adding zest to the sport of driving.
One of the most effective strategies is for a driver when approaching a
rotary to rigidly fix his head staring forward, of course, secretly using
peripheral vision to the limit. It is even more effective if the driver
on entering the rotary, starts to speed up, and some drivers embellish this
last step by adopting a look of maniacal glee. The effect is, of course,
one of intimidation, and a pecking order quickly develops.
The only reason there are not more accidents is that most drivers have a
second component to the strategy, namely, they assume everyone else may
be crazythey often are rightand every driver is really prepared to stop
with inches to spare. Again we see an example of a system where ambitious
tactics and prudent caution lead to an effective solution.
So far the examples I have given may suggest that the failures of ambitious
systems come from the human element and that at least the technical parts
of the system can be built correctly. In particular turning to computer
systems, it is only a matter of getting the code debugged. Some assume rigorous
testing will do the job. Some put their hopes in proving program correctness.
But unfortunately there are many cases where none of these techniques will
always work [15]. Let me offer a modest example illustrated in Figure 1.
Consider the case of an elaborate numerical calculation with a variable,
f, representing some physical value, being calculated for a set of points
over a range of a parameter, t. Now the property of physical variables
is that they normally do not exhibit abrupt changes or discontinuities.
So what has happened here? If we look at the expression for f, we
see it is the result of a constant, k, added to the product of two
other functions, g and h. Looking further we see that the
function g has a behavior that is exponentially increasing with t.
The function h, on the other hand, is exponentially decreasing with
t. The resultant product of g and h is almost constant
with increasing t until an abrupt jump occurs and the curve for f
goes flat.
What has gone wrong? The answer is that there has been floating point underflow
at the critical point in the curve, i.e. the representation of the negative
exponent has exceeded the field size in the floating point representation
for this particular computer, and the hardware has automatically set the
value for the function h to zero. Often this is reasonable since
small numbers are correctly approximated by zerobut not in this case, where
our results are grossly wrong. Worse yet, since the computation of f
might be internal, it is easy to imagine that the failure shown here would
not be noticed.
Because handling correctly the pathology that this example represents is
extra engineering bother, it should not be surprising that the problem of
underflow is frequently ignored. But the larger lesson to be learned from
this example is that subtle mistakes are very hard to avoid and to some
extent are inevitable.
My next example I encountered when I was a graduate student programming
on the pioneering Whirlwind computer. One night while awaiting my turn to
use it, the graduate student before me, began complaining of how “tough”
some of his calculations were. What he said he was doing was computing the
vibrational frequencies of a particular wing structure for a series of cases.
In fact his equations were cubics, and he was using the iterative Newton-Raphson
method. For reasons he did not understand, his method was finding one of
the roots but not “converging” for the others. The fix he was
making was to change his program so that when he encountered one of these
tough roots, the program would abandon the iteration after a fixed number
of tries.
Now there were several things wrong. First, the coefficients to his cubic
equations were based on experimental data and some of his points were simply
bad so that, in fact, as Figure 2 illustrates, he only had one real root
and a pair of imaginaries. Thus his iterative method could never converge
for the second and third roots and the value of his first root was pure
garbage. Second, cubic equations have an exact analytic closed form solution
so that it was entirely unnecessary to use an iterative method. And third,
based on his incomplete model and understanding of what was happening, he
exercised very poor judgment in patching his program to ignore values that
were seemingly difficult to compute.
Ambitious System Properties
Let me turn next to some of the general properties of ambitious systems.
First, they are often vast and have significant organizational structure
going beyond that of simple replication. Second, they are frequently complicated
or elaborate and are too much for even a small group to develop. Third,
if they really are ambitious, they are pushing the envelope of what people
know how to do, and as a result there is always a level of uncertainty of
when completion is possible. Because one has to be an optimist to begin
an ambitious project, it is not surprising that underestimation of completion
time is the norm. Fourth, ambitious systems when they work, often break
new ground, offer new services and soon become indispensable. Lastly, it
is often the case that ambitious systems by virtue of having opened up a
new domain of usage, invite a flood of improvements and changes.
Now one could argue that ambitious systems are really only difficult the
first time or two. It really is only a matter of learning how to do it.
Once one has, then one simply draws up the appropriate PERT charts, hires
good managers, ensures an adequate budget and gets on with it. Perhaps there
are some instances where this works, but at least in the area of computer
systems, there is a fundamental reason it does not.
A key reason we cannot seem to get ambitious systems right is change. The
computer field is intoxicated with change. We have seen galloping growth
over a period of four decades and it still does not seem to be slowing down.
The field is not mature yet and already it accounts for a significant percentage
of the Gross National Product both directly and indirectly. More importantly
the computer revolution, this second industrial revolution, has changed
our life styles and allowed the growth of countless new application areas.
And all this change and growth not only has changed the world we live in
but has raised our expectations, spurring on increasingly ambitious systems
in such diverse areas as airline reservations, banking, credit cards, and
air traffic control to name only a few.
Behind the incredible growth of the computer industry is, of course, the
equally mind-boggling change that has occurred in the raw performance of
digital logic. Figure 3, which is not precise and which many of you have
seen before in some form, gives the performance of a top-of-the-line computer
by decade. The ordinate in MIPS is logarithmic as you can see. In particular
in the last decade, the graph becomes problem dependent so that the upper
right hand end of the line should break up into some sort of whiskers as
more and more computers are tailored for special applications and for parallelism.
Complicating matters too is that parallelism is not a solution for every
problem. Certain calculations that are intrinsically serial, such as rocket
trajectories, have very limited benefit from parallel computers. And one
of course is reminded of the old joke about the Army way of speeding up
pregnancy by having nine women spend one month at the task.
As Figure 4 makes clear, it is not just performance that has fueled growth
but rather cost/performance, or simply put, the favorable economics. The
graph is an oversimplification but represents the cost for a given performance
computer model over the last four decades. Again the ordinate is logarithmic,
going from 10 million dollars in 1950 down to 1 thousand dollars in 1990.
As we approach the present, corresponding to a personal computer, the graph
really should become more complicated since one consequence of computers
becoming super-cheap is that increasingly, they are being embedded in other
equipment. The modern automobile is but one example. And it remains to be
seen how general-purpose the current wave of palm-sized computers will be
with their stylus inputs.
Further, when we look at a photograph taken back around 1960 of a”machine
room” staffed with one lone operator, we are reminded of the fantastic
changes that have occurred in computer technology. The boxes are huge, shower
stall sized, and the overall impression is of some small factory. You were
supposed to be impressed and the operator was expected to maintain decorum
by wearing a necktie. And if he did not, at least you could be sure an IBM
maintenance engineer would.
Another reminder of the immense technological change which has occurred
is in the physical dimensions of the main memories of computers. For example,
if one looks at old photographs taken in the mid 1950’s of core memory systems,
one typically sees a core memory plane roughly the size of a tennis racquet
head which could hold about 1000 bits of information. Contrast that with
today’s 4 megabit memory chips which are smaller than one’s thumb.
Now the basis of the award today is largely for my work on two pioneering
time-sharing systems, CTSS [1-2] and Multics [3-8]. Indeed it is out of
my involvement with those two systems that I gained the system building
perspective I am offering. It therefore seems appropriate to take a brief
retrospective look at these two systems as examples of ambitious systems
and to explore the reasons why the complexity of the tasks involved made
it almost impossible to build the systems correctly the first time [10].
CTSS, The Compatible Time-Sharing System
Looking first at CTSS, I have to remind you of the dark ages that then existed.
This was the early 1960’s. The computers of the day were big and expensive
and the administrators of computing centers felt obliged to husband the
precious resource. Users, i.e. programmers, were expected to submit a computing
job as a deck of punched cards; these were then combined into a batch with
other jobs onto a magnetic tape and the tape was processed by the computer
operating system. It had all the glamour and excitement of dropping one’s
clothes off at a laundromat.
The problem was that even for a trivial input typing mistake, the job would
be aborted. Time-sharing, as most of you know, was the solution to the problem
of not being able to interact with computers. The general vision of modern
time-sharing was primarily spelled out by John McCarthy, who I am pleased
to note is a featured speaker at this conference. In England, Christopher
Strachey independently came up with a limited kind of interactive computing
but it was aimed mostly at debugging. Soon there were many groups around
the country developing various forms of interactive computing, but in almost
all cases, the resulting systems had significant limitations.
It was in this context that my own group developed our version of the time-sharing
vision. We called it The Compatible Time-Sharing System, or CTSS for short.
Our initial aspirations were modest. First, it was meant to be a demonstration
prototype before more ambitious designs being attempted by others could
be implemented. Second, it was intended that general-purpose programming
could be done. And third, it was meant to be possible to run most of the
large body of software that had been developed over the years in the batch-processing
environment. Hence the name.
The basic scheme used to run CTSS was simple. The supervisor program, which
was always in main memory, would commutate among the user programs, running
each in turn for a brief interval with the help of an interval timer. As
Figure 5 indicates, user programs could do input/output with the typewriter-like
terminals and with the disk storage unit as well.
But the diagram is oversimplified. The key difficulty was that main memory
was in short supply and not all the programs of the active users could remain
in memory at once. Thus the supervisor program not only had to move programs
to and from the disk storage unit, but it also had to act as an intermediary
for all I/O initiated by user programs. Thus all the I/O lines should only
point to the supervisor program.
As a further complication, the supervisor program had to prevent user programs
from trampling over one another. To do this required special hardware modifications
to the processor such that there were memory bound registers that could
only be set by the supervisor. Nevertheless despite all the complications,
the simplicity of the initial supervisor program allowed it to occupy about
22K bytes of storageless storage than required for the text of this talk!
Most of the battles of creating CTSS involved solving problems which at
the time did not have standard solutions. For example:
There were no standard terminals. There were no simple modems. I/O to the
computer was by word and not by character, and worse yet, did not accommodate
lower case letters. The computers of the day had neither interrupt timers
nor calendar clocks. There was no way to prevent user programs from issuing
raw I/O instructions at random. There was no memory protection scheme. And,
there was no easy way to store large amounts of data with relatively rapid
random access.
The overall result of building CTSS was to change the style of computing
but there were several effects that seem worth noting. One of the most important
was that we discovered that writing interactive software was quite different
from software for batch operation and even today, in this era of personal
computers, the evolution of interactive interfaces continues.
In retrospect, several design decisions contributed to the success of CTSS,
but two were key. First, we could do general-purpose programming and, in
particular, develop new supervisor software using the system itself. Second,
by making the system able to accommodate older batch code, we inherited
a wealth of older software ready-to-go.
One important consequence of developing CTSS was that for the first time
users had persistent on-line storage of programs and data. Suddenly the
issues of privacy, protection and backup of information had to be faced.
Another byproduct of the development was that because we operated terminals
via modems, remote operation became the norm. Also the new-found freedom
of keeping information on-line in the central file system suddenly made
it especially convenient for users to share and exchange information among
themselves.
And there were surprises too. To our dismay, users who had been enduring
several hour waits between jobs run under batch processing, were suddenly
restless when response times were more than a second. Moreover many of the
simplifying assumptions that had allowed CTSS to be built so simply such
as a one level file system, suddenly began to chafe. It seemed like the
more we did, the more users wanted.
There are two other observations that can be made about the CTSS system.
First, it lasted far longer than we expected. Although CTSS had been demonstrated
in primitive form in November 1961, it was not until 1963 that it came into
wide use as the vehicle of a Project MAC Summer Study. For a time there
were two copies of the system hardware, but by 1973 the last copy was turned
off and scrapped primarily because the maintenance costs of the IBM 7094
hardware had become prohibitively expensive, and up to the bitter end, there
were users desperately trying to get in a few last hours of use.
Second, the then-new transistors and large random-access disk files were
absolutely critical to the success of time-sharing. The previous generation
of vacuum tubes was simply too unreliable for sustained real-time operation
and, of course, large disk files were crucial for the central storage of
user programs and data.
A Mishap
Now my central theme is to try to convince you that when you have a multitude
of novel issues to contend with while building a system, mistakes are inevitable.
And indeed, we had a beauty while using CTSS. Let me describe it:
What happened was that on one afternoon at Project MAC where CTSS was being
used as the main time-sharing workhorse, any user who logged in, found that
instead of the usual message-of-the-day typing out on his terminal, he had
the entire file of user passwords instead. This went on for fifteen or twenty
minutes until one particularly conscientious user called up the system administrator
and began the conversation with: “Did you know that…?” Needless
to say there was general consternation with this colossal breach of security,
the system was hastily shut down and the next twelve hours were spent heroically
changing everyone’s password. The question was how could this have happened?
Let me explain.
To simplify the organization of the initial CTSS system, a design decision
had been made to have each user at a terminal associated with his own directory
of files. Moreover the system itself was organized as a kind of quasi-user
with its own directory that included a large number of supporting applications
and files including the message-of-the day and the password file. So far,
so good. Normally a single system programmer could login to the system directory
and make any necessary changes. But the number of system programmers had
grown to about a dozen in number, and, further, the system by then was being
operated almost continuously so that the need to do live maintenance of
the system files became essential. Not surprisingly, the system programmers
saw the one-user-to-a-directory restriction as a big bottleneck for themselves.
They thereupon proceeded to cajole me into letting the system directory
be an exception so that more than one person at a time could be logged into
it. They assured me that they would be careful to not make mistakes.
But of course a mistake was made. Overlooked was a software design decision
in the standard system text editor. It was assumed that the editor would
only be used by one user at a time working in one directory so that a temporary
file could have the same name for all instantiations of the editor. But
with two system programmers editing at the same time in the system directory,
the editor temporary files became swapped and the disaster occurred.
One can draw two lessons from this: First, design bugs are often subtle
and occur by evolution with early assumptions being forgotten as new features
or uses are added to systems; and second, even skilled programmers make
mistakes.
Multics
Let me turn now to the development of Multics [9]. I will be brief since
the system has been documented well and there have already been two retrospective
papers written [11, 13]. The Multics system was meant to do time-sharing
“right” and replace the previous ad hoc systems such as CTSS.
It started as a cooperative effort among Project MAC of MIT, the Bell Telephone
Laboratories, and the Computer Department of General Electric, later acquired
by Honeywell. In our expansiveness of purpose we took on a long list of
innovations.
Some of the most important ones were: First, we introduced into the processor
hardware the mechanisms for paging and segmentation along with a careful
scheme for access control. Second, we introduced an idea for rings of protection
around the supervisor software. Third, we planned from the start that the
system would be composed of interchangeable multiple processors, memory
modules, and so forth. And fourth, we made the decision to implement nearly
all of the system in the newly defined compiler language, PL/I.
Let me make a few observations about the Multics experience. The novel hardware
we had commissioned meant that the system had to be built from the ground
up so that we had an immense task on our hands.
The decision to use a compiler to implement the system software was a good
one, but what we did not appreciate was that new language PL/I presented
us with two big difficulties: First, the language had constructs in it which
were intrinsically complicated, and it required a learning period on the
part of system programmers to learn to avoid them; second, no one knew how
to do a good job of implementing the compiler. Eventually we overcame these
difficulties but it took precious time.
That Multics succeeded is remarkable for it was the result of a cooperative
effort of three highly independent organizations and had no administrative
head. This meant decisions were made by persuasion and consensus. As a consequence,
it was difficult to reject weak ideas until considerable time and effort
had been spent on them.
The Multics system did turn into a commercial product. Some of its major
strengths were: the virtual memory system, the file system, the attention
to security, the ability to do online reconfiguration, and the information
backup system for the file system.
And, as was also true with CTSS, many of the alumni of the Multics development
have gone on to play important roles in the computing field [16].
A few more observations can be made about the ambitious Multics experience.
In particular, we were misled by our earlier successes with previous systems
such as CTSS, where we were able to build them “brick-by-brick,”
incrementally adding ideas to a large base of already working software.
We also were embarrassed by our inability to set and meet accurate schedules
for completion of the different phases of the project. In hindsight we should
not have been, for we had never done anything like it before. However in
many cases, our estimations should have been called guesses.
The UNIX system [12] was a reaction to Multics. Even the name was a joke.
Ken Thompson was part of the Bell Laboratories’ Multics effort, and, frustrated
with the attempts to bring a large system development under control, decided
to start over. His strategy was clear. Start small and build up the ideas
one by one as he saw how to implement them well. As we all know, UNIX has
evolved and become immensely successful as the system of choice for workstations.
Still there are aspects of Multics that have never been replicated in UNIX.
As a commercial product of Honeywell and Bull, Multics developed a loyal
following. At the peak there were about 77 sites worldwide and even today
many of the sites tenaciously continue for want of an alternative.
Sources of Complexity
The general problem with ambitious systems is complexity. Let me next try
to abstract some of the major causes. The most obvious complexity problems
arise from scale. In particular, the larger the personnel required, the
more levels of management there will be. We can see the problem even if
we use simplistic calculations. Thus if we assume a fixed supervision ratio,
for example six, the levels of management will grow as the logarithm of
the personnel. The difficulty is that with more layers of management, the
top most layers become out of touch with the relevant bottom issues and
the likelihood of random serendipitous communication decreases.
Another problem of organizations is that subordinates hate to report bad
news, sometimes for fear of “being shot as the messenger” and
at other times because they may have a different set of goals than the upper
management.
And lastly, large projects encourage specialization so that few team members
understand all of the project. Misunderstandings and miscommunication begin,
and soon a significant part of the project resources are spent fighting
internal confusion. And, of course, mistakes happen.
My next category of complexity arises because of new design domains. The
most vivid examples come from the world of physical systems, but software
too is subject to the same problems albeit often in more subtle ways.
Consider the destruction of the Tacoma Narrows Bridge, in Washington State,
on November 7, 1940. The bridge had been proudly opened about four months
earlier. Many of you have probably seen the amateur movie that was fortuitously
made of the collapse. What happened is that a strong but not unusual cross-wind
blew that day. Soon the roadbed, suspended by cables from the main span,
began to vibrate like a reed, and the more it flexed, the better cross-section
it presented to the wind. The result was that the bridge tore itself apart
as the oscillations became large and violent. What we had was a case of
a new design domain where the classic bridge builder, concerned with gravity-loaded
structures, had entered into the realm of aeronautics. The result was a
major mistake.
Next let us look at the complexities that arise from human usage of computer
systems. In using online systems which allow the sharing or exchanging of
information–and here networked workstations clearly fall in this class–one
is faced with a dilemma: if one places total trust in all other users, one
is vulnerable to the anti-social behavior of any malicious userconsider
the case of viruses; but if one tries to be totally reclusive and isolated,
one is not only bored, but one’s information universe will cease to grow
and be enhanced by interactions with others. The result is that most of
us operate in a complicated tradeoff zone with various arrangements of trust
and security mechanisms. Even such simple ideas as passwords are often a
problem: they are a nuisance to remember, they can easily be compromised
inadvertently, and they cannot be selectively revoked if shared. Privacy
and security issues are particularly difficult to deal with since responsibilities
are often split among users, managers, and vendors. Worse yet, there is
no way to simply “look” at a system and determine what the privacy
and security implications are. It is no wonder that mistakes happen all
the time in this area.
One of the consequences of using computer systems is that increasingly information
is being kept online in central storage devices. Computer storage devices
have become remarkably reliableexcept when they breakand that is the rub.
Even the most experienced computer user can find himself lulled into a false
sense of security by the almost perfect operation of today’s devices. The
problem is compounded by the attitude of vendors, not unlike the initial
attitude of the automobile industry toward safety, where inevitable disk
failure is treated as a negative issue that dampens sales.
What is needed is constant vigilance against a long list of “what ifs”:
hardware failure, human slips, vandalism, theft, fire, earthquakes, long-term
media failure, and even the loss of institutional memories concerning recovery
procedures. And as long as some individuals have to “learn the hard
way,” mistakes will continue to made.
A further complication in discussing risk or reliability is that there is
not a good language with which to carry on a dialog. Statistics are as often
misapplied as they are misunderstood. We also get absurd absolutes such
as “the Strategic Defense Initiative will produce a perfect unsaturatable
shield against nuclear attack” [14] or “it is impossible for the
reactor to overheat.” The problem is that we always have had risks
in our lives, we never have been very good at discussing them, and with
computers we now have a lot of new sources.
Another source of complexity arises with rapid change, change which is often
driven by technology improvements. A result is that changes in procedures
or usage occur and new vulnerabilities can arise. For example, in the area
of telephone networks, the economies and efficiencies of fiber optic cables
compared to copper wire are rapidly causing major upgrades and replacements
in the national telephone plant. Because one fiber cable can carry at a
reasonable cost the equivalent traffic of thousands of copper wires, fiber
is quickly replacing copper. As a result a transformation is likely to occur
where network links become sparser over a given area and multiply interconnected
nodes become less connected.
The difficulty is that there is reduced redundancy and a much higher vulnerability
to isolated accidents. In the Chicago area not long ago there was a fire
at a fiber optics switching center that caused a loss of service to a huge
number of customers for several weeks. More recently in New York City there
was a shutdown of the financial exchanges for several hours because of a
single mishap with a backhoe in New Jersey. Obviously in both instances,
efficiency had gotten ahead of robustness.
The last source of complexity that I will single out arises from the frailty
of human users when forced to deal with the multiplicity of technologies
in modern life. In a little more than a century, there has been an awesome
progression of technological changes from telephones and electricity, through
automobiles, movies and radioI won’t even try to complete the list since
we all know it well. The overall consequence has been to produce vast changes
in our lifestyles and we see these changes even happening today. Consider
the changes in the television editing styles that have occurred over a few
decades, the impact of viewgraph overhead projectors on college classrooms,
and the way we now do our banking with automatic teller machines. And the
progression of life style changes continues at a seemingly more rapid pace
with word processing, answering machines, facsimile machines, and electronic
mail.
One consequence of the many lifestyle changes is that some individuals feel
stressed and over stimulated by the plethora of inputs. The natural defense
is to increasingly depend on others to act as information filters. But the
combination of stressful life-styles and insulation from original data will
inevitably lead to more confusion and mistakes.
Conclusions
I have spent most of this talk trying to persuade you that failures in complex,
ambitious systems are inevitable. However I would be remiss if I did not
address what can be done about it. Unfortunately the list I can offer is
rather short but worthy of brief review.
First it is important to emphasize the value of simplicity and elegance,
for complexity has a way of compounding difficulties and as we have seen,
creating mistakes. My definition of elegance is the achievement of a given
functionality with a minimum of mechanism and a maximum of clarity.
Second, the value of metaphors should not be underestimated. Metaphors have
the virtue that they have an expected behavior that is understood by all.
Unnecessary communication and misunderstandings are reduced. Learning and
education are quicker. In effect metaphors are a way of internalizing and
abstracting concepts such that one’s thinking can be on a higher plane and
low-level mistakes are avoided.
Third, use of constrained languages for design or synthesis is a powerful
methodology. By not allowing a programmer or designer to express irrelevant
ideas, the domain of possible errors becomes far more limited.
Forth, one must try to anticipate both errors of human usage and of hardware
failure and properly develop the necessary contingency paths. This process
of playing “what if” is not as easy as it may sound since implicit
is the need to attach likelihoods of occurrence to events and to address
issues of the independence of failures.
Fifth, it should be assumed in the design of a system, that it will have
to be repaired or modified. The overall effect will be a much more robust
system, where there is a high degree of functional modularity and structure,
and repairs can be made easily.
Sixth, and lastly, on a large project, one of the best investments that
can be made is the cross-education of the team so that nearly everyone knows
more than he or she needs to know. Clearly with educational redundancy,
the team is more resilient to unexpected tragedies or departures. But in
addition, the increased awareness of team members can help catch global
or systemic mistakes early. It really is a case of “more heads are
better than one.”
Finally, I have touched on many different themes in this talk but I will
single out three:
First, the evolution of technology supports a rich future for ambitious
visions and dreams that will inevitably involve complex systems.
Second, one must always try to learn from past mistakes, but at the same
time be alert to the possibility that new circumstances require new solutions.
And third, one must remember that ambitious systems demand a defensive philosophy
of design and implementation. Or in other words, “Don’t wonder if
some mishap may happen, but rather ask what one will do about it
when it does occur.”
References
1. Corbató, F. J., Daggett, M. M., and Daley, R. C., “An Experimental
Time-Sharing System,” Proceedings of the Spring Joint Computer Conference,
May 1962.
2. Corbató, F. J., Daggett, M. M., Daley, R. C., Creasy, R. J., Hellwig,
J. D., Orenstein, R. H., and Horn, L. K., The Compatible Time-Sharing
System: A Programmer’s Guide, M.I.T. Press, June 1963.
3. Corbató, F. J., and Vyssotsky, V. A., “Introduction and Overview
of the Multics System,” Proceedings FJCC, 1965.
4. Glaser, E. L., Couleur, J. F. and Oliver, G. A. “System Design of
a Computer for Time-Sharing Applications,” Proceedings FJCC, 1965.
5. Vyssotsky, V. A., and Corbató, F. J., “Structure of the Multics
Supervisor,” Proceedings FJCC, 1965.
6. Daley, R. C. and Neumann, P. G. “A General-Purpose File System for
Secondary Storage,” Proceedings FJCC, 1965.
7. Ossanna, J. F., Mikus, L. and Dunten, S. D. “Communications and
Input-Output Switching in a Multiplex Computing System,” Proceedings
FJCC, 1965.
8. David, E. E., Jr. and Fano, R. M. “Some Thoughts About the Social
Implications of Accessible Computing,” Proceedings FJCC, 1965.
9. Organick, E. I. The Multics System: An Examination of its Structure,
MIT Press, 1972.
10. Corbató, F. J. “Sensitive Issues in the Design of Multi-Use
Systems,” (Unpublished), Lecture transcription of Feb. 1968, Project
MAC Memo M-383.
11. Corbató, F. J., Clingen, C. T., and Saltzer, J. H.,”Multics:
The First Seven Years,” Proceedings of the SJCC, May 1972, pp.
571-583.
12. Ritchie, D. M. and Thompson, K. The UNIX time-sharing system. CACM
17, 7 (July 1974), 365-375.
13. Corbató, F. J., and Clingen, C. T., “A Managerial View of
the Multics System Development,” an article in the book Research
Directions in Software Technology edited by P., Wegner, M.I.T. Press,
1979. (Also published in Tutorial: Software Management, Reifer,
Donald J. (ed), IEEE Computer Society Press, 1979; Second Edition 1981;
Third Edition, 1986.)
14. Parnas, D. L. “Software Aspects of Strategic Defense Systems,”
American Scientist, Nov. 1985. An excellent critique on the difficulties
of producing software for large-scale systems.
15. Brooks, F. P., Jr. “No Silver Bullet,” IEEE Computer,
April 1987, 10-19.
16. Apropos the theme of this lecture, P. G. Neumann, a Multics veteran,
has become a major contributor to the literature of computer related risks.
He is the editor of the widely-read network magazine “Risks-Forum”,
writes the “Inside Risks” column for the CACM, and periodically
creates digests in the ACM Software Engineering Notes.
—
