May 2nd, 2012
(written by lawrence, however indented passages are often quotes)
This sounds exhausting and tedious. It’s also incredible Kent M. Pitman was working on the Lisp standard for at least 10 years. He was new to formal standards processes in 1986, his book was published 10 years later, in 1996. That is a long time for a story to unfold.
5.2 Early Politics and Posturing
Never having been part of a formal standards process, I didn’t quite know what to expect. The very fact that there are a lot of rules was very daunting and confusing.
Work was divided up. Committees were assigned to work on various subtasks.
In researching this talk, I ran across notes about such division of labor that I had scribbled during an early meeting. It primarily illustrates how, in my youth, I was struggling to understand the organizational mechanisms at work. Among other things on the page I had scribbled the following phrases:
Lest someone find my handwriting illegible, the notes include these remarks:
“due process is an illusion”
“gerrymandering (Pittsburgh committee)”
“turn opponents on each other and let them battle each other down and/or demonstrate that you couldn’t have done better because problem was unreasonable in general.”
“soliciting volunteers gives critics a thing to do, which dilutes their passion and pacifies them by making them feel involved.”
“start process leaving details blind, then manipulate detail assumption, finally it will be too hard to back out of.”
I don’t know if these things really aptly described what was actually going on. They may have been, in some cases. They are just the personal guesses of someone who was new to the process and struggling to understand it. But I think it’s fair to say that early in the standards process there was a lot of tactical posturing between the committees.
It’s equally reasonable to note that while inter-corporate tactical posturing may have appeared to serve the individual vendors represented, it probably kept the vendors from cooperating in ways that later turned out to be essential. Before the process could move forward, a new understanding would have to be reached where we started to work more together, and less at odds with one another.
Ultimately, various committees were formed to work on various aspects of the standard, reviewing the relevant aspects of the language and determining whether changes or additions needed to be made. These committees included ones named Charter, Compiler, Conditions, Conformance, Namespaces, Iteration, and Objects. In addition there was a catch-all committee called Cleanup that handled small matters that didn’t seem to fit in any other committee.
All of these committees probably had stories worthy of note, but I will mention only a subset of them for reasons of space.
5.3 Charter: Susan Ennis (1986)
Sitting in a room for a good part of a meeting coming up with words to write as part of our mission did not seem like a good use of time to me at that moment. But I went along with it because there seemed no stopping it. In retrospect, I consider this a major administrative contribution and I credit the committee chair, Susan Ennis, for getting us to do it.
What I found later was that there were many times during work on the standard where people disagreed about what the right way to proceed was. In many of those cases, we might have been hopelessly deadlocked, each wanting to pursue a different agenda, but I was able instead to point to the charter and say, “No, we agreed that this is how we’d resolve things like this.”
Without a doubt, the most useful sentence in the charter was the one that said, “Aesthetic considerations shall also be weighed, but as subordinate criteria” [J13SD05]. Our goal in writing the charter had been to produce an industrial-strength language, and the time spent writing that one sentence, emphasizing the importance of pragmatics over abstract concerns about elegance, broke a lot of deadlocks. It’s not that any of us wanted the language to be unaesthetic. But in practice, given the compatibility constraints we had going into the project, there were a lot of details of the language that were simply fixed givens, and had we been obliged to fix all of those to the level of detail some were wishing for, it’s likely the process would never have terminated.
The charter also identified which projects were in scope and out of scope for our work, and which were required features and which were optional.
The time spent writing the charter later paid for itself many times over and it’s an exercise I recommend to any committee engaged in any large endeavor over a period of time.
5.4 Cleanup: Larry Masinter
Most people who have seen the Common Lisp HyperSpecTM [Pitman96] have spent at least some time browsing the X3J13 issues section, in which the nature of various issues considered in the design of the language are recorded for history. The forms used were insisted upon by Larry Masinter. Once again, I (and perhaps others) worried that having to fill out forms was just a lot of pointless make-work. However, it quickly became apparent that he was right in advocating this approach.
Using forms with standardized fields like “problem description” and “proposal” where each proposal had to be analyzed for “cost to users,” “cost to implementers,” etc. led those submitting changes to consider their proposal from all sides before making a suggestion. It also made it easy for reviewers to determine which proposed changes were adequately explained and which were controversial.
It had an additional benefit that is a little more subtle. There was implicitly a kind of philosophy of how contributions from collaborating individuals were merged using these forms. For example, a good problem description had to satisfy everyone. If two people saw a problem from a different point of view, both people’s points of view were merged into the problem description, making the problem more complicated, and making solutions sometimes harder to achieve. But this was essential to addressing porting problems, for example. One couldn’t just solve how a file system operation would work on one operating system unless the solution was going to work on other operating systems, too. On the other hand, the “proposal” field was very different. If two people disagreed on a proposal, they each wrote their own proposal so that proposals could be internally consistent and coherent. This meant that a single problem often had several proposed solutions with different costs and benefits, and the committee had to decide which was the stronger proposal.
The procedure worked well and solved hundreds of small issues that came up. But it was not a property of the ANSI process that we used this procedure. It was unique to Masinter’s way of doing things. This was just one of many details of the process that was greatly affected by the presence of and style of a single individual.
5.5 Project Editor: Kathy Chapman (1987)
The project was far too complicated to be completed by the committee itself. It required some sort of outside investment or it would never be done.
In 1987, Gary Brown, of Digital Equipment Corporation (DEC), arranged for the services of Kathy Chapman to be made available to the committee. Kathy became the first Project Editor and did some very important foundational work.
CLTL had been accepted as a “base document.” Under Kathy’s supervision, and with Steele’s explicit approval, CLTL was reorganized from its tutorial style to a dictionary style, working out many of the book design details that would carry through into the final work.
Within the code, she kept meticulous back-pointers to the source location of each of the bits of moved text so that it was later possible to track down the origin of passages that surprised people.
Kathy also used markup internally that distinguished the selection of an appropriate font for a function name or a variable name even though the fonts would turn out to be the same in the printed text. Although this practice had no effect on the printed text, however, it did have a subtle effect later because the TEX to HTML processor used to produce the Common Lisp HyperSpec would be able to rely on that information to create better cross-reference links.
5.6 ISLISP (SC22/WG16) begins (1988)
At this time also, work began by an ISO committee designated as SC22/WG16, an international standards body concerned with Lisp. (Later, at the request of John McCarthy, Lisp’s creator, this body would clarify that its goal was to produce a dialect named ISLISP, and not to produce a standard for all Lisp. In fact, in researching this paper I found records from an early meeting of X3J13 stating that McCarthy had made a similar request there, too—that the American standard be one for Common Lisp, not for Lisp.)
Participants in the international standard included representatives from various Lisp communities worldwide, including Common Lisp, Eulisp, Le Lisp, and Scheme.
The first meeting was in Paris in 1988 and was co-located with International Workshop on Lisp Evolution and Standardization (IWOLES). Richard Gabriel was designated by X3J13 as the United States’ representative to this committee. I also attended.
5.7 New Project Editor: Kent Pitman (1989)
In 1989, Digital Equipment Corporation (DEC), having already invested a substantial amount in hosting a Project Editor, decided it wanted someone else to take over the role. There was a call for a new Project Editor.
I volunteered—without first talking to my employer, Symbolics—with a few conditions. One condition was that funding could be found. Another was that I not have to write any status reports. I wanted to spend all my time writing. I told the committee it was fine if they sent people to peek into my office and see if I was working, but that I didn’t want to have the task be high overhead. I told them that if they could get a better offer, they should take it.
I wish I had recorded Richard Gabriel’s response to my offer because it was classic. He said something tepid like “I think Kent is the minimal acceptable kind of person to get this done.” I’m not sure by what standard he said this. But I came to believe that he was wrong. Perhaps he was focused on my technical or writing capabilities. I wasn’t the strongest technical person nor the best writer. But the job didn’t turn out to need that.
What it needed was someone who had a mix of skills, and I had a reasonably good mix. It needed me to be a technical person one day and a writer on other days.
Also, if it was going to involve someone with technical skills, it needed that person to be someone who was able to separate partisan technical advocacy (which could be done at meetings) from neutral editorial action (while editing the document). There was a lot of text changing, and it needed to change in purely editorial ways. Had I confused my being allowed to edit the document for editorial reasons with my being allowed to edit the document for technical reasons, the community would have lost faith in me. They needed to believe that I would work hard to make sure that the only changes made to the standard were those consistent with technical votes taken in the committee.
Editing, I found, is really mostly about trust.