The other night I arrived early at the San Francisco Brewing Company in anticipation of the San Francisco Linuxstammtisch. Passing the time by drinking IPAs and chatting with the bartender, I ran across two greybeards from Oregon who were in the city for MacWorld and had just wandered into the pub for a brew.
They asked me what I did for a living, and I told them that I work for a Linux company. This piqued their interest, and they asked me what Linux was and where it came from. I pondered the question for a moment, wondering how far I should go back. Should I tell them about the Finnish graduate student named Linus, the MIT hacker named Richard, or should I tell them about Marshall Kirk McKusick and his contemporaries?
Realizing that I was probably about an hour early for the event and that I could probably swing a few free ales out of the story, I decided to go all the way back--to 1969.
What follows is a rather embellished version of Unix history, and the reader might do well to read the following documents:
1 New Jersey
The year 1969 has often been heralded as the peak of good times in the 20th Century (at least for America). The main significance that it has for hackers, however, is that 1969 was the year that Unix was born.
AT&T Bell Labs in New Jersey had just pulled the plug on their contribution to the Multics system, leaving a number of competent OS hackers floating in a void. Three of them (Ken Thompson, Dennis Ritchie, and J.F. Ossanna) set about trying to revive the community and group-hack spirit that had been created by the project. Ken Thompson had come up with an idea for a really cool filesystem, and the three of them carried on the momentum, hacking an OS at least on paper.
Bell Labs wasn't quick to oblige them, however. Multics was one of the biggest hogs for system resources and had grown out of control during its development there. The administration didn't want to see another Multics happen so long as they were alive. All requests for a computer to hack an OS on were denied.
Gradually, the ex-Multics hackers got used to the idea that they weren't going to be allowed to keep the OS development ball rolling, and went on with their assigned projects. Quite by accident, however, they were granted use of a PDP-7 (a tiny machine, by their standards) to work with.
In his spare time, Ken Thompson worked on this machine to allow him to play his favorite Multics space travel game. Of course, a game needs to be able to manipulate files for scores and data and such, so he and Dennis Ritchie wrote a tiny kernel to allow use of his filesystem.
After a while, the system running on the PDP-7 (which was considered little more than a glorified graphics terminal for the larger machines) developed into the very project that Thompson and Ritchie had been hoping to start up.
Although it was not until well into 1970 that Brian Kernighan suggested the name Unix, in a somewhat treacherous pun on Multics, the operating system we know today was born. --Dennis Ritchie, 1979
I should note that Digital Equipment Corporation named their first series of minicomputers Programmable Data Processors in order to sneak into IBM's market. DEC made quite sure that they were never referred to as computers as the prevailing opinion at the time was that IBM was the only reputable company that made computers!
Soon enough, the Bell Labs hackers were given a PDP-11 in order to work on an office automation system. The machine was different enough from the PDP-7 that programs written on the 7 could not be run on the 11. It was a vastly different architecture, and required completely different programming techniques.
They had, however, come up with a high-level programming language for the machines. This allowed them to write programs algorithmically, and let the details of a particular machine's architecture be handled by the compiler. So long as a compiler existed for the machine in question, it was trivial to recompile a program to run on it.
The language Thompson and his colleagues created, C, was advanced enough to be easy to program in, but powerful enough to do most of what they would have done using assembly. They rewrote most of Unix in C, leaving only very few parts in assembly language. This was revolutionary at the time, as most people would have either created a PDP-7 emulator for the PDP-11, or they would have written a similar but completely new OS for the PDP-11. The idea that an operating system could be portable was almost unheard of.
And so, Ken Thompson started mailing magnetic tapes with the Unix source code and utilities to his friends, labeling them simply "Love, ken". And so, in the early 1970's, a culture of Unix hackers sprang up, working with the Bell Labs source code.
Around the mid 1970s, a professor by the name of John Lions at the University of New South Wales in Australia decided to use Unix to teach operating system architecture. Since Unix was written in C, it was much easier for a student to trace the flow of the code algorithmically. The university was, of course, fortunate enough to have a source license to Unix.
AT&T had actually been rather forthcoming with licenses to Unix for academic use. According to The Cranberry Book, Unix version 6 (circa 1976) was free for Universities and version 7 cost $100 (government labs and commercial entities had to pay $21,000). This was partly due to a consent decree that forbade AT&T from selling software commercially.
If you had taken Lions' class at the time, you would have bought two books (one red and one orange), they were the Source Code and Commentary on UNIX Level 6. The class became quite popular, and soon Bell Labs took notice.
The curtain fell. AT&T halted publication of the books, as they gave away and discussed proprietary code from Bell Labs. As a consequence, those two books are now quite possibly the most photocopied works in computing history. They were passed around samizdat for a decade, and educated a whole generation of OS hackers. They're considered ridiculously obsolete nowadays, but they still hold amazing historical value. I myself have a copy in my desk at work, to remind me of the rebel spirit that founded Unix.
Because we couldn't legally discuss the book in the University's operating systems class, several of us would meet at night in an empty classroom to discuss the book. It was the only time in my life that I was an active member of an underground.
--Peter B. Reintjes, on the back of the 1996 reprinting of Lions' Commentary
The whole Unix culture was based upon sending C source code from person to person, adding features and fixing bugs as time went on. In many cases, the only documentation or distribution medium for a program or OS tool was the source code itself. It was assumed that everybody had a C compiler and the ability to compile the code properly. Portability was one of Unix's major strengths, and it would have been foolish to interfere with this by distributing binary-only versions of programs and utilities.
2 Berkeley
Enter UC Berkeley in the late 1970's. UCB was a licensee of the Unix source code, and they used it extensively for research projects and development. Pretty much the whole of the Cal CS department was hacking Unix at one time or another during the late 1970's and early 1980's. The Computer Systems Research Group had hacked virtual memory into it in order to make the Franz LISP system more useful. This attracted the attention of the Defense Department, and they were soon working on DARPA projects for things like networking.
In fact, TCP/IP and the socket model for network programming were invented on BSD (Berkeley Systems Distribution, the Computer Systems Research Group's name for Berkeley Unix). By the mid-1980's, the hacker's system of choice was a VAX (DEC's 32-bit version of the PDP-11) running BSD 4.x. BSD networking provided newer and faster ways to send source code over the Internet from one person to another. Large collaborative projects started, thanks to the ability for programmers from around the world to work together on one machine.
Berkeley had improved Unix beyond anything Bell Labs had invented, and an East coast-West coast feud erupted versus proponents of ATT and BSD Unix. Many feel to this day that BSD is far more elegant and advanced than even SysVR4, the current version of ATT Unix that reports to have incorporated all the advances of BSD. Of course, AT&T no longer develops Unix. And as of this writing the rights to ATT Unix have been bounced around rather a lot, and have finally landed in the hands of Novell.
Of course, Bell Labs noticed yet again that their source code was practically being given away, even though BSD was so wildly different from the Unix that had been given to Cal. A hideous tangle of lawsuits erupted, including anyone who had marketed a product based on the Net/1 or Net/2 release of 4.3BSD (which were available off the Internet for free in the early 1990's). AT&T sued the CSRG for BSD, and UC Berkeley sued various companies for not giving credit to UCB.
The BSD wars, as they were later called, were the backdrop for the development of 4.4BSD, the last version to be distributed from UC Berkeley. The idea was to release two versions: an encumbered version, which required an AT&T license to run, and an unencumbered version that contained no proprietary code from Bell Labs Unix. In the end, the warring parties settled out of court, and 4.4BSD-Lite was released as the unencumbered and only legal release of BSD.
Many modern operating systems are based on 4.4BSD-Lite, including FreeBSD, NetBSD, OpenBSD and BSDI. These remain (with the exception of BSDI) almost entirely under the BSD license, which permits redistribution in binary or source code form, so long as adequate credit is given to the University of California at Berkeley for code contributed by the CSRG.
3 Cambridge
Parallel to the BSD/ATT feuds, there was a strong and dedicated culture of hackers at MIT. MIT's AI lab had long been a testbed for DEC's PDP series of computers, and they had written their own operating system for the PDP-10, called ITS. ITS stood for Incompatible Timesharing System, and it was optimized to within an inch of its life. The PDP-10 and ITS were inseparable. The MIT hackers had made use of every timing feature and minor flaw in the system, and their OS was arguably the best system ever written for a PDP-10.
Of course, ITS wasn't portable by any stretch of the imagination. It was an operating system in the old school of development, and it died rather suddenly when DEC announced that they were discontinuing the PDP-10 and its descendants in favor of the PDP-11 and VAX systems.
This was devastating to ITS culture, but the MIT hackers regrouped under the notion of building a machine that used the LISP language (popular mostly for Artificial Intelligence work) as its low-level assembly code. Unfortunately for the MIT AI lab, corporate interest in AI work increased in the late 1970's (spurred mostly by ridiculous claims from the Japanese electronics industry about its AI capabilities). Before too long, many of the public-domain projects worked on by the MIT hackers were the proprietary products of various corporations.
One particularly loud holdout named Richard Stallman (often known by his initials and login name, RMS) refused to give in to the massive commercialization of his home community of hackers, and set about finding a way to preserve the freedom he'd lost through the buyout of the AI lab's talent. Having witnessed both the death of ITS due to its lack of portability and the death of the LISP machine due to corporate greed, he decided to give in to the Unix way and write a new operating system with two important features: it would be portable, and it would be licensed in such a way that it would always be the property of the free development community.
It was a tradition at MIT to name projects using recursive acronyms (acronyms that can never be fully expanded). Stallman was notorious for writing an editor called EMACS, for example, which stood simply for "Editing MACroS". Another hacker wrote a similar program and called it EINE, for "EINE Is Not EMACS". EINE was then almost completely rewritten and then renamed ZWEI, for "ZWEI Was EINE Initially". Wordplay has always been at the heart of hacker culture, and RMS wanted to name his new OS in this spirit. He finally chose GNU, which stands for GNU's Not Unix (one of the first three-letter recursive acronyms, according to him).
While almost everyone is grateful to RMS for starting the GNU project, and while GCC, GDB, and EMACS are widely regarded as the three most important development tools ever written (and yes, I am a vi user), RMS's most brilliant hack was the license he put these programs under. The GPL (GNU General Public License) has a syntax that uses the Copyright system to preserve the public availability and modification rights to program source code. The basic gyst of the GPL is that you must provide source code for any executable or binary release of a program under the GPL, and that you must put all derived works under the GPL as well.
The project began in 1983 or so, and it really helped sustain the Unix hacker community during the "dark ages" that the 1980s proved to be for OS hacking. Proprietary versions of Unix were becoming popular in the corporate arena, and quite often these versions lacked source code (making them nearly unusable). Sun even distributed versions without C compilers! The personal computer was taking off, championed by proprietary software vendors Microsoft and IBM. If not for GNU, some argue, this disturbing trend of proprietary operating systems might have become the standard.
The GNU project worked from the outside in. That is to say, they wrote utilities and programs to replace those run on other Unix systems, trusting that they'd be able to replace the low-level parts later. The project worked from the assumption that portable utilities that worked soon after release were far more useful than an operating system kernel that would soon have a suite of utilities to follow it. Most Unix systems today include many tools from the GNU project simply by default, even though they are not necessarily GNU systems!
When it came time to write the kernel, however, the GNU project decided that it would be easier to split that up into two parts, and begin by writing the GNU kernel as a personality on the Mach microkernel. This proved to be more difficult than was anticipated, as development on Mach waned, and the technical details in writing a Mach personality were far more complex than those in writing a "monolithic" kernel.
4 Helsinki
By 1991, GNU was a complete OS with everything but a kernel, BSD was locked in lawsuits (Though there was a project by the Jolitzes to write a kernel called 386BSD based on the Net/2 release, it eventually stalled during the settlement.), and Unix was widely regarded as a big proprietary system for diskless workstations and large-scale installations. At about this time, Linus Torvalds, a student in Helsinki, Finland, started a kernel inspired by a system known as Minix.
Minix was written by Andrew Tannenbaum as a teaching tool for learning operating systems. His book centered on operating system design, and Minix was the focus of the book's planned curriculum. It wasn't very useful as a proper Unix system, but it provided insights into how to design a Unix-like kernel. Linus Torvalds was sick of the insufficiencies of Minix and the inability to get a free modem line into the University's Unix machines, so he set about writing his own kernel to fit under the GNU and BSD utilities.
Linus and his colleague Lars wrote the kernel partly so that they could learn C. They posted the code up on the net, and instantly started receiving patches and fixes and enhancements to the code. It was all licensed under the GPL, so the developers were assured that their work would remain useful to anyone who wanted to use it. In many ways, the success of the kernel owes to Linus' humility and ability to ask for help as necessary.
Eight years later, as I write this, Linus Torvalds is revered as one of the sharpest minds in OS design. His kernel has won him worldwide acclaim, and Linus even appeared on the cover of Forbes.
5 Back to San Francisco Bay
Of course, the GNU project was a set of OS utilities without a kernel, and Linux was an OS kernel without a set of utilities. It was up to third parties to take the Linux kernel and bundle it with the requisite GNU and BSD utilities.
One of the first widely popular distributions of the GNU and Linux combination was called SLS. SLS initially required that the user run Minix, and install GNU and Linux from there. SLS was quite popular, although it required prior experience with Unix in order to successfully install it.
Soon after SLS came Slackware, compiled by Patrick Volkerding. Slackware made it incredibly easy to install a Linux-based GNU OS, and soon there were tons of people installing it on home machines. Slackware allowed the expert Unix hacks to install and tune their systems manually, while providing a nice script called "setup" that walked newer users through the install process.