The classical view of an operating system is that is consists of a kernel that provides a set of kernel functions, and these functions provide all of the kernel services. Such a kernel is large, and from the outisde, it looks monolithic, although internally, it may have a well engineered structure with clear modular boundaries.
The border between kernel and users has shifted over the years. In the late 1960's, it was common for advertisers to claim that the FORTRAN and COBOL compilers were tightly integrated into the system, and even when this claim was empty, the command language interpreter was almost always part of the kernel. The term kernel, however, was not used in those days.
UNIX represented something of a revolution. In UNIX, all of the compilers are simple user programs, and the command language interpreter requires no special privileges so it too is a user program. If you don't like the command language interpreter, write your own? (This is why UNIX users have the choice between so many shells: bash, csh, tcsh, etc.) Furthermore, even most of the window manager, a relatively new add-on and certainly an afterthought, is largely non-privileged, so UNIX users have a choice between many different look-and-feel models for their windows.
Nonetheless, UNIX still has a large number of subsystems within the UNIX kernel, and there are more extreme approaches to kernel design.
The word kernel originally emerged in the discussion of the design of Multics. This system was conceived of as being constructed as a set of nested rings, sometimes described (long before Shrek) as the onion model (not because it smelled bad, but because it had layers). In this context, the innermost layer was described as the kernel of the system.
A question that arose, first as an academic question, and then as a very serious question, was how small could the kernel be? It was in the context of this discussion that the designers of UNIX (K. Thompson and D. Ritchie) adopted the term kernel to refer to what we now call the UNIX kernel. This is the innermost clearly identifiable layer of the UNIX system, with the shell, compilers and many utility programs "wrapped around" this kernel. Unlike Multics, the parts of UNIX outside the kernel do not form complete layers around the kernel, but rather, they are separate components. If you write only shell-scripts, indeed, the shell is wrapped around the kernel, but you are not obligated to use the shell.
The system designers who took the challenge of a minimal kernel seriously quickly adopted a new term, microkernel, to refer to their minimal designs. These designs were, at first, an academic curiosity, but in the context of distributed systems and multicomputers, they have emerged as a major design alternative. Demos, Mach, Amoeba are example microkernels.
The key observation that drove the development of microkernels was that huge parts of the traditonal oprating system could be modeled as servers. The first example where this was obvious was the file server.
If we can afford to map all file operations to network requests that are sent to a remote file server, why design the system with a file-system interface that hides this? Why not design the file system with a user interface that is explicitly based on message passing between client and server. Thus, for example, the user's view of the file system could be:
This model still forces the file server to manage directories as well as files, and there are reasons to ask for these two jobs to be separated. This leads to the following multi-server model.
The file server only does one thing, it creates "virtual disk devices" out of collections of sectors on a physical disk. Each file, a virtual disk, has a handle that can be stored in a directory, but that is outside of the control of the file server.
The directory server associates textual file names with file handles. It probably uses a file server to store directories, but the entries in these directories are not necessarily for files on the same file server as the directory.
Thus, we modify the above list to:
What other items found in a classic monolithic operating system can be broken out and stored in servers? In the end, almost everything! We can have a timer server for real-time services, a window server for maintaining windows on the screen (this is how the X-window system operates), and even a synchronization server for providing mutual exclusion services. At the extreme, we can even put process and memory management in a server, if the kernel cooperates with this server in a special way. What is left? Interprocess communication -- the tools clients use to communicate with servers.
There are many interprocess communication models, but if the system is intended to run in a distributed environment, message passing models are particularly appropriate. The DEMOS kernel was one of the very first to fully exploit this possiblity, and its simplicity makes it a good example.
If the only tools in the kernel are interprocess communication tools, how does the process do anything. The answer adopted on DEMOS is quite similar to the answer to some simpler questions adopted on UNIX. Each process has a standard environment that holds information that process needs to do its job. In UNIX, this is just a set of strings, many of which are file names.
In DEMOS, the standard environment is a set of communication links to servers. Thus, when a process is created, it must be given, by its creator, an environment, a set of links. It needs a link to the file server, a link to the open files used for standard input and output, a link to the process manager (so it can create processes), and so on. The very first process (created at startup time) has links for the default servers, but any process may, for example, launch a new server and begin using that server instead of the default server.