Plan 9 is a fascinating time capsule of a period where text terminals were just being replaced with graphics ones. Which of course Plan 9 had a cleverly designed one for.
However if it had become mainstream, it would be just as cluttered up with inelegant stuff as Linux is now, in the endless pursuit of, say, graphics bandwidth performance, first for videos, then for 3D immersive games and now to merely draw your desktop. Mount an audio device remotely. Lovely. But try to get that working with Bluetooth, something mainstream desktop Linux has only just recently managed (i.e. use bluetooth headsets reliably and without fuss).
Ditto for 10GB ethernet or what have you. Elegance is quickly sacrificed at the altar of efficiency and expediency. At the risk of inciting disagreement, look at what happened to the originally relatively elegant and simple X protocol.
The fact that the lowest level of interaction you can have with a computer is writing memory. When writing device drivers, or interfacing with microcontroller hardware, the way you give commands and transfer data is by writing your command to a memory-mapped device register, or an area of memory that will be copied to the target device. This, coupled with the ability to map a file to memory, allows this paradigm to compete with the most efficient bare-metal implementations. One particular example is the /dev/draw interface mentioned in the article, where anyone could literally write graphics program just by fiddling with bits in memory, just like in DOS or the C64.
The other thing is, modern computers are networks in a box. For example, what if your GPU is a separate computer networked over a high-speed link. What if I could upload a texture just by opening a 'file' on the GPU, mmaping -it and memcpy-ing the data into it? I don't see anything particularly inefficient here, particularly if the data transfer can be handle by a special network protocol that takes advantage of PCIExpress.
> The fact that the lowest level of interaction you can have with a computer is writing memory.
While this is true as far as it goes, it's worth keeping in mind that not all memory controllers are equally low-level. Some of them expose cache architecture or the rectangular organization of DRAM in interesting ways.
> The fact that the lowest level of interaction you can have with a computer is writing memory. When writing device drivers, or interfacing with microcontroller hardware, the way you give commands and transfer data is by writing your command to a memory-mapped device register, or an area of memory that will be copied to the target device.
This is somewhat true, but usually these devices are not designed for multiple processes to come in and start writing memory to them. On Linux, the job of the DRM kernel driver for graphics is to mostly facilitate command buffer submission, scheduling, and results gathering. We'd really just see the same exact situation on Plan 9: user-space GPU drivers which do the work of translating graphics APIs into command buffers, and talking to a kernel-side component by submitting dedicated command buffers.
That they talk over a file interface isn't really relevant here, since it's just used as userspace -> kernel IPC.
> The other thing is, modern computers are networks in a box. For example, what if your GPU is a separate computer networked over a high-speed link. What if I could upload a texture just by opening a 'file' on the GPU, mmaping -it and memcpy-ing the data into it?
Well, first, you'd need to figure out the layout of the texture you want (e.g. AMD has these layouts https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd... ), allocate the full mip chain size, allocate your texture descriptors, then swizzle the texture data into a form that the GPU can understand.
Oh, and if you wanted a fast texture on desktop GPUs, you'd have to store it in host-inaccessible memory (host-accessible memory is often slower). So first you'd need to allocate a scratch buffer in slower, host-accessible memory, copy your texture data there, and then allocate the real texture storage in the faster host-inaccessible memory, and submit a copy command. That's a lot more work than a memmap and a memcpy.
Maybe we put all those smarts in the driver, but the driver is going to need a lot more info about the intended usage of the texture, and by that point we're basically creating /dev/vulkan rather than a low-level GPU API.
AIUI, a mmap file could in fact be "backed" by host-inaccessible memory. Then copying data to and from that memory would be a job for the driver, as for any block device. Other things can almost certainly be done in userspace, such as figuring out texture formats and converting if necessary.
> usually these devices are not designed for multiple processes to come in and start writing memory to them.
Multiplexing devices as needed can also be done in userspace. That's what, e.g. pulseaudio and pipewire do for audio devices, even those that don't support outputing audio for multiple processes at the same time.
> For example, what if your GPU is a separate computer networked over a high-speed link. What if I could upload a texture just by opening a 'file' on the GPU, mmaping -it and memcpy-ing the data into it?
AIUI, you need something like CXL to implement a proper concurrency model that works seamlessly for both local and remote memory. So we're kinda close to what you're describing but not quite there yet.
HMM is on the way for a page migration-based impl that works on most hardware and allows seamlessly to use all CPU memory from GPU programs, including mmap’d files. CUDA 12 is going to be fun.
CXL 2.0 and earlier just aren’t it because of a coherency strategy that is absolutely dreadful. CXL 3.0 is better though, but that’s a long time away from hardware still.
OpenBSD is a good example of an OS that tries to adhere to old school design principals while offering modern features. It has some pretty bad rough edges but is impressive nonetheless.
I use linux on my desktop but have an ancient Thinkpad that runs OpenBSD as well. I love following the changelog for each new release.
I'm in a similar space. I love OpenBSD for it's simplicity and consistency, but, to my way of thinking, it's missing to much to act at a modern desktop replacement. To that end, I've started running ChromeOS Flex on the desktop, taking advantage of it's Debian VM where ChromeOS falls short, while still using OpenBSD everywhere else that I can and makes sense.
I pretty much love all of the free-unix but dislike them for graphical use. I have generally stuck with Windows or MacOS for general desktop usage.
I've considered a combination setup of a terminal system for my unix needs and a tablet for web and video. ChromeOS would work well for this too, I think.
> However if it had become mainstream, it would be just as cluttered up with inelegant stuff as Linux is now
This is a community and distro problem. If you accept patches for the sake of adding stuff with no good reason then you're going to get linux. For example just the other day on the 9front mailing list someone submitted a patch for an ethernet driver they were working on to enable tcp/udp checksum offloading. Another user replied along the lines of "do we really need this? first thing I do as an admin is turn off weird/buggy
offloading" which the submitter agreed with and that was that. No one really needs that patch; its code for ONE specific ethernet device few if any other users own.
The community strives to keep the OS small and practical as it was intended. That's a big part of the culture behind plan 9: Keep it simple, stupid!
Another thing is porting software for the sake of porting software is also discouraged. If you need it, then you port it and maintain it. Dont port things you don't need.
> But try to get that working with Bluetooth, something mainstream desktop Linux has only just recently managed (i.e. use bluetooth headsets reliably and without fuss).
Not really a fair statement. BT is a horror show with its batshit layercake of drivers, protocols and profiles. Plus the Linux audio situation has never stabilized and invents a new audio subsystem every decade: OSS, ALSA, Jack, pulse, pipewire, etc. The BT situation on Linux is the fault of the constantly moving linux dev community who's focus seems to favor server side stuff and DE colors.
True no doubt, but as of one or at most two Fedora releases or so, you can plug in a bluetooth dongle (if you don't already have Bluetooth), pair a headset to it and use it! With selectable audio profiles even (the phone quality bidirectional one or the hifi stereo ones). And except for a rare glitch, it works. As well as Bluetooth ever does, judging from the experience in the car.
This was not so previously. I tried every release or two, and always ended up with a locked-up driver within 10 minutes of playing with it. This is totally Bluetooth's complexity's fault of course.
> If you accept patches for the sake of adding stuff with no good reason...
Linux kernel is used on billions of devices by millions of people. If you don't see a good reason for some feature in Linux, it doesn't mean there is no such reason.
I'd even argue it is impossible to have something small, simple and practical, but simultaneously supporting such a vast sea of hardware and software combinations.
That’s... true, but what “file” means in that sentence is a bit tricky. The graphics protocol (or at least I think it was the graphics protocol) requires each command to be written in a single write() call, so a Plan 9 file is neither an array nor a stream of bytes, it includes those implicit boundaries as well. The interface used for impersonating users, IIRC, looks like a kernel-implemented file server but that file server essentially uses its kernel nature by referencing the process that opened the file, so this interface only deserves being called a file if /dev/stdin and /proc/self in classic Unix do as well.
I like Plan 9, mind you, but I also think we ought to be careful in treating it as an existence proof for what the pure everything-as-a-file model can do. Even outside of things Plan 9 doesn’t and can’t implement (e.g. modern bandwidth-limited 3D graphics), it also has some hacks in parts it does.
My naive assumption when looking at the design of Plan 9 from a distance, is that the filesystem could easily become leaky abstraction.
Sort of like how we now have USB-C everywhere, but you have to know what kind of cable is actually in the middle in many cases. Maybe it would better to split major feature sets across a few different physical plugs, so you know exactly what you have just by looking at it.
That sounds strange - mind you I'm not familiar with Plan 9, but was the graphics wire protocol text-based? If yes, why wasn't the separation handled by newlines?
I'm sure you can encode command separators in binary without the need to rely on write flushes.
Nope, much like 9p itself[1] the /dev/draw protocol[2] is a stream of (simple) binary records. And it’s not that the implicit separation is essential for parsing it, it’s that the servers don’t care to implement the additional buffering necessary in case a single message spans several read()s.
(I still don’t remember whether I looked at one of the /dev/draw implementations or at one of the 9p implementations when I was curious about the fiddly buffering question, sorry.)
> "Computers should feel instant whenever possible. This involves the event path, whatever processing is done, the speed of drawing, and the way the drawing is displayed."
This was 27 years ago and still no progress on this field.
Wayland and X compositors provide for tear-free video updates, which is what Carmack is talking about in those posts. Further progress will come from lowering the added latency, but that's a very different problem.
He goes on to complain about precursors to compositing.
By the late 1990's Linux supported tear free with lower latency than compositors currently support.
Anyway, computers have been getting worse at output latency since the mid 1980's. Compositing is just yet another ratcheted performance and usability regression.
Late-1990s systems supported practically tear-free updates for most practical uses, but AIUI there was not enough memory bandwidth for whole-screen updates to be tear-free, especially at higher resolutions and color depths. This is what Carmack is complaining about here.
Even then, there are better and worse ways of coping with limited video memory bandwidth, and what's "best" depends on the given application.
FWIW, the Plan 9 window system is in fact a (very barebones) compositor in modern terms, in that it (unlike classic X and like Wayland) maintains a backbuffer for every (top-level) window instead of communicating “damage” back to applications. That seems to be its very thing—there’s a distinct feeling of “look what computers let us do now” in the papers.
I don’t think it knows anything about the widgets inside those windows, either (again unlike classic X and like Wayland). (On the other hand, like classic X and unlike Wayland, it makes the client funnel every drawing operation through the window system.)
I don’t understand this comment. Wayland has already been mentioned. macOS and Windows have both also made huge progress in this area. And iOS is the gold standard for modern UI responsiveness.
All of those have worse latency than DOS or X11 (even when the latter are configured to prevent tearing by adding latency)
Memory, power and compute efficiency have also regressed badly vs. those systems. Carmack explains how if would be possible to have tear free, framerate level updates "if only" people would upgrade to 8MB of video ram and 100MB/sec busses.
I'm old enough to remember the DOS days. On average, iOS apps are significantly less responsive than old 386 DOS, SDL or X11 programs.
> Carmack explains how if would be possible to have tear free, framerate level updates "if only" people would upgrade to 8MB of video ram and 100MB/sec busses.
With 1280x1024x8@75Hz and a single blit per frame. For 32-bit colour like you have virtually everywhere these days, multiply that bandwidth by four; for a 2560x1600 screen like the one I’m writing this on, multiply further by three and a bit; (admittedly,) for 60Hz instead of 75Hz, multiply by 4/5.
All in all, this laptop I’m sitting before needs a gigabyte per second to just barely sustain one blit per frame. And it needs a full framebuffer of video RAM (15M bytes) for the pretty security-message crossfade, probably at least a couple framebuffers per tab for the tiled renderer in the browser, something in that ballpark so that I can smoothly scroll through a PDF I’m reading and not wait for it to rasterize... If I open a Chinese webpage, how much video RAM for the subpixel-positioned glyph cache, I wonder? And remember, none of that OpenGL floating-point-colour nonsense, that would multiply both memory and bandwidth by three again.
Carmack could have probably squeezed a smooth UI into a 8M main + 8M (16M?) video RAM, 100 MB/s machine at 800x600, if the UI you wanted was the one from 1995. And that would honestly be grand, I’d be happy to see that on a 2005-specced machine, even. But let’s not kid ourselves—we do in fact demand much more from our computers now than we did then.
DOS ran on VGA, with a resolution of 640x480, with 256 colors per pixel (307,200 bytes per frame). The iPhone 14 has a resolution of 2532x1170 [0], at 32 bits per pixel (11,849,760 bytes per frame) -- about two orders of magnitude higher. Not to mention the lack of compositing and multitasking on DOS. Compositing was a big feature. We started trading off throughput for latency because our old approaches just wouldn't scale.
There are lots of other factors, for sure, but that's a pretty big one!
While Plan 9 is cool and such, I prefer what they built afterwards with the lessons of Plan 9, while trying to compete against Java and Sun.
Inferno and Limbo, which tend to be ignored with too much focus on Plan 9, a middle stop in their whole experience designing OSes and programming languages after being done with UNIX and C.
I've always wondered. Bell Labs' Inferno, Sun's JavaOS, and Microsoft's Singularity were built on the same principles of OS being almost entirely built on top of a virtual machine, and it's a very interesting idea from many points of view. But all of them seem to have failed to gain enough traction to enter the mainstream.
Was there any reason for that? Were the CPUs not powerful enough, or were the compilers not quite there yet? Or did it have more to do with the business side of things?
Yep. *n*x is a victim of its own success. Thompson, Ritchie, Kernighan, et al built something so much better than most of the competition that it ate the world.
There have been better ideas since then (including Plan 9) but nothing able to knock *n*x off its perch.
IMO, LMI and Symbolics might have done it, if their offering hadn't required enormously expensive machines (by the standards of the time), while *n*x would run on cheap hardware (again, by the standards of the time).
Perhaps we'll eventually see an innovative OS written in WASM. That's about the only way I see to get around the vendor lock-in (I mean, even Microsoft appears to be converging on a "Windows UI wrapped around a *n*x kernel" model... Apple, of course, has been using a "Mac UI wrapped around a *n*x kernel" for a couple of decades now).
WSL is running on its own VM. Windows kernel still owns the game.
On "Cloud OS" the underlying kernel only matters for classical workloads being pushed onto the cloud.
Any language that doesn't depend on POSIX and has a rich library ecosystem, can happily run on top of type 1 hypervisors, which are in a way, the revenge of microkernels.
Awesome, thanks. Do you have a take on unikernels in general?
My main side project at the moment involves trying to use QEMU+WHPX on Windows to sandbox selfhosted services. Long term I think it would be awesome to support unikernels, but I get the feeling a lot of the momentum for them has died down the last couple years.
I work with NanoVMs and nanos.org can work under both qemu (by default) and deploy to hyper-v under WSL2. Also, nanos.org itself is a go unikernel so we have excellent go support.
Also, from parent comment, ops, can deploy to esxi/vsphere perfectly fine.
I wouldn't say that Plan 9 has better (basic) ideas than Unix, it's more like a Unix that follows the old ideas even for new things. AFAIU many of the ugly aspects of "contemporary Unix" stem from the Unix wars and later additions by people with less design sense than the Bell Labs group.
Well, most of the world uses Windows (not a UNIX), macOS/iOS (owes more to NextSTEP than UNIX) and Android (owes more to BeOS than UNIX). The latter two use a UNIX kernel to get basics like scheduling and a filesystem but the userland APIs look nothing like UNIX or Linux, and both have a strong micro-kernelish vibe to them with Mach and the Binder respectively.
The mainstream version of those ideas are Android, ChromeOS and servless computing on cloud environments.
Also note that these ideas are quite old, IBM and Unisys mainframes and micro-computers are quite different from their original versions, yet most applications keep running thanks to their language environments instead of shipping pure native code.
At least for JavaOS and Singularity there were a bunch of problems:
1. They were primarily research OS', Singularity was never even meant to be used at all. Its successor Midori was, but again it never made it outside of MS. JavaOS was sort of targeted at embedded devices but again never had any real usage.
2. Operating systems are ideally very tightly programmed with minimal overhead. Given a choice between elegance and performance, people pick the latter. A slow app can be optimized but if your kernel/drivers are slow then you're often stuck. Whether it's microkernels or operating systems written in managed languages, these better designs sell performance to buy convenience for the implementors, but convenience for users is more important.
3. Midori tried to fix this but ended up spending all its design budget on trying to turn C# into C++ and do fashionable things with concurrency/thread safety, which isn't a compelling basis for an OS. Plan9 at least had features that were interesting for the actual users of the computer, but the Singularity/Midori approach was mostly just a PL nerdout.
Single address space operating systems have some design issues that have never been well addressed, or at least the solutions didn't seem convincing. Unfortunately OS research is stagnant so very little has changed over time.
For example, they need everything to be written in a single (GCd/managed) language, and so existing C/C++ libraries can't be used (without a win3.1 style stability model at least). But the VM itself is written in C/C++, so the first step is to either switch the entire model to AOT compilation (the MS approach) or to write the language VM in the language itself (the Sun approach, which eventually surfaced in real products in the form of GraalVM). The Graal approach seems to solve this problem by being able to run any kind of language on top fot the JVM, even C/C++ code, and it can run it safely/GCd.
Another problem is the question of what exactly replaces a process. Processes in classical operating systems do a lot of different things - sandboxing, data isolation, scheduling, but also things like defining a failure domain. The big advantage of a single address space OS is you can allocate objects and then just pass them around freely without needing all the IPC/serialization/socket bindings/etc stuff that classical kernels require. So the obvious approach is to just not have such a thing as a process - but if you do that, you hit the question of how to manage executing code. What's the equivalent of SIGKILL if you don't have a process? What does a CPU usage graph look like? What happens if a driver does a callback into a piece of code which then deadlocks? This problem combined with the difficulty of finding a one-size-fits-all garbage collection algorithm pushes SAS operating systems towards reintroducing a process-like concept, for instance the Singularity process equivalent was actually way more constraining than a UNIX process, and objects could no longer be passed around freely between subsystems but instead required a convoluted exchange heap + quasi-RPC approach.
I used to think inferno was a great idea until I realized plan 9 makes cross-platform easy. Why go through all that VM nonsense when you can build the OS on top of a cross platform tool chain and c library? The OS was designed to run on as many architectures as they could get their hands on: x86, arm, power, m68k, mips, etc. You just set the objtype env var to the target arch, e.g. objtype=amd64 and then run mk (plan 9's better make) and/or mk install.
Java and Inferno were designed to fix the abysmal portability problems other operating system vendors created.
"Alef appeared in the first and second editions of Plan 9, but was abandoned during development of the third edition.[1][2] Rob Pike later explained Alef's demise by pointing to its lack of automatic memory management, despite Pike's and other people's urging Winterbottom to add garbage collection to the language;[3] also, in a February 2000 slideshow, Pike noted: "…although Alef was a fruitful language, it proved too difficult to maintain a variant language across multiple architectures, so we took what we learned from it and built the thread library for C."[4]
Alef was superseded by two programming environments. The Limbo programming language can be considered a direct successor of Alef"
> The plumber is cool, it’s like “what if xdg-open was good actually”
Yeah, plumber is way better than xdg-open in that it is extensible and it can be invoked from a script more easily via 9P. Given how hard it to customize xdg-open, I use a xdg-open wrapper that gets the caller's name using procfs and sends it to the plumber service. That way I can open links on a different browser depending from where the link was clicked/opened from .
#!/usr/bin/env bash
# this is named xdg-open and placed in a directory that is
# before /usr/bin in the $PATH
PARENT_COMMAND=$(cat "/proc/$PPID/comm")
case "$PARENT_COMMAND" in
slack)
/home/puercopop/src/plan9/bin/9 plumb -s slack "$@"
;;
zoom)
/home/puercopop/src/plan9/bin/9 plumb -s zoom "$@"
;;
*)
/usr/bin/xdg-open "$@"
;;
esac
And then on my plumbling file I have
type is text
src is zoom
data matches 'https?://.*'
plumb to firefox-trunk $data
type is text
plumb to xdg-open $data
> you open /net/tcp/clone to reserve a connection, and read the connection ID from it. Then you open /net/tcp/n/ctl and write "connect 127.0.0.1!80"
This feels so ham fisted to me. Why not just open /net/tcp/127.0.0.1/80? I am sure there are reasons, but if the goal is to make everything a file, this feels like a more natural representation.
The ctl file also lets you control other connection settings, such as keep alive. It would also be unfortunate to run ls in /net/tcp and dump the entire IPv4/6 space into your terminal.
Not sure I buy that last sentence. Why would the system populate these directories with anything other than a) what’s configured to be assigned to the system, and b) what the caller requests to be created by simply opening a specific path?
Sure, you could script opening each address of the entire address space and have an unfortunate situation, but you don’t get that by default when you simply query what exists.
That said, it’s been a veeeery long time since I tried out Plan9 and Inferno.
> The ctl file also lets you control other connection settings, such as keep alive.
Yes, though this could have been achieved with a bunch of sysfs-like entries under /net/tcp/127.0.0.1/80/<port>/ with the added benefit of being easily discoverable.
> It would also be unfortunate to run ls in /net/tcp and dump the entire IPv4/6 space into your terminal.
A reasonable semantics here would be to only list hosts to which there are active connections.
Probably so that you could also write additional options for connections, the same way you would with setsockopt. Besides, using connection IDs, and thus having a more or less persistent filename for the current connection, makes it easier for several programs to use the same connection.
I never saw what the Bell Labs folks had against Berkeley sockets, aside from them being from Berkeley and not Bell Labs. It's a crufty interface either way, because you can't shove all the complexity under the rug.
They're optimizing for as few syscalls as possible, I suppose, but in Plan 9 you still create pipes with pipe(), not by writing to a special file. And if you already have pipe(), why not socket()?
The TLI/XTI t_open() call takes a path to a character device to identify the desired protocol, instead of a bunch of numbers like BSD socket() does. Isn’t the former approach both more Unix-like and more elegant-and also more extensible? Supporting new protocols, multiple TCP/IP stacks - all easily achievable just by creating a different device driver and telling the app to open a different path (or even using something like a mount namespace to make the hardcoded path go to a different device for each app.) With Berkeley sockets it isn’t clear how to achieve that.
Even if each connection has the same remote IP address + TCP port, it has a distinct local port. What you get back from opening /net/tcp/clone is essentially a directory name, with the local port as a file name in it, along with a few more files representing various attributes of interest.
Yeah, the filesystem has no "array append" operation, it's usually hamfisted into place by writing to a special key in the dictionary. Having a native fs op for "append to dir" would solve some small level of Plan 9 awkwardness...
But now there isn't a path to represent that specific connection we just opened, and there is no way to open it again, say from within another process. The only handle for the open connection you have is the FD, and you'd have to send that around which is harder (but also safer I admit).
The approach would be Linux-y but probbly not enough according to 9p standards.
The thing that I find hard to defend is that plan9 turns all internal service / api calls into text based / filed based protocols, with parsing involved. This feels so inefficient and adhoc, and requires more documentation
I'm not sure - first of all, turning everything to a text-based protocol might be something can be fixed - it's not impossible to amend the specification to allow for binary protocols, or - if the caller and callee are on the same machine - literal system calls.
But the important thing to not is this fixes the biggest issue of modern Linux - the lack of stable API/ABI - that requires everything to be compiled for every distro.
It also naturally documents each program's interface allowing them to be easily rewritten/mocked/logged/debugged etc.
The reason software has to be recompiled so often on Linux is not to do with ABIs vs text based protocols. It's because the act of compiling on Linux is used by the toolchain developers as a signal to automatically 'upgrade' the resultant binaries in various ways. In other words, the tooling and frameworks don't place any cultural value on making it easy to target older versions of the OS from newer versions.
There's no particular reason it must be so. The Java compiler has a --release flag that lets you say "I want to run the results on Java N" and then the compiler won't use any bytecode or library features that were added after that time, even if it otherwise would automatically and implicitly do so. The result is still binary and non-text based, but there's nothing magic about it. That sort of feature simply takes work that the Linux community is culturally opposed to doing thanks to the fully centralized model.
It does make one wonder what happens if everything is a type instead of a file. Do you get a command line that looks like Mathematica? I too think every program containing a parser is a waste.
What's the alternative? Binary protocols tend to be dependent on machine specifics such as endianness and alignment requirements, which would be a non-starter on a network-focused OS like Plan9 - as well as poorly extensible and not always properly documented. There are some well-known pitfalls wrt. text formats, such as parsing and emitting floating point numbers (which is why hexfloats are a thing) but for mostly everything else they're good enough.
There is no reason that binary protocols have to rely on endianess or alignment. Reversing or realigning a field is orders of magnitude faster than parsing text, so just pick one and stick with it.
> Plan 9 failed, in a sense, because Unix was simply too big and too entrenched by the time Plan 9 came around.
The Unix renaissance in the ‘90s had a lot to do with open source source OSes such as (primarily) Linux and the BSDs. Until then we had a bunch of corporate unixes, no two alike but expensive and kernel hacking was basically off limits. With the advent of i386 we had affordable & powerful computers that could run “real” OSes and us hackers were hungry for an open source OS. Computing history might’ve been different if plan9 was made open source before Linux became available. But that was not to be.
Note that Linux is basically getting the whole Plan9 feature set added to it, if in incremental and unplanned ways. AIUI, a kernel feature for implementing block devices in user space was added quite recently.
IMHO the kitchensink like approach of adding more features to Linux misses the point of the simplicity of plan9. Representing & accessing resources as file systems was the real insight. If the OS provides N different ways of doing something, it has to continue supporting that which has a real cost. I once compared compiling plan9 & Linux from scratch on the original RaspberryPi. 1 minute versus many hours. Granted that the Linux kernel does a lot more and supported a few more devices on 'pi and plan9 pushes more drivers in the user space but even compiling everything on it took about 4 minutes. That included Ghostscript and two editors and the windowing system and more.
Linux has to support its kernel syscall interface, so that Linux-native programs continue to run. Pretty much everything else is up for grabs - but if you want native Linux software to interoperate within a more Plan9-like system, adding features that allow for implementing/managing these kernel-provided interfaces in userspace is helpful.
Whenever I read a praise for the conceptual cleanliness and power of niche operating systems, I remind myself:
The current mainstream platforms are as complicated as they are because of performance.
Abstractions are fine until you realize you're doing thousands of network round-trips to draw a window, so what if you short-circuited that?
Yes the 9P protocol sounds great and cleaner than FUSE but what's the IOPS?
The reason only some of Plan 9s concepts made it elsewhere was because those were the ones that could be implemented without too steep a performance penalty (or where performance didn't matter).
I think a bigger problem is that a pseudo-file is not actually an especially good API for any use case. It's a sort of compromise solution that makes nobody happy. The sockets example is a good example. What programmers want is a function or OOP style API where they can pass strings, type safe structures and so on, but a file is just a stream of bytes so they invented some ad-hoc socket-opening-protocol thing, presumably so you can shell script it. But then shell scripts want to use higher level protocols so it's not useful for them, and programs would wrap that pseudo-file with a library API anyway, so it's just an implementation detail and not so great for that job either. Like, it can introduce a world of parsing/escaping/versioning bugs, race conditions and overheads.
In contrast a custom syscall that takes a structure is a more direct interface, more suited for writing actual programs.
It also means the API is much more tightly defined. Everything-is-a-file can create a lot of edge cases just like how HTTP creates a lot of edge cases in web programming by pretending everything is a document. What happens if you delete /dev/draw, what does that mean? You need to define the semantics of that. Does it mean closing the window? What about trying to move or copy it - does that make sense? Do you have time to think about all these operations and give them meaningful results?
Once you go down the road of saying that there should be a consistent set of operations you can perform on conceptual 'objects' using a generic set of tools and commands, you may start to wonder why you'd pick a file API for that. Why not just invest in objects as a core tech - why not allow binding directly to an object in a remote process or over the network and then have methods and functions actually work? Why pretend it's a file and force everyone to constantly invent mini-protocols and formats, when types and functions are what you need 90% of the time anyway? That's why Microsoft ended up going down the DCOM path, why Apple ended up with XPC/Mach, why Android/BeOS ended up with the Binder and so on. The Plan9 approach wasn't taken up in any big way because if you're going to make a big effort to unify everything it might as well be around objects, not files.
A pseudo file works just fine as a low-level API/ABI, that you could layer higher level interfaces on top of. What does it mean to delete/move/copy etc. a pseudo-file? That's going to vary. The operation might simply be disabled and return a generic error. AIUI, generalized OOP interfaces end up having the same issue to an even greater extent.
OOP interfaces as the core concept have several advantages over a pseudo-file based system, even if you assume a higher level RPC system that uses files under the hood:
1. OOP can at least in principle abstract over whether code is in-process or in a remote process, with efficient stack based calls for in-process calls and serialization/RPC for remote calls. Yes you can't make them be identical, but you can bring them very close together. But files are fundamentally a kernel controlled object. For a pseudo-file system to hold together at all, you end up having to talk to the kernel all the time, which is slow (ish).
2. OOP has the notion of a queryable set of interfaces. UNIX style file objects don't have any equivalent, which is why you suggest that operations that don't seem to make sense should just return a generic error code. But this is bad. If we saw a junior programmer both design and then implement an interface in which most methods returned error codes we would school them (it can of course be acceptable if you don't control that interface but even then, really not ideal). A much better approach is to define interfaces better so that your objects only implement operations that make sense, and a generic client can use type casting / IUnknown::QueryInterface style operations to figure out what an object can do.
3. Just as you can't opt-out of generic file operations, you also can't add more. That's why you end up having this split world in which some toy operations are just basic file IO and then the moment you need anything complex enough for the real world, your file becomes just a sockety thing speaking some ad-hoc protocol and you can't use the file directly anymore, e.g. you can't shell script it, you need some ad-hoc library that wraps it. There are tons of files in /dev but how often do you see code that directly uses them? Almost never - apps use libraries that wrap the file interface.
4. The lack of proper interfaces and type safety means that evolving stuff is way too hard. Doesn't matter for a research OS but matters in reality. Look at the docs for /dev/draw, it's all ad-hoc protocols with no proper versioning or evolvability at all, just stuff like "open the device and read 12 strings each 11 characters wide" (!).
I think that even though they suffer from execution issues, Microsoft was heading down the right path with COM and PowerShell. COM provides you with objects, which is way more commonly what you actually want. DCOM attempts to abstract over location, so the serialization and sockety/filey stuff only gets involved when necessary. And PowerShell attempts to give you a shell-like syntax for accessing them. For various reasons it doesn't quite work as well as it could, hence why concepts like Plan9 have enduring fascination, but the core ideas are more general and robust.
It definitely has lots of interesting ideas. Especially the filesystem mounting stuff (and the shunning of symlinks).
I'm still unconvinced by "everything is a file". Writing `connect 1.1.1.1!80` to a file is a particularly shitty completely untyped, unchecked, fragile and slow alternative to an actual ABI.
It's obviously easier to use from shell scripts but I don't think that's what you should optimise for.
IMO there should be a proper API with a typed IDL and then you can automatically make it easy to access from a shell without compromising other languages.
Byte streams are "untyped and unchecked" too, but that's because typing and checking can be deferred to a higher layer in the stack. I.e. one could easily define a typed IDL to provide a semantics over these bare text streams. No different from how most languages provide type checking over, e.g. ABI-standard subroutine calls.
Right, but if you're going to have a higher layer that presents a nicely typed interface why use a text-based interface in the first place? It's just an opportunity for inefficiency and bugs.
The fact that you can paper over a bad interface doesn't mean it isn't a bad interface.
Unless your OS is virtual machine that checks the structure of the user program, at some point, your typed data will have to be a set of numbers and/or a byte stream. Granted, the set of numbers required for utf-8 `connect` is a little less efficient than your system call no. but that's hardly a highly typed API. Anything with less mechanical sympathy that's designed for _any_ user program to access it will have to pay similar costs.
It's not highly typed no, but it does force the use of a highly typed API. How many languages provide a string-based interface to the connect() syscall?
Languages and kernels don't exactly have the same design considerations. Types and compilers are trying to help you write valid programs for your benefit. A kernel only has access to machine code representation of your possibly buggy or malicious program. Sure, your language or libc might provide a convenient and object based API to the kernel but these are just wrappers over the basic number and pointer to C struct soups based syscall APIs that machine code can interface with. I.e. there's nothing exactly preventing you from providing the wrong numbers and bad structs aside from the "higher layer in the stack". At least you're allowed to mispell `cnect` accessibly so from the shell.
There are ways around this ofc like the Android Runtime that requires any code that interfaces with it to be written in Java adjacent bytecode that gets compiled to machine code (and cached) by the runtime (after being checked I imagine).
> there's nothing exactly preventing you from providing the wrong numbers and bad structs aside from the "higher layer in the stack"
Right, mistakes are possible in both cases. But why pick the design that is more likely to lead to mistakes (on both sides of the interface) and is slower? There's just no good reason to do it like that.
I admire Drew Devault's work, but I personally have very different sensibilities on what I consider good software. So this seems to be a case of "Tell me who praises you, and I'll tell you what your mistake is" ;-)
"Everything is a file" is a nice idea, but we almost have that in Unix, and it is something that can be emulated perfectly in a programming language or a library. There is no need for the low level stuff to be neat. It has to be performant, secure, and support my hardware. Everything else can - and I think should - be built as abstractions on top. That I can basically run the same applications on macOS, Linux, and Windows shows that it is possible and works well.
One thing where Plan 9 is lacking IMHO is in the GUI department. ACME is novel but where are the really new radical (G)UI concepts? I wonder what we would have got if BeOS or Longhorn would have been successful. Both played with the idea that the filesystem is a database. Your file system browser morphs into a mail client, a MP3 player, a photo browser depending on the circumstances. You don't deal with "video files" anymore but "episodes", for example. And I think it is not really a technical but a UX challenge to make something like that work well. I hope that people start experimenting with that stuff again!
> Everything else can - and I think should - be built as abstractions on top.
The problem is that those abstractions cannot implement the interfaces that, e.g. native Linux programs will expect to use. Features like FUSE (filesystems implemented in userspace) are useful precisely as means of increasing the level of abstraction.
The article touches on how to implement the filesystem half of those ideas in plan 9 when it talks about implementing virtual hardware devices with shell scripts.
Go's io/fs[0] design is one of the more successful ideas inspired by Plan 9 imho. If we can't have a 9p-centric OS, the next best thing is a 9p-like interface in a language's standard library.
For example, I have been developing a static site generator where I implement the output folder as an fs.FS[1]. The output generating code is now a simple function that copies from folder A to folder B, without even knowing that A is a virtual filesystem. Now how do I implement a preview server? Simply pass said filesystem to the standard library's http.FileServer. Done. (okay you actually have to pass it through the http.FS() adapter, but that's only because http.FileServer predates io/fs)
Of course this kind of abstraction can be done in any language, but Go explicitly specifies this interface, which can already be used by multiple utilities in the standard library (e.g. http.FileServer, go:embed). This nudges people to the same interoperable interface, and I'm all for it.
Do you have a link to your code? I thought fs.FS is read-only, so I’m curious where/how you bring writing data into it (“copies from folder A to folder B”)? Or do you mean that your implementation of fs.FS generates the content on-demand, when net/http serves it?
> Or do you mean that your implementation of fs.FS generates the content on-demand, when net/http serves it?
It's closer to this. The actual content was already written to an sqlite db, and my read-only FS allows reading that data back out but as a filesystem.
The admittedly not that interesting (and probably bad - I'm new to Go) code is here[1], and this "BlogFS" is used in 2 places:
- During "export" where it generates the static site. "Folder A" is my FS and "Folder B" is the destination i.e. an actual folder on the real filesystem.
- In a preview server that just serves "Folder A" directly.
In contrast to what Drew says, Plan 9 probably failed because it was proprietary, not because of the Unix predecessor. If it had a free licence at the time, I doubt RMS would have felt the need for GNU, though I don't know what he'd make of some of the design.
Other than being different from what you might be used to, and not having a lot of software available, what other tradeoffs would you be making if you wanted to try and use Plan 9 in production? How's the performance?
I have fond memories of setting up a Plan9 installation in 2000-2003 and throughly messing around with it. Really opened my eyes on what “distributed computing” could mean at the architectural level. Their backup soliton (name escapes me) was also very interesting — too many things to mention in a single article. If I’m not mistaken when I ran it they had an older GUI called 9½.
Something not mentioned is Plan 9 security. Specifically Unix file handles are often quoted as like capabilities, but you do need namespaces to make that useful for isolation. I don't remember exactly what the paper on security in Plan 9 says, but it does talk about capabilities.
How interoperable with each other are the various Plan9 variants (including plan9port and hosted Inferno)? If all of the Plan9-like servers on a network have the exact same OS, does that enable possibilities that wouldn't otherwise be available, like migrating running processes between them?
So essentially, based on the above, the security of Plan 9 depends on the identification of a host? Meaning that if one trusted host is compromised in the network, then you can just mount anyone's devices or filesystems?
Compared to Linux or OpenBSD, is Plan9 as "secure", i.e. randomization of tcp seq numbers, cryptography, authentication, mitigation of spectre and hardware vulnerabilities etc. Or it needs some improvements? Meaning, if I had a simple C application that could easily be ported to plan9 but it has to be super secure, would 9front be advisable, or it's better to stick with Linux or OpenBSD for the time being?
> When everything is supposed to be a file on Unix, why is it that the networking API is entirely implemented with special-purpose syscalls and ioctls?
The internet existed for most of a decade before Berkeley Sockets. Surely there were other APIs out there. If none of them forced everything to go through a filesystem, maybe that means it's just a bad fit?
I'm sure Plan 9 is cool and all but y'all Plan 9 nerds are like dudes who keep pushing network TV for a Barney Miller revival even though like the entire cast has been dead for years :-D <3
Good ideas are worth preserving. And preserving these ideas means keeping the discussion alive. Drew mentions in the article that quite a lot of ideas have since ported in one way or another to other unices, but Plan 9 was more than just the sum of its parts. More of a product of its parts, if you will, heh.
However if it had become mainstream, it would be just as cluttered up with inelegant stuff as Linux is now, in the endless pursuit of, say, graphics bandwidth performance, first for videos, then for 3D immersive games and now to merely draw your desktop. Mount an audio device remotely. Lovely. But try to get that working with Bluetooth, something mainstream desktop Linux has only just recently managed (i.e. use bluetooth headsets reliably and without fuss).
Ditto for 10GB ethernet or what have you. Elegance is quickly sacrificed at the altar of efficiency and expediency. At the risk of inciting disagreement, look at what happened to the originally relatively elegant and simple X protocol.