> they start by, without evidence, asserting that the main thing fork() does is to insert and remove processes from a list of processes
Since fork() as far as I understand is effectively copy-on-write in modern architectures on linux, that's probably not a terrible model at least for the initial step but it does seem kinda over-simplified if you think about the rest of the lifetime of the process and the different execution models with processes on a computer vs a supercomputing cluster.
What looks like it happened is that they had a cool enough idea: with supercomputers, the scheduler is kinda like the operating system. The scheduler becomes the bottleneck in some workloads. So why not replace the core data structures and operations with a DBMS which is kinda built for this sort of highly distributed, high contention state management situation. Also the larger general idea of comparing it to an OS sounds curiously close to the mesos / kubernetes worldview of things in some ways: "the operating system of your datacenter".
Then some annoying but probably correct person was like "hey we need a basis for comparison and a benchmark because this is an academic paper and that's how systems papers must be because academia in 2018" so they shoehorned in some apples and oranges comparison and we got this.
> Then some annoying but probably correct person was like "hey we need a basis for comparison and a benchmark because this is an academic paper and that's how systems papers must be because academia in 2018"
That person was correct regarding the fact that a benchmark is needed, if only in order to validate their mathematical model. But they hardly benchmark the correct thing. Adding a process to the process tree is only on of the things that happens when you fork() a process, and frankly the most trivial one.
Quite unsurprisingly, sparse matrix associative multiplication in a distributed system (which is how they simulate forking) is faster than a local fork() implementation that actually implements process creation. There's no mention about how well this DBMS-backed scheduler deals with process and resource management, process group management, core- or node-pinning, process tracing and so on.
Edit: to clarify this point with a trivial example. In Linux, fork() relies heavily on copy-on-write, so that process creation can be as cheap as possible. When a child is created, it doesn't get new memory pages -- by default, it just shares the parent's. A long time ago, on systems that didn't have a MMU, the system would copy the entire memory space of the parent (!!!); nowadays it just copies a couple of system structures (which is not trivial, either, because there's potential for race conditions there). Only when the process wants to write to a memory page does the operating system actually create a copy of that page for the process that wanted to do the modification.
Managing memory pages isn't all that easy, because the memory pages aren't a software-only construct, they're backed by an actual hardware device (the MMU) which has a sort of cache (the TLB) because, as the authors like to remind us, this ain't a PDP-11 anymore. fork()-ing 2^32 processes at the same time is not a trivial problem, but it's a piece of cake compared to 2^31 processes saying "hey yeah I kindda need a new copy of this page" at seemingly random times, some of them in very particular patterns that need to be serviced really quickly -- especially when some processes request pages one at a time, while others just request their whole space because they do exec().
It's anyone's guess how well a DBMS would handle this sort of stuff. Frankly, I don't know enough about DBMSes to do more than speculate -- but I don't see this article shedding light upon this problem, either.
Edit done :).
The conclusion that "... the simulations show that TabulaROSA has the potential to perform operating system functions on a massively parallel scale." (emphasis mine) is greatly inflated. The simulations show that it can efficiently perform a small part of one operating system function; if you squint and you're optimistic, you can maybe conclude that they show it has the potential to implement process creation on a massively parallel scale.
Even later edit: I feel like this is one of those upsetting cases of re-discovering something that we knew already. There's a lot of literature dating from 1990s about how one could manage processes running on a massively-distributed scale, and while I recall most of it too dimly for actual details, I think there was a widespread feeling that whatever orchestration solution we're finally gonna figure out is going to look a lot like a DBMS system. This paper doesn't seem to cite any of it, though, and so it unsurprisingly doesn't try to solve any of the many problems that have been shown to exist.
Since fork() as far as I understand is effectively copy-on-write in modern architectures on linux, that's probably not a terrible model at least for the initial step but it does seem kinda over-simplified if you think about the rest of the lifetime of the process and the different execution models with processes on a computer vs a supercomputing cluster.
What looks like it happened is that they had a cool enough idea: with supercomputers, the scheduler is kinda like the operating system. The scheduler becomes the bottleneck in some workloads. So why not replace the core data structures and operations with a DBMS which is kinda built for this sort of highly distributed, high contention state management situation. Also the larger general idea of comparing it to an OS sounds curiously close to the mesos / kubernetes worldview of things in some ways: "the operating system of your datacenter".
Then some annoying but probably correct person was like "hey we need a basis for comparison and a benchmark because this is an academic paper and that's how systems papers must be because academia in 2018" so they shoehorned in some apples and oranges comparison and we got this.