Will go really multiplex goroutines which are blocking on system calls as well as those blocking on go-level entities (e.g. channels)?
The underlying OS threads will do that, but if goroutines aren't 1-1 with OS threads (which I don't believe they are), how is this achieved? Are all current syscalls intercepted?
What about calls out to external C libs which in turn call blocking syscalls?
(I could test, but if go happens to be running all my goroutines in their own OS threads, I could get a false positive "pass")
The magic trick is that it runs the scheduler just before the blocking call, and if need be it will spawn a new thread to make the blocking call in. That way the thread that thinks it made the blocking call is free to continue running other goroutines.
I don't know if/how Go does this, but what I would do is something like the following: I'd have a thread pool of worker threads which execute the goroutines, so that if a system call blocks the goroutine, the thread will block. The threads would also use work-stealing so that if a thread runs out of tasks (goroutines), they can steal them from another threads task queue. As long as system calls are reasonably rare, this will work with if threads 1-1 with processor cores/hardware threads, but if system calls are common, this this could cause livelock (all threads blocked by system calls, but runnable goroutines cannot run). I imagine that often this would solve itself as system calls unblock the threads when they complete, but to protect against this, I would also have a low priority maintenance thread (or some other mechanism to run this check every so often - perhaps a wrapper around system calls) which checks how many runnable threads are in the thread pool and if there are less than a certain amount (eg number of available hardware threads), then it would spawn additional threads (or, again, perhaps a pool of these backup threads is kept around for cheaper spawning/despawning). Then later as the system calls unblock, the threads can be retired again.
Thats just off the top of my head how I'd handle it. I imagine with some more thought you can come up with a better mechanism.
The underlying OS threads will do that, but if goroutines aren't 1-1 with OS threads (which I don't believe they are), how is this achieved? Are all current syscalls intercepted?
What about calls out to external C libs which in turn call blocking syscalls?
(I could test, but if go happens to be running all my goroutines in their own OS threads, I could get a false positive "pass")