Why can’t you suspend a CPU bound task?

Why would you want to do that (in C++ context)?

To segment your tasks so that their scheduling is fair irrespective of operating system

No, you played ostrich to have the pleasure of having the final word like OS threads have nothing to do with the os

With segmented processing in coroutines, that will never happen

Simply adopting C++ coroutines will not automagically segment anything

Wow, that exactly what I said in the very first place

Now we have a consensus. Cheers 🍻

Well, of course if your task segmentation is non-sensical your time utilisation will be non sensical too. Why do you think the Erlang runtime prohibits loops for? Precisely task segmentation

Let’s make this easier with an illustration. This is just based on my understanding of async programming in other languages. Suppose we have 8 OS threads prepared to run coroutines, and we create 50 coroutines (representing 50 request connections). The mapping here is m:n, meaning 8 OS threads will execute 50 coroutines. Now, imagine 8 requests come into our server: 1. We spawn 8 coroutines. 2. Our 8 OS threads pick them up and execute them. 3. One of the requests makes a Postgres query. 4. The query code runs → we pause that coroutine → the OS thread is now free and can pick up another coroutine (because the thread is not blocked). 5. After some time, the query result is ready. 6. One of the OS threads resumes the paused coroutine and continues execution until it’s done. So far, everything looks good. But what if one of the requests involves 8 minutes of heavy computation inside its handler? Here’s what happens : A new connection arrives → wrapped in a coroutine → an OS thread picks it up. This coroutine contains pure CPU-bound work. Unlike IO tasks, CPU tasks should not be paused. IO can run in the background while the OS takes care of it, but CPU computation requires an OS thread to keep running continuously. If we try to pause this coroutine, we’re also pausing the computation itself. So: If the computation takes 8 minutes, but we pause it for 1 minute → the total latency becomes 8 + 1 = 9 minutes. Pausing/resuming repeatedly just makes it slower and adds pointless delays. Now, let’s improve this by moving long running CPU tasks to a separate dedicated thread pool: A new connection arrives → wrapped in a coroutine → one of our 8 OS threads picks it up. The coroutine sees a long CPU task → instead of blocking, it delegates the work to the dedicated thread pool: > “Hey bro, give me one OS thread from your pool to handle this heavy task. I’ll go back to serving new requests.” The dedicated thread pool runs the heavy computation until it finishes, then returns the result to the user. Meanwhile, the connection handler thread has already moved on to accept and execute other coroutines. If a coroutine needs to wait for IO → it pauses. If a coroutine needs heavy CPU work → it gets delegated to the dedicated pool. This way: No request handler thread is ever blocked. No CPU computation task is paused (which would only delay results). Request handling remains fast and responsive, because the handler threads are always free to accept new work. IO tasks → can be paused safely, no problem. Long CPU tasks → should not be paused. Instead, offload them to a dedicated thread pool so request handling stays smooth. (And true paralel)

The very first message in this discussion I sent was about how segmenting coroutines is essential

Given a good task segmentation, what stops one from calling a scheduler directly?

Nothing, I used to do that with callbacks and fanin fanout before, it is just made tremendously easier

Yup. So this has nothing to do with coroutines right.

Coroutines just make it very much easier and make the unsegmented code and segmented code almost identical

Please illustrate

Let’s make this easier with an illustration. This is just based on my understanding of async programming in other languages. Suppose we have 8 OS threads prepared to run coroutines, and we create 50 coroutines (representing 50 request connections). The mapping here is m:n, meaning 8 OS threads will execute 50 coroutines. Now, imagine 8 requests come into our server: 1. We spawn 8 coroutines. 2. Our 8 OS threads pick them up and execute them. 3. One of the requests makes a Postgres query. 4. The query code runs → we pause that coroutine → the OS thread is now free and can pick up another coroutine (because the thread is not blocked). 5. After some time, the query result is ready. 6. One of the OS threads resumes the paused coroutine and continues execution until it’s done. So far, everything looks good. But what if one of the requests involves 8 minutes of heavy computation inside its handler? Here’s what happens : A new connection arrives → wrapped in a coroutine → an OS thread picks it up. This coroutine contains pure CPU-bound work. Unlike IO tasks, CPU tasks should not be paused. IO can run in the background while the OS takes care of it, but CPU computation requires an OS thread to keep running continuously. If we try to pause this coroutine, we’re also pausing the computation itself. So: If the computation takes 8 minutes, but we pause it for 1 minute → the total latency becomes 8 + 1 = 9 minutes. Pausing/resuming repeatedly just makes it slower and adds pointless delays. Now, let’s improve this by moving long running CPU tasks to a separate dedicated thread pool: A new connection arrives → wrapped in a coroutine → one of our 8 OS threads picks it up. The coroutine sees a long CPU task → instead of blocking, it delegates the work to the dedicated thread pool: > “Hey bro, give me one OS thread from your pool to handle this heavy task. I’ll go back to serving new requests.” The dedicated thread pool runs the heavy computation until it finishes, then returns the result to the user. Meanwhile, the connection handler thread has already moved on to accept and execute other coroutines. If a coroutine needs to wait for IO → it pauses. If a coroutine needs heavy CPU work → it gets delegated to the dedicated pool. This way: No request handler thread is ever blocked. No CPU computation task is paused (which would only delay results). Request handling remains fast and responsive, because the handler threads are always free to accept new work. IO tasks → can be paused safely, no problem. Long CPU tasks → should not be paused. Instead, offload them to a dedicated thread pool so request handling stays smooth. (And true paralel)

Let’s make this easier with an illustration. This is just based on my understanding of async programming in other languages. Suppose we have 8 OS threads prepared to run coroutines, and we create 50 coroutines (representing 50 request connections). The mapping here is m:n, meaning 8 OS threads will execute 50 coroutines. Now, imagine 8 requests come into our server: 1. We spawn 8 coroutines. 2. Our 8 OS threads pick them up and execute them. 3. One of the requests makes a Postgres query. 4. The query code runs → we pause that coroutine → the OS thread is now free and can pick up another coroutine (because the thread is not blocked). 5. After some time, the query result is ready. 6. One of the OS threads resumes the paused coroutine and continues execution until it’s done. So far, everything looks good. But what if one of the requests involves 8 minutes of heavy computation inside its handler? Here’s what happens : A new connection arrives → wrapped in a coroutine → an OS thread picks it up. This coroutine contains pure CPU-bound work. Unlike IO tasks, CPU tasks should not be paused. IO can run in the background while the OS takes care of it, but CPU computation requires an OS thread to keep running continuously. If we try to pause this coroutine, we’re also pausing the computation itself. So: If the computation takes 8 minutes, but we pause it for 1 minute → the total latency becomes 8 + 1 = 9 minutes. Pausing/resuming repeatedly just makes it slower and adds pointless delays. Now, let’s improve this by moving long running CPU tasks to a separate dedicated thread pool: A new connection arrives → wrapped in a coroutine → one of our 8 OS threads picks it up. The coroutine sees a long CPU task → instead of blocking, it delegates the work to the dedicated thread pool: > “Hey bro, give me one OS thread from your pool to handle this heavy task. I’ll go back to serving new requests.” The dedicated thread pool runs the heavy computation until it finishes, then returns the result to the user. Meanwhile, the connection handler thread has already moved on to accept and execute other coroutines. If a coroutine needs to wait for IO → it pauses. If a coroutine needs heavy CPU work → it gets delegated to the dedicated pool. This way: No request handler thread is ever blocked. No CPU computation task is paused (which would only delay results). Request handling remains fast and responsive, because the handler threads are always free to accept new work. IO tasks → can be paused safely, no problem. Long CPU tasks → should not be paused. Instead, offload them to a dedicated thread pool so request handling stays smooth. (And true paralel)

Your example supposes that only one of the connections required 8 minutes to handle. Suppose that all 50 of them do require processing, all 50 connections will receive their results at the same time, fairly. Similarly, by using segmented coroutines, the other, newer connections will still be handled and be given roughly as much CPU time as the processing, even if the os is not fair in its time distribution.

To not block worker threads? Well

@QNeko basically you mean this?

I just make i only 1 that does 8 minutes compuation to not repeat typing the explanation, it can be scalled easily to 30 of the connections has 8 minutes computation in it, and it behave the same. No request handler thread is blocked, and does not add unnecerery delay/latency to the computation result

Also, it is generally easier to make coroutines cancelable this way than os thread based non segmented tasks

It does not if and only if your threads are not context switched, your os scheduling is fair, and the results are useful (task not cancelled). All of that goes to the drain if more threads than cpu cores are ever used

The question is, why we are looking for fairness in the first place?

It does not if and only if your threads are not context switched, your os scheduling is fair, and the results are useful (task not cancelled). All of that goes to the drain if more threads than cpu cores are ever used

Having more threads than CPU cores is actually fine. But if you try to context switch a CPU bound task with coroutines, you’re only hurting performance : It slows down request handling because fewer requests can enter the processing stage. It delays the math heavy computation itself, since it keeps getting paused and resumed over and over. That’s why offloading long heavy CPU work to a dedicated thread pool is much better, overall, the whole application runs faster and more smoothly.

That would also be the case with having more threads than os core, except you would have no control over it

The dedicated thread won’t be spawned unless it’s actually needed. When a request handler thread that executes coroutine that wraps the connection encounters a long running computation, it asks for a dedicated thread. That thread is then spawned, takes over the task, and runs it until completion. Once the computation is done, the thread doesn’t get destroyed right away. Instead, it’s kept in the pool for, say, 5 minutes, just in case another long heavy CPU task shows up soon. If no new task arrives within that time, the thread is removed and the pool goes back to empty.

And in what does that mean your threadpool does not compete with your connection acceptor coroutines threads?

Say you have 50 connections number crunching. If you do fifo, and do not check for cancellation (which many do because it is annoying) in a fair system, not everyone will wait the same amount of time, so it is very likely your queue will be tailed by cancelled operations that you will perform anyway, leading to subsequent connections also waiting too long and cancelling etc etc

I thought it's a common sense in [any languages/runtimes with an async equivalent mechanism] not to block the worker threads no?

They’re competing for resources, but if no one takes a thread from the pool within 5 minutes, it gets released right away. The good part is, this doesn’t add additional delay to accepting new connections, the OS handles that with context switching not OS context switch + user space context switch On the other hand, if we don’t do it this way, our coroutine could get blocked or experience extra delay, because then we’d be dealing with OS context switching + user-space context switching. In contrast, the first approach only involves OS context switching, which is the total is fewer operation in the total That is my understanding :v

Yeah, same with the user space scheduler

The model that I type above is the same that is uses on Go’s goroutines, Java’s virtual threads, or Kotlin’s coroutines. If the blocking is caused by IO, the coroutine (or goroutine/virtual thread) simply gets paused and the underlying thread is freed. But if the blocking comes from a long running CPU task, the scheduler will let it block the thread, and the scheduler will spin up a new OS thread to replace the blocked thread. If the previous thread is done and arrive, it continue the execution until is done and be freed

All of these are stackful coroutines and roughly on the scale as an os thread to switch to, minus the system call

I do not think they are the same. Because CPU bound task need OS thread otherwise it will not run, if you add user space context switch in pure CPU task, you just add new delay, because time of OS thread context switch + time of user space context switch

How about rust async that is also stackless, and they also use dedicated thread pool in tokio::spawn_blocking?

Why would the CPU bound task not run, and why do you consider that the user space scheduler will ba slower than the kernel scheduler when the user space scheduler does not have the following concerns/responsibilities: - process based fairness - memory eviction - interrupt timer - other processes to schedule - permissions - cgroups

Because many people still can't get into their head to segment their tasks and they instead rely on the kernel interrupting their control flow to implement it

nah tokio also have this https://docs.rs/tokio/latest/tokio/task/fn.yield_now.html

I thought it's a common sense in [any languages/runtimes with an async equivalent mechanism] not to block the worker threads no?

Apparently this is IO and that's why we use coroutines. In a purely CPU-bound setup such as one-off offline batch processing, I don't see the need.

Apparently this is IO and that's why we use coroutines. In a purely CPU-bound setup such as one-off offline batch processing, I don't see the need.

Yes, in a batch process with absolutely no IO it is not needed, but pure batch processes are fairly rare

Depends on the requirement, say running as a Job in a k8s cluster

Still fairly rare that you have a batch job that does all its thing in memory, most of the time you have database shenanigans or a big folder of files to process in batch jobs

But those tasks do not require fairness

Why would the CPU bound task not run, and why do you consider that the user space scheduler will ba slower than the kernel scheduler when the user space scheduler does not have the following concerns/responsibilities: - process based fairness - memory eviction - interrupt timer - other processes to schedule - permissions - cgroups

Because many people still can't get into their head to segment their tasks and they instead rely on the kernel interrupting their control flow to implement it

I came again, I just take a bath first :v Because you pause the coroutine sir, so the underlying thread is freed and CPU bound task can not run without our own thread unlike IO bound task I think I should type in better, what I mean is you are adding user space context switch on top of kernel level context switch which is 2 operations

You can schedule the CPU bound task on the coroutine thread as a coroutine

I still do not understand :< do you have working example that implement that? Like other lang or something