Concurrency needs a budget.

A worker limit answers a specific question: how much work may be active at once? It does not, by itself, tell you how much work is waiting.

Keeping those two counts separate makes a system easier to reason about.

Start with a small model

Take twelve independent jobs, each needing one unit of time. Give every worker one job per batch. If there are no other costs, the batch count is:

Here, $n$ is the number of jobs and $w > 0$ is the number of workers.

That arithmetic explains the interactive model in the lab. It counts batches rather than predicting a real program’s runtime. Uneven jobs, shared resources, and coordination costs change the picture.

Active is different from admitted

Imagine a service that accepts every incoming request but only processes four at a time. Active work is bounded. Waiting work may still keep growing.

The practical questions are separate:

• How many jobs may execute at once?
• How many may wait?
• What happens when there is no space left?
• Can waiting jobs expire or be cancelled?

A concurrency limit controls the first question. Admission rules and queue capacity address the others.

Put the boundary where it matters

A semaphore is one useful tool. In Tokio, a caller waits for an available permit; dropping the permit releases capacity. The placement matters: acquiring inside a newly spawned task limits that protected work, while acquiring before spawning can also limit how many such tasks are created. The Tokio semaphore documentation shows both patterns.

A permit is not automatically a bound on everything outside it. A queue can grow before acquisition. A job can hold memory while waiting. An external resource can remain the bottleneck.

A useful starting point is to name the resource being protected, identify exactly where work waits, and decide what happens when demand exceeds capacity. Then measure the behavior under that demand.

A limit is most useful when its boundary is clear.