New request
#1912: Subprocess pool refactor for profile-isolated quota probes (follow-up to #1873)
We'll provision a sandbox, run an agent against the issue, and open a draft PR. You can pull the branch and iterate from there.
Follow-up to v0.51.25 PR #1873: subprocess pool refactor for profile-isolated quota probes
Spinning out from Opus advisor on stage-320: the subprocess-isolation approach in v0.51.25 PR #1873 (api/providers.py) is correct as a synchronous spawn-per-probe pattern but has three operational concerns that should be addressed before this hits busy multi-profile installs.
Concerns
-
Spawn cost. Each probe creates a fresh
subprocess.run([PY, "-c", CODE, ...])per probe. The child re-importsagent.account_usageand transitive deps. Warm: 150-400ms; cold: 1s+. Synchronoussubprocess.runblocks the calling thread. Multiplies with profile count × provider count when a user's quota panel polls. -
No PDEATHSIG. Children are not actively killed on parent SIGKILL — risk of orphaned
python -cprocesses briefly outliving the WebUI. Visible as zombie/orphan rows inpsafter a hard restart. -
No concurrency cap. Nothing prevents many simultaneous interpreters across profiles × providers under concurrent UI polls. A user with 5 profiles × 3 providers each polling every 30s could spawn 15 interpreters simultaneously during a UI burst.
Suggested approach
- Long-lived worker pool keyed by profile-home. Spawn one worker per profile on first probe; reuse for subsequent probes. Workers exit on idle timeout (e.g. 5 minutes).
prctl(PR_SET_PDEATHSIG)viapreexec_fnon Linux so children die when the parent does.BoundedSemaphorefor max concurrent probes (default 4-8).stdin=DEVNULLto prevent any chance of children blocking on stdin.
Severity
Medium / operational follow-up. Functional today; ops risk under load. Not a blocker for v0.51.25 release.
Acceptance criteria
- Quota probe latency stays <500ms in the warm-pool case.
- No orphaned
python -cprocesses survive apkill -9 hermes-webui. - Concurrent-probe semaphore documented in
api/providers.pymodule docstring. - Existing
tests/test_provider_quota_status.pytests still pass. - New test: spawn 10 concurrent probes, assert no more than
MAX_CONCURRENT_PROBESPython interpreter processes exist mid-test.
References
- Original PR: #1873 (shipped in v0.51.25)
- Original issue: #1831
- Opus advisor verdict on stage-320: "current synchronous-spawn-per-probe is correct but inefficient and lacks PDEATHSIG / concurrency cap"
Refactoring synchronous subprocess spawns into a long-lived, profile-isolated worker pool is an internal operational improvement with no functional delta.
- api/providers.py
- api/subprocess_pool.py
- tests/test_providers.py
- api/config.py