github →

New request

#1912: Subprocess pool refactor for profile-isolated quota probes (follow-up to #1873)

We'll provision a sandbox, run an agent against the issue, and open a draft PR. You can pull the branch and iterate from there.

Issue
enhancement

Follow-up to v0.51.25 PR #1873: subprocess pool refactor for profile-isolated quota probes

Spinning out from Opus advisor on stage-320: the subprocess-isolation approach in v0.51.25 PR #1873 (api/providers.py) is correct as a synchronous spawn-per-probe pattern but has three operational concerns that should be addressed before this hits busy multi-profile installs.

Concerns

  1. Spawn cost. Each probe creates a fresh subprocess.run([PY, "-c", CODE, ...]) per probe. The child re-imports agent.account_usage and transitive deps. Warm: 150-400ms; cold: 1s+. Synchronous subprocess.run blocks the calling thread. Multiplies with profile count × provider count when a user's quota panel polls.

  2. No PDEATHSIG. Children are not actively killed on parent SIGKILL — risk of orphaned python -c processes briefly outliving the WebUI. Visible as zombie/orphan rows in ps after a hard restart.

  3. No concurrency cap. Nothing prevents many simultaneous interpreters across profiles × providers under concurrent UI polls. A user with 5 profiles × 3 providers each polling every 30s could spawn 15 interpreters simultaneously during a UI burst.

Suggested approach

  • Long-lived worker pool keyed by profile-home. Spawn one worker per profile on first probe; reuse for subsequent probes. Workers exit on idle timeout (e.g. 5 minutes).
  • prctl(PR_SET_PDEATHSIG) via preexec_fn on Linux so children die when the parent does.
  • BoundedSemaphore for max concurrent probes (default 4-8).
  • stdin=DEVNULL to prevent any chance of children blocking on stdin.

Severity

Medium / operational follow-up. Functional today; ops risk under load. Not a blocker for v0.51.25 release.

Acceptance criteria

  • Quota probe latency stays <500ms in the warm-pool case.
  • No orphaned python -c processes survive a pkill -9 hermes-webui.
  • Concurrent-probe semaphore documented in api/providers.py module docstring.
  • Existing tests/test_provider_quota_status.py tests still pass.
  • New test: spawn 10 concurrent probes, assert no more than MAX_CONCURRENT_PROBES Python interpreter processes exist mid-test.

References

  • Original PR: #1873 (shipped in v0.51.25)
  • Original issue: #1831
  • Opus advisor verdict on stage-320: "current synchronous-spawn-per-probe is correct but inefficient and lacks PDEATHSIG / concurrency cap"
Assessmentadvisory
chore●● medium85% confidence

Refactoring synchronous subprocess spawns into a long-lived, profile-isolated worker pool is an internal operational improvement with no functional delta.

Likely files
  • api/providers.py
  • api/subprocess_pool.py
  • tests/test_providers.py
  • api/config.py
Create the request

This opens a fresh agent run and a draft PR for issue #1912.

Cancel