github →

New request

#2476: macOS WebUI high CPU/battery during active Codex run with 300s stale timeouts

We'll provision a sandbox, run an agent against the issue, and open a draft PR. You can pull the branch and iterate from there.

Issue
bugupstream-changeinvestigationperformance

Bug Description

On macOS, Hermes WebUI can remain busy and consume noticeable CPU/battery during or after a long active run. In the same run, the underlying Hermes Agent call to the OpenAI Codex provider repeatedly hit the 300s stale-call cutoff:

Non-streaming API call timed out after 300s with no response (threshold: 300s)

This seems to be a combined WebUI/runtime observability issue:

  1. The WebUI background service can show sustained CPU use while an active run is open.
  2. The active run can keep retrying long provider calls for many minutes.
  3. The timeout/error surfaced as "non-streaming" even though the logs show a Codex streaming request path.
  4. The stale-call detector logged context=~0 tokens, while the final failure logged a very large context estimate (~98k tokens), suggesting the timeout estimator may not be reading the Codex Responses payload correctly.

Observed Behavior

Hermes WebUI was running as a macOS launchd service on 127.0.0.1:8787.

CPU samples for the WebUI server process while the active run was open:

  • 7.7%
  • 20.2%
  • 14.6%

Other Hermes services were mostly idle at the same time:

  • regular Hermes gateway: ~0.0% CPU
  • other profile gateways: ~0.0% CPU
  • dashboard: ~0.1% CPU
  • scheduled cron job: not running at the time

macOS power assertions did not show Hermes/Python/WebUI preventing sleep, so this appears to be CPU/energy usage rather than a sleep-prevention assertion.

The WebUI /health endpoint showed an active run, for example:

{
  "status": "ok",
  "sessions": 2,
  "active_streams": 1,
  "active_runs": 1,
  "runs": [
    {
      "phase": "running",
      "model": "gpt-5.5",
      "provider": "openai-codex",
      "ephemeral": false,
      "age_seconds": 80.5
    }
  ],
  "accept_loop": {
    "requests_total": 2650
  },
  "oldest_run_age_seconds": 80.5
}

Agent logs for the same WebUI session showed repeated stale-call timeouts:

Non-streaming API call stale for 300s (threshold 300s). model=gpt-5.5 context=~0 tokens. Killing connection.
API call failed (attempt 1/3) error_type=TimeoutError provider=openai-codex model=gpt-5.5 summary=Non-streaming API call timed out after 300s with no response (threshold: 300s)
Retrying API call ...
API call failed (attempt 2/3) ...
API call failed (attempt 3/3) ...
API call failed after 3 retries. Non-streaming API call timed out after 300s with no response (threshold: 300s) | provider=openai-codex model=gpt-5.5 msgs=69 tokens=~98,262

Nearby logs also showed the request path as Codex streaming:

OpenAI client created (codex_stream_request, shared=False) provider=openai-codex base_url=https://chatgpt.com/backend-api/codex model=gpt-5.5

Expected Behavior

  • WebUI should have low idle/background CPU usage when not actively doing useful work.
  • If a run is still active and consuming resources, the UI should make this obvious and provide an easy stop/cancel control.
  • If provider calls are expected to take longer for very large contexts, the stale timeout should scale correctly for Codex Responses requests.
  • Diagnostics should not report context=~0 tokens for a request that later logs ~98k tokens.
  • The error should accurately distinguish between non-streaming and streaming/Codex Responses code paths, if applicable.

Environment

  • OS: macOS 26.4.1
  • Hermes Agent: v0.13.0
  • Python: 3.11.15
  • Hermes WebUI: v0.51.74-dirty
  • WebUI git status at time of observation: behind origin/master by 79 commits, with one dirty local test file
  • Launch method: macOS launchd user LaunchAgent
  • LaunchAgent label: com.stefanvanbiljon.hermes-webui
  • WebUI port: 127.0.0.1:8787
  • Provider/model in failing run: openai-codex / gpt-5.5

Notes

This may already be improved in newer upstream commits; the local checkout was behind origin/master at the time of observation. Opening this issue to track the observed macOS energy impact, stuck/long active-run behavior, and the possible Codex Responses stale-timeout accounting mismatch.

Potentially relevant areas to inspect:

  • WebUI active-run lifecycle and polling/accept-loop behavior while runs are active or failing
  • Whether the UI exposes enough status/control for long active runs
  • Hermes Agent stale-call detector for Codex Responses payloads, especially token estimation and the non-streaming wording for codex_stream_request
Assessmentadvisory
bug●●● hard80% confidence

Performance bug rooted in timeout estimation, streaming-state misdetection, and run lifecycle management requiring deep cross-layer investigation.

Likely files
  • api/providers/codex.py
  • api/services/streaming.py
  • api/utils/timeouts.py
  • api/routes/health.py
  • api/models/runs.py
  • static/js/services/sse.js
  • server.py
Create the request

This opens a fresh agent run and a draft PR for issue #2476.

Cancel