github →

New request

#3432: Harden recall-prefill role adjacency on WebUI chat paths (strict chat templates)

We'll provision a sandbox, run an agent against the issue, and open a draft PR. You can pull the branch and iterate from there.

Issue
bug

Summary

Two related, pre-existing (not newly introduced) hardening items in the WebUI chat prompt-assembly paths, surfaced by the Codex + Opus review of #3324 (which fixed the guaranteed consecutive-user-turn case). Filing as a follow-up so #3324 can ship.

1. Recall-prefill role adjacency not guarded (strict chat templates)

When an admin configures a recall-prefill source (HERMES_PREFILL_MESSAGES_FILE or the prefill script hook) whose last message has role: "user", both WebUI chat paths can send:

system → … → user (recall prefill tail) → user (actual message)

Models with strict chat templates (Mistral, Gemma via llama.cpp) reject consecutive user turns with a Jinja 500.

  • This is conditional on user configuration and strictly better than before #3324: prior to #3324, _prefill_messages_with_webui_context always appended a session-context user message after the prefill, so consecutive-user turns were guaranteed whenever any prefill was configured. #3324 removed that always-appended message, leaving only this config-dependent possibility.
  • Fix shape: add a role-sequence normalizer on both the in-process (api/streaming.py) and gateway (api/gateway_chat.py) paths so prefill messages cannot end with a user role immediately before the browser's user turn (e.g. coalesce, or insert a minimal separator, or drop a trailing empty user prefill). Add a regression asserting no adjacent user roles for plain-text prefill.

2. _handle_chat_sync builds a WebUI prompt without _webui_ephemeral_system_prompt

api/routes.py _handle_chat_sync (the POST /api/chat fallback, commented "Not used by frontend") constructs its own prompt and does not route through _webui_ephemeral_system_prompt, so it would not carry the session/delivery context that #3324 centralized. Since it's documented as unused-by-frontend this is low priority, but if it's ever exercised it should use the same helper for consistency.

Acceptance

  • Both WebUI chat paths (in-process + gateway) never emit adjacent user roles regardless of configured prefill contents.
  • Regression test covering a prefill source whose last message is a user turn.
  • (Optional) _handle_chat_sync either routed through _webui_ephemeral_system_prompt or confirmed dead and removed.

Context: surfaced during the v0.51.216 (#3324) release gate. #3324 fixed the guaranteed consecutive-user case by moving session/delivery context into the ephemeral system prompt.

Assessmentadvisory
bug●● medium85% confidence

Need to add role-sequence normalization to both in-process and gateway chat paths to prevent consecutive user turns.

Likely files
  • api/streaming.py
  • api/gateway_chat.py
  • api/helpers.py
  • tests/test_chat_templates.py
Create the request

This opens a fresh agent run and a draft PR for issue #3432.

Cancel