Feature Request: Interactive Visualizations in Hermes WebUI

Description

I would like to have the ability to display and use interactive visualizations directly in the Hermes WebUI, similar to what Claude recently introduced (Text-Center Article) and how it is being attempted in OpenWebUI (Inline Visualizer v2 Plugin, OpenWebUI Reddit Discussion, OpenWebUI Plugin by thesysdev).

Concrete examples of desired features:

• Interactive Diagrams: Mermaid diagrams that are directly editable in the chat (e.g., flowcharts, sequence diagrams, Gantt charts)
• Dynamic Tables: Tables that are sortable, filterable, and expandable (e.g., for data analysis or project overviews)
• Interactive Graphics: Plotly charts, Matplotlib graphics, or other interactive visualizations that can be displayed and manipulated directly in the chat window
• Live Updates: Visualizations that automatically update when underlying data or code changes
• Export/Import: Ability to export or import visualizations as files (e.g., PNG, SVG, JSON)

Why is this important?

1. Improved User Experience and Productivity

• Faster Decision Making: Users can visualize data, processes, or systems directly without having to open external tools. This saves time and reduces context switching.
• Better Comprehension: Complex information (e.g., architectural diagrams, data flows, project plans) becomes easier to understand through interactive visualizations.
• Real-time Feedback: When working with data or code, users can immediately see the impact of changes without leaving the WebUI.

2. Competitive Advantage over other WebUIs

• Claude Sets the Standard: The introduction of interactive visualizations by Claude has shown that users strongly demand this feature. Hermes should not lag behind to remain attractive to developers and data analysts.
• OpenWebUI Shows the Way: The community has already developed solutions (e.g., the Inline Visualizer plugin) that can serve as inspiration. However, a native implementation in Hermes would be more elegant and better integrated.

3. Technical Feasibility

• Existing Architecture: The open Feature Request [#501: "Feature: Web UI Gateway — Local Browser-Based Interface with Streaming, Artifacts & Rich Rendering"] already supports the idea of richer rendering and better integration. The WebUI already uses a WebSocket-to-RPC bridge, which can serve as a foundation for interactive visualizations.
• Compatibility with Existing Tools: Hermes already supports Mermaid diagrams and can render Markdown. Extending this with interactive components would be a logical next step.
• Plugin Architecture: If a native implementation is too complex, a plugin interface for visualizations could be created, similar to OpenWebUI.

4. Target Audience and Use Cases

• Developers: Visualization of code structures, architectural diagrams, data flows
• Data Analysts: Interactive data visualizations (e.g., Plotly, Matplotlib)
• Project Managers: Gantt charts, Kanban boards, project progress visualizations
• System Designers: Mermaid diagrams for system architectures

Comparison with Existing Solutions

Feature	Hermes WebUI (current)	OpenWebUI (Plugin)	Claude	Desired in Hermes
Mermaid Diagrams	✅ (static)	✅ (static)	✅ (interactive)	✅ (interactive)
Dynamic Tables	❌	❌	✅	✅
Interactive Graphics	❌	❌	✅	✅
Live Updates	❌	❌	✅	✅
Export/Import	❌	❌	✅	✅

Proposal for Implementation

Phase 1: Basic Interactive Visualizations

• Mermaid Diagrams: Enable the creation and editing of diagrams directly in the chat window (e.g., via /diagram commands or by clicking on generated diagrams).
• Dynamic Tables: Enable the display of tables as interactive components (e.g., sortable, filterable).
• Live Updates: Visualizations automatically update when underlying data or code changes.

Phase 2: Advanced Visualizations

• Plotly/Matplotlib Support: Enable the display and manipulation of interactive data visualizations.
• Export/Import: Enable the export of visualizations as files (e.g., PNG, SVG, JSON) and the import of visualizations from files.
• Plugin Interface: Enable the integration of third-party plugins for specific visualization types.

Phase 3: Integration into Existing Workflows

• Kanban Boards: Enable the display of Kanban boards directly in the WebUI.
• Project Progress Visualization: Display project progress in interactive diagrams.
• Data Analysis Dashboards: Enable the creation of dashboards with multiple visualizations.

Technical Details

Dependencies

• WebSocket/RPC Architecture: The existing WebSocket-to-RPC bridge (Issue #501) should be used as the foundation for transmitting visualization data.
• Frontend Libraries: For displaying interactive components, libraries such as Mermaid.js, Plotly.js, Tabulator, or D3.js could be used.
• Backend Integration: The Hermes Agent API should be extended with endpoints that can generate and return visualization data.

Security Aspects

• Sandboxing: Visualizations should be executed in a sandbox environment to minimize security risks.
• Data Validation: All inputs and data used in visualizations must be validated to prevent injection attacks.

Conclusion

The introduction of interactive visualizations in the Hermes WebUI would significantly improve the user experience, increase productivity, and strengthen Hermes' competitiveness against other WebUIs (like OpenWebUI or Claude). Since the technical foundation already partially exists (WebSocket/RPC architecture, Mermaid support), this would be a realistic and valuable feature for the next development phase.

I kindly request that this feature be considered a priority and added to the roadmap. I am available for further discussions or details!

---

References:

• Claude: Interactive Visualizations
• OpenWebUI: Inline Visualizer v2 Plugin
• OpenWebUI: Reddit Discussion
• OpenWebUI: thesysdev Plugin
• Hermes Issue #501: Web UI Gateway