Synthetic Heart Documentation

The Documentation for Synthetic Heart is below; use the quickstart to get running and the User / Developer sections to explore features and internals.

Welcome to the Synthetic Heart documentation. These pages are built with Sphinx and hosted on Read the Docs. Every push to the repository triggers a new build of this wiki.

The following sections provide an overview of the project and instructions for getting started.

User guide:

• Quickstart
  • Security & operational notes
  • Stop Synthetic Heart
  • Component Auto-Discovery
• Usage Overview
  • Component Management
  • Runtime Commands
  • Configuration
  • WebUI and VRM Avatars
• Features
  • Modular Architecture
  • Adaptive Intelligence
  • Multi-Platform Integration
  • Progressive Web App (PWA) Mode (Experimental)
  • Desktop Notifications (Web UI)
  • Web UI Chat Archiving
  • Avatar System
  • Web UI Animation Sync
  • Plugin Ecosystem
  • Contextual Memory
  • Security & Access Control
  • Extensibility
  • AI Diary
• Cortex
  • Quick summary (for users)
  • User-facing notes
  • Switching and configuration
  • Available Cortexes (user guide)
  • Selenium (browser-driven) — what & how
  • Gemini API (official API)
  • Live (Gemini Live — real-time voice)
  • Agent (action-capable runtimes)
  • Quick FAQ
  • See also
  • Related pages
• Interfaces
  • Interface Architecture
  • Available Interfaces
  • Discord Interface
  • Telegram Bot Interface
  • Reddit Interface
  • Web UI Interface
  • Plugins
  • Ollama-Compatible Server
  • Interface Registration System
  • Developing Interfaces
  • Interface Actions
  • Security and Validation
  • Best Practices
• Plugins
  • Plugin Architecture
  • Available Action Plugins
  • Bio Manager Plugin
  • Emotion Manager Plugin
  • Recent Chats Plugin
  • Terminal Plugin
  • Event Plugin
  • AI Diary Personal Memory
  • Plugin Registration System
  • Developing Plugins
  • Plugin Flow
  • Best Practices
• External Endpoints
  • Overview
  • Note
  • Built-in default endpoint
  • Getting started
  • Provider presets
  • Probe status
  • Model list and default model
  • Subsystem mapping
  • Use cases
  • Legacy HTTP TTS endpoints
  • Troubleshooting
  • API Reference
• Auris & Vox — Audio Subsystem
  • Overview
  • Architecture
  • Registry Pattern
  • Auris — STT Subsystem
  • Vox — TTS & Lip-sync Subsystem
  • Lip-sync Integration
  • Migration from tts_lipsync
  • Live — Bidirectional Streaming
  • Testing
• VRM Avatar Animations
  • Overview
  • Animation States
  • Animation System Architecture
  • Integration with Message Chain
  • 3D Environment
  • Adding Custom Animations
  • File Locations
  • Troubleshooting
  • Browser Compatibility
  • Future Enhancements
  • See Also
• Animation System
  • Overview
  • Architecture
  • Animation States
  • Animation Flow
  • Usage
  • Frontend WebSocket Protocol
  • Centralized Animation State
  • Adding New Animations
  • Temporary Animation Uploads
  • Integration with Interfaces
  • Flexible Animation Sections (Intro/Loop/Outro)
  • effective_loop Determination
  • IDLE Rotation Loop
  • Smart Eye Behaviour
  • Known Issues Fixed
  • Debugging
  • Limitations
  • Transport Layer
  • Priority & Preemption
  • Audio State Tracking
  • Watchdog
  • Karada REST & WebSocket API
  • Future Enhancements
  • See Also
• WebUI configuration control types
  • Notes
  • See also
• Gemini Live API — SyntH Discord Voice Integration
  • Overview
  • Key files
  • Audio format details
  • Session lifecycle
  • Tool / function calling
  • Voice state cleanup
  • Dependencies
  • Configuration
  • Troubleshooting
  • Remaining work
• FAQ
  • What does “Synthetic Heart” mean? Is it emulating a biological heart?
  • Isn’t the LLM the core of the AI? Changing it would make it a different persona.
  • Does the LLM lie or hallucinate?
  • Can I integrate the synth into video games to play together?
  • Can the synth speak, not just text?
  • If the synth communicates on multiple platforms like Telegram and Discord, is it the same persona?
  • What about privacy concerns?
  • Why doesn’t the synth have more animations in 3D space to seem more alive?

Developer guide:

• Component Development Guide - Configuration Management
  • Overview
  • The Standard Pattern
  • ConfigVar API
  • When to Use get_value() vs get_var()
  • Complete Example: Telegram Bot Interface
  • Configuration Options
  • Configuration Precedence
  • Testing Your Component
  • Common Patterns
  • Migration from Old Pattern
  • Need Help?
  • Summary
• Development Components System
  • Overview
  • Key Features
  • Architecture
  • Directory Structure
  • Usage
  • WebUI Indicators
  • API Reference
  • Safety Considerations
  • Best Practices
  • Troubleshooting
  • See Also
• API Endpoints Reference
  • General Convention
  • System & Configuration
  • Components
  • Diary & Chat History
  • Agent‑related APIs
  • Animations, VRM & Emotion State
  • WebSocket Endpoints
  • Integration & Extended API
  • Config & Miscellaneous
  • Ollama‑Compatibility Server
  • Further Reading
• Component Development Pattern - Two-Phase Initialization
  • Overview
  • Why Two Phases?
  • Implementation Pattern
  • Core Initialization Sequence
  • Hot Reload Support
  • Rules for Component Developers
  • Migration Guide
  • Benefits
  • Testing
  • See Also
• Two-Phase Initialization Implementation
  • Problem
  • Solution: Two-Phase Initialization
  • Implementation
  • Verification
  • Benefits
  • Next Steps
  • Documentation
  • Related Files
• Configuration Management Guide
  • ⚠️ IMPORTANT: Use ConfigVar for Global Variables
  • Migration from Old Pattern
  • Registering Exposed Variables
  • Need Help?
  • Summary
• Prompt Engine JSON Prompt
  • Overview
  • Prompt structure
  • Input section
  • Context section
  • Recon contributions
  • Instructions and chat mode
  • Actions block
  • Prompt reduction
  • Developer notes
• Memory Search and Management
  • How memory search is triggered
  • Search pipeline and query behavior
  • Deduplication and ordering
  • How memories are injected into the prompt
  • Memory config flags and behavior toggles
  • Memory management internals
  • Developer notes
  • See also
• Contributing

Building the Documentation

Install the documentation requirements from the repository root and run:

sphinx-build -b html docs docs/_build/html

The generated HTML files will be available under docs/_build/html.