SideCar

Overview

The SideCar is a system component that manages standalone Python server processes running outside of TouchDesigner. Operators like Florence-2, OCR, and ACE-Step delegate their GPU-intensive model loading and inference to these sidecar servers, keeping TouchDesigner responsive while heavy computation runs in separate processes with isolated virtual environments. The SideCar auto-discovers which servers to manage from its embedded VFS, handles their full lifecycle (export, install, start, stop), and continuously monitors their health.

System Component

The SideCar is not a placeable operator you wire into networks. It is a system-level service that runs alongside TouchDesigner. Operators that need it communicate with their sidecar servers automatically over HTTP.

Key Features

• Auto-discovery — Scans its embedded VFS for sidecar folders containing a sidecar.json config and automatically registers each one on init.
• Multi-sidecar support — Manages multiple independent sidecar processes simultaneously, each with its own port, health endpoint, and virtual environment.
• VFS export with version tracking — Embeds sidecar server code in the operator’s VFS. Compares VFS version against what is on disk and re-exports automatically when starting a newer version.
• Dependency management — Uses uv sync to create and maintain isolated Python virtual environments per sidecar.
• Health monitoring — Periodic one-shot health checks update connection state, populate a stats table with system and per-model metrics, and dispatch any queued tasks when a sidecar comes online.
• Task queue — Operators can store tasks that execute automatically once their sidecar becomes available.

Requirements

• uv — The uv package manager must be installed on your system. The SideCar uses uv sync to create and manage isolated Python virtual environments per sidecar. Install it with pip install uv or follow the uv installation guide.
• CUDA GPU (recommended) — While the servers can run on CPU, a CUDA-capable GPU is strongly recommended for reasonable inference performance with ML models.

Supported Model Endpoints

The default sidecar server (vision_sidecar) hosts the following endpoints:

Model	Endpoint	Use Case
EasyOCR	/ocr/	Text extraction from images
PaddleOCR	/ocr/	Text extraction from images (alternative engine)
Florence-2	/florence/process	Vision tasks — captioning, object detection, OCR, segmentation
ACE-Step	/acestep/generate	Music generation

Models are loaded on demand and report their status, memory usage, and request statistics through the /health endpoint. Additional sidecar servers (e.g., embedding_sidecar) can be embedded and auto-discovered from the VFS without code changes.

First-Time Setup

1. Open the SideCar operator (found inside the LOPs system components).
2. On the Server page, pulse Export Sidecar Files to extract the server code from the operator’s VFS to disk.
3. Pulse Check Dependencies to verify the virtual environment status.
4. If the status shows “Venv not found”, pulse Install Dependencies. This runs uv sync in a console window — wait for it to complete.
5. Once dependencies are installed, pulse Start Server. A new console window opens with the sidecar server.
6. The Connected indicator turns on once the server responds to health checks.

Automatic Export

When you pulse Start Server, the SideCar automatically checks if the VFS version is newer than what is on disk and re-exports if needed. You only need to manually export if you want to inspect or modify the server files.

Starting and Stopping

• Start Server — Launches all registered sidecar processes. If a virtual environment does not exist yet, you will be prompted to install dependencies first.
• Stop Server — Sends a graceful shutdown request to all running sidecar servers. Each process terminates after completing any in-flight requests.
• Enable Monitoring — When enabled (default), the SideCar periodically checks each server’s health endpoint and updates the connection status. When a sidecar comes online, any queued tasks from operators that tried to use the server before it was ready are automatically dispatched.

Port Configuration

Each registered sidecar gets its own port parameter on the Ports page (e.g., “vision_sidecar port”, “embedding_sidecar port”). These are created automatically during auto-discovery from the sidecar.json defaults. You can change a port at any time — the SideCar updates the sidecar’s URL immediately and writes the new port back to the exported sidecar.json on disk so it persists across sessions.

Version Management

The SideCar embeds its server code in the operator’s VFS (Virtual File System). Two version fields on the Server page help track whether the exported files on disk are current:

• VFS Version — The version embedded in the operator.
• Disk Version — The version currently exported to disk.

If VFS Version is newer than Disk Version, the SideCar will automatically re-export when you start the server, or you can manually pulse Export Sidecar Files.

How Operators Connect

Operators like Florence-2 and OCR do not need manual configuration to find the SideCar. They call EnsureSidecar() which either returns the server URL if already connected, or starts the server automatically. The typical flow is:

1. An operator (e.g., Florence-2) calls EnsureSidecar('vision_sidecar') on the SideCar.
2. If the server is running and healthy, the SideCar returns the base URL immediately.
3. If not running, the SideCar starts the server process and the operator stores its task in the queue.
4. Once the server passes its health check, queued tasks are automatically dispatched.

Monitoring and Stats

When monitoring is active, the SideCar runs one-shot health checks on a 2-second interval using delayMilliSeconds rescheduling. Each check queries every registered sidecar’s health endpoint and updates an internal stats_table DAT with:

• System CPU and memory usage
• GPU memory and utilization (when available)
• Per-model request counts, average processing times, and loaded status
• Server uptime and request rate

The Connected parameter reflects whether any registered sidecar is currently healthy. A sidecar is marked disconnected only after 2 consecutive health check failures, preventing brief network hiccups from triggering false disconnections.

Troubleshooting

• “uv not found” dialog — Install uv with pip install uv or via the official installer, then restart TouchDesigner so it picks up the updated PATH.
• Server starts but Connected stays off — The server may still be loading. Check the sidecar console window for startup progress. The health check runs every 2 seconds and requires 2 consecutive failures before marking disconnected.
• Install Dependencies opens a console and closes immediately — Check if uv is accessible from the command line. Open a terminal and run uv --version to verify.
• Model loading fails in sidecar console — The model’s dependencies (e.g., torch, transformers, easyocr) may not be installed in the sidecar’s venv. Check the console output for missing package errors.
• Server crashes on startup — Look for sidecar_error.log in the sidecar’s temp directory. Common causes are missing Python packages or CUDA version mismatches.
• Port conflict — If a sidecar fails to start because its port is already in use, change the port on the Ports page and pulse Start Server again.

Parameters

Server

Status (Status) op('sidecar').par.Status Str

Default:

"" (Empty String)

Process Management Header

Start Server (Startserver) op('sidecar').par.Startserver Pulse

Default:

False

Stop Server (Stopserver) op('sidecar').par.Stopserver Pulse

Default:

False

Enable Monitoring (Enablemonitoring) op('sidecar').par.Enablemonitoring Toggle

Default:

False

Connection Header

Connected (Connected) op('sidecar').par.Connected Toggle

Default:

False

Server URL (Serverurl) op('sidecar').par.Serverurl Str

Default:

"" (Empty String)

Dependencies Header

Depstatus (Depstatus) op('sidecar').par.Depstatus Str

Default:

"" (Empty String)

Check Dependencies (Checkdeps) op('sidecar').par.Checkdeps Pulse

Default:

False

Install Dependencies (Installdeps) op('sidecar').par.Installdeps Pulse

Default:

False

Version Header

VFS Version (Vfsversion) op('sidecar').par.Vfsversion Str

Default:

"" (Empty String)

Disk Version (Diskversion) op('sidecar').par.Diskversion Str

Default:

"" (Empty String)

Export Sidecar Files (Exportsidecar) op('sidecar').par.Exportsidecar Pulse

Default:

False

Ports

vision_sidecar port (Visionsidecarport) op('sidecar').par.Visionsidecarport Int

Default:

0

Range:

0 to 1

Slider Range:

0 to 1

embedding_sidecar port (Embeddingsidecarport) op('sidecar').par.Embeddingsidecarport Int

Default:

0

Range:

0 to 1

Slider Range:

0 to 1

Changelog

v1.1.02026-03-26

• Refactor monitoring from persistent async loop to one-shot health checks with delayMilliSeconds rescheduling - Add _monitoring_enabled bool mirror for safe async context reads - Add completion callbacks for stop and health check operations - Make _update_stats_table synchronous - Guard ChatTD/TDAsyncIO access with null checks
• Generic sidecar process manager with auto-discovery and health monitoring - Implements VFS export, dependency management, and process lifecycle - Added stats table and persistent monitoring loop

v1.0.02026-03-13

• remove all hardcoded ml_server references from button callbacks and internal helpers - start/stop/deps/install now iterate all registered sidecars generically - add _get_primary_sidecar() helper for single-sidecar UI display - add _update_connected_state() to derive Connected from any sidecar being connected - fix StartMonitoring to cancel any leftover task before starting a new one - simplify monitor loop to sleep check_interval per cycle instead of sub-polling at 0.5s
• Fix export to loop over all registered sidecars

v0.2.02026-03-01

• Rewrite SideCarExt as generic sidecar process manager - Add RegisterSidecar/EnsureSidecar/StartSidecar/StopSidecar API - Single monitoring loop for all registered sidecars - Install deps via uv sync instead of pip - Remove ML-specific methods (ProcessOCR, ProcessFlorence, LoadModel) - Legacy Startserver/Stopserver route to ml_server sidecar - sidecar_server.py: lazy imports for ML libraries - sidecar_server.py: dependency pre-check with console persistence on crash
• Replace __import__ dependency checks with importlib.metadata for TD 32050+ compatibility
• Initial commit

Related Operators

• Florence
• Ocr
• Acestep