Python Manager

Overview

The Python Manager LOP simplifies managing external Python libraries within TouchDesigner. It allows you to create isolated Python environments (virtual environments) for your projects, install packages using pip or from Git repositories, and run Python scripts or shell commands without interfering with TouchDesigner’s core Python installation.

This ensures that project-specific dependencies don’t cause conflicts across different TouchDesigner projects or with the base TouchDesigner Python environment.

Key Concepts

Virtual Environments (venv)

A virtual environment is an isolated directory containing a specific Python version and its own set of installed packages.

• Isolation: Packages installed in a venv are only available when that environment is active or added to the project’s path. This prevents conflicts between different projects that might require different versions of the same library.
• Project-Specific: You typically create one venv per project (or group of related projects) in a designated Base Folder.
• Activation: The Python Manager handles adding the venv’s site-packages directory (where libraries are installed) to your TouchDesigner project’s sys.path, making the installed libraries importable.

Python Version Requirement

Critical Requirement: Python 3.11

TouchDesigner currently uses Python 3.11. For the Python Manager to create compatible virtual environments, you MUST have an external Python 3.11 installation on your system. Using other Python versions (e.g., 3.9, 3.10, 3.12) to create the virtual environment will likely cause errors when trying to use packages within TouchDesigner.

• Windows: Download and install Python 3.11.9 specifically from the official Python 3.11.9 download page.
• macOS: It’s recommended to use Homebrew to install Python 3.11 (brew install python@3.11).

The Python Manager will attempt to auto-detect a compatible installation, but you can manually specify the path if needed using the parameters on the Python Setup page.

Usage in LOPS

Many LOPS operators (like Florence, OCR, RAG Index, etc.) rely on external Python libraries that require specific versions or CUDA dependencies (e.g., PyTorch). These operators automatically use the Python Manager instance located within the ChatTD LOP (/tox/ChatTD/python_manager) to manage these dependencies.

You typically do not need to interact directly with the Python Manager for these LOPS dependencies. Here’s how it works:

1. Requirement Check: When a LOP initializes or runs, it checks if its required Python libraries (like torch, transformers) are available in the environment managed by ChatTD’s Python Manager.
2. Installation Prompt: If requirements are missing, the LOP will trigger the Python Manager to display a pop-up dialog. This dialog will inform you which packages need to be installed (e.g., “PyTorch 2.1.1 with CUDA 11.8/12.1 support”).
3. User Confirmation: You will be asked to confirm the installation or select options (like the CUDA version for PyTorch).
4. Managed Installation: Upon confirmation, the Python Manager handles the download and installation process within its designated virtual environment. TouchDesigner might pause during this process.
5. Completion: Once installed, the LOP can import and use the libraries.

This automated process ensures that complex dependencies like PyTorch (with specific CUDA versions compatible with TouchDesigner’s Python version) are installed correctly without manual intervention via the Python Manager interface.

Basic Workflow (Standalone Usage)

While LOPS manage their own dependencies, you can also use the Python Manager standalone to manage libraries for your custom tools or general project needs. The video tutorial above provides a great walkthrough. The basic steps are:

1. Select Python: Ensure the correct Python 3.11 executable is selected on the Python Setup page (usually auto-detected).
2. Set Base Folder: Choose or create a folder where the virtual environment will reside using the Base Folder parameter.
3. Create venv: Pulse the Create venv parameter. A command window might briefly appear.
4. Add to Path: Pulse Add venv to sys.path to make the environment’s packages available to your current TD project. Use Auto Add onStart or Auto Add onCreate for convenience in the future.
5. Install Packages: Use the Pip or Git pages to install the libraries you need into the created virtual environment.

Parameter Reference

Parameters are organized into pages mirroring the operator’s interface.

Status (Status) op('python_manager').par.Status Str

Default:

None

Select Python Installation Header

Set Python Executable (Setpython) op('python_manager').par.Setpython Pulse

Default:

None

Python Executable (Pythonexe) op('python_manager').par.Pythonexe File

Default:

None

Use Custom Python (Custompython) op('python_manager').par.Custompython Toggle

Default:

on

Python Executable Menu (Pythonexemenu) op('python_manager').par.Pythonexemenu Menu

Default:

None

Virtual Environment Create / Manage Header

Create venv (Createvenv) op('python_manager').par.Createvenv Pulse

Default:

None

Base Folder (Basefolder) op('python_manager').par.Basefolder Folder

Default:

None

Open venv (Openvenv) op('python_manager').par.Openvenv Pulse

Default:

None

Import Python Path To Project Header

Add venv to sys.path (Addtosyspath) op('python_manager').par.Addtosyspath Pulse

Default:

None

Auto Add onStart (Addonstart) op('python_manager').par.Addonstart Toggle

Default:

on

Auto Add onCreate (Addoncreate) op('python_manager').par.Addoncreate Toggle

Default:

on

Print venv Report (Printpythoninfo) op('python_manager').par.Printpythoninfo Pulse

Default:

None

Print Project sys.path (Printsyspath) op('python_manager').par.Printsyspath Pulse

Default:

None

Pip Package Manager Header

Python Package / Library (Pippackage) op('python_manager').par.Pippackage StrMenu

Default:

None

Install (Pipinstall) op('python_manager').par.Pipinstall Pulse

Default:

None

Update (Pipupdate) op('python_manager').par.Pipupdate Pulse

Default:

None

Uninstall (Pipuninstall) op('python_manager').par.Pipuninstall Pulse

Default:

None

Install from List [ in1 ] (Pipinstallfromlist) op('python_manager').par.Pipinstallfromlist Pulse

Default:

None

List Current Packages (Listinstalledpackages) op('python_manager').par.Listinstalledpackages Pulse

Default:

None

Current Packages (Currentpackages) op('python_manager').par.Currentpackages Str

Default:

0 installed packages

Create requirements.txt (Createreqtxt) op('python_manager').par.Createreqtxt Pulse

Default:

None

Upgrade pip (Upgradepip) op('python_manager').par.Upgradepip Pulse

Default:

None

Git Repository Download / Install Header

Git Folder (Gitfolder) op('python_manager').par.Gitfolder Folder

Default:

None

Clone Git Repo (Clonegitrepo) op('python_manager').par.Clonegitrepo Pulse

Default:

None

GitHub Repo URL (Gitrepourl) op('python_manager').par.Gitrepourl Str

Default:

None

Install After Clone (Installafterclone) op('python_manager').par.Installafterclone Toggle

Default:

off

Install Method (Installmethod) op('python_manager').par.Installmethod StrMenu

Default:

pip install .

Console Settings + Commands Header

Show cmd / terminal (Showcmdwindow) op('python_manager').par.Showcmdwindow Toggle

Default:

on

Block TD Thread (Blockthread) op('python_manager').par.Blockthread Toggle

Default:

on

Run External Commands [ in venv ] Header

Run Python Script (Runpythonscript) op('python_manager').par.Runpythonscript Pulse

Default:

None

Script (Pyscript) op('python_manager').par.Pyscript Str

Default:

None

Run Shell Command (Runshellcommand) op('python_manager').par.Runshellcommand Pulse

Default:

None

Command (Shellcmd) op('python_manager').par.Shellcmd Str

Default:

None

Txheader (Txheader) op('python_manager').par.Txheader Str

Default:

python manager - TD Tool by Dotsimulate

Version (Txversion) op('python_manager').par.Txversion Str

Default:

1.1.0

Date (Txdate) op('python_manager').par.Txdate Str

Default:

11/06/2024

About Me (Txopenaboutme) op('python_manager').par.Txopenaboutme Pulse

Default:

None

View readme (Viewreadme) op('python_manager').par.Viewreadme Pulse

Default:

None

Op Display (Opdisplay) op('python_manager').par.Opdisplay Menu

Default:

Packages

Options:

Packages, Logs

Clear Log On Start (Clearlogonstart) op('python_manager').par.Clearlogonstart Toggle

Default:

on

Pop-Up Help (Popups) op('python_manager').par.Popups Toggle

Default:

off

Show Built in Pars (Showbuiltin) op('python_manager').par.Showbuiltin Toggle

Default:

off

Technical Notes

• Python 3.11: Requires an external installation of Python 3.11.x.
• git Requirement: Cloning Git repositories requires git to be installed on your system and accessible via the command line (in the system’s PATH).
• sys.path: Adding the venv to sys.path only affects the current TouchDesigner process. Use the Auto Add onStart parameter for persistence across project restarts.
• Background Tasks: Be mindful when disabling Block TD Thread. While it keeps TD responsive, subsequent operations that depend on the background task completing might fail if initiated too early.

Related Operators

• ChatTD: Often contains the primary Python Manager instance used by other LOPS.