MCP Server

Helios includes an optional Model Context Protocol (MCP) server for compatible AI clients and local automation tools. It can inspect status, configure supported plugins, select scripts and setups, and help diagnose a workflow. Ordinary Helios use does not require MCP. The server does not include an AI model or chat interface; choose and configure a client separately.

Connect a Client

Configure the client to launch a stdio server with:

Command: C:\Helios\lib\HeliosMCP.exe
Arguments: --mcp

Replace C:\Helios with the actual installation folder. Use the client’s command and arguments fields; this is a local executable connection, not a web address. No separate Python or Node.js installation is needed for the MCP server.

Run the client under the same Windows account as Helios. Call helios_launch to start the matching installation or connect to its running instance. If multiple instances are running, add --helios-pid <PID> to the startup arguments to choose the intended Helios application process. Update an explicit PID when that process restarts.

If your client uses an older root-level or versions\<release>\lib\HeliosMCP.exe path, change it once to the installation’s lib\HeliosMCP.exe and reconnect. Keep this stable path for future updates. Clients supporting tool-list change notifications refresh the available tools automatically; otherwise refresh discovery or reconnect after an update.

Closing the MCP connection does not close Helios or stop its tools. Closing Helios leaves the MCP connection available; call helios_launch to reopen it. A client pinned to an explicit PID must be reconfigured when that process restarts.

Discover Before Acting

For a plugin call, supply the discovered values in this shape:

{
  "plugin": "<plugin ID from helios_plugins>",
  "command": "<command from helios_describe_plugin>",
  "arguments": {}
}

Fill arguments according to the command’s live schema. Available actions depend on the installed release, selected device, account access, and current plugin state. Loading a plugin does not necessarily start capture, run its script, or connect its device.

Common Workflows

Use the live tool catalog for complete argument definitions. For GPC 3, target an existing slot from the reported slot_count; adding another slot is an explicit action. Titan Bridge has three separate device slots. Supported output plugins also advertise bounded input actions; use them only for an intended test and inspect their final status before resuming normal input.

Titan Bridge direct-input actions require a connected, ready bridge; manual KMG Capture need not be running. For observations, controller.output describes Helios’s intended output, while titan.controller.report reads the Titan device report when available. A completed send does not prove that a console or target application accepted it.

Controller Tests and Calibration

helios_controller can read controller state or submit a bounded action. Choose raw_input to combine a test with physical input before GPC 3, or output plus an exact plugin to test that destination directly. A neutral test does not release a physically held control. Discover the destination’s readiness and enabled channels first. Sequences accept at most 32 steps and 10 seconds total; omitted controls are neutral. Inspect the returned action’s status and release, and cancel only that action if needed.

helios_control_capture combines bounded input with fresh observations before, during, or after it. Start with the live schema, then poll action: "status" using the returned job_id. It accepts up to eight captures; a status response returns one selected image. Completion requires phase: "complete" and input.restored: true. Inspect timestamps rather than assuming a captured frame represents a particular game response. Retrieve the desired observations before starting another job.

helios_calibration offers catalog, start, status, and cancel for supported measurements. For look_latency, start capture and a ready output, face a stationary textured scene, release controls, and request sample_count from 1 through 32. It temporarily takes control of the participating outputs; optional plugin selects just one. Poll until busy: false, phase: "complete", and restored: true. Partial samples in a failed or cancelled job are not a completed result. See Calibration for axis, held-control, and measurement guidance.

Frame Observations

For snapshots, recordings, and combined control/capture jobs, use raw only when no CV script is running, and processed while a CV script is running. Changing that state during capture ends the job. For a preview without Helios overlays, save the current disable_overlay_drawing preference, enable it, then restore the previous value after capture finishes. Pixel edits made by the script can still be present.

Workspace and Preferences

helios_layouts reports exact names, workspace_mode, Activity Bar state, and setup_locked. Named layout selection and editing require the Activity Bar to be enabled. Turning it off through helios_set_activity_bar switches to the all-panel workspace; if it would reveal hidden panels, inspect confirmation_required and supply confirm_show_all: true only when that change is intended.

Setup locking prevents setup changes, plugin loading/unloading, and workspace or run-group edits. Existing named layouts and configured run groups remain usable. Wait for pending operations to finish before changing the lock.

Preferences support the advertised GPU, Python environment, capture audio, Fuser display, overlay, keep-awake, and experimental enhanced-inference controls. The tensorrt_readiness_enabled preference requires an available NVIDIA TensorRT GPU and applies on the next CV start. Sign-in, account changes, credentials, and Developer Mode remain manual. Run groups accept up to six eligible plugins; inspect each member’s result because an accepted group action does not mean every member finished successfully.

Dataset Capture and Engine Builder

Both tools use an action and an arguments object. Begin with this request to the selected tool:

{"action": "describe", "arguments": {}}

For Dataset Capture, use an existing absolute output directory. MCP configuration requires capture to be disarmed and inference engines disconnected; check the reported availability before changing settings. Starting requires duration_ms from 1 through 600000 and at least one trigger. It does not start a video source or inference script. Inspect the session and saved files after stopping. Use helios_capture_frames instead when you need a lossless .hframes recording.

Engine Builder offers refresh, import, remove_source, probe, build, rebuild, and cancel. Wait for runtime discovery, probe a listed source before choosing dimensions, and poll describe for the operation’s terminal response. Build requires a compatible NVIDIA/TensorRT environment; protected models still require account access. Cancel with the reported operation_id and keep checking while the result says cancelling. MCP operations have a 30-minute limit. Closing the MCP connection leaves an accepted build running.

Use import for an absolute path to a nonempty, self-contained plain .onnx file with an explicit task. Import creates a new local source entry and does not publish it. The live schema includes optional model-family, class-count, priority, and class-recommendation fields; see Model Compatibility. Supply either class_priority or class_recommendations, not both.

remove_source removes the exact listed source and its metadata while retaining built engines. Rebuilding then requires the original source again. Inspect the final result, including source_removed, before retrying a partially failed removal. Cancellation applies to probe/build/rebuild; an accepted import finishes normally.

Python Packages and Saved Selections

Online Resources advertises environment.list, environment.packages, environment.packages.install, environment.packages.remove, and environment.packages.upgrade. Package inspection and changes can be asynchronous: poll the plugin’s latest operation. Install takes requirement specifications; remove and upgrade take package names. Use the live schema for the exact environment ID and fields.

Stop CV Python, CV C++, and Engine Builder work using the target environment before modifying packages, including any pending start or stop. Other unused environments remain manageable. Package operations do not select the interpreter in Preferences or replace its requirements file. If an operation times out or partly succeeds, inspect the environment before retrying.

CV Python and CV C++ advertise remove_script for saved dropdown entries; Dataset Playback advertises remove_folder. These remove history entries without deleting the underlying files or folders. The bundled CVTest entries are retained. Input Translator’s document.open accepts .ipt and .git; document.save flushes pending edits.

Titan Bridge’s configuration.read and configuration.set use its ordinary device configuration controls. Wait for completion and readback_confirmed: true; inspect the actual settings after a failure rather than assuming nothing changed.

Publish Online Resources

Enable Developer Mode and link Discord and a verified email in Helios first. The normal publishing limits and review requirements apply.

Inspect a local file through helios_call_plugin:

{
  "plugin": "Online Resources",
  "command": "resource.upload.inspect",
  "arguments": {"file_path": "C:/MyScripts/color_indicator.py"}
}

Inspection reports whether the file would create a resource or update one of your existing resources, plus its editable metadata. It does not upload the file. Review the result, then publish explicitly:

{
  "plugin": "Online Resources",
  "command": "resource.upload",
  "arguments": {
    "file_path": "C:/MyScripts/color_indicator.py",
    "metadata": {
      "name": "Color Indicator",
      "description": "Highlights a configured color region in the video preview.",
      "instructions": "Select this script in CV Python.",
      "tags": ["vision"],
      "version": 1.0,
      "visibility_type": "public"
    }
  }
}

Existing-resource matching uses your account and the full filename. Supplied metadata updates those fields; omitted metadata is retained for an existing resource. The live schema also describes optional image_file, controller-source compilation, and model task/class recommendations. For segmentation class recommendations, mask_anchor accepts "bbox" or "mask", mask_color uses #RRGGBB, mask_opacity accepts 0..1, mask_threshold is strictly between 0 and 1, and draw_mask is boolean. Metadata opacity uses 0..1; the Python and C++ drawing setters use 0..255. Follow the live publishing schema for the complete class row. Publishing does not run the file.

If the response is review_required, nothing has been submitted by that request. Add allow_review: true only when you intend to submit for manual review. Poll helios_describe_plugin for the upload operation and check its final resource ID and approval status; uploaded content may still be awaiting approval. If request_may_have_executed is reported, inspect the existing resource before retrying.

Completion and Troubleshooting

• Pending is not finished. Inspect the returned operation or job until it completes or fails. A setup change is checked through helios_layouts; plugin operations through helios_describe_plugin; frame jobs through helios_frame_status; Dataset Capture and Engine Builder through their describe actions.
• Timeout or lost connection: inspect current state before retrying a change. The original request may already have taken effect.
• Server missing: install the current complete Helios package and check the installation’s lib\HeliosMCP.exe path used in the client configuration.
• Workspace edit rejected: check setup_locked and Activity Bar state. Unlock the setup for an intended edit; enable the Activity Bar to select or edit named layouts.
• Wrong or ambiguous instance: close an unwanted instance or configure the intended PID. An old PID must be updated after restarting Helios.
• Command unavailable: refresh the plugin description and satisfy its visible requirements. Sign-in, access, device readiness, and Developer Mode requirements still apply where required by the action.
• No video: start one capture source and check its Write FPS before requesting a frame.

Agent actions appear with the MCP source in Output Panel. Connect clients you trust, and review their requested actions: supported commands can change settings, run scripts, operate devices, or install content using your current Helios session.

See Also

• Workspace & Core Tools
• Script API Reference
• Output Panel