Python Inference

helios.inference gives CV Python scripts access to compatible models installed in Helios. Helios uses the Compute GPU selected in Preferences.

Start a Model

from helios import inference


class CVWorker:
    def __init__(self, width, height):
        models = inference.list_models()
        if not models:
            raise RuntimeError("No inference models are installed")

        self.engine = inference.create_inference_engine(models[0]["uuid"])
        self.engine.set_confidence_threshold(0.45)
        self.engine.set_nms_threshold(0.50)
        self.engine.start()

    def process(self, frame):
        result = self.engine.wait_for_results_raw()
        if result is None:
            return

        count = int(result["num_detections"])
        detections = result["detections"][:count]
        if count:
            first = detections[0]
            class_id = int(first["class_id"])
            confidence = float(first["confidence"])
            center_x = float(first["center_x"])
            center_y = float(first["center_y"])

Create the engine in __init__ and reuse it. wait_for_results_raw() waits indefinitely by default so the result stays paired with the current frame. Pass timeout_ms=0 or another non-negative value only when the script intentionally accepts abandoning that frame's result after the deadline.

Structured Result View

wait_for_results_raw() returns a zero-dimensional NumPy structured view with named fields:

Only detections[:num_detections] is valid. Detection, pose, and segmentation share one global limit of 64 published detections per frame, not 64 per class.

Each detection has these named fields:

apparent_distance is a dimensionless, size-derived relative distance: larger values represent a smaller, apparently farther detection. It is not a physical distance. Configure the expected height-to-width ratio globally or for selected classes:

self.engine.set_apparent_distance_aspect_ratio(
    expected_height_per_width=2.5,
    class_ids=None,
)

The default expected ratio is 2.5. Use model metadata to map class ids to labels.

The result and nested arrays are borrowed reusable views. Read or copy the values needed for the current callback and do not retain those views across later frames.

Segmentation Results

Segmentation models provide the structured detection result plus one binary mask per published detection. After a successful wait_for_results_raw(), call:

mask_result = self.engine.segmentation_results_raw()
if mask_result is not None:
    (
        mask_frame_sequence,
        mask_format,
        mask_count,
        mask_width,
        mask_height,
        mask_row_bytes,
        mask_plane_stride,
        source_width,
        source_height,
        transform,
        masks,
    ) = mask_result

Mask i corresponds to detection i. masks is a read-only borrowed view containing mask_count packed mask planes. Within a plane, row y starts at y * mask_row_bytes; pixel x is bit x & 7 of byte x >> 3. Use mask_plane_stride to move between masks. The transform tuple maps mask coordinates to the source frame. Consume the view immediately and do not retain it across frames.

Model and Session Functions

The session exposes model_type, model_info, loaded_model_description, task, and is_paused. It supports load_model(uuid), unload_model(), pause(), resume(), stop(), and destroy().

One session holds one object model. Create additional sessions for additional models and account for the added CPU/GPU load. Helios supports up to 30 simultaneously started inference sessions in total, and each session supports up to 16 OCR regions. Calling pause() or resume() during process() changes inference beginning with the next frame; a wait during that transition returns immediately.

Every model must declare its task exactly as detect, pose, or segment.

Constants

Use the named constants rather than relying on their numeric values:

Filtering and ROI

Normalized positions use 0.0..1.0. New sessions begin with the model's recommended ROI and class priority when that metadata exists. Explicit ROI or priority calls replace the recommendation until its corresponding set_use_recommended_...(True) call is used again.

Search Activation

Use set_search_buttons(...) to run inference only for configured controller conditions. It accepts simple button indices, (button_index, state) conditions, or up to eight OR clauses. State 1 means pressed and 0 means released.

Use a stick-magnitude trigger with:

self.engine.set_search_stick(1, radius=25)                  # magnitude above 25
self.engine.set_search_stick(2, radius=25, less_than=True)  # magnitude below 25
self.engine.set_search_stick(None)                          # disable stick trigger

stick is 1 or 2; radius is required and accepts 0..100. Search buttons and the stick are alternatives, so either can activate inference. With neither configured, inference runs on every frame.

Drawing

set_draw_detections(True) enables boxes. Related settings include box thickness, confidence text, ROI drawing, and BGR colors. Pose sessions also provide keypoint and skeleton controls. Drawing is independent of returned results: hiding a box does not remove its result.

Segmentation masks draw by default. Use set_draw_segmentation_masks(enabled, class_ids=None), set_segmentation_mask_opacity(0..255), and set_color_segmentation_mask(b, g, r, class_ids=None) to control their display without changing raw masks.

Apparent-distance text is off by default:

self.engine.set_draw_apparent_distance(True)

It draws the rounded dimensionless apparent_distance value beside each detection.

Priorities, Anchors, and Optional Prediction

Use recommended class priorities unless the script needs a different ordering. Only the highest-priority tier represented among retained candidates is published; lower tiers do not fill unused slots. Classes grouped in the winning tier have equal priority and share the 64-result capacity.

Static and pose-derived anchor calls, polar-origin settings, ignore regions, and target-delay settings adjust which result geometry is returned. Optional anchor prediction is configured through set_anchor_prediction(...); its output and status are reported through the named prediction fields. Use it only when the script can provide reliable, correctly timestamped measurements. The full call reference is included under the installed SDK's docs directory.

For text recognition, see the OCR reference.

Troubleshooting

• Confirm the selected model appears in list_models() and declares detect, pose, or segment.
• Confirm the Compute GPU is available in Helios Preferences.
• Start the session before waiting for results and keep engine creation out of process(frame).
• Read only detections[:num_detections] and do not retain borrowed result or mask views.
• A hard runtime failure raises inference.InferenceCoreError; do not treat it as an ordinary zero-detection frame.
• For INSTALLATION_INVALID (error code 18), install the latest complete official Helios release and check whether security software quarantined a packaged file.
• If stick-gated inference never runs, confirm the stick is 1 or 2 and its radius is 0..100.
• Report a persistent setup failure once instead of printing it every frame.
• Check the Output Panel for model or GPU errors.