Skip to main content

Inference Workflow

If you are new to SiMa NEAT, keep this sequence in mind:

  1. Load a compiled model package (.tar.gz) with Model.
  2. Compose a Session with input nodes, model stages, and output nodes.
  3. Build a Run in sync or async mode.
  4. Push inputs and pull outputs as Tensor or Sample.

Synchronous inference snippet​

Sync: use run(...) or push_and_pull(...) for request/response style execution.

simaai::neat::Model model("resnet_50_model.tar.gz");
simaai::neat::Session session;
session.add(model.session());

cv::Mat img = /* your frame (RGB/BGR as configured) */;
auto run = session.build(img, simaai::neat::RunMode::Sync);
auto out = run.push_and_pull(img, /*timeout_ms=*/1000);

Asynchronous inference snippet​

Use async mode when you want to decouple producers and consumers, control queueing, or overlap IO and compute. Async: use push(...) / pull(...) with RunOptions to tune queueing and drop behavior.

simaai::neat::Model model("resnet_50_model.tar.gz");
simaai::neat::Session session;
session.add(model.session());

cv::Mat img = /* your frame */;

simaai::neat::RunOptions opt;
opt.queue_depth = 8;
opt.overflow_policy = simaai::neat::OverflowPolicy::Block;
opt.enable_metrics = true;

auto run = session.build(img, simaai::neat::RunMode::Async, opt);
run.push(img);
auto out = run.pull(/*timeout_ms=*/1000);

Learn the concepts​

  • Model: model pack loading and model-driven pipeline fragments.
  • Session: assembly, validation, and run/build entry point.
  • Node: atomic pipeline building block and composition unit.
  • Pipeline: deterministic node composition and execution handles.
  • Graph: hybrid DAG runtime for pipeline + stage composition.
  • Tensor and Sample: payload vs metadata envelope.
  • Input and Output: sources, sinks, groups, and I/O contracts.

Tutorials​