03 · Backends

Configs & adapters

Swap sim, robots, vision, IK, policy, and language backends via YAML — without rewriting the control loop.

Configs

configs/default.yamlMock sim (CI / fast)
configs/mujoco.yamlPhysics + viewer
configs/bitnet.yamlOptional 1.58-bit reasoner

Robot backends

mock_simDefault lightweight arm
mujocoTabletop physics + cameras
ros2Joint trajectory (dry-run by default)

New arms: implement robots.base.RobotAdapter.

Optional backends

control.ik_backend: dls

Jacobian damped-least-squares IK

scene.backend: yolo

Pretrained detector (pip install ultralytics)

policy.backend: lerobot

Run pretrained HF VLA checkpoints — SmolVLA, ACT — through the loop (pip install "lerobot[smolvla]"; CPU works: ~27s per 50-action chunk)

BitNet (optional LLM)

Use Microsoft bitnet.cpp as Kinetic's Language Reasoner for 1.58-bit LLM inference on CPU (GPU kernels available upstream). Control / policy / vision stay separate — BitNet only emits intent JSON.

1.58-bit weightsTernary BitNet; official lossless kernels
CPU speedupsUpstream reports ~1.4–6×; large energy savings
Event-onlyBitNet runs on new instruction / replan, never in the control loop
Compact contextPrompt = system rules + WorldState.summary_for_llm()
FallbackRule reasoner if BitNet is missing

Architecture placement

User instruction
      │
      ▼  (event only)
 BitNet b1.58  (run_inference.py)  →  GoalSpec JSON
      │
      ▼
 Task Planner → World Model → Policy → Controller → Robot

Never call BitNet every control tick.

Config sketch

language:
  backend: bitnet
  bitnet:
    repo_dir: F:/BitNet
    model_path: F:/BitNet/models/.../ggml-model-i2_s.gguf
    threads: 8
    n_predict: 128
    temperature: 0.1
    strict: false

One-shot setup

# Prefer VS2022 Developer PowerShell (clang + MSVC toolset)
cd F:\Kinetic
powershell -ExecutionPolicy Bypass -File scripts\setup_bitnet_light.ps1

This page covers optional backends for Kinetic (YOLO, BitNet, LeRobot, ROS2). Details match the in-repo BitNet setup notes.