02 · Architecture

Technical blueprint

Lightweight modular Vision-Language-Action stack. Key principle: modularity + persistent world state. Components are specialized, loosely coupled, and communicate primarily through the World Model.

High-level data flow

User Instruction → Language Reasoner → Task Planner → World Model
Vision Encoder + Sensors → Scene Encoder → World Model
World Model + Subtask → Action Policy → Motion Controller → Robot
Sensors + Vision feedback → World Model

Modules

schemasWorld / task / action models
vision · scene · worldPerceive + persistent state
language · plannerIntent + subtasks
policy · control · robotsAct + safety + adapters
loop · factoryWire + run
ui · demos · benchCLI + demos + benchmarks

Design principles

  1. 01World Model is the integration bus
  2. 02Language never emits joint commands
  3. 03Safety lives in the motion controller
  4. 04LLM / planner run on events, not every tick

Core components

01

Vision Encoder

kinetic.vision

Compact visual features from RGB (optional depth). Default: MockVisionEncoder (block pooling + change detection). Production targets: FastViT, MobileViT, EfficientViT, DINOv2/v3 Tiny, SigLIP.

02

Scene Encoder

kinetic.scene

Structured symbolic scene — detections, 6D poses, attributes, scene graph edges — not raw tokens for the LLM. Default: MockSceneEncoder + rule-based build_scene_graph.

03

World Model

kinetic.world

Central persistent memory: objects, robot proprioception, scene graph, task progress, short/long-term memory. Event-driven, delta-aware updates; optional SQLite persistence; compact summary_for_llm().

04

Language Reasoner

kinetic.language

Pure intent — no action generation. Input: user command + world summary → GoalSpec. Default RuleBasedReasoner; BitNetReasoner via bitnet.cpp; optional HttpLLMReasoner. Event-only — never inside the control cycle.

05

Task Planner

kinetic.planner

Hierarchical decomposition: Goal → ordered Subtask list with pre/postconditions. Default HierarchicalPlanner (fetch / place / navigate templates).

06

Action Policy

kinetic.policy

World state + current subtask → ActionCommand (EE pose, joints, gripper, stop). Default HeuristicPolicy; production path: Diffusion Policy / small transformer / BC.

07

Motion Controller

kinetic.control

Safety filter, rate limits, joint clamps, pseudo-IK trajectory. Production: MoveIt / full IK, collision checking, force limits.

08

Robot Adapter

kinetic.robots

Generic trajectory → robot API. Default MockSimRobot; adapters for ROS2, Franka, UR5 and more via robots.base.RobotAdapter.

Event-driven loop kinetic.loop

Efficiency: planner/LLM not every frame; only changed objects matter.

  1. 01New instruction or sensor tick
  2. 02Vision → Scene → World Model (delta)
  3. 03If task-relevant: Reasoner + Planner
  4. 04Policy selects primitive
  5. 05Controller → Robot
  6. 06Feedback → World Model
  7. 07Repeat

System diagram

                 User Instruction
                        │
                        ▼
               Language Reasoner
                        │ Goal
                        ▼
                  Task Planner
                        │ Subtasks
                        ▼
 ┌────────────── World Model ◄──────── feedback ──────────┐
 │                     │                                    │
 │            Vision + Scene Encoder                        │
 │                     │                                    │
 │              Action Policy                               │
 │                     │                                    │
 │            Motion Controller                             │
 │                     │                                    │
 └──────────────────► Robot ────────────────────────────────┘

This page is the architecture reference for Kinetic on the DifrasAI site.