How it works

Architecture

The whole system on one page: layers, data flow, the models we use and why, the pipeline, the stack and the API.

System layers

Frontend

Svelte 5Electron

Operator interface: live POV, overlays, spatial 3D view, roster and analytics.

Backend

PythonThreads

Orchestrates capture, analysis and persistence.

Inference Engine

PyTorchCUDA

Runs the model catalogue with FP16 acceleration.

Model Pipeline

Plugins

Composes models into the 13-stage processing flow.

Storage

SQLite

Local persistence for sources, events, identities and settings.

API Layer

FastAPIWebSocket

Streams live data and exposes REST endpoints.

Visualization Layer

three.jsWebGL

Renders overlays and the 3D reconstruction.

Export Layer

ClipsScenes

Turns analysis into shareable artifacts.

Data flow

How a frame becomes an event.

Frame to event

  Camera / RTSP / File / Image
            |
     StreamReader (thread)
            v
     FrameBuffer  (drop-oldest, bounded)
            v
   AnalysisWorker (thread)  ──►  Model Pipeline
            |                    ├─ YOLO11 + ByteTrack   (detection / tracking)
            |                    ├─ Depth Anything V2     (depth)
            |                    ├─ Seg / Pose / Re-ID    (masks / gait / identity)
            |                    └─ Geometry + SR         (3D / reconstruction)
            v
     AnalysisResult ──► EventBus ──┬─► Recorder (clips)
            |                      ├─► Alert + Threat engine
            |                      ├─► SQLite (events, subjects, settings)
            v                      └─► FastAPI
     WebSocket stream ─────────────────► Frontend (overlays + 3D viewport)

Models & why

A focused catalogue. Each row links to its full spec sheet.

Model	Role	Why we use it
YOLO11	Object Detection	Locate and classify people, vehicles, animals and hazards on every analysed frame.
ByteTrack	Multi-Object Tracking	Assign persistent identities to detections across frames, surviving brief occlusions.
Depth Anything V2	Monocular Depth	Estimate a dense depth field from a single ordinary camera, with no stereo rig.
Real-ESRGAN	Super-Resolution	Reconstruct a sharper, super-resolved image of a face or licence plate from many sightings.
YOLO11-seg	Segmentation	Produce per-pixel masks that separate standing objects from the background plate.
Pose Estimator	Body Keypoints	Extract body keypoints to support intent, posture cues and gait sampling.
ReID + Gait	Re-identification	Re-identify subjects across cameras and time to build long-term dossiers.
ViT Vehicle Classifier	Vehicle Attributes	Estimate vehicle make and support plate / speed attributes for tracked vehicles.

Pipeline

Every frame flows through thirteen stages.

Input→Frame Capture→Preprocessing→Detection→Segmentation→Depth Estimation→Pose Estimation→Geometry Fusion→3D Reconstruction→Mesh Optimization→Texture Mapping→Spatial Analysis→Output

#	Stage	Does
01	Input	Acquire a source: live camera, RTSP/ONVIF stream, recorded video or a single image.
02	Frame Capture	Pull frames into a bounded buffer with backpressure.
03	Preprocessing	Normalise, low-light enhance and resize before inference.
04	Detection	Detect and classify objects with YOLO11 + class gating.
05	Segmentation	Mask standing objects vs the static background plate.
06	Depth Estimation	Estimate a dense monocular depth field.
07	Pose Estimation	Extract body keypoints for intent and gait.
08	Geometry Fusion	Fuse depth, masks and detections into scene geometry.
09	3D Reconstruction	Back-project depth into a coloured point cloud.
10	Mesh Optimization	Triangulate and clean the surface.
11	Texture Mapping	Project RGB (and inpainted texture) onto the mesh.
12	Spatial Analysis	Reason over the scene: tracks, foresight, identity, alerts.
13	Output	Stream results and export scenes, clips and events.