Under the hood

Technology

The stack behind Overseer, in seven areas.

Computer Vision

Detection, tracking, segmentation and re-identification that turn pixels into structured entities.

Component	Role	Notes
YOLO11	Object detection	Ultralytics YOLO11 (n/s tiers) with per-class confidence floors, tiled inference for small objects, and detector-level class gating.
ByteTrack	Multi-object tracking	Associates high- and low-score boxes for stable ids through occlusion, with a coasting layer for momentary drops.
YOLO11-seg	Segmentation	Instance / foreground masks that separate movers from the static plate for scene completion.
MOG2	Motion detection	Background-subtraction motion mask feeding motion-percentage, recording triggers and the heatmap.
Appearance ReID + Gait	Re-identification	Embedding-based appearance matching plus z-scored gait descriptors and soft biometrics for cross-camera identity.

Machine Learning

The model runtime, from monocular depth to learned super-resolution and vehicle attributes.

Component	Role	Notes
Depth Anything V2	Monocular depth	Relative inverse-depth per frame, temporally median-fused; the backbone of the spatial pipeline.
Real-ESRGAN	Super-resolution	Vendored SRVGGNetCompact (realesr-general-x4v3) 4x upscaler for reconstructing blurry faces and plates.
ViT classifier	Vehicle attributes	A vision transformer estimates vehicle make, confidence-gated and voted across frames.
Pose estimation	Keypoints	Body keypoints for intent, fall / posture cues and gait sampling.
PyTorch + Ultralytics	Runtime	Torch 2.x with CUDA; models are vendored so weights load with no package sprawl.

Geometry Processing

Turning depth fields into clean, navigable geometry.

Component	Role	Notes
Pinhole back-projection	Lift to 3D	Depth grid unprojected through a pinhole model into a coloured point cloud.
Depth smoothing	Continuity	Ramps depth jumps so surfaces stay continuous; straddling triangles across discontinuities are culled.
Ground-plane fitting	Structure	Fits disp = a + b·y + c·x behind occluders for structure-aware background depth.
ECC alignment + median fusion	Reconstruction	Sub-pixel frame alignment then median fusion for multi-frame face / plate reconstruction.
TELEA inpainting	Completion	Fills depth and texture behind removed foreground objects for a continuous background layer.

Spatial Computing

Reasoning about position, motion and identity across space and time.

Component	Role	Notes
Foresight engine	Prediction	Per-track ground-plane velocity and short-horizon position prediction; flags converging paths.
Tactical god-view	Top-down radar	Inverse-perspective projection places contacts on a bird's-eye scope using real depth + FOV.
Ego-motion compensation	Camera pose	Per-frame flow model recovers camera motion so object speeds are ground-relative.
Cross-camera Re-ID	Identity	Embeddings + gait + soft biometrics link subjects across cameras into long-term dossiers.

Rendering

How scenes, overlays and 3D reconstructions reach the screen.

Component	Role	Notes
three.js	3D viewport	Renders the point cloud / mesh scene with a fly-through camera, fog and depth-tinted sky.
WebGL feed layer	Live overlay	GPU-composited camera feed under the detection, ghost and heatmap overlays.
SVG / Canvas HUD	Overlays	Detection reticles, the tactical radar and density heatmaps drawn as lightweight vector / canvas layers.
Svelte 5 + Electron	Shell	A runes-based Svelte UI in an Electron desktop shell, or any modern browser against the backend.

Data Processing

The backend that moves frames, events and identities.

Component	Role	Notes
FastAPI bridge	API + WS	An async FastAPI server streams frames, detections, metrics and alerts over WebSocket and exposes REST.
Threaded capture	Ingest	StreamReader → drop-oldest FrameBuffer → AnalysisWorker, with results marshalled onto the event loop.
SQLite store	Persistence	Sources, events, alerts, subjects, dossiers and settings persist locally; no external database.
Event bus	Fan-out	A pub/sub bus fans analysis results to recording, alerting, the timeline and the API layer.

Performance Optimizations

Where the latency budget is spent, and how it is kept low.

Component	Role	Notes
FP16 on CUDA	Throughput	Half-precision inference for detection, depth and super-resolution when a GPU is present.
Frame interval	Adaptive load	Heavy passes run every N frames; results are cached and coasted between runs.
Class gating	Load shedding	Disabled detection classes are dropped at the detector, freeing tracking, Re-ID and analytics too.
Low-res fuse, hi-res finish	Reconstruction	Alignment and fusion run at capped resolution, then a single super-resolution pass upscales.
Bounded buffers	Backpressure	Drop-oldest frame buffers and rolling windows keep memory flat under load.

Deep dive

Each model is specced individually (inputs, outputs, latency, hardware, benchmarks) on the Models page, and the end-to-end flow is on the Pipeline and Architecture pages.