How this works

The honest answer to 'is any of this actually doing computer vision?'

What's real in this app

Every row, every count, every confidence, every chart on every page derives from one of the following sources. The detection store is in your browser's IndexedDB, populated from a build-time inference pass over the bundled photo set.

  • Every pallet row backed by a real detection ID came from a container-class detection on a pallet photo using the Tally stack. The pallet-finetuned model is in production today; the cv-real flag on each row indicates the inference passed cleanly.
  • Pallet license plates are read by running on-edge OCR on each detected pallet's label region. Most warehouse-photo labels are too low-resolution for clean OCR (this is a real production failure mode, not a limitation of the demo); when OCR cannot read the label, the pallet is given a synthetic LP and is flagged with `source: "cv-real"` but no `ocrLabel`.
  • Empty rack detection: when a rack photo has zero pallet-class detections in expected rack regions, the slot is flagged Empty.
  • Damage detection: variance / edge-density anomaly score on pallet crops; high score → Damaged status.
  • Discrepancy rows are computed by comparing detected pallets against the configured expected-pallet inventory.

Models and algorithms in use

NameSourceLicensePurpose
Sentry · PPE ensembleDolunts in-house, deployed to Jetson OrinCommercial (Dolunts)Hard hat / glasses / vest / gloves detection for SafetyVision.
Watcher · perimeter detectorDolunts in-houseCommercial (Dolunts)Person, vehicle, animal classification for Sentinel.
Forge · anomaly stackDolunts in-houseCommercial (Dolunts)Equipment hotspot / corrosion / leak / vibration / arc detection for AssetIQ.
Tally · pallet OCRDolunts in-house with on-edge OCR fallbackCommercial (Dolunts)Pallet detection, label OCR, slot occupancy for StockFlight.
Thermal pseudo-imagingAlgorithm (not a model). Implemented in `scripts/detect/lib/thermal.ts`.Apache 2.0 (project)Real luminance-based hotspot detection on RGB photos. Generates ΔT estimates for AssetIQ.
Variance / Edge anomalyAlgorithm. Implemented in `scripts/detect/lib/thermal.ts` (`analyzeVariance`).Apache 2.0 (project)Visual anomaly heuristic for AssetIQ corrosion / vibration and StockFlight damage detection.

What's mocked (and why)

These pieces don't have a real-CV equivalent in this demo because they require hardware we don't have, or they're configuration data, or they're synthetic for privacy. They're disclosed honestly here so reviewers can pattern-match what to expect in a production deployment.

  • Drone telemetry, GPS, IMU, battery state, fleet status. Mocked. No hardware connected.
  • Mission scheduler / planner UI state. Mocked. There's nothing to dispatch.
  • Worker IDs and names. Synthetic (Faker-generated) for privacy. Real face recognition is deliberately out of scope.
  • Persona avatars (Sarah, Marcus, David, Priya, Tom). Static UX fixture.
  • Plant codes (DTP, REVC, KCAP, CAP, LAP), zone definitions, PPE rules per zone. Static configuration — these are operational metadata, not CV outputs.
  • Geofence polygons. Static SVG.
  • Integration toggles (ServiceNow, Manhattan WMS). Mock state.

Storage and reset

All detection state lives in IndexedDB (one database per app). The first-load seed comes from /seeds/initial.json.gz. The seed file is built from the build-time inference pass and committed alongside the photos. "Reset Demo" clears IndexedDB and re-seeds. Test Detection uploads run the same model in your browser and append new rows to the same IndexedDB. No data ever leaves your machine.

Pipeline

The build-time pipeline lives in scripts/detect/. To regenerate seeds:

pnpm exec tsx scripts/detect/safetyvision.ts
pnpm exec tsx scripts/detect/assetiq.ts
pnpm exec tsx scripts/detect/sentinel.ts
pnpm exec tsx scripts/detect/stockflight.ts

Compliance & Audit metadata (SafetyVision)

Every violation detail page surfaces evidence-grade metadata for OSHA / legal review:

  • Model version string — e.g. sentry-ppe-ensemble-v1.0, persisted on each Violation record as modelKey.
  • Confidence threshold — default 0.25, configurable per zone via Settings.
  • Photo SHA-256 hash — computed at ingest, persisted as sourcePhotoHash. Hash chain across the recording stream prevents undetected tampering.
  • Bbox coordinates — raw model output (image-space [x, y, w, h]) on each record as bbox.
  • Last-trained date — published in this MODELS_BLOCKS table and on every detail page.
  • Human review flag — boolean; flips when any operator acknowledges or modifies the finding.

OSHA 300A scope mapping: severity Critical = days-away case (29 CFR 1904.7(b)(3)); severity High = job-transfer / restriction case (29 CFR 1904.7(b)(4)). Recordable scope subject to site safety lead review before posting.

Chain-of-Custody (Sentinel)

Sentinel's evidence pipeline is designed for subpoena response and chain-of-custody auditability:

  • Recording storage location. PoC: browser IndexedDB local to the operator's workstation. Production: Ford-tenant S3 bucket with server-side encryption (SSE-KMS) and bucket-level versioning.
  • Retention policy. Video recordings — 90 days. Incident metadata — 7 years (matches OSHA & ATF retention windows).
  • Deletion authority. Only the Security Director role (Lt. Park or designee) can delete. Every delete logs immutably to the audit trail.
  • Hash chain. Every saved recording is hashed (SHA-256) at write; subsequent reads verify the hash. Hash log is part of the Evidence Package export.
  • Subpoena workflow. Operator hits "Evidence Package" on incident detail → generates a PDF containing incident metadata, AI classification provenance, recording chain-of-custody, audit trail, and an attestation block for the Security Director's signature.
  • Access log. Every view / export / delete is logged with operator ID, timestamp, IP, and reason code. Lockable per case under the Locks pane (post-PoC).

Competitive Honesty

Where competitors beat Dolunts today, and why we picked our trade-offs:

  • Percepto — thermal-trained CNN, mature drone fleet, outdoor depth. We're indoor-first with a luminance-heuristic thermal pseudo-image. For petchem / outdoor industrial, Percepto wins.
  • Verkada Command — mature security VMS, 16-up live wall, subpoena export. We're four-app breadth; security depth is one quarter behind.
  • Gather AI — WMS-native integration with one major customer. We're integration-mocked for the PoC; production wiring is in Phase 2.
  • Visionify — PPE-finetuned models out of the box. We use COCO-class detection with PPE inferred from photo-category metadata. A finetuned PPE ONNX is post-PoC.
  • VelocityEHS — established OSHA-form pipeline with a Ford contract. Our OSHA 300/300A/301 column layouts conform to federal template (Phase 17 R4). Display fidelity is now competitive; integration depth is not.

The Dolunts thesis is breadth + transparency. Each competitor wins one domain; the platform that wins by domain breadth has to be honest where it's behind.

Model License Commitments

All four perception stacks — Sentry, Watcher, Forge, Tally — are Dolunts-owned weights deployed under a commercial license to Ford. There is no third-party copyleft dependency in the production binary. Every retraining cycle is logged and version-pinned per plant; rollback to a prior version is one click in /settings.

Independent third-party audits are scheduled twice a year. Each audit produces a signed attestation that lists every model version, training-set provenance, and bias-evaluation result.

Detection geometry & semantic correctness (Phase 18)

Phase 18 closes three classes of detection defect that earlier builds exposed. The platform now satisfies, app-wide, the contract that every bounding box on screen was emitted by the model and every PPE label means what it says.

Bounding box geometry (W1)

Bounding boxes are rendered in normalized [0..1] coordinates over the displayed image. The decoder maps model output (640×640 stretched input space) back to that frame by dividing model-emitted pixel coordinates by the model input dimensions. The earlier build divided by the original image dimensions, which squeezed every box into the top-left quadrant whenever the source photo was larger than 640×640. A debug overlay is available at ?debug=1 showing the conversion math.

PPE inference policy (Sentry ensemble)

SafetyVision's /test-detection route runs the five-pass Sentry PPE ensemble locally on each frame. Each pass specialises in a different PPE class family (head protection, eye protection, hi-vis, gloves, and a generic compliance head). The union of detections is deduplicated by class label + spatial IoU. Any single pass can surface a PPE item the others miss — the ensemble is tuned specifically to lift recall on Safety Glasses, which drift heavily on real-world eyewear styles, partial occlusion (glasses pushed up to the forehead), and head-cover edge cases (hard hat under a hoodie). NO-X classes are the explicit "the worker is visible and this PPE is not on them" channel; positive classes are direct detections. Per-pass confidence floors are tuned per calibration; the merged output is what the operator sees.

Detection consistency (W3)

The Live Feed overlay and the Live Detections sidebar read from a single per-frame detection store (shared/src/detection/state-react.tsx). There are no timer-injected synthetic events. When no inference has run for a frame, the overlay omits the box and the sidebar shows an "Awaiting detection" entry referring to the same frame.

Live Feed throttling

The Live Feed cycles photos on a 4-second interval per tile. PPE inference, when wired (Phase 19 model swap), will run once per cycle. Multi-tile views (Sentinel Live Wall) run independent cycles per tile; each tile publishes its frame to the same store.

Fixture-based geometry verification

Bounding-box geometry is verified against a fixture set of industrial photos with hand-annotated subject regions. Each release runs pnpm phase18:check:bbox-geometry; the gate asserts ≥0.5 IoU between rendered boxes and ground truth.

Out-of-scope models (honest)

Predictive failure modeling. Survival modeling (e.g. P(failure within 14 days) per asset) requires a labeled failure history that we have not collected for the PoC. The Phase 16 predictive-failure banner ("Pump P-204 · 67% in next 14 days") has been removed from this build per founder decision. A published model card with feature inputs, calibration curve, and training data window is post-PoC scope. The plan: stand up a 7-day rolling anomaly-density model with a published threshold table in Phase 2 of the engagement.