# Road Hazard Detection

![YOLO](https://img.shields.io/badge/YOLO-TF.js%20%2B%20Core%20ML-00FFFF) ![Firebase](https://img.shields.io/badge/Firebase-Firestore%20rules-FFCA28) ![iOS](https://img.shields.io/badge/iOS-SwiftUI-F05138) ![team](https://img.shields.io/badge/team-2%20people-lightgrey)

Detect potholes and cracks from a phone camera, attach the location, and turn them into reports a city team can triage. Runs as a web PWA and as a native iOS app, with on-device inference on both.

![Detection demo: YOLO26n finds a pothole in a street photo](docs/images/demo-detect.jpg)

**Try it:** [open the live demo](https://shachafemanoel.github.io/Hazard-Detection-Web/demo/) (no account, no backend), or run it locally:

```bash
python3 -m http.server 8000   # from the repo root
# open http://localhost:8000/demo/
```

The [demo](demo/) runs the project's real models and Web Worker on sample photos or your own, and walks through the access rules with fictional data.

| Comparing the two models | Who can do what |
|---|---|
| ![YOLO12n finds two hazards that the end-to-end model misses](docs/images/demo-models.jpg) | ![Allowed and denied actions per role, checked against the rules](docs/images/demo-rules.png) |

## The problem

Road damage is reported late and inconsistently, often with no photo or exact location. We wanted reporting to take one tap: point the camera, let the model flag the hazard, and file a report with the image, GPS position and address, while an admin dashboard tracks each report from *new* to *fixed*.

## What we built

| Part | What it does |
|---|---|
| **Web PWA** | Live camera detection and photo upload, Mapbox map, filterable report table, admin status workflow. Installable, works offline for the shell. |
| **Native iOS app** (SwiftUI) | Core ML detection through Vision/AVFoundation, the same report schema, and an offline-first upload queue. |
| **Backend** | Firebase Auth (email + Google), Firestore with role-based security rules, Cloudinary for images, and a Node server for local hosting and the admin cleanup job. |
| **Models** | Two YOLO detectors (crack, pothole) trained on Kaggle, exported to TensorFlow.js and Core ML. |

## Architecture

```mermaid
flowchart LR
    subgraph Client["Browser / iPhone"]
        Cam[Camera or photo] --> W["Inference<br/>(Web Worker · TF.js / Core ML)"]
        W --> T["Tracker<br/>dedupe across frames"]
        T --> R[Report draft]
        R --> Q["Offline queue<br/>(iOS · SwiftData)"]
    end
    Q --> CL[(Cloudinary<br/>images)]
    Q --> FS[(Firestore<br/>reports)]
    R --> GEO[Reverse geocoding]
    FS --> D["Dashboard<br/>map · table · status"]
    Rules{{"firestore.rules<br/>roles · schema · counter"}} -.enforces.-> FS
```

## Engineering decisions

| Decision | Why | Trade-off |
|---|---|---|
| **Inference on the device, in a Web Worker** | No server GPU costs, works with a weak signal, and photos never leave the phone unless a report is filed. The worker keeps the camera view and map responsive. | Accuracy is capped by what a nano model can run on a phone. |
| **Two models for two jobs** | Uploads use **YOLO26n**, an end-to-end model that needs no NMS step. Live video uses **YOLO12n** with NMS (IoU 0.45). On the demo's sample photos it flags more low-contrast damage, so try both. | Two models to keep in sync and cache (~11 MB together). |
| **Track hazards across frames before reporting** | A pothole stays in view for many frames. IoU matching plus a small state machine (*tentative → confirmed → reported*) files it once. Above ~50 km/h the confidence threshold drops so fast-passing hazards still register. | More state to reason about than per-frame detection. |
| **Security lives in Firestore rules, not the UI** | Any client can call Firestore directly, so the rules decide: roles can't be self-assigned, reports are schema-checked on create, and the report counter can only advance by one. | Rules are harder to test than app code, so they have their own emulator suite. |
| **Offline-first reports on iOS** | Queued reports are stored with SwiftData and uploaded by a background `URLSession`, with retries at 1 s, 4 s, 16 s, then 60 s. A client-generated `clientReportId` makes each Firestore write idempotent, so a retry never creates a duplicate. | The queue adds a persistence layer and its own failure states. |
| **Config injected at runtime** | `server.js` serves `/env.js` from environment variables, so keys aren't committed for the Node deployment. | Firebase web keys are public by design, so they're restricted in the cloud consoles rather than hidden. |

## Security and testing

- **Firestore rules** ([`firestore.rules`](firestore.rules)): role assignment, report schema validation and counter integrity, covered by **24 emulator tests** in [`tests/rules`](tests/rules).
- **Node server** ([`server.js`](server.js)): serves only the public web app. The admin cleanup endpoint requires a verified admin ID token.
- **Images and builds**: credentials are excluded from Docker images and Firebase Hosting.

```bash
cd tests/rules && npm install && npm test   # needs Java for the Firestore emulator
```

## Team and my role

Built as a two-person team with [Nirel Jano](https://github.com/NirelJano). We designed the architecture and trained the models together, and both worked on the web app.

**My focus:**
- The native iOS app: Core ML detection, the shared report pipeline, and the offline queue with background uploads.
- The Firestore security model and its emulator tests.
- The security review and hardening of the server and deployment config.

## Tech stack

**ML:** Ultralytics YOLO (YOLO26n, YOLO12n) · TensorFlow.js · Core ML · Vision
**Web:** Vanilla JS (ES modules) · Web Workers · Service Worker · Tailwind · Mapbox GL JS
**iOS:** Swift · SwiftUI · SwiftData · AVFoundation · background URLSession
**Backend:** Firebase Auth · Firestore + security rules · Cloudinary · Node.js/Express · Docker

## Repository map

```
demo/            standalone demo (no backend)
js/worker.js     inference, NMS and tracking state machine
js/              web app modules (upload, live detection, dashboard)
pages/           SPA page fragments
assets/models/   TF.js models
ios-app/         native SwiftUI app
firestore.rules  security rules · tests/rules/ emulator tests
server.js        local/Docker server · admin-tools/ Cloudinary cleanup
```

## Running the full app

The full app needs your own Firebase project, Cloudinary account and Mapbox token. Copy [`.env.example`](.env.example) to `.env` and fill it in, then:

```bash
npm install && npm start   # http://localhost:3000
```

## Limitations and next steps

- Model accuracy hasn't been benchmarked on a held-out set in this repo, so the confidence scores aren't calibrated accuracy figures.
- The iOS offline queue (milestone M1) is in progress. Detection quality, live UX and performance work are planned in [`ios-app/HazardDetection/docs`](ios-app/HazardDetection/docs/UPGRADE_ROADMAP.md).
- Images are uploaded with an unsigned Cloudinary preset. Signed uploads would be the next hardening step.

## Credits and licensing

- Models are trained with [Ultralytics YOLO](https://github.com/ultralytics/ultralytics), which is licensed **AGPL-3.0**. Use of the model files follows that license.
- Demo sample photos come from Wikimedia Commons under CC0 or CC BY 4.0. See [credits](demo/samples/CREDITS.md).
- A license for the application code hasn't been chosen yet. Until one is added, all rights are reserved by the authors.
