# Frame Processor — Plan

## Context

A self-hosted ASP.NET Core service that turns arbitrary images (uploaded as JPG/PNG or pulled from a URL) into something a 6-color e-ink frame (Spectra 6) can render. The frame's accepted colors don't match what's *actually displayed* — `#FF0000` in the PNG renders as a muted `#62201E` red on the device — so the pipeline has to dither against the *displayed* palette and then remap each pixel to the device-input color. After the PNG is ready, the service publishes an MQTT signal; the frame wakes and pulls its image from `/i/{mac}.png`.

The firmware is already written, so we get to design both sides. The service must support multiple frames (each with its own MAC, resolution, orientation, palette) so it can grow with the home setup.

---

## Architecture

Single ASP.NET Core (.NET 10) service, sync request handling, packaged as a Docker image with docker-compose.

```
[client] --POST image--> [Service] --dither+remap+save PNG--> [disk] --GET /i/{mac}.png--> [frame]
                              |
                              +--MQTT publish "update"--> [broker] -----> [frame]
```

Key non-functional choices:
- **Sync** request model (1600×1200 dither finishes well under 1s; no queue needed).
- **Per-frame lock** to serialize concurrent uploads to the same frame.
- **Latest-only storage** on local disk with atomic temp-file rename.
- **MQTT non-blocking on failure**: PNG saved regardless, background retry loop publishes the `update` when broker recovers.

---

## Project layout

```
src/FrameProcessor/
  Program.cs                            // host + DI + middleware wiring
  Domain/                               // value types — own their JSON + route binders
    MacAddress.cs                       // record struct; permissive parse, canonical lowercase-no-separator ToString
    FrameName.cs                        // record struct; validates URL-safe charset (RFC 3986 unreserved)
    Orientation.cs                      // enum Landscape | Portrait with JSON converter
    Resolution.cs                       // record; ForOrientation(Orientation) returns rotated pair
    PaletteEntry.cs                     // record with parsed ImageSharp Colors (no hex strings)
    DitherAlgorithm.cs                  // enum (floyd-steinberg | atkinson | stucki | jarvis) with JSON converter
    ApiKey.cs                           // record struct with constant-time comparison
  Configuration/
    FramesOptions.cs                    // bound from frames.json (IOptionsMonitor)
    MqttOptions.cs                      // bound from appsettings.json
    ApiKeyOptions.cs                    // wraps an ApiKey
  Controllers/
    FramesController.cs                 // POST /api/frames/{name}/image, /image-url
    ImageController.cs                  // GET /i/{mac}.png
    HealthController.cs                 // GET /health
  ImagePipeline/
    IImagePipeline.cs
    ImagePipeline.cs                    // scale → dither → remap → encode (indexed PNG)
    PaletteFactory.cs                   // builds ImageSharp ReadOnlyMemory<Color> from PaletteEntry list
    DitheringRegistry.cs                // maps DitherAlgorithm → IDither
  Mqtt/
    MqttPublisher.cs                    // persistent MQTTnet client, QoS 1, background retry queue
  Storage/
    ImageStore.cs                       // atomic write, path resolution from MacAddress
  Middleware/
    ApiKeyMiddleware.cs                 // enforces API key on /api/* only
  Concurrency/
    FrameLockProvider.cs                // keyed SemaphoreSlim per FrameName
tests/FrameProcessor.Tests/
  ImagePipelineTests.cs                 // golden-image fixtures
  PaletteFactoryTests.cs
  MacAddressTests.cs                    // round-trips colons/hyphens/uppercase/whitespace → canonical
  FrameNameTests.cs                     // URL-safety validation
Dockerfile
docker-compose.yml
appsettings.json
frames.json                             // mounted; not in image
```

---

## Settings schemas

### `appsettings.json` (in image, overridable by env)

```jsonc
{
  "Mqtt": {
    "Host": "mosquitto",
    "Port": 1883,
    "ClientId": "frame-processor",
    "Username": null,
    "Password": null,
    "UseTls": false,
    "BaseTopic": "frames",
    "PublishQos": 1,
    "RetryBackoffSeconds": [1, 2, 5, 15, 60]
  },
  "ApiKey": "change-me",
  "Storage": {
    "ImageDirectory": "/data/images"
  },
  "UrlFetch": {
    "MaxBytes": 52428800,
    "TimeoutSeconds": 30,
    "MaxRedirects": 3
  },
  "Logging": { "LogLevel": { "Default": "Information" } }
}
```

### `frames.json` (mounted volume, hot-reloaded via `IOptionsMonitor`)

```jsonc
{
  "Frames": [
    {
      "name": "living-room",
      "mac": "AA:BB:CC:DD:EE:FF",     // normalized to "aabbccddeeff" at load
      "resolution": { "width": 1600, "height": 1200 },
      "orientation": "landscape",     // landscape | portrait
      "dithering": "floyd-steinberg", // floyd-steinberg | atkinson | stucki | jarvis
      "palette": [
        { "name": "black",  "color": "#1F2226", "deviceColor": "#000000" },
        { "name": "white",  "color": "#B9C7C9", "deviceColor": "#FFFFFF" },
        { "name": "blue",   "color": "#233F8E", "deviceColor": "#0000FF" },
        { "name": "green",  "color": "#35563A", "deviceColor": "#00FF00" },
        { "name": "red",    "color": "#62201E", "deviceColor": "#FF0000" },
        { "name": "yellow", "color": "#C1BB1E", "deviceColor": "#FFFF00" }
      ]
    }
  ]
}
```

Validation on load: name URL-safe, MAC parses to 6 bytes, palette has ≥2 entries, all hex colors parse, dithering key is known. Fail startup with a clear message if any frame is invalid; log a warning and skip the bad frame on hot-reload (so a typo doesn't crash a live service).

---

## HTTP API

| Method | Path                                | Auth     | Body                          | Response                                                       |
|--------|-------------------------------------|----------|-------------------------------|----------------------------------------------------------------|
| POST   | `/api/frames/{name}/image`          | API key  | `multipart/form-data` (file)  | `200 { frame, mac, url, processedAt, mqttPublished: bool }`    |
| POST   | `/api/frames/{name}/image-url`      | API key  | `application/json { url }`    | same                                                           |
| GET    | `/i/{mac}.png`                      | none     | —                             | `200` PNG with `Cache-Control: no-store`, ETag from mtime; `404` if no image yet |
| GET    | `/health`                           | none     | —                             | `200 { status, mqttConnected }`                                |

`{name}` resolves to a frame via `FramesOptions` lookup; 404 if unknown.

`{mac}` in `/i/{mac}.png` is normalized on entry (lowercase, separators stripped) and matched against normalized stored MACs.

---

## Image pipeline (the meat)

For each upload:

1. **Load** the input via ImageSharp (`Image.Load<Rgba32>(stream)`).
2. **Resize** with `ResizeMode.Crop` (center-crop cover) to the frame's resolution — swap W/H if `orientation == portrait`.
3. **Dither** against a palette built from the already-parsed `DisplayColor` of each `PaletteEntry`, using the configured dither algorithm:
   ```csharp
   var displayPalette = frame.Palette.Select(p => p.DisplayColor).ToArray();
   img.Mutate(x => x.Dither(ditherings[frame.Dithering], displayPalette));
   ```
   `frame.Dithering` is a `DitherAlgorithm` enum; `ditherings` is `IReadOnlyDictionary<DitherAlgorithm, IDither>`.
4. **Remap** each pixel from `DisplayColor` → `DeviceColor`. Build a `Dictionary<Rgba32, Rgba32>` once per frame from the palette and do a single pixel-buffer pass. Since the dither in step 3 produces only palette colors, every pixel finds a mapping.
5. **Rotate** 90° if `orientation == portrait` so the output PNG is always `width × height` as declared (e.g., 1600×1200), with content rotated for portrait mounting.
6. **Encode** as **indexed PNG** (`PngColorType.Palette`, `PngBitDepth.Bit4` — fits 6 colors in a nibble). ImageSharp's `PngEncoder` with `Quantizer = new PaletteQuantizer(deviceColorPalette)` produces this in one call.
7. **Atomic write**: `{tmpPath} → {finalPath}` via `File.Move(..., overwrite: true)` so the GET endpoint never sees a half-written file.

`IImagePipeline` is a single injected service. Encapsulates 1–6 and returns the bytes; `ImageStore` does step 7.

---

## MQTT

- `MQTTnet` v4 with `IMqttClient` wrapped in a hosted service (`MqttPublisher : IHostedService`).
- Single persistent connection with `WithReconnectDelay` and `WithCleanSession(false)`. Connection state surfaced to `/health`.
- Publish: `topic = {BaseTopic}/{normalized-mac}`, payload bytes `"update"`, QoS 1, not retained.
- **Failure handling**: `MqttPublisher.PublishAsync` returns a `Result` (success/failure). If failure, the publisher enqueues the (topic, payload) into an in-memory `Channel<Pending>` and a background loop drains it with the configured backoff sequence. The upload response includes `mqttPublished: false` so callers know.

---

## Concurrency

`FrameLockProvider` exposes `Task<IDisposable> AcquireAsync(FrameName frame, CancellationToken ct)` and hands out a per-frame `SemaphoreSlim(1,1)` keyed by `FrameName`. `FramesController` wraps the whole pipeline (download → process → save → publish-attempt) in the lock for that frame so the latest-written PNG always corresponds to the latest publish attempt.

---

## Tests

`tests/FrameProcessor.Tests` (xUnit + ImageSharp):

- **`ImagePipelineTests`** — golden-image fixtures. Check in 3-4 small test inputs (`landscape.jpg`, `portrait.jpg`, `tiny-1x1.png`, `non-divisible.jpg`) and expected outputs for one canonical Spectra 6 palette. Assert byte-equality of the indexed PNG output. ImageSharp dither output is deterministic for a given input/palette/algorithm, so this is a strong correctness anchor.
- **`PaletteFactoryTests`** — hex parsing, count, error messages on bad hex.
- **`MacAddressTests`** — round-trip: `"AA:BB:CC:DD:EE:FF"`, `"aa-bb-cc-dd-ee-ff"`, `"AABBCCDDEEFF"`, and surrounding whitespace all parse to the same value; `ToString()` always emits the canonical lowercase-no-separator form; equality compares on the 6 bytes.
- **`FrameNameTests`** — accepts URL-safe names, rejects names with reserved or whitespace characters with a clear error.

No controller/integration tests in v1 — pipeline correctness is the load-bearing part; HTTP plumbing is thin enough to verify manually with curl.

---

## Verification (end-to-end)

1. `docker compose up --build` — service comes up, connects to broker, `/health` returns `{ status: "Healthy", mqttConnected: true }`.
2. `curl -H "X-Api-Key: ..." -F "image=@photo.jpg" http://localhost:8080/api/frames/living-room/image` — returns 200 with a `url`.
3. Open `http://localhost:8080/i/aabbccddeeff.png` in a browser → confirm the dithered, color-remapped PNG looks reasonable visually.
4. Subscribe with `mosquitto_sub -t 'frames/#' -v` → confirm `frames/aabbccddeeff update` appears within ~1s of the upload.
5. Repeat with `/image-url` and a public image URL.
6. Concurrency check: fire two `curl` uploads to the same frame in parallel → both return 200, only one final PNG on disk, two MQTT publishes (in serial order).
7. Stop the broker, upload an image → response is 200 with `mqttPublished: false`, PNG saved, log shows retry attempts. Start the broker → background retry publishes successfully.
8. Run `dotnet test` → golden-image fixtures pass.
9. Power on the actual Spectra 6 frame mounted with the new firmware → it fetches the PNG and renders it correctly with the expected color mapping.
