209 lines
12 KiB
Markdown
209 lines
12 KiB
Markdown
# Frame Processor — Plan
|
||
|
||
## Context
|
||
|
||
A self-hosted ASP.NET Core service that turns arbitrary images (uploaded as JPG/PNG or pulled from a URL) into something a 6-color e-ink frame (Spectra 6) can render. The frame's accepted colors don't match what's *actually displayed* — `#FF0000` in the PNG renders as a muted `#62201E` red on the device — so the pipeline has to dither against the *displayed* palette and then remap each pixel to the device-input color. After the PNG is ready, the service publishes an MQTT signal; the frame wakes and pulls its image from `/i/{mac}.png`.
|
||
|
||
The firmware is already written, so we get to design both sides. The service must support multiple frames (each with its own MAC, resolution, orientation, palette) so it can grow with the home setup.
|
||
|
||
---
|
||
|
||
## Architecture
|
||
|
||
Single ASP.NET Core (.NET 10) service, sync request handling, packaged as a Docker image with docker-compose.
|
||
|
||
```
|
||
[client] --POST image--> [Service] --dither+remap+save PNG--> [disk] --GET /i/{mac}.png--> [frame]
|
||
|
|
||
+--MQTT publish "update"--> [broker] -----> [frame]
|
||
```
|
||
|
||
Key non-functional choices:
|
||
- **Sync** request model (1600×1200 dither finishes well under 1s; no queue needed).
|
||
- **Per-frame lock** to serialize concurrent uploads to the same frame.
|
||
- **Latest-only storage** on local disk with atomic temp-file rename.
|
||
- **MQTT non-blocking on failure**: PNG saved regardless, background retry loop publishes the `update` when broker recovers.
|
||
|
||
---
|
||
|
||
## Project layout
|
||
|
||
```
|
||
src/FrameProcessor/
|
||
Program.cs // host + DI + middleware wiring
|
||
Domain/ // value types — own their JSON + route binders
|
||
MacAddress.cs // record struct; permissive parse, canonical lowercase-no-separator ToString
|
||
FrameName.cs // record struct; validates URL-safe charset (RFC 3986 unreserved)
|
||
Orientation.cs // enum Landscape | Portrait with JSON converter
|
||
Resolution.cs // record; ForOrientation(Orientation) returns rotated pair
|
||
PaletteEntry.cs // record with parsed ImageSharp Colors (no hex strings)
|
||
DitherAlgorithm.cs // enum (floyd-steinberg | atkinson | stucki | jarvis) with JSON converter
|
||
ApiKey.cs // record struct with constant-time comparison
|
||
Configuration/
|
||
FramesOptions.cs // bound from frames.json (IOptionsMonitor)
|
||
MqttOptions.cs // bound from appsettings.json
|
||
ApiKeyOptions.cs // wraps an ApiKey
|
||
Controllers/
|
||
FramesController.cs // POST /api/frames/{name}/image, /image-url
|
||
ImageController.cs // GET /i/{mac}.png
|
||
HealthController.cs // GET /health
|
||
ImagePipeline/
|
||
IImagePipeline.cs
|
||
ImagePipeline.cs // scale → dither → remap → encode (indexed PNG)
|
||
PaletteFactory.cs // builds ImageSharp ReadOnlyMemory<Color> from PaletteEntry list
|
||
DitheringRegistry.cs // maps DitherAlgorithm → IDither
|
||
Mqtt/
|
||
MqttPublisher.cs // persistent MQTTnet client, QoS 1, background retry queue
|
||
Storage/
|
||
ImageStore.cs // atomic write, path resolution from MacAddress
|
||
Middleware/
|
||
ApiKeyMiddleware.cs // enforces API key on /api/* only
|
||
Concurrency/
|
||
FrameLockProvider.cs // keyed SemaphoreSlim per FrameName
|
||
tests/FrameProcessor.Tests/
|
||
ImagePipelineTests.cs // golden-image fixtures
|
||
PaletteFactoryTests.cs
|
||
MacAddressTests.cs // round-trips colons/hyphens/uppercase/whitespace → canonical
|
||
FrameNameTests.cs // URL-safety validation
|
||
Dockerfile
|
||
docker-compose.yml
|
||
appsettings.json
|
||
frames.json // mounted; not in image
|
||
```
|
||
|
||
---
|
||
|
||
## Settings schemas
|
||
|
||
### `appsettings.json` (in image, overridable by env)
|
||
|
||
```jsonc
|
||
{
|
||
"Mqtt": {
|
||
"Host": "mosquitto",
|
||
"Port": 1883,
|
||
"ClientId": "frame-processor",
|
||
"Username": null,
|
||
"Password": null,
|
||
"UseTls": false,
|
||
"BaseTopic": "frames",
|
||
"PublishQos": 1,
|
||
"RetryBackoffSeconds": [1, 2, 5, 15, 60]
|
||
},
|
||
"ApiKey": "change-me",
|
||
"Storage": {
|
||
"ImageDirectory": "/data/images"
|
||
},
|
||
"UrlFetch": {
|
||
"MaxBytes": 52428800,
|
||
"TimeoutSeconds": 30,
|
||
"MaxRedirects": 3
|
||
},
|
||
"Logging": { "LogLevel": { "Default": "Information" } }
|
||
}
|
||
```
|
||
|
||
### `frames.json` (mounted volume, hot-reloaded via `IOptionsMonitor`)
|
||
|
||
```jsonc
|
||
{
|
||
"Frames": [
|
||
{
|
||
"name": "living-room",
|
||
"mac": "AA:BB:CC:DD:EE:FF", // normalized to "aabbccddeeff" at load
|
||
"resolution": { "width": 1600, "height": 1200 },
|
||
"orientation": "landscape", // landscape | portrait
|
||
"dithering": "floyd-steinberg", // floyd-steinberg | atkinson | stucki | jarvis
|
||
"palette": [
|
||
{ "name": "black", "color": "#1F2226", "deviceColor": "#000000" },
|
||
{ "name": "white", "color": "#B9C7C9", "deviceColor": "#FFFFFF" },
|
||
{ "name": "blue", "color": "#233F8E", "deviceColor": "#0000FF" },
|
||
{ "name": "green", "color": "#35563A", "deviceColor": "#00FF00" },
|
||
{ "name": "red", "color": "#62201E", "deviceColor": "#FF0000" },
|
||
{ "name": "yellow", "color": "#C1BB1E", "deviceColor": "#FFFF00" }
|
||
]
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
Validation on load: name URL-safe, MAC parses to 6 bytes, palette has ≥2 entries, all hex colors parse, dithering key is known. Fail startup with a clear message if any frame is invalid; log a warning and skip the bad frame on hot-reload (so a typo doesn't crash a live service).
|
||
|
||
---
|
||
|
||
## HTTP API
|
||
|
||
| Method | Path | Auth | Body | Response |
|
||
|--------|-------------------------------------|----------|-------------------------------|----------------------------------------------------------------|
|
||
| POST | `/api/frames/{name}/image` | API key | `multipart/form-data` (file) | `200 { frame, mac, url, processedAt, mqttPublished: bool }` |
|
||
| POST | `/api/frames/{name}/image-url` | API key | `application/json { url }` | same |
|
||
| GET | `/i/{mac}.png` | none | — | `200` PNG with `Cache-Control: no-store`, ETag from mtime; `404` if no image yet |
|
||
| GET | `/health` | none | — | `200 { status, mqttConnected }` |
|
||
|
||
`{name}` resolves to a frame via `FramesOptions` lookup; 404 if unknown.
|
||
|
||
`{mac}` in `/i/{mac}.png` is normalized on entry (lowercase, separators stripped) and matched against normalized stored MACs.
|
||
|
||
---
|
||
|
||
## Image pipeline (the meat)
|
||
|
||
For each upload:
|
||
|
||
1. **Load** the input via ImageSharp (`Image.Load<Rgba32>(stream)`).
|
||
2. **Resize** with `ResizeMode.Crop` (center-crop cover) to the frame's resolution — swap W/H if `orientation == portrait`.
|
||
3. **Dither** against a palette built from the already-parsed `DisplayColor` of each `PaletteEntry`, using the configured dither algorithm:
|
||
```csharp
|
||
var displayPalette = frame.Palette.Select(p => p.DisplayColor).ToArray();
|
||
img.Mutate(x => x.Dither(ditherings[frame.Dithering], displayPalette));
|
||
```
|
||
`frame.Dithering` is a `DitherAlgorithm` enum; `ditherings` is `IReadOnlyDictionary<DitherAlgorithm, IDither>`.
|
||
4. **Remap** each pixel from `DisplayColor` → `DeviceColor`. Build a `Dictionary<Rgba32, Rgba32>` once per frame from the palette and do a single pixel-buffer pass. Since the dither in step 3 produces only palette colors, every pixel finds a mapping.
|
||
5. **Rotate** 90° if `orientation == portrait` so the output PNG is always `width × height` as declared (e.g., 1600×1200), with content rotated for portrait mounting.
|
||
6. **Encode** as **indexed PNG** (`PngColorType.Palette`, `PngBitDepth.Bit4` — fits 6 colors in a nibble). ImageSharp's `PngEncoder` with `Quantizer = new PaletteQuantizer(deviceColorPalette)` produces this in one call.
|
||
7. **Atomic write**: `{tmpPath} → {finalPath}` via `File.Move(..., overwrite: true)` so the GET endpoint never sees a half-written file.
|
||
|
||
`IImagePipeline` is a single injected service. Encapsulates 1–6 and returns the bytes; `ImageStore` does step 7.
|
||
|
||
---
|
||
|
||
## MQTT
|
||
|
||
- `MQTTnet` v4 with `IMqttClient` wrapped in a hosted service (`MqttPublisher : IHostedService`).
|
||
- Single persistent connection with `WithReconnectDelay` and `WithCleanSession(false)`. Connection state surfaced to `/health`.
|
||
- Publish: `topic = {BaseTopic}/{normalized-mac}`, payload bytes `"update"`, QoS 1, not retained.
|
||
- **Failure handling**: `MqttPublisher.PublishAsync` returns a `Result` (success/failure). If failure, the publisher enqueues the (topic, payload) into an in-memory `Channel<Pending>` and a background loop drains it with the configured backoff sequence. The upload response includes `mqttPublished: false` so callers know.
|
||
|
||
---
|
||
|
||
## Concurrency
|
||
|
||
`FrameLockProvider` exposes `Task<IDisposable> AcquireAsync(FrameName frame, CancellationToken ct)` and hands out a per-frame `SemaphoreSlim(1,1)` keyed by `FrameName`. `FramesController` wraps the whole pipeline (download → process → save → publish-attempt) in the lock for that frame so the latest-written PNG always corresponds to the latest publish attempt.
|
||
|
||
---
|
||
|
||
## Tests
|
||
|
||
`tests/FrameProcessor.Tests` (xUnit + ImageSharp):
|
||
|
||
- **`ImagePipelineTests`** — golden-image fixtures. Check in 3-4 small test inputs (`landscape.jpg`, `portrait.jpg`, `tiny-1x1.png`, `non-divisible.jpg`) and expected outputs for one canonical Spectra 6 palette. Assert byte-equality of the indexed PNG output. ImageSharp dither output is deterministic for a given input/palette/algorithm, so this is a strong correctness anchor.
|
||
- **`PaletteFactoryTests`** — hex parsing, count, error messages on bad hex.
|
||
- **`MacAddressTests`** — round-trip: `"AA:BB:CC:DD:EE:FF"`, `"aa-bb-cc-dd-ee-ff"`, `"AABBCCDDEEFF"`, and surrounding whitespace all parse to the same value; `ToString()` always emits the canonical lowercase-no-separator form; equality compares on the 6 bytes.
|
||
- **`FrameNameTests`** — accepts URL-safe names, rejects names with reserved or whitespace characters with a clear error.
|
||
|
||
No controller/integration tests in v1 — pipeline correctness is the load-bearing part; HTTP plumbing is thin enough to verify manually with curl.
|
||
|
||
---
|
||
|
||
## Verification (end-to-end)
|
||
|
||
1. `docker compose up --build` — service comes up, connects to broker, `/health` returns `{ status: "Healthy", mqttConnected: true }`.
|
||
2. `curl -H "X-Api-Key: ..." -F "image=@photo.jpg" http://localhost:8080/api/frames/living-room/image` — returns 200 with a `url`.
|
||
3. Open `http://localhost:8080/i/aabbccddeeff.png` in a browser → confirm the dithered, color-remapped PNG looks reasonable visually.
|
||
4. Subscribe with `mosquitto_sub -t 'frames/#' -v` → confirm `frames/aabbccddeeff update` appears within ~1s of the upload.
|
||
5. Repeat with `/image-url` and a public image URL.
|
||
6. Concurrency check: fire two `curl` uploads to the same frame in parallel → both return 200, only one final PNG on disk, two MQTT publishes (in serial order).
|
||
7. Stop the broker, upload an image → response is 200 with `mqttPublished: false`, PNG saved, log shows retry attempts. Start the broker → background retry publishes successfully.
|
||
8. Run `dotnet test` → golden-image fixtures pass.
|
||
9. Power on the actual Spectra 6 frame mounted with the new firmware → it fetches the PNG and renders it correctly with the expected color mapping.
|