# Frame Processor — Implementation Increments Step-by-step build order for assembling the service described in `SPEC.md` and `PLAN.md`. Each increment is sized to compile, run, and be independently verified before moving to the next. Order is chosen so a minimal end-to-end path exists early; quality, resilience, and tests layer on after. When working through this file, mark increments complete by changing `[ ]` to `[x]`. If the design deviates from `PLAN.md`/`SPEC.md` mid-stream, update those documents too. --- ## Phase 0 — Scaffolding ### [x] 0.1 Solution + projects - Create `FrameProcessor.sln` at repo root. - Create `src/FrameProcessor/FrameProcessor.csproj` targeting `net10.0`, `Microsoft.NET.Sdk.Web`. - Create `tests/FrameProcessor.Tests/FrameProcessor.Tests.csproj` (xUnit) referencing the main project. - Add `.gitignore` (dotnet template), `global.json` pinning .NET 10 SDK. - **DoD:** `dotnet build` and `dotnet test` both succeed (no tests yet, but harness runs). ### [x] 0.2 Minimal Program.cs - Minimal hosting + controllers wired up. - `GET /health` returning `{ status: "Healthy", mqttConnected: false }` (mqttConnected hardcoded for now). - **DoD:** `dotnet run --project src/FrameProcessor` then `curl localhost:8080/health` returns 200 with the JSON. ### [x] 0.3 Package references - Add to `FrameProcessor.csproj`: - `SixLabors.ImageSharp` - `MQTTnet` - `Serilog.AspNetCore`, `Serilog.Sinks.Console`, `Serilog.Sinks.File` - **DoD:** restore + build clean. --- ## Phase 1 — Domain value types Each type lives in `src/FrameProcessor/Domain/`. Tests in `tests/FrameProcessor.Tests/`. ### [x] 1.1 `MacAddress` - `record struct` storing 6 bytes. - `static bool TryParse(string, out MacAddress)` accepting `AA:BB:CC:DD:EE:FF`, `aa-bb-cc-dd-ee-ff`, `AABBCCDDEEFF`, surrounding whitespace. - `ToString()` → canonical lowercase no-separator (`aabbccddeeff`). - JSON converter (System.Text.Json) using the permissive parse + canonical write. - ASP.NET route value binding (e.g., `IParsable`). - **Tests:** `MacAddressTests` covering round-trip from all four input forms, equality on bytes, rejection of bad input. ### [x] 1.2 `FrameName` - `record struct` wrapping a validated string. - Constructor / `TryParse` enforces RFC 3986 unreserved (`A-Z a-z 0-9 - . _ ~`), non-empty. - JSON converter + `IParsable`. - **Tests:** `FrameNameTests` — accepts URL-safe, rejects whitespace/reserved chars with clear message. ### [x] 1.3 `Orientation` - `enum { Landscape, Portrait }`. - JSON converter reading/writing kebab-case lowercase. ### [x] 1.4 `Resolution` - `record(int Width, int Height)`. - `Resolution ForOrientation(Orientation o)` — swaps when portrait. ### [x] 1.5 `PaletteEntry` - `record(string Name, Color DisplayColor, Color DeviceColor)` using `SixLabors.ImageSharp.Color`. - JSON converter parses hex strings into `Color` at deserialize time (don't store hex strings on the type). ### [x] 1.6 `DitherAlgorithm` - `enum { FloydSteinberg, Atkinson, Stucki, Jarvis }`. - JSON converter reading kebab-case (`floyd-steinberg`, etc.). ### [x] 1.7 `ApiKey` - `record struct` wrapping a string. - `bool Matches(string candidate)` using `CryptographicOperations.FixedTimeEquals` over UTF-8 bytes. --- ## Phase 2 — Configuration binding ### [x] 2.1 `MqttOptions`, `StorageOptions`, `UrlFetchOptions`, `ApiKeyOptions` - POCOs in `src/FrameProcessor/Configuration/`. - Bound from `appsettings.json` via `builder.Services.Configure(...)`. - Validate on startup (`ValidateOnStart` + `IValidateOptions` or DataAnnotations). ### [x] 2.2 `FramesOptions` (from `frames.json`) - Top-level `{ Frames: Frame[] }` POCO. - Register `frames.json` as an additional config source with `reloadOnChange: true`. - Bind via `IOptionsMonitor`. - Custom validator enforcing rules from `SPEC.md` §6.2 (URL-safe name, MAC parseable, palette ≥2, hex parseable, dithering known). ### [x] 2.3 Startup vs reload asymmetry - On startup: throw on any invalid frame (fail-fast). - On `OnChange`: log warning, skip invalid frame, keep valid ones serving (`PLAN.md` line 130, `CLAUDE.md` "frames.json reload asymmetry"). - Add a `FramesRegistry` service that exposes `TryGetByName(FrameName)` / `TryGetByMac(MacAddress)` over the current valid set. ### [x] 2.4 Drop sample configs in repo root - `appsettings.json` from `PLAN.md` §"Settings schemas". - `frames.json` with one example frame (Spectra 6 palette). - **DoD:** Service starts and logs the loaded frames; an invalid `frames.json` fails startup with a clear error. --- ## Phase 3 — Image pipeline ### [x] 3.1 `PaletteFactory` - `static ReadOnlyMemory BuildDisplay(IReadOnlyList)` and `BuildDevice(...)`. - **Tests:** `PaletteFactoryTests` — count, ordering preserved. ### [x] 3.2 `DitheringRegistry` - `IReadOnlyDictionary` mapping each enum to the corresponding ImageSharp `KnownDitherings.*`. ### [x] 3.3 `IImagePipeline` + `ImagePipeline` - Single method: `byte[] Process(Stream input, Frame frame)`. - Steps (`SPEC.md` §3.2): decode → resize (cover/center-crop to oriented resolution) → dither against display palette → remap pixels display→device → rotate 90° CW if portrait → encode indexed PNG (`PngColorType.Palette`, smallest bit depth fitting the palette, `PaletteQuantizer(deviceColors)`). - Output dimensions always match the frame's *declared* `width × height` regardless of orientation. ### [x] 3.4 `ImagePipelineTests` golden fixtures - Check in `tests/FrameProcessor.Tests/Fixtures/inputs/` (landscape.jpg, portrait.jpg, tiny-1x1.png, non-divisible.jpg). - Generate expected outputs once, commit under `Fixtures/expected//.png`. - Assert byte-equality of produced PNG vs golden. - Run for all four dithering algorithms against the canonical Spectra 6 palette. - **DoD:** `dotnet test` green; deleting a fixture output and rerunning fails loud. --- ## Phase 4 — Storage ### [x] 4.1 `ImageStore` - Constructor takes `IOptions`; ensures `ImageDirectory` exists on startup. - `Task WriteAsync(MacAddress, ReadOnlyMemory, CancellationToken)` — writes to `{mac}.png.tmp`, then `File.Move(tmp, final, overwrite: true)`. - `bool TryGetPath(MacAddress, out string path)` returning the on-disk path if present. - **Manual check:** call from a unit test or scratch endpoint, confirm atomic rename behavior. --- ## Phase 5 — First end-to-end happy path (no MQTT yet) ### [x] 5.1 `FramesController.UploadImage` (multipart) - Route: `POST /api/frames/{name}/image`. - Resolve frame by name → 404 if unknown. - Read multipart file part `image` (return 400 if missing). - Call `ImagePipeline.Process`, then `ImageStore.WriteAsync`. - Return `200 { frame, mac, url, processedAt, mqttPublished: false }` (MQTT stubbed). ### [ ] 5.2 `ImageController.GetImage` - Route: `GET /i/{mac}.png`. - Normalize `{mac}` via `MacAddress.TryParse` → 404 on bad form. - Look up frame by MAC → 404 if unknown or file absent. - Return `FileStreamResult` with `Content-Type: image/png`, `Cache-Control: no-store`, `ETag` derived from file mtime. ### [ ] 5.3 Manual end-to-end smoke - Start service, `curl -F image=@photo.jpg .../api/frames/living-room/image` → 200. - `curl .../i/aabbccddeeff.png > out.png` → image opens and looks dithered + remapped. - **DoD:** above two commands work. --- ## Phase 6 — MQTT ### [ ] 6.1 `MqttPublisher` hosted service - Singleton `IHostedService` wrapping `IMqttClient` (MQTTnet v4). - `WithReconnectDelay`, `WithCleanSession(false)`, credentials/TLS from `MqttOptions`. - Exposes `bool IsConnected` for `/health`. - On `StartAsync`: connect; on failure, log and continue (background reconnect handles it). ### [ ] 6.2 `PublishAsync(MacAddress, CancellationToken) → Result` - Topic: `{BaseTopic}/{mac}`, payload UTF-8 `"update"`, QoS 1, retained false. - Returns success/failure (no throw). ### [ ] 6.3 Wire into `FramesController` - After successful save, call `PublishAsync`; set `mqttPublished` accordingly in the response. ### [ ] 6.4 `/health` reports MQTT status - Replace hardcoded `mqttConnected` with `MqttPublisher.IsConnected`. ### [ ] 6.5 Background retry queue - In-memory `Channel` (one slot per frame; newer publish supersedes older — per `SPEC.md` §5.1 "Multiple queued publishes for the same frame collapse to the most recent one"). - Background loop drains with backoff sequence from `MqttOptions.RetryBackoffSeconds`. - On reconnect, drain immediately. - **Manual check:** stop broker, upload → 200 with `mqttPublished: false`; restart broker → `mosquitto_sub` sees the message within backoff window. --- ## Phase 7 — URL-fetch ingestion ### [ ] 7.1 `IImageUrlFetcher` - Constructor takes `HttpClient` (via `IHttpClientFactory`) configured with `UrlFetch.TimeoutSeconds` and `MaxRedirects`. - `Task FetchAsync(Uri, CancellationToken)` — streams response, aborts if content exceeds `MaxBytes`. - Throws a typed exception (`ImageFetchException`) for timeout / non-2xx / too-large / redirect-loop. ### [ ] 7.2 `FramesController.UploadImageUrl` - Route: `POST /api/frames/{name}/image-url`, body `{ "url": "..." }`. - Fetch → pipeline → store → publish. Same response shape as 5.1. - Map `ImageFetchException` to `502 Bad Gateway`. ### [ ] 7.3 Manual check - `curl -H "Content-Type: application/json" -d '{"url":"https://..."}' .../image-url` works end-to-end. --- ## Phase 8 — Auth + concurrency + robustness ### [ ] 8.1 `ApiKeyMiddleware` - Matches request path `/api/*`; reads `X-Api-Key` header; constant-time compare against `ApiKeyOptions`. - 401 on mismatch. `/i/{mac}.png` and `/health` unaffected. ### [ ] 8.2 `FrameLockProvider` - `ConcurrentDictionary` (each `SemaphoreSlim(1, 1)`). - `Task AcquireAsync(FrameName, CancellationToken)` returning a disposable that releases on dispose. ### [ ] 8.3 Wrap full pipeline in lock - In `FramesController`, acquire the frame's lock before fetch/decode and release after publish-attempt completes (`CLAUDE.md` "Per-frame serialization"). - **Manual check:** fire two concurrent uploads to the same frame → both return 200, only one PNG on disk reflects whichever finished last, two MQTT publishes (or one if collapsed by retry queue). --- ## Phase 9 — Observability ### [ ] 9.1 Serilog wiring - `UseSerilog` with console + rolling file sinks (`logs/frame-processor-.log`, daily). - Structured fields: `FrameName`, `MacAddress`, `InputSource` (file/url), `InputBytes`, `OutputBytes`, `ElapsedMs`, `MqttPublished`. - Log one info line per upload at completion. --- ## Phase 10 — Packaging ### [ ] 10.1 `Dockerfile` - Multi-stage: SDK build → runtime (`mcr.microsoft.com/dotnet/aspnet:10.0`). - Expose 8080; create `/data/images`. - Non-root user. ### [ ] 10.2 `docker-compose.yml` - `frame-processor` service (build from Dockerfile, mount `frames.json` and `/data/images`). - `mosquitto` broker with anonymous access on the local network. - Single network so the service can reach `mosquitto:1883`. ### [ ] 10.3 End-to-end via compose - `docker compose up --build` → `/health` returns `mqttConnected: true`. - Run the full `PLAN.md` §Verification checklist (steps 2–7). --- ## Phase 11 — Final verification ### [ ] 11.1 Run `PLAN.md` verification checklist top to bottom - All 8 steps pass (step 9, real frame, is optional from this codebase's perspective). ### [ ] 11.2 README - Short `README.md` pointing at `SPEC.md`, `PLAN.md`, `CLAUDE.md` and showing the `docker compose up` + smoke-test commands. --- ## Notes for future-me - **Don't reorder dither/remap.** Dither against `DisplayColor`; remap to `DeviceColor` after. Reversing produces visibly wrong output. - **Normalize MAC at every boundary.** Config load, route binding, MQTT topic, file path — all lowercase no-separator. - **Atomic writes are non-negotiable.** Never write directly to `{mac}.png`; always tmp + rename. - **MQTT failure ≠ upload failure.** A broker outage produces a 200 with `mqttPublished: false`, not a 5xx. - **Hot-reload asymmetry.** Startup is strict; reload is lenient and skips bad frames. - **No premature abstractions.** If a step says "single class," resist the urge to introduce interfaces beyond what `PLAN.md` lists.