Frame Processor — Plan

Context

A self-hosted ASP.NET Core service that turns arbitrary images (uploaded as JPG/PNG or pulled from a URL) into something a 6-color e-ink frame (Spectra 6) can render. The frame's accepted colors don't match what's actually displayed — #FF0000 in the PNG renders as a muted #62201E red on the device — so the pipeline has to dither against the displayed palette and then remap each pixel to the device-input color. After the PNG is ready, the service publishes an MQTT signal; the frame wakes and pulls its image from /i/{mac}.png.

The firmware is already written, so we get to design both sides. The service must support multiple frames (each with its own MAC, resolution, orientation, palette) so it can grow with the home setup.

Architecture

Single ASP.NET Core (.NET 10) service, sync request handling, packaged as a Docker image with docker-compose.

[client] --POST image--> [Service] --dither+remap+save PNG--> [disk] --GET /i/{mac}.png--> [frame]
                              |
                              +--MQTT publish "update"--> [broker] -----> [frame]

Key non-functional choices:

Sync request model (1600×1200 dither finishes well under 1s; no queue needed).
Per-frame lock to serialize concurrent uploads to the same frame.
Latest-only storage on local disk with atomic temp-file rename.
MQTT non-blocking on failure: PNG saved regardless, background retry loop publishes the update when broker recovers.

Project layout

src/FrameProcessor/
  Program.cs                            // host + DI + middleware wiring
  Domain/                               // value types — own their JSON + route binders
    MacAddress.cs                       // record struct; permissive parse, canonical lowercase-no-separator ToString
    FrameName.cs                        // record struct; validates URL-safe charset (RFC 3986 unreserved)
    Orientation.cs                      // enum Landscape | Portrait with JSON converter
    Resolution.cs                       // record; ForOrientation(Orientation) returns rotated pair
    PaletteEntry.cs                     // record with parsed ImageSharp Colors (no hex strings)
    DitherAlgorithm.cs                  // enum (floyd-steinberg | atkinson | stucki | jarvis) with JSON converter
    ApiKey.cs                           // record struct with constant-time comparison
  Configuration/
    FramesOptions.cs                    // bound from frames.json (IOptionsMonitor)
    MqttOptions.cs                      // bound from appsettings.json
    ApiKeyOptions.cs                    // wraps an ApiKey
  Controllers/
    FramesController.cs                 // POST /api/frames/{name}/image, /image-url
    ImageController.cs                  // GET /i/{mac}.png
    HealthController.cs                 // GET /health
  ImagePipeline/
    IImagePipeline.cs
    ImagePipeline.cs                    // scale → dither → remap → encode (indexed PNG)
    PaletteFactory.cs                   // builds ImageSharp ReadOnlyMemory<Color> from PaletteEntry list
    DitheringRegistry.cs                // maps DitherAlgorithm → IDither
  Mqtt/
    MqttPublisher.cs                    // persistent MQTTnet client, QoS 1, background retry queue
  Storage/
    ImageStore.cs                       // atomic write, path resolution from MacAddress
  Middleware/
    ApiKeyMiddleware.cs                 // enforces API key on /api/* only
  Concurrency/
    FrameLockProvider.cs                // keyed SemaphoreSlim per FrameName
tests/FrameProcessor.Tests/
  ImagePipelineTests.cs                 // golden-image fixtures
  PaletteFactoryTests.cs
  MacAddressTests.cs                    // round-trips colons/hyphens/uppercase/whitespace → canonical
  FrameNameTests.cs                     // URL-safety validation
Dockerfile
docker-compose.yml
appsettings.json
frames.json                             // mounted; not in image

Settings schemas

`appsettings.json` (in image, overridable by env)

{
  "Mqtt": {
    "Host": "mosquitto",
    "Port": 1883,
    "ClientId": "frame-processor",
    "Username": null,
    "Password": null,
    "UseTls": false,
    "BaseTopic": "frames",
    "PublishQos": 1,
    "RetryBackoffSeconds": [1, 2, 5, 15, 60]
  },
  "ApiKey": "change-me",
  "Storage": {
    "ImageDirectory": "/data/images"
  },
  "UrlFetch": {
    "MaxBytes": 52428800,
    "TimeoutSeconds": 30,
    "MaxRedirects": 3
  },
  "Logging": { "LogLevel": { "Default": "Information" } }
}

`frames.json` (mounted volume, hot-reloaded via `IOptionsMonitor`)

{
  "Frames": [
    {
      "name": "living-room",
      "mac": "AA:BB:CC:DD:EE:FF",     // normalized to "aabbccddeeff" at load
      "resolution": { "width": 1600, "height": 1200 },
      "orientation": "landscape",     // landscape | portrait
      "dithering": "floyd-steinberg", // floyd-steinberg | atkinson | stucki | jarvis
      "palette": [
        { "name": "black",  "color": "#1F2226", "deviceColor": "#000000" },
        { "name": "white",  "color": "#B9C7C9", "deviceColor": "#FFFFFF" },
        { "name": "blue",   "color": "#233F8E", "deviceColor": "#0000FF" },
        { "name": "green",  "color": "#35563A", "deviceColor": "#00FF00" },
        { "name": "red",    "color": "#62201E", "deviceColor": "#FF0000" },
        { "name": "yellow", "color": "#C1BB1E", "deviceColor": "#FFFF00" }
      ]
    }
  ]
}

Validation on load: name URL-safe, MAC parses to 6 bytes, palette has ≥2 entries, all hex colors parse, dithering key is known. Fail startup with a clear message if any frame is invalid; log a warning and skip the bad frame on hot-reload (so a typo doesn't crash a live service).

HTTP API

Method	Path	Auth	Body	Response
POST	`/api/frames/{name}/image`	API key	`multipart/form-data` (file)	`200 { frame, mac, url, processedAt, mqttPublished: bool }`
POST	`/api/frames/{name}/image-url`	API key	`application/json { url }`	same
GET	`/i/{mac}.png`	none	—	`200` PNG with `Cache-Control: no-store`, ETag from mtime; `404` if no image yet
GET	`/health`	none	—	`200 { status, mqttConnected }`

{name} resolves to a frame via FramesOptions lookup; 404 if unknown.

{mac} in /i/{mac}.png is normalized on entry (lowercase, separators stripped) and matched against normalized stored MACs.

Image pipeline (the meat)

For each upload:

Load the input via ImageSharp (Image.Load<Rgba32>(stream)).
Resize with ResizeMode.Crop (center-crop cover) to the frame's resolution — swap W/H if orientation == portrait.
Dither against a palette built from the already-parsed DisplayColor of each PaletteEntry, using the configured dither algorithm:
```
var displayPalette = frame.Palette.Select(p => p.DisplayColor).ToArray();
img.Mutate(x => x.Dither(ditherings[frame.Dithering], displayPalette));
```
frame.Dithering is a DitherAlgorithm enum; ditherings is IReadOnlyDictionary<DitherAlgorithm, IDither>.
Remap each pixel from DisplayColor → DeviceColor. Build a Dictionary<Rgba32, Rgba32> once per frame from the palette and do a single pixel-buffer pass. Since the dither in step 3 produces only palette colors, every pixel finds a mapping.
Rotate 90° if orientation == portrait so the output PNG is always width × height as declared (e.g., 1600×1200), with content rotated for portrait mounting.
Encode as indexed PNG (PngColorType.Palette, PngBitDepth.Bit4 — fits 6 colors in a nibble). ImageSharp's PngEncoder with Quantizer = new PaletteQuantizer(deviceColorPalette) produces this in one call.
Atomic write: {tmpPath} → {finalPath} via File.Move(..., overwrite: true) so the GET endpoint never sees a half-written file.

IImagePipeline is a single injected service. Encapsulates 1–6 and returns the bytes; ImageStore does step 7.

MQTT

MQTTnet v4 with IMqttClient wrapped in a hosted service (MqttPublisher : IHostedService).
Single persistent connection with WithReconnectDelay and WithCleanSession(false). Connection state surfaced to /health.
Publish: topic = {BaseTopic}/{normalized-mac}, payload bytes "update", QoS 1, not retained.
Failure handling: MqttPublisher.PublishAsync returns a Result (success/failure). If failure, the publisher enqueues the (topic, payload) into an in-memory Channel<Pending> and a background loop drains it with the configured backoff sequence. The upload response includes mqttPublished: false so callers know.

Concurrency

FrameLockProvider exposes Task<IDisposable> AcquireAsync(FrameName frame, CancellationToken ct) and hands out a per-frame SemaphoreSlim(1,1) keyed by FrameName. FramesController wraps the whole pipeline (download → process → save → publish-attempt) in the lock for that frame so the latest-written PNG always corresponds to the latest publish attempt.

Tests

tests/FrameProcessor.Tests (xUnit + ImageSharp):

ImagePipelineTests — golden-image fixtures. Check in 3-4 small test inputs (landscape.jpg, portrait.jpg, tiny-1x1.png, non-divisible.jpg) and expected outputs for one canonical Spectra 6 palette. Assert byte-equality of the indexed PNG output. ImageSharp dither output is deterministic for a given input/palette/algorithm, so this is a strong correctness anchor.
PaletteFactoryTests — hex parsing, count, error messages on bad hex.
MacAddressTests — round-trip: "AA:BB:CC:DD:EE:FF", "aa-bb-cc-dd-ee-ff", "AABBCCDDEEFF", and surrounding whitespace all parse to the same value; ToString() always emits the canonical lowercase-no-separator form; equality compares on the 6 bytes.
FrameNameTests — accepts URL-safe names, rejects names with reserved or whitespace characters with a clear error.

No controller/integration tests in v1 — pipeline correctness is the load-bearing part; HTTP plumbing is thin enough to verify manually with curl.

Verification (end-to-end)

docker compose up --build — service comes up, connects to broker, /health returns { status: "Healthy", mqttConnected: true }.
curl -H "X-Api-Key: ..." -F "image=@photo.jpg" http://localhost:8080/api/frames/living-room/image — returns 200 with a url.
Open http://localhost:8080/i/aabbccddeeff.png in a browser → confirm the dithered, color-remapped PNG looks reasonable visually.
Subscribe with mosquitto_sub -t 'frames/#' -v → confirm frames/aabbccddeeff update appears within ~1s of the upload.
Repeat with /image-url and a public image URL.
Concurrency check: fire two curl uploads to the same frame in parallel → both return 200, only one final PNG on disk, two MQTT publishes (in serial order).
Stop the broker, upload an image → response is 200 with mqttPublished: false, PNG saved, log shows retry attempts. Start the broker → background retry publishes successfully.
Run dotnet test → golden-image fixtures pass.
Power on the actual Spectra 6 frame mounted with the new firmware → it fetches the PNG and renders it correctly with the expected color mapping.

12 KiB Raw Permalink Blame History Unescape Escape