frame-processor/SPEC.md

Frame Processor — Specification

1. Purpose

Frame Processor is a self-hosted HTTP service that converts arbitrary images into PNGs suitable for color e-ink frames (e.g., Spectra 6). It performs dithering against the frame's actually-displayed color palette, remaps the result to the device's input-color encoding, stores the output, and notifies the frame via MQTT to fetch it.

The service supports multiple frames, each with its own MAC address, resolution, orientation, and color palette.

2. Concepts

Term	Definition
Frame	A physical e-ink device, identified in the service by a URL-friendly name and uniquely by its mac.
Palette	The set of colors a frame can render. Each entry has a color (what the device displays for that swatch) and a deviceColor (the RGB value the device expects in the input PNG to produce that swatch).
Display color	The hex color visible on the e-ink panel — what the human eye sees.
Device color	The hex color the firmware expects in the input PNG to render the corresponding display color.
Update	The MQTT signal published after a new image is ready, prompting the frame to fetch it.

3. Functional requirements

3.1 Image ingestion

The service accepts an image for a named frame in one of two forms:

• Direct file upload as multipart/form-data (image/jpeg, image/png, or any format ImageSharp can decode).
• URL reference as a JSON body { "url": "https://..." }; the service fetches it over HTTPS or HTTP.

Both forms produce identical downstream behavior.

3.2 Processing pipeline

For each accepted image, the service performs the following stages in order:

1. Decode the input into an in-memory RGBA bitmap.
2. Resize to the frame's resolution using cover semantics — scale the input so it fully covers the target rectangle, then center-crop overflow. Aspect ratio is preserved; no stretching.
3. Dither the bitmap against a palette built from the frame's color (display) values, using the dithering algorithm configured for that frame. Output of this stage contains only colors from the display palette.
4. Remap each pixel from its display color to its corresponding deviceColor per the palette. Output of this stage contains only colors from the device-color palette.
5. Rotate the bitmap 90° clockwise if the frame's orientation is portrait. The output dimensions always match the frame's declared resolution (e.g., 1600×1200) regardless of orientation.
6. Encode as an indexed PNG with a palette of the frame's device colors, using the smallest bit depth that fits the palette size.
7. Persist the resulting PNG to storage, replacing any previous image for the same frame atomically (no half-written file is ever observable via the HTTP fetch endpoint).
8. Notify by publishing an MQTT message announcing that a new image is available.

3.3 Output delivery

The service exposes the most recent processed PNG for each frame at a stable URL keyed by the frame's MAC address. The frame is expected to fetch this URL on receipt of an MQTT notification.

3.4 Concurrency

For any single frame, ingestion requests are serialized: if a second request arrives while the first is still processing, it waits for the first to complete before starting. Requests against different frames may proceed concurrently. The last successfully-processed image is the one served from the fetch URL.

3.5 MQTT resiliency

If MQTT publish fails (broker unreachable, network error), the image is still saved and the upload request still returns successfully (with a flag indicating MQTT failed). The service retries the publish in the background using an exponential backoff schedule until it succeeds. This ensures one transient broker outage does not lose the image.

4. HTTP API

All API responses are JSON unless otherwise noted. All /api/* endpoints require an X-Api-Key header matching the configured value; otherwise 401 Unauthorized.

4.1 POST /api/frames/{name}/image

Upload an image file for a named frame.

• Path: {name} — the URL-friendly frame identifier from configuration.
• Body: multipart/form-data with a single file part named image.
• Success (200 OK):

  {
    "frame": "living-room",
    "mac": "aabbccddeeff",
    "url": "/i/aabbccddeeff.png",
    "processedAt": "2026-06-07T10:15:30Z",
    "mqttPublished": true
  }
  

• Errors:
  • 404 Not Found — unknown frame name.
  • 400 Bad Request — missing/invalid file part, unsupported image format.
  • 413 Payload Too Large — file exceeds configured maximum.

4.2 POST /api/frames/{name}/image-url

Process an image fetched from a URL.

• Path: {name} — the URL-friendly frame identifier.
• Body: application/json

  { "url": "https://example.com/photo.jpg" }
  

• Success: identical to §4.1.
• Errors: as §4.1, plus:
  • 502 Bad Gateway — remote URL did not return a usable image (timeout, non-2xx, too large, redirect loop).

4.3 GET /i/{mac}.png

Fetch the most recent processed PNG for a frame.

• Path: {mac} — the frame's MAC address; normalized server-side (lowercase, separators stripped). Match is performed against normalized configured MACs.
• Auth: none.
• Success (200 OK):
  • Content-Type: image/png
  • Cache-Control: no-store
  • ETag derived from file mtime
• Errors:
  • 404 Not Found — no image has been processed yet for this frame, or unknown MAC.

4.4 GET /health

• Auth: none.
• Success (200 OK):

  { "status": "Healthy", "mqttConnected": true }
  

5. MQTT contract

• Broker: configured in appsettings.json (host, port, credentials, TLS).
• Client: persistent connection with automatic reconnect; clean session = false.
• Publish:
  • Topic: {baseTopic}/{normalized-mac} — baseTopic is configurable (default frames); normalized-mac is lowercase hex with no separators (e.g., aabbccddeeff).
  • Payload: the UTF-8 bytes of the literal string update.
  • QoS: 1 (at-least-once).
  • Retained: false.

The service is a publisher only. It does not subscribe to any topics.

5.1 Background retry

If a publish attempt fails, the (topic, payload) is enqueued for retry. Retries occur on a configurable backoff schedule (default [1, 2, 5, 15, 60] seconds). A successful publish drains the corresponding queue entry. Multiple queued publishes for the same frame collapse to the most recent one.

6. Configuration

6.1 Application configuration — appsettings.json

{
  "Mqtt": {
    "Host": "mosquitto",
    "Port": 1883,
    "ClientId": "frame-processor",
    "Username": null,
    "Password": null,
    "UseTls": false,
    "BaseTopic": "frames",
    "PublishQos": 1,
    "RetryBackoffSeconds": [1, 2, 5, 15, 60]
  },
  "ApiKey": "change-me",
  "Storage": {
    "ImageDirectory": "/data/images"
  },
  "UrlFetch": {
    "MaxBytes": 52428800,
    "TimeoutSeconds": 30,
    "MaxRedirects": 3
  }
}

All values overridable by environment variables using ASP.NET's __ separator (e.g., Mqtt__Host).

6.2 Frame configuration — frames.json

{
  "Frames": [
    {
      "name": "living-room",
      "mac": "AA:BB:CC:DD:EE:FF",
      "resolution": { "width": 1600, "height": 1200 },
      "orientation": "landscape",
      "dithering": "floyd-steinberg",
      "palette": [
        { "name": "black",  "color": "#1F2226", "deviceColor": "#000000" },
        { "name": "white",  "color": "#B9C7C9", "deviceColor": "#FFFFFF" },
        { "name": "blue",   "color": "#233F8E", "deviceColor": "#0000FF" },
        { "name": "green",  "color": "#35563A", "deviceColor": "#00FF00" },
        { "name": "red",    "color": "#62201E", "deviceColor": "#FF0000" },
        { "name": "yellow", "color": "#C1BB1E", "deviceColor": "#FFFF00" }
      ]
    }
  ]
}

Field semantics:

Field	Type	Constraints
name	string	Required. URL-friendly (RFC 3986 unreserved characters). Unique. Bound internally to a FrameName value type.
mac	string	Required. Accepts colon-, hyphen-, or unseparated hex; case-insensitive. Bound internally to a MacAddress value type whose canonical form is lowercase hex with no separators (used for URL matching, MQTT topics, and file paths).
resolution.width, .height	int	Required. Positive. The device's native resolution.
orientation	enum	Required. landscape or portrait. Bound to an Orientation enum.
dithering	enum	Required. One of floyd-steinberg, atkinson, stucki, jarvis. Bound to a DitherAlgorithm enum.
palette	array	Required. ≥2 entries. Each entry has name, color, deviceColor.
palette[].color / .deviceColor	string	Required. Hex #RRGGBB. Parsed once into ImageSharp Color values.

frames.json is hot-reloaded — changes apply without restart. A reload that contains an invalid frame logs a warning and skips that frame; valid frames continue to function. (Initial startup, by contrast, fails fast on any invalid frame.)

7. Storage

7.1 Location

PNGs are written to {Storage.ImageDirectory}/{normalized-mac}.png. The directory is created on startup if absent.

7.2 Atomicity

Writes occur via a temp file ({normalized-mac}.png.tmp) followed by an atomic rename. The fetch endpoint never returns a partially-written file.

7.3 Retention

Latest-only: each new write replaces the previous PNG for that frame. No history is kept.

8. URL-fetch guardrails

When fetching an image from a remote URL (§4.2), the service enforces:

• Maximum response size: UrlFetch.MaxBytes (default 50 MB). Exceeding aborts the download and returns 502.
• Request timeout: UrlFetch.TimeoutSeconds (default 30 s).
• Maximum redirect hops: UrlFetch.MaxRedirects (default 3).

The service does not restrict by Content-Type or destination network — it trusts that callers will not deliberately point at malicious or internal URLs. Decoding failures from non-image responses naturally surface as 400 errors.

9. Security model

The service is intended for trusted-network (LAN) deployment.

• All POST /api/* endpoints require a shared X-Api-Key header. The key is configured in appsettings.json (or via environment variable).
• GET /i/{mac}.png is unauthenticated. The MAC address in the URL is treated as a sufficiently unguessable identifier for hobby-scale use.
• GET /health is unauthenticated.
• The service does not implement HTTPS itself; deploy behind a reverse proxy if TLS is needed.

10. Observability

• Logging: structured logging via Serilog with console + rolling file sinks. Each upload logs the frame name, input source (file/url), input size, output size, total processing time, and MQTT publish status (success/queued-for-retry).
• Metrics: none in v1.
• Tracing: none in v1.

11. Non-functional requirements

Property	Target
Latency (1600×1200)	< 1 s end-to-end (upload → MQTT publish), excluding URL fetch
Concurrency	≥ 1 in-flight request per frame, unbounded across frames
Memory	Bounded by max input file size (50 MB) × concurrent requests
Restart behavior	Persisted PNGs survive restart; MQTT retry queue is in-memory and resets
Platform	.NET 10 on Linux/x64 and Linux/ARM64 (Docker)

12. Out of scope (v1)

• Multi-user authentication / user accounts.
• Per-frame image history or rollback.
• Image scheduling / playlists.
• Frame status reporting back to the service (battery, last-seen, etc.).
• A web UI.
• HTTPS termination inside the service.
• Internet-facing deployment hardening (SSRF protection, rate limiting).