Add saliency-based smart crop before resize
Replace the center-crop cover resize with a Sobel edge-density search that picks the target-aspect window with the most edge energy, so a subject off-center isn't sliced when the input aspect differs from the frame's. Failures fall back to the centered crop. Golden fixtures are regenerated and an asymmetric-edges input exercises the bias. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
No files matched your search
@@ -49,9 +49,12 @@ src/FrameProcessor/
|
||||
HealthController.cs // GET /health
|
||||
ImagePipeline/
|
||||
IImagePipeline.cs
|
||||
ImagePipeline.cs // scale → dither → remap → encode (indexed PNG)
|
||||
ImagePipeline.cs // smart-crop → resize → dither → remap → encode (indexed PNG)
|
||||
PaletteFactory.cs // builds ImageSharp ReadOnlyMemory<Color> from PaletteEntry list
|
||||
DitheringRegistry.cs // maps DitherAlgorithm → IDither
|
||||
SmartCrop/
|
||||
ISmartCropper.cs // computes a crop rect in source coordinates
|
||||
SaliencySmartCropper.cs // Sobel edge-density saliency search
|
||||
Mqtt/
|
||||
MqttPublisher.cs // persistent MQTTnet client, QoS 1, background retry queue
|
||||
Storage/
|
||||
@@ -154,19 +157,39 @@ Validation on load: name URL-safe, MAC parses to 6 bytes, palette has ≥2 entri
|
||||
For each upload:
|
||||
|
||||
1. **Load** the input via ImageSharp (`Image.Load<Rgba32>(stream)`).
|
||||
2. **Resize** with `ResizeMode.Crop` (center-crop cover) to the frame's resolution — swap W/H if `orientation == portrait`.
|
||||
3. **Dither** against a palette built from the already-parsed `DisplayColor` of each `PaletteEntry`, using the configured dither algorithm:
|
||||
2. **Smart crop** — `ISmartCropper.ComputeCropRect` picks a source-coordinate rectangle whose aspect matches the frame's oriented resolution (W/H swapped for portrait). If aspects already match within tolerance, the full source bounds is returned and no crop is applied. Otherwise the crop is applied via `Crop(rect)`. See "Smart cropping" below.
|
||||
3. **Resize** to the frame's oriented resolution. Aspects already match after the smart crop, so this is a pure scale (no cover/crop semantics needed).
|
||||
4. **Dither** against a palette built from the already-parsed `DisplayColor` of each `PaletteEntry`, using the configured dither algorithm:
|
||||
```csharp
|
||||
var displayPalette = frame.Palette.Select(p => p.DisplayColor).ToArray();
|
||||
img.Mutate(x => x.Dither(ditherings[frame.Dithering], displayPalette));
|
||||
```
|
||||
`frame.Dithering` is a `DitherAlgorithm` enum; `ditherings` is `IReadOnlyDictionary<DitherAlgorithm, IDither>`.
|
||||
4. **Remap** each pixel from `DisplayColor` → `DeviceColor`. Build a `Dictionary<Rgba32, Rgba32>` once per frame from the palette and do a single pixel-buffer pass. Since the dither in step 3 produces only palette colors, every pixel finds a mapping.
|
||||
5. **Rotate** 90° if `orientation == portrait` so the output PNG is always `width × height` as declared (e.g., 1600×1200), with content rotated for portrait mounting.
|
||||
6. **Encode** as **indexed PNG** (`PngColorType.Palette`, `PngBitDepth.Bit4` — fits 6 colors in a nibble). ImageSharp's `PngEncoder` with `Quantizer = new PaletteQuantizer(deviceColorPalette)` produces this in one call.
|
||||
7. **Atomic write**: `{tmpPath} → {finalPath}` via `File.Move(..., overwrite: true)` so the GET endpoint never sees a half-written file.
|
||||
5. **Remap** each pixel from `DisplayColor` → `DeviceColor`. Build a `Dictionary<Rgba32, Rgba32>` once per frame from the palette and do a single pixel-buffer pass. Since the dither in step 4 produces only palette colors, every pixel finds a mapping.
|
||||
6. **Rotate** 90° if `orientation == portrait` so the output PNG is always `width × height` as declared (e.g., 1600×1200), with content rotated for portrait mounting.
|
||||
7. **Encode** as **indexed PNG** (`PngColorType.Palette`, `PngBitDepth.Bit4` — fits 6 colors in a nibble). ImageSharp's `PngEncoder` with `Quantizer = new PaletteQuantizer(deviceColorPalette)` produces this in one call.
|
||||
8. **Atomic write**: `{tmpPath} → {finalPath}` via `File.Move(..., overwrite: true)` so the GET endpoint never sees a half-written file.
|
||||
|
||||
`IImagePipeline` is a single injected service. Encapsulates 1–6 and returns the bytes; `ImageStore` does step 7.
|
||||
`IImagePipeline` is a single injected service. Encapsulates 1–7 and returns the bytes; `ImageStore` does step 8.
|
||||
|
||||
### Smart cropping
|
||||
|
||||
A hard center crop slices the subject in half when going between landscape and portrait. The pipeline instead picks the crop window by Sobel edge density — edge-rich regions tend to coincide with the photo's subject (faces, in particular, contribute a lot of edges via eyes, nose, and hair).
|
||||
|
||||
Algorithm (`SaliencySmartCropper`):
|
||||
|
||||
1. If source aspect matches target aspect within ~0.5%, return full source bounds — no crop needed.
|
||||
2. Compute the **max-fitting centered rect** at the target aspect. This is the upper bound on crop size and the fallback if any later step throws.
|
||||
3. Downscale a clone of the source to **256 px on the long edge** (cheap analysis canvas).
|
||||
4. Compute per-pixel **Sobel magnitude** on Rec. 601 luma, build a **summed-area table** for O(1) rectangle sums.
|
||||
5. Slide a target-aspect window over the downscaled image at three sizes (100%, 85%, 70% of the max-fitting size) with an 8 px step, pick the position with maximum total edge energy.
|
||||
6. Scale the winning rect back to source coordinates, clamp to image bounds, return.
|
||||
|
||||
Failure mode: the whole `ComputeCropRect` is wrapped in `try/catch`; on any exception we log a warning and return the centered max-fitting rect. A bad photo must never 500 the upload — same doctrine as MQTT failure being non-fatal.
|
||||
|
||||
Determinism: pure managed integer/float math, no native deps, no ML model. Golden-image byte equality holds across runs.
|
||||
|
||||
No configuration knob in v1 — smart crop is always on, and same-aspect inputs hit the zero-cost fast path.
|
||||
|
||||
---
|
||||
|
||||
@@ -190,7 +213,7 @@ For each upload:
|
||||
|
||||
`tests/FrameProcessor.Tests` (xUnit + ImageSharp):
|
||||
|
||||
- **`ImagePipelineTests`** — golden-image fixtures. Check in 3-4 small test inputs (`landscape.jpg`, `portrait.jpg`, `tiny-1x1.png`, `non-divisible.jpg`) and expected outputs for one canonical Spectra 6 palette. Assert byte-equality of the indexed PNG output. ImageSharp dither output is deterministic for a given input/palette/algorithm, so this is a strong correctness anchor.
|
||||
- **`ImagePipelineTests`** — golden-image fixtures. Check in synthetic test inputs (`landscape.jpg`, `portrait.jpg`, `tiny-1x1.png`, `non-divisible.jpg`, `asymmetric-edges.jpg`) and expected outputs for one canonical Spectra 6 palette. Assert byte-equality of the indexed PNG output. The pipeline (dither + smart crop) is deterministic for a given input/palette/algorithm, so this is a strong correctness anchor. `asymmetric-edges.jpg` exercises the saliency search by placing a high-edge checkerboard on one half of the source and flat gray on the other — the smart crop must bias toward the edge-rich half.
|
||||
- **`PaletteFactoryTests`** — hex parsing, count, error messages on bad hex.
|
||||
- **`MacAddressTests`** — round-trip: `"AA:BB:CC:DD:EE:FF"`, `"aa-bb-cc-dd-ee-ff"`, `"AABBCCDDEEFF"`, and surrounding whitespace all parse to the same value; `ToString()` always emits the canonical lowercase-no-separator form; equality compares on the 6 bytes.
|
||||
- **`FrameNameTests`** — accepts URL-safe names, rejects names with reserved or whitespace characters with a clear error.
|
||||
|
||||
@@ -32,13 +32,14 @@ Both forms produce identical downstream behavior.
|
||||
For each accepted image, the service performs the following stages in order:
|
||||
|
||||
1. **Decode** the input into an in-memory RGBA bitmap.
|
||||
2. **Resize** to the frame's resolution using `cover` semantics — scale the input so it fully covers the target rectangle, then center-crop overflow. Aspect ratio is preserved; no stretching.
|
||||
3. **Dither** the bitmap against a palette built from the frame's `color` (display) values, using the dithering algorithm configured for that frame. Output of this stage contains only colors from the display palette.
|
||||
4. **Remap** each pixel from its display color to its corresponding `deviceColor` per the palette. Output of this stage contains only colors from the device-color palette.
|
||||
5. **Rotate** the bitmap 90° clockwise if the frame's orientation is `portrait`. The output dimensions always match the frame's declared resolution (e.g., 1600×1200) regardless of orientation.
|
||||
6. **Encode** as an **indexed PNG** with a palette of the frame's device colors, using the smallest bit depth that fits the palette size.
|
||||
7. **Persist** the resulting PNG to storage, replacing any previous image for the same frame **atomically** (no half-written file is ever observable via the HTTP fetch endpoint).
|
||||
8. **Notify** by publishing an MQTT message announcing that a new image is available.
|
||||
2. **Smart crop** to the frame's target aspect ratio. The service selects the source-coordinate crop rectangle by saliency (Sobel edge density): the rectangle of target aspect with the highest total edge energy is kept. This preserves the most visually interesting region of the source when the input aspect ratio differs from the frame's. If the input already matches the frame's aspect ratio (within a small tolerance), no cropping is performed. Smart-crop failure is non-fatal: on any internal error the service falls back to a centered max-fitting crop and logs a warning — a difficult photo never fails the upload.
|
||||
3. **Resize** the cropped bitmap to the frame's resolution. Aspect ratio is preserved by the preceding crop, so this stage is a pure scale.
|
||||
4. **Dither** the bitmap against a palette built from the frame's `color` (display) values, using the dithering algorithm configured for that frame. Output of this stage contains only colors from the display palette.
|
||||
5. **Remap** each pixel from its display color to its corresponding `deviceColor` per the palette. Output of this stage contains only colors from the device-color palette.
|
||||
6. **Rotate** the bitmap 90° clockwise if the frame's orientation is `portrait`. The output dimensions always match the frame's declared resolution (e.g., 1600×1200) regardless of orientation.
|
||||
7. **Encode** as an **indexed PNG** with a palette of the frame's device colors, using the smallest bit depth that fits the palette size.
|
||||
8. **Persist** the resulting PNG to storage, replacing any previous image for the same frame **atomically** (no half-written file is ever observable via the HTTP fetch endpoint).
|
||||
9. **Notify** by publishing an MQTT message announcing that a new image is available.
|
||||
|
||||
### 3.3 Output delivery
|
||||
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
using FrameProcessor.Domain;
|
||||
using FrameProcessor.ImagePipeline.SmartCrop;
|
||||
using SixLabors.ImageSharp;
|
||||
using SixLabors.ImageSharp.Formats.Png;
|
||||
using SixLabors.ImageSharp.PixelFormats;
|
||||
@@ -9,6 +10,13 @@ namespace FrameProcessor.ImagePipeline;
|
||||
|
||||
public sealed class ImagePipeline : IImagePipeline
|
||||
{
|
||||
private readonly ISmartCropper _smartCropper;
|
||||
|
||||
public ImagePipeline(ISmartCropper smartCropper)
|
||||
{
|
||||
_smartCropper = smartCropper;
|
||||
}
|
||||
|
||||
public byte[] Process(Stream input, Frame frame)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(input);
|
||||
@@ -21,13 +29,14 @@ public sealed class ImagePipeline : IImagePipeline
|
||||
|
||||
using var image = Image.Load<Rgba32>(input);
|
||||
|
||||
var crop = _smartCropper.ComputeCropRect(image, oriented.Width, oriented.Height);
|
||||
if (crop != image.Bounds)
|
||||
{
|
||||
image.Mutate(ctx => ctx.Crop(crop));
|
||||
}
|
||||
|
||||
image.Mutate(ctx => ctx
|
||||
.Resize(new ResizeOptions
|
||||
{
|
||||
Size = new Size(oriented.Width, oriented.Height),
|
||||
Mode = ResizeMode.Crop,
|
||||
Position = AnchorPositionMode.Center,
|
||||
})
|
||||
.Resize(oriented.Width, oriented.Height)
|
||||
.Dither(dither, displayPalette));
|
||||
|
||||
RemapDisplayToDevice(image, frame.Palette);
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
using SixLabors.ImageSharp;
|
||||
using SixLabors.ImageSharp.PixelFormats;
|
||||
|
||||
namespace FrameProcessor.ImagePipeline.SmartCrop;
|
||||
|
||||
/// <summary>
|
||||
/// Picks the source-coordinate crop rectangle to apply before resizing an
|
||||
/// input image to a frame's target resolution. Implementations may use
|
||||
/// saliency or other heuristics to keep the most interesting region.
|
||||
/// </summary>
|
||||
public interface ISmartCropper
|
||||
{
|
||||
/// <summary>
|
||||
/// Returns a crop rectangle in <paramref name="source"/>'s coordinate
|
||||
/// space whose aspect ratio matches <paramref name="targetWidth"/> :
|
||||
/// <paramref name="targetHeight"/>. If the source already matches the
|
||||
/// target aspect, returns the full source bounds.
|
||||
/// </summary>
|
||||
Rectangle ComputeCropRect(Image<Rgba32> source, int targetWidth, int targetHeight);
|
||||
}
|
||||
@@ -0,0 +1,216 @@
|
||||
using Microsoft.Extensions.Logging;
|
||||
using SixLabors.ImageSharp;
|
||||
using SixLabors.ImageSharp.PixelFormats;
|
||||
using SixLabors.ImageSharp.Processing;
|
||||
|
||||
namespace FrameProcessor.ImagePipeline.SmartCrop;
|
||||
|
||||
/// <summary>
|
||||
/// Smart cropper that picks the region with the highest Sobel edge density.
|
||||
/// Edge-rich regions tend to coincide with the photo's subject (including
|
||||
/// faces, which are high in edge content from eyes, nose and hair), so this
|
||||
/// is a face-detection-free proxy for "the interesting part of the image".
|
||||
/// </summary>
|
||||
public sealed class SaliencySmartCropper : ISmartCropper
|
||||
{
|
||||
private const double AspectMatchTolerance = 0.005;
|
||||
private const int SmallLongEdge = 256;
|
||||
private const int SlideStep = 8;
|
||||
private static readonly double[] CandidateScales = { 1.0, 0.85, 0.7 };
|
||||
|
||||
private readonly ILogger<SaliencySmartCropper> _logger;
|
||||
|
||||
public SaliencySmartCropper(ILogger<SaliencySmartCropper> logger)
|
||||
{
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
public Rectangle ComputeCropRect(Image<Rgba32> source, int targetWidth, int targetHeight)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(source);
|
||||
ArgumentOutOfRangeException.ThrowIfNegativeOrZero(targetWidth);
|
||||
ArgumentOutOfRangeException.ThrowIfNegativeOrZero(targetHeight);
|
||||
|
||||
try
|
||||
{
|
||||
return ComputeInternal(source, targetWidth, targetHeight);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogWarning(ex, "Smart crop failed; falling back to centered crop.");
|
||||
return MaxFittingCenteredRect(source.Width, source.Height, (double)targetWidth / targetHeight);
|
||||
}
|
||||
}
|
||||
|
||||
private static Rectangle ComputeInternal(Image<Rgba32> source, int targetWidth, int targetHeight)
|
||||
{
|
||||
var srcW = source.Width;
|
||||
var srcH = source.Height;
|
||||
var tgtAspect = (double)targetWidth / targetHeight;
|
||||
var srcAspect = (double)srcW / srcH;
|
||||
|
||||
if (Math.Abs(srcAspect - tgtAspect) / tgtAspect < AspectMatchTolerance)
|
||||
{
|
||||
return source.Bounds;
|
||||
}
|
||||
|
||||
var fallback = MaxFittingCenteredRect(srcW, srcH, tgtAspect);
|
||||
if (fallback.Width < 4 || fallback.Height < 4)
|
||||
{
|
||||
return fallback;
|
||||
}
|
||||
|
||||
var (smallW, smallH) = SmallDimensions(srcW, srcH);
|
||||
using var small = source.Clone(ctx => ctx.Resize(smallW, smallH));
|
||||
|
||||
var mag = ComputeSobelMagnitude(small);
|
||||
var sat = BuildSat(mag, smallW, smallH);
|
||||
|
||||
var smallMaxRect = MaxFittingCenteredRect(smallW, smallH, tgtAspect);
|
||||
var bestSmallRect = smallMaxRect;
|
||||
double bestEnergy = -1;
|
||||
|
||||
foreach (var scale in CandidateScales)
|
||||
{
|
||||
var winW = (int)Math.Round(smallMaxRect.Width * scale);
|
||||
var winH = (int)Math.Round(smallMaxRect.Height * scale);
|
||||
if (winW < 2 || winH < 2 || winW > smallW || winH > smallH)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
var maxX = smallW - winW;
|
||||
var maxY = smallH - winH;
|
||||
var stride = smallW + 1;
|
||||
|
||||
for (var y = 0; y <= maxY; y += SlideStep)
|
||||
{
|
||||
for (var x = 0; x <= maxX; x += SlideStep)
|
||||
{
|
||||
var energy = SumRect(sat, stride, x, y, x + winW, y + winH);
|
||||
if (energy > bestEnergy)
|
||||
{
|
||||
bestEnergy = energy;
|
||||
bestSmallRect = new Rectangle(x, y, winW, winH);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (bestEnergy < 0)
|
||||
{
|
||||
return fallback;
|
||||
}
|
||||
|
||||
var sx = (double)srcW / smallW;
|
||||
var sy = (double)srcH / smallH;
|
||||
var x0 = (int)Math.Round(bestSmallRect.X * sx);
|
||||
var y0 = (int)Math.Round(bestSmallRect.Y * sy);
|
||||
var x1 = (int)Math.Round((bestSmallRect.X + bestSmallRect.Width) * sx);
|
||||
var y1 = (int)Math.Round((bestSmallRect.Y + bestSmallRect.Height) * sy);
|
||||
|
||||
x0 = Math.Clamp(x0, 0, srcW - 1);
|
||||
y0 = Math.Clamp(y0, 0, srcH - 1);
|
||||
x1 = Math.Clamp(x1, x0 + 1, srcW);
|
||||
y1 = Math.Clamp(y1, y0 + 1, srcH);
|
||||
|
||||
return new Rectangle(x0, y0, x1 - x0, y1 - y0);
|
||||
}
|
||||
|
||||
private static (int W, int H) SmallDimensions(int srcW, int srcH)
|
||||
{
|
||||
if (srcW >= srcH)
|
||||
{
|
||||
var w = Math.Min(SmallLongEdge, srcW);
|
||||
var h = Math.Max(1, (int)Math.Round((double)srcH * w / srcW));
|
||||
return (w, h);
|
||||
}
|
||||
else
|
||||
{
|
||||
var h = Math.Min(SmallLongEdge, srcH);
|
||||
var w = Math.Max(1, (int)Math.Round((double)srcW * h / srcH));
|
||||
return (w, h);
|
||||
}
|
||||
}
|
||||
|
||||
private static Rectangle MaxFittingCenteredRect(int srcW, int srcH, double tgtAspect)
|
||||
{
|
||||
var srcAspect = (double)srcW / srcH;
|
||||
int cropW, cropH;
|
||||
if (srcAspect >= tgtAspect)
|
||||
{
|
||||
cropH = srcH;
|
||||
cropW = Math.Min(srcW, (int)Math.Round(srcH * tgtAspect));
|
||||
}
|
||||
else
|
||||
{
|
||||
cropW = srcW;
|
||||
cropH = Math.Min(srcH, (int)Math.Round(srcW / tgtAspect));
|
||||
}
|
||||
|
||||
var x = (srcW - cropW) / 2;
|
||||
var y = (srcH - cropH) / 2;
|
||||
return new Rectangle(x, y, cropW, cropH);
|
||||
}
|
||||
|
||||
private static float[] ComputeSobelMagnitude(Image<Rgba32> img)
|
||||
{
|
||||
var w = img.Width;
|
||||
var h = img.Height;
|
||||
var lum = new byte[w * h];
|
||||
|
||||
img.ProcessPixelRows(accessor =>
|
||||
{
|
||||
for (var y = 0; y < accessor.Height; y++)
|
||||
{
|
||||
var row = accessor.GetRowSpan(y);
|
||||
var off = y * w;
|
||||
for (var x = 0; x < row.Length; x++)
|
||||
{
|
||||
var p = row[x];
|
||||
// Rec. 601 luma (integer-fixed to keep determinism across runtimes).
|
||||
lum[off + x] = (byte)((p.R * 299 + p.G * 587 + p.B * 114 + 500) / 1000);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
var mag = new float[w * h];
|
||||
for (var y = 1; y < h - 1; y++)
|
||||
{
|
||||
for (var x = 1; x < w - 1; x++)
|
||||
{
|
||||
var i = y * w + x;
|
||||
int a = lum[i - w - 1], b = lum[i - w], c = lum[i - w + 1];
|
||||
int d = lum[i - 1], e = lum[i + 1];
|
||||
int f = lum[i + w - 1], g = lum[i + w], hh = lum[i + w + 1];
|
||||
|
||||
var gx = (c + 2 * e + hh) - (a + 2 * d + f);
|
||||
var gy = (f + 2 * g + hh) - (a + 2 * b + c);
|
||||
mag[i] = Math.Abs(gx) + Math.Abs(gy);
|
||||
}
|
||||
}
|
||||
return mag;
|
||||
}
|
||||
|
||||
private static double[] BuildSat(float[] mag, int w, int h)
|
||||
{
|
||||
var stride = w + 1;
|
||||
var sat = new double[stride * (h + 1)];
|
||||
for (var y = 0; y < h; y++)
|
||||
{
|
||||
double rowSum = 0;
|
||||
for (var x = 0; x < w; x++)
|
||||
{
|
||||
rowSum += mag[y * w + x];
|
||||
sat[(y + 1) * stride + (x + 1)] = sat[y * stride + (x + 1)] + rowSum;
|
||||
}
|
||||
}
|
||||
return sat;
|
||||
}
|
||||
|
||||
private static double SumRect(double[] sat, int stride, int x0, int y0, int x1, int y1)
|
||||
=> sat[y1 * stride + x1]
|
||||
- sat[y0 * stride + x1]
|
||||
- sat[y1 * stride + x0]
|
||||
+ sat[y0 * stride + x0];
|
||||
}
|
||||
@@ -2,6 +2,7 @@ using System.Net;
|
||||
using FrameProcessor.Concurrency;
|
||||
using FrameProcessor.Configuration;
|
||||
using FrameProcessor.ImagePipeline;
|
||||
using FrameProcessor.ImagePipeline.SmartCrop;
|
||||
using FrameProcessor.Middleware;
|
||||
using FrameProcessor.Mqtt;
|
||||
using FrameProcessor.Storage;
|
||||
@@ -150,6 +151,7 @@ builder.Services.AddOptions<FramesOptions>()
|
||||
.Bind(builder.Configuration);
|
||||
builder.Services.AddSingleton<FramesOptionsValidator>();
|
||||
builder.Services.AddSingleton<FramesRegistry>();
|
||||
builder.Services.AddSingleton<ISmartCropper, SaliencySmartCropper>();
|
||||
builder.Services.AddSingleton<IImagePipeline, FrameProcessor.ImagePipeline.ImagePipeline>();
|
||||
builder.Services.AddSingleton<ImageStore>();
|
||||
builder.Services.AddSingleton<MqttPublisher>();
|
||||
|
||||
|
After Width: | Height: | Size: 115 KiB |
|
Before Width: | Height: | Size: 252 KiB After Width: | Height: | Size: 251 KiB |
|
Before Width: | Height: | Size: 234 KiB After Width: | Height: | Size: 235 KiB |
|
Before Width: | Height: | Size: 259 KiB After Width: | Height: | Size: 230 KiB |
|
After Width: | Height: | Size: 101 KiB |
|
Before Width: | Height: | Size: 266 KiB After Width: | Height: | Size: 269 KiB |
|
Before Width: | Height: | Size: 248 KiB After Width: | Height: | Size: 248 KiB |
|
Before Width: | Height: | Size: 261 KiB After Width: | Height: | Size: 258 KiB |
|
After Width: | Height: | Size: 162 KiB |
|
Before Width: | Height: | Size: 308 KiB After Width: | Height: | Size: 308 KiB |
|
Before Width: | Height: | Size: 274 KiB After Width: | Height: | Size: 273 KiB |
|
Before Width: | Height: | Size: 288 KiB After Width: | Height: | Size: 248 KiB |
|
After Width: | Height: | Size: 143 KiB |
|
Before Width: | Height: | Size: 293 KiB After Width: | Height: | Size: 296 KiB |
|
Before Width: | Height: | Size: 268 KiB After Width: | Height: | Size: 266 KiB |
|
Before Width: | Height: | Size: 280 KiB After Width: | Height: | Size: 246 KiB |
|
After Width: | Height: | Size: 34 KiB |
@@ -1,6 +1,8 @@
|
||||
using System.Runtime.CompilerServices;
|
||||
using FrameProcessor.Domain;
|
||||
using FrameProcessor.ImagePipeline;
|
||||
using FrameProcessor.ImagePipeline.SmartCrop;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using SixLabors.ImageSharp;
|
||||
using SixLabors.ImageSharp.Formats.Jpeg;
|
||||
using SixLabors.ImageSharp.Formats.Png;
|
||||
@@ -35,8 +37,12 @@ public class ImagePipelineTests
|
||||
"portrait.jpg",
|
||||
"tiny-1x1.png",
|
||||
"non-divisible.jpg",
|
||||
"asymmetric-edges.jpg",
|
||||
};
|
||||
|
||||
private static FrameProcessor.ImagePipeline.ImagePipeline CreatePipeline()
|
||||
=> new(new SaliencySmartCropper(NullLogger<SaliencySmartCropper>.Instance));
|
||||
|
||||
public static IEnumerable<object[]> AllCombinations()
|
||||
{
|
||||
foreach (var algo in Enum.GetValues<DitherAlgorithm>())
|
||||
@@ -61,7 +67,7 @@ public class ImagePipelineTests
|
||||
$"Missing golden fixture: {expectedPath}. Regenerate by running 'UPDATE_GOLDEN=1 dotnet test'.");
|
||||
|
||||
var frame = LandscapeFrame with { Dithering = algorithm };
|
||||
var pipeline = new FrameProcessor.ImagePipeline.ImagePipeline();
|
||||
var pipeline = CreatePipeline();
|
||||
using var stream = File.OpenRead(inputPath);
|
||||
var actual = pipeline.Process(stream, frame);
|
||||
var expected = File.ReadAllBytes(expectedPath);
|
||||
@@ -79,7 +85,7 @@ public class ImagePipelineTests
|
||||
|
||||
WriteInputFixtures();
|
||||
|
||||
var pipeline = new FrameProcessor.ImagePipeline.ImagePipeline();
|
||||
var pipeline = CreatePipeline();
|
||||
foreach (var algorithm in Enum.GetValues<DitherAlgorithm>())
|
||||
{
|
||||
foreach (var input in InputNames)
|
||||
@@ -106,6 +112,45 @@ public class ImagePipelineTests
|
||||
WriteJpeg(Path.Combine(inputsDir, "portrait.jpg"), 1200, 1600);
|
||||
WriteJpeg(Path.Combine(inputsDir, "non-divisible.jpg"), 1234, 789);
|
||||
WriteTinyPng(Path.Combine(inputsDir, "tiny-1x1.png"));
|
||||
WriteAsymmetricEdges(Path.Combine(inputsDir, "asymmetric-edges.jpg"));
|
||||
}
|
||||
|
||||
private static void WriteAsymmetricEdges(string path)
|
||||
{
|
||||
// 1600x900 (aspect ~1.78) processed by a 1600x1200 landscape frame
|
||||
// (aspect ~1.33) requires horizontal cropping. Left half is flat gray
|
||||
// (low edge energy); right half is an 8x8 black/white checkerboard
|
||||
// (high edge energy). Saliency should bias the crop window toward the
|
||||
// checkerboard side.
|
||||
const int width = 1600;
|
||||
const int height = 900;
|
||||
using var image = new Image<Rgba32>(width, height);
|
||||
var gray = new Rgba32(0x80, 0x80, 0x80, 0xFF);
|
||||
var black = new Rgba32(0x00, 0x00, 0x00, 0xFF);
|
||||
var white = new Rgba32(0xFF, 0xFF, 0xFF, 0xFF);
|
||||
image.ProcessPixelRows(accessor =>
|
||||
{
|
||||
for (var y = 0; y < accessor.Height; y++)
|
||||
{
|
||||
var row = accessor.GetRowSpan(y);
|
||||
for (var x = 0; x < row.Length; x++)
|
||||
{
|
||||
if (x < width / 2)
|
||||
{
|
||||
row[x] = gray;
|
||||
}
|
||||
else
|
||||
{
|
||||
var cellX = (x - width / 2) / 8;
|
||||
var cellY = y / 8;
|
||||
row[x] = ((cellX + cellY) & 1) == 0 ? black : white;
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
var encoder = new JpegEncoder { Quality = 90 };
|
||||
using var fs = File.Create(path);
|
||||
image.SaveAsJpeg(fs, encoder);
|
||||
}
|
||||
|
||||
private static void WriteJpeg(string path, int width, int height)
|
||||
|
||||