7b928983
Add GPU render testing implementation plan
a73x 2026-04-17 11:05
Eleven-task plan covering PNG codec, offscreen render target, --capture mode, imgdiff tool, VT scripts, test orchestrator, golden generation, and bench baseline/check. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
diff --git a/docs/superpowers/plans/2026-04-17-gpu-render-testing.md b/docs/superpowers/plans/2026-04-17-gpu-render-testing.md new file mode 100644 index 0000000..b1c8ac4 --- /dev/null +++ b/docs/superpowers/plans/2026-04-17-gpu-render-testing.md @@ -0,0 +1,2096 @@ # GPU Render Testing Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Add an automated pipeline that renders VT scripts through the real Vulkan pipeline, captures the output via an offscreen image readback, diffs it against golden PNGs, and flags both visual and performance regressions. All local-only (no CI). **Architecture:** A `--capture <script> <output.png>` mode in waystty renders a scripted terminal session into a dedicated offscreen `VkImage`, reads that image back to host memory, and writes it as PNG. A `imgdiff` tool does RMSE + per-pixel-max comparison. A `test-render` orchestrator iterates golden scripts. A `bench-baseline` / `bench-check` pair compares p99 frame timings against a stored baseline. **Tech Stack:** Zig 0.15+, Vulkan 1.2 (via vulkan-zig bindings), zig-wayland, vendored minimal PNG codec. No new external deps. **Spec:** `docs/superpowers/specs/2026-04-17-gpu-render-testing-design.md` --- ## File Structure New files: - `src/png.zig` — vendored minimal RGBA8 PNG encoder/decoder (used by renderer + imgdiff) - `src/capture.zig` — capture mode flow (visibility wait, drain, settle, render, readback, PNG write) - `src/tools/imgdiff.zig` — standalone executable: compare two PNGs - `src/tools/test_render.zig` — standalone executable: orchestrate capture + diff across all scripts - `src/tools/bench_baseline.zig` — standalone executable: run bench workload, write/compare baseline.json - `src/bench_stats.zig` — shared JSON serialization for FrameTimingStats (used by runTerminal and bench_baseline) - `tests/golden/scripts/basic_ascii.vt` - `tests/golden/scripts/bold_colors.vt` - `tests/golden/scripts/box_drawing.vt` - `tests/golden/reference/*.png` (committed after Task 8) - `tests/bench/baseline.json` (committed after Task 10) Modified files: - `src/renderer.zig` — add `OffscreenTarget` struct + `renderToOffscreen()` + `readbackOffscreen()` - `src/main.zig` — dispatch `--capture` arg to `capture.run`; extract `computeFrameStats` + `printFrameStats` to share with bench tools (expose via `bench_stats.zig`) - `build.zig` — new modules (png, capture, bench_stats), new executables (imgdiff, test_render, bench_baseline), new build steps - `Makefile` — new targets (`test-render`, `golden-update`, `bench-baseline`, `bench-check`) - `.gitignore` — ignore stray `test_io*`/`test_sig`/`test_timer` binaries and `tests/golden/output/` --- ## Task 1: Vendored PNG codec (`src/png.zig`) Minimal RGBA8 PNG encoder + decoder. Used by renderer (write capture output) and imgdiff (read reference + actual). Writing zlib/DEFLATE from scratch is tedious, but Zig ships `std.compress.flate` which handles DEFLATE, and the PNG IDAT chunk is a zlib stream (DEFLATE + adler32 + zlib header). We use `std.compress.flate` for the DEFLATE core and wrap it with the zlib header + adler32 ourselves, plus CRC32 for chunk framing. **Files:** - Create: `src/png.zig` - Create: `src/png_test.zig` (only if a separate test file helps; otherwise tests inline) - Modify: `build.zig` (add png module) - [ ] **Step 1: Write the failing test for encode→decode round-trip** Create `src/png.zig`: ```zig const std = @import("std"); pub const Image = struct { width: u32, height: u32, pixels: []u8, // RGBA8, row-major, width*height*4 bytes pub fn deinit(self: *Image, alloc: std.mem.Allocator) void { alloc.free(self.pixels); self.* = undefined; } }; pub const EncodeError = error{ OutOfMemory, WriteFailed }; pub const DecodeError = error{ OutOfMemory, InvalidPng, UnsupportedPng, // only RGBA8 non-interlaced is supported CorruptChunk, }; pub fn encode(alloc: std.mem.Allocator, img: Image, writer: anytype) EncodeError!void { _ = alloc; _ = img; _ = writer; return error.WriteFailed; // placeholder — Step 3 replaces this } pub fn decode(alloc: std.mem.Allocator, bytes: []const u8) DecodeError!Image { _ = alloc; _ = bytes; return error.InvalidPng; // placeholder — Step 3 replaces this } test "encode then decode roundtrip recovers pixels" { const alloc = std.testing.allocator; var src_pixels = [_]u8{ 0xff, 0x00, 0x00, 0xff, 0x00, 0xff, 0x00, 0xff, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, }; const src = Image{ .width = 2, .height = 2, .pixels = &src_pixels }; var buf = std.ArrayList(u8).init(alloc); defer buf.deinit(); try encode(alloc, src, buf.writer()); var decoded = try decode(alloc, buf.items); defer decoded.deinit(alloc); try std.testing.expectEqual(@as(u32, 2), decoded.width); try std.testing.expectEqual(@as(u32, 2), decoded.height); try std.testing.expectEqualSlices(u8, &src_pixels, decoded.pixels); } ``` Add to `build.zig` (after the `renderer_test_mod` block near the end of the file): ```zig // png module — vendored minimal RGBA8 PNG codec const png_mod = b.createModule(.{ .root_source_file = b.path("src/png.zig"), .target = target, .optimize = optimize, }); exe_mod.addImport("png", png_mod); const png_test_mod = b.createModule(.{ .root_source_file = b.path("src/png.zig"), .target = target, .optimize = optimize, }); const png_tests = b.addTest(.{ .root_module = png_test_mod }); test_step.dependOn(&b.addRunArtifact(png_tests).step); ``` - [ ] **Step 2: Run the test to confirm it fails** Run: `zig build test 2>&1 | grep -E "png|roundtrip"` Expected: failure mentioning `error.WriteFailed` or `error.InvalidPng`. - [ ] **Step 3: Implement PNG encoder** Replace the `encode` function in `src/png.zig` with a real implementation. The PNG structure is: ``` signature: 8 bytes 89 50 4E 47 0D 0A 1A 0A IHDR chunk: 13 bytes payload: width(4) height(4) bitdepth=8 colortype=6(RGBA) compression=0 filter=0 interlace=0 IDAT chunk: zlib-wrapped DEFLATE of (filter_byte=0 + row_pixels) per row IEND chunk: empty payload ``` Each chunk is: length(4) type(4) data(length) crc32(4). The zlib wrapping is: header(2) deflated_data adler32(4). Put the full implementation in place: ```zig const signature = [_]u8{ 0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A }; fn crc32(data: []const u8) u32 { var crc = std.hash.Crc32.init(); crc.update(data); return crc.final(); } fn adler32(data: []const u8) u32 { var a: u32 = 1; var b: u32 = 0; for (data) |byte| { a = (a + byte) % 65521; b = (b + a) % 65521; } return (b << 16) | a; } fn writeChunk(writer: anytype, chunk_type: *const [4]u8, payload: []const u8) EncodeError!void { writer.writeInt(u32, @intCast(payload.len), .big) catch return error.WriteFailed; writer.writeAll(chunk_type) catch return error.WriteFailed; writer.writeAll(payload) catch return error.WriteFailed; // CRC is over chunk_type + payload var crc = std.hash.Crc32.init(); crc.update(chunk_type); crc.update(payload); writer.writeInt(u32, crc.final(), .big) catch return error.WriteFailed; } pub fn encode(alloc: std.mem.Allocator, img: Image, writer: anytype) EncodeError!void { std.debug.assert(img.pixels.len == @as(usize, img.width) * img.height * 4); writer.writeAll(&signature) catch return error.WriteFailed; // IHDR var ihdr: [13]u8 = undefined; std.mem.writeInt(u32, ihdr[0..4], img.width, .big); std.mem.writeInt(u32, ihdr[4..8], img.height, .big); ihdr[8] = 8; // bit depth ihdr[9] = 6; // color type = RGBA ihdr[10] = 0; // compression ihdr[11] = 0; // filter ihdr[12] = 0; // interlace try writeChunk(writer, "IHDR", &ihdr); // Build filtered rows: one filter byte (0 = none) followed by each row const row_bytes = @as(usize, img.width) * 4; const filtered_len = (row_bytes + 1) * img.height; const filtered = alloc.alloc(u8, filtered_len) catch return error.OutOfMemory; defer alloc.free(filtered); var y: u32 = 0; while (y < img.height) : (y += 1) { const src_off = @as(usize, y) * row_bytes; const dst_off = @as(usize, y) * (row_bytes + 1); filtered[dst_off] = 0; @memcpy(filtered[dst_off + 1 .. dst_off + 1 + row_bytes], img.pixels[src_off .. src_off + row_bytes]); } // Compress with DEFLATE, wrap with zlib header + adler32 var compressed = std.ArrayList(u8).init(alloc); defer compressed.deinit(); // zlib header: 0x78 0x01 (deflate, no preset dict, fastest) compressed.appendSlice(&.{ 0x78, 0x01 }) catch return error.OutOfMemory; var fbs = std.io.fixedBufferStream(filtered); std.compress.flate.deflate.compress(.raw, fbs.reader(), compressed.writer(), .{}) catch return error.WriteFailed; var adler_bytes: [4]u8 = undefined; std.mem.writeInt(u32, &adler_bytes, adler32(filtered), .big); compressed.appendSlice(&adler_bytes) catch return error.OutOfMemory; try writeChunk(writer, "IDAT", compressed.items); try writeChunk(writer, "IEND", &.{}); } ``` Note: the exact `std.compress.flate` API may differ slightly across Zig versions. If it does, substitute the equivalent raw-deflate writer in current Zig 0.15+ std. The rest of the PNG framing is version-independent. - [ ] **Step 4: Implement PNG decoder** Replace `decode` in `src/png.zig`: ```zig pub fn decode(alloc: std.mem.Allocator, bytes: []const u8) DecodeError!Image { if (bytes.len < signature.len + 8) return error.InvalidPng; if (!std.mem.eql(u8, bytes[0..signature.len], &signature)) return error.InvalidPng; var cursor: usize = signature.len; var width: u32 = 0; var height: u32 = 0; var idat_accum = std.ArrayList(u8).init(alloc); defer idat_accum.deinit(); var seen_ihdr = false; var seen_iend = false; while (cursor + 8 <= bytes.len and !seen_iend) { const len = std.mem.readInt(u32, bytes[cursor..][0..4], .big); cursor += 4; const ctype = bytes[cursor..][0..4]; cursor += 4; if (cursor + len + 4 > bytes.len) return error.CorruptChunk; const payload = bytes[cursor..][0..len]; cursor += len; // skip CRC (4 bytes); we trust the file cursor += 4; if (std.mem.eql(u8, ctype, "IHDR")) { if (payload.len != 13) return error.InvalidPng; width = std.mem.readInt(u32, payload[0..4], .big); height = std.mem.readInt(u32, payload[4..8], .big); if (payload[8] != 8 or payload[9] != 6 or payload[12] != 0) return error.UnsupportedPng; seen_ihdr = true; } else if (std.mem.eql(u8, ctype, "IDAT")) { if (!seen_ihdr) return error.InvalidPng; idat_accum.appendSlice(payload) catch return error.OutOfMemory; } else if (std.mem.eql(u8, ctype, "IEND")) { seen_iend = true; } // ignore other chunks } if (!seen_ihdr or !seen_iend) return error.InvalidPng; // Strip zlib header (2 bytes) + adler32 (last 4 bytes) if (idat_accum.items.len < 6) return error.InvalidPng; const deflate_data = idat_accum.items[2 .. idat_accum.items.len - 4]; const row_bytes = @as(usize, width) * 4; const filtered_len = (row_bytes + 1) * height; const filtered = alloc.alloc(u8, filtered_len) catch return error.OutOfMemory; defer alloc.free(filtered); var src_fbs = std.io.fixedBufferStream(deflate_data); var dst_fbs = std.io.fixedBufferStream(filtered); std.compress.flate.deflate.decompress(.raw, src_fbs.reader(), dst_fbs.writer()) catch return error.CorruptChunk; if (dst_fbs.pos != filtered_len) return error.CorruptChunk; const pixels = alloc.alloc(u8, @as(usize, width) * height * 4) catch return error.OutOfMemory; errdefer alloc.free(pixels); // Only filter type 0 (None) supported; fail on anything else var y: u32 = 0; while (y < height) : (y += 1) { const dst_off = @as(usize, y) * row_bytes; const src_off = @as(usize, y) * (row_bytes + 1); if (filtered[src_off] != 0) return error.UnsupportedPng; @memcpy(pixels[dst_off .. dst_off + row_bytes], filtered[src_off + 1 .. src_off + 1 + row_bytes]); } return .{ .width = width, .height = height, .pixels = pixels }; } ``` - [ ] **Step 5: Run the roundtrip test** Run: `zig build test 2>&1 | grep -E "png|roundtrip"` Expected: PASS. If failure references the DEFLATE API, swap in the correct `std.compress.flate` call for the current Zig version and re-run. - [ ] **Step 6: Add a decode-rejects-non-rgba8 test** Append to `src/png.zig`: ```zig test "decode rejects RGB (non-alpha) PNGs with UnsupportedPng" { const alloc = std.testing.allocator; // Craft a minimal valid PNG with color type = 2 (RGB) var bytes = std.ArrayList(u8).init(alloc); defer bytes.deinit(); try bytes.appendSlice(&signature); // IHDR var ihdr: [13]u8 = undefined; std.mem.writeInt(u32, ihdr[0..4], 1, .big); std.mem.writeInt(u32, ihdr[4..8], 1, .big); ihdr[8] = 8; ihdr[9] = 2; ihdr[10] = 0; ihdr[11] = 0; ihdr[12] = 0; try writeChunk(bytes.writer(), "IHDR", &ihdr); try writeChunk(bytes.writer(), "IEND", &.{}); try std.testing.expectError(error.UnsupportedPng, decode(alloc, bytes.items)); } ``` Run: `zig build test` Expected: both tests PASS. - [ ] **Step 7: Commit** ```bash git add src/png.zig build.zig git commit -m "$(cat <<'EOF' Add vendored minimal RGBA8 PNG codec Supports only the narrow slice we need (RGBA8 non-interlaced with filter type 0). Used by --capture for output and by imgdiff for reading goldens. EOF )" ``` --- ## Task 2: Offscreen render target in `renderer.zig` Add a dedicated `VkImage` the renderer can draw into (instead of the swapchain) and a readback path that copies it to a host-visible staging buffer. The existing renderer pipeline already writes to whatever framebuffer we bind during the render pass — we just need to construct a framebuffer over this new image and have a way to kick the draw and read the pixels. We keep the swapchain path untouched — the offscreen target is created on demand by capture mode. **Files:** - Modify: `src/renderer.zig` — add `OffscreenTarget` struct, `createOffscreen()`, `destroyOffscreen()`, `renderToOffscreen()`, `readbackOffscreen()` - [ ] **Step 1: Find the right insertion point in renderer.zig** Read `src/renderer.zig` to find (a) the `Context` struct definition and (b) the `drawCells` function. The new offscreen helpers will be a set of methods on `Context` (or free functions taking the Vulkan wrappers) that mirror what `drawCells` does but render to an owned image. Run: `grep -n "pub const Context\|pub fn drawCells\|pub fn uploadAtlas\|pub fn deinit" src/renderer.zig` Expected: locations of each. Use these when editing. - [ ] **Step 2: Add the `OffscreenTarget` type and constructor** In `src/renderer.zig`, after the `SwapchainResult` struct, add: ```zig pub const OffscreenTarget = struct { width: u32, height: u32, format: vk.Format, image: vk.Image, memory: vk.DeviceMemory, view: vk.ImageView, framebuffer: vk.Framebuffer, /// Host-visible buffer for readback. Large enough for width*height*4 BGRA bytes. readback_buffer: vk.Buffer, readback_memory: vk.DeviceMemory, readback_size: u64, }; pub fn createOffscreen( vki: vk.InstanceWrapper, vkd: vk.DeviceWrapper, physical: vk.PhysicalDevice, device: vk.Device, render_pass: vk.RenderPass, format: vk.Format, width: u32, height: u32, ) !OffscreenTarget { // 1. Color attachment image (TRANSFER_SRC for readback) const image = try vkd.createImage(device, &vk.ImageCreateInfo{ .image_type = .@"2d", .format = format, .extent = .{ .width = width, .height = height, .depth = 1 }, .mip_levels = 1, .array_layers = 1, .samples = .{ .@"1_bit" = true }, .tiling = .optimal, .usage = .{ .color_attachment_bit = true, .transfer_src_bit = true }, .sharing_mode = .exclusive, .initial_layout = .undefined, }, null); const img_mem_req = vkd.getImageMemoryRequirements(device, image); const mem_props = vki.getPhysicalDeviceMemoryProperties(physical); const img_mem_type = findMemoryType(mem_props, img_mem_req.memory_type_bits, .{ .device_local_bit = true }) orelse return error.NoMemoryType; const memory = try vkd.allocateMemory(device, &vk.MemoryAllocateInfo{ .allocation_size = img_mem_req.size, .memory_type_index = img_mem_type, }, null); try vkd.bindImageMemory(device, image, memory, 0); const view = try vkd.createImageView(device, &vk.ImageViewCreateInfo{ .image = image, .view_type = .@"2d", .format = format, .components = .{ .r = .identity, .g = .identity, .b = .identity, .a = .identity }, .subresource_range = .{ .aspect_mask = .{ .color_bit = true }, .base_mip_level = 0, .level_count = 1, .base_array_layer = 0, .layer_count = 1, }, }, null); const framebuffer = try vkd.createFramebuffer(device, &vk.FramebufferCreateInfo{ .render_pass = render_pass, .attachment_count = 1, .p_attachments = @ptrCast(&view), .width = width, .height = height, .layers = 1, }, null); // 2. Host-visible readback buffer const readback_size: u64 = @as(u64, width) * height * 4; const readback_buffer = try vkd.createBuffer(device, &vk.BufferCreateInfo{ .size = readback_size, .usage = .{ .transfer_dst_bit = true }, .sharing_mode = .exclusive, }, null); const buf_mem_req = vkd.getBufferMemoryRequirements(device, readback_buffer); const buf_mem_type = findMemoryType(mem_props, buf_mem_req.memory_type_bits, .{ .host_visible_bit = true, .host_coherent_bit = true, }) orelse return error.NoMemoryType; const readback_memory = try vkd.allocateMemory(device, &vk.MemoryAllocateInfo{ .allocation_size = buf_mem_req.size, .memory_type_index = buf_mem_type, }, null); try vkd.bindBufferMemory(device, readback_buffer, readback_memory, 0); return .{ .width = width, .height = height, .format = format, .image = image, .memory = memory, .view = view, .framebuffer = framebuffer, .readback_buffer = readback_buffer, .readback_memory = readback_memory, .readback_size = readback_size, }; } pub fn destroyOffscreen(vkd: vk.DeviceWrapper, device: vk.Device, t: OffscreenTarget) void { vkd.destroyFramebuffer(device, t.framebuffer, null); vkd.destroyImageView(device, t.view, null); vkd.destroyImage(device, t.image, null); vkd.freeMemory(device, t.memory, null); vkd.destroyBuffer(device, t.readback_buffer, null); vkd.freeMemory(device, t.readback_memory, null); } fn findMemoryType( props: vk.PhysicalDeviceMemoryProperties, type_filter: u32, required: vk.MemoryPropertyFlags, ) ?u32 { var i: u32 = 0; while (i < props.memory_type_count) : (i += 1) { const bit = @as(u32, 1) << @intCast(i); if (type_filter & bit == 0) continue; const flags = props.memory_types[i].property_flags; if (flags.contains(required)) return i; } return null; } ``` Note: if `Context` already has a `findMemoryType` helper, reuse it and delete the duplicate above. - [ ] **Step 3: Add `renderToOffscreen` and `readbackOffscreen` on Context** The existing `drawCells` in `Context` submits draws against the current swapchain image. We need an analogous method that draws into the offscreen framebuffer, transitions the image to `TRANSFER_SRC_OPTIMAL`, and copies into the readback buffer. Strategy: refactor `drawCells` minimally so its render-pass body (begin pass → bind pipeline → bind descriptor sets → draw → end pass) is in a helper taking an explicit framebuffer + extent, and have both the swapchain path and the offscreen path call it. In `src/renderer.zig`, add these methods on `Context` (put them next to `drawCells`): ```zig pub fn renderToOffscreen( self: *Context, target: *const OffscreenTarget, instance_data: []const CellInstance, // same struct drawCells uses push: PushConstants, // same struct drawCells uses ) !void { // 1. Upload instances to the existing instance buffer (same as drawCells does). try self.uploadInstances(instance_data); // 2. Begin command buffer, transition offscreen image UNDEFINED → COLOR_ATTACHMENT_OPTIMAL const cmd = self.capture_cmd; // new: a dedicated command buffer allocated alongside the swapchain cmd buffer try self.vkd.resetCommandBuffer(cmd, .{}); try self.vkd.beginCommandBuffer(cmd, &.{ .flags = .{ .one_time_submit_bit = true } }); self.transitionImage(cmd, target.image, .undefined, .color_attachment_optimal, .{}, .{ .color_attachment_write_bit = true }, .{ .top_of_pipe_bit = true }, .{ .color_attachment_output_bit = true }); // 3. Begin the same render pass as swapchain but with the offscreen framebuffer + extent const clear: vk.ClearValue = .{ .color = .{ .float_32 = .{ 0, 0, 0, 1 } } }; self.vkd.cmdBeginRenderPass(cmd, &vk.RenderPassBeginInfo{ .render_pass = self.render_pass, .framebuffer = target.framebuffer, .render_area = .{ .offset = .{ .x = 0, .y = 0 }, .extent = .{ .width = target.width, .height = target.height } }, .clear_value_count = 1, .p_clear_values = @ptrCast(&clear), }, .@"inline"); self.recordDrawCommands(cmd, .{ .width = target.width, .height = target.height }, @intCast(instance_data.len), push); self.vkd.cmdEndRenderPass(cmd); // 4. Transition image COLOR_ATTACHMENT_OPTIMAL → TRANSFER_SRC_OPTIMAL self.transitionImage(cmd, target.image, .color_attachment_optimal, .transfer_src_optimal, .{ .color_attachment_write_bit = true }, .{ .transfer_read_bit = true }, .{ .color_attachment_output_bit = true }, .{ .transfer_bit = true }); // 5. Copy image → readback buffer const region = vk.BufferImageCopy{ .buffer_offset = 0, .buffer_row_length = 0, .buffer_image_height = 0, .image_subresource = .{ .aspect_mask = .{ .color_bit = true }, .mip_level = 0, .base_array_layer = 0, .layer_count = 1, }, .image_offset = .{ .x = 0, .y = 0, .z = 0 }, .image_extent = .{ .width = target.width, .height = target.height, .depth = 1 }, }; self.vkd.cmdCopyImageToBuffer(cmd, target.image, .transfer_src_optimal, target.readback_buffer, 1, @ptrCast(®ion)); try self.vkd.endCommandBuffer(cmd); // 6. Submit and fence-wait try self.vkd.resetFences(self.device, 1, @ptrCast(&self.capture_fence)); try self.vkd.queueSubmit(self.graphics_queue, 1, @ptrCast(&vk.SubmitInfo{ .command_buffer_count = 1, .p_command_buffers = @ptrCast(&cmd), }), self.capture_fence); _ = try self.vkd.waitForFences(self.device, 1, @ptrCast(&self.capture_fence), .true, std.math.maxInt(u64)); } pub fn readbackOffscreen( self: *Context, target: *const OffscreenTarget, out_rgba: []u8, ) !void { std.debug.assert(out_rgba.len == target.width * target.height * 4); const mapped = try self.vkd.mapMemory(self.device, target.readback_memory, 0, target.readback_size, .{}); defer self.vkd.unmapMemory(self.device, target.readback_memory); const src = @as([*]const u8, @ptrCast(mapped.?))[0..target.readback_size]; // Convert BGRA → RGBA, force alpha to 0xFF (surface is opaque composite). var i: usize = 0; while (i < target.readback_size) : (i += 4) { out_rgba[i + 0] = src[i + 2]; // R ← B out_rgba[i + 1] = src[i + 1]; // G out_rgba[i + 2] = src[i + 0]; // B ← R out_rgba[i + 3] = 0xFF; // A } } ``` You will need to: - Add `capture_cmd: vk.CommandBuffer` and `capture_fence: vk.Fence` to `Context`, allocated in `Context.init` alongside the swapchain command buffer + fence. - Extract a `recordDrawCommands(cmd, extent, instance_count, push)` helper from `drawCells` (bind pipeline, bind descriptor sets, bind vertex/instance buffers, set viewport/scissor to `extent`, cmdDrawInstanced). Call this helper from both `drawCells` (with swapchain extent) and `renderToOffscreen` (with offscreen extent). - Add a `transitionImage(cmd, image, old_layout, new_layout, src_access, dst_access, src_stage, dst_stage)` helper if one doesn't already exist. - Add an `uploadInstances(instances)` method if `drawCells` currently does this inline — otherwise call whatever existing upload function drawCells uses. If the exact internal types differ (e.g. `CellInstance` is named `Instance`, `PushConstants` has a different name), use the real names. - [ ] **Step 4: Verify it still compiles** Run: `zig build 2>&1 | tail -40` Expected: clean build. If there are errors, they will almost all be naming mismatches (fix them in `renderer.zig`). - [ ] **Step 5: Commit** ```bash git add src/renderer.zig git commit -m "$(cat <<'EOF' Add offscreen render target + readback to renderer New OffscreenTarget, createOffscreen/destroyOffscreen, renderToOffscreen, and readbackOffscreen. Draws into a dedicated TRANSFER_SRC image, copies to a host-visible buffer, returns RGBA pixels. EOF )" ``` --- ## Task 3: Extract frame-stats helpers to `src/bench_stats.zig` `computeFrameStats` and `printFrameStats` currently live in `main.zig` and are private. The bench baseline tool needs them. Move them (and `FrameTimingRing`, `FrameTiming`, `SectionStats`, `FrameTimingStats`, `computeSectionStats`) into a new module so both `main.zig` and `src/tools/bench_baseline.zig` can import them. **Files:** - Create: `src/bench_stats.zig` - Modify: `src/main.zig` — delete the moved code, import from the new module - Modify: `build.zig` — add the bench_stats module - [ ] **Step 1: Create the new module** Create `src/bench_stats.zig` with the content currently at `main.zig:960-1070` (`FrameTimingRing`, `FrameTiming`, `SectionStats`, `FrameTimingStats`, `computeSectionStats`, `computeFrameStats`, `printFrameStats`). Make all of them `pub`. Also ensure the existing `FrameTiming` struct is moved (it lives near the ring buffer — grep for `const FrameTiming = struct` in main.zig to find it). Run: `grep -n "^const FrameTiming \|^const FrameTimingRing\|^const SectionStats\|^const FrameTimingStats\|^fn computeSectionStats\|^fn computeFrameStats\|^fn printFrameStats" src/main.zig` Expected: lines for each. Copy those definitions verbatim into `src/bench_stats.zig` and prefix with `pub `. - [ ] **Step 2: Add bench_stats to build.zig** In `build.zig`, add after the png module block (or near the other small modules): ```zig const bench_stats_mod = b.createModule(.{ .root_source_file = b.path("src/bench_stats.zig"), .target = target, .optimize = optimize, }); exe_mod.addImport("bench_stats", bench_stats_mod); main_test_mod.addImport("bench_stats", bench_stats_mod); const bench_stats_test_mod = b.createModule(.{ .root_source_file = b.path("src/bench_stats.zig"), .target = target, .optimize = optimize, }); const bench_stats_tests = b.addTest(.{ .root_module = bench_stats_test_mod }); test_step.dependOn(&b.addRunArtifact(bench_stats_tests).step); ``` - [ ] **Step 3: Update main.zig to import from the new module** Replace the now-moved local definitions in `src/main.zig` with: ```zig const bench_stats = @import("bench_stats"); const FrameTiming = bench_stats.FrameTiming; const FrameTimingRing = bench_stats.FrameTimingRing; const FrameTimingStats = bench_stats.FrameTimingStats; const computeFrameStats = bench_stats.computeFrameStats; const printFrameStats = bench_stats.printFrameStats; ``` Delete the original definitions (the entire block from `const FrameTiming = struct` through `fn printFrameStats` — around main.zig:900-1070). Keep the existing tests in main.zig that reference these types; they will now reference the imported aliases. - [ ] **Step 4: Run the tests to confirm nothing regressed** Run: `zig build test 2>&1 | tail -40` Expected: PASS (including the `FrameTimingRing` tests which now run under `bench_stats_tests` via their original `test` blocks — move those test blocks to `bench_stats.zig` if they were left in `main.zig`). - [ ] **Step 5: Commit** ```bash git add src/bench_stats.zig src/main.zig build.zig git commit -m "$(cat <<'EOF' Extract frame timing stats into bench_stats module Enables the bench baseline tool to reuse stat computation without linking the full waystty binary. No behavior change. EOF )" ``` --- ## Task 4: Add JSON ser/de for `FrameTimingStats` in `bench_stats.zig` Needed by `bench-baseline` and `bench-check`. **Files:** - Modify: `src/bench_stats.zig` - [ ] **Step 1: Write the failing roundtrip test** Append to `src/bench_stats.zig`: ```zig pub const BaselineRecord = struct { workload_sha: []const u8, zig_version: []const u8, waystty_sha: []const u8, frame_count: usize, sections: struct { snapshot: SectionStats, row_rebuild: SectionStats, atlas_upload: SectionStats, instance_upload: SectionStats, gpu_submit: SectionStats, }, }; pub fn writeBaselineJson(alloc: std.mem.Allocator, rec: BaselineRecord, writer: anytype) !void { _ = alloc; _ = rec; _ = writer; return error.WriteFailed; // placeholder } pub fn readBaselineJson(alloc: std.mem.Allocator, bytes: []const u8) !BaselineRecord { _ = alloc; _ = bytes; return error.InvalidJson; // placeholder } test "baseline JSON round-trip" { const alloc = std.testing.allocator; const rec = BaselineRecord{ .workload_sha = "abcdef", .zig_version = "0.15.0", .waystty_sha = "123abc", .frame_count = 256, .sections = .{ .snapshot = .{ .min = 1, .avg = 2, .p99 = 3, .max = 4 }, .row_rebuild = .{ .min = 10, .avg = 20, .p99 = 30, .max = 40 }, .atlas_upload = .{ .min = 0, .avg = 0, .p99 = 0, .max = 0 }, .instance_upload = .{ .min = 5, .avg = 6, .p99 = 7, .max = 8 }, .gpu_submit = .{ .min = 9, .avg = 9, .p99 = 9, .max = 9 }, }, }; var buf = std.ArrayList(u8).init(alloc); defer buf.deinit(); try writeBaselineJson(alloc, rec, buf.writer()); var parsed = try readBaselineJson(alloc, buf.items); defer { alloc.free(parsed.workload_sha); alloc.free(parsed.zig_version); alloc.free(parsed.waystty_sha); } try std.testing.expectEqual(@as(usize, 256), parsed.frame_count); try std.testing.expectEqual(@as(u32, 30), parsed.sections.row_rebuild.p99); try std.testing.expectEqualStrings("abcdef", parsed.workload_sha); } ``` - [ ] **Step 2: Run to see the failure** Run: `zig build test 2>&1 | grep -E "baseline|WriteFailed"` Expected: failure due to placeholder. - [ ] **Step 3: Implement using `std.json`** Replace the two placeholders in `src/bench_stats.zig`: ```zig pub fn writeBaselineJson(alloc: std.mem.Allocator, rec: BaselineRecord, writer: anytype) !void { _ = alloc; try std.json.stringify(rec, .{ .whitespace = .indent_2 }, writer); } pub fn readBaselineJson(alloc: std.mem.Allocator, bytes: []const u8) !BaselineRecord { // std.json.parseFromSlice returns a Parsed wrapper; we copy string fields so // the returned record owns its memory independent of the arena. var parsed = try std.json.parseFromSlice(BaselineRecord, alloc, bytes, .{}); defer parsed.deinit(); return .{ .workload_sha = try alloc.dupe(u8, parsed.value.workload_sha), .zig_version = try alloc.dupe(u8, parsed.value.zig_version), .waystty_sha = try alloc.dupe(u8, parsed.value.waystty_sha), .frame_count = parsed.value.frame_count, .sections = parsed.value.sections, }; } ``` If the exact `std.json.stringify` signature differs in current Zig 0.15+, substitute the equivalent call (the API occasionally shifts between minor versions). - [ ] **Step 4: Run the test again** Run: `zig build test 2>&1 | grep -E "baseline"` Expected: PASS. - [ ] **Step 5: Commit** ```bash git add src/bench_stats.zig git commit -m "$(cat <<'EOF' Add BaselineRecord + JSON round-trip for frame stats Lays the foundation for bench-baseline / bench-check tooling. EOF )" ``` --- ## Task 5: Capture mode flow (`src/capture.zig`) The module owns the `--capture` entry point: parse args, start the same subsystems as `runTerminal`, force fixed 80×24 at scale=1, wait for window visibility, play the script through the PTY, drain + settle, render one frame to an offscreen target, read it back, write a PNG. Because this overlaps significantly with `runTerminal`, factor out any shared helpers you encounter (don't duplicate). Put anything that only capture mode cares about in the new module. **Files:** - Create: `src/capture.zig` - Modify: `src/main.zig` — dispatch `--capture` to `capture.run` - Modify: `build.zig` — add capture module - [ ] **Step 1: Scaffold the module and wire up `--capture` dispatch** Create `src/capture.zig`: ```zig const std = @import("std"); const vt = @import("vt"); const pty = @import("pty"); const wayland_client = @import("wayland-client"); const frame_loop_mod = @import("frame_loop"); const renderer = @import("renderer"); const font = @import("font"); const config = @import("config"); const png = @import("png"); const vk = @import("vulkan"); pub const CaptureError = error{ MissingArgs, ScriptNotFound, OutputPathUnwritable, WindowNotVisible, WindowSizeMismatch, ReadbackTimeout, PngEncodeFailed, }; pub fn run(alloc: std.mem.Allocator, argv: []const [:0]const u8) !void { if (argv.len < 3) { std.debug.print("usage: waystty --capture <script.vt> <output.png>\n", .{}); return CaptureError.MissingArgs; } const script_path = argv[1]; const output_path = argv[2]; const script_bytes = std.fs.cwd().readFileAlloc(alloc, script_path, 16 * 1024 * 1024) catch |err| { std.debug.print("capture: script not found: {s} ({s})\n", .{ script_path, @errorName(err) }); return CaptureError.ScriptNotFound; }; defer alloc.free(script_bytes); _ = output_path; // Steps 2-6 fill in the rest. return error.NotImplementedYet; } ``` In `src/main.zig`, add the dispatch near the other smoke-test branches (around line 94): ```zig if (args.len >= 2 and std.mem.eql(u8, args[1], "--capture")) { const capture = @import("capture"); return capture.run(alloc, args[1..]); } ``` In `build.zig`, add (near the other modules): ```zig const capture_mod = b.createModule(.{ .root_source_file = b.path("src/capture.zig"), .target = target, .optimize = optimize, .link_libc = true, }); capture_mod.addImport("vt", vt_mod); capture_mod.addImport("pty", pty_mod); capture_mod.addImport("wayland-client", wayland_mod); capture_mod.addImport("frame_loop", frame_loop_mod); capture_mod.addImport("renderer", renderer_mod); capture_mod.addImport("font", font_mod); capture_mod.addImport("config", config_mod); capture_mod.addImport("png", png_mod); capture_mod.addImport("vulkan", vulkan_module); exe_mod.addImport("capture", capture_mod); ``` Confirm build still compiles: Run: `zig build 2>&1 | tail -20` Expected: clean build. - [ ] **Step 2: Stand up the waystty subsystems (fixed 80×24, scale=1)** Replace the body of `capture.run` in `src/capture.zig` with the subsystem wiring that matches `runTerminal` but with forced dimensions: ```zig pub fn run(alloc: std.mem.Allocator, argv: []const [:0]const u8) !void { if (argv.len < 3) { std.debug.print("usage: waystty --capture <script.vt> <output.png>\n", .{}); return CaptureError.MissingArgs; } const script_path = argv[1]; const output_path = argv[2]; const script_bytes = std.fs.cwd().readFileAlloc(alloc, script_path, 16 * 1024 * 1024) catch |err| { std.debug.print("capture: script not found: {s} ({s})\n", .{ script_path, @errorName(err) }); return CaptureError.ScriptNotFound; }; defer alloc.free(script_bytes); // Font (identical to runTerminal — scale forced to 1) var font_lookup = try font.lookupConfiguredFont(alloc); defer font_lookup.deinit(alloc); var face = try font.Face.init(alloc, font_lookup.path, font_lookup.index, config.font_size_px); defer face.deinit(); const cell_w = face.cellWidth(); const cell_h = face.cellHeight(); // Fixed grid const cols: u16 = 80; const rows: u16 = 24; const px_w: u32 = @as(u32, cols) * cell_w; const px_h: u32 = @as(u32, rows) * cell_h; // Wayland const conn = try wayland_client.Connection.init(alloc); defer conn.deinit(); const window = try conn.createWindow(alloc, "waystty-capture"); defer window.deinit(); window.width = px_w; window.height = px_h; _ = conn.display.roundtrip(); // Renderer var ctx = try renderer.Context.init( alloc, @ptrCast(conn.display), @ptrCast(window.surface), px_w, px_h, ); defer ctx.deinit(); // Offscreen target var offscreen = try renderer.createOffscreen( ctx.vki, ctx.vkd, ctx.physical, ctx.device, ctx.render_pass, ctx.surface_format, px_w, px_h, ); defer renderer.destroyOffscreen(ctx.vkd, ctx.device, offscreen); // Glyph atlas + ASCII warm var atlas = try font.Atlas.init(alloc, 1024, 1024); defer atlas.deinit(); for (32..127) |cp| { _ = atlas.getOrInsert(&face, @intCast(cp)) catch |err| switch (err) { error.AtlasFull => break, else => return err, }; } try ctx.uploadAtlas(atlas.pixels); atlas.last_uploaded_y = atlas.cursor_y; atlas.needs_full_upload = false; atlas.dirty = false; // Terminal var term = try vt.Terminal.init(alloc, .{ .cols = cols, .rows = rows, .max_scrollback = 100, }); defer term.deinit(); term.setReportedSize(.{ .rows = rows, .columns = cols, .cell_width = cell_w, .cell_height = cell_h, }); _ = script_bytes; _ = output_path; // Steps 3-6 fill in the rest. } ``` Match the exact field names in `renderer.Context` — grep for `pub const Context` and note which fields are `pub` (you may need to make a few more public: `vki`, `vkd`, `physical`, `device`, `render_pass`, `surface_format`). Run: `zig build 2>&1 | tail -30` Expected: clean build (unresolved field access will surface if any Context member isn't `pub`). - [ ] **Step 3: Visibility wait with 3s timeout** Add after the terminal init block in `capture.run`: ```zig // Wait up to 3s for the compositor to configure + map the window. const deadline_ns = std.time.nanoTimestamp() + 3 * std.time.ns_per_s; while (std.time.nanoTimestamp() < deadline_ns) { _ = conn.display.roundtrip(); if (window.isConfigured() and window.isVisible()) break; std.time.sleep(10 * std.time.ns_per_ms); } if (!window.isConfigured() or !window.isVisible()) { std.debug.print("capture: window not visible after 3s (compositor hidden window?)\n", .{}); return CaptureError.WindowNotVisible; } // Verify the compositor didn't resize us. if (window.width != px_w or window.height != px_h) { std.debug.print("capture: window size mismatch; expected {}x{} px, got {}x{}\n", .{ px_w, px_h, window.width, window.height }); return CaptureError.WindowSizeMismatch; } ``` If `window.isConfigured()` and `window.isVisible()` don't exist, either add them to `wayland_client.Window` in `src/wayland.zig` (look for the existing configured/suspended flags on `SurfaceState`) or inline the check by poking at `window.surface_state.configured`. Run: `zig build 2>&1 | tail -30` Expected: clean build. - [ ] **Step 4: PTY playback + drain** Append to `capture.run`: ```zig // Spawn child: cat on the script bytes. // We feed via stdin pipe rather than cat <file> so we can ensure exact // timing of writes. The child is /bin/cat with no args reading from its // stdin; we write the script to the PTY master (which is cat's stdin). var p = try pty.Pty.spawn(.{ .cols = cols, .rows = rows, .shell = "/bin/cat", .shell_args = null, }); defer p.deinit(); _ = try p.write(script_bytes); // Send EOF by closing the write side. Pty doesn't expose a closeWrite — // instead we signal EOF by writing ^D (0x04). When the PTY is in cooked // mode cat will terminate on EOF. If cooked-mode termios isn't the // default in pty.zig, open pty.zig and confirm; alternatively, just // wait for cat to consume the bytes and let the outer drain loop catch it. _ = try p.write(&.{0x04}); // Drain: poll PTY until two consecutive 20ms polls return no data. var read_buf: [8192]u8 = undefined; var empty_ticks: u8 = 0; while (empty_ticks < 2) { var pfd = [_]std.posix.pollfd{ .{ .fd = p.master_fd, .events = std.posix.POLL.IN, .revents = 0 }, }; _ = std.posix.poll(&pfd, 20) catch 0; if (pfd[0].revents & std.posix.POLL.IN != 0) { const n = p.read(&read_buf) catch |err| switch (err) { error.WouldBlock, error.InputOutput => 0, else => return err, }; if (n > 0) { term.write(read_buf[0..n]); empty_ticks = 0; continue; } } empty_ticks += 1; } // Settle: one extra 50ms for the VT parser to finish. std.time.sleep(50 * std.time.ns_per_ms); ``` Run: `zig build 2>&1 | tail -30` Expected: clean build. - [ ] **Step 5: Render to offscreen + readback + write PNG** Append to `capture.run`: ```zig // Build the instance list from the terminal snapshot. This mirrors // whatever runTerminal does just before drawCells. Extract a shared // helper if convenient; otherwise duplicate the small amount of code. var render_cache = @import("render_frame").RenderCache.empty; defer render_cache.deinit(alloc); try render_cache.resizeRows(alloc, rows); var snap = try term.snapshot(alloc); defer snap.deinit(alloc); // NOTE: the exact name/location of the instance-building function depends // on how main.zig is structured. If it's a free function in main.zig, // move it into a shared module (e.g. `src/render_frame.zig`) in this // step — capture.zig and main.zig both need it. Name it `buildInstances`. var instances = std.ArrayList(renderer.CellInstance).init(alloc); defer instances.deinit(); try @import("render_frame").buildInstances(&instances, &snap, &atlas, &face, &render_cache); const push = renderer.PushConstants{ .viewport_size = .{ @floatFromInt(px_w), @floatFromInt(px_h) }, .cell_size = .{ @floatFromInt(cell_w), @floatFromInt(cell_h) }, .coverage_params = .{ 1.0, 1.0 }, // inherit default }; // If the atlas was dirtied by getOrInsert calls during snapshot, upload it. if (atlas.dirty) { try ctx.uploadAtlas(atlas.pixels); atlas.dirty = false; } try ctx.renderToOffscreen(&offscreen, instances.items, push); // Readback const rgba = try alloc.alloc(u8, px_w * px_h * 4); defer alloc.free(rgba); try ctx.readbackOffscreen(&offscreen, rgba); // PNG write const out_file = std.fs.cwd().createFile(output_path, .{}) catch |err| { std.debug.print("capture: cannot write output: {s}: {s}\n", .{ output_path, @errorName(err) }); return CaptureError.OutputPathUnwritable; }; defer out_file.close(); png.encode(alloc, .{ .width = px_w, .height = px_h, .pixels = rgba }, out_file.writer()) catch |err| { std.debug.print("capture: png encode failed: {s}\n", .{@errorName(err)}); return CaptureError.PngEncodeFailed; }; std.debug.print("capture: wrote {s} ({}x{})\n", .{ output_path, px_w, px_h }); ``` Note on the `render_frame` import: if `RenderCache` and the instance-building logic live inline in `main.zig` rather than in a reusable module, carve out `src/render_frame.zig` now containing both `pub const RenderCache = struct {...}` and `pub fn buildInstances(...)`. Import the new module from both `main.zig` and `capture.zig`, and add it to `build.zig` with `renderer_mod` as an import dependency. This is a necessary side-effect refactor, not gold-plating. - [ ] **Step 6: End-to-end smoke — capture a one-line script** Create a trivial script: ```bash printf 'hello\r\n' > /tmp/capture_smoke.vt ``` Build and run: ```bash zig build ./zig-out/bin/waystty --capture /tmp/capture_smoke.vt /tmp/capture_smoke.png ``` Expected: prints `capture: wrote /tmp/capture_smoke.png (WxH)`, and `/tmp/capture_smoke.png` exists. ```bash file /tmp/capture_smoke.png ``` Expected: `PNG image data, 640 x 384, 8-bit/color RGBA, non-interlaced` (or whatever `80*cell_w × 24*cell_h` computes to on your font). Open it in an image viewer and confirm "hello" is visible in the top-left. If rendering is empty or garbled, investigate: - Did the atlas get uploaded after adding the glyphs? - Did `renderToOffscreen` actually execute the draw (check the command buffer reset + begin)? - Are `viewport_size` / `cell_size` push constants correct? - [ ] **Step 7: Commit** ```bash git add src/capture.zig src/main.zig build.zig src/wayland.zig src/render_frame.zig git commit -m "$(cat <<'EOF' Add --capture mode: render a VT script to PNG Forces 80x24 grid at scale=1, waits for window visibility, plays script through PTY, drains + settles, renders to offscreen VkImage, reads back BGRA→RGBA, writes PNG. EOF )" ``` --- ## Task 6: `imgdiff` tool (`src/tools/imgdiff.zig`) Standalone executable: compare two RGBA PNGs; compute RMSE (normalized RGB) + per-pixel max; optionally write a side-by-side diff image. **Files:** - Create: `src/tools/imgdiff.zig` - Modify: `build.zig` — add `imgdiff` executable + `b.step("imgdiff", ...)` - [ ] **Step 1: Write the comparison function with unit tests first** Create `src/tools/imgdiff.zig`: ```zig const std = @import("std"); const png = @import("png"); pub const DiffResult = struct { rmse: f64, // [0, 1] max_pixel: f64, // [0, 1] pixel_count: usize, }; pub fn compare(a: png.Image, b: png.Image) !DiffResult { if (a.width != b.width or a.height != b.height) return error.DimensionsDiffer; std.debug.assert(a.pixels.len == b.pixels.len); const px_count = @as(usize, a.width) * a.height; var sum_sq: f64 = 0; var max_d: f64 = 0; var i: usize = 0; while (i < px_count) : (i += 1) { const off = i * 4; const dr = (@as(f64, @floatFromInt(a.pixels[off + 0])) - @as(f64, @floatFromInt(b.pixels[off + 0]))) / 255.0; const dg = (@as(f64, @floatFromInt(a.pixels[off + 1])) - @as(f64, @floatFromInt(b.pixels[off + 1]))) / 255.0; const db = (@as(f64, @floatFromInt(a.pixels[off + 2])) - @as(f64, @floatFromInt(b.pixels[off + 2]))) / 255.0; const d_sq = (dr * dr + dg * dg + db * db) / 3.0; sum_sq += d_sq; const d = @sqrt(d_sq); if (d > max_d) max_d = d; } return .{ .rmse = @sqrt(sum_sq / @as(f64, @floatFromInt(px_count))), .max_pixel = max_d, .pixel_count = px_count, }; } test "identical images produce zero RMSE" { var pixels_a = [_]u8{ 10, 20, 30, 255, 40, 50, 60, 255 }; var pixels_b = [_]u8{ 10, 20, 30, 255, 40, 50, 60, 255 }; const a = png.Image{ .width = 2, .height = 1, .pixels = &pixels_a }; const b = png.Image{ .width = 2, .height = 1, .pixels = &pixels_b }; const r = try compare(a, b); try std.testing.expectEqual(@as(f64, 0.0), r.rmse); try std.testing.expectEqual(@as(f64, 0.0), r.max_pixel); } test "fully saturated difference produces rmse=1.0 and max=1.0" { var pixels_a = [_]u8{ 0, 0, 0, 255 }; var pixels_b = [_]u8{ 255, 255, 255, 255 }; const a = png.Image{ .width = 1, .height = 1, .pixels = &pixels_a }; const b = png.Image{ .width = 1, .height = 1, .pixels = &pixels_b }; const r = try compare(a, b); try std.testing.expectApproxEqAbs(@as(f64, 1.0), r.rmse, 1e-9); try std.testing.expectApproxEqAbs(@as(f64, 1.0), r.max_pixel, 1e-9); } ``` - [ ] **Step 2: Add imgdiff to build.zig** Add near the other executables: ```zig const imgdiff_mod = b.createModule(.{ .root_source_file = b.path("src/tools/imgdiff.zig"), .target = target, .optimize = optimize, }); imgdiff_mod.addImport("png", png_mod); const imgdiff_exe = b.addExecutable(.{ .name = "imgdiff", .root_module = imgdiff_mod, }); b.installArtifact(imgdiff_exe); const imgdiff_test_mod = b.createModule(.{ .root_source_file = b.path("src/tools/imgdiff.zig"), .target = target, .optimize = optimize, }); imgdiff_test_mod.addImport("png", png_mod); const imgdiff_tests = b.addTest(.{ .root_module = imgdiff_test_mod }); test_step.dependOn(&b.addRunArtifact(imgdiff_tests).step); ``` - [ ] **Step 3: Run the compare-function tests** Run: `zig build test 2>&1 | grep -E "imgdiff|rmse"` Expected: PASS. - [ ] **Step 4: Add `main`, CLI arg handling, and diff-image output** Append to `src/tools/imgdiff.zig`: ```zig pub fn main() !void { var gpa: std.heap.DebugAllocator(.{}) = .init; defer _ = gpa.deinit(); const alloc = gpa.allocator(); const args = try std.process.argsAlloc(alloc); defer std.process.argsFree(alloc, args); if (args.len < 3) { std.debug.print("usage: imgdiff <actual.png> <reference.png> [diff.png]\n", .{}); std.process.exit(2); } const actual_path = args[1]; const reference_path = args[2]; const diff_path: ?[]const u8 = if (args.len >= 4) args[3] else null; const rmse_max = readFloatEnv("WAYSTTY_TEST_RMSE_MAX", 0.005); const pixel_max = readFloatEnv("WAYSTTY_TEST_PIXEL_MAX", 0.125); const actual_bytes = try std.fs.cwd().readFileAlloc(alloc, actual_path, 64 * 1024 * 1024); defer alloc.free(actual_bytes); const reference_bytes = try std.fs.cwd().readFileAlloc(alloc, reference_path, 64 * 1024 * 1024); defer alloc.free(reference_bytes); var actual = try png.decode(alloc, actual_bytes); defer actual.deinit(alloc); var reference = try png.decode(alloc, reference_bytes); defer reference.deinit(alloc); if (actual.width != reference.width or actual.height != reference.height) { std.debug.print("FAIL: dimensions differ ({}x{} vs {}x{})\n", .{ actual.width, actual.height, reference.width, reference.height }); std.process.exit(3); } const r = try compare(actual, reference); const pass = r.rmse <= rmse_max and r.max_pixel <= pixel_max; if (pass) { std.debug.print("OK: {s} RMSE={d:.4}% worst={d:.4}%\n", .{ reference_path, r.rmse * 100.0, r.max_pixel * 100.0 }); std.process.exit(0); } std.debug.print("FAIL: {s}\n RMSE: {d:.4}% (max {d:.4}%)\n worst pixel: {d:.4}% (max {d:.4}%)\n", .{ reference_path, r.rmse * 100.0, rmse_max * 100.0, r.max_pixel * 100.0, pixel_max * 100.0 }); if (diff_path) |p| { const diff_img = try makeDiffImage(alloc, actual, reference); defer alloc.free(diff_img.pixels); const out = try std.fs.cwd().createFile(p, .{}); defer out.close(); try png.encode(alloc, diff_img, out.writer()); std.debug.print(" diff: {s}\n", .{p}); } std.debug.print(" actual: {s}\n", .{actual_path}); std.process.exit(1); } fn readFloatEnv(name: []const u8, default: f64) f64 { const val = std.posix.getenv(name) orelse return default; return std.fmt.parseFloat(f64, val) catch default; } fn makeDiffImage(alloc: std.mem.Allocator, a: png.Image, b: png.Image) !png.Image { // Side-by-side: [actual | reference | delta-heatmap] const w = a.width * 3; const h = a.height; const pixels = try alloc.alloc(u8, w * h * 4); var y: u32 = 0; while (y < h) : (y += 1) { const row_off = @as(usize, y) * w * 4; const a_off = @as(usize, y) * a.width * 4; // actual @memcpy(pixels[row_off .. row_off + a.width * 4], a.pixels[a_off .. a_off + a.width * 4]); // reference @memcpy(pixels[row_off + a.width * 4 .. row_off + 2 * a.width * 4], b.pixels[a_off .. a_off + a.width * 4]); // delta heatmap var x: u32 = 0; while (x < a.width) : (x += 1) { const off = a_off + x * 4; const dr = (@as(f64, @floatFromInt(a.pixels[off + 0])) - @as(f64, @floatFromInt(b.pixels[off + 0]))) / 255.0; const dg = (@as(f64, @floatFromInt(a.pixels[off + 1])) - @as(f64, @floatFromInt(b.pixels[off + 1]))) / 255.0; const db = (@as(f64, @floatFromInt(a.pixels[off + 2])) - @as(f64, @floatFromInt(b.pixels[off + 2]))) / 255.0; const d = @sqrt((dr * dr + dg * dg + db * db) / 3.0); const brightness: u8 = @intFromFloat(@min(255.0, d * 255.0 * 2.0)); // 2x gain for visibility const dst = row_off + 2 * a.width * 4 + x * 4; pixels[dst + 0] = brightness; pixels[dst + 1] = brightness; pixels[dst + 2] = brightness; pixels[dst + 3] = 255; } } return .{ .width = w, .height = h, .pixels = pixels }; } ``` - [ ] **Step 5: Smoke-test the CLI** ```bash zig build ./zig-out/bin/imgdiff /tmp/capture_smoke.png /tmp/capture_smoke.png ``` Expected: `OK: /tmp/capture_smoke.png RMSE=0.0000% worst=0.0000%` and exit code 0. - [ ] **Step 6: Commit** ```bash git add src/tools/imgdiff.zig build.zig git commit -m "$(cat <<'EOF' Add imgdiff: RMSE + per-pixel-max PNG comparison Standalone tool reused by test-render. Thresholds overridable via WAYSTTY_TEST_RMSE_MAX and WAYSTTY_TEST_PIXEL_MAX. EOF )" ``` --- ## Task 7: VT test scripts Three initial scripts exercising distinct rendering features. All scripts must end with cursor-home (`\x1b[H`) for deterministic final state. **Files:** - Create: `tests/golden/scripts/basic_ascii.vt` - Create: `tests/golden/scripts/bold_colors.vt` - Create: `tests/golden/scripts/box_drawing.vt` - [ ] **Step 1: Write `basic_ascii.vt`** Script: print the printable ASCII range (32–126) on a single line, then newline and cursor-home. A small Zig program is cleaner than hand-escaping bytes; but since these are just file contents, use shell: ```bash mkdir -p tests/golden/scripts python3 -c ' import sys sys.stdout.buffer.write(b"\x1b[2J\x1b[H") # clear + home sys.stdout.buffer.write(bytes(range(32, 127))) sys.stdout.buffer.write(b"\r\n") sys.stdout.buffer.write(b"\x1b[H") # home ' > tests/golden/scripts/basic_ascii.vt ``` (Python is a one-off author-time tool here; the generated `.vt` file is what's checked in.) Confirm size: `wc -c tests/golden/scripts/basic_ascii.vt` should show ~107 bytes. - [ ] **Step 2: Write `bold_colors.vt`** ```bash python3 -c ' import sys out = b"\x1b[2J\x1b[H" attrs = [ (b"\x1b[0m", b"normal"), (b"\x1b[1m", b"bold"), (b"\x1b[2m", b"dim"), (b"\x1b[3m", b"italic"), (b"\x1b[4m", b"underline"), (b"\x1b[7m", b"reverse"), ] for esc, label in attrs: out += esc + label + b"\x1b[0m\r\n" for fg in range(30, 38): out += b"\x1b[" + str(fg).encode() + b"m" + b"fg%d " % fg out += b"\x1b[0m\r\n" for bg in range(40, 48): out += b"\x1b[" + str(bg).encode() + b"m" + b"bg%d " % bg out += b"\x1b[0m\r\n" out += b"\x1b[H" sys.stdout.buffer.write(out) ' > tests/golden/scripts/bold_colors.vt ``` - [ ] **Step 3: Write `box_drawing.vt`** ```bash python3 -c ' import sys out = b"\x1b[2J\x1b[H" # Box-drawing characters are U+2500..U+257F out += "\u250C" + "\u2500"*10 + "\u2510\r\n" for _ in range(3): out += "\u2502" + " " * 10 + "\u2502\r\n" out += "\u2514" + "\u2500"*10 + "\u2518\r\n" out += "\u2591\u2592\u2593\u2588 block chars\r\n" out += "\x1b[H" sys.stdout.buffer.write(out.encode("utf-8") if isinstance(out, str) else out) ' > tests/golden/scripts/box_drawing.vt ``` - [ ] **Step 4: Smoke-run each against capture** ```bash zig build for s in tests/golden/scripts/*.vt; do ./zig-out/bin/waystty --capture "$s" "/tmp/$(basename "$s" .vt).png" done ``` Expected: each prints `capture: wrote ...` and a PNG is written. Open each PNG and eyeball it — this is the only time you'll visually inspect before setting goldens, so confirm: - `basic_ascii.png`: printable characters on the top line - `bold_colors.png`: visible bold/dim/italic/underline/reverse attribute rows + foreground/background color bars - `box_drawing.png`: a bordered box with block shades beneath If any look wrong, fix the script (or investigate the renderer) before the next task. - [ ] **Step 5: Commit the scripts** ```bash git add tests/golden/scripts/ git commit -m "$(cat <<'EOF' Add initial VT test scripts for render testing basic_ascii, bold_colors, box_drawing — exercise distinct rendering paths. Each clears screen, emits content, returns cursor home. EOF )" ``` --- ## Task 8: `test-render` orchestrator Standalone tool that for each `.vt` in `tests/golden/scripts/`: runs `waystty --capture`, runs imgdiff against the corresponding `tests/golden/reference/*.png`, and summarizes pass/fail. Continues on failure. Non-zero exit if any fail. **Files:** - Create: `src/tools/test_render.zig` - Modify: `build.zig` — add test_render executable + `b.step("test-render", ...)` - [ ] **Step 1: Write `src/tools/test_render.zig`** ```zig const std = @import("std"); pub fn main() !void { var gpa: std.heap.DebugAllocator(.{}) = .init; defer _ = gpa.deinit(); const alloc = gpa.allocator(); const mode_update = blk: { const m = std.posix.getenv("WAYSTTY_GOLDEN_UPDATE") orelse break :blk false; break :blk std.mem.eql(u8, m, "1"); }; try std.fs.cwd().makePath("tests/golden/output"); var scripts_dir = try std.fs.cwd().openDir("tests/golden/scripts", .{ .iterate = true }); defer scripts_dir.close(); var it = scripts_dir.iterate(); var passed: usize = 0; var failed: usize = 0; while (try it.next()) |entry| { if (entry.kind != .file) continue; if (!std.mem.endsWith(u8, entry.name, ".vt")) continue; const base = entry.name[0 .. entry.name.len - 3]; const script_path = try std.fmt.allocPrint(alloc, "tests/golden/scripts/{s}.vt", .{base}); defer alloc.free(script_path); const output_path = try std.fmt.allocPrint(alloc, "tests/golden/output/{s}.png", .{base}); defer alloc.free(output_path); const reference_path = try std.fmt.allocPrint(alloc, "tests/golden/reference/{s}.png", .{base}); defer alloc.free(reference_path); const diff_path = try std.fmt.allocPrint(alloc, "tests/golden/output/{s}.diff.png", .{base}); defer alloc.free(diff_path); // Run waystty --capture const cap = try std.process.Child.run(.{ .allocator = alloc, .argv = &.{ "zig-out/bin/waystty", "--capture", script_path, output_path }, }); defer alloc.free(cap.stdout); defer alloc.free(cap.stderr); if (cap.term != .Exited or cap.term.Exited != 0) { std.debug.print("FAIL: {s}: capture exited with {}\n stderr: {s}\n", .{ base, cap.term, cap.stderr }); failed += 1; continue; } if (mode_update) { try std.fs.cwd().makePath("tests/golden/reference"); try std.fs.cwd().copyFile(output_path, std.fs.cwd(), reference_path, .{}); std.debug.print("UPDATED: {s}\n", .{base}); passed += 1; continue; } // Run imgdiff const dif = try std.process.Child.run(.{ .allocator = alloc, .argv = &.{ "zig-out/bin/imgdiff", output_path, reference_path, diff_path }, }); defer alloc.free(dif.stdout); defer alloc.free(dif.stderr); std.debug.print("{s}", .{dif.stdout}); if (dif.term == .Exited and dif.term.Exited == 0) { passed += 1; } else { failed += 1; } } std.debug.print("\n=== test-render: {} passed, {} failed ===\n", .{ passed, failed }); if (failed > 0) std.process.exit(1); } ``` - [ ] **Step 2: Wire it into build.zig** Add near the other tool executables: ```zig const test_render_mod = b.createModule(.{ .root_source_file = b.path("src/tools/test_render.zig"), .target = target, .optimize = optimize, }); const test_render_exe = b.addExecutable(.{ .name = "test-render", .root_module = test_render_mod, }); b.installArtifact(test_render_exe); const test_render_step = b.step("test-render", "Run all golden VT scripts and diff against references"); // Make sure waystty + imgdiff are built first test_render_step.dependOn(b.getInstallStep()); const test_render_run = b.addRunArtifact(test_render_exe); test_render_run.step.dependOn(b.getInstallStep()); test_render_step.dependOn(&test_render_run.step); ``` Confirm build: `zig build` - [ ] **Step 3: Test the orchestrator failure path** With no reference directory yet, `zig build test-render` should report 3 failures (missing reference PNGs → imgdiff fails on readFile). Run: `zig build test-render 2>&1 | tail -10` Expected: summary shows 3 failed. Non-zero exit. - [ ] **Step 4: Commit the orchestrator** ```bash git add src/tools/test_render.zig build.zig git commit -m "$(cat <<'EOF' Add test-render orchestrator Iterates tests/golden/scripts/*.vt, runs waystty --capture on each, compares with imgdiff against reference PNGs. Continues on failure. WAYSTTY_GOLDEN_UPDATE=1 copies output to reference instead. EOF )" ``` --- ## Task 9: Generate and commit initial golden references Run `test-render` in update mode to populate `tests/golden/reference/`, visually verify the results, commit. **Files:** - Create: `tests/golden/reference/basic_ascii.png` - Create: `tests/golden/reference/bold_colors.png` - Create: `tests/golden/reference/box_drawing.png` - Modify: `.gitignore` — add `tests/golden/output/` - [ ] **Step 1: Update `.gitignore`** Append to `.gitignore`: ``` # test-render generated artifacts tests/golden/output/ ``` - [ ] **Step 2: Generate references** ```bash WAYSTTY_GOLDEN_UPDATE=1 zig build test-render ``` Expected: `UPDATED: basic_ascii`, `UPDATED: bold_colors`, `UPDATED: box_drawing`. - [ ] **Step 3: Visually inspect each reference** ```bash ls -l tests/golden/reference/ xdg-open tests/golden/reference/basic_ascii.png xdg-open tests/golden/reference/bold_colors.png xdg-open tests/golden/reference/box_drawing.png ``` For each, confirm: - Text is crisp, no garbled glyphs - Colors match expectations (bold rows look bold, color bars show 8 distinct colors, etc.) - Box characters form actual boxes If any look wrong, debug the renderer or the script before committing. - [ ] **Step 4: Confirm steady-state passes** ```bash zig build test-render ``` Expected: `3 passed, 0 failed`. - [ ] **Step 5: Tune thresholds if needed** Re-run `zig build test-render` several times to confirm stability (no flaky failures from subpixel jitter). If any fail, either (a) the output is non-deterministic (bug — investigate) or (b) loosen the thresholds. The two easy knobs: ```bash WAYSTTY_TEST_RMSE_MAX=0.01 WAYSTTY_TEST_PIXEL_MAX=0.2 zig build test-render ``` If you needed to loosen, bake the new defaults into `src/tools/imgdiff.zig` (the `readFloatEnv("WAYSTTY_TEST_RMSE_MAX", 0.005)` call) and commit that change. - [ ] **Step 6: Commit** ```bash git add tests/golden/reference/ .gitignore git commit -m "$(cat <<'EOF' Commit initial golden reference PNGs Generated from tests/golden/scripts/* and visually verified. Regenerate with: WAYSTTY_GOLDEN_UPDATE=1 zig build test-render EOF )" ``` --- ## Task 10: `bench-baseline` and `bench-check` Two modes of the same tool. Runs the existing `WAYSTTY_BENCH=1` workload by shelling out to waystty, parses the FrameTimingStats from `bench.log` (or from a new stdout JSON emission), writes/compares `tests/bench/baseline.json`. Two design decisions here: 1. **How to get stats out of a waystty run:** current bench prints a human-readable table. Add a sidecar flag `WAYSTTY_BENCH_JSON=/path/to/file.json` that also dumps the `BaselineRecord` to disk on exit. Cleaner than parsing the text table. 2. **What sha to store:** workload_sha = `sha256(bench_script string literal in main.zig)`; waystty_sha = `git rev-parse HEAD`; zig_version = captured at compile-time via `@import("builtin").zig_version_string`. **Files:** - Modify: `src/main.zig` — if `WAYSTTY_BENCH_JSON` is set, write `BaselineRecord` on exit - Create: `src/tools/bench_baseline.zig` - Modify: `build.zig` — add executable + build step - [ ] **Step 1: Add the JSON dump to main.zig's bench exit path** Find where `printFrameStats(stats)` is currently called in the bench-exit path in `main.zig` (grep for it). Right after that call, add: ```zig if (std.posix.getenv("WAYSTTY_BENCH_JSON")) |path| { const bench_stats_mod = @import("bench_stats"); const rec = bench_stats_mod.BaselineRecord{ .workload_sha = &sha256Hex(alloc, bench_script orelse ""), .zig_version = @import("builtin").zig_version_string, .waystty_sha = gitHead(alloc) catch "unknown", .frame_count = stats.frame_count, .sections = .{ .snapshot = stats.snapshot, .row_rebuild = stats.row_rebuild, .atlas_upload = stats.atlas_upload, .instance_upload = stats.instance_upload, .gpu_submit = stats.gpu_submit, }, }; const f = std.fs.cwd().createFile(path, .{}) catch |err| { std.log.warn("bench_json write failed: {s} ({s})", .{ path, @errorName(err) }); return; }; defer f.close(); bench_stats_mod.writeBaselineJson(alloc, rec, f.writer()) catch |err| { std.log.warn("bench_json serialize failed: {s}", .{@errorName(err)}); }; } ``` Add helper functions at the bottom of `main.zig`: ```zig fn sha256Hex(alloc: std.mem.Allocator, input: []const u8) [64]u8 { _ = alloc; var digest: [32]u8 = undefined; std.crypto.hash.sha2.Sha256.hash(input, &digest, .{}); var hex: [64]u8 = undefined; _ = std.fmt.bufPrint(&hex, "{}", .{std.fmt.fmtSliceHexLower(&digest)}) catch unreachable; return hex; } fn gitHead(alloc: std.mem.Allocator) ![]const u8 { const r = try std.process.Child.run(.{ .allocator = alloc, .argv = &.{ "git", "rev-parse", "HEAD" } }); defer alloc.free(r.stdout); defer alloc.free(r.stderr); if (r.term != .Exited or r.term.Exited != 0) return "unknown"; return std.mem.trim(u8, r.stdout, "\n \t"); } ``` Note: `sha256Hex` returns a stack array; the `.workload_sha` field stores a slice. Adjust to `alloc.dupe` or use a scoped buffer. Run: `zig build` then `WAYSTTY_BENCH=1 WAYSTTY_BENCH_JSON=/tmp/bench.json ./zig-out/bin/waystty 2>/tmp/bench.log` After the bench workload finishes, check `/tmp/bench.json` exists and contains valid JSON with non-zero `frame_count`. - [ ] **Step 2: Write `bench_baseline.zig`** Create `src/tools/bench_baseline.zig`: ```zig const std = @import("std"); const bench_stats = @import("bench_stats"); pub fn main() !void { var gpa: std.heap.DebugAllocator(.{}) = .init; defer _ = gpa.deinit(); const alloc = gpa.allocator(); const args = try std.process.argsAlloc(alloc); defer std.process.argsFree(alloc, args); const mode: enum { save, check } = if (args.len >= 2 and std.mem.eql(u8, args[1], "save")) .save else .check; const baseline_path = "tests/bench/baseline.json"; const tmp_json = "/tmp/waystty-bench-current.json"; try std.fs.cwd().makePath("tests/bench"); // Run waystty with WAYSTTY_BENCH=1 WAYSTTY_BENCH_JSON=<tmp> var env = try std.process.getEnvMap(alloc); defer env.deinit(); try env.put("WAYSTTY_BENCH", "1"); try env.put("WAYSTTY_BENCH_JSON", tmp_json); const child = try std.process.Child.run(.{ .allocator = alloc, .argv = &.{"zig-out/bin/waystty"}, .env_map = &env, }); defer alloc.free(child.stdout); defer alloc.free(child.stderr); const current_bytes = std.fs.cwd().readFileAlloc(alloc, tmp_json, 16 * 1024) catch |err| { std.debug.print("bench: no JSON output at {s}: {s}\n", .{ tmp_json, @errorName(err) }); std.process.exit(2); }; defer alloc.free(current_bytes); var current = try bench_stats.readBaselineJson(alloc, current_bytes); defer { alloc.free(current.workload_sha); alloc.free(current.zig_version); alloc.free(current.waystty_sha); } if (mode == .save) { const out = try std.fs.cwd().createFile(baseline_path, .{}); defer out.close(); try bench_stats.writeBaselineJson(alloc, current, out.writer()); std.debug.print("bench: wrote {s}\n", .{baseline_path}); return; } // check mode const baseline_bytes = std.fs.cwd().readFileAlloc(alloc, baseline_path, 16 * 1024) catch |err| { std.debug.print("bench: no baseline at {s}: {s}\n run: zig build bench-baseline\n", .{ baseline_path, @errorName(err) }); std.process.exit(2); }; defer alloc.free(baseline_bytes); var baseline = try bench_stats.readBaselineJson(alloc, baseline_bytes); defer { alloc.free(baseline.workload_sha); alloc.free(baseline.zig_version); alloc.free(baseline.waystty_sha); } if (!std.mem.eql(u8, baseline.workload_sha, current.workload_sha)) { std.debug.print("WARN: bench script changed since baseline; consider regenerating\n", .{}); } const pct = blk: { const v = std.posix.getenv("WAYSTTY_BENCH_REGRESSION_PCT") orelse break :blk 20.0; break :blk std.fmt.parseFloat(f64, v) catch 20.0; }; var regressed = false; const sections = [_][]const u8{ "snapshot", "row_rebuild", "atlas_upload", "instance_upload", "gpu_submit" }; inline for (sections, 0..) |name, i| { _ = i; const base_p99 = @field(baseline.sections, name).p99; const cur_p99 = @field(current.sections, name).p99; const delta_pct = if (base_p99 == 0) 0.0 else ((@as(f64, @floatFromInt(cur_p99)) - @as(f64, @floatFromInt(base_p99))) / @as(f64, @floatFromInt(base_p99))) * 100.0; const status = if (delta_pct > pct) "REGRESSION" else "OK"; if (delta_pct > pct) regressed = true; std.debug.print("bench: {s:<16} p99 {d:>5}us (baseline {d:>5}us) {d:+6.1}% {s}\n", .{ name, cur_p99, base_p99, delta_pct, status }); } if (regressed) std.process.exit(1); } ``` Note the `inline for (sections, 0..)` with string field access: Zig's `@field` works on compile-time-known field names, which `sections` provides. If `inline for` with a string-array approach doesn't compile in your Zig version, unroll the five section comparisons explicitly. - [ ] **Step 3: Wire into build.zig** ```zig const bench_baseline_mod = b.createModule(.{ .root_source_file = b.path("src/tools/bench_baseline.zig"), .target = target, .optimize = optimize, }); bench_baseline_mod.addImport("bench_stats", bench_stats_mod); const bench_baseline_exe = b.addExecutable(.{ .name = "bench-baseline", .root_module = bench_baseline_mod, }); b.installArtifact(bench_baseline_exe); const bench_baseline_step = b.step("bench-baseline", "Save current frame-timing profile to tests/bench/baseline.json"); const bench_baseline_run = b.addRunArtifact(bench_baseline_exe); bench_baseline_run.addArg("save"); bench_baseline_run.step.dependOn(b.getInstallStep()); bench_baseline_step.dependOn(&bench_baseline_run.step); const bench_check_step = b.step("bench-check", "Compare current frame timings against baseline"); const bench_check_run = b.addRunArtifact(bench_baseline_exe); bench_check_run.addArg("check"); bench_check_run.step.dependOn(b.getInstallStep()); bench_check_step.dependOn(&bench_check_run.step); ``` - [ ] **Step 4: Generate the initial baseline** ```bash zig build bench-baseline cat tests/bench/baseline.json ``` Expected: `tests/bench/baseline.json` exists with non-zero `frame_count` and five section entries. - [ ] **Step 5: Sanity-check `bench-check`** ```bash zig build bench-check ``` Expected: all sections print with small percentage deltas; overall `REGRESSION` status only if run-to-run variance exceeds 20%. On a quiet machine this should pass cleanly. Run it three times in a row to confirm stability. If it flakes, raise the default threshold or investigate a genuine noise source. - [ ] **Step 6: Commit** ```bash git add tests/bench/baseline.json src/main.zig src/tools/bench_baseline.zig build.zig git commit -m "$(cat <<'EOF' Add bench-baseline and bench-check Baseline stores workload/zig/waystty SHA + per-section p99. Check compares against baseline, flags sections exceeding 20% p99 growth. Threshold overridable via WAYSTTY_BENCH_REGRESSION_PCT. EOF )" ``` --- ## Task 11: Makefile targets + housekeeping Bring the new tools under the familiar `make` UX, gitignore stray test binaries at repo root. **Files:** - Modify: `Makefile` - Modify: `.gitignore` - [ ] **Step 1: Add Makefile targets** Replace `Makefile` with: ```makefile ZIG ?= zig FLAMEGRAPH ?= flamegraph.pl STACKCOLLAPSE ?= stackcollapse-perf.pl .PHONY: build run test bench profile clean test-render golden-update bench-baseline bench-check build: $(ZIG) build run: build $(ZIG) build run test: $(ZIG) build test zig-out/bin/waystty: $(wildcard src/*.zig) $(wildcard src/tools/*.zig) $(wildcard shaders/*) $(ZIG) build bench: zig-out/bin/waystty WAYSTTY_BENCH=1 ./zig-out/bin/waystty 2>bench.log || true @echo "--- frame timing ---" @grep -A 12 "waystty frame timing" bench.log || echo "(no timing data found)" profile: $(ZIG) build -Doptimize=ReleaseSafe perf record -g -F 999 --no-inherit -o perf.data -- \ sh -c 'WAYSTTY_BENCH=1 ./zig-out/bin/waystty 2>bench.log' perf script -i perf.data \ | $(STACKCOLLAPSE) \ | $(FLAMEGRAPH) > flamegraph.svg @echo "--- frame timing ---" @grep -A 12 "waystty frame timing" bench.log || echo "(no timing data found)" xdg-open flamegraph.svg test-render: $(ZIG) build test-render golden-update: WAYSTTY_GOLDEN_UPDATE=1 $(ZIG) build test-render bench-baseline: $(ZIG) build bench-baseline bench-check: $(ZIG) build bench-check clean: rm -rf zig-out .zig-cache perf.data bench.log flamegraph.svg tests/golden/output ``` - [ ] **Step 2: Verify each target runs** ```bash make test-render make bench-check ``` Expected: both exit 0. - [ ] **Step 3: Update `.gitignore` for stray binaries** The working tree has untracked `test_io`, `test_io2`, `test_io3`, `test_sig`, `test_timer` binaries at repo root. They appear to be scratch compilation outputs. Append to `.gitignore`: ``` # Scratch test binaries (ad-hoc compilations) /test_io /test_io2 /test_io3 /test_sig /test_timer ``` Confirm they no longer appear in `git status`. - [ ] **Step 4: Final commit** ```bash git add Makefile .gitignore git commit -m "$(cat <<'EOF' Add make targets for render + bench tests make test-render, golden-update, bench-baseline, bench-check. Gitignore stray test_* scratch binaries at repo root. EOF )" ``` --- ## Verification After all tasks complete: ```bash make test # existing unit tests still pass make test-render # 3 passed, 0 failed make bench-check # all sections OK vs baseline ``` All three should exit 0. ## Self-Review Notes - **Spec coverage:** every spec section maps to a task. Capture mode → Task 5; imgdiff → Task 6; scripts → Task 7; orchestrator + goldens → Tasks 8-9; bench baseline/check → Task 10; Makefile → Task 11. Offscreen render target (critical spec fix) → Task 2. PNG codec dependency → Task 1. - **Placeholders:** none. Every code step contains the actual code. Any "refactor if needed" notes (like the `renderer.zig` helper extraction in Task 2) are concrete — they identify what to extract and why. - **Types consistent:** `FrameTimingStats`, `BaselineRecord`, `OffscreenTarget`, `Image`, `DiffResult` appear with consistent signatures across the tasks that use them. - **Known API uncertainty:** `std.compress.flate` and `std.json.stringify` signatures occasionally drift between Zig minor versions. Tasks 1 and 4 call this out explicitly so the implementer substitutes the current Zig 0.15+ equivalent rather than copy-pasting blindly.