#19 feat(demo): show hint mode on fullscreen Firefox with a real site

state: merged · 95445f0 · fast-forward
branch: feat/demo-hints → main
opened: bnema · 16h ago
updated: 16h ago

Reworks the README demo (tools/demo, docs/media/demo.mp4).

Same flow (side-by-side windows, bands, stash, overview), now 49 s.
Firefox opens a live DuckDuckGo search; the fake tools/demo/site page is gone.
Ending: back on workspace 1, Firefox fullscreen (Super+Shift+f), hint mode (Super+g), the label of the "Wayland (protocol) - Wikipedia" result is typed and Wikipedia opens.
The compositor runs inside the Docker image with tesseract and AT-SPI; demo-hint-label finds the target label by OCR so the typed label is deterministic despite live content.
record.sh fails on truncated (timeout) recordings and on hosts without an AMD GPU (vulkan-radeon is required for hint scans).

Headless screenshots encode each frame once with BestSpeed PNG and publish latest.png as a hard link: 75 ms to ~38 ms per frame, max 12 to 19 unique frames/s in the video.

Checks: make check, make race, make mocks-check, staticcheck, shellcheck. Wrap-up reviews (Go + demo tooling) done, all findings fixed or explicitly kept.

Known limits: live site content varies between recordings (the script fails loudly if the target result is missing); unique frame rate is still below 30 fps (Go PNG encoder floor).

Commit	Subject	Author	Date	Signature
95445f0	fix(demo): fail truncated recordings and document the AMD GPU requirement	brice	16h ago	GPG Verified
b35bb20	test(headless): assert screenshot hard links and cover skip and rename failure paths	brice	16h ago	GPG Verified
b970441	refactor(headless): share the screenshot security gate	brice	16h ago	GPG Verified
9ccafd9	chore(demo): re-record the demo with faster screenshot frames	brice	16h ago	GPG Verified
0eeac95	perf(headless): encode each screenshot frame once with fast PNG compression	brice	16h ago	GPG Verified
3a0a985	feat(demo): open the hint scene on fullscreen Firefox	brice	16h ago	GPG Verified
d5c3b34	feat(demo): show hint mode on real sites with the compositor in the image	brice	16h ago	GPG Verified

Checks 2

13 files · +492 -80

1/2

M docs/dev/headless.md +9 -5


@@ -7,7 +7,7 @@ neferwl --backend=headless --screenshot /tmp/neferwl-shots
7	7		WAYLAND_DISPLAY=<name from the log> foot
8	8		```
9	9
10		-	- `latest.png` shows the current frame; `frames.ffconcat` records frame timing. Every frame is also saved as `frame-NNNNNN.png`, so the directory grows.
	10	+	- `latest.png` shows the current frame; `frames.ffconcat` records frame timing. Every frame is also saved as `frame-NNNNNN.png`, so the directory grows. `latest.png` is a hard link to the newest frame, so replace it rather than editing it in place.
11	11		- `--size 640x360,640x360` creates two outputs, arranged left to right,
12	12		for testing `focus-monitor-left/right` and moving columns between outputs.
13	13		- With several outputs, each writes to `<dir>/HEADLESS-<n>/`.

@@ -25,10 +25,14 @@ ffmpeg -f concat -safe 0 -i /tmp/shots/frames.ffconcat -vf "fps=60,format=yuv420
25	25		For a GIF, convert `demo.mp4` with ffmpeg's `palettegen` and `paletteuse` filters.
26	26
27	27		### Demo recording
28		-	Run `tools/demo/record.sh` with Go 1.27, rootless Docker and ffmpeg installed.
29		-	It builds an Arch app image and the host compositor, then records the 1080p README demo.
30		-	Apps run in a disposable, offline container; only the headless Wayland socket is mounted.
31		-	A FIFO synchronizes `input.txt` with app launches after container startup.
	28	+	Run `tools/demo/record.sh`. It requires Go 1.27, rootless Docker, ffmpeg, network access and an AMD GPU.
	29	+	It builds the Arch image (apps, tesseract, AT-SPI) and the compositor binary, then records the 1080p README demo.
	30	+	The compositor runs headless inside a disposable container, with the binary and a work directory bind-mounted and `/dev/dri` passed for the GPU.
	31	+	The image only carries Mesa's radeon Vulkan driver, and hint scans need the sync files that a hardware driver exports; the script stops early without an AMD render node.
	32	+	Firefox opens a live DuckDuckGo page, so the recording needs the network and its content varies.
	33	+	A FIFO feeds `input.txt` while the script launches the apps on the same clock.
	34	+	The hint step makes Firefox fullscreen and labels the page with `hint`; `hint-click <words>` then types the label of the link that shows those words, found from the scan's dump and an OCR read of the still page (`hint-read`).
	35	+	The recording only counts if the compositor stops on the quit key that ends `input.txt`; reaching its `--timeout` first fails the run as truncated.
32	36		Temporary captures are removed by default; set `KEEP_CAPTURE=1` to keep successful runs.
33	37		The recording and poster replace `docs/media/demo.mp4` and `docs/media/demo.png`.
34	38

 WAYLAND_DISPLAY=<name from the log> foot
 ```
-- `latest.png` shows the current frame; `frames.ffconcat` records frame timing. Every frame is also saved as `frame-NNNNNN.png`, so the directory grows.
+- `latest.png` shows the current frame; `frames.ffconcat` records frame timing. Every frame is also saved as `frame-NNNNNN.png`, so the directory grows. `latest.png` is a hard link to the newest frame, so replace it rather than editing it in place.
 - `--size 640x360,640x360` creates two outputs, arranged left to right,
   for testing `focus-monitor-left/right` and moving columns between outputs.
 - With several outputs, each writes to `<dir>/HEADLESS-<n>/`.
 For a GIF, convert `demo.mp4` with ffmpeg's `palettegen` and `paletteuse` filters.
 ### Demo recording
-Run `tools/demo/record.sh` with Go 1.27, rootless Docker and ffmpeg installed.
+Run `tools/demo/record.sh`. It requires Go 1.27, rootless Docker, ffmpeg, network access and an AMD GPU.
-It builds an Arch app image and the host compositor, then records the 1080p README demo.
+It builds the Arch image (apps, tesseract, AT-SPI) and the compositor binary, then records the 1080p README demo.
-Apps run in a disposable, offline container; only the headless Wayland socket is mounted.
+The compositor runs headless inside a disposable container, with the binary and a work directory bind-mounted and `/dev/dri` passed for the GPU.
-A FIFO synchronizes `input.txt` with app launches after container startup.
+The image only carries Mesa's radeon Vulkan driver, and hint scans need the sync files that a hardware driver exports; the script stops early without an AMD render node.
+Firefox opens a live DuckDuckGo page, so the recording needs the network and its content varies.
+A FIFO feeds `input.txt` while the script launches the apps on the same clock.
+The hint step makes Firefox fullscreen and labels the page with `hint`; `hint-click <words>` then types the label of the link that shows those words, found from the scan's dump and an OCR read of the still page (`hint-read`).
+The recording only counts if the compositor stops on the quit key that ends `input.txt`; reaching its `--timeout` first fails the run as truncated.
 Temporary captures are removed by default; set `KEEP_CAPTURE=1` to keep successful runs.
 The recording and poster replace `docs/media/demo.mp4` and `docs/media/demo.png`.

M docs/media/demo.mp4 +0 -0 binary file: no text diff

binary file: no text diff

M docs/media/demo.png +0 -0 binary file: no text diff

binary file: no text diff

M internal/adapters/headless/output.go +53 -4


@@ -8,6 +8,7 @@ import (
8	8		"image/png"
9	9		"os"
10	10		"path/filepath"
	11	+	"sync"
11	12		"time"
12	13
13	14		"golang.org/x/sys/unix"

@@ -583,7 +584,8 @@ func Run(ctx context.Context, opts Options, scenes <-chan ports.Scene, contents
583	584		if !written {
584	585		continue
585	586		}
586		-	written, err = writePNGSecureResult(filepath.Join(opts.ScreenshotDir, "latest.png"), shot, opts.Security, scene.Security)
	587	+	// latest.png is the frame just written, linked rather than encoded again.
	588	+	written, err = linkPNGSecure(filepath.Join(opts.ScreenshotDir, name), filepath.Join(opts.ScreenshotDir, "latest.png"), shot, opts.Security, scene.Security)
587	589		if err != nil {
588	590		return fmt.Errorf("latest screenshot: %w", err)
589	591		}

@@ -640,9 +642,31 @@ func writePNGSecure(path string, img image.Image, security ports.SessionSecurity
640	642		return err
641	643		}
642	644
	645	+	// screenshotPNG encodes the PNGs written by the headless output (frames, the
	646	+	// latest.png fallback, raw HDR latest-pq.png): encoding is by far the largest
	647	+	// cost of a headless frame, so it trades file size for speed. Its buffers are
	648	+	// reused between files.
	649	+	var screenshotPNG = png.Encoder{CompressionLevel: png.BestSpeed, BufferPool: new(pngBuffers)}
	650	+
	651	+	// pngBuffers is a png.EncoderBufferPool.
	652	+	type pngBuffers struct{ pool sync.Pool }
	653	+
	654	+	func (p pngBuffers) Get() png.EncoderBuffer {
	655	+	b, _ := p.pool.Get().(*png.EncoderBuffer)
	656	+	return b
	657	+	}
	658	+
	659	+	func (p pngBuffers) Put(b png.EncoderBuffer) { p.pool.Put(b) }
	660	+
	661	+	// gated reports whether a screenshot must not be published: the session is
	662	+	// protected, or its security state is no longer the one the frame was drawn in.
	663	+	func gated(security ports.SessionSecurity, state ports.SecurityState) bool {
	664	+	return security != nil && (state.Protected \|\| security.Snapshot() != state)
	665	+	}
	666	+
643	667		// writePNGSecureResult distinguishes a security skip from a published PNG.
644	668		func writePNGSecureResult(path string, img image.Image, security ports.SessionSecurity, state ports.SecurityState) (bool, error) {
645		-	if security != nil && (state.Protected \|\| security.Snapshot() != state) {
	669	+	if gated(security, state) {
646	670		return false, nil
647	671		}
648	672		f, err := os.CreateTemp(filepath.Dir(path), ".frame-*.png")

@@ -650,14 +674,14 @@ func writePNGSecureResult(path string, img image.Image, security ports.SessionSe
650	674		return false, err
651	675		}
652	676		defer os.Remove(f.Name())
653		-	if err := png.Encode(f, img); err != nil {
	677	+	if err := screenshotPNG.Encode(f, img); err != nil {
654	678		_ = f.Close()
655	679		return false, err
656	680		}
657	681		if err := f.Close(); err != nil {
658	682		return false, err
659	683		}
660		-	if security != nil && security.Snapshot() != state {
	684	+	if gated(security, state) {
661	685		return false, nil
662	686		}
663	687		if err := os.Rename(f.Name(), path); err != nil {

@@ -666,6 +690,31 @@ func writePNGSecureResult(path string, img image.Image, security ports.SessionSe
666	690		return true, nil
667	691		}
668	692
	693	+	// linkPNGSecure publishes the PNG already written at src as dst without
	694	+	// encoding it again: a hard link under a temporary name, renamed over dst, so
	695	+	// readers see the old or the new frame, never a partial one. Frame files are
	696	+	// never modified after they are published, which makes the shared file safe.
	697	+	// It gates on security like writePNGSecureResult, and falls back to encoding
	698	+	// img where the file system cannot link.
	699	+	func linkPNGSecure(src, dst string, img image.Image, security ports.SessionSecurity, state ports.SecurityState) (bool, error) {
	700	+	if gated(security, state) {
	701	+	return false, nil
	702	+	}
	703	+	tmp := dst + ".tmp"
	704	+	_ = os.Remove(tmp)
	705	+	if err := os.Link(src, tmp); err != nil {
	706	+	return writePNGSecureResult(dst, img, security, state)
	707	+	}
	708	+	defer os.Remove(tmp)
	709	+	if gated(security, state) {
	710	+	return false, nil
	711	+	}
	712	+	if err := os.Rename(tmp, dst); err != nil {
	713	+	return false, err
	714	+	}
	715	+	return true, nil
	716	+	}
	717	+
669	718		// report queues what the output read, with flip for a drawn frame, and
670	719		// pushes it to the sink.
671	720		func (opts Options) report(q presented.Queue, flip ports.FlipInfo, seen map[ports.WindowID]uint64) {


@@ -8,6 +8,7 @@ import (
8		"image/png"	8		"image/png"
9		"os"	9		"os"
10		"path/filepath"	10		"path/filepath"
			11	+	"sync"
11		"time"	12		"time"
12			13
13		"golang.org/x/sys/unix"	14		"golang.org/x/sys/unix"

@@ -583,7 +584,8 @@ func Run(ctx context.Context, opts Options, scenes <-chan ports.Scene, contents
583		if !written {	584		if !written {
584		continue	585		continue
585		}	586		}
586	-	written, err = writePNGSecureResult(filepath.Join(opts.ScreenshotDir, "latest.png"), shot, opts.Security, scene.Security)	587	+	// latest.png is the frame just written, linked rather than encoded again.
			588	+	written, err = linkPNGSecure(filepath.Join(opts.ScreenshotDir, name), filepath.Join(opts.ScreenshotDir, "latest.png"), shot, opts.Security, scene.Security)
587		if err != nil {	589		if err != nil {
588		return fmt.Errorf("latest screenshot: %w", err)	590		return fmt.Errorf("latest screenshot: %w", err)
589		}	591		}

@@ -640,9 +642,31 @@ func writePNGSecure(path string, img image.Image, security ports.SessionSecurity
640		return err	642		return err
641		}	643		}
642			644
			645	+	// screenshotPNG encodes the PNGs written by the headless output (frames, the
			646	+	// latest.png fallback, raw HDR latest-pq.png): encoding is by far the largest
			647	+	// cost of a headless frame, so it trades file size for speed. Its buffers are
			648	+	// reused between files.
			649	+	var screenshotPNG = png.Encoder{CompressionLevel: png.BestSpeed, BufferPool: new(pngBuffers)}
			650	+
			651	+	// pngBuffers is a png.EncoderBufferPool.
			652	+	type pngBuffers struct{ pool sync.Pool }
			653	+
			654	+	func (p pngBuffers) Get() png.EncoderBuffer {
			655	+	b, _ := p.pool.Get().(*png.EncoderBuffer)
			656	+	return b
			657	+	}
			658	+
			659	+	func (p pngBuffers) Put(b png.EncoderBuffer) { p.pool.Put(b) }
			660	+
			661	+	// gated reports whether a screenshot must not be published: the session is
			662	+	// protected, or its security state is no longer the one the frame was drawn in.
			663	+	func gated(security ports.SessionSecurity, state ports.SecurityState) bool {
			664	+	return security != nil && (state.Protected \|\| security.Snapshot() != state)
			665	+	}
			666	+
643		// writePNGSecureResult distinguishes a security skip from a published PNG.	667		// writePNGSecureResult distinguishes a security skip from a published PNG.
644		func writePNGSecureResult(path string, img image.Image, security ports.SessionSecurity, state ports.SecurityState) (bool, error) {	668		func writePNGSecureResult(path string, img image.Image, security ports.SessionSecurity, state ports.SecurityState) (bool, error) {
645	-	if security != nil && (state.Protected \|\| security.Snapshot() != state) {	669	+	if gated(security, state) {
646		return false, nil	670		return false, nil
647		}	671		}
648		f, err := os.CreateTemp(filepath.Dir(path), ".frame-*.png")	672		f, err := os.CreateTemp(filepath.Dir(path), ".frame-*.png")

@@ -650,14 +674,14 @@ func writePNGSecureResult(path string, img image.Image, security ports.SessionSe
650		return false, err	674		return false, err
651		}	675		}
652		defer os.Remove(f.Name())	676		defer os.Remove(f.Name())
653	-	if err := png.Encode(f, img); err != nil {	677	+	if err := screenshotPNG.Encode(f, img); err != nil {
654		_ = f.Close()	678		_ = f.Close()
655		return false, err	679		return false, err
656		}	680		}
657		if err := f.Close(); err != nil {	681		if err := f.Close(); err != nil {
658		return false, err	682		return false, err
659		}	683		}
660	-	if security != nil && security.Snapshot() != state {	684	+	if gated(security, state) {
661		return false, nil	685		return false, nil
662		}	686		}
663		if err := os.Rename(f.Name(), path); err != nil {	687		if err := os.Rename(f.Name(), path); err != nil {

@@ -666,6 +690,31 @@ func writePNGSecureResult(path string, img image.Image, security ports.SessionSe
666		return true, nil	690		return true, nil
667		}	691		}
668			692
			693	+	// linkPNGSecure publishes the PNG already written at src as dst without
			694	+	// encoding it again: a hard link under a temporary name, renamed over dst, so
			695	+	// readers see the old or the new frame, never a partial one. Frame files are
			696	+	// never modified after they are published, which makes the shared file safe.
			697	+	// It gates on security like writePNGSecureResult, and falls back to encoding
			698	+	// img where the file system cannot link.
			699	+	func linkPNGSecure(src, dst string, img image.Image, security ports.SessionSecurity, state ports.SecurityState) (bool, error) {
			700	+	if gated(security, state) {
			701	+	return false, nil
			702	+	}
			703	+	tmp := dst + ".tmp"
			704	+	_ = os.Remove(tmp)
			705	+	if err := os.Link(src, tmp); err != nil {
			706	+	return writePNGSecureResult(dst, img, security, state)
			707	+	}
			708	+	defer os.Remove(tmp)
			709	+	if gated(security, state) {
			710	+	return false, nil
			711	+	}
			712	+	if err := os.Rename(tmp, dst); err != nil {
			713	+	return false, err
			714	+	}
			715	+	return true, nil
			716	+	}
			717	+
669		// report queues what the output read, with flip for a drawn frame, and	718		// report queues what the output read, with flip for a drawn frame, and
670		// pushes it to the sink.	719		// pushes it to the sink.
671		func (opts Options) report(q presented.Queue, flip ports.FlipInfo, seen map[ports.WindowID]uint64) {	720		func (opts Options) report(q presented.Queue, flip ports.FlipInfo, seen map[ports.WindowID]uint64) {

M internal/adapters/headless/output_test.go +171 -0


@@ -14,6 +14,7 @@ import (
14	14		"strconv"
15	15		"strings"
16	16		"sync"
	17	+	"syscall"
17	18		"testing"
18	19		"time"
19	20

@@ -269,6 +270,176 @@ func TestRunTrimsRendererOnTick(t *testing.T) {
269	270		}
270	271		}
271	272
	273	+	// latest.png is the frame file linked, not encoded again: the same pixels,
	274	+	// published atomically, and nothing left behind.
	275	+	func TestLinkPNGSecure(t *testing.T) {
	276	+	dir := t.TempDir()
	277	+	img := image.NewRGBA(image.Rect(0, 0, 3, 2))
	278	+	for i := range img.Pix {
	279	+	img.Pix[i] = byte(i * 7)
	280	+	}
	281	+	frame, latest := filepath.Join(dir, "frame-000001.png"), filepath.Join(dir, "latest.png")
	282	+	if err := writePNGSecure(frame, img, nil, ports.SecurityState{}); err != nil {
	283	+	t.Fatal(err)
	284	+	}
	285	+	// An older latest.png is replaced.
	286	+	if err := os.WriteFile(latest, []byte("old"), 0o600); err != nil {
	287	+	t.Fatal(err)
	288	+	}
	289	+	if written, err := linkPNGSecure(frame, latest, img, nil, ports.SecurityState{}); err != nil \|\| !written {
	290	+	t.Fatalf("link = %v, %v", written, err)
	291	+	}
	292	+	a, err := os.ReadFile(frame)
	293	+	if err != nil {
	294	+	t.Fatal(err)
	295	+	}
	296	+	b, err := os.ReadFile(latest)
	297	+	if err != nil {
	298	+	t.Fatal(err)
	299	+	}
	300	+	if !bytes.Equal(a, b) {
	301	+	t.Fatal("latest.png differs from the frame")
	302	+	}
	303	+	fi, err := os.Stat(frame)
	304	+	if err != nil {
	305	+	t.Fatal(err)
	306	+	}
	307	+	li, err := os.Stat(latest)
	308	+	if err != nil {
	309	+	t.Fatal(err)
	310	+	}
	311	+	if !os.SameFile(fi, li) {
	312	+	t.Fatal("latest.png is a copy, not a hard link to the frame")
	313	+	}
	314	+	got, err := png.Decode(bytes.NewReader(b))
	315	+	if err != nil {
	316	+	t.Fatal(err)
	317	+	}
	318	+	for y := range 2 {
	319	+	for x := range 3 {
	320	+	if g, w := color.RGBAModel.Convert(got.At(x, y)), img.RGBAAt(x, y); g != color.Color(w) {
	321	+	t.Errorf("(%d,%d) = %v, want %v", x, y, g, w)
	322	+	}
	323	+	}
	324	+	}
	325	+	entries, err := os.ReadDir(dir)
	326	+	if err != nil {
	327	+	t.Fatal(err)
	328	+	}
	329	+	if len(entries) != 2 {
	330	+	t.Fatalf("leftover files: %v", entries)
	331	+	}
	332	+	}
	333	+
	334	+	// A frame that crossed a security change is not published as latest.png, a
	335	+	// failed rename leaves no temporary file, and a source that cannot be linked
	336	+	// is encoded instead.
	337	+	func TestLinkPNGSecureSkipsAndFallsBack(t *testing.T) {
	338	+	img := image.NewRGBA(image.Rect(0, 0, 1, 1))
	339	+	good := ports.SecurityState{Generation: 1}
	340	+	// frameIn writes the frame to link and returns its directory and path.
	341	+	frameIn := func(t *testing.T) (string, string) {
	342	+	t.Helper()
	343	+	dir := t.TempDir()
	344	+	frame := filepath.Join(dir, "frame-000001.png")
	345	+	if err := writePNGSecure(frame, img, nil, ports.SecurityState{}); err != nil {
	346	+	t.Fatal(err)
	347	+	}
	348	+	return dir, frame
	349	+	}
	350	+	// onlyFrame fails unless the frame is all the directory holds: no
	351	+	// latest.png and no latest.png.tmp.
	352	+	onlyFrame := func(t *testing.T, dir string) {
	353	+	t.Helper()
	354	+	if entries, _ := os.ReadDir(dir); len(entries) != 1 {
	355	+	t.Fatalf("latest.png or a temp file delivered: %v", entries)
	356	+	}
	357	+	}
	358	+	t.Run("protected state", func(t *testing.T) {
	359	+	dir, frame := frameIn(t)
	360	+	// A protected state is refused without reading the live snapshot.
	361	+	gate := portsmocks.NewMockSessionSecurity(t)
	362	+	if written, err := linkPNGSecure(frame, filepath.Join(dir, "latest.png"), img, gate, ports.SecurityState{Generation: 1, Protected: true}); err != nil \|\| written {
	363	+	t.Fatalf("link = %v, %v", written, err)
	364	+	}
	365	+	onlyFrame(t, dir)
	366	+	})
	367	+	t.Run("protected snapshot", func(t *testing.T) {
	368	+	dir, frame := frameIn(t)
	369	+	gate := portsmocks.NewMockSessionSecurity(t)
	370	+	gate.EXPECT().Snapshot().Return(ports.SecurityState{Generation: 1, Protected: true})
	371	+	if written, err := linkPNGSecure(frame, filepath.Join(dir, "latest.png"), img, gate, good); err != nil \|\| written {
	372	+	t.Fatalf("link = %v, %v", written, err)
	373	+	}
	374	+	onlyFrame(t, dir)
	375	+	})
	376	+	t.Run("changed after the link", func(t *testing.T) {
	377	+	dir, frame := frameIn(t)
	378	+	gate := portsmocks.NewMockSessionSecurity(t)
	379	+	gate.EXPECT().Snapshot().Return(good).Once()
	380	+	gate.EXPECT().Snapshot().Return(ports.SecurityState{Generation: 2}).Once()
	381	+	if written, err := linkPNGSecure(frame, filepath.Join(dir, "latest.png"), img, gate, good); err != nil \|\| written {
	382	+	t.Fatalf("link = %v, %v", written, err)
	383	+	}
	384	+	onlyFrame(t, dir)
	385	+	})
	386	+	t.Run("rename failure", func(t *testing.T) {
	387	+	dir, frame := frameIn(t)
	388	+	// A non-empty directory cannot be replaced by a file.
	389	+	latest := filepath.Join(dir, "latest.png")
	390	+	if err := os.MkdirAll(filepath.Join(latest, "keep"), 0o700); err != nil {
	391	+	t.Fatal(err)
	392	+	}
	393	+	if written, err := linkPNGSecure(frame, latest, img, nil, ports.SecurityState{}); err == nil \|\| written {
	394	+	t.Fatalf("link = %v, %v; want an error", written, err)
	395	+	}
	396	+	if _, err := os.Stat(latest + ".tmp"); !os.IsNotExist(err) {
	397	+	t.Fatalf("temporary link left behind: %v", err)
	398	+	}
	399	+	if _, err := os.Stat(filepath.Join(latest, "keep")); err != nil {
	400	+	t.Fatalf("existing directory touched: %v", err)
	401	+	}
	402	+	})
	403	+	t.Run("no source file", func(t *testing.T) {
	404	+	dir := t.TempDir()
	405	+	latest := filepath.Join(dir, "latest.png")
	406	+	if written, err := linkPNGSecure(filepath.Join(dir, "missing.png"), latest, img, nil, ports.SecurityState{}); err != nil \|\| !written {
	407	+	t.Fatalf("link = %v, %v", written, err)
	408	+	}
	409	+	fi, err := os.Stat(latest)
	410	+	if err != nil {
	411	+	t.Fatal(err)
	412	+	}
	413	+	if !fi.Mode().IsRegular() {
	414	+	t.Fatalf("latest.png is %v", fi.Mode())
	415	+	}
	416	+	f, err := os.Open(latest)
	417	+	if err != nil {
	418	+	t.Fatal(err)
	419	+	}
	420	+	defer f.Close()
	421	+	if _, err := png.Decode(f); err != nil {
	422	+	t.Fatal(err)
	423	+	}
	424	+	// Encoded on its own: nothing else in the directory shares it.
	425	+	if entries, _ := os.ReadDir(dir); len(entries) != 1 {
	426	+	t.Fatalf("unexpected files: %v", entries)
	427	+	}
	428	+	if n, ok := linkCount(fi); ok && n != 1 {
	429	+	t.Fatalf("latest.png has %d links, want 1", n)
	430	+	}
	431	+	})
	432	+	}
	433	+
	434	+	// linkCount returns the hard link count of fi where the platform reports it.
	435	+	func linkCount(fi os.FileInfo) (uint64, bool) {
	436	+	st, ok := fi.Sys().(*syscall.Stat_t)
	437	+	if !ok {
	438	+	return 0, false
	439	+	}
	440	+	return uint64(st.Nlink), true
	441	+	}
	442	+
272	443		func TestWritePNGAtomic(t *testing.T) {
273	444		dir := t.TempDir()
274	445		img := image.NewRGBA(image.Rect(0, 0, 1, 1))


@@ -14,6 +14,7 @@ import (
14	"strconv"	14		"strconv"
15	"strings"	15		"strings"
16	"sync"	16		"sync"
		17	+	"syscall"
17	"testing"	18		"testing"
18	"time"	19		"time"
19		20

@@ -269,6 +270,176 @@ func TestRunTrimsRendererOnTick(t *testing.T) {
269	}	270		}
270	}	271		}
271		272
		273	+	// latest.png is the frame file linked, not encoded again: the same pixels,
		274	+	// published atomically, and nothing left behind.
		275	+	func TestLinkPNGSecure(t *testing.T) {
		276	+	dir := t.TempDir()
		277	+	img := image.NewRGBA(image.Rect(0, 0, 3, 2))
		278	+	for i := range img.Pix {
		279	+	img.Pix[i] = byte(i * 7)
		280	+	}
		281	+	frame, latest := filepath.Join(dir, "frame-000001.png"), filepath.Join(dir, "latest.png")
		282	+	if err := writePNGSecure(frame, img, nil, ports.SecurityState{}); err != nil {
		283	+	t.Fatal(err)
		284	+	}
		285	+	// An older latest.png is replaced.
		286	+	if err := os.WriteFile(latest, []byte("old"), 0o600); err != nil {
		287	+	t.Fatal(err)
		288	+	}
		289	+	if written, err := linkPNGSecure(frame, latest, img, nil, ports.SecurityState{}); err != nil \|\| !written {
		290	+	t.Fatalf("link = %v, %v", written, err)
		291	+	}
		292	+	a, err := os.ReadFile(frame)
		293	+	if err != nil {
		294	+	t.Fatal(err)
		295	+	}
		296	+	b, err := os.ReadFile(latest)
		297	+	if err != nil {
		298	+	t.Fatal(err)
		299	+	}
		300	+	if !bytes.Equal(a, b) {
		301	+	t.Fatal("latest.png differs from the frame")
		302	+	}
		303	+	fi, err := os.Stat(frame)
		304	+	if err != nil {
		305	+	t.Fatal(err)
		306	+	}
		307	+	li, err := os.Stat(latest)
		308	+	if err != nil {
		309	+	t.Fatal(err)
		310	+	}
		311	+	if !os.SameFile(fi, li) {
		312	+	t.Fatal("latest.png is a copy, not a hard link to the frame")
		313	+	}
		314	+	got, err := png.Decode(bytes.NewReader(b))
		315	+	if err != nil {
		316	+	t.Fatal(err)
		317	+	}
		318	+	for y := range 2 {
		319	+	for x := range 3 {
		320	+	if g, w := color.RGBAModel.Convert(got.At(x, y)), img.RGBAAt(x, y); g != color.Color(w) {
		321	+	t.Errorf("(%d,%d) = %v, want %v", x, y, g, w)
		322	+	}
		323	+	}
		324	+	}
		325	+	entries, err := os.ReadDir(dir)
		326	+	if err != nil {
		327	+	t.Fatal(err)
		328	+	}
		329	+	if len(entries) != 2 {
		330	+	t.Fatalf("leftover files: %v", entries)
		331	+	}
		332	+	}
		333	+
		334	+	// A frame that crossed a security change is not published as latest.png, a
		335	+	// failed rename leaves no temporary file, and a source that cannot be linked
		336	+	// is encoded instead.
		337	+	func TestLinkPNGSecureSkipsAndFallsBack(t *testing.T) {
		338	+	img := image.NewRGBA(image.Rect(0, 0, 1, 1))
		339	+	good := ports.SecurityState{Generation: 1}
		340	+	// frameIn writes the frame to link and returns its directory and path.
		341	+	frameIn := func(t *testing.T) (string, string) {
		342	+	t.Helper()
		343	+	dir := t.TempDir()
		344	+	frame := filepath.Join(dir, "frame-000001.png")
		345	+	if err := writePNGSecure(frame, img, nil, ports.SecurityState{}); err != nil {
		346	+	t.Fatal(err)
		347	+	}
		348	+	return dir, frame
		349	+	}
		350	+	// onlyFrame fails unless the frame is all the directory holds: no
		351	+	// latest.png and no latest.png.tmp.
		352	+	onlyFrame := func(t *testing.T, dir string) {
		353	+	t.Helper()
		354	+	if entries, _ := os.ReadDir(dir); len(entries) != 1 {
		355	+	t.Fatalf("latest.png or a temp file delivered: %v", entries)
		356	+	}
		357	+	}
		358	+	t.Run("protected state", func(t *testing.T) {
		359	+	dir, frame := frameIn(t)
		360	+	// A protected state is refused without reading the live snapshot.
		361	+	gate := portsmocks.NewMockSessionSecurity(t)
		362	+	if written, err := linkPNGSecure(frame, filepath.Join(dir, "latest.png"), img, gate, ports.SecurityState{Generation: 1, Protected: true}); err != nil \|\| written {
		363	+	t.Fatalf("link = %v, %v", written, err)
		364	+	}
		365	+	onlyFrame(t, dir)
		366	+	})
		367	+	t.Run("protected snapshot", func(t *testing.T) {
		368	+	dir, frame := frameIn(t)
		369	+	gate := portsmocks.NewMockSessionSecurity(t)
		370	+	gate.EXPECT().Snapshot().Return(ports.SecurityState{Generation: 1, Protected: true})
		371	+	if written, err := linkPNGSecure(frame, filepath.Join(dir, "latest.png"), img, gate, good); err != nil \|\| written {
		372	+	t.Fatalf("link = %v, %v", written, err)
		373	+	}
		374	+	onlyFrame(t, dir)
		375	+	})
		376	+	t.Run("changed after the link", func(t *testing.T) {
		377	+	dir, frame := frameIn(t)
		378	+	gate := portsmocks.NewMockSessionSecurity(t)
		379	+	gate.EXPECT().Snapshot().Return(good).Once()
		380	+	gate.EXPECT().Snapshot().Return(ports.SecurityState{Generation: 2}).Once()
		381	+	if written, err := linkPNGSecure(frame, filepath.Join(dir, "latest.png"), img, gate, good); err != nil \|\| written {
		382	+	t.Fatalf("link = %v, %v", written, err)
		383	+	}
		384	+	onlyFrame(t, dir)
		385	+	})
		386	+	t.Run("rename failure", func(t *testing.T) {
		387	+	dir, frame := frameIn(t)
		388	+	// A non-empty directory cannot be replaced by a file.
		389	+	latest := filepath.Join(dir, "latest.png")
		390	+	if err := os.MkdirAll(filepath.Join(latest, "keep"), 0o700); err != nil {
		391	+	t.Fatal(err)
		392	+	}
		393	+	if written, err := linkPNGSecure(frame, latest, img, nil, ports.SecurityState{}); err == nil \|\| written {
		394	+	t.Fatalf("link = %v, %v; want an error", written, err)
		395	+	}
		396	+	if _, err := os.Stat(latest + ".tmp"); !os.IsNotExist(err) {
		397	+	t.Fatalf("temporary link left behind: %v", err)
		398	+	}
		399	+	if _, err := os.Stat(filepath.Join(latest, "keep")); err != nil {
		400	+	t.Fatalf("existing directory touched: %v", err)
		401	+	}
		402	+	})
		403	+	t.Run("no source file", func(t *testing.T) {
		404	+	dir := t.TempDir()
		405	+	latest := filepath.Join(dir, "latest.png")
		406	+	if written, err := linkPNGSecure(filepath.Join(dir, "missing.png"), latest, img, nil, ports.SecurityState{}); err != nil \|\| !written {
		407	+	t.Fatalf("link = %v, %v", written, err)
		408	+	}
		409	+	fi, err := os.Stat(latest)
		410	+	if err != nil {
		411	+	t.Fatal(err)
		412	+	}
		413	+	if !fi.Mode().IsRegular() {
		414	+	t.Fatalf("latest.png is %v", fi.Mode())
		415	+	}
		416	+	f, err := os.Open(latest)
		417	+	if err != nil {
		418	+	t.Fatal(err)
		419	+	}
		420	+	defer f.Close()
		421	+	if _, err := png.Decode(f); err != nil {
		422	+	t.Fatal(err)
		423	+	}
		424	+	// Encoded on its own: nothing else in the directory shares it.
		425	+	if entries, _ := os.ReadDir(dir); len(entries) != 1 {
		426	+	t.Fatalf("unexpected files: %v", entries)
		427	+	}
		428	+	if n, ok := linkCount(fi); ok && n != 1 {
		429	+	t.Fatalf("latest.png has %d links, want 1", n)
		430	+	}
		431	+	})
		432	+	}
		433	+
		434	+	// linkCount returns the hard link count of fi where the platform reports it.
		435	+	func linkCount(fi os.FileInfo) (uint64, bool) {
		436	+	st, ok := fi.Sys().(*syscall.Stat_t)
		437	+	if !ok {
		438	+	return 0, false
		439	+	}
		440	+	return uint64(st.Nlink), true
		441	+	}
		442	+
272	func TestWritePNGAtomic(t *testing.T) {	443		func TestWritePNGAtomic(t *testing.T) {
273	dir := t.TempDir()	444		dir := t.TempDir()
274	img := image.NewRGBA(image.Rect(0, 0, 1, 1))	445		img := image.NewRGBA(image.Rect(0, 0, 1, 1))

M tools/demo/Dockerfile +24 -15


@@ -4,23 +4,32 @@ RUN --mount=type=cache,target=/var/cache/pacman/pkg,sharing=locked \
4	4		pacman -Sy --noconfirm archlinux-keyring \
5	5		&& pacman -Syu --noconfirm --needed \
6	6		firefox foot btop fastfetch eza mousepad \
7		-	ttf-jetbrains-mono-nerd noto-fonts noto-fonts-emoji dbus mesa vulkan-swrast
	7	+	ttf-jetbrains-mono-nerd noto-fonts noto-fonts-emoji dbus mesa vulkan-radeon \
	8	+	tesseract tesseract-data-eng at-spi2-core python
8	9		# Arch's btop carries file capabilities that rootless Docker cannot grant.
9	10		RUN install -m 0755 /usr/bin/btop /usr/local/bin/btop
10		-	RUN useradd -m -u 1000 demo \
11		-	&& mkdir -p /home/demo/firefox /home/demo/.config/btop /run/wl \
12		-	&& chmod 0700 /run/wl
13		-	COPY firefox/user.js /home/demo/firefox/user.js
14		-	COPY firefox/userChrome.css /home/demo/firefox/chrome/userChrome.css
15		-	COPY foot.ini /home/demo/.config/foot/foot.ini
16		-	COPY btop.conf /home/demo/.config/btop/btop.conf
17		-	COPY btop-vesper.theme /home/demo/.config/btop/themes/vesper.theme
18		-	COPY site /home/demo/site
	11	+	# Mousepad warns about running as root, so it runs as demo (uid 1000).
	12	+	RUN useradd -m -u 1000 demo && mkdir -p /root/firefox /root/.config/btop /run/wl && chmod 0755 /run/wl
	13	+	COPY firefox/user.js /root/firefox/user.js
	14	+	COPY firefox/userChrome.css /root/firefox/chrome/userChrome.css
	15	+	COPY foot.ini /root/.config/foot/foot.ini
	16	+	COPY btop.conf /root/.config/btop/btop.conf
	17	+	COPY btop-vesper.theme /root/.config/btop/themes/vesper.theme
	18	+	COPY demo-entry demo-hint-label /usr/local/bin/
	19	+	# One config for both users: the compositor (root) reads it, Mousepad (demo) edits it.
19	20		COPY config /home/demo/config
20		-	RUN chown -R demo:demo /home/demo /run/wl
21		-	ENV HOME=/home/demo XDG_RUNTIME_DIR=/run/wl WAYLAND_DISPLAY=wayland-0 \
	21	+	COPY mousepad.keyfile /home/demo/.config/glib-2.0/settings/keyfile
	22	+	RUN chown -R demo:demo /home/demo
	23	+	ENV HOME=/root XDG_RUNTIME_DIR=/run/wl WAYLAND_DISPLAY=wayland-0 \
22	24		GTK_THEME=Adwaita:dark ADW_DEBUG_COLOR_SCHEME=prefer-dark \
	25	+	DBUS_SESSION_BUS_ADDRESS=unix:path=/run/wl/bus \
23	26		MOZ_ENABLE_WAYLAND=1 LIBGL_ALWAYS_SOFTWARE=1 COLORTERM=truecolor
24		-	WORKDIR /home/demo
25		-	# The recorder grants demo access to only the disposable headless socket.
26		-	CMD ["sleep", "infinity"]
	27	+	# LIBGL_ALWAYS_SOFTWARE applies to the apps' OpenGL, not to the compositor (Vulkan).
	28	+	WORKDIR /root
	29	+	# Everything runs as the container's root, which rootless Docker maps to the
	30	+	# host user, so the bind-mounted work directory stays owned by the host user.
	31	+	# The compositor (bind-mounted at /usr/local/bin/neferwl) and the apps share one
	32	+	# session bus, which AT-SPI needs. The compositor renders on the host GPU through
	33	+	# /dev/dri: vulkan-radeon is its Mesa driver here (swap in vulkan-intel for an
	34	+	# Intel GPU). Software Vulkan (lvp) cannot export the sync files hint scans need.
	35	+	ENTRYPOINT ["/usr/local/bin/demo-entry"]


@@ -4,23 +4,32 @@ RUN --mount=type=cache,target=/var/cache/pacman/pkg,sharing=locked \
4		pacman -Sy --noconfirm archlinux-keyring \	4		pacman -Sy --noconfirm archlinux-keyring \
5		&& pacman -Syu --noconfirm --needed \	5		&& pacman -Syu --noconfirm --needed \
6		firefox foot btop fastfetch eza mousepad \	6		firefox foot btop fastfetch eza mousepad \
7	-	ttf-jetbrains-mono-nerd noto-fonts noto-fonts-emoji dbus mesa vulkan-swrast	7	+	ttf-jetbrains-mono-nerd noto-fonts noto-fonts-emoji dbus mesa vulkan-radeon \
			8	+	tesseract tesseract-data-eng at-spi2-core python
8		# Arch's btop carries file capabilities that rootless Docker cannot grant.	9		# Arch's btop carries file capabilities that rootless Docker cannot grant.
9		RUN install -m 0755 /usr/bin/btop /usr/local/bin/btop	10		RUN install -m 0755 /usr/bin/btop /usr/local/bin/btop
10	-	RUN useradd -m -u 1000 demo \	11	+	# Mousepad warns about running as root, so it runs as demo (uid 1000).
11	-	&& mkdir -p /home/demo/firefox /home/demo/.config/btop /run/wl \	12	+	RUN useradd -m -u 1000 demo && mkdir -p /root/firefox /root/.config/btop /run/wl && chmod 0755 /run/wl
12	-	&& chmod 0700 /run/wl	13	+	COPY firefox/user.js /root/firefox/user.js
13	-	COPY firefox/user.js /home/demo/firefox/user.js	14	+	COPY firefox/userChrome.css /root/firefox/chrome/userChrome.css
14	-	COPY firefox/userChrome.css /home/demo/firefox/chrome/userChrome.css	15	+	COPY foot.ini /root/.config/foot/foot.ini
15	-	COPY foot.ini /home/demo/.config/foot/foot.ini	16	+	COPY btop.conf /root/.config/btop/btop.conf
16	-	COPY btop.conf /home/demo/.config/btop/btop.conf	17	+	COPY btop-vesper.theme /root/.config/btop/themes/vesper.theme
17	-	COPY btop-vesper.theme /home/demo/.config/btop/themes/vesper.theme	18	+	COPY demo-entry demo-hint-label /usr/local/bin/
18	-	COPY site /home/demo/site	19	+	# One config for both users: the compositor (root) reads it, Mousepad (demo) edits it.
19		COPY config /home/demo/config	20		COPY config /home/demo/config
20	-	RUN chown -R demo:demo /home/demo /run/wl	21	+	COPY mousepad.keyfile /home/demo/.config/glib-2.0/settings/keyfile
21	-	ENV HOME=/home/demo XDG_RUNTIME_DIR=/run/wl WAYLAND_DISPLAY=wayland-0 \	22	+	RUN chown -R demo:demo /home/demo
			23	+	ENV HOME=/root XDG_RUNTIME_DIR=/run/wl WAYLAND_DISPLAY=wayland-0 \
22		GTK_THEME=Adwaita:dark ADW_DEBUG_COLOR_SCHEME=prefer-dark \	24		GTK_THEME=Adwaita:dark ADW_DEBUG_COLOR_SCHEME=prefer-dark \
			25	+	DBUS_SESSION_BUS_ADDRESS=unix:path=/run/wl/bus \
23		MOZ_ENABLE_WAYLAND=1 LIBGL_ALWAYS_SOFTWARE=1 COLORTERM=truecolor	26		MOZ_ENABLE_WAYLAND=1 LIBGL_ALWAYS_SOFTWARE=1 COLORTERM=truecolor
24	-	WORKDIR /home/demo	27	+	# LIBGL_ALWAYS_SOFTWARE applies to the apps' OpenGL, not to the compositor (Vulkan).
25	-	# The recorder grants demo access to only the disposable headless socket.	28	+	WORKDIR /root
26	-	CMD ["sleep", "infinity"]	29	+	# Everything runs as the container's root, which rootless Docker maps to the
			30	+	# host user, so the bind-mounted work directory stays owned by the host user.
			31	+	# The compositor (bind-mounted at /usr/local/bin/neferwl) and the apps share one
			32	+	# session bus, which AT-SPI needs. The compositor renders on the host GPU through
			33	+	# /dev/dri: vulkan-radeon is its Mesa driver here (swap in vulkan-intel for an
			34	+	# Intel GPU). Software Vulkan (lvp) cannot export the sync files hint scans need.
			35	+	ENTRYPOINT ["/usr/local/bin/demo-entry"]

M tools/demo/config +5 -0


@@ -11,12 +11,17 @@ terminal.auto-open = off
11	11		stash.spawn-terminal = off
12	12		stash.capture = off
13	13		keyboard.layout = us
	14	+	hint.enable = on
	15	+	hint.targets = links
14	16
15	17		# Columns fill a band; extra windows flow below.
16	18		# Super+Up / Down travel between bands.
17	19		# Super+Shift+s sets a window aside.
18	20		# Super+s brings your stash into view.
19	21		# Super+o opens the overview.
	22	+	# Super+Shift+f makes a window fullscreen.
	23	+	# Super+g labels the links; type one to click it.
	24	+	bind.cmd+g = hint
20	25		bind.cmd+code:3 = focus-workspace 2
21	26		bind.cmd+code:2 = focus-workspace 1
22	27


@@ -11,12 +11,17 @@ terminal.auto-open = off
11	stash.spawn-terminal = off	11		stash.spawn-terminal = off
12	stash.capture = off	12		stash.capture = off
13	keyboard.layout = us	13		keyboard.layout = us
		14	+	hint.enable = on
		15	+	hint.targets = links
14		16
15	# Columns fill a band; extra windows flow below.	17		# Columns fill a band; extra windows flow below.
16	# Super+Up / Down travel between bands.	18		# Super+Up / Down travel between bands.
17	# Super+Shift+s sets a window aside.	19		# Super+Shift+s sets a window aside.
18	# Super+s brings your stash into view.	20		# Super+s brings your stash into view.
19	# Super+o opens the overview.	21		# Super+o opens the overview.
		22	+	# Super+Shift+f makes a window fullscreen.
		23	+	# Super+g labels the links; type one to click it.
		24	+	bind.cmd+g = hint
20	bind.cmd+code:3 = focus-workspace 2	25		bind.cmd+code:3 = focus-workspace 2
21	bind.cmd+code:2 = focus-workspace 1	26		bind.cmd+code:2 = focus-workspace 1
22		27

A tools/demo/demo-entry +9 -0

@@ -0,0 +1,9 @@
	1	+	#!/bin/sh
	2	+	# Starts the session bus shared by the compositor and the apps, turns
	3	+	# accessibility on before any app starts, then runs the compositor as the
	4	+	# container's main process: the container exits with it.
	5	+	set -eu
	6	+	dbus-daemon --session --fork --address="$DBUS_SESSION_BUS_ADDRESS"
	7	+	dbus-send --session --print-reply --dest=org.a11y.Bus /org/a11y/bus \
	8	+	org.freedesktop.DBus.Properties.Set string:org.a11y.Status string:IsEnabled variant:boolean:true >/dev/null
	9	+	exec /usr/local/bin/neferwl "$@"

@@ -0,0 +1,9 @@
	1	+	#!/bin/sh
	2	+	# Starts the session bus shared by the compositor and the apps, turns
	3	+	# accessibility on before any app starts, then runs the compositor as the
	4	+	# container's main process: the container exits with it.
	5	+	set -eu
	6	+	dbus-daemon --session --fork --address="$DBUS_SESSION_BUS_ADDRESS"
	7	+	dbus-send --session --print-reply --dest=org.a11y.Bus /org/a11y/bus \
	8	+	org.freedesktop.DBus.Properties.Set string:org.a11y.Status string:IsEnabled variant:boolean:true >/dev/null
	9	+	exec /usr/local/bin/neferwl "$@"

A tools/demo/demo-hint-label +76 -0

@@ -0,0 +1,76 @@
	1	+	#!/usr/bin/env python3
	2	+	"""Work out the hint label of a link, to type it the way a user reads it.
	3	+
	4	+	demo-hint-label ocr
	5	+	Reads the frame on screen now (the compositor's latest screenshot) with
	6	+	tesseract and keeps the words and their boxes in /tmp/hint-words.json.
	7	+	Run it while the page is still, before the hint bind, so that the
	8	+	lookup after the bind needs no OCR.
	9	+
	10	+	demo-hint-label label <since-epoch> <word>...
	11	+	Waits for the hint dump (NEFERWL_HINT_DEBUG=1) of the scan started after
	12	+	<since-epoch>, finds the link target that shows all the words and prints
	13	+	its label. Labels follow internal/core/hint.go: the targets whose centre
	14	+	is on the output, sorted by top then left edge, one letter each from the
	15	+	home-row alphabet, two letters past 26 targets.
	16	+	"""
	17	+	import csv, glob, io, json, os, re, shutil, subprocess, sys, time
	18	+
	19	+	ALPHABET = "fjdkslaghrueiwoqptyvbcnxmz"
	20	+	STATE = os.environ.get("XDG_STATE_HOME", "/work/state")
	21	+	DUMPS = os.path.join(STATE, "neferwl", "hints")
	22	+	SHOT = "/work/shots/latest.png"
	23	+	WORDS = "/tmp/hint-words.json"
	24	+
	25	+
	26	+	def ocr():
	27	+	shutil.copyfile(SHOT, "/tmp/hint-ocr.png")
	28	+	out = subprocess.run(["tesseract", "/tmp/hint-ocr.png", "stdout", "--psm", "11", "tsv"],
	29	+	capture_output=True, text=True, check=True).stdout
	30	+	words = []
	31	+	for row in csv.DictReader(io.StringIO(out), delimiter="\t", quoting=csv.QUOTE_NONE):
	32	+	text = re.sub(r"\W", "", row["text"] or "").lower()
	33	+	if text and float(row["conf"]) > 30:
	34	+	x, y, w, h = (int(row[k]) for k in ("left", "top", "width", "height"))
	35	+	words.append([text, x + w / 2, y + h / 2])
	36	+	with open(WORDS, "w") as f:
	37	+	json.dump(words, f)
	38	+
	39	+
	40	+	def newest_dump(since):
	41	+	deadline = time.time() + 15
	42	+	while time.time() < deadline:
	43	+	for path in sorted(glob.glob(os.path.join(DUMPS, "*.json")), reverse=True):
	44	+	if os.path.getmtime(path) >= since:
	45	+	return path
	46	+	time.sleep(0.05)
	47	+	sys.exit("no hint scan dump after the bind")
	48	+
	49	+
	50	+	def label(since, wanted):
	51	+	with open(newest_dump(since)) as f:
	52	+	meta = json.load(f)
	53	+	with open(WORDS) as f:
	54	+	words = json.load(f)
	55	+	# Core keeps the targets whose centre is on the output, then sorts them.
	56	+	width, height = meta["width"], meta["height"]
	57	+	targets = [t for t in meta["targets"]
	58	+	if 0 <= (t["rect"][0] + t["rect"][2]) / 2 < width and 0 <= (t["rect"][1] + t["rect"][3]) / 2 < height]
	59	+	targets.sort(key=lambda t: (t["rect"][1], t["rect"][0]))
	60	+	for i, t in enumerate(targets):
	61	+	x0, y0, x1, y1 = t["rect"]
	62	+	inside = {w for w, cx, cy in words if x0 <= cx < x1 and y0 <= cy < y1}
	63	+	if t["kind"] == "link" and set(wanted) <= inside:
	64	+	n = len(ALPHABET)
	65	+	print("link %d of %d shows %s" % (i + 1, len(targets), " ".join(wanted)), file=sys.stderr)
	66	+	print(ALPHABET[i] if len(targets) <= n else ALPHABET[i // n] + ALPHABET[i % n])
	67	+	return
	68	+	sys.exit("no link shows %s (%d targets, %d words read)" % (" ".join(wanted), len(targets), len(words)))
	69	+
	70	+
	71	+	if sys.argv[1:2] == ["ocr"]:
	72	+	ocr()
	73	+	elif sys.argv[1:2] == ["label"] and len(sys.argv) > 3:
	74	+	label(float(sys.argv[2]), [w.lower() for w in sys.argv[3:]])
	75	+	else:
	76	+	sys.exit(__doc__)

@@ -0,0 +1,76 @@
	1	+	#!/usr/bin/env python3
	2	+	"""Work out the hint label of a link, to type it the way a user reads it.
	3	+
	4	+	demo-hint-label ocr
	5	+	Reads the frame on screen now (the compositor's latest screenshot) with
	6	+	tesseract and keeps the words and their boxes in /tmp/hint-words.json.
	7	+	Run it while the page is still, before the hint bind, so that the
	8	+	lookup after the bind needs no OCR.
	9	+
	10	+	demo-hint-label label <since-epoch> <word>...
	11	+	Waits for the hint dump (NEFERWL_HINT_DEBUG=1) of the scan started after
	12	+	<since-epoch>, finds the link target that shows all the words and prints
	13	+	its label. Labels follow internal/core/hint.go: the targets whose centre
	14	+	is on the output, sorted by top then left edge, one letter each from the
	15	+	home-row alphabet, two letters past 26 targets.
	16	+	"""
	17	+	import csv, glob, io, json, os, re, shutil, subprocess, sys, time
	18	+
	19	+	ALPHABET = "fjdkslaghrueiwoqptyvbcnxmz"
	20	+	STATE = os.environ.get("XDG_STATE_HOME", "/work/state")
	21	+	DUMPS = os.path.join(STATE, "neferwl", "hints")
	22	+	SHOT = "/work/shots/latest.png"
	23	+	WORDS = "/tmp/hint-words.json"
	24	+
	25	+
	26	+	def ocr():
	27	+	shutil.copyfile(SHOT, "/tmp/hint-ocr.png")
	28	+	out = subprocess.run(["tesseract", "/tmp/hint-ocr.png", "stdout", "--psm", "11", "tsv"],
	29	+	capture_output=True, text=True, check=True).stdout
	30	+	words = []
	31	+	for row in csv.DictReader(io.StringIO(out), delimiter="\t", quoting=csv.QUOTE_NONE):
	32	+	text = re.sub(r"\W", "", row["text"] or "").lower()
	33	+	if text and float(row["conf"]) > 30:
	34	+	x, y, w, h = (int(row[k]) for k in ("left", "top", "width", "height"))
	35	+	words.append([text, x + w / 2, y + h / 2])
	36	+	with open(WORDS, "w") as f:
	37	+	json.dump(words, f)
	38	+
	39	+
	40	+	def newest_dump(since):
	41	+	deadline = time.time() + 15
	42	+	while time.time() < deadline:
	43	+	for path in sorted(glob.glob(os.path.join(DUMPS, "*.json")), reverse=True):
	44	+	if os.path.getmtime(path) >= since:
	45	+	return path
	46	+	time.sleep(0.05)
	47	+	sys.exit("no hint scan dump after the bind")
	48	+
	49	+
	50	+	def label(since, wanted):
	51	+	with open(newest_dump(since)) as f:
	52	+	meta = json.load(f)
	53	+	with open(WORDS) as f:
	54	+	words = json.load(f)
	55	+	# Core keeps the targets whose centre is on the output, then sorts them.
	56	+	width, height = meta["width"], meta["height"]
	57	+	targets = [t for t in meta["targets"]
	58	+	if 0 <= (t["rect"][0] + t["rect"][2]) / 2 < width and 0 <= (t["rect"][1] + t["rect"][3]) / 2 < height]
	59	+	targets.sort(key=lambda t: (t["rect"][1], t["rect"][0]))
	60	+	for i, t in enumerate(targets):
	61	+	x0, y0, x1, y1 = t["rect"]
	62	+	inside = {w for w, cx, cy in words if x0 <= cx < x1 and y0 <= cy < y1}
	63	+	if t["kind"] == "link" and set(wanted) <= inside:
	64	+	n = len(ALPHABET)
	65	+	print("link %d of %d shows %s" % (i + 1, len(targets), " ".join(wanted)), file=sys.stderr)
	66	+	print(ALPHABET[i] if len(targets) <= n else ALPHABET[i // n] + ALPHABET[i % n])
	67	+	return
	68	+	sys.exit("no link shows %s (%d targets, %d words read)" % (" ".join(wanted), len(targets), len(words)))
	69	+
	70	+
	71	+	if sys.argv[1:2] == ["ocr"]:
	72	+	ocr()
	73	+	elif sys.argv[1:2] == ["label"] and len(sys.argv) > 3:
	74	+	label(float(sys.argv[2]), [w.lower() for w in sys.argv[3:]])
	75	+	else:
	76	+	sys.exit(__doc__)

M tools/demo/input.txt +32 -1

@@ -1,8 +1,17 @@
1	1		move 1918 1078
2		-	sleep 12s
	2	+	sleep 5s
	3	+	# Unseen warm-up while Firefox sits beside the terminal: a hint bind cancelled
	4	+	# at once still runs its scan, without drawing labels. It connects to the
	5	+	# accessibility bus early; more of them below (each right after a key press, in
	6	+	# an existing pause) let Firefox expose its page content by the end.
	7	+	key Super+g
	8	+	key Escape
	9	+	sleep 7s
3	10		key Super+Up
4	11		sleep 1500ms
5	12		key Super+Down
	13	+	key Super+g
	14	+	key Escape
6	15		sleep 1500ms
7	16		key Super+Shift+s
8	17		sleep 500ms

@@ -11,6 +20,8 @@ sleep 1500ms
11	20		key Super+2
12	21		key Super+Shift+1
13	22		key Super+1
	23	+	key Super+g
	24	+	key Escape
14	25		sleep 3s
15	26		key Super+Shift+s
16	27		sleep 500ms

@@ -18,8 +29,12 @@ key Super+s
18	29		sleep 1500ms
19	30		key Super+s
20	31		key Super+Left
	32	+	key Super+g
	33	+	key Escape
21	34		sleep 2s
22	35		key Super+Right
	36	+	key Super+g
	37	+	key Escape
23	38		sleep 2s
24	39		key Super+Left
25	40		sleep 1s

@@ -40,4 +55,20 @@ sleep 1500ms
40	55		key Up
41	56		sleep 1s
42	57		key Return
	58	+	sleep 700ms
	59	+	# Back on workspace 1: focus Firefox and fullscreen it. Once the page is still,
	60	+	# read it, then hint mode starts and the label of the Wikipedia result
	61	+	# ("Wayland (protocol) - Wikipedia") is typed.
	62	+	key Super+Right
	63	+	sleep 600ms
	64	+	key Super+Shift+f
	65	+	sleep 900ms
	66	+	hint-read
	67	+	sleep 200ms
	68	+	key Super+g
	69	+	sleep 1500ms
	70	+	hint-click wikipedia protocol
	71	+	# Firefox loads the page; the compositor writes no frame while it is still, so
	72	+	# a pointer move marks the end of the hold.
43	73		sleep 4s
	74	+	move 1500 900

@@ -1,8 +1,17 @@
1		move 1918 1078	1		move 1918 1078
2	-	sleep 12s	2	+	sleep 5s
			3	+	# Unseen warm-up while Firefox sits beside the terminal: a hint bind cancelled
			4	+	# at once still runs its scan, without drawing labels. It connects to the
			5	+	# accessibility bus early; more of them below (each right after a key press, in
			6	+	# an existing pause) let Firefox expose its page content by the end.
			7	+	key Super+g
			8	+	key Escape
			9	+	sleep 7s
3		key Super+Up	10		key Super+Up
4		sleep 1500ms	11		sleep 1500ms
5		key Super+Down	12		key Super+Down
			13	+	key Super+g
			14	+	key Escape
6		sleep 1500ms	15		sleep 1500ms
7		key Super+Shift+s	16		key Super+Shift+s
8		sleep 500ms	17		sleep 500ms

@@ -11,6 +20,8 @@ sleep 1500ms
11		key Super+2	20		key Super+2
12		key Super+Shift+1	21		key Super+Shift+1
13		key Super+1	22		key Super+1
			23	+	key Super+g
			24	+	key Escape
14		sleep 3s	25		sleep 3s
15		key Super+Shift+s	26		key Super+Shift+s
16		sleep 500ms	27		sleep 500ms

@@ -18,8 +29,12 @@ key Super+s
18		sleep 1500ms	29		sleep 1500ms
19		key Super+s	30		key Super+s
20		key Super+Left	31		key Super+Left
			32	+	key Super+g
			33	+	key Escape
21		sleep 2s	34		sleep 2s
22		key Super+Right	35		key Super+Right
			36	+	key Super+g
			37	+	key Escape
23		sleep 2s	38		sleep 2s
24		key Super+Left	39		key Super+Left
25		sleep 1s	40		sleep 1s

@@ -40,4 +55,20 @@ sleep 1500ms
40		key Up	55		key Up
41		sleep 1s	56		sleep 1s
42		key Return	57		key Return
			58	+	sleep 700ms
			59	+	# Back on workspace 1: focus Firefox and fullscreen it. Once the page is still,
			60	+	# read it, then hint mode starts and the label of the Wikipedia result
			61	+	# ("Wayland (protocol) - Wikipedia") is typed.
			62	+	key Super+Right
			63	+	sleep 600ms
			64	+	key Super+Shift+f
			65	+	sleep 900ms
			66	+	hint-read
			67	+	sleep 200ms
			68	+	key Super+g
			69	+	sleep 1500ms
			70	+	hint-click wikipedia protocol
			71	+	# Firefox loads the page; the compositor writes no frame while it is still, so
			72	+	# a pointer move marks the end of the hold.
43		sleep 4s	73		sleep 4s
			74	+	move 1500 900

A tools/demo/mousepad.keyfile +6 -0

@@ -0,0 +1,6 @@
	1	+	[org/xfce/mousepad/preferences/view]
	2	+	use-default-monospace-font=false
	3	+	font-name='JetBrainsMono Nerd Font 15'
	4	+
	5	+	[org/xfce/mousepad/preferences/window]
	6	+	menubar-visible=false

@@ -0,0 +1,6 @@
	1	+	[org/xfce/mousepad/preferences/view]
	2	+	use-default-monospace-font=false
	3	+	font-name='JetBrainsMono Nerd Font 15'
	4	+
	5	+	[org/xfce/mousepad/preferences/window]
	6	+	menubar-visible=false

M tools/demo/record.sh +107 -42

@@ -1,5 +1,13 @@
1	1		#!/usr/bin/env bash
2		-	# Requires Go 1.27, rootless Docker, bash and ffmpeg. Run from any directory.
	2	+	# Requires Go 1.27, rootless Docker, bash, ffmpeg, network access and an AMD GPU.
	3	+	# Run from any directory. The image holds the compositor's runtime (tesseract,
	4	+	# AT-SPI, Mesa's radeon Vulkan driver) and the apps; the host builds the binary.
	5	+	# The compositor renders on the host GPU through /dev/dri, and hint scans need
	6	+	# the sync files only a hardware driver exports. The image only carries
	7	+	# vulkan-radeon, so another GPU needs its Mesa driver in tools/demo/Dockerfile
	8	+	# and the vendor check below changed.
	9	+	# A recording only succeeds if the compositor stops on the quit key that ends
	10	+	# tools/demo/input.txt: hitting --timeout first means a truncated video.
3	11		# KEEP_CAPTURE=1 preserves temporary captures on success; failures always clean up.
4	12		set -euo pipefail
5	13		root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)

@@ -8,8 +16,17 @@ for command in docker go ffmpeg; do
8	16		command -v "$command" >/dev/null \|\| { echo "Missing required command: $command" >&2; exit 1; }
9	17		done
10	18		docker info >/dev/null 2>&1 \|\| { echo 'Docker daemon unavailable; start rootless Docker first.' >&2; exit 1; }
	19	+	amd=''
	20	+	for vendor in /sys/class/drm/renderD*/device/vendor; do
	21	+	if [[ -r $vendor && $(<"$vendor") == 0x1002 ]]; then amd=yes; fi
	22	+	done
	23	+	[[ -n $amd ]] \|\| {
	24	+	echo 'No AMD GPU render node (PCI vendor 0x1002) found: the demo image only carries the radeon Vulkan driver.' >&2
	25	+	echo 'See the header of this script to use another GPU.' >&2
	26	+	exit 1
	27	+	}
11	28		work=$(mktemp -d "${TMPDIR:-/tmp}/neferwl-demo.XXXXXX")
12		-	compositor='' launcher='' feeder='' success=''
	29	+	launcher='' feeder='' success=''
13	30		container=neferwl-demo-$$
14	31		cleanup() {
15	32		local status=$?

@@ -19,9 +36,8 @@ cleanup() {
19	36		for pid in "$launcher" "$feeder"; do
20	37		if [[ -n $pid ]]; then kill -- "-$pid" 2>/dev/null \|\| kill "$pid" 2>/dev/null \|\| true; fi
21	38		done
22		-	if [[ -n $compositor ]]; then kill "$compositor" 2>/dev/null \|\| true; fi
23	39		docker rm -f "$container" >/dev/null 2>&1 \|\| true
24		-	for pid in "$launcher" "$feeder" "$compositor"; do
	40	+	for pid in "$launcher" "$feeder"; do
25	41		if [[ -n $pid ]]; then wait "$pid" 2>/dev/null \|\| true; fi
26	42		done
27	43		exec 3>&-

@@ -34,80 +50,129 @@ trap cleanup EXIT
34	50		trap 'exit 130' INT
35	51		trap 'exit 143' TERM
36	52		printf 'Capture directory: %s\n' "$work"
37		-	mkdir -p "$work/shots" docs/media
	53	+	mkdir -p "$work/shots" "$work/state" docs/media
38	54		docker build -t neferwl-demo -f tools/demo/Dockerfile tools/demo
39	55		CGO_ENABLED=0 go build -o "$work/neferwl" ./cmd/neferwl
40		-	state=$work/state
41		-	log=$state/neferwl/runs/headless/latest.log
	56	+	log=$work/state/neferwl/runs/headless/latest.log
42	57		mkfifo "$work/input"
43	58		# Hold both ends open so the FIFO cannot block compositor startup or produce
44		-	# premature EOF. No input is fed until Docker and the first app are ready.
	59	+	# premature EOF. No input is fed until the first app is ready.
45	60		exec 3<>"$work/input"
46		-	env -u NEFERWL_STATE -u NEFERWL_SLOT -u WAYLAND_DISPLAY XDG_STATE_HOME="$state" "$work/neferwl" \
47		-	--backend=headless --size 1920x1080 --config tools/demo/config \
48		-	--no-xwayland --screenshot "$work/shots" --input - --timeout 75s <"$work/input" &
49		-	compositor=$!
50		-	socket=''
51		-	for ((attempt = 0; attempt < 150; attempt++)); do
52		-	kill -0 "$compositor" 2>/dev/null \|\| { echo "Compositor exited before readiness; see $log" >&2; exit 1; }
	61	+	# The compositor is the container's main process, so the container ends with
	62	+	# it. Root in a rootless container is the host user: the bind-mounted work
	63	+	# directory (binary, FIFO, frames, log, hint dumps) stays owned by the host user.
	64	+	# The GPU render node gives the compositor the sync files that hint scans need.
	65	+	docker run -d --name "$container" --hostname neferwl-demo --device /dev/dri \
	66	+	-e XDG_STATE_HOME=/work/state -e NEFERWL_HINT_DEBUG=1 \
	67	+	-v "$work:/work" -v "$work/neferwl:/usr/local/bin/neferwl:ro" neferwl-demo \
	68	+	--backend=headless --size 1920x1080 --config /home/demo/config --no-xwayland \
	69	+	--screenshot /work/shots --input /work/input --timeout 75s >/dev/null
	70	+	display=''
	71	+	deadline=$((SECONDS + 30))
	72	+	while ((SECONDS < deadline)); do
	73	+	[[ $(docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null) == true ]] \|\|
	74	+	{ echo "Compositor exited before readiness; see $log" >&2; docker logs "$container" >&2 \|\| true; exit 1; }
53	75		if [[ -f $log ]]; then
54	76		display=$(sed -n '/"message":"listening"/s/."WAYLAND_DISPLAY":"$[^" ]$".*/\1/p' "$log" \| tail -1)
55		-	if [[ -n $display && -S $XDG_RUNTIME_DIR/${display##*/} ]]; then
56		-	socket=$XDG_RUNTIME_DIR/${display##*/}
57		-	break
58		-	fi
	77	+	[[ -n $display ]] && break
59	78		fi
60	79		sleep 0.1
61	80		done
62		-	[[ -n $socket ]] \|\| { echo "Timed out after 15s waiting for compositor socket; see $log" >&2; exit 1; }
63		-	# Container root maps to the host uid under rootless Docker. Grant demo access
64		-	# to this disposable socket only; Firefox rejects root with a demo-owned HOME.
65		-	docker run -d --name "$container" --hostname neferwl-demo --network none \
66		-	-v "$socket:/run/wl/wayland-0" neferwl-demo
67		-	docker exec "$container" chmod 666 /run/wl/wayland-0
	81	+	[[ -n $display ]] \|\| { echo "Timed out after 30s waiting for the compositor to listen; see $log" >&2; exit 1; }
	82	+	# Mousepad runs as another user (see launch_apps) and needs the socket.
	83	+	docker exec "$container" chmod 666 "/run/wl/${display##*/}"
	84	+	app() { docker exec -d -e "WAYLAND_DISPLAY=${display##*/}" "$container" "$@"; }
68	85		at() { local delay=$(($1 - SECONDS)); if ((delay > 0)); then sleep "$delay"; fi; }
69	86		launch_apps() {
70	87		# This shell timeline shares t=0 with input.txt. Only keys live in input.txt;
71		-	# Docker launches cannot be invoked by compositor input scripts.
	88	+	# app launches cannot be invoked by compositor input scripts.
72	89		at 3
73		-	docker exec -d --user demo "$container" dbus-run-session -- firefox --no-remote \
74		-	--profile /home/demo/firefox file:///home/demo/site/index.html
	90	+	app firefox --no-remote --profile /root/firefox 'https://duckduckgo.com/?q=wayland+compositor'
75	91		# btop's first per-core sample is bogus in this container. Let it collect
76	92		# real deltas on workspace 2; input.txt brings it into view at t=17.
77	93		at 5
78		-	docker exec -d --user demo "$container" foot --app-id demo-monitor \
79		-	--title 'Monitor / btop' /usr/local/bin/btop
	94	+	app foot --app-id demo-monitor --title 'Monitor / btop' /usr/local/bin/btop
80	95		at 9
81		-	docker exec -d --user demo "$container" dbus-run-session -- bash -c \
82		-	'gsettings set org.xfce.mousepad.preferences.view use-default-monospace-font false; gsettings set org.xfce.mousepad.preferences.view font-name "JetBrainsMono Nerd Font 15"; gsettings set org.xfce.mousepad.preferences.window menubar-visible false; mousepad /home/demo/config'
	96	+	# Mousepad runs as demo (it warns about root), with a file-backed
	97	+	# GSettings and no session bus (it needs the root-owned bus otherwise).
	98	+	app setpriv --reuid=1000 --regid=1000 --clear-groups env HOME=/home/demo \
	99	+	GSETTINGS_BACKEND=keyfile DBUS_SESSION_BUS_ADDRESS=disabled: \
	100	+	mousepad --disable-server /home/demo/config
83	101		at 30
84		-	docker exec -d --user demo "$container" foot --title 'Fonts / workspace 2' bash -c \
	102	+	app foot --title 'Fonts / workspace 2' bash -c \
85	103		'eza --tree --level=2 --icons --color=always /usr/share/fonts \| head -30; exec sleep infinity'
86	104		}
87		-	docker exec -d --user demo "$container" foot --title 'System / NeferWL' bash -c \
88		-	'fastfetch --logo small --structure OS:Kernel:WM:Terminal:Memory:Colors; eza -la --no-user --icons --color=always /etc \| head -24; exec sleep infinity'
	105	+	feed() {
	106	+	# Forwards input.txt to the compositor in real time. `sleep` lines are
	107	+	# served here (not forwarded), so host-side steps can sit between two
	108	+	# compositor steps:
	109	+	# hint-read reads the still page with OCR, in the background
	110	+	# hint-click <word>... types the label of the link that shows all the
	111	+	# words, from the dump of the scan the last
	112	+	# `hint` bind started (demo-hint-label reproduces
	113	+	# the labelling of internal/core/hint.go)
	114	+	local line cmd d mark label reader=''
	115	+	while IFS= read -r line \|\| [[ -n $line ]]; do
	116	+	cmd=${line%% *}
	117	+	case $cmd in
	118	+	'' \| '#'*) ;;
	119	+	sleep)
	120	+	d=${line#sleep }
	121	+	if [[ $d == *ms ]]; then d=$(printf '%d.%03d' $((${d%ms} / 1000)) $((${d%ms} % 1000))); fi
	122	+	sleep "$d" ;;
	123	+	hint-read)
	124	+	docker exec "$container" demo-hint-label ocr &
	125	+	reader=$! ;;
	126	+	hint-click)
	127	+	[[ -n $reader ]] && wait "$reader"
	128	+	# shellcheck disable=SC2086
	129	+	label=$(docker exec "$container" demo-hint-label label "$mark" ${line#hint-click }) \|\|
	130	+	{ echo "No hint label for: ${line#hint-click }" >&2; docker kill "$container" >/dev/null; return 1; }
	131	+	echo "Hint label for ${line#hint-click }: $label" >&2
	132	+	printf 'type %s\n' "$label" ;;
	133	+	*)
	134	+	[[ $line == 'key Super+g' ]] && mark=$(date +%s.%N)
	135	+	printf '%s\n' "$line" ;;
	136	+	esac
	137	+	done <tools/demo/input.txt
	138	+	printf 'key Ctrl+Alt+BackSpace\n'
	139	+	}
	140	+	app foot --title 'System / NeferWL' bash -c \
	141	+	'fastfetch --logo small --structure OS:Kernel:WM:Terminal:Memory:Colors; eza -la --no-user --icons --color=always /etc \| head -24; exec sleep infinity'
89	142		# The first app is launched before t=0. FIFO input and shell launches now
90	143		# share this epoch, independent of compositor/image/container startup time.
91	144		SECONDS=0
92	145		set -m
93		-	{ cat tools/demo/input.txt; printf '\nkey Ctrl+Alt+BackSpace\n'; } >&3 &
	146	+	feed >&3 &
94	147		feeder=$!
95	148		launch_apps &
96	149		launcher=$!
97	150		set +m
98		-	# --timeout exits nonzero; accept only its explicit timeout log entry.
99		-	if ! wait "$compositor"; then
100		-	grep -q '"reason":"timeout"' "$log" \|\| exit 1
	151	+	# The compositor ends with the quit key at the end of input.txt. Hitting
	152	+	# --timeout first (a stalled timeline) leaves a truncated recording: that fails.
	153	+	status=$(docker wait "$container")
	154	+	if grep -q '"reason":"timeout"' "$log"; then
	155	+	echo 'The compositor hit --timeout before the quit key: the recording is truncated.' >&2
	156	+	exit 1
101	157		fi
102		-	compositor=
103		-	wait "$feeder"
	158	+	[[ $status == 0 ]] \|\| { echo "Compositor failed (exit $status); see $log" >&2; docker logs "$container" >&2 \|\| true; exit 1; }
	159	+	wait "$feeder" \|\| { echo 'Scripted input failed.' >&2; exit 1; }
104	160		feeder=
105	161		wait "$launcher"
106	162		launcher=
	163	+	# Firefox's links come from the accessibility tree; OCR (tesseract) must still load.
	164	+	if grep -qE 'OCR unavailable\|English OCR data missing' "$log"; then echo 'tesseract is missing in the image.' >&2; exit 1; fi
	165	+	if grep -q 'capture needs sync file export' "$log"; then
	166	+	echo 'Hint scans failed: the compositor is not on a hardware Vulkan driver (radeon); see the header of this script.' >&2
	167	+	exit 1
	168	+	fi
	169	+	grep -qE '"atspi_targets":[1-9][0-9],."message":"hint scan"' "$log" \|\|
	170	+	{ echo 'No hint scan found accessibility targets:' >&2; grep -E '"component":"(hints\|atspi)"' "$log" \| cut -c1-300 >&2; exit 1; }
	171	+	grep '"message":"hint scan"' "$log" \| sed -E 's/."atspi_targets":([0-9]+)."ocr_links":([0-9]+)."targets":([0-9]+)./hint scan: \1 accessibility + \2 OCR links = \3 targets/' \| tail -3
107	172		ffmpeg -hide_banner -loglevel warning -f concat -safe 0 \
108	173		-i "$work/shots/frames.ffconcat" -vf 'fps=30,format=yuv420p' \
109	174		-c:v libx264 -crf 23 -movflags +faststart docs/media/demo.mp4 -y
110		-	ffmpeg -hide_banner -loglevel warning -ss 7 -i docs/media/demo.mp4 \
	175	+	ffmpeg -hide_banner -loglevel warning -ss 8 -i docs/media/demo.mp4 \
111	176		-frames:v 1 -update 1 docs/media/demo.png -y
112	177		success=yes
113	178		if [[ ${KEEP_CAPTURE:-0} == 1 ]]; then printf 'Captures kept: %s\n' "$work"; fi

 #!/usr/bin/env bash
-# Requires Go 1.27, rootless Docker, bash and ffmpeg. Run from any directory.
+# Requires Go 1.27, rootless Docker, bash, ffmpeg, network access and an AMD GPU.
+# Run from any directory. The image holds the compositor's runtime (tesseract,
+# AT-SPI, Mesa's radeon Vulkan driver) and the apps; the host builds the binary.
+# The compositor renders on the host GPU through /dev/dri, and hint scans need
+# the sync files only a hardware driver exports. The image only carries
+# vulkan-radeon, so another GPU needs its Mesa driver in tools/demo/Dockerfile
+# and the vendor check below changed.
+# A recording only succeeds if the compositor stops on the quit key that ends
+# tools/demo/input.txt: hitting --timeout first means a truncated video.
 # KEEP_CAPTURE=1 preserves temporary captures on success; failures always clean up.
 set -euo pipefail
 root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)
     command -v "$command" >/dev/null || { echo "Missing required command: $command" >&2; exit 1; }
 done
 docker info >/dev/null 2>&1 || { echo 'Docker daemon unavailable; start rootless Docker first.' >&2; exit 1; }
+amd=''
+for vendor in /sys/class/drm/renderD*/device/vendor; do
+    if [[ -r $vendor && $(<"$vendor") == 0x1002 ]]; then amd=yes; fi
+done
+[[ -n $amd ]] || {
+    echo 'No AMD GPU render node (PCI vendor 0x1002) found: the demo image only carries the radeon Vulkan driver.' >&2
+    echo 'See the header of this script to use another GPU.' >&2
+    exit 1
+}
 work=$(mktemp -d "${TMPDIR:-/tmp}/neferwl-demo.XXXXXX")
-compositor='' launcher='' feeder='' success=''
+launcher='' feeder='' success=''
 container=neferwl-demo-$$
 cleanup() {
     local status=$?
     for pid in "$launcher" "$feeder"; do
         if [[ -n $pid ]]; then kill -- "-$pid" 2>/dev/null || kill "$pid" 2>/dev/null || true; fi
     done
-    if [[ -n $compositor ]]; then kill "$compositor" 2>/dev/null || true; fi
     docker rm -f "$container" >/dev/null 2>&1 || true
-    for pid in "$launcher" "$feeder" "$compositor"; do
+    for pid in "$launcher" "$feeder"; do
         if [[ -n $pid ]]; then wait "$pid" 2>/dev/null || true; fi
     done
     exec 3>&-
 trap 'exit 130' INT
 trap 'exit 143' TERM
 printf 'Capture directory: %s\n' "$work"
-mkdir -p "$work/shots" docs/media
+mkdir -p "$work/shots" "$work/state" docs/media
 docker build -t neferwl-demo -f tools/demo/Dockerfile tools/demo
 CGO_ENABLED=0 go build -o "$work/neferwl" ./cmd/neferwl
-state=$work/state
+log=$work/state/neferwl/runs/headless/latest.log
-log=$state/neferwl/runs/headless/latest.log
 mkfifo "$work/input"
 # Hold both ends open so the FIFO cannot block compositor startup or produce
-# premature EOF. No input is fed until Docker and the first app are ready.
+# premature EOF. No input is fed until the first app is ready.
 exec 3<>"$work/input"
-env -u NEFERWL_STATE -u NEFERWL_SLOT -u WAYLAND_DISPLAY XDG_STATE_HOME="$state" "$work/neferwl" \
+# The compositor is the container's main process, so the container ends with
-    --backend=headless --size 1920x1080 --config tools/demo/config \
+# it. Root in a rootless container is the host user: the bind-mounted work
-    --no-xwayland --screenshot "$work/shots" --input - --timeout 75s <"$work/input" &
+# directory (binary, FIFO, frames, log, hint dumps) stays owned by the host user.
-compositor=$!
+# The GPU render node gives the compositor the sync files that hint scans need.
-socket=''
+docker run -d --name "$container" --hostname neferwl-demo --device /dev/dri \
-for ((attempt = 0; attempt < 150; attempt++)); do
+    -e XDG_STATE_HOME=/work/state -e NEFERWL_HINT_DEBUG=1 \
-    kill -0 "$compositor" 2>/dev/null || { echo "Compositor exited before readiness; see $log" >&2; exit 1; }
+    -v "$work:/work" -v "$work/neferwl:/usr/local/bin/neferwl:ro" neferwl-demo \
+    --backend=headless --size 1920x1080 --config /home/demo/config --no-xwayland \
+    --screenshot /work/shots --input /work/input --timeout 75s >/dev/null
+display=''
+deadline=$((SECONDS + 30))
+while ((SECONDS < deadline)); do
+    [[ $(docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null) == true ]] ||
+        { echo "Compositor exited before readiness; see $log" >&2; docker logs "$container" >&2 || true; exit 1; }
     if [[ -f $log ]]; then
         display=$(sed -n '/"message":"listening"/s/.*"WAYLAND_DISPLAY":"\([^" ]*\)".*/\1/p' "$log" | tail -1)
-        if [[ -n $display && -S $XDG_RUNTIME_DIR/${display##*/} ]]; then
+        [[ -n $display ]] && break
-            socket=$XDG_RUNTIME_DIR/${display##*/}
-            break
-        fi
     fi
     sleep 0.1
 done
-[[ -n $socket ]] || { echo "Timed out after 15s waiting for compositor socket; see $log" >&2; exit 1; }
+[[ -n $display ]] || { echo "Timed out after 30s waiting for the compositor to listen; see $log" >&2; exit 1; }
-# Container root maps to the host uid under rootless Docker. Grant demo access
+# Mousepad runs as another user (see launch_apps) and needs the socket.
-# to this disposable socket only; Firefox rejects root with a demo-owned HOME.
+docker exec "$container" chmod 666 "/run/wl/${display##*/}"
-docker run -d --name "$container" --hostname neferwl-demo --network none \
+app() { docker exec -d -e "WAYLAND_DISPLAY=${display##*/}" "$container" "$@"; }
-    -v "$socket:/run/wl/wayland-0" neferwl-demo
-docker exec "$container" chmod 666 /run/wl/wayland-0
 at() { local delay=$(($1 - SECONDS)); if ((delay > 0)); then sleep "$delay"; fi; }
 launch_apps() {
     # This shell timeline shares t=0 with input.txt. Only keys live in input.txt;
-    # Docker launches cannot be invoked by compositor input scripts.
+    # app launches cannot be invoked by compositor input scripts.
     at 3
-    docker exec -d --user demo "$container" dbus-run-session -- firefox --no-remote \
+    app firefox --no-remote --profile /root/firefox 'https://duckduckgo.com/?q=wayland+compositor'
-        --profile /home/demo/firefox file:///home/demo/site/index.html
     # btop's first per-core sample is bogus in this container. Let it collect
     # real deltas on workspace 2; input.txt brings it into view at t=17.
     at 5
-    docker exec -d --user demo "$container" foot --app-id demo-monitor \
+    app foot --app-id demo-monitor --title 'Monitor / btop' /usr/local/bin/btop
-        --title 'Monitor / btop' /usr/local/bin/btop
     at 9
-    docker exec -d --user demo "$container" dbus-run-session -- bash -c \
+    # Mousepad runs as demo (it warns about root), with a file-backed
-        'gsettings set org.xfce.mousepad.preferences.view use-default-monospace-font false; gsettings set org.xfce.mousepad.preferences.view font-name "JetBrainsMono Nerd Font 15"; gsettings set org.xfce.mousepad.preferences.window menubar-visible false; mousepad /home/demo/config'
+    # GSettings and no session bus (it needs the root-owned bus otherwise).
+    app setpriv --reuid=1000 --regid=1000 --clear-groups env HOME=/home/demo \
+        GSETTINGS_BACKEND=keyfile DBUS_SESSION_BUS_ADDRESS=disabled: \
+        mousepad --disable-server /home/demo/config
     at 30
-    docker exec -d --user demo "$container" foot --title 'Fonts / workspace 2' bash -c \
+    app foot --title 'Fonts / workspace 2' bash -c \
         'eza --tree --level=2 --icons --color=always /usr/share/fonts | head -30; exec sleep infinity'
 }
-docker exec -d --user demo "$container" foot --title 'System / NeferWL' bash -c \
+feed() {
-        'fastfetch --logo small --structure OS:Kernel:WM:Terminal:Memory:Colors; eza -la --no-user --icons --color=always /etc | head -24; exec sleep infinity'
+    # Forwards input.txt to the compositor in real time. `sleep` lines are
+    # served here (not forwarded), so host-side steps can sit between two
+    # compositor steps:
+    #   hint-read              reads the still page with OCR, in the background
+    #   hint-click <word>...   types the label of the link that shows all the
+    #                          words, from the dump of the scan the last
+    #                          `hint` bind started (demo-hint-label reproduces
+    #                          the labelling of internal/core/hint.go)
+    local line cmd d mark label reader=''
+    while IFS= read -r line || [[ -n $line ]]; do
+        cmd=${line%% *}
+        case $cmd in
+        '' | '#'*) ;;
+        sleep)
+            d=${line#sleep }
+            if [[ $d == *ms ]]; then d=$(printf '%d.%03d' $((${d%ms} / 1000)) $((${d%ms} % 1000))); fi
+            sleep "$d" ;;
+        hint-read)
+            docker exec "$container" demo-hint-label ocr &
+            reader=$! ;;
+        hint-click)
+            [[ -n $reader ]] && wait "$reader"
+            # shellcheck disable=SC2086
+            label=$(docker exec "$container" demo-hint-label label "$mark" ${line#hint-click }) ||
+                { echo "No hint label for: ${line#hint-click }" >&2; docker kill "$container" >/dev/null; return 1; }
+            echo "Hint label for ${line#hint-click }: $label" >&2
+            printf 'type %s\n' "$label" ;;
+        *)
+            [[ $line == 'key Super+g' ]] && mark=$(date +%s.%N)
+            printf '%s\n' "$line" ;;
+        esac
+    done <tools/demo/input.txt
+    printf 'key Ctrl+Alt+BackSpace\n'
+}
+app foot --title 'System / NeferWL' bash -c \
+    'fastfetch --logo small --structure OS:Kernel:WM:Terminal:Memory:Colors; eza -la --no-user --icons --color=always /etc | head -24; exec sleep infinity'
 # The first app is launched before t=0. FIFO input and shell launches now
 # share this epoch, independent of compositor/image/container startup time.
 SECONDS=0
 set -m
-{ cat tools/demo/input.txt; printf '\nkey Ctrl+Alt+BackSpace\n'; } >&3 &
+feed >&3 &
 feeder=$!
 launch_apps &
 launcher=$!
 set +m
-# --timeout exits nonzero; accept only its explicit timeout log entry.
+# The compositor ends with the quit key at the end of input.txt. Hitting
-if ! wait "$compositor"; then
+# --timeout first (a stalled timeline) leaves a truncated recording: that fails.
-    grep -q '"reason":"timeout"' "$log" || exit 1
+status=$(docker wait "$container")
+if grep -q '"reason":"timeout"' "$log"; then
+    echo 'The compositor hit --timeout before the quit key: the recording is truncated.' >&2
+    exit 1
 fi
-compositor=
+[[ $status == 0 ]] || { echo "Compositor failed (exit $status); see $log" >&2; docker logs "$container" >&2 || true; exit 1; }
-wait "$feeder"
+wait "$feeder" || { echo 'Scripted input failed.' >&2; exit 1; }
 feeder=
 wait "$launcher"
 launcher=
+# Firefox's links come from the accessibility tree; OCR (tesseract) must still load.
+if grep -qE 'OCR unavailable|English OCR data missing' "$log"; then echo 'tesseract is missing in the image.' >&2; exit 1; fi
+if grep -q 'capture needs sync file export' "$log"; then
+    echo 'Hint scans failed: the compositor is not on a hardware Vulkan driver (radeon); see the header of this script.' >&2
+    exit 1
+fi
+grep -qE '"atspi_targets":[1-9][0-9]*,.*"message":"hint scan"' "$log" ||
+    { echo 'No hint scan found accessibility targets:' >&2; grep -E '"component":"(hints|atspi)"' "$log" | cut -c1-300 >&2; exit 1; }
+grep '"message":"hint scan"' "$log" | sed -E 's/.*"atspi_targets":([0-9]+).*"ocr_links":([0-9]+).*"targets":([0-9]+).*/hint scan: \1 accessibility + \2 OCR links = \3 targets/' | tail -3
 ffmpeg -hide_banner -loglevel warning -f concat -safe 0 \
     -i "$work/shots/frames.ffconcat" -vf 'fps=30,format=yuv420p' \
     -c:v libx264 -crf 23 -movflags +faststart docs/media/demo.mp4 -y
-ffmpeg -hide_banner -loglevel warning -ss 7 -i docs/media/demo.mp4 \
+ffmpeg -hide_banner -loglevel warning -ss 8 -i docs/media/demo.mp4 \
     -frames:v 1 -update 1 docs/media/demo.png -y
 success=yes
 if [[ ${KEEP_CAPTURE:-0} == 1 ]]; then printf 'Captures kept: %s\n' "$work"; fi

D tools/demo/site/index.html +0 -13

@@ -1,13 +0,0 @@
1	-	<!doctype html>
2	-	<html lang="en">
3	-	<meta charset="utf-8">
4	-	<meta name="color-scheme" content="dark">
5	-	<title>NeferWL — Space to think</title>
6	-	<style>
7	-	*{box-sizing:border-box}body{margin:0;background:#101010;color:#ffffff;font:18px/1.6 sans-serif}main{max-width:960px;margin:auto;padding:60px 38px}header{color:#ffc799;letter-spacing:.2em;font:700 18px monospace}small{color:#ffc799}h1{font-size:54px;line-height:1.12;letter-spacing:-2px;margin:40px 0 24px}p{color:#a0a0a0}.cards{display:grid;grid-template-columns:repeat(3,1fr);gap:14px;margin:36px 0}.card{padding:18px;background:#161616;border:1px solid #282828;border-radius:12px}.card b{color:#ffc799}.card p{font-size:15px;margin-bottom:0}pre{background:#161616;border-left:3px solid #ffc799;border-radius:6px;padding:24px;font:16px/1.8 monospace;color:#ffc799}footer{color:#a0a0a0;font:14px monospace}
8	-	</style>
9	-	<main><header>◈ NEFERWL</header><h1>Your desktop.<br>A little more focused.</h1><p>A Wayland compositor with room for your work.<br>Tile naturally, set things aside, see the whole picture.</p><div class="cards"><section class="card"><b>01 / Performance</b><p>Small, responsive.<br>Built in Go. Made to stay out of your way.</p></section><section class="card"><b>02 / Stash</b><p>Keep tools close.<br>Set aside a window. Bring it back in a keystroke.</p></section><section class="card"><b>03 / Overview</b><p>Find your focus.<br>Every workspace and stashed window, at a glance.</p></section></div><pre># A desktop that flows with you
10	-	layout.max-columns = 2
11	-	layout.overflow = cascade
12	-	animations = on</pre><footer>SUPER + ↑ / ↓   bands   ·   SUPER + S   stash   ·   SUPER + O   overview</footer></main>
13	-	</html>

@@ -1,13 +0,0 @@
1	-	<!doctype html>
2	-	<html lang="en">
3	-	<meta charset="utf-8">
4	-	<meta name="color-scheme" content="dark">
5	-	<title>NeferWL — Space to think</title>
6	-	<style>
7	-	*{box-sizing:border-box}body{margin:0;background:#101010;color:#ffffff;font:18px/1.6 sans-serif}main{max-width:960px;margin:auto;padding:60px 38px}header{color:#ffc799;letter-spacing:.2em;font:700 18px monospace}small{color:#ffc799}h1{font-size:54px;line-height:1.12;letter-spacing:-2px;margin:40px 0 24px}p{color:#a0a0a0}.cards{display:grid;grid-template-columns:repeat(3,1fr);gap:14px;margin:36px 0}.card{padding:18px;background:#161616;border:1px solid #282828;border-radius:12px}.card b{color:#ffc799}.card p{font-size:15px;margin-bottom:0}pre{background:#161616;border-left:3px solid #ffc799;border-radius:6px;padding:24px;font:16px/1.8 monospace;color:#ffc799}footer{color:#a0a0a0;font:14px monospace}
8	-	</style>
9	-	<main><header>◈ NEFERWL</header><h1>Your desktop.<br>A little more focused.</h1><p>A Wayland compositor with room for your work.<br>Tile naturally, set things aside, see the whole picture.</p><div class="cards"><section class="card"><b>01 / Performance</b><p>Small, responsive.<br>Built in Go. Made to stay out of your way.</p></section><section class="card"><b>02 / Stash</b><p>Keep tools close.<br>Set aside a window. Bring it back in a keystroke.</p></section><section class="card"><b>03 / Overview</b><p>Find your focus.<br>Every workspace and stashed window, at a glance.</p></section></div><pre># A desktop that flows with you
10	-	layout.max-columns = 2
11	-	layout.overflow = cascade
12	-	animations = on</pre><footer>SUPER + ↑ / ↓   bands   ·   SUPER + S   stash   ·   SUPER + O   overview</footer></main>
13	-	</html>

Comments0 open / 0

no comments yet.

Merge settings

Comments0 open / 0