feat/hints a471fa4

#5 WIP: feat(hints): add experimental keyboard hint mode

state
closed
branch
feat/hints → main
opened
bnema ·
updated

Experimental keyboard hint mode: press the hint bind, every link and button on the focused output gets a short label, type the label to click it.

Off by default. hint.enable = on turns it on (restart required). When it is off, nothing starts: no goroutine, no accessibility bus connection, no capture routing, no buffers.

Where targets come from

  • Accessibility tree (AT-SPI) for windows that expose it (Firefox, Chromium with --force-renderer-accessibility, GTK, Qt, Electron): exact links and buttons, a few milliseconds. New adapter atspi over zerobus, one connection owned by the hints worker, connected on the first scan with bounded attempts and reconnect backoff.
  • OCR (tesseract CLI, no cgo) only on the visible windows the accessibility tree does not cover: visible URLs, domains, email addresses and underlined words. Grayscale bands fed as PGM, 4 processes max per scan, per-band word cache. Without tesseract, OCR is off and only accessibility targets are labelled.

Click

  • Core keeps each target on the window it was scanned on, and cancels hints when the window, workspace, output geometry, scale or transform changes.
  • Every hint click sends an explicit modifier state to the client (new SetModifiers client command), then restores the real state. Links read by OCR in text use hint.text-click (default ctrl+shift) so terminals open them; underlined words and accessibility targets get a plain click.

Labels

  • Drawn by the Vulkan renderer, anchored outside the top-left corner of their target, kept out of captured frames.
  • Size follows the output pixel density (EDID millimetres), colours derive from border.active with WCAG 4.5:1 contrast. Overrides: hint.size, hint.background, hint.foreground.

Memory

  • Captures go to anonymous mmap released right after cropping; no capture when every window is covered by the accessibility tree. About 92 KB of Go heap per scan at 5120x2160.

Debugging

  • NEFERWL_HINT_DEBUG=1 writes each scanned frame, an annotated frame and a JSON of targets to $XDG_STATE_HOME/neferwl/hints/ (0700 / 0600, 50 most recent kept).

Checks

  • make check, make race, make mocks-check, staticcheck ./... pass; every commit builds.
  • Headless end to end at 1920x1080 and 5120x2160: Chromium (11 accessibility targets in ~2 ms), kitty (URL and domain links by OCR, none on filenames, paths, prompt or cursor), side by side (OCR only on the kitty rect), dark page without accessibility (underlined links only), ctrl+shift click opens a kitty link and a plain click does not, accessibility click reaches Chromium, hint.enable = off starts nothing.

Known limits

  • OCR is best effort: it misses links whose address is not written on screen and not underlined, and reads two full-height windows in about 1 s at 5120x2160.
  • Browsers drawing web content off screen (CEF OSR) expose no accessibility tree; they fall back to OCR.
  • zerobus has no dial deadline: a stalled bus is bounded by one connection attempt in flight, not by a timeout.
  • Modifier masks use the standard xkb indices.
CommitSubjectAuthorDateSignatureStatus
a471fa4docs(config): document the experimental hint modebriceGPG Verified
d7c4662feat(app): wire hint mode when hint.enable is onbriceGPG Verified
463ec0afeat(hints): find hint targets with the accessibility tree and OCRbriceGPG Verified
d9f6a44feat(atspi): find hint targets in the accessibility treebriceGPG Verified
e368cc0feat(config): add hint mode keysbriceGPG Verified
2082615feat(wayland): apply temporary modifier overrides for hint clicksbriceGPG Verified
6319bfafeat(vulkan): draw hint labels and keep them out of capturesbriceGPG Verified
4b849d0feat(hintlabel): rasterize hint labels sized and coloured for the outputbriceGPG Verified
40c7803feat(core): add keyboard hint modebriceGPG Verified
689e994feat(ports): add hint scan, modifier override and hint config messagesbriceGPG Verified

Checks 1

75 files · +7747 -37
1/1
M .hexcheck.yaml +1 -1
@@ -25,7 +25,7 @@ allow:
2525 ruleSettings:
2626 missing-generated-mock-for-port:
2727 # Sealed message unions are channel data, not ports; tested by sending messages (ADR 005).
28- excludePaths: [internal/ports/client_events.go, internal/ports/client_commands.go, internal/ports/input_events.go, internal/ports/output_events.go, internal/ports/lease.go, internal/ports/session_lock_messages.go, internal/ports/screencast_messages.go, internal/ports/session_backend.go]
28+ excludePaths: [internal/ports/client_events.go, internal/ports/client_commands.go, internal/ports/input_events.go, internal/ports/output_events.go, internal/ports/lease.go, internal/ports/session_lock_messages.go, internal/ports/screencast_messages.go, internal/ports/session_backend.go, internal/ports/hint_messages.go]
2929 suspicious-business-logic-in-adapter:
3030 # app is the composition root; starting core and wiring channels is its job.
3131 excludePaths: [internal/app/app.go, internal/app/session.go]
@@ -25,7 +25,7 @@ allow:
25 ruleSettings:25 ruleSettings:
26 missing-generated-mock-for-port:26 missing-generated-mock-for-port:
27 # Sealed message unions are channel data, not ports; tested by sending messages (ADR 005).27 # Sealed message unions are channel data, not ports; tested by sending messages (ADR 005).
28- excludePaths: [internal/ports/client_events.go, internal/ports/client_commands.go, internal/ports/input_events.go, internal/ports/output_events.go, internal/ports/lease.go, internal/ports/session_lock_messages.go, internal/ports/screencast_messages.go, internal/ports/session_backend.go]28+ excludePaths: [internal/ports/client_events.go, internal/ports/client_commands.go, internal/ports/input_events.go, internal/ports/output_events.go, internal/ports/lease.go, internal/ports/session_lock_messages.go, internal/ports/screencast_messages.go, internal/ports/session_backend.go, internal/ports/hint_messages.go]
29 suspicious-business-logic-in-adapter:29 suspicious-business-logic-in-adapter:
30 # app is the composition root; starting core and wiring channels is its job.30 # app is the composition root; starting core and wiring channels is its job.
31 excludePaths: [internal/app/app.go, internal/app/session.go]31 excludePaths: [internal/app/app.go, internal/app/session.go]
M .mockery.yml +19 -0
@@ -111,6 +111,25 @@ packages:
111111 dir: internal/adapters/statefile
112112 pkgname: statefile
113113 filename: proc_tree_mock_test.go
114+ github.com/bnema/neferwl/internal/adapters/hints:
115+ interfaces:
116+ targetSource:
117+ config:
118+ dir: internal/adapters/hints
119+ pkgname: hints
120+ filename: target_source_mock_test.go
121+ github.com/bnema/neferwl/internal/adapters/atspi:
122+ interfaces:
123+ transport:
124+ config:
125+ dir: internal/adapters/atspi
126+ pkgname: atspi
127+ filename: transport_mock_test.go
128+ connector:
129+ config:
130+ dir: internal/adapters/atspi
131+ pkgname: atspi
132+ filename: connector_mock_test.go
114133 github.com/bnema/neferwl/internal/adapters/screencast:
115134 interfaces:
116135 bus:
@@ -111,6 +111,25 @@ packages:
111 dir: internal/adapters/statefile111 dir: internal/adapters/statefile
112 pkgname: statefile112 pkgname: statefile
113 filename: proc_tree_mock_test.go113 filename: proc_tree_mock_test.go
114+ github.com/bnema/neferwl/internal/adapters/hints:
115+ interfaces:
116+ targetSource:
117+ config:
118+ dir: internal/adapters/hints
119+ pkgname: hints
120+ filename: target_source_mock_test.go
121+ github.com/bnema/neferwl/internal/adapters/atspi:
122+ interfaces:
123+ transport:
124+ config:
125+ dir: internal/adapters/atspi
126+ pkgname: atspi
127+ filename: transport_mock_test.go
128+ connector:
129+ config:
130+ dir: internal/adapters/atspi
131+ pkgname: atspi
132+ filename: connector_mock_test.go
114 github.com/bnema/neferwl/internal/adapters/screencast:133 github.com/bnema/neferwl/internal/adapters/screencast:
115 interfaces:134 interfaces:
116 bus:135 bus:
M Makefile +1 -1
@@ -61,7 +61,7 @@ arch: adapter-imports
6161 # Adapters stay independent: production code may import only the shared
6262 # adapter libraries below. outputkit holds what both output backends (drm,
6363 # headless) run; the others are leaf helpers.
64-ADAPTER_LIBS := fdwait|busretry|outputkit/capture|outputkit/corewake|outputkit/presented|outputkit/surfaces|captureallow|clock|logging|peerexe|sessionsecurity|syncfile|workspaceid|xkb
64+ADAPTER_LIBS := fdwait|busretry|hintlabel|outputkit/capture|outputkit/corewake|outputkit/presented|outputkit/surfaces|captureallow|clock|logging|peerexe|sessionsecurity|syncfile|workspaceid|xkb
6565 adapter-imports:
6666 @edges=$$(CGO_ENABLED=0 go list -f '{{$$p := .ImportPath}}{{range .Imports}}{{$$p}} {{.}}{{"\n"}}{{end}}' ./internal/adapters/...) || exit 1; \
6767 bad=$$(echo "$$edges" | grep -E ' \S+/internal/adapters/' | grep -vE '/internal/adapters/($(ADAPTER_LIBS))$$' \
@@ -61,7 +61,7 @@ arch: adapter-imports
61 # Adapters stay independent: production code may import only the shared61 # Adapters stay independent: production code may import only the shared
62 # adapter libraries below. outputkit holds what both output backends (drm,62 # adapter libraries below. outputkit holds what both output backends (drm,
63 # headless) run; the others are leaf helpers.63 # headless) run; the others are leaf helpers.
64-ADAPTER_LIBS := fdwait|busretry|outputkit/capture|outputkit/corewake|outputkit/presented|outputkit/surfaces|captureallow|clock|logging|peerexe|sessionsecurity|syncfile|workspaceid|xkb64+ADAPTER_LIBS := fdwait|busretry|hintlabel|outputkit/capture|outputkit/corewake|outputkit/presented|outputkit/surfaces|captureallow|clock|logging|peerexe|sessionsecurity|syncfile|workspaceid|xkb
65 adapter-imports:65 adapter-imports:
66 @edges=$$(CGO_ENABLED=0 go list -f '{{$$p := .ImportPath}}{{range .Imports}}{{$$p}} {{.}}{{"\n"}}{{end}}' ./internal/adapters/...) || exit 1; \66 @edges=$$(CGO_ENABLED=0 go list -f '{{$$p := .ImportPath}}{{range .Imports}}{{$$p}} {{.}}{{"\n"}}{{end}}' ./internal/adapters/...) || exit 1; \
67 bad=$$(echo "$$edges" | grep -E ' \S+/internal/adapters/' | grep -vE '/internal/adapters/($(ADAPTER_LIBS))$$' \67 bad=$$(echo "$$edges" | grep -E ' \S+/internal/adapters/' | grep -vE '/internal/adapters/($(ADAPTER_LIBS))$$' \
M README.md +1 -0
@@ -281,6 +281,7 @@ bind.cmd+shift+f = toggle-fullscreen
281281 bind.cmd+s = toggle-stash-visible
282282 bind.cmd+shift+s = toggle-window-stash
283283 bind.cmd+o = toggle-overview
284+# bind.cmd+g = hint
284285 bind.cmd+tab = switch-column-next
285286 bind.cmd+shift+tab = switch-column-prev
286287
@@ -281,6 +281,7 @@ bind.cmd+shift+f = toggle-fullscreen
281 bind.cmd+s = toggle-stash-visible281 bind.cmd+s = toggle-stash-visible
282 bind.cmd+shift+s = toggle-window-stash282 bind.cmd+shift+s = toggle-window-stash
283 bind.cmd+o = toggle-overview283 bind.cmd+o = toggle-overview
284+# bind.cmd+g = hint
284 bind.cmd+tab = switch-column-next285 bind.cmd+tab = switch-column-next
285 bind.cmd+shift+tab = switch-column-prev286 bind.cmd+shift+tab = switch-column-prev
286 287
M docs/config.md +50 -0
@@ -55,6 +55,12 @@ See [examples/config](../examples/config) for every key with its default.
5555 | `focus.flash-opacity` | `0.6` | Starting opacity of the flash, `0.1` to `1` |
5656 | `focus.flash-duration` | `200ms` | Fade duration, `50ms` to `1s`; not scaled by `animations.speed` |
5757 | `focus.flash-color` | `border.active` | Flash color `#rrggbb`; empty uses `border.active` |
58+| `hint.enable` | `off` | Turn on the experimental [hint mode](#hint-mode) (restart required) |
59+| `hint.targets` | `clickable` | What [hint mode](#hint-mode) labels: `clickable` (links, URLs and buttons) or `links` (links and URLs only) |
60+| `hint.text-click` | `ctrl+shift` | Modifiers held for clicks on links found by OCR (terminals open links on a modified click); `none` sends a plain click |
61+| `hint.size` | `auto` | Label text size in logical pixels (6 to 72), or `auto` for screen pixel density |
62+| `hint.background` | empty | Label background (`#rrggbb`); empty uses `border.active` |
63+| `hint.foreground` | empty | Label text (`#rrggbb`); empty chooses a contrasting colour |
5864 | `workspace.<name>.monitor` | focused output | Home monitor of a named workspace |
5965 | `workspace.<name>.size` | `inherit` | [Smaller viewport](#workspace-size), e.g. `1920x1080` |
6066 | `workspace.<name>.max-columns`, `.overflow` | output values | Per-workspace layout |
@@ -191,6 +197,7 @@ bind.XF86MonBrightnessDown = spawn brightnessctl set 5%-
191197 | `toggle-window-stash` | Move the focused window to the [stash](#stash), or back to its column |
192198 | `toggle-stash-visible` | Show or hide the stash |
193199 | `toggle-overview` | Open or close the [overview](#overview) |
200+| `hint` | Label the links and buttons on the focused screen, then click the one you type ([hint mode](#hint-mode)) |
194201 | `switch-column-next` / `-prev` | Switch to the previously used column. Hold `cmd` and press again to go further back |
195202 | `maximize-column` | Toggle full width for the focused column |
196203 | `cycle-column-width` | Grow the focused column by one [cell](#column-widths), then back to equal widths |
@@ -295,3 +302,46 @@ A workspace's stash shows as a pile of cards. Selecting the pile unfolds it into
295302 With `layout.overflow = cascade`, workspaces sit side by side, each band of columns is a lane, and the stash sits under the lanes. Down (or fingers down on the touchpad, with natural scroll off) moves to the lane below; past the last lane it unfolds the stash. Left/right walk the cards; where the carousel does not wrap round, they go to the neighbor workspace past either end. Up leaves the stash for the last lane, at the column you came from.
296303
297304 Otherwise, left from the first column unfolds the stash. Up/down walk the cards, the focus-workspace binds change workspace, and right goes back to the columns.
305+
306+## Hint mode
307+
308+Hint mode is experimental and off by default: set `hint.enable = on` and restart NeferWL.
309+While it is off, nothing of hint mode runs or uses memory, and the `hint` bind
310+logs a notice once and does nothing. A reload does not change it.
311+
312+`hint` labels the links and buttons on the focused output. Type a label to
313+click its target; BackSpace erases a letter, Escape cancels. No bind is set
314+by default; for example `bind.cmd+g = hint`.
315+
316+Labels use the `border.active` colour with a contrasting text colour, and
317+their size follows the screen's pixel density. Each label sits just outside
318+the top-left corner of its target. Links found by OCR (terminal text) are
319+clicked with the `hint.text-click` modifiers held, which is how terminals open
320+a link; accessibility targets get a plain click.
321+
322+### Where targets come from
323+
324+| Source | Apps | Accuracy | Speed |
325+|---|---|---|---|
326+| Accessibility tree (AT-SPI) | Firefox, Chromium, GTK, Qt, Electron | Exact: the app reports its own links and buttons | A few milliseconds |
327+| Text recognition (OCR) | Everything else: terminals, apps without accessibility | Best effort: URLs, domains and underlined text written on screen | 150 ms or more, more on large screens |
328+
329+NeferWL asks the accessibility tree first. Windows it does not cover are read
330+with OCR, which finds visible URLs, domains, email addresses and underlined
331+text. Underlined words get a plain click, and the words of one continuous
332+underline form one target; URLs keep the `hint.text-click` modifiers. OCR
333+cannot see a link whose address is hidden behind plain text, and it can misread
334+small or low-contrast text.
335+
336+### Requirements
337+
338+- Accessibility: NeferWL turns it on at startup. Some apps need a flag:
339+ - Chromium: `--force-renderer-accessibility`
340+ - Electron: `ACCESSIBILITY_ENABLED=1`
341+ - Qt: `QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1`
342+- OCR: `tesseract` and `tesseract-data-eng`. Without tesseract, OCR is off
343+ and hint mode labels only apps that expose the accessibility tree.
344+
345+### Debugging hint mode
346+
347+With `hint.enable = on` and `NEFERWL_HINT_DEBUG=1` set when NeferWL starts (for example `NEFERWL_HINT_DEBUG=1 make tty`, or the variable in the launch environment), every successful scan writes three files to `$XDG_STATE_HOME/neferwl/hints/` (`~/.local/state/neferwl/hints/` when unset): `<time>-<output>.png`, the frame as scanned; `<time>-<output>.annotated.png`, the same with each target outlined (OCR links green, accessibility targets magenta, OCR window crops grey); and `<time>-<output>.json`, the targets (with their `source`, `atspi` or `ocr`) and timings. NeferWL keeps the 50 most recent scans and logs a warning at startup. The files contain the screen in clear: delete them after use.
@@ -55,6 +55,12 @@ See [examples/config](../examples/config) for every key with its default.
55 | `focus.flash-opacity` | `0.6` | Starting opacity of the flash, `0.1` to `1` |55 | `focus.flash-opacity` | `0.6` | Starting opacity of the flash, `0.1` to `1` |
56 | `focus.flash-duration` | `200ms` | Fade duration, `50ms` to `1s`; not scaled by `animations.speed` |56 | `focus.flash-duration` | `200ms` | Fade duration, `50ms` to `1s`; not scaled by `animations.speed` |
57 | `focus.flash-color` | `border.active` | Flash color `#rrggbb`; empty uses `border.active` |57 | `focus.flash-color` | `border.active` | Flash color `#rrggbb`; empty uses `border.active` |
58+| `hint.enable` | `off` | Turn on the experimental [hint mode](#hint-mode) (restart required) |
59+| `hint.targets` | `clickable` | What [hint mode](#hint-mode) labels: `clickable` (links, URLs and buttons) or `links` (links and URLs only) |
60+| `hint.text-click` | `ctrl+shift` | Modifiers held for clicks on links found by OCR (terminals open links on a modified click); `none` sends a plain click |
61+| `hint.size` | `auto` | Label text size in logical pixels (6 to 72), or `auto` for screen pixel density |
62+| `hint.background` | empty | Label background (`#rrggbb`); empty uses `border.active` |
63+| `hint.foreground` | empty | Label text (`#rrggbb`); empty chooses a contrasting colour |
58 | `workspace.<name>.monitor` | focused output | Home monitor of a named workspace |64 | `workspace.<name>.monitor` | focused output | Home monitor of a named workspace |
59 | `workspace.<name>.size` | `inherit` | [Smaller viewport](#workspace-size), e.g. `1920x1080` |65 | `workspace.<name>.size` | `inherit` | [Smaller viewport](#workspace-size), e.g. `1920x1080` |
60 | `workspace.<name>.max-columns`, `.overflow` | output values | Per-workspace layout |66 | `workspace.<name>.max-columns`, `.overflow` | output values | Per-workspace layout |
@@ -191,6 +197,7 @@ bind.XF86MonBrightnessDown = spawn brightnessctl set 5%-
191 | `toggle-window-stash` | Move the focused window to the [stash](#stash), or back to its column |197 | `toggle-window-stash` | Move the focused window to the [stash](#stash), or back to its column |
192 | `toggle-stash-visible` | Show or hide the stash |198 | `toggle-stash-visible` | Show or hide the stash |
193 | `toggle-overview` | Open or close the [overview](#overview) |199 | `toggle-overview` | Open or close the [overview](#overview) |
200+| `hint` | Label the links and buttons on the focused screen, then click the one you type ([hint mode](#hint-mode)) |
194 | `switch-column-next` / `-prev` | Switch to the previously used column. Hold `cmd` and press again to go further back |201 | `switch-column-next` / `-prev` | Switch to the previously used column. Hold `cmd` and press again to go further back |
195 | `maximize-column` | Toggle full width for the focused column |202 | `maximize-column` | Toggle full width for the focused column |
196 | `cycle-column-width` | Grow the focused column by one [cell](#column-widths), then back to equal widths |203 | `cycle-column-width` | Grow the focused column by one [cell](#column-widths), then back to equal widths |
@@ -295,3 +302,46 @@ A workspace's stash shows as a pile of cards. Selecting the pile unfolds it into
295 With `layout.overflow = cascade`, workspaces sit side by side, each band of columns is a lane, and the stash sits under the lanes. Down (or fingers down on the touchpad, with natural scroll off) moves to the lane below; past the last lane it unfolds the stash. Left/right walk the cards; where the carousel does not wrap round, they go to the neighbor workspace past either end. Up leaves the stash for the last lane, at the column you came from.302 With `layout.overflow = cascade`, workspaces sit side by side, each band of columns is a lane, and the stash sits under the lanes. Down (or fingers down on the touchpad, with natural scroll off) moves to the lane below; past the last lane it unfolds the stash. Left/right walk the cards; where the carousel does not wrap round, they go to the neighbor workspace past either end. Up leaves the stash for the last lane, at the column you came from.
296 303
297 Otherwise, left from the first column unfolds the stash. Up/down walk the cards, the focus-workspace binds change workspace, and right goes back to the columns.304 Otherwise, left from the first column unfolds the stash. Up/down walk the cards, the focus-workspace binds change workspace, and right goes back to the columns.
305+
306+## Hint mode
307+
308+Hint mode is experimental and off by default: set `hint.enable = on` and restart NeferWL.
309+While it is off, nothing of hint mode runs or uses memory, and the `hint` bind
310+logs a notice once and does nothing. A reload does not change it.
311+
312+`hint` labels the links and buttons on the focused output. Type a label to
313+click its target; BackSpace erases a letter, Escape cancels. No bind is set
314+by default; for example `bind.cmd+g = hint`.
315+
316+Labels use the `border.active` colour with a contrasting text colour, and
317+their size follows the screen's pixel density. Each label sits just outside
318+the top-left corner of its target. Links found by OCR (terminal text) are
319+clicked with the `hint.text-click` modifiers held, which is how terminals open
320+a link; accessibility targets get a plain click.
321+
322+### Where targets come from
323+
324+| Source | Apps | Accuracy | Speed |
325+|---|---|---|---|
326+| Accessibility tree (AT-SPI) | Firefox, Chromium, GTK, Qt, Electron | Exact: the app reports its own links and buttons | A few milliseconds |
327+| Text recognition (OCR) | Everything else: terminals, apps without accessibility | Best effort: URLs, domains and underlined text written on screen | 150 ms or more, more on large screens |
328+
329+NeferWL asks the accessibility tree first. Windows it does not cover are read
330+with OCR, which finds visible URLs, domains, email addresses and underlined
331+text. Underlined words get a plain click, and the words of one continuous
332+underline form one target; URLs keep the `hint.text-click` modifiers. OCR
333+cannot see a link whose address is hidden behind plain text, and it can misread
334+small or low-contrast text.
335+
336+### Requirements
337+
338+- Accessibility: NeferWL turns it on at startup. Some apps need a flag:
339+ - Chromium: `--force-renderer-accessibility`
340+ - Electron: `ACCESSIBILITY_ENABLED=1`
341+ - Qt: `QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1`
342+- OCR: `tesseract` and `tesseract-data-eng`. Without tesseract, OCR is off
343+ and hint mode labels only apps that expose the accessibility tree.
344+
345+### Debugging hint mode
346+
347+With `hint.enable = on` and `NEFERWL_HINT_DEBUG=1` set when NeferWL starts (for example `NEFERWL_HINT_DEBUG=1 make tty`, or the variable in the launch environment), every successful scan writes three files to `$XDG_STATE_HOME/neferwl/hints/` (`~/.local/state/neferwl/hints/` when unset): `<time>-<output>.png`, the frame as scanned; `<time>-<output>.annotated.png`, the same with each target outlined (OCR links green, accessibility targets magenta, OCR window crops grey); and `<time>-<output>.json`, the targets (with their `source`, `atspi` or `ocr`) and timings. NeferWL keeps the 50 most recent scans and logs a warning at startup. The files contain the screen in clear: delete them after use.
M examples/config +7 -0
@@ -94,6 +94,11 @@ focus.flash-width = 2
9494 focus.flash-opacity = 0.6
9595 focus.flash-duration = 200ms
9696 focus.flash-color =
97+# Hint mode is experimental and off: turn it on here (restart required).
98+# hint.enable = off
99+# Hint mode labels the links, URLs and buttons on the focused screen:
100+# clickable (links, URLs and buttons) or links (links and URLs only).
101+# hint.targets = clickable
97102
98103 # Named workspaces are outside the numbered list and require a bind. The bind
99104 # shows one; pressing it again returns to the previous workspace.
@@ -195,6 +200,8 @@ bind.cmd+shift+f = toggle-fullscreen
195200 bind.cmd+s = toggle-stash-visible
196201 bind.cmd+shift+s = toggle-window-stash
197202 bind.cmd+o = toggle-overview
203+# Label the links and buttons on screen and click one by typing its label.
204+# bind.cmd+g = hint
198205 bind.cmd+q = close-window
199206 bind.ctrl+alt+backspace = quit
200207 bind.ctrl+cmd+space = spawn fuzzel
@@ -94,6 +94,11 @@ focus.flash-width = 2
94 focus.flash-opacity = 0.694 focus.flash-opacity = 0.6
95 focus.flash-duration = 200ms95 focus.flash-duration = 200ms
96 focus.flash-color =96 focus.flash-color =
97+# Hint mode is experimental and off: turn it on here (restart required).
98+# hint.enable = off
99+# Hint mode labels the links, URLs and buttons on the focused screen:
100+# clickable (links, URLs and buttons) or links (links and URLs only).
101+# hint.targets = clickable
97 102
98 # Named workspaces are outside the numbered list and require a bind. The bind103 # Named workspaces are outside the numbered list and require a bind. The bind
99 # shows one; pressing it again returns to the previous workspace.104 # shows one; pressing it again returns to the previous workspace.
@@ -195,6 +200,8 @@ bind.cmd+shift+f = toggle-fullscreen
195 bind.cmd+s = toggle-stash-visible200 bind.cmd+s = toggle-stash-visible
196 bind.cmd+shift+s = toggle-window-stash201 bind.cmd+shift+s = toggle-window-stash
197 bind.cmd+o = toggle-overview202 bind.cmd+o = toggle-overview
203+# Label the links and buttons on screen and click one by typing its label.
204+# bind.cmd+g = hint
198 bind.cmd+q = close-window205 bind.cmd+q = close-window
199 bind.ctrl+alt+backspace = quit206 bind.ctrl+alt+backspace = quit
200 bind.ctrl+cmd+space = spawn fuzzel207 bind.ctrl+cmd+space = spawn fuzzel
M go.mod +2 -0
@@ -15,6 +15,7 @@ require (
1515 github.com/godbus/dbus/v5 v5.2.2
1616 github.com/rs/zerolog v1.35.1
1717 github.com/stretchr/testify v1.12.1
18+ golang.org/x/image v0.46.0
1819 golang.org/x/sys v0.48.0
1920 golang.org/x/term v0.46.0
2021 )
@@ -24,5 +25,6 @@ require (
2425 github.com/mattn/go-isatty v0.0.22 // indirect
2526 github.com/stretchr/objx v0.5.3 // indirect
2627 go.yaml.in/yaml/v3 v3.0.5 // indirect
28+ golang.org/x/text v0.42.0 // indirect
2729 gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
2830 )
@@ -15,6 +15,7 @@ require (
15 github.com/godbus/dbus/v5 v5.2.215 github.com/godbus/dbus/v5 v5.2.2
16 github.com/rs/zerolog v1.35.116 github.com/rs/zerolog v1.35.1
17 github.com/stretchr/testify v1.12.117 github.com/stretchr/testify v1.12.1
18+ golang.org/x/image v0.46.0
18 golang.org/x/sys v0.48.019 golang.org/x/sys v0.48.0
19 golang.org/x/term v0.46.020 golang.org/x/term v0.46.0
20 )21 )
@@ -24,5 +25,6 @@ require (
24 github.com/mattn/go-isatty v0.0.22 // indirect25 github.com/mattn/go-isatty v0.0.22 // indirect
25 github.com/stretchr/objx v0.5.3 // indirect26 github.com/stretchr/objx v0.5.3 // indirect
26 go.yaml.in/yaml/v3 v3.0.5 // indirect27 go.yaml.in/yaml/v3 v3.0.5 // indirect
28+ golang.org/x/text v0.42.0 // indirect
27 gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect29 gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
28 )30 )
M go.sum +4 -0 generated file: collapsed
A internal/adapters/atspi/bus_test.go +197 -0
@@ -0,0 +1,197 @@
1+package atspi
2+
3+import (
4+ "bufio"
5+ "os"
6+ "os/exec"
7+ "strings"
8+ "testing"
9+ "time"
10+
11+ "github.com/bnema/zerobus"
12+ "github.com/stretchr/testify/require"
13+)
14+
15+// startBus starts a private dbus-daemon, so the test never touches the
16+// desktop's buses. It skips when the daemon is not installed.
17+func startBus(t *testing.T) string {
18+ t.Helper()
19+ bin, err := exec.LookPath("dbus-daemon")
20+ if err != nil {
21+ if os.Getenv("CI") != "" {
22+ t.Fatal("dbus-daemon not installed")
23+ }
24+ t.Skip("dbus-daemon not installed")
25+ }
26+ cmd := exec.Command(bin, "--session", "--nofork", "--nopidfile", "--print-address=1")
27+ out, err := cmd.StdoutPipe()
28+ require.NoError(t, err)
29+ require.NoError(t, cmd.Start())
30+ t.Cleanup(func() { _ = cmd.Process.Kill(); _ = cmd.Wait() })
31+ line, err := bufio.NewReader(out).ReadString('\n')
32+ require.NoError(t, err)
33+ return strings.TrimSpace(line)
34+}
35+
36+// TestDoDecodesMatchesPipelined runs Do against an app on a real bus: the
37+// GetMatches reply is decoded, and replies to several calls sent before any
38+// is read are matched by serial.
39+func TestDoDecodesMatchesPipelined(t *testing.T) {
40+ addr := startBus(t)
41+ app, err := zerobus.Dial(addr)
42+ require.NoError(t, err)
43+ t.Cleanup(func() { _ = app.Close() })
44+ _, err = app.RequestName("org.test.App", 0)
45+ require.NoError(t, err)
46+
47+ done := make(chan struct{})
48+ go func() {
49+ defer close(done)
50+ for {
51+ m, err := app.ReadMessage()
52+ if err != nil {
53+ return
54+ }
55+ if m.Type != zerobus.TypeMethodCall {
56+ continue
57+ }
58+ req := &zerobus.Message{Sender: strings.Clone(m.Sender), Serial: m.Serial}
59+ switch m.Member {
60+ case "GetMatches":
61+ // Check the rule: struct, then sort, count, traverse.
62+ r := m.Body()
63+ r.Struct()
64+ end := r.Array('i')
65+ var states []int32
66+ for r.More(end) {
67+ states = append(states, r.Int32())
68+ }
69+ _ = r.Int32()
70+ ad := r.Array('{')
71+ _ = ad
72+ _ = r.Int32()
73+ end = r.Array('i')
74+ var roles []int32
75+ for r.More(end) {
76+ roles = append(roles, r.Int32())
77+ }
78+ ok := r.Err() == nil && len(states) == 2 && states[0] == 1<<25|1<<30 && len(roles) == 5 && uint32(roles[2]) == 1<<24|1<<18|1<<31
79+ if !ok {
80+ app.NewError(req, "org.test.Error.BadRule", "s").Str("bad rule")
81+ _, _ = app.Send()
82+ continue
83+ }
84+ e := app.NewReply(req, "a(so)")
85+ a := e.BeginArray('(')
86+ for _, p := range []string{"/one", "/two"} {
87+ e.Struct()
88+ e.Str(":1.99")
89+ e.ObjectPath(p)
90+ }
91+ e.EndArray(a)
92+ case "GetExtents":
93+ e := app.NewReply(req, "(iiii)")
94+ e.Struct()
95+ e.Int32(1)
96+ e.Int32(2)
97+ e.Int32(30)
98+ e.Int32(40)
99+ case "GetRole":
100+ app.NewReply(req, "u").Uint32(roleLink)
101+ case "GetState":
102+ e := app.NewReply(req, "au")
103+ a := e.BeginArray('u')
104+ e.Uint32(stShowing)
105+ e.Uint32(0)
106+ e.EndArray(a)
107+ default:
108+ app.NewError(req, "org.freedesktop.DBus.Error.UnknownMethod", "s").Str("no")
109+ }
110+ _, _ = app.Send()
111+ }
112+ }()
113+
114+ cc, err := zerobus.Dial(addr)
115+ require.NoError(t, err)
116+ b, err := newConn(cc)
117+ require.NoError(t, err)
118+ t.Cleanup(func() { _ = b.Close() })
119+
120+ calls := []call{
121+ {op: opMatches, name: "org.test.App", path: "/frame", rule: matchRule{linksOnly: true}},
122+ {op: opExtents, name: "org.test.App", path: "/one", arg: coordWindow},
123+ {op: opRole, name: "org.test.App", path: "/one"},
124+ {op: opState, name: "org.test.App", path: "/one"},
125+ {op: opChildren, name: "org.test.App", path: "/one"},
126+ }
127+ rep := b.Do(time.Now().Add(2*time.Second), time.Second, calls)
128+ require.Len(t, rep, 5)
129+ require.NoError(t, rep[0].err)
130+ require.Equal(t, []ref{{":1.99", "/one"}, {":1.99", "/two"}}, rep[0].refs)
131+ require.NoError(t, rep[1].err)
132+ require.Equal(t, [4]int32{1, 2, 30, 40}, rep[1].rect)
133+ require.Equal(t, uint32(roleLink), rep[2].u)
134+ require.Equal(t, uint64(stShowing), rep[3].state)
135+ var be *busError
136+ require.ErrorAs(t, rep[4].err, &be)
137+ require.Equal(t, "org.freedesktop.DBus.Error.UnknownMethod", be.name)
138+
139+ // A call to a name nobody owns fails fast, others still succeed.
140+ rep = b.Do(time.Now().Add(2*time.Second), time.Second, []call{
141+ {op: opRole, name: "org.test.Nobody", path: "/x"},
142+ {op: opRole, name: "org.test.App", path: "/x"},
143+ })
144+ require.Error(t, rep[0].err)
145+ require.NoError(t, rep[1].err)
146+}
147+
148+func TestDoTimesOutOnSilentApp(t *testing.T) {
149+ addr := startBus(t)
150+ app, err := zerobus.Dial(addr)
151+ require.NoError(t, err)
152+ t.Cleanup(func() { _ = app.Close() })
153+ _, err = app.RequestName("org.test.Mute", 0)
154+ require.NoError(t, err)
155+
156+ cc, err := zerobus.Dial(addr)
157+ require.NoError(t, err)
158+ b, err := newConn(cc)
159+ require.NoError(t, err)
160+ t.Cleanup(func() { _ = b.Close() })
161+
162+ t0 := time.Now()
163+ rep := b.Do(time.Now().Add(time.Second), 50*time.Millisecond, []call{{op: opRole, name: "org.test.Mute", path: "/x"}})
164+ require.ErrorIs(t, rep[0].err, ErrTimeout)
165+ require.Less(t, time.Since(t0), 500*time.Millisecond)
166+}
167+
168+// A call left unanswered for idle gets ErrTimeout without breaking the
169+// connection; a batch that hits its deadline breaks it, and Do returns near
170+// the deadline even though the idle timeout is longer.
171+func TestDoDeadlineBreaksConnection(t *testing.T) {
172+ addr := startBus(t)
173+ app, err := zerobus.Dial(addr)
174+ require.NoError(t, err)
175+ t.Cleanup(func() { _ = app.Close() })
176+ _, err = app.RequestName("org.test.Mute", 0)
177+ require.NoError(t, err)
178+
179+ cc, err := zerobus.Dial(addr)
180+ require.NoError(t, err)
181+ b, err := newConn(cc)
182+ require.NoError(t, err)
183+ t.Cleanup(func() { _ = b.Close() })
184+
185+ mute := []call{{op: opRole, name: "org.test.Mute", path: "/x"}}
186+ rep := b.Do(time.Now().Add(time.Second), 40*time.Millisecond, mute)
187+ require.ErrorIs(t, rep[0].err, ErrTimeout)
188+ require.NotErrorIs(t, rep[0].err, ErrBroken)
189+
190+ t0 := time.Now()
191+ rep = b.Do(time.Now().Add(60*time.Millisecond), time.Second, mute)
192+ require.ErrorIs(t, rep[0].err, ErrBroken)
193+ require.Less(t, time.Since(t0), 300*time.Millisecond)
194+ // The socket is closed: the next batch fails at once.
195+ rep = b.Do(time.Now().Add(time.Second), time.Second, mute)
196+ require.ErrorIs(t, rep[0].err, ErrBroken)
197+}
@@ -0,0 +1,197 @@
1+package atspi
2+
3+import (
4+ "bufio"
5+ "os"
6+ "os/exec"
7+ "strings"
8+ "testing"
9+ "time"
10+
11+ "github.com/bnema/zerobus"
12+ "github.com/stretchr/testify/require"
13+)
14+
15+// startBus starts a private dbus-daemon, so the test never touches the
16+// desktop's buses. It skips when the daemon is not installed.
17+func startBus(t *testing.T) string {
18+ t.Helper()
19+ bin, err := exec.LookPath("dbus-daemon")
20+ if err != nil {
21+ if os.Getenv("CI") != "" {
22+ t.Fatal("dbus-daemon not installed")
23+ }
24+ t.Skip("dbus-daemon not installed")
25+ }
26+ cmd := exec.Command(bin, "--session", "--nofork", "--nopidfile", "--print-address=1")
27+ out, err := cmd.StdoutPipe()
28+ require.NoError(t, err)
29+ require.NoError(t, cmd.Start())
30+ t.Cleanup(func() { _ = cmd.Process.Kill(); _ = cmd.Wait() })
31+ line, err := bufio.NewReader(out).ReadString('\n')
32+ require.NoError(t, err)
33+ return strings.TrimSpace(line)
34+}
35+
36+// TestDoDecodesMatchesPipelined runs Do against an app on a real bus: the
37+// GetMatches reply is decoded, and replies to several calls sent before any
38+// is read are matched by serial.
39+func TestDoDecodesMatchesPipelined(t *testing.T) {
40+ addr := startBus(t)
41+ app, err := zerobus.Dial(addr)
42+ require.NoError(t, err)
43+ t.Cleanup(func() { _ = app.Close() })
44+ _, err = app.RequestName("org.test.App", 0)
45+ require.NoError(t, err)
46+
47+ done := make(chan struct{})
48+ go func() {
49+ defer close(done)
50+ for {
51+ m, err := app.ReadMessage()
52+ if err != nil {
53+ return
54+ }
55+ if m.Type != zerobus.TypeMethodCall {
56+ continue
57+ }
58+ req := &zerobus.Message{Sender: strings.Clone(m.Sender), Serial: m.Serial}
59+ switch m.Member {
60+ case "GetMatches":
61+ // Check the rule: struct, then sort, count, traverse.
62+ r := m.Body()
63+ r.Struct()
64+ end := r.Array('i')
65+ var states []int32
66+ for r.More(end) {
67+ states = append(states, r.Int32())
68+ }
69+ _ = r.Int32()
70+ ad := r.Array('{')
71+ _ = ad
72+ _ = r.Int32()
73+ end = r.Array('i')
74+ var roles []int32
75+ for r.More(end) {
76+ roles = append(roles, r.Int32())
77+ }
78+ ok := r.Err() == nil && len(states) == 2 && states[0] == 1<<25|1<<30 && len(roles) == 5 && uint32(roles[2]) == 1<<24|1<<18|1<<31
79+ if !ok {
80+ app.NewError(req, "org.test.Error.BadRule", "s").Str("bad rule")
81+ _, _ = app.Send()
82+ continue
83+ }
84+ e := app.NewReply(req, "a(so)")
85+ a := e.BeginArray('(')
86+ for _, p := range []string{"/one", "/two"} {
87+ e.Struct()
88+ e.Str(":1.99")
89+ e.ObjectPath(p)
90+ }
91+ e.EndArray(a)
92+ case "GetExtents":
93+ e := app.NewReply(req, "(iiii)")
94+ e.Struct()
95+ e.Int32(1)
96+ e.Int32(2)
97+ e.Int32(30)
98+ e.Int32(40)
99+ case "GetRole":
100+ app.NewReply(req, "u").Uint32(roleLink)
101+ case "GetState":
102+ e := app.NewReply(req, "au")
103+ a := e.BeginArray('u')
104+ e.Uint32(stShowing)
105+ e.Uint32(0)
106+ e.EndArray(a)
107+ default:
108+ app.NewError(req, "org.freedesktop.DBus.Error.UnknownMethod", "s").Str("no")
109+ }
110+ _, _ = app.Send()
111+ }
112+ }()
113+
114+ cc, err := zerobus.Dial(addr)
115+ require.NoError(t, err)
116+ b, err := newConn(cc)
117+ require.NoError(t, err)
118+ t.Cleanup(func() { _ = b.Close() })
119+
120+ calls := []call{
121+ {op: opMatches, name: "org.test.App", path: "/frame", rule: matchRule{linksOnly: true}},
122+ {op: opExtents, name: "org.test.App", path: "/one", arg: coordWindow},
123+ {op: opRole, name: "org.test.App", path: "/one"},
124+ {op: opState, name: "org.test.App", path: "/one"},
125+ {op: opChildren, name: "org.test.App", path: "/one"},
126+ }
127+ rep := b.Do(time.Now().Add(2*time.Second), time.Second, calls)
128+ require.Len(t, rep, 5)
129+ require.NoError(t, rep[0].err)
130+ require.Equal(t, []ref{{":1.99", "/one"}, {":1.99", "/two"}}, rep[0].refs)
131+ require.NoError(t, rep[1].err)
132+ require.Equal(t, [4]int32{1, 2, 30, 40}, rep[1].rect)
133+ require.Equal(t, uint32(roleLink), rep[2].u)
134+ require.Equal(t, uint64(stShowing), rep[3].state)
135+ var be *busError
136+ require.ErrorAs(t, rep[4].err, &be)
137+ require.Equal(t, "org.freedesktop.DBus.Error.UnknownMethod", be.name)
138+
139+ // A call to a name nobody owns fails fast, others still succeed.
140+ rep = b.Do(time.Now().Add(2*time.Second), time.Second, []call{
141+ {op: opRole, name: "org.test.Nobody", path: "/x"},
142+ {op: opRole, name: "org.test.App", path: "/x"},
143+ })
144+ require.Error(t, rep[0].err)
145+ require.NoError(t, rep[1].err)
146+}
147+
148+func TestDoTimesOutOnSilentApp(t *testing.T) {
149+ addr := startBus(t)
150+ app, err := zerobus.Dial(addr)
151+ require.NoError(t, err)
152+ t.Cleanup(func() { _ = app.Close() })
153+ _, err = app.RequestName("org.test.Mute", 0)
154+ require.NoError(t, err)
155+
156+ cc, err := zerobus.Dial(addr)
157+ require.NoError(t, err)
158+ b, err := newConn(cc)
159+ require.NoError(t, err)
160+ t.Cleanup(func() { _ = b.Close() })
161+
162+ t0 := time.Now()
163+ rep := b.Do(time.Now().Add(time.Second), 50*time.Millisecond, []call{{op: opRole, name: "org.test.Mute", path: "/x"}})
164+ require.ErrorIs(t, rep[0].err, ErrTimeout)
165+ require.Less(t, time.Since(t0), 500*time.Millisecond)
166+}
167+
168+// A call left unanswered for idle gets ErrTimeout without breaking the
169+// connection; a batch that hits its deadline breaks it, and Do returns near
170+// the deadline even though the idle timeout is longer.
171+func TestDoDeadlineBreaksConnection(t *testing.T) {
172+ addr := startBus(t)
173+ app, err := zerobus.Dial(addr)
174+ require.NoError(t, err)
175+ t.Cleanup(func() { _ = app.Close() })
176+ _, err = app.RequestName("org.test.Mute", 0)
177+ require.NoError(t, err)
178+
179+ cc, err := zerobus.Dial(addr)
180+ require.NoError(t, err)
181+ b, err := newConn(cc)
182+ require.NoError(t, err)
183+ t.Cleanup(func() { _ = b.Close() })
184+
185+ mute := []call{{op: opRole, name: "org.test.Mute", path: "/x"}}
186+ rep := b.Do(time.Now().Add(time.Second), 40*time.Millisecond, mute)
187+ require.ErrorIs(t, rep[0].err, ErrTimeout)
188+ require.NotErrorIs(t, rep[0].err, ErrBroken)
189+
190+ t0 := time.Now()
191+ rep = b.Do(time.Now().Add(60*time.Millisecond), time.Second, mute)
192+ require.ErrorIs(t, rep[0].err, ErrBroken)
193+ require.Less(t, time.Since(t0), 300*time.Millisecond)
194+ // The socket is closed: the next batch fails at once.
195+ rep = b.Do(time.Now().Add(time.Second), time.Second, mute)
196+ require.ErrorIs(t, rep[0].err, ErrBroken)
197+}
A internal/adapters/atspi/connector_mock_test.go +109 -0
@@ -0,0 +1,109 @@
1+// Code generated by mockery; DO NOT EDIT.
2+// github.com/vektra/mockery
3+// template: testify
4+
5+package atspi
6+
7+import (
8+ "context"
9+
10+ mock "github.com/stretchr/testify/mock"
11+)
12+
13+// newMockconnector creates a new instance of mockconnector. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
14+// The first argument is typically a *testing.T value.
15+func newMockconnector(t interface {
16+ mock.TestingT
17+ Cleanup(func())
18+}) *mockconnector {
19+ if helper, ok := t.(interface{ Helper() }); ok {
20+ helper.Helper()
21+ }
22+
23+ mock := &mockconnector{}
24+ mock.Mock.Test(t)
25+
26+ t.Cleanup(func() {
27+ if helper, ok := t.(interface{ Helper() }); ok {
28+ helper.Helper()
29+ }
30+ mock.AssertExpectations(t)
31+ })
32+
33+ return mock
34+}
35+
36+// mockconnector is an autogenerated mock type for the connector type
37+type mockconnector struct {
38+ mock.Mock
39+}
40+
41+type mockconnector_Expecter struct {
42+ mock *mock.Mock
43+}
44+
45+func (_m *mockconnector) EXPECT() *mockconnector_Expecter {
46+ return &mockconnector_Expecter{mock: &_m.Mock}
47+}
48+
49+// Connect provides a mock function for the type mockconnector
50+func (_mock *mockconnector) Connect(ctx context.Context) (transport, error) {
51+ ret := _mock.Called(ctx)
52+
53+ if len(ret) == 0 {
54+ panic("no return value specified for Connect")
55+ }
56+
57+ var r0 transport
58+ var r1 error
59+ if returnFunc, ok := ret.Get(0).(func(context.Context) (transport, error)); ok {
60+ return returnFunc(ctx)
61+ }
62+ if returnFunc, ok := ret.Get(0).(func(context.Context) transport); ok {
63+ r0 = returnFunc(ctx)
64+ } else {
65+ if ret.Get(0) != nil {
66+ r0 = ret.Get(0).(transport)
67+ }
68+ }
69+ if returnFunc, ok := ret.Get(1).(func(context.Context) error); ok {
70+ r1 = returnFunc(ctx)
71+ } else {
72+ r1 = ret.Error(1)
73+ }
74+ return r0, r1
75+}
76+
77+// mockconnector_Connect_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Connect'
78+type mockconnector_Connect_Call struct {
79+ *mock.Call
80+}
81+
82+// Connect is a helper method to define mock.On call
83+// - ctx context.Context
84+func (_e *mockconnector_Expecter) Connect(ctx any) *mockconnector_Connect_Call {
85+ return &mockconnector_Connect_Call{Call: _e.mock.On("Connect", ctx)}
86+}
87+
88+func (_c *mockconnector_Connect_Call) Run(run func(ctx context.Context)) *mockconnector_Connect_Call {
89+ _c.Call.Run(func(args mock.Arguments) {
90+ var arg0 context.Context
91+ if args[0] != nil {
92+ arg0 = args[0].(context.Context)
93+ }
94+ run(
95+ arg0,
96+ )
97+ })
98+ return _c
99+}
100+
101+func (_c *mockconnector_Connect_Call) Return(transportMoqParam transport, err error) *mockconnector_Connect_Call {
102+ _c.Call.Return(transportMoqParam, err)
103+ return _c
104+}
105+
106+func (_c *mockconnector_Connect_Call) RunAndReturn(run func(ctx context.Context) (transport, error)) *mockconnector_Connect_Call {
107+ _c.Call.Return(run)
108+ return _c
109+}
@@ -0,0 +1,109 @@
1+// Code generated by mockery; DO NOT EDIT.
2+// github.com/vektra/mockery
3+// template: testify
4+
5+package atspi
6+
7+import (
8+ "context"
9+
10+ mock "github.com/stretchr/testify/mock"
11+)
12+
13+// newMockconnector creates a new instance of mockconnector. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
14+// The first argument is typically a *testing.T value.
15+func newMockconnector(t interface {
16+ mock.TestingT
17+ Cleanup(func())
18+}) *mockconnector {
19+ if helper, ok := t.(interface{ Helper() }); ok {
20+ helper.Helper()
21+ }
22+
23+ mock := &mockconnector{}
24+ mock.Mock.Test(t)
25+
26+ t.Cleanup(func() {
27+ if helper, ok := t.(interface{ Helper() }); ok {
28+ helper.Helper()
29+ }
30+ mock.AssertExpectations(t)
31+ })
32+
33+ return mock
34+}
35+
36+// mockconnector is an autogenerated mock type for the connector type
37+type mockconnector struct {
38+ mock.Mock
39+}
40+
41+type mockconnector_Expecter struct {
42+ mock *mock.Mock
43+}
44+
45+func (_m *mockconnector) EXPECT() *mockconnector_Expecter {
46+ return &mockconnector_Expecter{mock: &_m.Mock}
47+}
48+
49+// Connect provides a mock function for the type mockconnector
50+func (_mock *mockconnector) Connect(ctx context.Context) (transport, error) {
51+ ret := _mock.Called(ctx)
52+
53+ if len(ret) == 0 {
54+ panic("no return value specified for Connect")
55+ }
56+
57+ var r0 transport
58+ var r1 error
59+ if returnFunc, ok := ret.Get(0).(func(context.Context) (transport, error)); ok {
60+ return returnFunc(ctx)
61+ }
62+ if returnFunc, ok := ret.Get(0).(func(context.Context) transport); ok {
63+ r0 = returnFunc(ctx)
64+ } else {
65+ if ret.Get(0) != nil {
66+ r0 = ret.Get(0).(transport)
67+ }
68+ }
69+ if returnFunc, ok := ret.Get(1).(func(context.Context) error); ok {
70+ r1 = returnFunc(ctx)
71+ } else {
72+ r1 = ret.Error(1)
73+ }
74+ return r0, r1
75+}
76+
77+// mockconnector_Connect_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Connect'
78+type mockconnector_Connect_Call struct {
79+ *mock.Call
80+}
81+
82+// Connect is a helper method to define mock.On call
83+// - ctx context.Context
84+func (_e *mockconnector_Expecter) Connect(ctx any) *mockconnector_Connect_Call {
85+ return &mockconnector_Connect_Call{Call: _e.mock.On("Connect", ctx)}
86+}
87+
88+func (_c *mockconnector_Connect_Call) Run(run func(ctx context.Context)) *mockconnector_Connect_Call {
89+ _c.Call.Run(func(args mock.Arguments) {
90+ var arg0 context.Context
91+ if args[0] != nil {
92+ arg0 = args[0].(context.Context)
93+ }
94+ run(
95+ arg0,
96+ )
97+ })
98+ return _c
99+}
100+
101+func (_c *mockconnector_Connect_Call) Return(transportMoqParam transport, err error) *mockconnector_Connect_Call {
102+ _c.Call.Return(transportMoqParam, err)
103+ return _c
104+}
105+
106+func (_c *mockconnector_Connect_Call) RunAndReturn(run func(ctx context.Context) (transport, error)) *mockconnector_Connect_Call {
107+ _c.Call.Return(run)
108+ return _c
109+}
A internal/adapters/atspi/doc.go +10 -0
@@ -0,0 +1,10 @@
1+// Package atspi finds hint targets in the accessibility tree of the apps on
2+// screen. It talks to the AT-SPI registry over the accessibility bus with
3+// zerobus, from one connection used by one goroutine at a time, and pipelines
4+// the calls it needs: several are sent before their replies are read, and the
5+// replies are matched by serial.
6+//
7+// The approach (Collection.GetMatches on the window frame, bounded walk when
8+// the toolkit has no Collection, window-relative extents) is adapted from
9+// neru's AT-SPI client, https://github.com/y3owk1n/neru (MIT licence).
10+package atspi
@@ -0,0 +1,10 @@
1+// Package atspi finds hint targets in the accessibility tree of the apps on
2+// screen. It talks to the AT-SPI registry over the accessibility bus with
3+// zerobus, from one connection used by one goroutine at a time, and pipelines
4+// the calls it needs: several are sent before their replies are read, and the
5+// replies are matched by serial.
6+//
7+// The approach (Collection.GetMatches on the window frame, bounded walk when
8+// the toolkit has no Collection, window-relative extents) is adapted from
9+// neru's AT-SPI client, https://github.com/y3owk1n/neru (MIT licence).
10+package atspi
A internal/adapters/atspi/model.go +206 -0
@@ -0,0 +1,206 @@
1+package atspi
2+
3+import "github.com/bnema/neferwl/internal/ports"
4+
5+// AT-SPI constants (atspi-constants.h). The role ids are the AtspiRole enum
6+// values; the state ids are bit indices of the state set.
7+const (
8+ roleCheckBox = 7
9+ roleComboBox = 11
10+ roleDialog = 16
11+ roleFrame = 23
12+ roleMenuItem = 35
13+ rolePageTab = 37
14+ rolePushButton = 43
15+ roleRadioButton = 44
16+ roleToggleButton = 62
17+ roleWindow = 69
18+ roleLink = 88
19+ roleDocumentFrame = 82
20+ roleDocumentWeb = 95
21+
22+ stateActive = 1
23+ stateShowing = 25
24+ stateVisible = 30
25+
26+ roleWords = 5 // int32 words of a role set in a MatchRule
27+ stateWords = 2 // int32 words of a state set in a MatchRule
28+
29+ // AtspiCollectionMatchType and AtspiCollectionSortOrder values.
30+ matchAll = 1
31+ matchAny = 2
32+ sortCanonical = 1
33+ matchCount = 500
34+ maxDepth = 30
35+ maxWalkNodes = 3000
36+ maxFrames = 32
37+ sizeTolerance = 40
38+ maxMatchesRead = matchCount
39+)
40+
41+// clickableRoles are the roles that become hint targets.
42+var clickableRoles = []uint32{
43+ roleLink, rolePushButton, roleToggleButton, roleCheckBox,
44+ roleRadioButton, roleMenuItem, rolePageTab, roleComboBox,
45+}
46+
47+// matchRule is the Collection query: showing and visible objects of the
48+// clickable roles (only links when linksOnly), plus web documents for coverage.
49+type matchRule struct{ linksOnly bool }
50+
51+func (r matchRule) roles() []uint32 {
52+ if r.linksOnly {
53+ return []uint32{roleLink, roleDocumentFrame, roleDocumentWeb}
54+ }
55+ return append(append([]uint32(nil), clickableRoles...), roleDocumentFrame, roleDocumentWeb)
56+}
57+
58+// roleBits packs role ids into the bit set of a MatchRule: bit id%32 of word
59+// id/32.
60+func roleBits(ids []uint32) []int32 {
61+ words := make([]int32, roleWords)
62+ for _, id := range ids {
63+ if w := int(id / 32); w < len(words) {
64+ words[w] |= int32(uint32(1) << (id % 32))
65+ }
66+ }
67+ return words
68+}
69+
70+// stateBits packs state ids into the bit set of a MatchRule.
71+func stateBits(ids ...uint32) []int32 {
72+ words := make([]int32, stateWords)
73+ for _, id := range ids {
74+ if w := int(id / 32); w < len(words) {
75+ words[w] |= int32(uint32(1) << (id % 32))
76+ }
77+ }
78+ return words
79+}
80+
81+func hasState(set uint64, state uint) bool { return set&(1<<state) != 0 }
82+
83+// isClickable reports whether role makes a target.
84+func isClickable(role uint32, linksOnly bool) bool {
85+ if linksOnly {
86+ return role == roleLink
87+ }
88+ for _, r := range clickableRoles {
89+ if r == role {
90+ return true
91+ }
92+ }
93+ return false
94+}
95+
96+// isWindowRole reports whether role is a top-level window role.
97+func isWindowRole(role uint32) bool {
98+ return role == roleFrame || role == roleWindow || role == roleDialog
99+}
100+
101+// kindOf maps a role to the hint kind: links are links, the rest buttons.
102+func kindOf(role uint32) ports.HintKind {
103+ if role == roleLink {
104+ return ports.HintLink
105+ }
106+ return ports.HintButton
107+}
108+
109+// frameInfo is a showing top-level window of an app.
110+type frameInfo struct {
111+ ref ref
112+ active bool
113+ w, h int // size of the frame, from its window extents
114+}
115+
116+// matchFrames pairs the windows of one app with its frames, one to one. A
117+// window is paired with a frame when the pairing is certain:
118+// - one window and one frame;
119+// - the focused window with the one ACTIVE frame; that frame is then
120+// reserved and no other window can take it;
121+// - otherwise the one unreserved frame whose size is within sizeTolerance
122+// of the window, provided no other unmatched window claims it as well.
123+//
124+// Ambiguous windows are left out. The result maps the index of a window in
125+// wins to the index of its frame in frames.
126+func matchFrames(wins []ports.HintWindow, frames []frameInfo) map[int]int {
127+ out := map[int]int{}
128+ if len(wins) == 1 && len(frames) == 1 {
129+ out[0] = 0
130+ return out
131+ }
132+ // Active matches first: they are reserved against the size matches.
133+ active, n := -1, 0
134+ for j, f := range frames {
135+ if f.active {
136+ active, n = j, n+1
137+ }
138+ }
139+ reserved := make([]bool, len(frames))
140+ done := make([]bool, len(wins))
141+ if n == 1 {
142+ focused := -1
143+ count := 0
144+ for i, w := range wins {
145+ if w.Focused {
146+ focused, count = i, count+1
147+ }
148+ }
149+ if count == 1 {
150+ out[focused] = active
151+ reserved[active] = true
152+ done[focused] = true
153+ }
154+ }
155+ claims := make([][]int, len(wins)) // size candidates per window
156+ owners := make([]int, len(frames)) // windows that size-match a frame
157+ for i, w := range wins {
158+ if done[i] {
159+ continue
160+ }
161+ for j, f := range frames {
162+ if !reserved[j] && absInt(f.w-w.Rect.W) <= sizeTolerance && absInt(f.h-w.Rect.H) <= sizeTolerance {
163+ claims[i] = append(claims[i], j)
164+ owners[j]++
165+ }
166+ }
167+ }
168+ for i := range wins {
169+ if !done[i] && len(claims[i]) == 1 && owners[claims[i][0]] == 1 {
170+ out[i] = claims[i][0]
171+ }
172+ }
173+ return out
174+}
175+
176+func absInt(v int) int {
177+ if v < 0 {
178+ return -v
179+ }
180+ return v
181+}
182+
183+// element is a match with its extents.
184+type element struct {
185+ role uint32
186+ x, y, w, h int32 // window-relative
187+}
188+
189+// toTarget converts an element of a window into a logical output-local
190+// target: the element extents are relative to the top-level frame, whose own
191+// window-relative origin is (fx, fy); the result is clipped to the window.
192+// It reports false for an empty result.
193+func toTarget(win ports.Rect, fx, fy int32, e element) (ports.HintTarget, bool) {
194+ if e.w <= 0 || e.h <= 0 {
195+ return ports.HintTarget{}, false
196+ }
197+ r := ports.Rect{X: win.X + int(e.x-fx), Y: win.Y + int(e.y-fy), W: int(e.w), H: int(e.h)}
198+ x0, y0 := max(r.X, win.X), max(r.Y, win.Y)
199+ x1, y1 := min(r.X+r.W, win.X+win.W), min(r.Y+r.H, win.Y+win.H)
200+ if x1 <= x0 || y1 <= y0 {
201+ return ports.HintTarget{}, false
202+ }
203+ return ports.HintTarget{Rect: ports.Rect{X: x0, Y: y0, W: x1 - x0, H: y1 - y0}, Kind: kindOf(e.role)}, true
204+}
205+
206+func isDocument(role uint32) bool { return role == roleDocumentFrame || role == roleDocumentWeb }
@@ -0,0 +1,206 @@
1+package atspi
2+
3+import "github.com/bnema/neferwl/internal/ports"
4+
5+// AT-SPI constants (atspi-constants.h). The role ids are the AtspiRole enum
6+// values; the state ids are bit indices of the state set.
7+const (
8+ roleCheckBox = 7
9+ roleComboBox = 11
10+ roleDialog = 16
11+ roleFrame = 23
12+ roleMenuItem = 35
13+ rolePageTab = 37
14+ rolePushButton = 43
15+ roleRadioButton = 44
16+ roleToggleButton = 62
17+ roleWindow = 69
18+ roleLink = 88
19+ roleDocumentFrame = 82
20+ roleDocumentWeb = 95
21+
22+ stateActive = 1
23+ stateShowing = 25
24+ stateVisible = 30
25+
26+ roleWords = 5 // int32 words of a role set in a MatchRule
27+ stateWords = 2 // int32 words of a state set in a MatchRule
28+
29+ // AtspiCollectionMatchType and AtspiCollectionSortOrder values.
30+ matchAll = 1
31+ matchAny = 2
32+ sortCanonical = 1
33+ matchCount = 500
34+ maxDepth = 30
35+ maxWalkNodes = 3000
36+ maxFrames = 32
37+ sizeTolerance = 40
38+ maxMatchesRead = matchCount
39+)
40+
41+// clickableRoles are the roles that become hint targets.
42+var clickableRoles = []uint32{
43+ roleLink, rolePushButton, roleToggleButton, roleCheckBox,
44+ roleRadioButton, roleMenuItem, rolePageTab, roleComboBox,
45+}
46+
47+// matchRule is the Collection query: showing and visible objects of the
48+// clickable roles (only links when linksOnly), plus web documents for coverage.
49+type matchRule struct{ linksOnly bool }
50+
51+func (r matchRule) roles() []uint32 {
52+ if r.linksOnly {
53+ return []uint32{roleLink, roleDocumentFrame, roleDocumentWeb}
54+ }
55+ return append(append([]uint32(nil), clickableRoles...), roleDocumentFrame, roleDocumentWeb)
56+}
57+
58+// roleBits packs role ids into the bit set of a MatchRule: bit id%32 of word
59+// id/32.
60+func roleBits(ids []uint32) []int32 {
61+ words := make([]int32, roleWords)
62+ for _, id := range ids {
63+ if w := int(id / 32); w < len(words) {
64+ words[w] |= int32(uint32(1) << (id % 32))
65+ }
66+ }
67+ return words
68+}
69+
70+// stateBits packs state ids into the bit set of a MatchRule.
71+func stateBits(ids ...uint32) []int32 {
72+ words := make([]int32, stateWords)
73+ for _, id := range ids {
74+ if w := int(id / 32); w < len(words) {
75+ words[w] |= int32(uint32(1) << (id % 32))
76+ }
77+ }
78+ return words
79+}
80+
81+func hasState(set uint64, state uint) bool { return set&(1<<state) != 0 }
82+
83+// isClickable reports whether role makes a target.
84+func isClickable(role uint32, linksOnly bool) bool {
85+ if linksOnly {
86+ return role == roleLink
87+ }
88+ for _, r := range clickableRoles {
89+ if r == role {
90+ return true
91+ }
92+ }
93+ return false
94+}
95+
96+// isWindowRole reports whether role is a top-level window role.
97+func isWindowRole(role uint32) bool {
98+ return role == roleFrame || role == roleWindow || role == roleDialog
99+}
100+
101+// kindOf maps a role to the hint kind: links are links, the rest buttons.
102+func kindOf(role uint32) ports.HintKind {
103+ if role == roleLink {
104+ return ports.HintLink
105+ }
106+ return ports.HintButton
107+}
108+
109+// frameInfo is a showing top-level window of an app.
110+type frameInfo struct {
111+ ref ref
112+ active bool
113+ w, h int // size of the frame, from its window extents
114+}
115+
116+// matchFrames pairs the windows of one app with its frames, one to one. A
117+// window is paired with a frame when the pairing is certain:
118+// - one window and one frame;
119+// - the focused window with the one ACTIVE frame; that frame is then
120+// reserved and no other window can take it;
121+// - otherwise the one unreserved frame whose size is within sizeTolerance
122+// of the window, provided no other unmatched window claims it as well.
123+//
124+// Ambiguous windows are left out. The result maps the index of a window in
125+// wins to the index of its frame in frames.
126+func matchFrames(wins []ports.HintWindow, frames []frameInfo) map[int]int {
127+ out := map[int]int{}
128+ if len(wins) == 1 && len(frames) == 1 {
129+ out[0] = 0
130+ return out
131+ }
132+ // Active matches first: they are reserved against the size matches.
133+ active, n := -1, 0
134+ for j, f := range frames {
135+ if f.active {
136+ active, n = j, n+1
137+ }
138+ }
139+ reserved := make([]bool, len(frames))
140+ done := make([]bool, len(wins))
141+ if n == 1 {
142+ focused := -1
143+ count := 0
144+ for i, w := range wins {
145+ if w.Focused {
146+ focused, count = i, count+1
147+ }
148+ }
149+ if count == 1 {
150+ out[focused] = active
151+ reserved[active] = true
152+ done[focused] = true
153+ }
154+ }
155+ claims := make([][]int, len(wins)) // size candidates per window
156+ owners := make([]int, len(frames)) // windows that size-match a frame
157+ for i, w := range wins {
158+ if done[i] {
159+ continue
160+ }
161+ for j, f := range frames {
162+ if !reserved[j] && absInt(f.w-w.Rect.W) <= sizeTolerance && absInt(f.h-w.Rect.H) <= sizeTolerance {
163+ claims[i] = append(claims[i], j)
164+ owners[j]++
165+ }
166+ }
167+ }
168+ for i := range wins {
169+ if !done[i] && len(claims[i]) == 1 && owners[claims[i][0]] == 1 {
170+ out[i] = claims[i][0]
171+ }
172+ }
173+ return out
174+}
175+
176+func absInt(v int) int {
177+ if v < 0 {
178+ return -v
179+ }
180+ return v
181+}
182+
183+// element is a match with its extents.
184+type element struct {
185+ role uint32
186+ x, y, w, h int32 // window-relative
187+}
188+
189+// toTarget converts an element of a window into a logical output-local
190+// target: the element extents are relative to the top-level frame, whose own
191+// window-relative origin is (fx, fy); the result is clipped to the window.
192+// It reports false for an empty result.
193+func toTarget(win ports.Rect, fx, fy int32, e element) (ports.HintTarget, bool) {
194+ if e.w <= 0 || e.h <= 0 {
195+ return ports.HintTarget{}, false
196+ }
197+ r := ports.Rect{X: win.X + int(e.x-fx), Y: win.Y + int(e.y-fy), W: int(e.w), H: int(e.h)}
198+ x0, y0 := max(r.X, win.X), max(r.Y, win.Y)
199+ x1, y1 := min(r.X+r.W, win.X+win.W), min(r.Y+r.H, win.Y+win.H)
200+ if x1 <= x0 || y1 <= y0 {
201+ return ports.HintTarget{}, false
202+ }
203+ return ports.HintTarget{Rect: ports.Rect{X: x0, Y: y0, W: x1 - x0, H: y1 - y0}, Kind: kindOf(e.role)}, true
204+}
205+
206+func isDocument(role uint32) bool { return role == roleDocumentFrame || role == roleDocumentWeb }
A internal/adapters/atspi/model_test.go +117 -0
@@ -0,0 +1,117 @@
1+package atspi
2+
3+import (
4+ "testing"
5+
6+ "github.com/stretchr/testify/require"
7+
8+ "github.com/bnema/neferwl/internal/ports"
9+)
10+
11+func TestToTargetConvertsWindowCoordinates(t *testing.T) {
12+ win := ports.Rect{X: 100, Y: 50, W: 800, H: 600}
13+ // The frame origin (fx, fy) is subtracted: toolkit offsets cancel out.
14+ got, ok := toTarget(win, 10, 20, element{role: roleLink, x: 110, y: 120, w: 40, h: 16})
15+ require.True(t, ok)
16+ require.Equal(t, ports.HintTarget{Rect: ports.Rect{X: 200, Y: 150, W: 40, H: 16}, Kind: ports.HintLink}, got)
17+}
18+
19+func TestToTargetClipsToWindow(t *testing.T) {
20+ win := ports.Rect{X: 100, Y: 50, W: 200, H: 100}
21+ got, ok := toTarget(win, 0, 0, element{role: rolePushButton, x: 180, y: 90, w: 60, h: 30})
22+ require.True(t, ok)
23+ require.Equal(t, ports.Rect{X: 280, Y: 140, W: 20, H: 10}, got.Rect)
24+ require.Equal(t, ports.HintButton, got.Kind)
25+}
26+
27+func TestToTargetDropsEmptyAndOutside(t *testing.T) {
28+ win := ports.Rect{X: 0, Y: 0, W: 100, H: 100}
29+ for name, e := range map[string]element{
30+ "zero width": {role: roleLink, x: 1, y: 1, w: 0, h: 10},
31+ "neg height": {role: roleLink, x: 1, y: 1, w: 10, h: -1},
32+ "outside": {role: roleLink, x: 500, y: 1, w: 10, h: 10},
33+ "above": {role: roleLink, x: 1, y: -50, w: 10, h: 10},
34+ "left of win": {role: roleLink, x: -30, y: 1, w: 10, h: 10},
35+ } {
36+ _, ok := toTarget(win, 0, 0, e)
37+ require.False(t, ok, name)
38+ }
39+}
40+
41+func TestRoleKindMapping(t *testing.T) {
42+ require.Equal(t, ports.HintLink, kindOf(roleLink))
43+ for _, r := range []uint32{rolePushButton, roleToggleButton, roleCheckBox, roleRadioButton, roleMenuItem, rolePageTab, roleComboBox} {
44+ require.Equal(t, ports.HintButton, kindOf(r), r)
45+ require.True(t, isClickable(r, false), r)
46+ require.False(t, isClickable(r, true), r)
47+ }
48+ require.True(t, isClickable(roleLink, true))
49+ require.False(t, isClickable(roleFrame, false))
50+ require.True(t, isWindowRole(roleFrame) && isWindowRole(roleWindow) && isWindowRole(roleDialog))
51+ require.False(t, isWindowRole(roleLink))
52+}
53+
54+func TestRoleAndStateBits(t *testing.T) {
55+ require.Equal(t, []int32{0, 0, 0, 0, 0}, roleBits(nil))
56+ // link is role 88: word 2, bit 24; push button is 43: word 1, bit 11.
57+ require.Equal(t, []int32{0, 1 << 11, 1 << 24, 0, 0}, roleBits([]uint32{roleLink, rolePushButton}))
58+ // showing is state 25, visible 30: both in word 0.
59+ require.Equal(t, []int32{1<<25 | 1<<30, 0}, stateBits(stateShowing, stateVisible))
60+}
61+
62+func wins(ws ...ports.HintWindow) []ports.HintWindow { return ws }
63+
64+func win(id ports.WindowID, w, h int, focused bool) ports.HintWindow {
65+ return ports.HintWindow{ID: id, PID: 1, Rect: ports.Rect{W: w, H: h}, Focused: focused}
66+}
67+
68+func TestMatchFramesSingle(t *testing.T) {
69+ got := matchFrames(wins(win(1, 800, 600, false)), []frameInfo{{w: 100, h: 100}})
70+ require.Equal(t, map[int]int{0: 0}, got)
71+}
72+
73+func TestMatchFramesActive(t *testing.T) {
74+ frames := []frameInfo{{w: 800, h: 600}, {w: 800, h: 600, active: true}}
75+ got := matchFrames(wins(win(1, 800, 600, true), win(2, 800, 600, false)), frames)
76+ // The focused window takes the ACTIVE frame and reserves it; the other
77+ // window then only has the remaining frame to match.
78+ require.Equal(t, map[int]int{0: 1, 1: 0}, got)
79+}
80+
81+func TestMatchFramesSize(t *testing.T) {
82+ frames := []frameInfo{{w: 1000, h: 700}, {w: 400, h: 300}}
83+ got := matchFrames(wins(win(1, 1000, 700, false), win(2, 420, 290, false)), frames)
84+ require.Equal(t, map[int]int{0: 0, 1: 1}, got)
85+}
86+
87+func TestMatchFramesSizeTolerance(t *testing.T) {
88+ frames := []frameInfo{{w: 840, h: 640}, {w: 100, h: 100}}
89+ other := win(2, 90, 90, false)
90+ // 40 px off still matches; 41 px does not.
91+ require.Equal(t, map[int]int{0: 0, 1: 1}, matchFrames(wins(win(1, 800, 600, false), other), frames))
92+ require.Equal(t, map[int]int{1: 1}, matchFrames(wins(win(1, 799, 600, false), other), frames))
93+}
94+
95+func TestMatchFramesAmbiguous(t *testing.T) {
96+ // Two windows and two frames of the same size, none active: skipped.
97+ frames := []frameInfo{{w: 800, h: 600}, {w: 810, h: 600}}
98+ got := matchFrames(wins(win(1, 800, 600, false), win(2, 800, 600, false)), frames)
99+ require.Empty(t, got)
100+ // One window, two same-size frames, not focused: skipped.
101+ require.Empty(t, matchFrames(wins(win(1, 800, 600, false)), frames))
102+ // No frame close enough.
103+ require.Empty(t, matchFrames(wins(win(1, 200, 200, false), win(2, 300, 300, false)), frames))
104+}
105+
106+// The active frame is reserved by the focused window: an unfocused window of
107+// the same size as that frame must not be paired with it as well.
108+func TestMatchFramesOneToOne(t *testing.T) {
109+ frames := []frameInfo{{ref: ref{"a", "/active"}, active: true, w: 400, h: 300}, {ref: ref{"a", "/idle"}, w: 800, h: 600}}
110+ got := matchFrames(wins(win(1, 800, 600, true), win(2, 400, 300, false)), frames)
111+ seen := map[int]bool{}
112+ for _, f := range got {
113+ require.False(t, seen[f], "frame %d assigned twice: %v", f, got)
114+ seen[f] = true
115+ }
116+ require.Equal(t, map[int]int{0: 0}, got)
117+}
@@ -0,0 +1,117 @@
1+package atspi
2+
3+import (
4+ "testing"
5+
6+ "github.com/stretchr/testify/require"
7+
8+ "github.com/bnema/neferwl/internal/ports"
9+)
10+
11+func TestToTargetConvertsWindowCoordinates(t *testing.T) {
12+ win := ports.Rect{X: 100, Y: 50, W: 800, H: 600}
13+ // The frame origin (fx, fy) is subtracted: toolkit offsets cancel out.
14+ got, ok := toTarget(win, 10, 20, element{role: roleLink, x: 110, y: 120, w: 40, h: 16})
15+ require.True(t, ok)
16+ require.Equal(t, ports.HintTarget{Rect: ports.Rect{X: 200, Y: 150, W: 40, H: 16}, Kind: ports.HintLink}, got)
17+}
18+
19+func TestToTargetClipsToWindow(t *testing.T) {
20+ win := ports.Rect{X: 100, Y: 50, W: 200, H: 100}
21+ got, ok := toTarget(win, 0, 0, element{role: rolePushButton, x: 180, y: 90, w: 60, h: 30})
22+ require.True(t, ok)
23+ require.Equal(t, ports.Rect{X: 280, Y: 140, W: 20, H: 10}, got.Rect)
24+ require.Equal(t, ports.HintButton, got.Kind)
25+}
26+
27+func TestToTargetDropsEmptyAndOutside(t *testing.T) {
28+ win := ports.Rect{X: 0, Y: 0, W: 100, H: 100}
29+ for name, e := range map[string]element{
30+ "zero width": {role: roleLink, x: 1, y: 1, w: 0, h: 10},
31+ "neg height": {role: roleLink, x: 1, y: 1, w: 10, h: -1},
32+ "outside": {role: roleLink, x: 500, y: 1, w: 10, h: 10},
33+ "above": {role: roleLink, x: 1, y: -50, w: 10, h: 10},
34+ "left of win": {role: roleLink, x: -30, y: 1, w: 10, h: 10},
35+ } {
36+ _, ok := toTarget(win, 0, 0, e)
37+ require.False(t, ok, name)
38+ }
39+}
40+
41+func TestRoleKindMapping(t *testing.T) {
42+ require.Equal(t, ports.HintLink, kindOf(roleLink))
43+ for _, r := range []uint32{rolePushButton, roleToggleButton, roleCheckBox, roleRadioButton, roleMenuItem, rolePageTab, roleComboBox} {
44+ require.Equal(t, ports.HintButton, kindOf(r), r)
45+ require.True(t, isClickable(r, false), r)
46+ require.False(t, isClickable(r, true), r)
47+ }
48+ require.True(t, isClickable(roleLink, true))
49+ require.False(t, isClickable(roleFrame, false))
50+ require.True(t, isWindowRole(roleFrame) && isWindowRole(roleWindow) && isWindowRole(roleDialog))
51+ require.False(t, isWindowRole(roleLink))
52+}
53+
54+func TestRoleAndStateBits(t *testing.T) {
55+ require.Equal(t, []int32{0, 0, 0, 0, 0}, roleBits(nil))
56+ // link is role 88: word 2, bit 24; push button is 43: word 1, bit 11.
57+ require.Equal(t, []int32{0, 1 << 11, 1 << 24, 0, 0}, roleBits([]uint32{roleLink, rolePushButton}))
58+ // showing is state 25, visible 30: both in word 0.
59+ require.Equal(t, []int32{1<<25 | 1<<30, 0}, stateBits(stateShowing, stateVisible))
60+}
61+
62+func wins(ws ...ports.HintWindow) []ports.HintWindow { return ws }
63+
64+func win(id ports.WindowID, w, h int, focused bool) ports.HintWindow {
65+ return ports.HintWindow{ID: id, PID: 1, Rect: ports.Rect{W: w, H: h}, Focused: focused}
66+}
67+
68+func TestMatchFramesSingle(t *testing.T) {
69+ got := matchFrames(wins(win(1, 800, 600, false)), []frameInfo{{w: 100, h: 100}})
70+ require.Equal(t, map[int]int{0: 0}, got)
71+}
72+
73+func TestMatchFramesActive(t *testing.T) {
74+ frames := []frameInfo{{w: 800, h: 600}, {w: 800, h: 600, active: true}}
75+ got := matchFrames(wins(win(1, 800, 600, true), win(2, 800, 600, false)), frames)
76+ // The focused window takes the ACTIVE frame and reserves it; the other
77+ // window then only has the remaining frame to match.
78+ require.Equal(t, map[int]int{0: 1, 1: 0}, got)
79+}
80+
81+func TestMatchFramesSize(t *testing.T) {
82+ frames := []frameInfo{{w: 1000, h: 700}, {w: 400, h: 300}}
83+ got := matchFrames(wins(win(1, 1000, 700, false), win(2, 420, 290, false)), frames)
84+ require.Equal(t, map[int]int{0: 0, 1: 1}, got)
85+}
86+
87+func TestMatchFramesSizeTolerance(t *testing.T) {
88+ frames := []frameInfo{{w: 840, h: 640}, {w: 100, h: 100}}
89+ other := win(2, 90, 90, false)
90+ // 40 px off still matches; 41 px does not.
91+ require.Equal(t, map[int]int{0: 0, 1: 1}, matchFrames(wins(win(1, 800, 600, false), other), frames))
92+ require.Equal(t, map[int]int{1: 1}, matchFrames(wins(win(1, 799, 600, false), other), frames))
93+}
94+
95+func TestMatchFramesAmbiguous(t *testing.T) {
96+ // Two windows and two frames of the same size, none active: skipped.
97+ frames := []frameInfo{{w: 800, h: 600}, {w: 810, h: 600}}
98+ got := matchFrames(wins(win(1, 800, 600, false), win(2, 800, 600, false)), frames)
99+ require.Empty(t, got)
100+ // One window, two same-size frames, not focused: skipped.
101+ require.Empty(t, matchFrames(wins(win(1, 800, 600, false)), frames))
102+ // No frame close enough.
103+ require.Empty(t, matchFrames(wins(win(1, 200, 200, false), win(2, 300, 300, false)), frames))
104+}
105+
106+// The active frame is reserved by the focused window: an unfocused window of
107+// the same size as that frame must not be paired with it as well.
108+func TestMatchFramesOneToOne(t *testing.T) {
109+ frames := []frameInfo{{ref: ref{"a", "/active"}, active: true, w: 400, h: 300}, {ref: ref{"a", "/idle"}, w: 800, h: 600}}
110+ got := matchFrames(wins(win(1, 800, 600, true), win(2, 400, 300, false)), frames)
111+ seen := map[int]bool{}
112+ for _, f := range got {
113+ require.False(t, seen[f], "frame %d assigned twice: %v", f, got)
114+ seen[f] = true
115+ }
116+ require.Equal(t, map[int]int{0: 0}, got)
117+}
A internal/adapters/atspi/source.go +482 -0
@@ -0,0 +1,482 @@
1+package atspi
2+
3+import (
4+ "context"
5+ "errors"
6+ "fmt"
7+ "strings"
8+ "time"
9+
10+ "github.com/bnema/zerobus"
11+
12+ "github.com/bnema/neferwl/internal/adapters/logging"
13+ "github.com/bnema/neferwl/internal/ports"
14+)
15+
16+const (
17+ // callTimeout is how long a batch waits for the next reply.
18+ callTimeout = 100 * time.Millisecond
19+ // budget is the whole time Targets may take.
20+ budget = 300 * time.Millisecond
21+)
22+
23+// connectBudget bounds the whole connection setup (session bus,
24+// GetAddress, IsEnabled, dial of the accessibility bus) of one attempt.
25+const connectBudget = 300 * time.Millisecond
26+
27+// retryEvery is the minimum time between two connection attempts.
28+const retryEvery = 5 * time.Second
29+
30+// connector opens a transport on the accessibility bus.
31+type connector interface {
32+ Connect(ctx context.Context) (transport, error)
33+}
34+
35+// stallWarnAfter is how long an attempt may be in flight before it is
36+// reported as a stalled accessibility bus.
37+const stallWarnAfter = 5 * time.Second
38+
39+// dialResult is the outcome of a connection attempt.
40+type dialResult struct {
41+ tr transport
42+ err error
43+ at time.Time // when the attempt finished
44+}
45+
46+// Source reads hint targets from the accessibility tree. It owns one
47+// connection to the accessibility bus. It is NOT safe for concurrent use:
48+// it is owned by the hints worker goroutine, which calls Targets and,
49+// when it returns, Close. The connection is made on the first Targets call
50+// that needs it, never at startup, and made again after a failure, at most
51+// once every retryEvery; meanwhile Targets finds nothing and OCR takes over.
52+//
53+// At most one connection attempt (one goroutine and one socket) exists at
54+// any time: an attempt that outlives the wait of ensure stays in flight, is
55+// polled by the later Targets calls and is never doubled.
56+type Source struct {
57+ tr transport
58+ dial connector
59+ // attempt delivers the outcome of the attempt in flight (buffered, so
60+ // the attempt goroutine never blocks); nil when none runs.
61+ attempt chan dialResult
62+ attemptStarted time.Time
63+ warnedStall bool
64+ retryAt time.Time
65+ closed bool
66+ // wait is how long ensure waits for a new attempt; connectBudget if 0.
67+ wait time.Duration
68+}
69+
70+// New returns a Source. It does not touch the bus: the connection is made by
71+// the first Targets call.
72+func New() *Source { return &Source{dial: busConnector{}} }
73+
74+// busConnector connects to the session bus, enables accessibility and dials
75+// the accessibility bus.
76+type busConnector struct{}
77+
78+func (busConnector) Connect(ctx context.Context) (transport, error) {
79+ log := logging.For(ctx, "atspi")
80+ // zerobus has no deadline or context API: SessionBus and Dial run the
81+ // socket dial, SASL auth and Hello without a bound, and the watchdog
82+ // below cannot exist before the connection does. A stall there is
83+ // bounded by the single-in-flight rule of Source.ensure, not by a
84+ // deadline. Follow-up: a deadline-capable zerobus API.
85+ sess, err := zerobus.SessionBus()
86+ if err != nil {
87+ return nil, fmt.Errorf("atspi: session bus: %w", err)
88+ }
89+ defer func() { _ = sess.Close() }()
90+ // Call has no deadline: closing the connection ends the wait for a
91+ // silent service.
92+ stop := time.AfterFunc(connectBudget, func() { _ = sess.Close() })
93+ defer stop.Stop()
94+
95+ sess.NewCall(a11yBusName, a11yBusPath, a11yBusName, "GetAddress", "")
96+ m, err := sess.Call()
97+ if err != nil {
98+ return nil, fmt.Errorf("atspi: org.a11y.Bus GetAddress: %w", err)
99+ }
100+ r := m.Body()
101+ addr := r.Str()
102+ if r.Err() != nil || addr == "" {
103+ return nil, errors.New("atspi: no accessibility bus address")
104+ }
105+ addr = strings.Clone(addr)
106+
107+ e := sess.NewCall(a11yBusName, a11yBusPath, ifaceProps, "Set", "ssv")
108+ e.Str(a11yStatus)
109+ e.Str("IsEnabled")
110+ e.Variant("b")
111+ e.Bool(true)
112+ if _, err := sess.Call(); err != nil {
113+ // Toolkits that already see the bus keep working.
114+ log.Warn().Err(err).Msg("could not enable accessibility")
115+ } else {
116+ log.Info().Msg("accessibility enabled")
117+ }
118+
119+ zc, err := zerobus.Dial(addr)
120+ if err != nil {
121+ return nil, fmt.Errorf("atspi: accessibility bus: %w", err)
122+ }
123+ tr, err := newConn(zc)
124+ if err != nil {
125+ return nil, err
126+ }
127+ log.Info().Msg("accessibility bus connected")
128+ return tr, nil
129+}
130+
131+// ensure returns the connection, making one if none exists and the last
132+// attempt is old enough. It waits at most the budget for a new attempt; an
133+// attempt that does not answer in time stays in flight (abandoned, never
134+// doubled) and is polled without blocking by the next calls.
135+func (s *Source) ensure(ctx context.Context) transport {
136+ if s.closed {
137+ return nil
138+ }
139+ if s.tr != nil {
140+ return s.tr
141+ }
142+ if s.dial == nil {
143+ return nil
144+ }
145+ now := time.Now()
146+ if s.attempt != nil {
147+ select {
148+ case r := <-s.attempt:
149+ s.finish(ctx, r)
150+ default:
151+ if !s.warnedStall && now.Sub(s.attemptStarted) > stallWarnAfter {
152+ s.warnedStall = true
153+ log := logging.For(ctx, "atspi")
154+ log.Warn().Dur("for", now.Sub(s.attemptStarted)).Msg("accessibility bus not answering")
155+ }
156+ }
157+ return s.tr
158+ }
159+ if now.Before(s.retryAt) {
160+ return nil
161+ }
162+ ch := make(chan dialResult, 1)
163+ s.attempt = ch
164+ s.attemptStarted = now
165+ s.warnedStall = false
166+ dial := s.dial
167+ go func() {
168+ tr, err := dial.Connect(ctx)
169+ ch <- dialResult{tr: tr, err: err, at: time.Now()}
170+ }()
171+ wait := s.wait
172+ if wait <= 0 {
173+ wait = connectBudget
174+ }
175+ timer := time.NewTimer(wait)
176+ defer timer.Stop()
177+ select {
178+ case r := <-ch:
179+ s.finish(ctx, r)
180+ case <-timer.C:
181+ log := logging.For(ctx, "atspi")
182+ log.Debug().Dur("after", wait).Msg("accessibility connect attempt abandoned in flight")
183+ case <-ctx.Done():
184+ }
185+ return s.tr
186+}
187+
188+// finish takes the outcome of the attempt in flight: a transport is kept,
189+// an error starts the backoff from the time the attempt finished.
190+func (s *Source) finish(ctx context.Context, r dialResult) {
191+ s.attempt = nil
192+ if r.err != nil || r.tr == nil {
193+ s.retryAt = r.at.Add(retryEvery)
194+ err := r.err
195+ if err == nil {
196+ err = errors.New("atspi: no transport")
197+ }
198+ log := logging.For(ctx, "atspi")
199+ log.Warn().Err(err).Msg("accessibility hint targets unavailable, using OCR only")
200+ return
201+ }
202+ s.tr = r.tr
203+}
204+
205+// drop closes a broken connection; the next attempt reconnects.
206+func (s *Source) drop() {
207+ s.retryAt = time.Now().Add(retryEvery)
208+ if s.tr != nil {
209+ _ = s.tr.Close()
210+ s.tr = nil
211+ }
212+}
213+
214+// Close closes the accessibility bus connection, and the one an attempt
215+// still in flight may deliver late. After Close, Targets finds nothing and
216+// never connects. Like Targets it is called by the owner goroutine only.
217+func (s *Source) Close() error {
218+ if s.closed {
219+ return nil
220+ }
221+ s.closed = true
222+ var err error
223+ if s.tr != nil {
224+ err = s.tr.Close()
225+ s.tr = nil
226+ }
227+ if ch := s.attempt; ch != nil {
228+ s.attempt = nil
229+ // One reaper at most: Close runs once, and one attempt exists.
230+ go func() {
231+ if r := <-ch; r.tr != nil {
232+ _ = r.tr.Close()
233+ }
234+ }()
235+ }
236+ return err
237+}
238+
239+// appInfo is an application of the registry whose process shows a window.
240+type appInfo struct {
241+ root ref
242+ pid int
243+}
244+
245+// winMatch is a window with the frame it was paired with.
246+type winMatch struct {
247+ win ports.HintWindow
248+ frame ref
249+ fx int32
250+ fy int32
251+}
252+
253+// Targets returns the clickable targets of the accessibility tree for
254+// windows (logical output-local, clipped to each window) and the set of
255+// windows those targets cover. A window is covered when its tree answered
256+// with a link target or web document; uncovered windows also get OCR. It
257+// returns what is ready within the budget.
258+func (s *Source) Targets(ctx context.Context, windows []ports.HintWindow, linksOnly bool) ([]ports.HintTarget, map[ports.WindowID]bool, error) {
259+ covered := map[ports.WindowID]bool{}
260+ deadline := time.Now().Add(budget)
261+ if d, ok := ctx.Deadline(); ok && d.Before(deadline) {
262+ deadline = d
263+ }
264+ wanted := map[int]bool{}
265+ for _, w := range windows {
266+ if w.PID > 0 {
267+ wanted[w.PID] = true
268+ }
269+ }
270+ if len(wanted) == 0 {
271+ return nil, covered, nil
272+ }
273+ tr := s.ensure(ctx)
274+ if tr == nil {
275+ return nil, covered, nil
276+ }
277+ broken := false
278+ do := func(calls []call) []reply {
279+ if len(calls) == 0 || ctx.Err() != nil {
280+ return make([]reply, len(calls))
281+ }
282+ if broken {
283+ out := make([]reply, len(calls))
284+ for i := range out {
285+ out[i].err = ErrBroken
286+ }
287+ return out
288+ }
289+ out := tr.Do(deadline, callTimeout, calls)
290+ for _, r := range out {
291+ if errors.Is(r.err, ErrBroken) {
292+ broken = true
293+ s.drop()
294+ break
295+ }
296+ }
297+ return out
298+ }
299+
300+ // Applications of the registry, kept when their process shows a window.
301+ rep := do([]call{{op: opChildren, name: registryName, path: registryRoot}})
302+ if len(rep) == 0 || rep[0].err != nil {
303+ if len(rep) > 0 {
304+ return nil, covered, fmt.Errorf("atspi: registry: %w", rep[0].err)
305+ }
306+ return nil, covered, nil
307+ }
308+ apps := rep[0].refs
309+ calls := make([]call, len(apps))
310+ for i, a := range apps {
311+ calls[i] = call{op: opPID, name: a.name}
312+ }
313+ var kept []appInfo
314+ for i, r := range do(calls) {
315+ if r.err == nil && wanted[int(r.u)] {
316+ kept = append(kept, appInfo{root: apps[i], pid: int(r.u)})
317+ }
318+ }
319+
320+ // The showing windows of each kept app.
321+ calls = calls[:0]
322+ for _, a := range kept {
323+ calls = append(calls, call{op: opChildren, name: a.root.name, path: a.root.path})
324+ }
325+ var cands []ref
326+ var candApp []int
327+ for i, r := range do(calls) {
328+ if r.err != nil {
329+ continue
330+ }
331+ for _, c := range r.refs[:min(len(r.refs), maxFrames)] {
332+ cands = append(cands, c)
333+ candApp = append(candApp, i)
334+ }
335+ }
336+ calls = calls[:0]
337+ for _, c := range cands {
338+ calls = append(calls,
339+ call{op: opRole, name: c.name, path: c.path},
340+ call{op: opState, name: c.name, path: c.path},
341+ call{op: opExtents, name: c.name, path: c.path, arg: coordWindow})
342+ }
343+ info := do(calls)
344+ type frame struct {
345+ frameInfo
346+ pid int
347+ fx, fy int32
348+ }
349+ var frames []frame
350+ for i, c := range cands {
351+ role, state, ext := info[3*i], info[3*i+1], info[3*i+2]
352+ if role.err != nil || state.err != nil || ext.err != nil || !isWindowRole(role.u) || !hasState(state.state, stateShowing) {
353+ continue
354+ }
355+ frames = append(frames, frame{
356+ frameInfo: frameInfo{ref: c, active: hasState(state.state, stateActive), w: int(ext.rect[2]), h: int(ext.rect[3])},
357+ pid: kept[candApp[i]].pid, fx: ext.rect[0], fy: ext.rect[1],
358+ })
359+ }
360+
361+ // Pair the windows with the frames, process by process.
362+ var matched []winMatch
363+ for pid := range wanted {
364+ var wins []ports.HintWindow
365+ for _, w := range windows {
366+ if w.PID == pid {
367+ wins = append(wins, w)
368+ }
369+ }
370+ var fs []frameInfo
371+ var origin []frame
372+ for _, f := range frames {
373+ if f.pid == pid {
374+ fs = append(fs, f.frameInfo)
375+ origin = append(origin, f)
376+ }
377+ }
378+ for wi, fi := range matchFrames(wins, fs) {
379+ matched = append(matched, winMatch{win: wins[wi], frame: origin[fi].ref, fx: origin[fi].fx, fy: origin[fi].fy})
380+ }
381+ }
382+
383+ targets := s.elements(do, matched, linksOnly, covered)
384+ log := logging.For(ctx, "atspi")
385+ log.Debug().Int("apps", len(kept)).Int("frames", len(frames)).Int("windows", len(matched)).Int("targets", len(targets)).Msg("atspi targets")
386+ return targets, covered, nil
387+}
388+
389+// elements reads the clickable elements of the matched frames.
390+func (s *Source) elements(do func([]call) []reply, matched []winMatch, linksOnly bool, covered map[ports.WindowID]bool) []ports.HintTarget {
391+ calls := make([]call, len(matched))
392+ for i, m := range matched {
393+ calls[i] = call{op: opMatches, name: m.frame.name, path: m.frame.path, rule: matchRule{linksOnly: linksOnly}}
394+ }
395+ found := make([][]ref, len(matched))
396+ var walk []int // frames without Collection
397+ for i, r := range do(calls) {
398+ if r.err != nil {
399+ var be *busError
400+ if errors.As(r.err, &be) {
401+ walk = append(walk, i)
402+ }
403+ continue
404+ }
405+ found[i] = r.refs
406+ }
407+ for _, i := range walk {
408+ found[i] = s.walk(do, matched[i].frame, linksOnly)
409+ }
410+
411+ calls = calls[:0]
412+ var owner []int
413+ for i, refs := range found {
414+ for _, r := range refs {
415+ calls = append(calls,
416+ call{op: opRole, name: r.name, path: r.path},
417+ call{op: opExtents, name: r.name, path: r.path, arg: coordWindow})
418+ owner = append(owner, i)
419+ }
420+ }
421+ info := do(calls)
422+ var out []ports.HintTarget
423+ for k, i := range owner {
424+ role, ext := info[2*k], info[2*k+1]
425+ if role.err == nil && isDocument(role.u) {
426+ covered[matched[i].win.ID] = true
427+ continue
428+ }
429+ if role.err != nil || ext.err != nil || !isClickable(role.u, linksOnly) {
430+ continue
431+ }
432+ m := matched[i]
433+ t, ok := toTarget(m.win.Rect, m.fx, m.fy, element{role: role.u, x: ext.rect[0], y: ext.rect[1], w: ext.rect[2], h: ext.rect[3]})
434+ if !ok {
435+ continue
436+ }
437+ t.Window = m.win.ID
438+ out = append(out, t)
439+ if t.Kind == ports.HintLink {
440+ covered[m.win.ID] = true
441+ }
442+ }
443+ return out
444+}
445+
446+// walk is the fallback for toolkits without Collection: a breadth-first
447+// walk of at most maxDepth levels and maxWalkNodes nodes, returning the
448+// clickable nodes.
449+func (s *Source) walk(do func([]call) []reply, root ref, linksOnly bool) []ref {
450+ var out []ref
451+ level := []ref{root}
452+ seen := 0
453+ for depth := 0; depth < maxDepth && len(level) > 0 && seen < maxWalkNodes; depth++ {
454+ calls := make([]call, 0, 3*len(level))
455+ for _, n := range level {
456+ calls = append(calls,
457+ call{op: opChildren, name: n.name, path: n.path},
458+ call{op: opRole, name: n.name, path: n.path},
459+ call{op: opState, name: n.name, path: n.path})
460+ }
461+ rep := do(calls)
462+ var next []ref
463+ for i, n := range level {
464+ ch, role, state := rep[3*i], rep[3*i+1], rep[3*i+2]
465+ if state.err == nil && !hasState(state.state, stateShowing) {
466+ continue
467+ }
468+ if role.err == nil && (isClickable(role.u, linksOnly) || isDocument(role.u)) && state.err == nil && hasState(state.state, stateVisible) {
469+ out = append(out, n)
470+ }
471+ if ch.err == nil {
472+ next = append(next, ch.refs...)
473+ }
474+ }
475+ seen += len(level)
476+ if room := maxWalkNodes - seen; len(next) > room {
477+ next = next[:max(room, 0)]
478+ }
479+ level = next
480+ }
481+ return out
482+}
@@ -0,0 +1,482 @@
1+package atspi
2+
3+import (
4+ "context"
5+ "errors"
6+ "fmt"
7+ "strings"
8+ "time"
9+
10+ "github.com/bnema/zerobus"
11+
12+ "github.com/bnema/neferwl/internal/adapters/logging"
13+ "github.com/bnema/neferwl/internal/ports"
14+)
15+
16+const (
17+ // callTimeout is how long a batch waits for the next reply.
18+ callTimeout = 100 * time.Millisecond
19+ // budget is the whole time Targets may take.
20+ budget = 300 * time.Millisecond
21+)
22+
23+// connectBudget bounds the whole connection setup (session bus,
24+// GetAddress, IsEnabled, dial of the accessibility bus) of one attempt.
25+const connectBudget = 300 * time.Millisecond
26+
27+// retryEvery is the minimum time between two connection attempts.
28+const retryEvery = 5 * time.Second
29+
30+// connector opens a transport on the accessibility bus.
31+type connector interface {
32+ Connect(ctx context.Context) (transport, error)
33+}
34+
35+// stallWarnAfter is how long an attempt may be in flight before it is
36+// reported as a stalled accessibility bus.
37+const stallWarnAfter = 5 * time.Second
38+
39+// dialResult is the outcome of a connection attempt.
40+type dialResult struct {
41+ tr transport
42+ err error
43+ at time.Time // when the attempt finished
44+}
45+
46+// Source reads hint targets from the accessibility tree. It owns one
47+// connection to the accessibility bus. It is NOT safe for concurrent use:
48+// it is owned by the hints worker goroutine, which calls Targets and,
49+// when it returns, Close. The connection is made on the first Targets call
50+// that needs it, never at startup, and made again after a failure, at most
51+// once every retryEvery; meanwhile Targets finds nothing and OCR takes over.
52+//
53+// At most one connection attempt (one goroutine and one socket) exists at
54+// any time: an attempt that outlives the wait of ensure stays in flight, is
55+// polled by the later Targets calls and is never doubled.
56+type Source struct {
57+ tr transport
58+ dial connector
59+ // attempt delivers the outcome of the attempt in flight (buffered, so
60+ // the attempt goroutine never blocks); nil when none runs.
61+ attempt chan dialResult
62+ attemptStarted time.Time
63+ warnedStall bool
64+ retryAt time.Time
65+ closed bool
66+ // wait is how long ensure waits for a new attempt; connectBudget if 0.
67+ wait time.Duration
68+}
69+
70+// New returns a Source. It does not touch the bus: the connection is made by
71+// the first Targets call.
72+func New() *Source { return &Source{dial: busConnector{}} }
73+
74+// busConnector connects to the session bus, enables accessibility and dials
75+// the accessibility bus.
76+type busConnector struct{}
77+
78+func (busConnector) Connect(ctx context.Context) (transport, error) {
79+ log := logging.For(ctx, "atspi")
80+ // zerobus has no deadline or context API: SessionBus and Dial run the
81+ // socket dial, SASL auth and Hello without a bound, and the watchdog
82+ // below cannot exist before the connection does. A stall there is
83+ // bounded by the single-in-flight rule of Source.ensure, not by a
84+ // deadline. Follow-up: a deadline-capable zerobus API.
85+ sess, err := zerobus.SessionBus()
86+ if err != nil {
87+ return nil, fmt.Errorf("atspi: session bus: %w", err)
88+ }
89+ defer func() { _ = sess.Close() }()
90+ // Call has no deadline: closing the connection ends the wait for a
91+ // silent service.
92+ stop := time.AfterFunc(connectBudget, func() { _ = sess.Close() })
93+ defer stop.Stop()
94+
95+ sess.NewCall(a11yBusName, a11yBusPath, a11yBusName, "GetAddress", "")
96+ m, err := sess.Call()
97+ if err != nil {
98+ return nil, fmt.Errorf("atspi: org.a11y.Bus GetAddress: %w", err)
99+ }
100+ r := m.Body()
101+ addr := r.Str()
102+ if r.Err() != nil || addr == "" {
103+ return nil, errors.New("atspi: no accessibility bus address")
104+ }
105+ addr = strings.Clone(addr)
106+
107+ e := sess.NewCall(a11yBusName, a11yBusPath, ifaceProps, "Set", "ssv")
108+ e.Str(a11yStatus)
109+ e.Str("IsEnabled")
110+ e.Variant("b")
111+ e.Bool(true)
112+ if _, err := sess.Call(); err != nil {
113+ // Toolkits that already see the bus keep working.
114+ log.Warn().Err(err).Msg("could not enable accessibility")
115+ } else {
116+ log.Info().Msg("accessibility enabled")
117+ }
118+
119+ zc, err := zerobus.Dial(addr)
120+ if err != nil {
121+ return nil, fmt.Errorf("atspi: accessibility bus: %w", err)
122+ }
123+ tr, err := newConn(zc)
124+ if err != nil {
125+ return nil, err
126+ }
127+ log.Info().Msg("accessibility bus connected")
128+ return tr, nil
129+}
130+
131+// ensure returns the connection, making one if none exists and the last
132+// attempt is old enough. It waits at most the budget for a new attempt; an
133+// attempt that does not answer in time stays in flight (abandoned, never
134+// doubled) and is polled without blocking by the next calls.
135+func (s *Source) ensure(ctx context.Context) transport {
136+ if s.closed {
137+ return nil
138+ }
139+ if s.tr != nil {
140+ return s.tr
141+ }
142+ if s.dial == nil {
143+ return nil
144+ }
145+ now := time.Now()
146+ if s.attempt != nil {
147+ select {
148+ case r := <-s.attempt:
149+ s.finish(ctx, r)
150+ default:
151+ if !s.warnedStall && now.Sub(s.attemptStarted) > stallWarnAfter {
152+ s.warnedStall = true
153+ log := logging.For(ctx, "atspi")
154+ log.Warn().Dur("for", now.Sub(s.attemptStarted)).Msg("accessibility bus not answering")
155+ }
156+ }
157+ return s.tr
158+ }
159+ if now.Before(s.retryAt) {
160+ return nil
161+ }
162+ ch := make(chan dialResult, 1)
163+ s.attempt = ch
164+ s.attemptStarted = now
165+ s.warnedStall = false
166+ dial := s.dial
167+ go func() {
168+ tr, err := dial.Connect(ctx)
169+ ch <- dialResult{tr: tr, err: err, at: time.Now()}
170+ }()
171+ wait := s.wait
172+ if wait <= 0 {
173+ wait = connectBudget
174+ }
175+ timer := time.NewTimer(wait)
176+ defer timer.Stop()
177+ select {
178+ case r := <-ch:
179+ s.finish(ctx, r)
180+ case <-timer.C:
181+ log := logging.For(ctx, "atspi")
182+ log.Debug().Dur("after", wait).Msg("accessibility connect attempt abandoned in flight")
183+ case <-ctx.Done():
184+ }
185+ return s.tr
186+}
187+
188+// finish takes the outcome of the attempt in flight: a transport is kept,
189+// an error starts the backoff from the time the attempt finished.
190+func (s *Source) finish(ctx context.Context, r dialResult) {
191+ s.attempt = nil
192+ if r.err != nil || r.tr == nil {
193+ s.retryAt = r.at.Add(retryEvery)
194+ err := r.err
195+ if err == nil {
196+ err = errors.New("atspi: no transport")
197+ }
198+ log := logging.For(ctx, "atspi")
199+ log.Warn().Err(err).Msg("accessibility hint targets unavailable, using OCR only")
200+ return
201+ }
202+ s.tr = r.tr
203+}
204+
205+// drop closes a broken connection; the next attempt reconnects.
206+func (s *Source) drop() {
207+ s.retryAt = time.Now().Add(retryEvery)
208+ if s.tr != nil {
209+ _ = s.tr.Close()
210+ s.tr = nil
211+ }
212+}
213+
214+// Close closes the accessibility bus connection, and the one an attempt
215+// still in flight may deliver late. After Close, Targets finds nothing and
216+// never connects. Like Targets it is called by the owner goroutine only.
217+func (s *Source) Close() error {
218+ if s.closed {
219+ return nil
220+ }
221+ s.closed = true
222+ var err error
223+ if s.tr != nil {
224+ err = s.tr.Close()
225+ s.tr = nil
226+ }
227+ if ch := s.attempt; ch != nil {
228+ s.attempt = nil
229+ // One reaper at most: Close runs once, and one attempt exists.
230+ go func() {
231+ if r := <-ch; r.tr != nil {
232+ _ = r.tr.Close()
233+ }
234+ }()
235+ }
236+ return err
237+}
238+
239+// appInfo is an application of the registry whose process shows a window.
240+type appInfo struct {
241+ root ref
242+ pid int
243+}
244+
245+// winMatch is a window with the frame it was paired with.
246+type winMatch struct {
247+ win ports.HintWindow
248+ frame ref
249+ fx int32
250+ fy int32
251+}
252+
253+// Targets returns the clickable targets of the accessibility tree for
254+// windows (logical output-local, clipped to each window) and the set of
255+// windows those targets cover. A window is covered when its tree answered
256+// with a link target or web document; uncovered windows also get OCR. It
257+// returns what is ready within the budget.
258+func (s *Source) Targets(ctx context.Context, windows []ports.HintWindow, linksOnly bool) ([]ports.HintTarget, map[ports.WindowID]bool, error) {
259+ covered := map[ports.WindowID]bool{}
260+ deadline := time.Now().Add(budget)
261+ if d, ok := ctx.Deadline(); ok && d.Before(deadline) {
262+ deadline = d
263+ }
264+ wanted := map[int]bool{}
265+ for _, w := range windows {
266+ if w.PID > 0 {
267+ wanted[w.PID] = true
268+ }
269+ }
270+ if len(wanted) == 0 {
271+ return nil, covered, nil
272+ }
273+ tr := s.ensure(ctx)
274+ if tr == nil {
275+ return nil, covered, nil
276+ }
277+ broken := false
278+ do := func(calls []call) []reply {
279+ if len(calls) == 0 || ctx.Err() != nil {
280+ return make([]reply, len(calls))
281+ }
282+ if broken {
283+ out := make([]reply, len(calls))
284+ for i := range out {
285+ out[i].err = ErrBroken
286+ }
287+ return out
288+ }
289+ out := tr.Do(deadline, callTimeout, calls)
290+ for _, r := range out {
291+ if errors.Is(r.err, ErrBroken) {
292+ broken = true
293+ s.drop()
294+ break
295+ }
296+ }
297+ return out
298+ }
299+
300+ // Applications of the registry, kept when their process shows a window.
301+ rep := do([]call{{op: opChildren, name: registryName, path: registryRoot}})
302+ if len(rep) == 0 || rep[0].err != nil {
303+ if len(rep) > 0 {
304+ return nil, covered, fmt.Errorf("atspi: registry: %w", rep[0].err)
305+ }
306+ return nil, covered, nil
307+ }
308+ apps := rep[0].refs
309+ calls := make([]call, len(apps))
310+ for i, a := range apps {
311+ calls[i] = call{op: opPID, name: a.name}
312+ }
313+ var kept []appInfo
314+ for i, r := range do(calls) {
315+ if r.err == nil && wanted[int(r.u)] {
316+ kept = append(kept, appInfo{root: apps[i], pid: int(r.u)})
317+ }
318+ }
319+
320+ // The showing windows of each kept app.
321+ calls = calls[:0]
322+ for _, a := range kept {
323+ calls = append(calls, call{op: opChildren, name: a.root.name, path: a.root.path})
324+ }
325+ var cands []ref
326+ var candApp []int
327+ for i, r := range do(calls) {
328+ if r.err != nil {
329+ continue
330+ }
331+ for _, c := range r.refs[:min(len(r.refs), maxFrames)] {
332+ cands = append(cands, c)
333+ candApp = append(candApp, i)
334+ }
335+ }
336+ calls = calls[:0]
337+ for _, c := range cands {
338+ calls = append(calls,
339+ call{op: opRole, name: c.name, path: c.path},
340+ call{op: opState, name: c.name, path: c.path},
341+ call{op: opExtents, name: c.name, path: c.path, arg: coordWindow})
342+ }
343+ info := do(calls)
344+ type frame struct {
345+ frameInfo
346+ pid int
347+ fx, fy int32
348+ }
349+ var frames []frame
350+ for i, c := range cands {
351+ role, state, ext := info[3*i], info[3*i+1], info[3*i+2]
352+ if role.err != nil || state.err != nil || ext.err != nil || !isWindowRole(role.u) || !hasState(state.state, stateShowing) {
353+ continue
354+ }
355+ frames = append(frames, frame{
356+ frameInfo: frameInfo{ref: c, active: hasState(state.state, stateActive), w: int(ext.rect[2]), h: int(ext.rect[3])},
357+ pid: kept[candApp[i]].pid, fx: ext.rect[0], fy: ext.rect[1],
358+ })
359+ }
360+
361+ // Pair the windows with the frames, process by process.
362+ var matched []winMatch
363+ for pid := range wanted {
364+ var wins []ports.HintWindow
365+ for _, w := range windows {
366+ if w.PID == pid {
367+ wins = append(wins, w)
368+ }
369+ }
370+ var fs []frameInfo
371+ var origin []frame
372+ for _, f := range frames {
373+ if f.pid == pid {
374+ fs = append(fs, f.frameInfo)
375+ origin = append(origin, f)
376+ }
377+ }
378+ for wi, fi := range matchFrames(wins, fs) {
379+ matched = append(matched, winMatch{win: wins[wi], frame: origin[fi].ref, fx: origin[fi].fx, fy: origin[fi].fy})
380+ }
381+ }
382+
383+ targets := s.elements(do, matched, linksOnly, covered)
384+ log := logging.For(ctx, "atspi")
385+ log.Debug().Int("apps", len(kept)).Int("frames", len(frames)).Int("windows", len(matched)).Int("targets", len(targets)).Msg("atspi targets")
386+ return targets, covered, nil
387+}
388+
389+// elements reads the clickable elements of the matched frames.
390+func (s *Source) elements(do func([]call) []reply, matched []winMatch, linksOnly bool, covered map[ports.WindowID]bool) []ports.HintTarget {
391+ calls := make([]call, len(matched))
392+ for i, m := range matched {
393+ calls[i] = call{op: opMatches, name: m.frame.name, path: m.frame.path, rule: matchRule{linksOnly: linksOnly}}
394+ }
395+ found := make([][]ref, len(matched))
396+ var walk []int // frames without Collection
397+ for i, r := range do(calls) {
398+ if r.err != nil {
399+ var be *busError
400+ if errors.As(r.err, &be) {
401+ walk = append(walk, i)
402+ }
403+ continue
404+ }
405+ found[i] = r.refs
406+ }
407+ for _, i := range walk {
408+ found[i] = s.walk(do, matched[i].frame, linksOnly)
409+ }
410+
411+ calls = calls[:0]
412+ var owner []int
413+ for i, refs := range found {
414+ for _, r := range refs {
415+ calls = append(calls,
416+ call{op: opRole, name: r.name, path: r.path},
417+ call{op: opExtents, name: r.name, path: r.path, arg: coordWindow})
418+ owner = append(owner, i)
419+ }
420+ }
421+ info := do(calls)
422+ var out []ports.HintTarget
423+ for k, i := range owner {
424+ role, ext := info[2*k], info[2*k+1]
425+ if role.err == nil && isDocument(role.u) {
426+ covered[matched[i].win.ID] = true
427+ continue
428+ }
429+ if role.err != nil || ext.err != nil || !isClickable(role.u, linksOnly) {
430+ continue
431+ }
432+ m := matched[i]
433+ t, ok := toTarget(m.win.Rect, m.fx, m.fy, element{role: role.u, x: ext.rect[0], y: ext.rect[1], w: ext.rect[2], h: ext.rect[3]})
434+ if !ok {
435+ continue
436+ }
437+ t.Window = m.win.ID
438+ out = append(out, t)
439+ if t.Kind == ports.HintLink {
440+ covered[m.win.ID] = true
441+ }
442+ }
443+ return out
444+}
445+
446+// walk is the fallback for toolkits without Collection: a breadth-first
447+// walk of at most maxDepth levels and maxWalkNodes nodes, returning the
448+// clickable nodes.
449+func (s *Source) walk(do func([]call) []reply, root ref, linksOnly bool) []ref {
450+ var out []ref
451+ level := []ref{root}
452+ seen := 0
453+ for depth := 0; depth < maxDepth && len(level) > 0 && seen < maxWalkNodes; depth++ {
454+ calls := make([]call, 0, 3*len(level))
455+ for _, n := range level {
456+ calls = append(calls,
457+ call{op: opChildren, name: n.name, path: n.path},
458+ call{op: opRole, name: n.name, path: n.path},
459+ call{op: opState, name: n.name, path: n.path})
460+ }
461+ rep := do(calls)
462+ var next []ref
463+ for i, n := range level {
464+ ch, role, state := rep[3*i], rep[3*i+1], rep[3*i+2]
465+ if state.err == nil && !hasState(state.state, stateShowing) {
466+ continue
467+ }
468+ if role.err == nil && (isClickable(role.u, linksOnly) || isDocument(role.u)) && state.err == nil && hasState(state.state, stateVisible) {
469+ out = append(out, n)
470+ }
471+ if ch.err == nil {
472+ next = append(next, ch.refs...)
473+ }
474+ }
475+ seen += len(level)
476+ if room := maxWalkNodes - seen; len(next) > room {
477+ next = next[:max(room, 0)]
478+ }
479+ level = next
480+ }
481+ return out
482+}
A internal/adapters/atspi/source_test.go +315 -0
@@ -0,0 +1,315 @@
1+package atspi
2+
3+import (
4+ "context"
5+ "errors"
6+ "fmt"
7+ "testing"
8+ "time"
9+
10+ "github.com/stretchr/testify/mock"
11+ "github.com/stretchr/testify/require"
12+
13+ "github.com/bnema/neferwl/internal/ports"
14+)
15+
16+const (
17+ stActive = 1 << stateActive
18+ stShowing = 1 << stateShowing
19+)
20+
21+// tree answers calls like an app: registry with one app (:1.5, pid 77) with
22+// one showing frame whose links sit at window-relative extents.
23+func answerTree(frameExt [4]int32, matches []ref, ext map[string][4]int32, roles map[string]uint32, noCollection bool) func(time.Time, time.Duration, []call) []reply {
24+ return func(_ time.Time, _ time.Duration, calls []call) []reply {
25+ out := make([]reply, len(calls))
26+ for i, c := range calls {
27+ switch {
28+ case c.op == opChildren && c.name == registryName:
29+ out[i].refs = []ref{{":1.5", "/root"}, {":1.9", "/root"}}
30+ case c.op == opPID:
31+ out[i].u = map[string]uint32{":1.5": 77, ":1.9": 99}[c.name]
32+ case c.op == opChildren && c.path == "/root":
33+ out[i].refs = []ref{{":1.5", "/frame"}}
34+ case c.op == opRole && c.path == "/frame":
35+ out[i].u = roleFrame
36+ case c.op == opState && c.path == "/frame":
37+ out[i].state = stShowing | stActive
38+ case c.op == opExtents && c.path == "/frame":
39+ out[i].rect = frameExt
40+ case c.op == opMatches:
41+ if noCollection {
42+ out[i].err = &busError{name: "org.freedesktop.DBus.Error.UnknownMethod"}
43+ } else {
44+ out[i].refs = matches
45+ }
46+ case c.op == opChildren && c.path == "/frame":
47+ out[i].refs = matches
48+ case c.op == opChildren:
49+ case c.op == opRole:
50+ out[i].u = roles[c.path]
51+ case c.op == opState:
52+ out[i].state = stShowing | 1<<stateVisible
53+ case c.op == opExtents:
54+ out[i].rect = ext[c.path]
55+ }
56+ }
57+ return out
58+ }
59+}
60+
61+func TestTargetsCoversWindowAndConverts(t *testing.T) {
62+ tr := newMocktransport(t)
63+ matches := []ref{{":1.5", "/a"}, {":1.5", "/b"}, {":1.5", "/zero"}}
64+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
65+ [4]int32{4, 30, 800, 600}, matches,
66+ map[string][4]int32{"/a": {14, 130, 50, 16}, "/b": {104, 230, 20, 20}, "/zero": {5, 5, 0, 0}},
67+ map[string]uint32{"/a": roleLink, "/b": rolePushButton, "/zero": roleLink}, false))
68+ s := &Source{tr: tr}
69+
70+ windows := []ports.HintWindow{
71+ {ID: 1, PID: 77, Rect: ports.Rect{X: 100, Y: 200, W: 800, H: 600}, Focused: true},
72+ {ID: 2, PID: 5, Rect: ports.Rect{X: 0, Y: 0, W: 50, H: 50}},
73+ }
74+ targets, covered, err := s.Targets(context.Background(), windows, false)
75+ require.NoError(t, err)
76+ require.Equal(t, map[ports.WindowID]bool{1: true}, covered)
77+ require.Equal(t, []ports.HintTarget{
78+ {Rect: ports.Rect{X: 110, Y: 300, W: 50, H: 16}, Kind: ports.HintLink, Window: 1},
79+ {Rect: ports.Rect{X: 200, Y: 400, W: 20, H: 20}, Kind: ports.HintButton, Window: 1},
80+ }, targets)
81+}
82+
83+func TestTargetsLinksOnlyDropsButtons(t *testing.T) {
84+ tr := newMocktransport(t)
85+ matches := []ref{{":1.5", "/a"}, {":1.5", "/b"}}
86+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
87+ [4]int32{0, 0, 800, 600}, matches,
88+ map[string][4]int32{"/a": {14, 130, 50, 16}, "/b": {104, 230, 20, 20}},
89+ map[string]uint32{"/a": roleLink, "/b": rolePushButton}, false))
90+ s := &Source{tr: tr}
91+ targets, _, err := s.Targets(context.Background(), []ports.HintWindow{{ID: 1, PID: 77, Rect: ports.Rect{W: 800, H: 600}}}, true)
92+ require.NoError(t, err)
93+ require.Len(t, targets, 1)
94+ require.Equal(t, ports.HintLink, targets[0].Kind)
95+}
96+
97+func TestTargetsWalkWithoutCollection(t *testing.T) {
98+ tr := newMocktransport(t)
99+ matches := []ref{{":1.5", "/a"}}
100+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
101+ [4]int32{0, 0, 800, 600}, matches,
102+ map[string][4]int32{"/a": {14, 130, 50, 16}},
103+ map[string]uint32{"/a": roleLink}, true))
104+ s := &Source{tr: tr}
105+ targets, covered, err := s.Targets(context.Background(), []ports.HintWindow{{ID: 1, PID: 77, Rect: ports.Rect{W: 800, H: 600}}}, false)
106+ require.NoError(t, err)
107+ require.Len(t, targets, 1)
108+ require.True(t, covered[1])
109+}
110+
111+func TestTargetsNoPIDMatchCoversNothing(t *testing.T) {
112+ tr := newMocktransport(t)
113+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree([4]int32{}, nil, nil, nil, false))
114+ s := &Source{tr: tr}
115+ targets, covered, err := s.Targets(context.Background(), []ports.HintWindow{{ID: 1, PID: 1234, Rect: ports.Rect{W: 8, H: 6}}}, false)
116+ require.NoError(t, err)
117+ require.Empty(t, targets)
118+ require.Empty(t, covered)
119+}
120+
121+func TestTargetsWithoutPIDsAsksNothing(t *testing.T) {
122+ s := &Source{tr: newMocktransport(t)}
123+ _, covered, err := s.Targets(context.Background(), []ports.HintWindow{{ID: 1, Rect: ports.Rect{W: 8, H: 6}}}, false)
124+ require.NoError(t, err)
125+ require.Empty(t, covered)
126+}
127+
128+func TestCoverageRequiresLinkOrDocument(t *testing.T) {
129+ for _, collection := range []bool{true, false} {
130+ for _, role := range []uint32{rolePushButton, roleDocumentFrame, roleDocumentWeb, roleLink} {
131+ t.Run(fmt.Sprintf("collection=%t/role=%d", collection, role), func(t *testing.T) {
132+ tr := newMocktransport(t)
133+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
134+ [4]int32{0, 0, 800, 600}, []ref{{":1.5", "/a"}}, map[string][4]int32{"/a": {10, 20, 30, 40}}, map[string]uint32{"/a": role}, !collection))
135+ s := &Source{tr: tr}
136+ targets, covered, err := s.Targets(context.Background(), []ports.HintWindow{{ID: 1, PID: 77, Rect: ports.Rect{W: 800, H: 600}}}, false)
137+ require.NoError(t, err)
138+ require.Equal(t, role != rolePushButton, covered[1])
139+ if isDocument(role) {
140+ require.Empty(t, targets)
141+ } else {
142+ require.Len(t, targets, 1)
143+ }
144+ })
145+ }
146+ }
147+}
148+
149+var oneWindow = []ports.HintWindow{{ID: 1, PID: 77, Rect: ports.Rect{W: 800, H: 600}}}
150+
151+// New does not touch the bus; the first Targets call connects, once.
152+func TestConnectsLazilyOnFirstTargets(t *testing.T) {
153+ tr := newMocktransport(t)
154+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
155+ [4]int32{0, 0, 800, 600}, []ref{{":1.5", "/a"}}, map[string][4]int32{"/a": {1, 1, 9, 9}}, map[string]uint32{"/a": roleLink}, false))
156+ c := newMockconnector(t)
157+ c.EXPECT().Connect(mock.Anything).Return(tr, nil).Once()
158+ s := &Source{dial: c}
159+ for range 2 {
160+ targets, _, err := s.Targets(context.Background(), oneWindow, false)
161+ require.NoError(t, err)
162+ require.Len(t, targets, 1)
163+ }
164+}
165+
166+// Without PIDs nothing is asked and nothing is connected.
167+func TestNoConnectWithoutWindows(t *testing.T) {
168+ s := &Source{dial: newMockconnector(t)}
169+ _, _, err := s.Targets(context.Background(), []ports.HintWindow{{ID: 1}}, false)
170+ require.NoError(t, err)
171+}
172+
173+// A failed connection is retried no sooner than retryEvery; meanwhile no
174+// targets are returned.
175+func TestConnectRetriesWithBackoff(t *testing.T) {
176+ c := newMockconnector(t)
177+ c.EXPECT().Connect(mock.Anything).Return(nil, errors.New("down")).Once()
178+ s := &Source{dial: c}
179+ for range 3 {
180+ targets, covered, err := s.Targets(context.Background(), oneWindow, false)
181+ require.NoError(t, err)
182+ require.Empty(t, targets)
183+ require.Empty(t, covered)
184+ }
185+ tr := newMocktransport(t)
186+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
187+ [4]int32{0, 0, 800, 600}, []ref{{":1.5", "/a"}}, map[string][4]int32{"/a": {1, 1, 9, 9}}, map[string]uint32{"/a": roleLink}, false))
188+ c.EXPECT().Connect(mock.Anything).Return(tr, nil).Once()
189+ s.retryAt = time.Now().Add(-time.Second)
190+ targets, _, err := s.Targets(context.Background(), oneWindow, false)
191+ require.NoError(t, err)
192+ require.Len(t, targets, 1)
193+}
194+
195+// A broken connection is closed and dropped; the next attempt reconnects.
196+func TestBrokenConnectionIsDropped(t *testing.T) {
197+ tr := newMocktransport(t)
198+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(
199+ func(_ time.Time, _ time.Duration, calls []call) []reply {
200+ out := make([]reply, len(calls))
201+ for i := range out {
202+ out[i].err = ErrBroken
203+ }
204+ return out
205+ }).Once()
206+ tr.EXPECT().Close().Return(nil).Once()
207+ c := newMockconnector(t)
208+ c.EXPECT().Connect(mock.Anything).Return(tr, nil).Once()
209+ s := &Source{dial: c}
210+ _, _, err := s.Targets(context.Background(), oneWindow, false)
211+ require.Error(t, err)
212+ require.Nil(t, s.tr)
213+ targets, _, err := s.Targets(context.Background(), oneWindow, false)
214+ require.NoError(t, err)
215+ require.Empty(t, targets) // backoff: no new attempt yet
216+}
217+
218+const testWait = 20 * time.Millisecond
219+
220+// blockedConnector returns a connector whose Connect blocks until release is
221+func blockedConnector(t *testing.T, release <-chan struct{}, tr transport, err error, times int) *mockconnector {
222+ t.Helper()
223+ c := newMockconnector(t)
224+ c.EXPECT().Connect(mock.Anything).RunAndReturn(func(context.Context) (transport, error) {
225+ <-release
226+ return tr, err
227+ }).Times(times)
228+ return c
229+}
230+
231+// A silent service cannot hold Targets beyond the wait, is not doubled while
232+// in flight, and its late transport is used.
233+func TestConnectIsBoundedAndNotDoubled(t *testing.T) {
234+ release := make(chan struct{})
235+ tr := newMocktransport(t)
236+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
237+ [4]int32{0, 0, 800, 600}, []ref{{":1.5", "/a"}}, map[string][4]int32{"/a": {1, 1, 9, 9}}, map[string]uint32{"/a": roleLink}, false))
238+ c := blockedConnector(t, release, tr, nil, 1)
239+ s := &Source{dial: c, wait: testWait}
240+ t0 := time.Now()
241+ targets, _, err := s.Targets(context.Background(), oneWindow, false)
242+ require.NoError(t, err)
243+ require.Empty(t, targets)
244+ require.Less(t, time.Since(t0), testWait+200*time.Millisecond)
245+ require.NotNil(t, s.attempt)
246+ // Still in flight: no second Connect (Times(1)), and no blocking.
247+ targets, _, err = s.Targets(context.Background(), oneWindow, false)
248+ require.NoError(t, err)
249+ require.Empty(t, targets)
250+ // It connects late: the next call uses it.
251+ close(release)
252+ require.Eventually(t, func() bool { return len(s.attempt) == 1 }, time.Second, time.Millisecond)
253+ targets, _, err = s.Targets(context.Background(), oneWindow, false)
254+ require.NoError(t, err)
255+ require.Len(t, targets, 1)
256+ require.Nil(t, s.attempt)
257+}
258+
259+// An abandoned attempt that fails applies the backoff, then one new attempt.
260+func TestAbandonedAttemptErrorBacksOff(t *testing.T) {
261+ release := make(chan struct{})
262+ tr := newMocktransport(t)
263+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
264+ [4]int32{0, 0, 800, 600}, []ref{{":1.5", "/a"}}, map[string][4]int32{"/a": {1, 1, 9, 9}}, map[string]uint32{"/a": roleLink}, false))
265+ c := newMockconnector(t)
266+ c.EXPECT().Connect(mock.Anything).RunAndReturn(func(context.Context) (transport, error) {
267+ <-release
268+ return nil, errors.New("down")
269+ }).Once()
270+ c.EXPECT().Connect(mock.Anything).Return(tr, nil).Once()
271+ s := &Source{dial: c, wait: testWait}
272+ targets, _, err := s.Targets(context.Background(), oneWindow, false)
273+ require.NoError(t, err)
274+ require.Empty(t, targets)
275+ close(release)
276+ require.Eventually(t, func() bool { return len(s.attempt) == 1 }, time.Second, time.Millisecond)
277+ // The error is collected; the backoff runs from its finish time.
278+ targets, _, err = s.Targets(context.Background(), oneWindow, false)
279+ require.NoError(t, err)
280+ require.Empty(t, targets)
281+ require.Nil(t, s.attempt)
282+ require.WithinDuration(t, time.Now().Add(retryEvery), s.retryAt, time.Second)
283+ targets, _, err = s.Targets(context.Background(), oneWindow, false) // backoff: no Connect
284+ require.NoError(t, err)
285+ require.Empty(t, targets)
286+ s.retryAt = time.Now().Add(-time.Second)
287+ targets, _, err = s.Targets(context.Background(), oneWindow, false)
288+ require.NoError(t, err)
289+ require.Len(t, targets, 1)
290+}
291+
292+// Close with an attempt in flight closes the late transport; afterwards
293+// Targets finds nothing and never connects.
294+func TestCloseReapsLateTransport(t *testing.T) {
295+ release := make(chan struct{})
296+ closed := make(chan struct{})
297+ tr := newMocktransport(t)
298+ tr.EXPECT().Close().RunAndReturn(func() error { close(closed); return nil }).Once()
299+ c := blockedConnector(t, release, tr, nil, 1)
300+ s := &Source{dial: c, wait: testWait}
301+ _, _, err := s.Targets(context.Background(), oneWindow, false)
302+ require.NoError(t, err)
303+ require.NoError(t, s.Close())
304+ close(release)
305+ select {
306+ case <-closed:
307+ case <-time.After(5 * time.Second):
308+ t.Fatal("late transport not closed")
309+ }
310+ targets, covered, err := s.Targets(context.Background(), oneWindow, false)
311+ require.NoError(t, err)
312+ require.Empty(t, targets)
313+ require.Empty(t, covered)
314+ require.NoError(t, s.Close()) // idempotent
315+}
@@ -0,0 +1,315 @@
1+package atspi
2+
3+import (
4+ "context"
5+ "errors"
6+ "fmt"
7+ "testing"
8+ "time"
9+
10+ "github.com/stretchr/testify/mock"
11+ "github.com/stretchr/testify/require"
12+
13+ "github.com/bnema/neferwl/internal/ports"
14+)
15+
16+const (
17+ stActive = 1 << stateActive
18+ stShowing = 1 << stateShowing
19+)
20+
21+// tree answers calls like an app: registry with one app (:1.5, pid 77) with
22+// one showing frame whose links sit at window-relative extents.
23+func answerTree(frameExt [4]int32, matches []ref, ext map[string][4]int32, roles map[string]uint32, noCollection bool) func(time.Time, time.Duration, []call) []reply {
24+ return func(_ time.Time, _ time.Duration, calls []call) []reply {
25+ out := make([]reply, len(calls))
26+ for i, c := range calls {
27+ switch {
28+ case c.op == opChildren && c.name == registryName:
29+ out[i].refs = []ref{{":1.5", "/root"}, {":1.9", "/root"}}
30+ case c.op == opPID:
31+ out[i].u = map[string]uint32{":1.5": 77, ":1.9": 99}[c.name]
32+ case c.op == opChildren && c.path == "/root":
33+ out[i].refs = []ref{{":1.5", "/frame"}}
34+ case c.op == opRole && c.path == "/frame":
35+ out[i].u = roleFrame
36+ case c.op == opState && c.path == "/frame":
37+ out[i].state = stShowing | stActive
38+ case c.op == opExtents && c.path == "/frame":
39+ out[i].rect = frameExt
40+ case c.op == opMatches:
41+ if noCollection {
42+ out[i].err = &busError{name: "org.freedesktop.DBus.Error.UnknownMethod"}
43+ } else {
44+ out[i].refs = matches
45+ }
46+ case c.op == opChildren && c.path == "/frame":
47+ out[i].refs = matches
48+ case c.op == opChildren:
49+ case c.op == opRole:
50+ out[i].u = roles[c.path]
51+ case c.op == opState:
52+ out[i].state = stShowing | 1<<stateVisible
53+ case c.op == opExtents:
54+ out[i].rect = ext[c.path]
55+ }
56+ }
57+ return out
58+ }
59+}
60+
61+func TestTargetsCoversWindowAndConverts(t *testing.T) {
62+ tr := newMocktransport(t)
63+ matches := []ref{{":1.5", "/a"}, {":1.5", "/b"}, {":1.5", "/zero"}}
64+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
65+ [4]int32{4, 30, 800, 600}, matches,
66+ map[string][4]int32{"/a": {14, 130, 50, 16}, "/b": {104, 230, 20, 20}, "/zero": {5, 5, 0, 0}},
67+ map[string]uint32{"/a": roleLink, "/b": rolePushButton, "/zero": roleLink}, false))
68+ s := &Source{tr: tr}
69+
70+ windows := []ports.HintWindow{
71+ {ID: 1, PID: 77, Rect: ports.Rect{X: 100, Y: 200, W: 800, H: 600}, Focused: true},
72+ {ID: 2, PID: 5, Rect: ports.Rect{X: 0, Y: 0, W: 50, H: 50}},
73+ }
74+ targets, covered, err := s.Targets(context.Background(), windows, false)
75+ require.NoError(t, err)
76+ require.Equal(t, map[ports.WindowID]bool{1: true}, covered)
77+ require.Equal(t, []ports.HintTarget{
78+ {Rect: ports.Rect{X: 110, Y: 300, W: 50, H: 16}, Kind: ports.HintLink, Window: 1},
79+ {Rect: ports.Rect{X: 200, Y: 400, W: 20, H: 20}, Kind: ports.HintButton, Window: 1},
80+ }, targets)
81+}
82+
83+func TestTargetsLinksOnlyDropsButtons(t *testing.T) {
84+ tr := newMocktransport(t)
85+ matches := []ref{{":1.5", "/a"}, {":1.5", "/b"}}
86+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
87+ [4]int32{0, 0, 800, 600}, matches,
88+ map[string][4]int32{"/a": {14, 130, 50, 16}, "/b": {104, 230, 20, 20}},
89+ map[string]uint32{"/a": roleLink, "/b": rolePushButton}, false))
90+ s := &Source{tr: tr}
91+ targets, _, err := s.Targets(context.Background(), []ports.HintWindow{{ID: 1, PID: 77, Rect: ports.Rect{W: 800, H: 600}}}, true)
92+ require.NoError(t, err)
93+ require.Len(t, targets, 1)
94+ require.Equal(t, ports.HintLink, targets[0].Kind)
95+}
96+
97+func TestTargetsWalkWithoutCollection(t *testing.T) {
98+ tr := newMocktransport(t)
99+ matches := []ref{{":1.5", "/a"}}
100+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
101+ [4]int32{0, 0, 800, 600}, matches,
102+ map[string][4]int32{"/a": {14, 130, 50, 16}},
103+ map[string]uint32{"/a": roleLink}, true))
104+ s := &Source{tr: tr}
105+ targets, covered, err := s.Targets(context.Background(), []ports.HintWindow{{ID: 1, PID: 77, Rect: ports.Rect{W: 800, H: 600}}}, false)
106+ require.NoError(t, err)
107+ require.Len(t, targets, 1)
108+ require.True(t, covered[1])
109+}
110+
111+func TestTargetsNoPIDMatchCoversNothing(t *testing.T) {
112+ tr := newMocktransport(t)
113+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree([4]int32{}, nil, nil, nil, false))
114+ s := &Source{tr: tr}
115+ targets, covered, err := s.Targets(context.Background(), []ports.HintWindow{{ID: 1, PID: 1234, Rect: ports.Rect{W: 8, H: 6}}}, false)
116+ require.NoError(t, err)
117+ require.Empty(t, targets)
118+ require.Empty(t, covered)
119+}
120+
121+func TestTargetsWithoutPIDsAsksNothing(t *testing.T) {
122+ s := &Source{tr: newMocktransport(t)}
123+ _, covered, err := s.Targets(context.Background(), []ports.HintWindow{{ID: 1, Rect: ports.Rect{W: 8, H: 6}}}, false)
124+ require.NoError(t, err)
125+ require.Empty(t, covered)
126+}
127+
128+func TestCoverageRequiresLinkOrDocument(t *testing.T) {
129+ for _, collection := range []bool{true, false} {
130+ for _, role := range []uint32{rolePushButton, roleDocumentFrame, roleDocumentWeb, roleLink} {
131+ t.Run(fmt.Sprintf("collection=%t/role=%d", collection, role), func(t *testing.T) {
132+ tr := newMocktransport(t)
133+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
134+ [4]int32{0, 0, 800, 600}, []ref{{":1.5", "/a"}}, map[string][4]int32{"/a": {10, 20, 30, 40}}, map[string]uint32{"/a": role}, !collection))
135+ s := &Source{tr: tr}
136+ targets, covered, err := s.Targets(context.Background(), []ports.HintWindow{{ID: 1, PID: 77, Rect: ports.Rect{W: 800, H: 600}}}, false)
137+ require.NoError(t, err)
138+ require.Equal(t, role != rolePushButton, covered[1])
139+ if isDocument(role) {
140+ require.Empty(t, targets)
141+ } else {
142+ require.Len(t, targets, 1)
143+ }
144+ })
145+ }
146+ }
147+}
148+
149+var oneWindow = []ports.HintWindow{{ID: 1, PID: 77, Rect: ports.Rect{W: 800, H: 600}}}
150+
151+// New does not touch the bus; the first Targets call connects, once.
152+func TestConnectsLazilyOnFirstTargets(t *testing.T) {
153+ tr := newMocktransport(t)
154+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
155+ [4]int32{0, 0, 800, 600}, []ref{{":1.5", "/a"}}, map[string][4]int32{"/a": {1, 1, 9, 9}}, map[string]uint32{"/a": roleLink}, false))
156+ c := newMockconnector(t)
157+ c.EXPECT().Connect(mock.Anything).Return(tr, nil).Once()
158+ s := &Source{dial: c}
159+ for range 2 {
160+ targets, _, err := s.Targets(context.Background(), oneWindow, false)
161+ require.NoError(t, err)
162+ require.Len(t, targets, 1)
163+ }
164+}
165+
166+// Without PIDs nothing is asked and nothing is connected.
167+func TestNoConnectWithoutWindows(t *testing.T) {
168+ s := &Source{dial: newMockconnector(t)}
169+ _, _, err := s.Targets(context.Background(), []ports.HintWindow{{ID: 1}}, false)
170+ require.NoError(t, err)
171+}
172+
173+// A failed connection is retried no sooner than retryEvery; meanwhile no
174+// targets are returned.
175+func TestConnectRetriesWithBackoff(t *testing.T) {
176+ c := newMockconnector(t)
177+ c.EXPECT().Connect(mock.Anything).Return(nil, errors.New("down")).Once()
178+ s := &Source{dial: c}
179+ for range 3 {
180+ targets, covered, err := s.Targets(context.Background(), oneWindow, false)
181+ require.NoError(t, err)
182+ require.Empty(t, targets)
183+ require.Empty(t, covered)
184+ }
185+ tr := newMocktransport(t)
186+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
187+ [4]int32{0, 0, 800, 600}, []ref{{":1.5", "/a"}}, map[string][4]int32{"/a": {1, 1, 9, 9}}, map[string]uint32{"/a": roleLink}, false))
188+ c.EXPECT().Connect(mock.Anything).Return(tr, nil).Once()
189+ s.retryAt = time.Now().Add(-time.Second)
190+ targets, _, err := s.Targets(context.Background(), oneWindow, false)
191+ require.NoError(t, err)
192+ require.Len(t, targets, 1)
193+}
194+
195+// A broken connection is closed and dropped; the next attempt reconnects.
196+func TestBrokenConnectionIsDropped(t *testing.T) {
197+ tr := newMocktransport(t)
198+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(
199+ func(_ time.Time, _ time.Duration, calls []call) []reply {
200+ out := make([]reply, len(calls))
201+ for i := range out {
202+ out[i].err = ErrBroken
203+ }
204+ return out
205+ }).Once()
206+ tr.EXPECT().Close().Return(nil).Once()
207+ c := newMockconnector(t)
208+ c.EXPECT().Connect(mock.Anything).Return(tr, nil).Once()
209+ s := &Source{dial: c}
210+ _, _, err := s.Targets(context.Background(), oneWindow, false)
211+ require.Error(t, err)
212+ require.Nil(t, s.tr)
213+ targets, _, err := s.Targets(context.Background(), oneWindow, false)
214+ require.NoError(t, err)
215+ require.Empty(t, targets) // backoff: no new attempt yet
216+}
217+
218+const testWait = 20 * time.Millisecond
219+
220+// blockedConnector returns a connector whose Connect blocks until release is
221+func blockedConnector(t *testing.T, release <-chan struct{}, tr transport, err error, times int) *mockconnector {
222+ t.Helper()
223+ c := newMockconnector(t)
224+ c.EXPECT().Connect(mock.Anything).RunAndReturn(func(context.Context) (transport, error) {
225+ <-release
226+ return tr, err
227+ }).Times(times)
228+ return c
229+}
230+
231+// A silent service cannot hold Targets beyond the wait, is not doubled while
232+// in flight, and its late transport is used.
233+func TestConnectIsBoundedAndNotDoubled(t *testing.T) {
234+ release := make(chan struct{})
235+ tr := newMocktransport(t)
236+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
237+ [4]int32{0, 0, 800, 600}, []ref{{":1.5", "/a"}}, map[string][4]int32{"/a": {1, 1, 9, 9}}, map[string]uint32{"/a": roleLink}, false))
238+ c := blockedConnector(t, release, tr, nil, 1)
239+ s := &Source{dial: c, wait: testWait}
240+ t0 := time.Now()
241+ targets, _, err := s.Targets(context.Background(), oneWindow, false)
242+ require.NoError(t, err)
243+ require.Empty(t, targets)
244+ require.Less(t, time.Since(t0), testWait+200*time.Millisecond)
245+ require.NotNil(t, s.attempt)
246+ // Still in flight: no second Connect (Times(1)), and no blocking.
247+ targets, _, err = s.Targets(context.Background(), oneWindow, false)
248+ require.NoError(t, err)
249+ require.Empty(t, targets)
250+ // It connects late: the next call uses it.
251+ close(release)
252+ require.Eventually(t, func() bool { return len(s.attempt) == 1 }, time.Second, time.Millisecond)
253+ targets, _, err = s.Targets(context.Background(), oneWindow, false)
254+ require.NoError(t, err)
255+ require.Len(t, targets, 1)
256+ require.Nil(t, s.attempt)
257+}
258+
259+// An abandoned attempt that fails applies the backoff, then one new attempt.
260+func TestAbandonedAttemptErrorBacksOff(t *testing.T) {
261+ release := make(chan struct{})
262+ tr := newMocktransport(t)
263+ tr.EXPECT().Do(mock.Anything, mock.Anything, mock.Anything).RunAndReturn(answerTree(
264+ [4]int32{0, 0, 800, 600}, []ref{{":1.5", "/a"}}, map[string][4]int32{"/a": {1, 1, 9, 9}}, map[string]uint32{"/a": roleLink}, false))
265+ c := newMockconnector(t)
266+ c.EXPECT().Connect(mock.Anything).RunAndReturn(func(context.Context) (transport, error) {
267+ <-release
268+ return nil, errors.New("down")
269+ }).Once()
270+ c.EXPECT().Connect(mock.Anything).Return(tr, nil).Once()
271+ s := &Source{dial: c, wait: testWait}
272+ targets, _, err := s.Targets(context.Background(), oneWindow, false)
273+ require.NoError(t, err)
274+ require.Empty(t, targets)
275+ close(release)
276+ require.Eventually(t, func() bool { return len(s.attempt) == 1 }, time.Second, time.Millisecond)
277+ // The error is collected; the backoff runs from its finish time.
278+ targets, _, err = s.Targets(context.Background(), oneWindow, false)
279+ require.NoError(t, err)
280+ require.Empty(t, targets)
281+ require.Nil(t, s.attempt)
282+ require.WithinDuration(t, time.Now().Add(retryEvery), s.retryAt, time.Second)
283+ targets, _, err = s.Targets(context.Background(), oneWindow, false) // backoff: no Connect
284+ require.NoError(t, err)
285+ require.Empty(t, targets)
286+ s.retryAt = time.Now().Add(-time.Second)
287+ targets, _, err = s.Targets(context.Background(), oneWindow, false)
288+ require.NoError(t, err)
289+ require.Len(t, targets, 1)
290+}
291+
292+// Close with an attempt in flight closes the late transport; afterwards
293+// Targets finds nothing and never connects.
294+func TestCloseReapsLateTransport(t *testing.T) {
295+ release := make(chan struct{})
296+ closed := make(chan struct{})
297+ tr := newMocktransport(t)
298+ tr.EXPECT().Close().RunAndReturn(func() error { close(closed); return nil }).Once()
299+ c := blockedConnector(t, release, tr, nil, 1)
300+ s := &Source{dial: c, wait: testWait}
301+ _, _, err := s.Targets(context.Background(), oneWindow, false)
302+ require.NoError(t, err)
303+ require.NoError(t, s.Close())
304+ close(release)
305+ select {
306+ case <-closed:
307+ case <-time.After(5 * time.Second):
308+ t.Fatal("late transport not closed")
309+ }
310+ targets, covered, err := s.Targets(context.Background(), oneWindow, false)
311+ require.NoError(t, err)
312+ require.Empty(t, targets)
313+ require.Empty(t, covered)
314+ require.NoError(t, s.Close()) // idempotent
315+}
A internal/adapters/atspi/transport.go +362 -0
@@ -0,0 +1,362 @@
1+package atspi
2+
3+import (
4+ "errors"
5+ "fmt"
6+ "strings"
7+ "sync/atomic"
8+ "syscall"
9+ "time"
10+
11+ "github.com/bnema/zerobus"
12+ "golang.org/x/sys/unix"
13+)
14+
15+// D-Bus names, paths and interfaces.
16+const (
17+ busDaemon = "org.freedesktop.DBus"
18+ busDaemonPath = "/org/freedesktop/DBus"
19+ ifaceProps = "org.freedesktop.DBus.Properties"
20+
21+ a11yBusName = "org.a11y.Bus"
22+ a11yBusPath = "/org/a11y/bus"
23+ a11yStatus = "org.a11y.Status"
24+ registryName = "org.a11y.atspi.Registry"
25+ registryRoot = "/org/a11y/atspi/accessible/root"
26+
27+ ifaceAccessible = "org.a11y.atspi.Accessible"
28+ ifaceComponent = "org.a11y.atspi.Component"
29+ ifaceCollection = "org.a11y.atspi.Collection"
30+
31+ // coordWindow is ATSPI_COORD_TYPE_WINDOW: extents relative to the
32+ // top-level window.
33+ coordWindow = 1
34+
35+ // maxInflight bounds the calls sent before a reply is read, so the bus
36+ // never waits on a full socket while this side waits on it.
37+ maxInflight = 64
38+ // maxRefs bounds the objects read from one reply.
39+ maxRefs = 4096
40+)
41+
42+// op is a call the source makes.
43+type op uint8
44+
45+const (
46+ opChildren op = iota // Accessible.GetChildren -> refs
47+ opPID // GetConnectionUnixProcessID(name) -> u
48+ opRole // Accessible.GetRole -> u
49+ opState // Accessible.GetState -> state
50+ opExtents // Component.GetExtents(arg) -> rect
51+ opMatches // Collection.GetMatches(rule) -> refs
52+)
53+
54+// ref is an accessible object: a bus name and a path.
55+type ref struct{ name, path string }
56+
57+// call is one request of a batch. name and path address the object; for
58+// opPID, name is the bus name asked about.
59+type call struct {
60+ op op
61+ name, path string
62+ arg uint32
63+ rule matchRule
64+}
65+
66+// reply answers the call at the same index of the batch.
67+type reply struct {
68+ err error
69+ refs []ref
70+ u uint32
71+ state uint64
72+ rect [4]int32 // x, y, w, h
73+}
74+
75+// transport sends batches of calls on one bus connection. It is used by one
76+// goroutine at a time.
77+type transport interface {
78+ // Do sends calls pipelined and returns one reply per call, in order.
79+ // The batch ends at deadline. A call not answered idle after it was
80+ // sent gets ErrTimeout. When the connection fails or the deadline
81+ // finds calls unanswered, those calls get an error wrapping
82+ // ErrBroken: the connection must be dropped.
83+ Do(deadline time.Time, idle time.Duration, calls []call) []reply
84+ Close() error
85+}
86+
87+// ErrTimeout is the error of a call not answered in time.
88+var ErrTimeout = errors.New("atspi: call timed out")
89+
90+// busError is a D-Bus error reply.
91+type busError struct{ name, msg string }
92+
93+func (e *busError) Error() string {
94+ if e.msg == "" {
95+ return e.name
96+ }
97+ return e.name + ": " + e.msg
98+}
99+
100+// conn is a transport on a zerobus connection.
101+type conn struct {
102+ c *zerobus.Conn
103+ raw syscall.RawConn
104+}
105+
106+func newConn(c *zerobus.Conn) (*conn, error) {
107+ raw, err := c.SyscallConn()
108+ if err != nil {
109+ _ = c.Close()
110+ return nil, err
111+ }
112+ return &conn{c: c, raw: raw}, nil
113+}
114+
115+func (b *conn) Close() error { return b.c.Close() }
116+
117+// readable waits until a message can be read or d passes.
118+func (b *conn) readable(d time.Duration) (bool, error) {
119+ if b.c.Buffered() {
120+ return true, nil
121+ }
122+ ms := int((d + time.Millisecond - 1) / time.Millisecond)
123+ if ms < 1 {
124+ ms = 1
125+ }
126+ var n int
127+ var perr error
128+ if err := b.raw.Control(func(fd uintptr) {
129+ fds := []unix.PollFd{{Fd: int32(fd), Events: unix.POLLIN}}
130+ for {
131+ n, perr = unix.Poll(fds, ms)
132+ if perr != unix.EINTR {
133+ return
134+ }
135+ }
136+ }); err != nil {
137+ return false, err
138+ }
139+ return n > 0, perr
140+}
141+
142+// ErrBroken is the error of calls on a connection that failed or was closed
143+// at the deadline; the owner must drop it and connect again.
144+var ErrBroken = errors.New("atspi: connection broken")
145+
146+// errDeadline is the error of calls still unanswered at the batch deadline:
147+// late replies and half-read messages leave the connection unusable.
148+var errDeadline = fmt.Errorf("%w: %w", ErrBroken, ErrTimeout)
149+
150+// watchdogGrace is how long after the deadline the watchdog closes a socket
151+// that the batch loop could not leave by itself (a stuck read or write).
152+const watchdogGrace = 20 * time.Millisecond
153+
154+// Do sends the calls pipelined. A watchdog closes the socket shortly after
155+// deadline, so neither a blocked write nor a read stuck in the middle of a
156+// message can outlive the batch; the connection is then broken. Each call
157+// has its own idle timeout, counted from its send: traffic for other calls
158+// does not extend it.
159+func (b *conn) Do(deadline time.Time, idle time.Duration, calls []call) []reply {
160+ out := make([]reply, len(calls))
161+ answered := make([]bool, len(calls))
162+ type inflight struct {
163+ idx int
164+ sent time.Time
165+ }
166+ pending := make(map[uint32]inflight, min(len(calls), maxInflight))
167+ next, done := 0, 0
168+ var fired atomic.Bool
169+ watchdog := time.AfterFunc(time.Until(deadline)+watchdogGrace, func() {
170+ fired.Store(true)
171+ _ = b.c.Close()
172+ })
173+ defer watchdog.Stop()
174+ // finish gives err to every call without an answer.
175+ finish := func(err error) []reply {
176+ if !errors.Is(err, ErrBroken) && (fired.Load() || !errors.Is(err, ErrTimeout)) {
177+ err = fmt.Errorf("%w: %w", ErrBroken, err)
178+ }
179+ if errors.Is(err, ErrBroken) {
180+ _ = b.c.Close() // late replies would desynchronize the next batch
181+ }
182+ for i := range out {
183+ if !answered[i] {
184+ out[i].err = err
185+ }
186+ }
187+ return out
188+ }
189+ for done < len(calls) {
190+ for next < len(calls) && len(pending) < maxInflight {
191+ if !time.Now().Before(deadline) {
192+ return finish(errDeadline)
193+ }
194+ serial, err := b.send(calls[next])
195+ if err != nil {
196+ return finish(err)
197+ }
198+ pending[serial] = inflight{idx: next, sent: time.Now()}
199+ next++
200+ }
201+ // Calls that waited longer than idle get no reply.
202+ now := time.Now()
203+ var oldest time.Time
204+ for serial, p := range pending {
205+ if now.Sub(p.sent) >= idle {
206+ out[p.idx].err = ErrTimeout
207+ answered[p.idx] = true
208+ done++
209+ delete(pending, serial)
210+ } else if oldest.IsZero() || p.sent.Before(oldest) {
211+ oldest = p.sent
212+ }
213+ }
214+ if len(pending) == 0 {
215+ continue
216+ }
217+ wait := min(time.Until(oldest.Add(idle)), time.Until(deadline))
218+ if wait <= 0 {
219+ if !time.Now().Before(deadline) {
220+ break
221+ }
222+ continue
223+ }
224+ ok, err := b.readable(wait)
225+ if err != nil {
226+ return finish(err)
227+ }
228+ if !ok {
229+ continue
230+ }
231+ m, err := b.c.ReadMessage()
232+ if err != nil {
233+ return finish(err)
234+ }
235+ p, found := pending[m.ReplySerial]
236+ if !found || (m.Type != zerobus.TypeMethodReturn && m.Type != zerobus.TypeError) {
237+ // A signal, or the late reply of an earlier batch.
238+ continue
239+ }
240+ delete(pending, m.ReplySerial)
241+ decode(m, calls[p.idx], &out[p.idx])
242+ answered[p.idx] = true
243+ done++
244+ }
245+ if done == len(calls) {
246+ return out
247+ }
248+ return finish(errDeadline)
249+}
250+
251+// send builds and sends one call and returns its serial.
252+func (b *conn) send(c call) (uint32, error) {
253+ switch c.op {
254+ case opChildren:
255+ b.c.NewCall(c.name, c.path, ifaceAccessible, "GetChildren", "")
256+ case opPID:
257+ b.c.NewCall(busDaemon, busDaemonPath, busDaemon, "GetConnectionUnixProcessID", "s").Str(c.name)
258+ case opRole:
259+ b.c.NewCall(c.name, c.path, ifaceAccessible, "GetRole", "")
260+ case opState:
261+ b.c.NewCall(c.name, c.path, ifaceAccessible, "GetState", "")
262+ case opExtents:
263+ b.c.NewCall(c.name, c.path, ifaceComponent, "GetExtents", "u").Uint32(c.arg)
264+ case opMatches:
265+ e := b.c.NewCall(c.name, c.path, ifaceCollection, "GetMatches", "(aiia{ss}iaiiasib)uib")
266+ encodeRule(e, c.rule)
267+ e.Uint32(sortCanonical)
268+ e.Int32(matchCount)
269+ e.Bool(true)
270+ }
271+ return b.c.Send()
272+}
273+
274+// encodeRule writes the MatchRule struct (aiia{ss}iaiiasib): states, state
275+// match, attributes, attribute match, roles, role match, interfaces,
276+// interface match, invert.
277+func encodeRule(e *zerobus.Encoder, r matchRule) {
278+ e.Struct()
279+ a := e.BeginArray('i')
280+ for _, w := range stateBits(stateShowing, stateVisible) {
281+ e.Int32(w)
282+ }
283+ e.EndArray(a)
284+ e.Int32(matchAll)
285+ m := e.BeginArray('{')
286+ e.EndArray(m)
287+ e.Int32(matchAll)
288+ a = e.BeginArray('i')
289+ for _, w := range roleBits(r.roles()) {
290+ e.Int32(w)
291+ }
292+ e.EndArray(a)
293+ e.Int32(matchAny)
294+ s := e.BeginArray('s')
295+ e.EndArray(s)
296+ e.Int32(matchAll)
297+ e.Bool(false)
298+}
299+
300+// decode reads the reply m of c into out.
301+func decode(m *zerobus.Message, c call, out *reply) {
302+ if m.Type == zerobus.TypeError {
303+ be := &busError{name: strings.Clone(m.ErrorName)}
304+ if strings.HasPrefix(m.Signature, "s") {
305+ be.msg = strings.Clone(m.Body().Str())
306+ }
307+ out.err = be
308+ return
309+ }
310+ r := m.Body()
311+ want := func(sig string) bool {
312+ if m.Signature != sig {
313+ out.err = fmt.Errorf("atspi: reply signature %q, want %q", m.Signature, sig)
314+ return false
315+ }
316+ return true
317+ }
318+ switch c.op {
319+ case opChildren, opMatches:
320+ if !want("a(so)") {
321+ return
322+ }
323+ end := r.Array('(')
324+ for r.More(end) {
325+ if len(out.refs) >= maxRefs {
326+ break
327+ }
328+ r.Struct()
329+ name := r.Str()
330+ path := r.ObjectPath()
331+ if r.Err() != nil {
332+ break
333+ }
334+ out.refs = append(out.refs, ref{strings.Clone(name), strings.Clone(path)})
335+ }
336+ case opPID, opRole:
337+ if want("u") {
338+ out.u = r.Uint32()
339+ }
340+ case opState:
341+ if !want("au") {
342+ return
343+ }
344+ end := r.Array('u')
345+ var words [2]uint32
346+ for i := 0; r.More(end); i++ {
347+ w := r.Uint32()
348+ if i < len(words) {
349+ words[i] = w
350+ }
351+ }
352+ out.state = uint64(words[0]) | uint64(words[1])<<32
353+ case opExtents:
354+ if want("(iiii)") {
355+ r.Struct()
356+ out.rect = [4]int32{r.Int32(), r.Int32(), r.Int32(), r.Int32()}
357+ }
358+ }
359+ if out.err == nil && r.Err() != nil {
360+ out.err = r.Err()
361+ }
362+}
@@ -0,0 +1,362 @@
1+package atspi
2+
3+import (
4+ "errors"
5+ "fmt"
6+ "strings"
7+ "sync/atomic"
8+ "syscall"
9+ "time"
10+
11+ "github.com/bnema/zerobus"
12+ "golang.org/x/sys/unix"
13+)
14+
15+// D-Bus names, paths and interfaces.
16+const (
17+ busDaemon = "org.freedesktop.DBus"
18+ busDaemonPath = "/org/freedesktop/DBus"
19+ ifaceProps = "org.freedesktop.DBus.Properties"
20+
21+ a11yBusName = "org.a11y.Bus"
22+ a11yBusPath = "/org/a11y/bus"
23+ a11yStatus = "org.a11y.Status"
24+ registryName = "org.a11y.atspi.Registry"
25+ registryRoot = "/org/a11y/atspi/accessible/root"
26+
27+ ifaceAccessible = "org.a11y.atspi.Accessible"
28+ ifaceComponent = "org.a11y.atspi.Component"
29+ ifaceCollection = "org.a11y.atspi.Collection"
30+
31+ // coordWindow is ATSPI_COORD_TYPE_WINDOW: extents relative to the
32+ // top-level window.
33+ coordWindow = 1
34+
35+ // maxInflight bounds the calls sent before a reply is read, so the bus
36+ // never waits on a full socket while this side waits on it.
37+ maxInflight = 64
38+ // maxRefs bounds the objects read from one reply.
39+ maxRefs = 4096
40+)
41+
42+// op is a call the source makes.
43+type op uint8
44+
45+const (
46+ opChildren op = iota // Accessible.GetChildren -> refs
47+ opPID // GetConnectionUnixProcessID(name) -> u
48+ opRole // Accessible.GetRole -> u
49+ opState // Accessible.GetState -> state
50+ opExtents // Component.GetExtents(arg) -> rect
51+ opMatches // Collection.GetMatches(rule) -> refs
52+)
53+
54+// ref is an accessible object: a bus name and a path.
55+type ref struct{ name, path string }
56+
57+// call is one request of a batch. name and path address the object; for
58+// opPID, name is the bus name asked about.
59+type call struct {
60+ op op
61+ name, path string
62+ arg uint32
63+ rule matchRule
64+}
65+
66+// reply answers the call at the same index of the batch.
67+type reply struct {
68+ err error
69+ refs []ref
70+ u uint32
71+ state uint64
72+ rect [4]int32 // x, y, w, h
73+}
74+
75+// transport sends batches of calls on one bus connection. It is used by one
76+// goroutine at a time.
77+type transport interface {
78+ // Do sends calls pipelined and returns one reply per call, in order.
79+ // The batch ends at deadline. A call not answered idle after it was
80+ // sent gets ErrTimeout. When the connection fails or the deadline
81+ // finds calls unanswered, those calls get an error wrapping
82+ // ErrBroken: the connection must be dropped.
83+ Do(deadline time.Time, idle time.Duration, calls []call) []reply
84+ Close() error
85+}
86+
87+// ErrTimeout is the error of a call not answered in time.
88+var ErrTimeout = errors.New("atspi: call timed out")
89+
90+// busError is a D-Bus error reply.
91+type busError struct{ name, msg string }
92+
93+func (e *busError) Error() string {
94+ if e.msg == "" {
95+ return e.name
96+ }
97+ return e.name + ": " + e.msg
98+}
99+
100+// conn is a transport on a zerobus connection.
101+type conn struct {
102+ c *zerobus.Conn
103+ raw syscall.RawConn
104+}
105+
106+func newConn(c *zerobus.Conn) (*conn, error) {
107+ raw, err := c.SyscallConn()
108+ if err != nil {
109+ _ = c.Close()
110+ return nil, err
111+ }
112+ return &conn{c: c, raw: raw}, nil
113+}
114+
115+func (b *conn) Close() error { return b.c.Close() }
116+
117+// readable waits until a message can be read or d passes.
118+func (b *conn) readable(d time.Duration) (bool, error) {
119+ if b.c.Buffered() {
120+ return true, nil
121+ }
122+ ms := int((d + time.Millisecond - 1) / time.Millisecond)
123+ if ms < 1 {
124+ ms = 1
125+ }
126+ var n int
127+ var perr error
128+ if err := b.raw.Control(func(fd uintptr) {
129+ fds := []unix.PollFd{{Fd: int32(fd), Events: unix.POLLIN}}
130+ for {
131+ n, perr = unix.Poll(fds, ms)
132+ if perr != unix.EINTR {
133+ return
134+ }
135+ }
136+ }); err != nil {
137+ return false, err
138+ }
139+ return n > 0, perr
140+}
141+
142+// ErrBroken is the error of calls on a connection that failed or was closed
143+// at the deadline; the owner must drop it and connect again.
144+var ErrBroken = errors.New("atspi: connection broken")
145+
146+// errDeadline is the error of calls still unanswered at the batch deadline:
147+// late replies and half-read messages leave the connection unusable.
148+var errDeadline = fmt.Errorf("%w: %w", ErrBroken, ErrTimeout)
149+
150+// watchdogGrace is how long after the deadline the watchdog closes a socket
151+// that the batch loop could not leave by itself (a stuck read or write).
152+const watchdogGrace = 20 * time.Millisecond
153+
154+// Do sends the calls pipelined. A watchdog closes the socket shortly after
155+// deadline, so neither a blocked write nor a read stuck in the middle of a
156+// message can outlive the batch; the connection is then broken. Each call
157+// has its own idle timeout, counted from its send: traffic for other calls
158+// does not extend it.
159+func (b *conn) Do(deadline time.Time, idle time.Duration, calls []call) []reply {
160+ out := make([]reply, len(calls))
161+ answered := make([]bool, len(calls))
162+ type inflight struct {
163+ idx int
164+ sent time.Time
165+ }
166+ pending := make(map[uint32]inflight, min(len(calls), maxInflight))
167+ next, done := 0, 0
168+ var fired atomic.Bool
169+ watchdog := time.AfterFunc(time.Until(deadline)+watchdogGrace, func() {
170+ fired.Store(true)
171+ _ = b.c.Close()
172+ })
173+ defer watchdog.Stop()
174+ // finish gives err to every call without an answer.
175+ finish := func(err error) []reply {
176+ if !errors.Is(err, ErrBroken) && (fired.Load() || !errors.Is(err, ErrTimeout)) {
177+ err = fmt.Errorf("%w: %w", ErrBroken, err)
178+ }
179+ if errors.Is(err, ErrBroken) {
180+ _ = b.c.Close() // late replies would desynchronize the next batch
181+ }
182+ for i := range out {
183+ if !answered[i] {
184+ out[i].err = err
185+ }
186+ }
187+ return out
188+ }
189+ for done < len(calls) {
190+ for next < len(calls) && len(pending) < maxInflight {
191+ if !time.Now().Before(deadline) {
192+ return finish(errDeadline)
193+ }
194+ serial, err := b.send(calls[next])
195+ if err != nil {
196+ return finish(err)
197+ }
198+ pending[serial] = inflight{idx: next, sent: time.Now()}
199+ next++
200+ }
201+ // Calls that waited longer than idle get no reply.
202+ now := time.Now()
203+ var oldest time.Time
204+ for serial, p := range pending {
205+ if now.Sub(p.sent) >= idle {
206+ out[p.idx].err = ErrTimeout
207+ answered[p.idx] = true
208+ done++
209+ delete(pending, serial)
210+ } else if oldest.IsZero() || p.sent.Before(oldest) {
211+ oldest = p.sent
212+ }
213+ }
214+ if len(pending) == 0 {
215+ continue
216+ }
217+ wait := min(time.Until(oldest.Add(idle)), time.Until(deadline))
218+ if wait <= 0 {
219+ if !time.Now().Before(deadline) {
220+ break
221+ }
222+ continue
223+ }
224+ ok, err := b.readable(wait)
225+ if err != nil {
226+ return finish(err)
227+ }
228+ if !ok {
229+ continue
230+ }
231+ m, err := b.c.ReadMessage()
232+ if err != nil {
233+ return finish(err)
234+ }
235+ p, found := pending[m.ReplySerial]
236+ if !found || (m.Type != zerobus.TypeMethodReturn && m.Type != zerobus.TypeError) {
237+ // A signal, or the late reply of an earlier batch.
238+ continue
239+ }
240+ delete(pending, m.ReplySerial)
241+ decode(m, calls[p.idx], &out[p.idx])
242+ answered[p.idx] = true
243+ done++
244+ }
245+ if done == len(calls) {
246+ return out
247+ }
248+ return finish(errDeadline)
249+}
250+
251+// send builds and sends one call and returns its serial.
252+func (b *conn) send(c call) (uint32, error) {
253+ switch c.op {
254+ case opChildren:
255+ b.c.NewCall(c.name, c.path, ifaceAccessible, "GetChildren", "")
256+ case opPID:
257+ b.c.NewCall(busDaemon, busDaemonPath, busDaemon, "GetConnectionUnixProcessID", "s").Str(c.name)
258+ case opRole:
259+ b.c.NewCall(c.name, c.path, ifaceAccessible, "GetRole", "")
260+ case opState:
261+ b.c.NewCall(c.name, c.path, ifaceAccessible, "GetState", "")
262+ case opExtents:
263+ b.c.NewCall(c.name, c.path, ifaceComponent, "GetExtents", "u").Uint32(c.arg)
264+ case opMatches:
265+ e := b.c.NewCall(c.name, c.path, ifaceCollection, "GetMatches", "(aiia{ss}iaiiasib)uib")
266+ encodeRule(e, c.rule)
267+ e.Uint32(sortCanonical)
268+ e.Int32(matchCount)
269+ e.Bool(true)
270+ }
271+ return b.c.Send()
272+}
273+
274+// encodeRule writes the MatchRule struct (aiia{ss}iaiiasib): states, state
275+// match, attributes, attribute match, roles, role match, interfaces,
276+// interface match, invert.
277+func encodeRule(e *zerobus.Encoder, r matchRule) {
278+ e.Struct()
279+ a := e.BeginArray('i')
280+ for _, w := range stateBits(stateShowing, stateVisible) {
281+ e.Int32(w)
282+ }
283+ e.EndArray(a)
284+ e.Int32(matchAll)
285+ m := e.BeginArray('{')
286+ e.EndArray(m)
287+ e.Int32(matchAll)
288+ a = e.BeginArray('i')
289+ for _, w := range roleBits(r.roles()) {
290+ e.Int32(w)
291+ }
292+ e.EndArray(a)
293+ e.Int32(matchAny)
294+ s := e.BeginArray('s')
295+ e.EndArray(s)
296+ e.Int32(matchAll)
297+ e.Bool(false)
298+}
299+
300+// decode reads the reply m of c into out.
301+func decode(m *zerobus.Message, c call, out *reply) {
302+ if m.Type == zerobus.TypeError {
303+ be := &busError{name: strings.Clone(m.ErrorName)}
304+ if strings.HasPrefix(m.Signature, "s") {
305+ be.msg = strings.Clone(m.Body().Str())
306+ }
307+ out.err = be
308+ return
309+ }
310+ r := m.Body()
311+ want := func(sig string) bool {
312+ if m.Signature != sig {
313+ out.err = fmt.Errorf("atspi: reply signature %q, want %q", m.Signature, sig)
314+ return false
315+ }
316+ return true
317+ }
318+ switch c.op {
319+ case opChildren, opMatches:
320+ if !want("a(so)") {
321+ return
322+ }
323+ end := r.Array('(')
324+ for r.More(end) {
325+ if len(out.refs) >= maxRefs {
326+ break
327+ }
328+ r.Struct()
329+ name := r.Str()
330+ path := r.ObjectPath()
331+ if r.Err() != nil {
332+ break
333+ }
334+ out.refs = append(out.refs, ref{strings.Clone(name), strings.Clone(path)})
335+ }
336+ case opPID, opRole:
337+ if want("u") {
338+ out.u = r.Uint32()
339+ }
340+ case opState:
341+ if !want("au") {
342+ return
343+ }
344+ end := r.Array('u')
345+ var words [2]uint32
346+ for i := 0; r.More(end); i++ {
347+ w := r.Uint32()
348+ if i < len(words) {
349+ words[i] = w
350+ }
351+ }
352+ out.state = uint64(words[0]) | uint64(words[1])<<32
353+ case opExtents:
354+ if want("(iiii)") {
355+ r.Struct()
356+ out.rect = [4]int32{r.Int32(), r.Int32(), r.Int32(), r.Int32()}
357+ }
358+ }
359+ if out.err == nil && r.Err() != nil {
360+ out.err = r.Err()
361+ }
362+}
A internal/adapters/atspi/transport_mock_test.go +156 -0
@@ -0,0 +1,156 @@
1+// Code generated by mockery; DO NOT EDIT.
2+// github.com/vektra/mockery
3+// template: testify
4+
5+package atspi
6+
7+import (
8+ "time"
9+
10+ mock "github.com/stretchr/testify/mock"
11+)
12+
13+// newMocktransport creates a new instance of mocktransport. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
14+// The first argument is typically a *testing.T value.
15+func newMocktransport(t interface {
16+ mock.TestingT
17+ Cleanup(func())
18+}) *mocktransport {
19+ if helper, ok := t.(interface{ Helper() }); ok {
20+ helper.Helper()
21+ }
22+
23+ mock := &mocktransport{}
24+ mock.Mock.Test(t)
25+
26+ t.Cleanup(func() {
27+ if helper, ok := t.(interface{ Helper() }); ok {
28+ helper.Helper()
29+ }
30+ mock.AssertExpectations(t)
31+ })
32+
33+ return mock
34+}
35+
36+// mocktransport is an autogenerated mock type for the transport type
37+type mocktransport struct {
38+ mock.Mock
39+}
40+
41+type mocktransport_Expecter struct {
42+ mock *mock.Mock
43+}
44+
45+func (_m *mocktransport) EXPECT() *mocktransport_Expecter {
46+ return &mocktransport_Expecter{mock: &_m.Mock}
47+}
48+
49+// Close provides a mock function for the type mocktransport
50+func (_mock *mocktransport) Close() error {
51+ ret := _mock.Called()
52+
53+ if len(ret) == 0 {
54+ panic("no return value specified for Close")
55+ }
56+
57+ var r0 error
58+ if returnFunc, ok := ret.Get(0).(func() error); ok {
59+ r0 = returnFunc()
60+ } else {
61+ r0 = ret.Error(0)
62+ }
63+ return r0
64+}
65+
66+// mocktransport_Close_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Close'
67+type mocktransport_Close_Call struct {
68+ *mock.Call
69+}
70+
71+// Close is a helper method to define mock.On call
72+func (_e *mocktransport_Expecter) Close() *mocktransport_Close_Call {
73+ return &mocktransport_Close_Call{Call: _e.mock.On("Close")}
74+}
75+
76+func (_c *mocktransport_Close_Call) Run(run func()) *mocktransport_Close_Call {
77+ _c.Call.Run(func(args mock.Arguments) {
78+ run()
79+ })
80+ return _c
81+}
82+
83+func (_c *mocktransport_Close_Call) Return(err error) *mocktransport_Close_Call {
84+ _c.Call.Return(err)
85+ return _c
86+}
87+
88+func (_c *mocktransport_Close_Call) RunAndReturn(run func() error) *mocktransport_Close_Call {
89+ _c.Call.Return(run)
90+ return _c
91+}
92+
93+// Do provides a mock function for the type mocktransport
94+func (_mock *mocktransport) Do(deadline time.Time, idle time.Duration, calls []call) []reply {
95+ ret := _mock.Called(deadline, idle, calls)
96+
97+ if len(ret) == 0 {
98+ panic("no return value specified for Do")
99+ }
100+
101+ var r0 []reply
102+ if returnFunc, ok := ret.Get(0).(func(time.Time, time.Duration, []call) []reply); ok {
103+ r0 = returnFunc(deadline, idle, calls)
104+ } else {
105+ if ret.Get(0) != nil {
106+ r0 = ret.Get(0).([]reply)
107+ }
108+ }
109+ return r0
110+}
111+
112+// mocktransport_Do_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Do'
113+type mocktransport_Do_Call struct {
114+ *mock.Call
115+}
116+
117+// Do is a helper method to define mock.On call
118+// - deadline time.Time
119+// - idle time.Duration
120+// - calls []call
121+func (_e *mocktransport_Expecter) Do(deadline any, idle any, calls any) *mocktransport_Do_Call {
122+ return &mocktransport_Do_Call{Call: _e.mock.On("Do", deadline, idle, calls)}
123+}
124+
125+func (_c *mocktransport_Do_Call) Run(run func(deadline time.Time, idle time.Duration, calls []call)) *mocktransport_Do_Call {
126+ _c.Call.Run(func(args mock.Arguments) {
127+ var arg0 time.Time
128+ if args[0] != nil {
129+ arg0 = args[0].(time.Time)
130+ }
131+ var arg1 time.Duration
132+ if args[1] != nil {
133+ arg1 = args[1].(time.Duration)
134+ }
135+ var arg2 []call
136+ if args[2] != nil {
137+ arg2 = args[2].([]call)
138+ }
139+ run(
140+ arg0,
141+ arg1,
142+ arg2,
143+ )
144+ })
145+ return _c
146+}
147+
148+func (_c *mocktransport_Do_Call) Return(replyMoqParams []reply) *mocktransport_Do_Call {
149+ _c.Call.Return(replyMoqParams)
150+ return _c
151+}
152+
153+func (_c *mocktransport_Do_Call) RunAndReturn(run func(deadline time.Time, idle time.Duration, calls []call) []reply) *mocktransport_Do_Call {
154+ _c.Call.Return(run)
155+ return _c
156+}
@@ -0,0 +1,156 @@
1+// Code generated by mockery; DO NOT EDIT.
2+// github.com/vektra/mockery
3+// template: testify
4+
5+package atspi
6+
7+import (
8+ "time"
9+
10+ mock "github.com/stretchr/testify/mock"
11+)
12+
13+// newMocktransport creates a new instance of mocktransport. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
14+// The first argument is typically a *testing.T value.
15+func newMocktransport(t interface {
16+ mock.TestingT
17+ Cleanup(func())
18+}) *mocktransport {
19+ if helper, ok := t.(interface{ Helper() }); ok {
20+ helper.Helper()
21+ }
22+
23+ mock := &mocktransport{}
24+ mock.Mock.Test(t)
25+
26+ t.Cleanup(func() {
27+ if helper, ok := t.(interface{ Helper() }); ok {
28+ helper.Helper()
29+ }
30+ mock.AssertExpectations(t)
31+ })
32+
33+ return mock
34+}
35+
36+// mocktransport is an autogenerated mock type for the transport type
37+type mocktransport struct {
38+ mock.Mock
39+}
40+
41+type mocktransport_Expecter struct {
42+ mock *mock.Mock
43+}
44+
45+func (_m *mocktransport) EXPECT() *mocktransport_Expecter {
46+ return &mocktransport_Expecter{mock: &_m.Mock}
47+}
48+
49+// Close provides a mock function for the type mocktransport
50+func (_mock *mocktransport) Close() error {
51+ ret := _mock.Called()
52+
53+ if len(ret) == 0 {
54+ panic("no return value specified for Close")
55+ }
56+
57+ var r0 error
58+ if returnFunc, ok := ret.Get(0).(func() error); ok {
59+ r0 = returnFunc()
60+ } else {
61+ r0 = ret.Error(0)
62+ }
63+ return r0
64+}
65+
66+// mocktransport_Close_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Close'
67+type mocktransport_Close_Call struct {
68+ *mock.Call
69+}
70+
71+// Close is a helper method to define mock.On call
72+func (_e *mocktransport_Expecter) Close() *mocktransport_Close_Call {
73+ return &mocktransport_Close_Call{Call: _e.mock.On("Close")}
74+}
75+
76+func (_c *mocktransport_Close_Call) Run(run func()) *mocktransport_Close_Call {
77+ _c.Call.Run(func(args mock.Arguments) {
78+ run()
79+ })
80+ return _c
81+}
82+
83+func (_c *mocktransport_Close_Call) Return(err error) *mocktransport_Close_Call {
84+ _c.Call.Return(err)
85+ return _c
86+}
87+
88+func (_c *mocktransport_Close_Call) RunAndReturn(run func() error) *mocktransport_Close_Call {
89+ _c.Call.Return(run)
90+ return _c
91+}
92+
93+// Do provides a mock function for the type mocktransport
94+func (_mock *mocktransport) Do(deadline time.Time, idle time.Duration, calls []call) []reply {
95+ ret := _mock.Called(deadline, idle, calls)
96+
97+ if len(ret) == 0 {
98+ panic("no return value specified for Do")
99+ }
100+
101+ var r0 []reply
102+ if returnFunc, ok := ret.Get(0).(func(time.Time, time.Duration, []call) []reply); ok {
103+ r0 = returnFunc(deadline, idle, calls)
104+ } else {
105+ if ret.Get(0) != nil {
106+ r0 = ret.Get(0).([]reply)
107+ }
108+ }
109+ return r0
110+}
111+
112+// mocktransport_Do_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Do'
113+type mocktransport_Do_Call struct {
114+ *mock.Call
115+}
116+
117+// Do is a helper method to define mock.On call
118+// - deadline time.Time
119+// - idle time.Duration
120+// - calls []call
121+func (_e *mocktransport_Expecter) Do(deadline any, idle any, calls any) *mocktransport_Do_Call {
122+ return &mocktransport_Do_Call{Call: _e.mock.On("Do", deadline, idle, calls)}
123+}
124+
125+func (_c *mocktransport_Do_Call) Run(run func(deadline time.Time, idle time.Duration, calls []call)) *mocktransport_Do_Call {
126+ _c.Call.Run(func(args mock.Arguments) {
127+ var arg0 time.Time
128+ if args[0] != nil {
129+ arg0 = args[0].(time.Time)
130+ }
131+ var arg1 time.Duration
132+ if args[1] != nil {
133+ arg1 = args[1].(time.Duration)
134+ }
135+ var arg2 []call
136+ if args[2] != nil {
137+ arg2 = args[2].([]call)
138+ }
139+ run(
140+ arg0,
141+ arg1,
142+ arg2,
143+ )
144+ })
145+ return _c
146+}
147+
148+func (_c *mocktransport_Do_Call) Return(replyMoqParams []reply) *mocktransport_Do_Call {
149+ _c.Call.Return(replyMoqParams)
150+ return _c
151+}
152+
153+func (_c *mocktransport_Do_Call) RunAndReturn(run func(deadline time.Time, idle time.Duration, calls []call) []reply) *mocktransport_Do_Call {
154+ _c.Call.Return(run)
155+ return _c
156+}
M internal/adapters/config/config.go +74 -0
@@ -114,6 +114,7 @@ var actions = map[string]bool{
114114 "move-workspace-prev": true,
115115 "move-workspace-next": true,
116116 "toggle-floating": true,
117+ "hint": true,
117118 }
118119 var components = map[string]bool{"core": true, "wayland": true, "input": true, "drm": true, "seat": true, "render": true, "sync": true, "config": true, "app": true}
119120 var color = regexp.MustCompile(`^#[0-9a-fA-F]{6}$`)
@@ -163,6 +164,9 @@ func Defaults() ports.Config {
163164 c.Terminal.Command = []string{"foot"}
164165 }
165166 c.Terminal.AutoOpen = "first"
167+ // Hint mode is experimental: off unless the config turns it on.
168+ c.Hint.Enable = false
169+ c.Hint.TextClick = ports.ModCtrl | ports.ModShift
166170 c.Xwayland = "xwayland-satellite"
167171 c.Background.Color = "#111111"
168172 c.Floating.Dim = 0.3
@@ -847,6 +851,46 @@ func set(c *ports.Config, key, v string) error {
847851 return err
848852 }
849853 c.Focus.Flash = b
854+ case "hint.enable":
855+ b, err := onOff(v)
856+ if err != nil {
857+ return err
858+ }
859+ c.Hint.Enable = b
860+ case "hint.size":
861+ if v == "auto" {
862+ c.Hint.Size = 0
863+ break
864+ }
865+ n, err := strconv.ParseFloat(v, 64)
866+ if err != nil || math.IsNaN(n) || n < 6 || n > 72 {
867+ return fmt.Errorf("must be auto or a number from 6 to 72 (logical pixels)")
868+ }
869+ c.Hint.Size = n
870+ case "hint.background", "hint.foreground":
871+ if v != "" && !color.MatchString(v) {
872+ return fmt.Errorf("must be #rrggbb or empty (derived)")
873+ }
874+ if key == "hint.background" {
875+ c.Hint.Background = v
876+ } else {
877+ c.Hint.Foreground = v
878+ }
879+ case "hint.targets":
880+ switch v {
881+ case "clickable":
882+ c.Hint.LinksOnly = false
883+ case "links":
884+ c.Hint.LinksOnly = true
885+ default:
886+ return fmt.Errorf("must be clickable or links")
887+ }
888+ case "hint.text-click":
889+ m, err := parseTextClick(v)
890+ if err != nil {
891+ return err
892+ }
893+ c.Hint.TextClick = m
850894 case "focus.flash-width":
851895 n, err := strconv.Atoi(v)
852896 if err != nil || n < 1 || n > 8 {
@@ -1270,6 +1314,36 @@ func checkAction(v string) error {
12701314 return nil
12711315 }
12721316
1317+// parseTextClick parses hint.text-click: `+`-separated modifier names as in
1318+// binds, or none. cmd is an alias of super here (it is not resolved through
1319+// keyboard.cmd, so the result does not depend on key order).
1320+func parseTextClick(v string) (ports.Mods, error) {
1321+ if strings.ToLower(v) == "none" {
1322+ return 0, nil
1323+ }
1324+ var mods ports.Mods
1325+ for _, p := range strings.Split(v, "+") {
1326+ var m ports.Mods
1327+ switch strings.ToLower(strings.TrimSpace(p)) {
1328+ case "shift":
1329+ m = ports.ModShift
1330+ case "ctrl":
1331+ m = ports.ModCtrl
1332+ case "alt":
1333+ m = ports.ModAlt
1334+ case "super", "cmd":
1335+ m = ports.ModSuper
1336+ default:
1337+ return 0, fmt.Errorf("unknown modifier %q (use shift, ctrl, alt, super or cmd joined by +, or none)", p)
1338+ }
1339+ if mods&m != 0 {
1340+ return 0, fmt.Errorf("duplicate modifier %q", p)
1341+ }
1342+ mods |= m
1343+ }
1344+ return mods, nil
1345+}
1346+
12731347 // parseCombo turns `ctrl+cmd+é` into the canonical `Cmd+Ctrl+eacute`:
12741348 // modifiers sorted, the key as an xkb keysym name.
12751349 func parseCombo(s string) (string, error) {
@@ -114,6 +114,7 @@ var actions = map[string]bool{
114 "move-workspace-prev": true,114 "move-workspace-prev": true,
115 "move-workspace-next": true,115 "move-workspace-next": true,
116 "toggle-floating": true,116 "toggle-floating": true,
117+ "hint": true,
117 }118 }
118 var components = map[string]bool{"core": true, "wayland": true, "input": true, "drm": true, "seat": true, "render": true, "sync": true, "config": true, "app": true}119 var components = map[string]bool{"core": true, "wayland": true, "input": true, "drm": true, "seat": true, "render": true, "sync": true, "config": true, "app": true}
119 var color = regexp.MustCompile(`^#[0-9a-fA-F]{6}$`)120 var color = regexp.MustCompile(`^#[0-9a-fA-F]{6}$`)
@@ -163,6 +164,9 @@ func Defaults() ports.Config {
163 c.Terminal.Command = []string{"foot"}164 c.Terminal.Command = []string{"foot"}
164 }165 }
165 c.Terminal.AutoOpen = "first"166 c.Terminal.AutoOpen = "first"
167+ // Hint mode is experimental: off unless the config turns it on.
168+ c.Hint.Enable = false
169+ c.Hint.TextClick = ports.ModCtrl | ports.ModShift
166 c.Xwayland = "xwayland-satellite"170 c.Xwayland = "xwayland-satellite"
167 c.Background.Color = "#111111"171 c.Background.Color = "#111111"
168 c.Floating.Dim = 0.3172 c.Floating.Dim = 0.3
@@ -847,6 +851,46 @@ func set(c *ports.Config, key, v string) error {
847 return err851 return err
848 }852 }
849 c.Focus.Flash = b853 c.Focus.Flash = b
854+ case "hint.enable":
855+ b, err := onOff(v)
856+ if err != nil {
857+ return err
858+ }
859+ c.Hint.Enable = b
860+ case "hint.size":
861+ if v == "auto" {
862+ c.Hint.Size = 0
863+ break
864+ }
865+ n, err := strconv.ParseFloat(v, 64)
866+ if err != nil || math.IsNaN(n) || n < 6 || n > 72 {
867+ return fmt.Errorf("must be auto or a number from 6 to 72 (logical pixels)")
868+ }
869+ c.Hint.Size = n
870+ case "hint.background", "hint.foreground":
871+ if v != "" && !color.MatchString(v) {
872+ return fmt.Errorf("must be #rrggbb or empty (derived)")
873+ }
874+ if key == "hint.background" {
875+ c.Hint.Background = v
876+ } else {
877+ c.Hint.Foreground = v
878+ }
879+ case "hint.targets":
880+ switch v {
881+ case "clickable":
882+ c.Hint.LinksOnly = false
883+ case "links":
884+ c.Hint.LinksOnly = true
885+ default:
886+ return fmt.Errorf("must be clickable or links")
887+ }
888+ case "hint.text-click":
889+ m, err := parseTextClick(v)
890+ if err != nil {
891+ return err
892+ }
893+ c.Hint.TextClick = m
850 case "focus.flash-width":894 case "focus.flash-width":
851 n, err := strconv.Atoi(v)895 n, err := strconv.Atoi(v)
852 if err != nil || n < 1 || n > 8 {896 if err != nil || n < 1 || n > 8 {
@@ -1270,6 +1314,36 @@ func checkAction(v string) error {
1270 return nil1314 return nil
1271 }1315 }
1272 1316
1317+// parseTextClick parses hint.text-click: `+`-separated modifier names as in
1318+// binds, or none. cmd is an alias of super here (it is not resolved through
1319+// keyboard.cmd, so the result does not depend on key order).
1320+func parseTextClick(v string) (ports.Mods, error) {
1321+ if strings.ToLower(v) == "none" {
1322+ return 0, nil
1323+ }
1324+ var mods ports.Mods
1325+ for _, p := range strings.Split(v, "+") {
1326+ var m ports.Mods
1327+ switch strings.ToLower(strings.TrimSpace(p)) {
1328+ case "shift":
1329+ m = ports.ModShift
1330+ case "ctrl":
1331+ m = ports.ModCtrl
1332+ case "alt":
1333+ m = ports.ModAlt
1334+ case "super", "cmd":
1335+ m = ports.ModSuper
1336+ default:
1337+ return 0, fmt.Errorf("unknown modifier %q (use shift, ctrl, alt, super or cmd joined by +, or none)", p)
1338+ }
1339+ if mods&m != 0 {
1340+ return 0, fmt.Errorf("duplicate modifier %q", p)
1341+ }
1342+ mods |= m
1343+ }
1344+ return mods, nil
1345+}
1346+
1273 // parseCombo turns `ctrl+cmd+é` into the canonical `Cmd+Ctrl+eacute`:1347 // parseCombo turns `ctrl+cmd+é` into the canonical `Cmd+Ctrl+eacute`:
1274 // modifiers sorted, the key as an xkb keysym name.1348 // modifiers sorted, the key as an xkb keysym name.
1275 func parseCombo(s string) (string, error) {1349 func parseCombo(s string) (string, error) {
M internal/adapters/config/config_test.go +92 -0
@@ -953,3 +953,95 @@ func TestLoadRejectsFIFO(t *testing.T) {
953953 t.Fatal("Load blocked on a FIFO")
954954 }
955955 }
956+
957+func TestHintTargets(t *testing.T) {
958+ if Defaults().Hint.LinksOnly {
959+ t.Fatal("hint targets default must be clickable")
960+ }
961+ c, w := parseString(t, "hint.targets = links\n")
962+ if !c.Hint.LinksOnly || len(w) != 0 {
963+ t.Fatal(c.Hint, w)
964+ }
965+ c, w = parseString(t, "hint.targets = clickable\n")
966+ if c.Hint.LinksOnly || len(w) != 0 {
967+ t.Fatal(c.Hint, w)
968+ }
969+ // An invalid value warns and keeps the default.
970+ c, w = parseString(t, "hint.targets = buttons\n")
971+ if c.Hint.LinksOnly || len(w) != 1 {
972+ t.Fatal(c.Hint, w)
973+ }
974+ c, w = parseString(t, "bind.cmd+g = hint\n")
975+ if len(w) != 0 || c.Binds["Cmd+g"] != "hint" {
976+ t.Fatal(c.Binds["Cmd+g"], w)
977+ }
978+}
979+
980+func TestHintStyleConfig(t *testing.T) {
981+ c, warnings := parseString(t, "hint.size = 17.5\nhint.background = #89b4fa\nhint.foreground = #111111\n")
982+ if len(warnings) != 0 || c.Hint.Size != 17.5 || c.Hint.Background != "#89b4fa" || c.Hint.Foreground != "#111111" {
983+ t.Fatalf("style: %+v, warnings: %v", c.Hint, warnings)
984+ }
985+ c, warnings = parseString(t, "hint.size = auto\nhint.background =\nhint.foreground =\n")
986+ if len(warnings) != 0 || c.Hint.Size != 0 || c.Hint.Background != "" || c.Hint.Foreground != "" {
987+ t.Fatalf("automatic: %+v %v", c.Hint, warnings)
988+ }
989+ for _, line := range []string{"hint.size = 0", "hint.size = -1", "hint.size = NaN", "hint.size = Inf", "hint.size = huge", "hint.size = 5.9", "hint.size = 72.1", "hint.size = 1e9", "hint.background = red", "hint.foreground = #fff"} {
990+ _, warnings := parseString(t, line+"\n")
991+ if len(warnings) == 0 {
992+ t.Errorf("expected validate-config warning for %s", line)
993+ }
994+ }
995+}
996+
997+func TestHintSizeBounds(t *testing.T) {
998+ for _, v := range []string{"6", "72", "13"} {
999+ c, w := parseString(t, "hint.size = "+v+"\n")
1000+ if len(w) != 0 || c.Hint.Size == 0 {
1001+ t.Errorf("hint.size = %s: %+v %v", v, c.Hint, w)
1002+ }
1003+ }
1004+}
1005+
1006+func TestHintTextClick(t *testing.T) {
1007+ if got := Defaults().Hint.TextClick; got != ports.ModCtrl|ports.ModShift {
1008+ t.Fatalf("default %d", got)
1009+ }
1010+ for in, want := range map[string]ports.Mods{
1011+ "ctrl+shift": ports.ModCtrl | ports.ModShift,
1012+ "Alt": ports.ModAlt,
1013+ "super": ports.ModSuper,
1014+ "cmd+shift": ports.ModSuper | ports.ModShift,
1015+ "none": 0,
1016+ "shift+ctrl+alt": ports.ModShift | ports.ModCtrl | ports.ModAlt,
1017+ } {
1018+ c, w := parseString(t, "hint.text-click = "+in+"\n")
1019+ if c.Hint.TextClick != want || len(w) != 0 {
1020+ t.Errorf("%q: %d %v", in, c.Hint.TextClick, w)
1021+ }
1022+ }
1023+ for _, bad := range []string{"hyper", "ctrl+", "ctrl+ctrl", "ctrl+none", ""} {
1024+ c, w := parseString(t, "hint.text-click = "+bad+"\n")
1025+ if len(w) != 1 || c.Hint.TextClick != ports.ModCtrl|ports.ModShift {
1026+ t.Errorf("%q: %d %v", bad, c.Hint.TextClick, w)
1027+ }
1028+ }
1029+}
1030+
1031+func TestHintEnable(t *testing.T) {
1032+ if Defaults().Hint.Enable {
1033+ t.Fatal("hint mode must be off by default")
1034+ }
1035+ c, w := parseString(t, "hint.enable = on\n")
1036+ if !c.Hint.Enable || len(w) != 0 {
1037+ t.Fatal(c.Hint, w)
1038+ }
1039+ c, w = parseString(t, "hint.enable = off\n")
1040+ if c.Hint.Enable || len(w) != 0 {
1041+ t.Fatal(c.Hint, w)
1042+ }
1043+ c, w = parseString(t, "hint.enable = maybe\n")
1044+ if c.Hint.Enable || len(w) != 1 {
1045+ t.Fatal(c.Hint, w)
1046+ }
1047+}
@@ -953,3 +953,95 @@ func TestLoadRejectsFIFO(t *testing.T) {
953 t.Fatal("Load blocked on a FIFO")953 t.Fatal("Load blocked on a FIFO")
954 }954 }
955 }955 }
956+
957+func TestHintTargets(t *testing.T) {
958+ if Defaults().Hint.LinksOnly {
959+ t.Fatal("hint targets default must be clickable")
960+ }
961+ c, w := parseString(t, "hint.targets = links\n")
962+ if !c.Hint.LinksOnly || len(w) != 0 {
963+ t.Fatal(c.Hint, w)
964+ }
965+ c, w = parseString(t, "hint.targets = clickable\n")
966+ if c.Hint.LinksOnly || len(w) != 0 {
967+ t.Fatal(c.Hint, w)
968+ }
969+ // An invalid value warns and keeps the default.
970+ c, w = parseString(t, "hint.targets = buttons\n")
971+ if c.Hint.LinksOnly || len(w) != 1 {
972+ t.Fatal(c.Hint, w)
973+ }
974+ c, w = parseString(t, "bind.cmd+g = hint\n")
975+ if len(w) != 0 || c.Binds["Cmd+g"] != "hint" {
976+ t.Fatal(c.Binds["Cmd+g"], w)
977+ }
978+}
979+
980+func TestHintStyleConfig(t *testing.T) {
981+ c, warnings := parseString(t, "hint.size = 17.5\nhint.background = #89b4fa\nhint.foreground = #111111\n")
982+ if len(warnings) != 0 || c.Hint.Size != 17.5 || c.Hint.Background != "#89b4fa" || c.Hint.Foreground != "#111111" {
983+ t.Fatalf("style: %+v, warnings: %v", c.Hint, warnings)
984+ }
985+ c, warnings = parseString(t, "hint.size = auto\nhint.background =\nhint.foreground =\n")
986+ if len(warnings) != 0 || c.Hint.Size != 0 || c.Hint.Background != "" || c.Hint.Foreground != "" {
987+ t.Fatalf("automatic: %+v %v", c.Hint, warnings)
988+ }
989+ for _, line := range []string{"hint.size = 0", "hint.size = -1", "hint.size = NaN", "hint.size = Inf", "hint.size = huge", "hint.size = 5.9", "hint.size = 72.1", "hint.size = 1e9", "hint.background = red", "hint.foreground = #fff"} {
990+ _, warnings := parseString(t, line+"\n")
991+ if len(warnings) == 0 {
992+ t.Errorf("expected validate-config warning for %s", line)
993+ }
994+ }
995+}
996+
997+func TestHintSizeBounds(t *testing.T) {
998+ for _, v := range []string{"6", "72", "13"} {
999+ c, w := parseString(t, "hint.size = "+v+"\n")
1000+ if len(w) != 0 || c.Hint.Size == 0 {
1001+ t.Errorf("hint.size = %s: %+v %v", v, c.Hint, w)
1002+ }
1003+ }
1004+}
1005+
1006+func TestHintTextClick(t *testing.T) {
1007+ if got := Defaults().Hint.TextClick; got != ports.ModCtrl|ports.ModShift {
1008+ t.Fatalf("default %d", got)
1009+ }
1010+ for in, want := range map[string]ports.Mods{
1011+ "ctrl+shift": ports.ModCtrl | ports.ModShift,
1012+ "Alt": ports.ModAlt,
1013+ "super": ports.ModSuper,
1014+ "cmd+shift": ports.ModSuper | ports.ModShift,
1015+ "none": 0,
1016+ "shift+ctrl+alt": ports.ModShift | ports.ModCtrl | ports.ModAlt,
1017+ } {
1018+ c, w := parseString(t, "hint.text-click = "+in+"\n")
1019+ if c.Hint.TextClick != want || len(w) != 0 {
1020+ t.Errorf("%q: %d %v", in, c.Hint.TextClick, w)
1021+ }
1022+ }
1023+ for _, bad := range []string{"hyper", "ctrl+", "ctrl+ctrl", "ctrl+none", ""} {
1024+ c, w := parseString(t, "hint.text-click = "+bad+"\n")
1025+ if len(w) != 1 || c.Hint.TextClick != ports.ModCtrl|ports.ModShift {
1026+ t.Errorf("%q: %d %v", bad, c.Hint.TextClick, w)
1027+ }
1028+ }
1029+}
1030+
1031+func TestHintEnable(t *testing.T) {
1032+ if Defaults().Hint.Enable {
1033+ t.Fatal("hint mode must be off by default")
1034+ }
1035+ c, w := parseString(t, "hint.enable = on\n")
1036+ if !c.Hint.Enable || len(w) != 0 {
1037+ t.Fatal(c.Hint, w)
1038+ }
1039+ c, w = parseString(t, "hint.enable = off\n")
1040+ if c.Hint.Enable || len(w) != 0 {
1041+ t.Fatal(c.Hint, w)
1042+ }
1043+ c, w = parseString(t, "hint.enable = maybe\n")
1044+ if c.Hint.Enable || len(w) != 1 {
1045+ t.Fatal(c.Hint, w)
1046+ }
1047+}
M internal/adapters/drm/overlay.go +3 -0
@@ -70,6 +70,9 @@ func overlayCandidate(s ports.Scene, surfaces map[ports.WindowID]ports.SurfaceCo
7070 if s.Flash != nil {
7171 return ports.SceneWindow{}, ports.SurfaceContent{}, colorBypass, "focus_flash"
7272 }
73+ if len(s.HintLabels) > 0 {
74+ return ports.SceneWindow{}, ports.SurfaceContent{}, colorBypass, "hint_labels"
75+ }
7376 if s.Transform != 0 {
7477 return ports.SceneWindow{}, ports.SurfaceContent{}, colorBypass, "output_transform"
7578 }
@@ -70,6 +70,9 @@ func overlayCandidate(s ports.Scene, surfaces map[ports.WindowID]ports.SurfaceCo
70 if s.Flash != nil {70 if s.Flash != nil {
71 return ports.SceneWindow{}, ports.SurfaceContent{}, colorBypass, "focus_flash"71 return ports.SceneWindow{}, ports.SurfaceContent{}, colorBypass, "focus_flash"
72 }72 }
73+ if len(s.HintLabels) > 0 {
74+ return ports.SceneWindow{}, ports.SurfaceContent{}, colorBypass, "hint_labels"
75+ }
73 if s.Transform != 0 {76 if s.Transform != 0 {
74 return ports.SceneWindow{}, ports.SurfaceContent{}, colorBypass, "output_transform"77 return ports.SceneWindow{}, ports.SurfaceContent{}, colorBypass, "output_transform"
75 }78 }
M internal/adapters/drm/scanout.go +4 -0
@@ -94,6 +94,10 @@ func scanoutCandidate(s ports.Scene, surfaces map[ports.WindowID]ports.SurfaceCo
9494 if s.Flash != nil {
9595 return ports.SurfaceContent{}, "focus_flash"
9696 }
97+ // Hint labels are composed over the window.
98+ if len(s.HintLabels) > 0 {
99+ return ports.SurfaceContent{}, "hint_labels"
100+ }
97101 if s.Transform != 0 {
98102 // The scanout buffer would have to be pre-rotated: rotated outputs compose.
99103 return ports.SurfaceContent{}, "output_transform"
@@ -94,6 +94,10 @@ func scanoutCandidate(s ports.Scene, surfaces map[ports.WindowID]ports.SurfaceCo
94 if s.Flash != nil {94 if s.Flash != nil {
95 return ports.SurfaceContent{}, "focus_flash"95 return ports.SurfaceContent{}, "focus_flash"
96 }96 }
97+ // Hint labels are composed over the window.
98+ if len(s.HintLabels) > 0 {
99+ return ports.SurfaceContent{}, "hint_labels"
100+ }
97 if s.Transform != 0 {101 if s.Transform != 0 {
98 // The scanout buffer would have to be pre-rotated: rotated outputs compose.102 // The scanout buffer would have to be pre-rotated: rotated outputs compose.
99 return ports.SurfaceContent{}, "output_transform"103 return ports.SurfaceContent{}, "output_transform"
A internal/adapters/hintlabel/hintlabel.go +137 -0
@@ -0,0 +1,137 @@
1+// Package hintlabel rasterizes the keyboard hint labels on the CPU: a small
2+// box with the label text in Go Mono, as premultiplied BGRA pixels for the
3+// renderer to upload and draw.
4+package hintlabel
5+
6+import (
7+ "image"
8+ "image/color"
9+ "image/draw"
10+ "math"
11+ "sync"
12+
13+ "golang.org/x/image/font"
14+ "golang.org/x/image/font/gofont/gomono"
15+ "golang.org/x/image/font/opentype"
16+ "golang.org/x/image/math/fixed"
17+)
18+
19+const (
20+ // fontSize is the text size in logical pixels.
21+ fontSize = 13
22+ // padX and padY are the padding around the text in logical pixels.
23+ padX, padY = 3, 1
24+ // maxEntries bounds the cache: it is cleared when it grows past it.
25+ maxEntries = 1024
26+)
27+
28+var (
29+ parseOnce sync.Once
30+ parsed *opentype.Font
31+ parseErr error
32+)
33+
34+type key struct {
35+ text string
36+ typed int
37+ scale float64
38+ style Style
39+}
40+
41+type entry struct {
42+ pix []byte
43+ w, h int
44+}
45+
46+// Raster draws and caches labels. It is not safe for concurrent use: the
47+// renderer owns it.
48+type Raster struct {
49+ faces map[float64]font.Face
50+ cache map[key]entry
51+}
52+
53+// Label returns the label box for text, whose first typed characters are
54+// drawn dimmed, as premultiplied BGRA at scale physical pixels per logical
55+// pixel. The slice is owned by the cache and stays valid until the cache is
56+// cleared by a later call: do not modify it. Empty text, or a font that does
57+// not load, yields a nil slice.
58+func (r *Raster) Label(text string, typed int, scale float64, styles ...Style) (pix []byte, w, h int) {
59+ if text == "" {
60+ return nil, 0, 0
61+ }
62+ if scale <= 0 || math.IsNaN(scale) || math.IsInf(scale, 0) {
63+ scale = 1
64+ }
65+ typed = max(0, min(typed, len(text)))
66+ bg, fg := Colors("", "", "")
67+ style := Style{fontSize * scale, bg, fg}
68+ if len(styles) > 0 {
69+ style = styles[0]
70+ }
71+ k := key{text, typed, scale, style}
72+ if e, ok := r.cache[k]; ok {
73+ return e.pix, e.w, e.h
74+ }
75+ face := r.face(style.FontPx)
76+ if face == nil {
77+ return nil, 0, 0
78+ }
79+ if r.cache == nil || len(r.cache) >= maxEntries {
80+ r.cache = map[key]entry{}
81+ }
82+ e := render(face, text, typed, style)
83+ r.cache[k] = e
84+ return e.pix, e.w, e.h
85+}
86+
87+func (r *Raster) face(scale float64) font.Face {
88+ if f, ok := r.faces[scale]; ok {
89+ return f
90+ }
91+ parseOnce.Do(func() { parsed, parseErr = opentype.Parse(gomono.TTF) })
92+ if parseErr != nil {
93+ return nil
94+ }
95+ f, err := opentype.NewFace(parsed, &opentype.FaceOptions{Size: scale, DPI: 72, Hinting: font.HintingFull})
96+ if err != nil {
97+ return nil
98+ }
99+ if r.faces == nil || len(r.faces) >= 16 {
100+ for _, old := range r.faces {
101+ _ = old.Close()
102+ }
103+ r.faces = map[float64]font.Face{}
104+ }
105+ r.faces[scale] = f
106+ return f
107+}
108+
109+// render draws an opaque box inside a one-physical-pixel translucent outline.
110+// All colours are premultiplied.
111+func render(face font.Face, text string, typed int, style Style) entry {
112+ scale := style.FontPx / fontSize
113+ background, ink := style.Background, style.Foreground
114+ typedInk := mix(ink, background)
115+ outline := color.RGBA{uint8(math.Round(float64(ink.R) * .35)), uint8(math.Round(float64(ink.G) * .35)), uint8(math.Round(float64(ink.B) * .35)), 89}
116+ m := face.Metrics()
117+ px := int(math.Round(padX * scale))
118+ py := int(math.Round(padY * scale))
119+ border := 1
120+ adv := font.MeasureString(face, text).Ceil()
121+ w := adv + 2*px + 2*border
122+ h := m.Height.Ceil() + 2*py + 2*border
123+ img := image.NewRGBA(image.Rect(0, 0, w, h))
124+ draw.Draw(img, img.Bounds(), image.NewUniform(outline), image.Point{}, draw.Src)
125+ draw.Draw(img, img.Bounds().Inset(border), image.NewUniform(background), image.Point{}, draw.Src)
126+ d := font.Drawer{Dst: img, Face: face, Dot: fixed.Point26_6{X: fixed.I(border + px), Y: fixed.I(border+py) + m.Ascent}}
127+ d.Src = image.NewUniform(typedInk)
128+ d.DrawString(text[:typed])
129+ d.Src = image.NewUniform(ink)
130+ d.DrawString(text[typed:])
131+ // Premultiplied RGBA to BGRA in place.
132+ pix := img.Pix
133+ for i := 0; i+3 < len(pix); i += 4 {
134+ pix[i], pix[i+2] = pix[i+2], pix[i]
135+ }
136+ return entry{pix: pix, w: w, h: h}
137+}
@@ -0,0 +1,137 @@
1+// Package hintlabel rasterizes the keyboard hint labels on the CPU: a small
2+// box with the label text in Go Mono, as premultiplied BGRA pixels for the
3+// renderer to upload and draw.
4+package hintlabel
5+
6+import (
7+ "image"
8+ "image/color"
9+ "image/draw"
10+ "math"
11+ "sync"
12+
13+ "golang.org/x/image/font"
14+ "golang.org/x/image/font/gofont/gomono"
15+ "golang.org/x/image/font/opentype"
16+ "golang.org/x/image/math/fixed"
17+)
18+
19+const (
20+ // fontSize is the text size in logical pixels.
21+ fontSize = 13
22+ // padX and padY are the padding around the text in logical pixels.
23+ padX, padY = 3, 1
24+ // maxEntries bounds the cache: it is cleared when it grows past it.
25+ maxEntries = 1024
26+)
27+
28+var (
29+ parseOnce sync.Once
30+ parsed *opentype.Font
31+ parseErr error
32+)
33+
34+type key struct {
35+ text string
36+ typed int
37+ scale float64
38+ style Style
39+}
40+
41+type entry struct {
42+ pix []byte
43+ w, h int
44+}
45+
46+// Raster draws and caches labels. It is not safe for concurrent use: the
47+// renderer owns it.
48+type Raster struct {
49+ faces map[float64]font.Face
50+ cache map[key]entry
51+}
52+
53+// Label returns the label box for text, whose first typed characters are
54+// drawn dimmed, as premultiplied BGRA at scale physical pixels per logical
55+// pixel. The slice is owned by the cache and stays valid until the cache is
56+// cleared by a later call: do not modify it. Empty text, or a font that does
57+// not load, yields a nil slice.
58+func (r *Raster) Label(text string, typed int, scale float64, styles ...Style) (pix []byte, w, h int) {
59+ if text == "" {
60+ return nil, 0, 0
61+ }
62+ if scale <= 0 || math.IsNaN(scale) || math.IsInf(scale, 0) {
63+ scale = 1
64+ }
65+ typed = max(0, min(typed, len(text)))
66+ bg, fg := Colors("", "", "")
67+ style := Style{fontSize * scale, bg, fg}
68+ if len(styles) > 0 {
69+ style = styles[0]
70+ }
71+ k := key{text, typed, scale, style}
72+ if e, ok := r.cache[k]; ok {
73+ return e.pix, e.w, e.h
74+ }
75+ face := r.face(style.FontPx)
76+ if face == nil {
77+ return nil, 0, 0
78+ }
79+ if r.cache == nil || len(r.cache) >= maxEntries {
80+ r.cache = map[key]entry{}
81+ }
82+ e := render(face, text, typed, style)
83+ r.cache[k] = e
84+ return e.pix, e.w, e.h
85+}
86+
87+func (r *Raster) face(scale float64) font.Face {
88+ if f, ok := r.faces[scale]; ok {
89+ return f
90+ }
91+ parseOnce.Do(func() { parsed, parseErr = opentype.Parse(gomono.TTF) })
92+ if parseErr != nil {
93+ return nil
94+ }
95+ f, err := opentype.NewFace(parsed, &opentype.FaceOptions{Size: scale, DPI: 72, Hinting: font.HintingFull})
96+ if err != nil {
97+ return nil
98+ }
99+ if r.faces == nil || len(r.faces) >= 16 {
100+ for _, old := range r.faces {
101+ _ = old.Close()
102+ }
103+ r.faces = map[float64]font.Face{}
104+ }
105+ r.faces[scale] = f
106+ return f
107+}
108+
109+// render draws an opaque box inside a one-physical-pixel translucent outline.
110+// All colours are premultiplied.
111+func render(face font.Face, text string, typed int, style Style) entry {
112+ scale := style.FontPx / fontSize
113+ background, ink := style.Background, style.Foreground
114+ typedInk := mix(ink, background)
115+ outline := color.RGBA{uint8(math.Round(float64(ink.R) * .35)), uint8(math.Round(float64(ink.G) * .35)), uint8(math.Round(float64(ink.B) * .35)), 89}
116+ m := face.Metrics()
117+ px := int(math.Round(padX * scale))
118+ py := int(math.Round(padY * scale))
119+ border := 1
120+ adv := font.MeasureString(face, text).Ceil()
121+ w := adv + 2*px + 2*border
122+ h := m.Height.Ceil() + 2*py + 2*border
123+ img := image.NewRGBA(image.Rect(0, 0, w, h))
124+ draw.Draw(img, img.Bounds(), image.NewUniform(outline), image.Point{}, draw.Src)
125+ draw.Draw(img, img.Bounds().Inset(border), image.NewUniform(background), image.Point{}, draw.Src)
126+ d := font.Drawer{Dst: img, Face: face, Dot: fixed.Point26_6{X: fixed.I(border + px), Y: fixed.I(border+py) + m.Ascent}}
127+ d.Src = image.NewUniform(typedInk)
128+ d.DrawString(text[:typed])
129+ d.Src = image.NewUniform(ink)
130+ d.DrawString(text[typed:])
131+ // Premultiplied RGBA to BGRA in place.
132+ pix := img.Pix
133+ for i := 0; i+3 < len(pix); i += 4 {
134+ pix[i], pix[i+2] = pix[i+2], pix[i]
135+ }
136+ return entry{pix: pix, w: w, h: h}
137+}
A internal/adapters/hintlabel/hintlabel_test.go +60 -0
@@ -0,0 +1,60 @@
1+package hintlabel
2+
3+import (
4+ "testing"
5+
6+ "github.com/stretchr/testify/require"
7+)
8+
9+func TestHintLabelSize(t *testing.T) {
10+ var r Raster
11+ _, w1, h1 := r.Label("f", 0, 1)
12+ _, w2, h2 := r.Label("fj", 0, 1)
13+ require.Greater(t, w1, 0)
14+ require.Greater(t, w2, w1, "a longer label is wider")
15+ require.Equal(t, h1, h2)
16+ _, w3, h3 := r.Label("fj", 0, 2)
17+ // The 1 px border does not scale: compare the boxes without it.
18+ require.InDelta(t, 2*(w2-2), w3-2, 1)
19+ require.InDelta(t, 2*(h2-2), h3-2, 1)
20+}
21+
22+func TestHintLabelCache(t *testing.T) {
23+ var r Raster
24+ a, w, h := r.Label("fj", 1, 1)
25+ b, _, _ := r.Label("fj", 1, 1)
26+ require.Len(t, a, w*h*4)
27+ require.Same(t, &a[0], &b[0], "same key returns the cached slice")
28+ c, _, _ := r.Label("fj", 0, 1)
29+ require.NotSame(t, &a[0], &c[0], "typed count is part of the key")
30+}
31+
32+func TestHintLabelPremultiplied(t *testing.T) {
33+ var r Raster
34+ pix, w, h := r.Label("ab", 1, 1.5)
35+ require.Len(t, pix, w*h*4)
36+ for i := 0; i < len(pix); i += 4 {
37+ a := pix[i+3]
38+ require.GreaterOrEqual(t, a, pix[i])
39+ require.GreaterOrEqual(t, a, pix[i+1])
40+ require.GreaterOrEqual(t, a, pix[i+2])
41+ }
42+}
43+
44+func TestHintLabelTypedIsDimmed(t *testing.T) {
45+ var r Raster
46+ plain, _, _ := r.Label("fj", 0, 1)
47+ dim, _, _ := r.Label("fj", 1, 1)
48+ require.NotEqual(t, plain, dim)
49+}
50+
51+func TestHintLabelEmptyAndCacheBound(t *testing.T) {
52+ var r Raster
53+ pix, w, h := r.Label("", 0, 1)
54+ require.Nil(t, pix)
55+ require.Zero(t, w+h)
56+ for i := 0; i < maxEntries+10; i++ {
57+ r.Label("a", 0, 1+float64(i)/100000)
58+ }
59+ require.LessOrEqual(t, len(r.cache), maxEntries)
60+}
@@ -0,0 +1,60 @@
1+package hintlabel
2+
3+import (
4+ "testing"
5+
6+ "github.com/stretchr/testify/require"
7+)
8+
9+func TestHintLabelSize(t *testing.T) {
10+ var r Raster
11+ _, w1, h1 := r.Label("f", 0, 1)
12+ _, w2, h2 := r.Label("fj", 0, 1)
13+ require.Greater(t, w1, 0)
14+ require.Greater(t, w2, w1, "a longer label is wider")
15+ require.Equal(t, h1, h2)
16+ _, w3, h3 := r.Label("fj", 0, 2)
17+ // The 1 px border does not scale: compare the boxes without it.
18+ require.InDelta(t, 2*(w2-2), w3-2, 1)
19+ require.InDelta(t, 2*(h2-2), h3-2, 1)
20+}
21+
22+func TestHintLabelCache(t *testing.T) {
23+ var r Raster
24+ a, w, h := r.Label("fj", 1, 1)
25+ b, _, _ := r.Label("fj", 1, 1)
26+ require.Len(t, a, w*h*4)
27+ require.Same(t, &a[0], &b[0], "same key returns the cached slice")
28+ c, _, _ := r.Label("fj", 0, 1)
29+ require.NotSame(t, &a[0], &c[0], "typed count is part of the key")
30+}
31+
32+func TestHintLabelPremultiplied(t *testing.T) {
33+ var r Raster
34+ pix, w, h := r.Label("ab", 1, 1.5)
35+ require.Len(t, pix, w*h*4)
36+ for i := 0; i < len(pix); i += 4 {
37+ a := pix[i+3]
38+ require.GreaterOrEqual(t, a, pix[i])
39+ require.GreaterOrEqual(t, a, pix[i+1])
40+ require.GreaterOrEqual(t, a, pix[i+2])
41+ }
42+}
43+
44+func TestHintLabelTypedIsDimmed(t *testing.T) {
45+ var r Raster
46+ plain, _, _ := r.Label("fj", 0, 1)
47+ dim, _, _ := r.Label("fj", 1, 1)
48+ require.NotEqual(t, plain, dim)
49+}
50+
51+func TestHintLabelEmptyAndCacheBound(t *testing.T) {
52+ var r Raster
53+ pix, w, h := r.Label("", 0, 1)
54+ require.Nil(t, pix)
55+ require.Zero(t, w+h)
56+ for i := 0; i < maxEntries+10; i++ {
57+ r.Label("a", 0, 1+float64(i)/100000)
58+ }
59+ require.LessOrEqual(t, len(r.cache), maxEntries)
60+}
A internal/adapters/hintlabel/style.go +102 -0
@@ -0,0 +1,102 @@
1+package hintlabel
2+
3+import (
4+ "image/color"
5+ "math"
6+ "strconv"
7+)
8+
9+// Style is the resolved, physical-pixel label style and is also a cache key.
10+type Style struct {
11+ FontPx float64
12+ Background, Foreground color.RGBA
13+}
14+
15+// FontPixels resolves an explicit logical size or the output's pixel density.
16+// Both millimetre dimensions must be plausible for the untransformed mode.
17+func FontPixels(width, height, mmWidth, mmHeight int, scale, logical float64) float64 {
18+ return min(maxFontPx, fontPixels(width, height, mmWidth, mmHeight, scale, logical))
19+}
20+
21+// maxFontPx bounds the physical font size, whatever the configuration or the
22+// scale ask for.
23+const maxFontPx = 256
24+
25+func fontPixels(width, height, mmWidth, mmHeight int, scale, logical float64) float64 {
26+ if scale <= 0 || math.IsNaN(scale) || math.IsInf(scale, 0) {
27+ scale = 1
28+ }
29+ if logical > 0 && !math.IsInf(logical, 0) {
30+ return logical * scale
31+ }
32+ if width > 0 && height > 0 && mmWidth > 0 && mmHeight > 0 {
33+ ratio := (float64(width) / float64(height)) / (float64(mmWidth) / float64(mmHeight))
34+ if math.Abs(ratio-1) <= .2 {
35+ return min(math.Round(40*scale), max(math.Round(13*scale), math.Round(3.2*float64(width)/float64(mmWidth))))
36+ }
37+ }
38+ return 13 * scale * math.Sqrt(max(1, float64(height)/(1080*scale)))
39+}
40+
41+func parseColor(s string, fallback color.RGBA) color.RGBA {
42+ if len(s) != 7 || s[0] != '#' {
43+ return fallback
44+ }
45+ n, err := strconv.ParseUint(s[1:], 16, 24)
46+ if err != nil {
47+ return fallback
48+ }
49+ return color.RGBA{uint8(n >> 16), uint8(n >> 8), uint8(n), 255}
50+}
51+
52+func luminance(c color.RGBA) float64 {
53+ linear := func(v uint8) float64 {
54+ x := float64(v) / 255
55+ if x <= .04045 {
56+ return x / 12.92
57+ }
58+ return math.Pow((x+.055)/1.055, 2.4)
59+ }
60+ return .2126*linear(c.R) + .7152*linear(c.G) + .0722*linear(c.B)
61+}
62+
63+func contrast(a, b color.RGBA) float64 {
64+ x, y := luminance(a), luminance(b)
65+ return (max(x, y) + .05) / (min(x, y) + .05)
66+}
67+
68+// Colors derives ink from the background. Near-black and near-white have a
69+// narrow luminance gap where neither meets AA; in that gap move HSL
70+// lightness away from .5, preserving hue and saturation.
71+// Explicit foreground overrides are honoured as supplied.
72+func Colors(active, background, foreground string) (color.RGBA, color.RGBA) {
73+ bg := parseColor(background, parseColor(active, color.RGBA{128, 128, 128, 255}))
74+ dark, light := color.RGBA{17, 17, 17, 255}, color.RGBA{245, 245, 245, 255}
75+ ink := dark
76+ if contrast(bg, light) > contrast(bg, dark) {
77+ ink = light
78+ }
79+ for contrast(bg, ink) < 4.5 {
80+ // Move HSL lightness toward zero while preserving hue and saturation.
81+ hi, lo := max(bg.R, bg.G, bg.B), min(bg.R, bg.G, bg.B)
82+ l := (float64(hi) + float64(lo)) / 510
83+ if l > .5 {
84+ // Above .5, moving toward white increases contrast with dark ink.
85+ bg.R += uint8(math.Ceil(float64(255-bg.R) * .01))
86+ bg.G += uint8(math.Ceil(float64(255-bg.G) * .01))
87+ bg.B += uint8(math.Ceil(float64(255-bg.B) * .01))
88+ ink = dark
89+ } else {
90+ bg.R = uint8(float64(bg.R) * .99)
91+ bg.G = uint8(float64(bg.G) * .99)
92+ bg.B = uint8(float64(bg.B) * .99)
93+ ink = light
94+ }
95+ }
96+ return bg, parseColor(foreground, ink)
97+}
98+
99+func mix(ink, bg color.RGBA) color.RGBA {
100+ channel := func(a, b uint8) uint8 { return uint8(math.Round(.45*float64(a) + .55*float64(b))) }
101+ return color.RGBA{channel(ink.R, bg.R), channel(ink.G, bg.G), channel(ink.B, bg.B), 255}
102+}
@@ -0,0 +1,102 @@
1+package hintlabel
2+
3+import (
4+ "image/color"
5+ "math"
6+ "strconv"
7+)
8+
9+// Style is the resolved, physical-pixel label style and is also a cache key.
10+type Style struct {
11+ FontPx float64
12+ Background, Foreground color.RGBA
13+}
14+
15+// FontPixels resolves an explicit logical size or the output's pixel density.
16+// Both millimetre dimensions must be plausible for the untransformed mode.
17+func FontPixels(width, height, mmWidth, mmHeight int, scale, logical float64) float64 {
18+ return min(maxFontPx, fontPixels(width, height, mmWidth, mmHeight, scale, logical))
19+}
20+
21+// maxFontPx bounds the physical font size, whatever the configuration or the
22+// scale ask for.
23+const maxFontPx = 256
24+
25+func fontPixels(width, height, mmWidth, mmHeight int, scale, logical float64) float64 {
26+ if scale <= 0 || math.IsNaN(scale) || math.IsInf(scale, 0) {
27+ scale = 1
28+ }
29+ if logical > 0 && !math.IsInf(logical, 0) {
30+ return logical * scale
31+ }
32+ if width > 0 && height > 0 && mmWidth > 0 && mmHeight > 0 {
33+ ratio := (float64(width) / float64(height)) / (float64(mmWidth) / float64(mmHeight))
34+ if math.Abs(ratio-1) <= .2 {
35+ return min(math.Round(40*scale), max(math.Round(13*scale), math.Round(3.2*float64(width)/float64(mmWidth))))
36+ }
37+ }
38+ return 13 * scale * math.Sqrt(max(1, float64(height)/(1080*scale)))
39+}
40+
41+func parseColor(s string, fallback color.RGBA) color.RGBA {
42+ if len(s) != 7 || s[0] != '#' {
43+ return fallback
44+ }
45+ n, err := strconv.ParseUint(s[1:], 16, 24)
46+ if err != nil {
47+ return fallback
48+ }
49+ return color.RGBA{uint8(n >> 16), uint8(n >> 8), uint8(n), 255}
50+}
51+
52+func luminance(c color.RGBA) float64 {
53+ linear := func(v uint8) float64 {
54+ x := float64(v) / 255
55+ if x <= .04045 {
56+ return x / 12.92
57+ }
58+ return math.Pow((x+.055)/1.055, 2.4)
59+ }
60+ return .2126*linear(c.R) + .7152*linear(c.G) + .0722*linear(c.B)
61+}
62+
63+func contrast(a, b color.RGBA) float64 {
64+ x, y := luminance(a), luminance(b)
65+ return (max(x, y) + .05) / (min(x, y) + .05)
66+}
67+
68+// Colors derives ink from the background. Near-black and near-white have a
69+// narrow luminance gap where neither meets AA; in that gap move HSL
70+// lightness away from .5, preserving hue and saturation.
71+// Explicit foreground overrides are honoured as supplied.
72+func Colors(active, background, foreground string) (color.RGBA, color.RGBA) {
73+ bg := parseColor(background, parseColor(active, color.RGBA{128, 128, 128, 255}))
74+ dark, light := color.RGBA{17, 17, 17, 255}, color.RGBA{245, 245, 245, 255}
75+ ink := dark
76+ if contrast(bg, light) > contrast(bg, dark) {
77+ ink = light
78+ }
79+ for contrast(bg, ink) < 4.5 {
80+ // Move HSL lightness toward zero while preserving hue and saturation.
81+ hi, lo := max(bg.R, bg.G, bg.B), min(bg.R, bg.G, bg.B)
82+ l := (float64(hi) + float64(lo)) / 510
83+ if l > .5 {
84+ // Above .5, moving toward white increases contrast with dark ink.
85+ bg.R += uint8(math.Ceil(float64(255-bg.R) * .01))
86+ bg.G += uint8(math.Ceil(float64(255-bg.G) * .01))
87+ bg.B += uint8(math.Ceil(float64(255-bg.B) * .01))
88+ ink = dark
89+ } else {
90+ bg.R = uint8(float64(bg.R) * .99)
91+ bg.G = uint8(float64(bg.G) * .99)
92+ bg.B = uint8(float64(bg.B) * .99)
93+ ink = light
94+ }
95+ }
96+ return bg, parseColor(foreground, ink)
97+}
98+
99+func mix(ink, bg color.RGBA) color.RGBA {
100+ channel := func(a, b uint8) uint8 { return uint8(math.Round(.45*float64(a) + .55*float64(b))) }
101+ return color.RGBA{channel(ink.R, bg.R), channel(ink.G, bg.G), channel(ink.B, bg.B), 255}
102+}
A internal/adapters/hintlabel/style_test.go +80 -0
@@ -0,0 +1,80 @@
1+package hintlabel
2+
3+import (
4+ "image/color"
5+ "testing"
6+
7+ "github.com/stretchr/testify/require"
8+)
9+
10+func TestDensitySize(t *testing.T) {
11+ for _, tt := range []struct {
12+ name string
13+ w, h, mw, mh int
14+ scale, logical, want float64
15+ }{
16+ {"ultrawide", 5120, 2160, 990, 420, 1, 0, 17},
17+ {"4k", 3840, 2160, 600, 340, 1.2, 0, 20},
18+ {"unknown", 1920, 1080, 0, 0, 1, 0, 13},
19+ {"bogus aspect", 1920, 1080, 100, 100, 1, 0, 13},
20+ {"fallback high", 5120, 2160, 0, 0, 1, 0, 18.38477631085024},
21+ {"minimum", 1920, 1080, 1000, 563, 1.2, 0, 16},
22+ {"maximum", 5120, 2160, 100, 42, 1, 0, 40},
23+ {"override", 5120, 2160, 990, 420, 1.2, 24, 28.8},
24+ } {
25+ t.Run(tt.name, func(t *testing.T) {
26+ require.InDelta(t, tt.want, FontPixels(tt.w, tt.h, tt.mw, tt.mh, tt.scale, tt.logical), .00001)
27+ })
28+ }
29+}
30+
31+func TestDerivedContrast(t *testing.T) {
32+ for _, s := range []string{"#f9e2af", "#89b4fa", "#808080", "#000000", "#ffffff", "#1e1e2e", "#747474", "#757575", "#ff0000", "#00ff00", "#0000ff"} {
33+ t.Run(s, func(t *testing.T) {
34+ bg, fg := Colors(s, "", "")
35+ require.GreaterOrEqual(t, contrast(bg, fg), 4.5)
36+ require.Contains(t, []color.RGBA{{17, 17, 17, 255}, {245, 245, 245, 255}}, fg)
37+ })
38+ }
39+ // Sweep the gap between near-black and near-white contrast thresholds.
40+ for i := 0; i < 256; i++ {
41+ c := color.RGBA{uint8(i), uint8(i), uint8(i), 255}
42+ s := "#" + hexByte(c.R) + hexByte(c.G) + hexByte(c.B)
43+ bg, fg := Colors(s, "", "")
44+ require.GreaterOrEqual(t, contrast(bg, fg), 4.5, s)
45+ }
46+}
47+
48+func hexByte(v uint8) string {
49+ const digits = "0123456789abcdef"
50+ return string([]byte{digits[v>>4], digits[v&15]})
51+}
52+
53+func TestOverridesAndTyped(t *testing.T) {
54+ bg, fg := Colors("#ffffff", "#000000", "#89b4fa")
55+ require.Equal(t, color.RGBA{0, 0, 0, 255}, bg)
56+ require.Equal(t, color.RGBA{137, 180, 250, 255}, fg)
57+ require.Equal(t, color.RGBA{62, 81, 113, 255}, mix(fg, bg))
58+ fallback, _ := Colors("", "", "")
59+ require.Equal(t, color.RGBA{128, 128, 128, 255}, fallback)
60+}
61+
62+func TestStyledRasterSizeAndCache(t *testing.T) {
63+ var r Raster
64+ bg, fg := Colors("#f9e2af", "", "")
65+ style := Style{13, bg, fg}
66+ a, _, small := r.Label("fj", 0, 1, style)
67+ style.FontPx = 26
68+ b, _, large := r.Label("fj", 0, 1, style)
69+ require.Greater(t, large, small)
70+ require.NotEqual(t, a, b)
71+ style.Background = color.RGBA{0, 0, 0, 255}
72+ c, _, _ := r.Label("fj", 0, 1, style)
73+ require.NotEqual(t, b, c)
74+ require.Equal(t, byte(89), c[3], "outline has 35 percent physical-pixel alpha")
75+}
76+
77+func TestFontPixelsClamped(t *testing.T) {
78+ require.InDelta(t, 256, FontPixels(1920, 1080, 0, 0, 3, 1e9), .00001)
79+ require.InDelta(t, 256, FontPixels(1920, 1080, 0, 0, 4, 72), .00001)
80+}
@@ -0,0 +1,80 @@
1+package hintlabel
2+
3+import (
4+ "image/color"
5+ "testing"
6+
7+ "github.com/stretchr/testify/require"
8+)
9+
10+func TestDensitySize(t *testing.T) {
11+ for _, tt := range []struct {
12+ name string
13+ w, h, mw, mh int
14+ scale, logical, want float64
15+ }{
16+ {"ultrawide", 5120, 2160, 990, 420, 1, 0, 17},
17+ {"4k", 3840, 2160, 600, 340, 1.2, 0, 20},
18+ {"unknown", 1920, 1080, 0, 0, 1, 0, 13},
19+ {"bogus aspect", 1920, 1080, 100, 100, 1, 0, 13},
20+ {"fallback high", 5120, 2160, 0, 0, 1, 0, 18.38477631085024},
21+ {"minimum", 1920, 1080, 1000, 563, 1.2, 0, 16},
22+ {"maximum", 5120, 2160, 100, 42, 1, 0, 40},
23+ {"override", 5120, 2160, 990, 420, 1.2, 24, 28.8},
24+ } {
25+ t.Run(tt.name, func(t *testing.T) {
26+ require.InDelta(t, tt.want, FontPixels(tt.w, tt.h, tt.mw, tt.mh, tt.scale, tt.logical), .00001)
27+ })
28+ }
29+}
30+
31+func TestDerivedContrast(t *testing.T) {
32+ for _, s := range []string{"#f9e2af", "#89b4fa", "#808080", "#000000", "#ffffff", "#1e1e2e", "#747474", "#757575", "#ff0000", "#00ff00", "#0000ff"} {
33+ t.Run(s, func(t *testing.T) {
34+ bg, fg := Colors(s, "", "")
35+ require.GreaterOrEqual(t, contrast(bg, fg), 4.5)
36+ require.Contains(t, []color.RGBA{{17, 17, 17, 255}, {245, 245, 245, 255}}, fg)
37+ })
38+ }
39+ // Sweep the gap between near-black and near-white contrast thresholds.
40+ for i := 0; i < 256; i++ {
41+ c := color.RGBA{uint8(i), uint8(i), uint8(i), 255}
42+ s := "#" + hexByte(c.R) + hexByte(c.G) + hexByte(c.B)
43+ bg, fg := Colors(s, "", "")
44+ require.GreaterOrEqual(t, contrast(bg, fg), 4.5, s)
45+ }
46+}
47+
48+func hexByte(v uint8) string {
49+ const digits = "0123456789abcdef"
50+ return string([]byte{digits[v>>4], digits[v&15]})
51+}
52+
53+func TestOverridesAndTyped(t *testing.T) {
54+ bg, fg := Colors("#ffffff", "#000000", "#89b4fa")
55+ require.Equal(t, color.RGBA{0, 0, 0, 255}, bg)
56+ require.Equal(t, color.RGBA{137, 180, 250, 255}, fg)
57+ require.Equal(t, color.RGBA{62, 81, 113, 255}, mix(fg, bg))
58+ fallback, _ := Colors("", "", "")
59+ require.Equal(t, color.RGBA{128, 128, 128, 255}, fallback)
60+}
61+
62+func TestStyledRasterSizeAndCache(t *testing.T) {
63+ var r Raster
64+ bg, fg := Colors("#f9e2af", "", "")
65+ style := Style{13, bg, fg}
66+ a, _, small := r.Label("fj", 0, 1, style)
67+ style.FontPx = 26
68+ b, _, large := r.Label("fj", 0, 1, style)
69+ require.Greater(t, large, small)
70+ require.NotEqual(t, a, b)
71+ style.Background = color.RGBA{0, 0, 0, 255}
72+ c, _, _ := r.Label("fj", 0, 1, style)
73+ require.NotEqual(t, b, c)
74+ require.Equal(t, byte(89), c[3], "outline has 35 percent physical-pixel alpha")
75+}
76+
77+func TestFontPixelsClamped(t *testing.T) {
78+ require.InDelta(t, 256, FontPixels(1920, 1080, 0, 0, 3, 1e9), .00001)
79+ require.InDelta(t, 256, FontPixels(1920, 1080, 0, 0, 4, 72), .00001)
80+}
A internal/adapters/hints/benchmark_test.go +81 -0
A internal/adapters/hints/dump.go +285 -0
A internal/adapters/hints/dump_test.go +40 -0
A internal/adapters/hints/hints.go +507 -0
A internal/adapters/hints/hints_test.go +436 -0
A internal/adapters/hints/hybrid_test.go +177 -0
A internal/adapters/hints/mmap.go +40 -0
A internal/adapters/hints/ocr.go +440 -0
A internal/adapters/hints/ocr_test.go +238 -0
A internal/adapters/hints/scan_bench_test.go +49 -0
A internal/adapters/hints/strips_test.go +124 -0
A internal/adapters/hints/target_source_mock_test.go +174 -0
A internal/adapters/hints/testdata/browser.png +0 -0 binary file: no text diff
A internal/adapters/hints/testdata/terminal.png +0 -0 binary file: no text diff
A internal/adapters/hints/testdata/underline.png +0 -0 binary file: no text diff
A internal/adapters/hints/underline.go +246 -0
A internal/adapters/hints/underline_test.go +252 -0
M internal/adapters/outputkit/capture/excluded.go +5 -5
M internal/adapters/outputkit/capture/indicator_test.go +40 -0
M internal/adapters/vulkan/damage.go +4 -2
M internal/adapters/vulkan/damage_delta_test.go +2 -2
A internal/adapters/vulkan/hint_labels.go +52 -0
A internal/adapters/vulkan/hint_labels_test.go +137 -0
M internal/adapters/vulkan/renderer.go +3 -0
M internal/adapters/vulkan/scene.go +4 -3
M internal/adapters/vulkan/shm.go +8 -0
M internal/adapters/vulkan/targets.go +1 -0
M internal/adapters/wayland/keyboard_test.go +88 -1
M internal/adapters/wayland/seat.go +56 -0
M internal/adapters/wayland/server.go +2 -2
M internal/app/app.go +7 -0
A internal/app/captured.go +41 -0
A internal/app/captured_test.go +72 -0
A internal/app/hintdebug.go +51 -0
A internal/app/hintoptin_test.go +68 -0
M internal/app/session.go +17 -2
M internal/app/wiring.go +34 -5
M internal/core/action.go +6 -0
M internal/core/core.go +30 -1
M internal/core/drag.go +1 -1
A internal/core/hint.go +361 -0
A internal/core/hint_test.go +834 -0
M internal/core/input.go +22 -8
M internal/core/scene.go +2 -0
M internal/ports/capture_messages.go +6 -0
M internal/ports/client_commands.go +14 -0
M internal/ports/config.go +14 -0
A internal/ports/hint_messages.go +67 -0
M internal/ports/scene.go +14 -1
M internal/ports/scene_same_test.go +4 -2

Comments0 open / 0

no comments yet.

Keyboard shortcuts

Single-key shortcuts are on. ctrl k and esc work whatever this says.

ctrl k :
go to a section, branch, tag, file or repository
?
this list
j k
move the row cursor down / up
enter
open the row under the cursor; on Files, a directory opens or closes in place
↑ ↓ ← → home end
in the file tree (focus on a row): previous / next row, close / open a directory or go to its parent / first entry, first / last row. They never open a file
l h
on Files: open a directory or a file / close a directory or go up to its parent, then to the parent page. Elsewhere: into the side panel (on a merge request, its files) / back to the content; there j k move between its controls
g g G
first / last row
/
filter the list; on Files, find a file anywhere in the repository (esc clears, enter opens the best match)
b
switch branch or tag
g s g f g c g b g m g i g r g a
tabs: summary, files, commits, branches & tags, merge requests, issues, releases, actions (when available)
g h
home
w
wide mode: use the whole width of the screen, or not (remembered)
y
copy a permalink pinned to the commit
d u x
diff: split / unified / collapse or expand all
e E
diff: the whole file in view / every file, with the lines the hunks hide
n
merge request: next note in the diff
l
merge request: the list of files (j k move, enter opens, esc back)
M
merge request: the merge menu; there 1–3 pick the mode, D deletes the branch, enter merges
R
merge request: mark the head read
esc
close a dialog, leave a field

Branches 25

Tags 10

↑↓ moveenter openesc close