feat/hints a471fa4

#5 WIP: feat(hints): add experimental keyboard hint mode

state
closed
branch
feat/hints → main
opened
bnema ·
updated

Experimental keyboard hint mode: press the hint bind, every link and button on the focused output gets a short label, type the label to click it.

Off by default. hint.enable = on turns it on (restart required). When it is off, nothing starts: no goroutine, no accessibility bus connection, no capture routing, no buffers.

Where targets come from

  • Accessibility tree (AT-SPI) for windows that expose it (Firefox, Chromium with --force-renderer-accessibility, GTK, Qt, Electron): exact links and buttons, a few milliseconds. New adapter atspi over zerobus, one connection owned by the hints worker, connected on the first scan with bounded attempts and reconnect backoff.
  • OCR (tesseract CLI, no cgo) only on the visible windows the accessibility tree does not cover: visible URLs, domains, email addresses and underlined words. Grayscale bands fed as PGM, 4 processes max per scan, per-band word cache. Without tesseract, OCR is off and only accessibility targets are labelled.

Click

  • Core keeps each target on the window it was scanned on, and cancels hints when the window, workspace, output geometry, scale or transform changes.
  • Every hint click sends an explicit modifier state to the client (new SetModifiers client command), then restores the real state. Links read by OCR in text use hint.text-click (default ctrl+shift) so terminals open them; underlined words and accessibility targets get a plain click.

Labels

  • Drawn by the Vulkan renderer, anchored outside the top-left corner of their target, kept out of captured frames.
  • Size follows the output pixel density (EDID millimetres), colours derive from border.active with WCAG 4.5:1 contrast. Overrides: hint.size, hint.background, hint.foreground.

Memory

  • Captures go to anonymous mmap released right after cropping; no capture when every window is covered by the accessibility tree. About 92 KB of Go heap per scan at 5120x2160.

Debugging

  • NEFERWL_HINT_DEBUG=1 writes each scanned frame, an annotated frame and a JSON of targets to $XDG_STATE_HOME/neferwl/hints/ (0700 / 0600, 50 most recent kept).

Checks

  • make check, make race, make mocks-check, staticcheck ./... pass; every commit builds.
  • Headless end to end at 1920x1080 and 5120x2160: Chromium (11 accessibility targets in ~2 ms), kitty (URL and domain links by OCR, none on filenames, paths, prompt or cursor), side by side (OCR only on the kitty rect), dark page without accessibility (underlined links only), ctrl+shift click opens a kitty link and a plain click does not, accessibility click reaches Chromium, hint.enable = off starts nothing.

Known limits

  • OCR is best effort: it misses links whose address is not written on screen and not underlined, and reads two full-height windows in about 1 s at 5120x2160.
  • Browsers drawing web content off screen (CEF OSR) expose no accessibility tree; they fall back to OCR.
  • zerobus has no dial deadline: a stalled bus is bounded by one connection attempt in flight, not by a timeout.
  • Modifier masks use the standard xkb indices.
CommitSubjectAuthorDateSignatureStatus
a471fa4docs(config): document the experimental hint modebriceGPG Verified
d7c4662feat(app): wire hint mode when hint.enable is onbriceGPG Verified
463ec0afeat(hints): find hint targets with the accessibility tree and OCRbriceGPG Verified
d9f6a44feat(atspi): find hint targets in the accessibility treebriceGPG Verified
e368cc0feat(config): add hint mode keysbriceGPG Verified
2082615feat(wayland): apply temporary modifier overrides for hint clicksbriceGPG Verified
6319bfafeat(vulkan): draw hint labels and keep them out of capturesbriceGPG Verified
4b849d0feat(hintlabel): rasterize hint labels sized and coloured for the outputbriceGPG Verified
40c7803feat(core): add keyboard hint modebriceGPG Verified
689e994feat(ports): add hint scan, modifier override and hint config messagesbriceGPG Verified

Checks 1

75 files · +7747 -37
1/1
M .hexcheck.yaml +1 -1
M .mockery.yml +19 -0
M Makefile +1 -1
M README.md +1 -0
M docs/config.md +50 -0
M examples/config +7 -0
M go.mod +2 -0
M go.sum +4 -0
A internal/adapters/atspi/bus_test.go +197 -0
A internal/adapters/atspi/connector_mock_test.go +109 -0
A internal/adapters/atspi/doc.go +10 -0
A internal/adapters/atspi/model.go +206 -0
A internal/adapters/atspi/model_test.go +117 -0
A internal/adapters/atspi/source.go +482 -0
A internal/adapters/atspi/source_test.go +315 -0
A internal/adapters/atspi/transport.go +362 -0
A internal/adapters/atspi/transport_mock_test.go +156 -0
M internal/adapters/config/config.go +74 -0
M internal/adapters/config/config_test.go +92 -0
M internal/adapters/drm/overlay.go +3 -0
M internal/adapters/drm/scanout.go +4 -0
A internal/adapters/hintlabel/hintlabel.go +137 -0
A internal/adapters/hintlabel/hintlabel_test.go +60 -0
A internal/adapters/hintlabel/style.go +102 -0
A internal/adapters/hintlabel/style_test.go +80 -0
A internal/adapters/hints/benchmark_test.go +81 -0
A internal/adapters/hints/dump.go +285 -0
A internal/adapters/hints/dump_test.go +40 -0
A internal/adapters/hints/hints.go +507 -0
A internal/adapters/hints/hints_test.go +436 -0
A internal/adapters/hints/hybrid_test.go +177 -0
A internal/adapters/hints/mmap.go +40 -0
A internal/adapters/hints/ocr.go +440 -0
A internal/adapters/hints/ocr_test.go +238 -0
A internal/adapters/hints/scan_bench_test.go +49 -0
A internal/adapters/hints/strips_test.go +124 -0
A internal/adapters/hints/target_source_mock_test.go +174 -0
A internal/adapters/hints/testdata/browser.png +0 -0 binary file: no text diff
A internal/adapters/hints/testdata/terminal.png +0 -0 binary file: no text diff
A internal/adapters/hints/testdata/underline.png +0 -0 binary file: no text diff
A internal/adapters/hints/underline.go +246 -0
A internal/adapters/hints/underline_test.go +252 -0
M internal/adapters/outputkit/capture/excluded.go +5 -5
M internal/adapters/outputkit/capture/indicator_test.go +40 -0
M internal/adapters/vulkan/damage.go +4 -2
M internal/adapters/vulkan/damage_delta_test.go +2 -2
A internal/adapters/vulkan/hint_labels.go +52 -0
A internal/adapters/vulkan/hint_labels_test.go +137 -0
M internal/adapters/vulkan/renderer.go +3 -0
M internal/adapters/vulkan/scene.go +4 -3
M internal/adapters/vulkan/shm.go +8 -0
@@ -6,6 +6,7 @@ import (
66 "runtime/debug"
77 "unsafe"
88
9+ "github.com/bnema/neferwl/internal/adapters/hintlabel"
910 "github.com/bnema/neferwl/internal/ports"
1011 vk "github.com/bnema/purego-vulkan/vulkan"
1112 "golang.org/x/sys/unix"
@@ -154,6 +155,13 @@ func (r *Renderer) freeGPUBuffer(b *gpuBuffer) {
154155 type shmKey struct {
155156 win ports.WindowID
156157 surface uint64
158+ // label, typed and scale name a compositor-drawn hint label (hint_labels.go)
159+ // instead of a client surface: win and surface are 0 and label is never
160+ // empty, so such a key cannot collide with a client's.
161+ label string
162+ typed int
163+ scale float64
164+ style hintlabel.Style
157165 }
158166
159167 // shmState is what a GPU buffer holds: a w×h client buffer at a content
@@ -6,6 +6,7 @@ import (
6 "runtime/debug"6 "runtime/debug"
7 "unsafe"7 "unsafe"
8 8
9+ "github.com/bnema/neferwl/internal/adapters/hintlabel"
9 "github.com/bnema/neferwl/internal/ports"10 "github.com/bnema/neferwl/internal/ports"
10 vk "github.com/bnema/purego-vulkan/vulkan"11 vk "github.com/bnema/purego-vulkan/vulkan"
11 "golang.org/x/sys/unix"12 "golang.org/x/sys/unix"
@@ -154,6 +155,13 @@ func (r *Renderer) freeGPUBuffer(b *gpuBuffer) {
154 type shmKey struct {155 type shmKey struct {
155 win ports.WindowID156 win ports.WindowID
156 surface uint64157 surface uint64
158+ // label, typed and scale name a compositor-drawn hint label (hint_labels.go)
159+ // instead of a client surface: win and surface are 0 and label is never
160+ // empty, so such a key cannot collide with a client's.
161+ label string
162+ typed int
163+ scale float64
164+ style hintlabel.Style
157 }165 }
158 166
159 // shmState is what a GPU buffer holds: a w×h client buffer at a content167 // shmState is what a GPU buffer holds: a w×h client buffer at a content
M internal/adapters/vulkan/targets.go +1 -0
@@ -43,6 +43,7 @@ type target struct {
4343 heldLayers []ports.SceneLayer
4444 heldHints []ports.Rect
4545 heldIndicators []ports.CaptureIndicator
46+ heldLabels []ports.HintLabel
4647 windows map[ports.WindowID]heldWindow
4748 }
4849
@@ -43,6 +43,7 @@ type target struct {
43 heldLayers []ports.SceneLayer43 heldLayers []ports.SceneLayer
44 heldHints []ports.Rect44 heldHints []ports.Rect
45 heldIndicators []ports.CaptureIndicator45 heldIndicators []ports.CaptureIndicator
46+ heldLabels []ports.HintLabel
46 windows map[ports.WindowID]heldWindow47 windows map[ports.WindowID]heldWindow
47 }48 }
48 49
M internal/adapters/wayland/keyboard_test.go +88 -1
@@ -130,13 +130,14 @@ type eventsProxy struct {
130130 opcodes []uint16
131131 keymaps []string
132132 locked []uint32 // locked modifiers of each modifiers event
133+ depress []uint32 // depressed modifiers of each modifiers event
133134 }
134135
135136 func (p *eventsProxy) Dispatch(e *wlturbo.Event) {
136137 p.opcodes = append(p.opcodes, e.Opcode)
137138 if e.Opcode == uint16(wayland.KeyboardEventModifiers) {
138139 e.Uint32() // serial
139- e.Uint32() // depressed
140+ p.depress = append(p.depress, e.Uint32())
140141 e.Uint32() // latched
141142 p.locked = append(p.locked, e.Uint32())
142143 return
@@ -254,3 +255,89 @@ func TestFootKeyboard(t *testing.T) {
254255 }
255256 }
256257 }
258+
259+// SetModifiers sends the requested depressed mask with the real latched and
260+// locked state, and Restore sends the real state back; the seat's own state
261+// is untouched. A window that is not focused is ignored.
262+func TestSetModifiersOverrideAndRestore(t *testing.T) {
263+ s, events, commands, dir := keyboardServer(t)
264+ target, proxy := focusedTarget(t, s, events, commands, dir)
265+ const capsLocked = 2
266+ var id ports.WindowID
267+ set := make(chan struct{})
268+ s.display.Do(func() { s.seat.modState = ports.ModState{Locked: capsLocked}; id = s.seat.focused; close(set) })
269+ <-set
270+ // A marker command applied on the display means every command sent before
271+ // it was applied (they apply in order); the roundtrip then delivers the
272+ // events they sent.
273+ var pending bool
274+ wait := func() {
275+ t.Helper()
276+ pending = !pending
277+ commands <- ports.SlotsPending{Pending: pending}
278+ deadline := time.Now().Add(2 * time.Second)
279+ for {
280+ var now bool
281+ if !s.display.Do(func() { now = s.slotsPending }) {
282+ t.Fatal("display stopped")
283+ }
284+ if now == pending {
285+ break
286+ }
287+ if time.Now().After(deadline) {
288+ t.Fatal("commands not applied")
289+ }
290+ time.Sleep(time.Millisecond)
291+ }
292+ roundtrip(t, target)
293+ }
294+ proxy.depress, proxy.locked = nil, nil
295+ commands <- ports.SetModifiers{ID: id + 100, Mods: ports.ModCtrl}
296+ wait()
297+ if len(proxy.depress) != 0 {
298+ t.Fatalf("unfocused window got modifiers %v", proxy.depress)
299+ }
300+ commands <- ports.SetModifiers{ID: id, Mods: ports.ModCtrl | ports.ModShift}
301+ wait()
302+ if !slices.Equal(proxy.depress, []uint32{xkbControl | xkbShift}) || !slices.Equal(proxy.locked, []uint32{capsLocked}) {
303+ t.Fatalf("override depressed %v locked %v", proxy.depress, proxy.locked)
304+ }
305+ done := make(chan ports.ModState, 1)
306+ commands <- ports.SetModifiers{ID: id, Restore: true, State: ports.ModState{Locked: capsLocked}}
307+ wait()
308+ if !slices.Equal(proxy.depress, []uint32{xkbControl | xkbShift, 0}) || !slices.Equal(proxy.locked, []uint32{capsLocked, capsLocked}) {
309+ t.Fatalf("restore depressed %v locked %v", proxy.depress, proxy.locked)
310+ }
311+ s.display.Do(func() { done <- s.seat.modState })
312+ if m := <-done; m != (ports.ModState{Locked: capsLocked}) {
313+ t.Fatalf("seat state changed: %+v", m)
314+ }
315+}
316+
317+// Restore carries the real state core saw, which swallowed keys (a latch a
318+// hint label key consumed) never gave the seat: it is sent and adopted.
319+func TestSetModifiersRestoreAdoptsCoreState(t *testing.T) {
320+ s, events, commands, dir := keyboardServer(t)
321+ target, proxy := focusedTarget(t, s, events, commands, dir)
322+ const shiftLatched, capsLocked = 1, 2
323+ var id ports.WindowID
324+ set := make(chan struct{})
325+ s.display.Do(func() { s.seat.modState = ports.ModState{Latched: shiftLatched}; id = s.seat.focused; close(set) })
326+ <-set
327+ proxy.depress, proxy.locked = nil, nil
328+ want := ports.ModState{Locked: capsLocked} // the latch was consumed
329+ commands <- ports.SetModifiers{ID: id, Restore: true, State: want}
330+ var got ports.ModState
331+ deadline := time.Now().Add(2 * time.Second)
332+ for got != want {
333+ if time.Now().After(deadline) {
334+ t.Fatalf("seat state %+v, want %+v", got, want)
335+ }
336+ s.display.Do(func() { got = s.seat.modState })
337+ time.Sleep(time.Millisecond)
338+ }
339+ roundtrip(t, target)
340+ if !slices.Equal(proxy.locked, []uint32{capsLocked}) || !slices.Equal(proxy.depress, []uint32{0}) {
341+ t.Fatalf("restore depressed %v locked %v", proxy.depress, proxy.locked)
342+ }
343+}
@@ -130,13 +130,14 @@ type eventsProxy struct {
130 opcodes []uint16130 opcodes []uint16
131 keymaps []string131 keymaps []string
132 locked []uint32 // locked modifiers of each modifiers event132 locked []uint32 // locked modifiers of each modifiers event
133+ depress []uint32 // depressed modifiers of each modifiers event
133 }134 }
134 135
135 func (p *eventsProxy) Dispatch(e *wlturbo.Event) {136 func (p *eventsProxy) Dispatch(e *wlturbo.Event) {
136 p.opcodes = append(p.opcodes, e.Opcode)137 p.opcodes = append(p.opcodes, e.Opcode)
137 if e.Opcode == uint16(wayland.KeyboardEventModifiers) {138 if e.Opcode == uint16(wayland.KeyboardEventModifiers) {
138 e.Uint32() // serial139 e.Uint32() // serial
139- e.Uint32() // depressed140+ p.depress = append(p.depress, e.Uint32())
140 e.Uint32() // latched141 e.Uint32() // latched
141 p.locked = append(p.locked, e.Uint32())142 p.locked = append(p.locked, e.Uint32())
142 return143 return
@@ -254,3 +255,89 @@ func TestFootKeyboard(t *testing.T) {
254 }255 }
255 }256 }
256 }257 }
258+
259+// SetModifiers sends the requested depressed mask with the real latched and
260+// locked state, and Restore sends the real state back; the seat's own state
261+// is untouched. A window that is not focused is ignored.
262+func TestSetModifiersOverrideAndRestore(t *testing.T) {
263+ s, events, commands, dir := keyboardServer(t)
264+ target, proxy := focusedTarget(t, s, events, commands, dir)
265+ const capsLocked = 2
266+ var id ports.WindowID
267+ set := make(chan struct{})
268+ s.display.Do(func() { s.seat.modState = ports.ModState{Locked: capsLocked}; id = s.seat.focused; close(set) })
269+ <-set
270+ // A marker command applied on the display means every command sent before
271+ // it was applied (they apply in order); the roundtrip then delivers the
272+ // events they sent.
273+ var pending bool
274+ wait := func() {
275+ t.Helper()
276+ pending = !pending
277+ commands <- ports.SlotsPending{Pending: pending}
278+ deadline := time.Now().Add(2 * time.Second)
279+ for {
280+ var now bool
281+ if !s.display.Do(func() { now = s.slotsPending }) {
282+ t.Fatal("display stopped")
283+ }
284+ if now == pending {
285+ break
286+ }
287+ if time.Now().After(deadline) {
288+ t.Fatal("commands not applied")
289+ }
290+ time.Sleep(time.Millisecond)
291+ }
292+ roundtrip(t, target)
293+ }
294+ proxy.depress, proxy.locked = nil, nil
295+ commands <- ports.SetModifiers{ID: id + 100, Mods: ports.ModCtrl}
296+ wait()
297+ if len(proxy.depress) != 0 {
298+ t.Fatalf("unfocused window got modifiers %v", proxy.depress)
299+ }
300+ commands <- ports.SetModifiers{ID: id, Mods: ports.ModCtrl | ports.ModShift}
301+ wait()
302+ if !slices.Equal(proxy.depress, []uint32{xkbControl | xkbShift}) || !slices.Equal(proxy.locked, []uint32{capsLocked}) {
303+ t.Fatalf("override depressed %v locked %v", proxy.depress, proxy.locked)
304+ }
305+ done := make(chan ports.ModState, 1)
306+ commands <- ports.SetModifiers{ID: id, Restore: true, State: ports.ModState{Locked: capsLocked}}
307+ wait()
308+ if !slices.Equal(proxy.depress, []uint32{xkbControl | xkbShift, 0}) || !slices.Equal(proxy.locked, []uint32{capsLocked, capsLocked}) {
309+ t.Fatalf("restore depressed %v locked %v", proxy.depress, proxy.locked)
310+ }
311+ s.display.Do(func() { done <- s.seat.modState })
312+ if m := <-done; m != (ports.ModState{Locked: capsLocked}) {
313+ t.Fatalf("seat state changed: %+v", m)
314+ }
315+}
316+
317+// Restore carries the real state core saw, which swallowed keys (a latch a
318+// hint label key consumed) never gave the seat: it is sent and adopted.
319+func TestSetModifiersRestoreAdoptsCoreState(t *testing.T) {
320+ s, events, commands, dir := keyboardServer(t)
321+ target, proxy := focusedTarget(t, s, events, commands, dir)
322+ const shiftLatched, capsLocked = 1, 2
323+ var id ports.WindowID
324+ set := make(chan struct{})
325+ s.display.Do(func() { s.seat.modState = ports.ModState{Latched: shiftLatched}; id = s.seat.focused; close(set) })
326+ <-set
327+ proxy.depress, proxy.locked = nil, nil
328+ want := ports.ModState{Locked: capsLocked} // the latch was consumed
329+ commands <- ports.SetModifiers{ID: id, Restore: true, State: want}
330+ var got ports.ModState
331+ deadline := time.Now().Add(2 * time.Second)
332+ for got != want {
333+ if time.Now().After(deadline) {
334+ t.Fatalf("seat state %+v, want %+v", got, want)
335+ }
336+ s.display.Do(func() { got = s.seat.modState })
337+ time.Sleep(time.Millisecond)
338+ }
339+ roundtrip(t, target)
340+ if !slices.Equal(proxy.locked, []uint32{capsLocked}) || !slices.Equal(proxy.depress, []uint32{0}) {
341+ t.Fatalf("restore depressed %v locked %v", proxy.depress, proxy.locked)
342+ }
343+}
M internal/adapters/wayland/seat.go +56 -0
@@ -83,6 +83,8 @@ func (s *Server) applyInput(cmd ports.ClientCommand) {
8383 target = c.ID
8484 case ports.ForwardKey:
8585 target = c.ID
86+ case ports.SetModifiers:
87+ target = c.ID
8688 }
8789 if target != 0 && !s.lockInputTarget(target) {
8890 return
@@ -177,6 +179,8 @@ func (s *Server) applyInput(cmd ports.ClientCommand) {
177179 if s.seat.focused != c.ID {
178180 s.changeFocus(c.ID)
179181 }
182+ case ports.SetModifiers:
183+ s.setModifiers(c)
180184 case ports.ForwardKey:
181185 if !(s.security != nil && s.security.Snapshot().Protected) && s.grabKey(c) {
182186 return
@@ -219,6 +223,58 @@ func (s *Server) applyInput(cmd ports.ClientCommand) {
219223 }
220224 }
221225
226+// xkb modifier masks of the default keymaps: Shift, Control, Mod1 (Alt) and
227+// Mod4 (Super). The server does not parse the keymap for its modifier indices.
228+const (
229+ xkbShift = 1 << 0
230+ xkbControl = 1 << 2
231+ xkbMod1 = 1 << 3
232+ xkbMod4 = 1 << 6
233+)
234+
235+// depressedMask maps core modifier flags to xkb depressed masks.
236+func depressedMask(m ports.Mods) uint32 {
237+ var d uint32
238+ for _, f := range []struct {
239+ flag ports.Mods
240+ mask uint32
241+ }{{ports.ModShift, xkbShift}, {ports.ModCtrl, xkbControl}, {ports.ModAlt, xkbMod1}, {ports.ModSuper, xkbMod4}} {
242+ if m&f.flag != 0 {
243+ d |= f.mask
244+ }
245+ }
246+ return d
247+}
248+
249+// setModifiers overrides, then restores, the modifiers the focused window's
250+// keyboards (or the input method grab) see. The override replaces the
251+// depressed modifiers and keeps latched, locked and group; the seat's real
252+// modState is only replaced by a Restore, with the state core carries.
253+func (s *Server) setModifiers(c ports.SetModifiers) {
254+ _, keyboards := s.focusTarget(c.ID)
255+ if c.ID != s.seat.focused || len(keyboards) == 0 {
256+ s.log.Debug().Uint64("id", uint64(c.ID)).Msg("ignored set modifiers")
257+ return
258+ }
259+ s.useKeymap(nil)
260+ s.serial++
261+ if c.Restore {
262+ // Swallowed keys never reached the seat: core's snapshot is the
263+ // real state.
264+ s.seat.modState = c.State
265+ }
266+ m := s.seat.modState
267+ if !c.Restore {
268+ m.Depressed = depressedMask(c.Mods)
269+ }
270+ for _, k := range keyboards {
271+ k.SendModifiers(s.serial, m.Depressed, m.Latched, m.Locked, m.Group)
272+ }
273+ if g := s.grab(); g != nil {
274+ g.res.SendModifiers(s.serial, m.Depressed, m.Latched, m.Locked, m.Group)
275+ }
276+}
277+
222278 func (s *Server) windowKeyboards(w *window) []*wayland.Keyboard {
223279 if w == nil || !w.mapped || !w.xdg.resource.Resource.Alive() {
224280 return nil
@@ -83,6 +83,8 @@ func (s *Server) applyInput(cmd ports.ClientCommand) {
83 target = c.ID83 target = c.ID
84 case ports.ForwardKey:84 case ports.ForwardKey:
85 target = c.ID85 target = c.ID
86+ case ports.SetModifiers:
87+ target = c.ID
86 }88 }
87 if target != 0 && !s.lockInputTarget(target) {89 if target != 0 && !s.lockInputTarget(target) {
88 return90 return
@@ -177,6 +179,8 @@ func (s *Server) applyInput(cmd ports.ClientCommand) {
177 if s.seat.focused != c.ID {179 if s.seat.focused != c.ID {
178 s.changeFocus(c.ID)180 s.changeFocus(c.ID)
179 }181 }
182+ case ports.SetModifiers:
183+ s.setModifiers(c)
180 case ports.ForwardKey:184 case ports.ForwardKey:
181 if !(s.security != nil && s.security.Snapshot().Protected) && s.grabKey(c) {185 if !(s.security != nil && s.security.Snapshot().Protected) && s.grabKey(c) {
182 return186 return
@@ -219,6 +223,58 @@ func (s *Server) applyInput(cmd ports.ClientCommand) {
219 }223 }
220 }224 }
221 225
226+// xkb modifier masks of the default keymaps: Shift, Control, Mod1 (Alt) and
227+// Mod4 (Super). The server does not parse the keymap for its modifier indices.
228+const (
229+ xkbShift = 1 << 0
230+ xkbControl = 1 << 2
231+ xkbMod1 = 1 << 3
232+ xkbMod4 = 1 << 6
233+)
234+
235+// depressedMask maps core modifier flags to xkb depressed masks.
236+func depressedMask(m ports.Mods) uint32 {
237+ var d uint32
238+ for _, f := range []struct {
239+ flag ports.Mods
240+ mask uint32
241+ }{{ports.ModShift, xkbShift}, {ports.ModCtrl, xkbControl}, {ports.ModAlt, xkbMod1}, {ports.ModSuper, xkbMod4}} {
242+ if m&f.flag != 0 {
243+ d |= f.mask
244+ }
245+ }
246+ return d
247+}
248+
249+// setModifiers overrides, then restores, the modifiers the focused window's
250+// keyboards (or the input method grab) see. The override replaces the
251+// depressed modifiers and keeps latched, locked and group; the seat's real
252+// modState is only replaced by a Restore, with the state core carries.
253+func (s *Server) setModifiers(c ports.SetModifiers) {
254+ _, keyboards := s.focusTarget(c.ID)
255+ if c.ID != s.seat.focused || len(keyboards) == 0 {
256+ s.log.Debug().Uint64("id", uint64(c.ID)).Msg("ignored set modifiers")
257+ return
258+ }
259+ s.useKeymap(nil)
260+ s.serial++
261+ if c.Restore {
262+ // Swallowed keys never reached the seat: core's snapshot is the
263+ // real state.
264+ s.seat.modState = c.State
265+ }
266+ m := s.seat.modState
267+ if !c.Restore {
268+ m.Depressed = depressedMask(c.Mods)
269+ }
270+ for _, k := range keyboards {
271+ k.SendModifiers(s.serial, m.Depressed, m.Latched, m.Locked, m.Group)
272+ }
273+ if g := s.grab(); g != nil {
274+ g.res.SendModifiers(s.serial, m.Depressed, m.Latched, m.Locked, m.Group)
275+ }
276+}
277+
222 func (s *Server) windowKeyboards(w *window) []*wayland.Keyboard {278 func (s *Server) windowKeyboards(w *window) []*wayland.Keyboard {
223 if w == nil || !w.mapped || !w.xdg.resource.Resource.Alive() {279 if w == nil || !w.mapped || !w.xdg.resource.Resource.Alive() {
224 return nil280 return nil
M internal/adapters/wayland/server.go +2 -2
@@ -752,13 +752,13 @@ func (s *Server) apply(cmd ports.ClientCommand) {
752752 // Only input/focus commands require an owner envelope. Config relay
753753 // also produces SetKeymap and other epoch-independent commands.
754754 switch cmd.(type) {
755- case ports.PointerFocus, ports.PointerMotionTo, ports.PointerButtonTo, ports.PointerAxisTo, ports.GestureBeginTo, ports.GestureUpdateTo, ports.GestureEndTo, ports.FocusWindow, ports.ForwardKey,
755+ case ports.PointerFocus, ports.PointerMotionTo, ports.PointerButtonTo, ports.PointerAxisTo, ports.GestureBeginTo, ports.GestureUpdateTo, ports.GestureEndTo, ports.FocusWindow, ports.ForwardKey, ports.SetModifiers,
756756 ports.DataDragFocus, ports.DataDragMotion, ports.DataDragDrop, ports.DataDragCancel:
757757 return
758758 }
759759 }
760760 switch c := cmd.(type) {
761- case ports.PointerFocus, ports.PointerMotionTo, ports.PointerButtonTo, ports.PointerAxisTo, ports.GestureBeginTo, ports.GestureUpdateTo, ports.GestureEndTo, ports.SetKeymap, ports.FocusWindow, ports.ForwardKey:
761+ case ports.PointerFocus, ports.PointerMotionTo, ports.PointerButtonTo, ports.PointerAxisTo, ports.GestureBeginTo, ports.GestureUpdateTo, ports.GestureEndTo, ports.SetKeymap, ports.FocusWindow, ports.ForwardKey, ports.SetModifiers:
762762 s.applyInput(c)
763763 case ports.DataDragFocus, ports.DataDragMotion, ports.DataDragDrop, ports.DataDragCancel:
764764 s.applyDrag(c)
@@ -752,13 +752,13 @@ func (s *Server) apply(cmd ports.ClientCommand) {
752 // Only input/focus commands require an owner envelope. Config relay752 // Only input/focus commands require an owner envelope. Config relay
753 // also produces SetKeymap and other epoch-independent commands.753 // also produces SetKeymap and other epoch-independent commands.
754 switch cmd.(type) {754 switch cmd.(type) {
755- case ports.PointerFocus, ports.PointerMotionTo, ports.PointerButtonTo, ports.PointerAxisTo, ports.GestureBeginTo, ports.GestureUpdateTo, ports.GestureEndTo, ports.FocusWindow, ports.ForwardKey,755+ case ports.PointerFocus, ports.PointerMotionTo, ports.PointerButtonTo, ports.PointerAxisTo, ports.GestureBeginTo, ports.GestureUpdateTo, ports.GestureEndTo, ports.FocusWindow, ports.ForwardKey, ports.SetModifiers,
756 ports.DataDragFocus, ports.DataDragMotion, ports.DataDragDrop, ports.DataDragCancel:756 ports.DataDragFocus, ports.DataDragMotion, ports.DataDragDrop, ports.DataDragCancel:
757 return757 return
758 }758 }
759 }759 }
760 switch c := cmd.(type) {760 switch c := cmd.(type) {
761- case ports.PointerFocus, ports.PointerMotionTo, ports.PointerButtonTo, ports.PointerAxisTo, ports.GestureBeginTo, ports.GestureUpdateTo, ports.GestureEndTo, ports.SetKeymap, ports.FocusWindow, ports.ForwardKey:761+ case ports.PointerFocus, ports.PointerMotionTo, ports.PointerButtonTo, ports.PointerAxisTo, ports.GestureBeginTo, ports.GestureUpdateTo, ports.GestureEndTo, ports.SetKeymap, ports.FocusWindow, ports.ForwardKey, ports.SetModifiers:
762 s.applyInput(c)762 s.applyInput(c)
763 case ports.DataDragFocus, ports.DataDragMotion, ports.DataDragDrop, ports.DataDragCancel:763 case ports.DataDragFocus, ports.DataDragMotion, ports.DataDragDrop, ports.DataDragCancel:
764 s.applyDrag(c)764 s.applyDrag(c)
M internal/app/app.go +7 -0
@@ -84,6 +84,7 @@ const busRetry = 2 * time.Second
8484 func relayConfig(ctx context.Context, cur ports.Config, in <-chan ports.ConfigChanged, out chan<- ports.ConfigChanged, keymaps chan *xkb.Keymap, deviceConfigs chan ports.InputDevicesConfig, curs *cursors, commands chan<- ports.ClientCommand, log zerowrap.Logger) {
8585 kb := cur.Keyboard
8686 devCfg := cur.InputDevicesConfig
87+ hintOn := cur.Hint.Enable
8788 for {
8889 var ev ports.ConfigChanged
8990 select {
@@ -91,6 +92,12 @@ func relayConfig(ctx context.Context, cur ports.Config, in <-chan ports.ConfigCh
9192 return
9293 case ev = <-in:
9394 }
95+ // hint.enable is read at startup: nothing of hint mode starts or
96+ // stops on a reload.
97+ if ev.Config.Hint.Enable != hintOn {
98+ log.Info().Bool("running", hintOn).Msg("hint.enable changes apply after restart")
99+ ev.Config.Hint.Enable = hintOn
100+ }
94101 next := ev.Config.Keyboard
95102 layoutChanged := next.Layout != kb.Layout || next.Variant != kb.Variant || next.Options != kb.Options
96103 repeatChanged := next.RepeatRate != kb.RepeatRate || next.RepeatDelay != kb.RepeatDelay
@@ -84,6 +84,7 @@ const busRetry = 2 * time.Second
84 func relayConfig(ctx context.Context, cur ports.Config, in <-chan ports.ConfigChanged, out chan<- ports.ConfigChanged, keymaps chan *xkb.Keymap, deviceConfigs chan ports.InputDevicesConfig, curs *cursors, commands chan<- ports.ClientCommand, log zerowrap.Logger) {84 func relayConfig(ctx context.Context, cur ports.Config, in <-chan ports.ConfigChanged, out chan<- ports.ConfigChanged, keymaps chan *xkb.Keymap, deviceConfigs chan ports.InputDevicesConfig, curs *cursors, commands chan<- ports.ClientCommand, log zerowrap.Logger) {
85 kb := cur.Keyboard85 kb := cur.Keyboard
86 devCfg := cur.InputDevicesConfig86 devCfg := cur.InputDevicesConfig
87+ hintOn := cur.Hint.Enable
87 for {88 for {
88 var ev ports.ConfigChanged89 var ev ports.ConfigChanged
89 select {90 select {
@@ -91,6 +92,12 @@ func relayConfig(ctx context.Context, cur ports.Config, in <-chan ports.ConfigCh
91 return92 return
92 case ev = <-in:93 case ev = <-in:
93 }94 }
95+ // hint.enable is read at startup: nothing of hint mode starts or
96+ // stops on a reload.
97+ if ev.Config.Hint.Enable != hintOn {
98+ log.Info().Bool("running", hintOn).Msg("hint.enable changes apply after restart")
99+ ev.Config.Hint.Enable = hintOn
100+ }
94 next := ev.Config.Keyboard101 next := ev.Config.Keyboard
95 layoutChanged := next.Layout != kb.Layout || next.Variant != kb.Variant || next.Options != kb.Options102 layoutChanged := next.Layout != kb.Layout || next.Variant != kb.Variant || next.Options != kb.Options
96 repeatChanged := next.RepeatRate != kb.RepeatRate || next.RepeatDelay != kb.RepeatDelay103 repeatChanged := next.RepeatRate != kb.RepeatRate || next.RepeatDelay != kb.RepeatDelay
A internal/app/captured.go +41 -0
@@ -0,0 +1,41 @@
1+package app
2+
3+import (
4+ "context"
5+
6+ "github.com/bnema/neferwl/internal/adapters/logging"
7+ "github.com/bnema/neferwl/internal/ports"
8+)
9+
10+// routeCaptured demultiplexes capture replies from the outputs: replies whose
11+// ID carries ports.CaptureIDInternal go to internal, all others to wayland.
12+// Both sends give way to ctx. A nil internal means no internal consumer is
13+// wired: internal replies are dropped (logged at debug) instead of blocking
14+// forever, since nothing would ever receive them. It returns when ctx is done
15+// or in is closed.
16+func routeCaptured(ctx context.Context, in <-chan ports.CaptureDone, wayland, internal chan<- ports.CaptureDone) {
17+ log := logging.For(ctx, "app")
18+ for {
19+ select {
20+ case <-ctx.Done():
21+ return
22+ case done, ok := <-in:
23+ if !ok {
24+ return
25+ }
26+ dst := wayland
27+ if done.ID&ports.CaptureIDInternal != 0 {
28+ if internal == nil {
29+ log.Debug().Uint64("id", done.ID).Msg("internal capture reply dropped: no consumer")
30+ continue
31+ }
32+ dst = internal
33+ }
34+ select {
35+ case dst <- done:
36+ case <-ctx.Done():
37+ return
38+ }
39+ }
40+ }
41+}
@@ -0,0 +1,41 @@
1+package app
2+
3+import (
4+ "context"
5+
6+ "github.com/bnema/neferwl/internal/adapters/logging"
7+ "github.com/bnema/neferwl/internal/ports"
8+)
9+
10+// routeCaptured demultiplexes capture replies from the outputs: replies whose
11+// ID carries ports.CaptureIDInternal go to internal, all others to wayland.
12+// Both sends give way to ctx. A nil internal means no internal consumer is
13+// wired: internal replies are dropped (logged at debug) instead of blocking
14+// forever, since nothing would ever receive them. It returns when ctx is done
15+// or in is closed.
16+func routeCaptured(ctx context.Context, in <-chan ports.CaptureDone, wayland, internal chan<- ports.CaptureDone) {
17+ log := logging.For(ctx, "app")
18+ for {
19+ select {
20+ case <-ctx.Done():
21+ return
22+ case done, ok := <-in:
23+ if !ok {
24+ return
25+ }
26+ dst := wayland
27+ if done.ID&ports.CaptureIDInternal != 0 {
28+ if internal == nil {
29+ log.Debug().Uint64("id", done.ID).Msg("internal capture reply dropped: no consumer")
30+ continue
31+ }
32+ dst = internal
33+ }
34+ select {
35+ case dst <- done:
36+ case <-ctx.Done():
37+ return
38+ }
39+ }
40+ }
41+}
A internal/app/captured_test.go +72 -0
@@ -0,0 +1,72 @@
1+package app
2+
3+import (
4+ "context"
5+ "testing"
6+ "time"
7+
8+ "github.com/stretchr/testify/require"
9+
10+ "github.com/bnema/neferwl/internal/ports"
11+)
12+
13+func TestCapturedDemux(t *testing.T) {
14+ ctx, cancel := context.WithCancel(context.Background())
15+ defer cancel()
16+ in := make(chan ports.CaptureDone, 4)
17+ wl := make(chan ports.CaptureDone, 4)
18+ internal := make(chan ports.CaptureDone, 4)
19+ exited := make(chan struct{})
20+ go func() { routeCaptured(ctx, in, wl, internal); close(exited) }()
21+
22+ in <- ports.CaptureDone{ID: 7}
23+ in <- ports.CaptureDone{ID: ports.CaptureIDInternal | 3}
24+ require.Equal(t, uint64(7), recvDone(t, wl).ID)
25+ require.Equal(t, ports.CaptureIDInternal|3, recvDone(t, internal).ID)
26+ require.Empty(t, wl)
27+ require.Empty(t, internal)
28+
29+ cancel()
30+ select {
31+ case <-exited:
32+ case <-time.After(2 * time.Second):
33+ t.Fatal("routeCaptured did not return on cancel")
34+ }
35+}
36+
37+func TestCapturedDemuxCancelWhileBlocked(t *testing.T) {
38+ ctx, cancel := context.WithCancel(context.Background())
39+ in := make(chan ports.CaptureDone, 1)
40+ wl := make(chan ports.CaptureDone) // never read
41+ exited := make(chan struct{})
42+ go func() { routeCaptured(ctx, in, wl, nil); close(exited) }()
43+ in <- ports.CaptureDone{ID: 1}
44+ cancel()
45+ select {
46+ case <-exited:
47+ case <-time.After(2 * time.Second):
48+ t.Fatal("routeCaptured blocked on a full destination after cancel")
49+ }
50+}
51+
52+func TestCapturedDemuxDropsInternalWithoutConsumer(t *testing.T) {
53+ ctx, cancel := context.WithCancel(context.Background())
54+ defer cancel()
55+ in := make(chan ports.CaptureDone, 2)
56+ wl := make(chan ports.CaptureDone, 2)
57+ go routeCaptured(ctx, in, wl, nil)
58+ in <- ports.CaptureDone{ID: ports.CaptureIDInternal | 1}
59+ in <- ports.CaptureDone{ID: 2}
60+ require.Equal(t, uint64(2), recvDone(t, wl).ID)
61+}
62+
63+func recvDone(t *testing.T, ch <-chan ports.CaptureDone) ports.CaptureDone {
64+ t.Helper()
65+ select {
66+ case d := <-ch:
67+ return d
68+ case <-time.After(2 * time.Second):
69+ t.Fatal("no capture reply")
70+ return ports.CaptureDone{}
71+ }
72+}
@@ -0,0 +1,72 @@
1+package app
2+
3+import (
4+ "context"
5+ "testing"
6+ "time"
7+
8+ "github.com/stretchr/testify/require"
9+
10+ "github.com/bnema/neferwl/internal/ports"
11+)
12+
13+func TestCapturedDemux(t *testing.T) {
14+ ctx, cancel := context.WithCancel(context.Background())
15+ defer cancel()
16+ in := make(chan ports.CaptureDone, 4)
17+ wl := make(chan ports.CaptureDone, 4)
18+ internal := make(chan ports.CaptureDone, 4)
19+ exited := make(chan struct{})
20+ go func() { routeCaptured(ctx, in, wl, internal); close(exited) }()
21+
22+ in <- ports.CaptureDone{ID: 7}
23+ in <- ports.CaptureDone{ID: ports.CaptureIDInternal | 3}
24+ require.Equal(t, uint64(7), recvDone(t, wl).ID)
25+ require.Equal(t, ports.CaptureIDInternal|3, recvDone(t, internal).ID)
26+ require.Empty(t, wl)
27+ require.Empty(t, internal)
28+
29+ cancel()
30+ select {
31+ case <-exited:
32+ case <-time.After(2 * time.Second):
33+ t.Fatal("routeCaptured did not return on cancel")
34+ }
35+}
36+
37+func TestCapturedDemuxCancelWhileBlocked(t *testing.T) {
38+ ctx, cancel := context.WithCancel(context.Background())
39+ in := make(chan ports.CaptureDone, 1)
40+ wl := make(chan ports.CaptureDone) // never read
41+ exited := make(chan struct{})
42+ go func() { routeCaptured(ctx, in, wl, nil); close(exited) }()
43+ in <- ports.CaptureDone{ID: 1}
44+ cancel()
45+ select {
46+ case <-exited:
47+ case <-time.After(2 * time.Second):
48+ t.Fatal("routeCaptured blocked on a full destination after cancel")
49+ }
50+}
51+
52+func TestCapturedDemuxDropsInternalWithoutConsumer(t *testing.T) {
53+ ctx, cancel := context.WithCancel(context.Background())
54+ defer cancel()
55+ in := make(chan ports.CaptureDone, 2)
56+ wl := make(chan ports.CaptureDone, 2)
57+ go routeCaptured(ctx, in, wl, nil)
58+ in <- ports.CaptureDone{ID: ports.CaptureIDInternal | 1}
59+ in <- ports.CaptureDone{ID: 2}
60+ require.Equal(t, uint64(2), recvDone(t, wl).ID)
61+}
62+
63+func recvDone(t *testing.T, ch <-chan ports.CaptureDone) ports.CaptureDone {
64+ t.Helper()
65+ select {
66+ case d := <-ch:
67+ return d
68+ case <-time.After(2 * time.Second):
69+ t.Fatal("no capture reply")
70+ return ports.CaptureDone{}
71+ }
72+}
A internal/app/hintdebug.go +51 -0
@@ -0,0 +1,51 @@
1+package app
2+
3+import (
4+ "context"
5+ "os"
6+ "path/filepath"
7+
8+ "github.com/bnema/neferwl/internal/adapters/atspi"
9+ "github.com/bnema/neferwl/internal/adapters/hints"
10+ "github.com/bnema/neferwl/internal/adapters/logging"
11+)
12+
13+// hintOptions builds the hint worker options: the accessibility source, and,
14+// when NEFERWL_HINT_DEBUG is "1", the dump directory of scanned frames,
15+// $XDG_STATE_HOME/neferwl/hints. The accessibility bus is only contacted
16+// by the first scan; without it the worker reads windows with OCR only.
17+func hintOptions(ctx context.Context) hints.Options {
18+ opts := hintDump(ctx)
19+ // The source connects on its first scan, and again after a failure. The
20+ // hints worker owns it and closes it when it returns.
21+ src := atspi.New()
22+ opts.Source = src
23+ return opts
24+}
25+
26+// hintDump reads NEFERWL_HINT_DEBUG once.
27+func hintDump(ctx context.Context) hints.Options {
28+ if os.Getenv("NEFERWL_HINT_DEBUG") != "1" {
29+ return hints.Options{}
30+ }
31+ log := logging.For(ctx, "hints")
32+ state := os.Getenv("XDG_STATE_HOME")
33+ if state == "" {
34+ home, err := os.UserHomeDir()
35+ if err != nil {
36+ log.Warn().Err(err).Msg("hint debug dumps disabled")
37+ return hints.Options{}
38+ }
39+ state = filepath.Join(home, ".local", "state")
40+ }
41+ dir := filepath.Join(state, "neferwl", "hints")
42+ log.Warn().Str("dir", dir).Msg("hint debug dumps enabled")
43+ return hints.Options{DumpDir: dir}
44+}
45+
46+// hintDisabledNote returns the once-only notice for a hint bind pressed while
47+// hint mode is off. The logger is made now; nothing else of hint mode exists.
48+func hintDisabledNote(ctx context.Context) func() {
49+ log := logging.For(ctx, "core")
50+ return func() { log.Info().Msg("hint mode is disabled; set hint.enable = on and restart") }
51+}
@@ -0,0 +1,51 @@
1+package app
2+
3+import (
4+ "context"
5+ "os"
6+ "path/filepath"
7+
8+ "github.com/bnema/neferwl/internal/adapters/atspi"
9+ "github.com/bnema/neferwl/internal/adapters/hints"
10+ "github.com/bnema/neferwl/internal/adapters/logging"
11+)
12+
13+// hintOptions builds the hint worker options: the accessibility source, and,
14+// when NEFERWL_HINT_DEBUG is "1", the dump directory of scanned frames,
15+// $XDG_STATE_HOME/neferwl/hints. The accessibility bus is only contacted
16+// by the first scan; without it the worker reads windows with OCR only.
17+func hintOptions(ctx context.Context) hints.Options {
18+ opts := hintDump(ctx)
19+ // The source connects on its first scan, and again after a failure. The
20+ // hints worker owns it and closes it when it returns.
21+ src := atspi.New()
22+ opts.Source = src
23+ return opts
24+}
25+
26+// hintDump reads NEFERWL_HINT_DEBUG once.
27+func hintDump(ctx context.Context) hints.Options {
28+ if os.Getenv("NEFERWL_HINT_DEBUG") != "1" {
29+ return hints.Options{}
30+ }
31+ log := logging.For(ctx, "hints")
32+ state := os.Getenv("XDG_STATE_HOME")
33+ if state == "" {
34+ home, err := os.UserHomeDir()
35+ if err != nil {
36+ log.Warn().Err(err).Msg("hint debug dumps disabled")
37+ return hints.Options{}
38+ }
39+ state = filepath.Join(home, ".local", "state")
40+ }
41+ dir := filepath.Join(state, "neferwl", "hints")
42+ log.Warn().Str("dir", dir).Msg("hint debug dumps enabled")
43+ return hints.Options{DumpDir: dir}
44+}
45+
46+// hintDisabledNote returns the once-only notice for a hint bind pressed while
47+// hint mode is off. The logger is made now; nothing else of hint mode exists.
48+func hintDisabledNote(ctx context.Context) func() {
49+ log := logging.For(ctx, "core")
50+ return func() { log.Info().Msg("hint mode is disabled; set hint.enable = on and restart") }
51+}
A internal/app/hintoptin_test.go +68 -0
@@ -0,0 +1,68 @@
1+package app
2+
3+import (
4+ "context"
5+ "runtime"
6+ "testing"
7+ "time"
8+
9+ "github.com/stretchr/testify/require"
10+
11+ "github.com/bnema/neferwl/internal/adapters/clock"
12+ "github.com/bnema/neferwl/internal/adapters/config"
13+ "github.com/bnema/neferwl/internal/adapters/logging"
14+ "github.com/bnema/neferwl/internal/adapters/xkb"
15+ "github.com/bnema/neferwl/internal/ports"
16+)
17+
18+func TestHintPipesOffByDefault(t *testing.T) {
19+ require.False(t, config.Defaults().Hint.Enable)
20+ off := newPipes(false, false)
21+ require.Nil(t, off.hintScans)
22+ require.Nil(t, off.hintResults)
23+ require.Nil(t, off.hintCaptured)
24+ // No demultiplexing: the outputs reply on the wayland channel itself.
25+ require.Equal(t, off.captured, off.capturedAll)
26+ ch := off.core()
27+ require.Nil(t, ch.HintScans)
28+ require.Nil(t, ch.Hints)
29+ on := newPipes(false, true)
30+ require.NotNil(t, on.hintScans)
31+ require.NotNil(t, on.hintResults)
32+ require.NotNil(t, on.hintCaptured)
33+ require.NotEqual(t, on.captured, on.capturedAll)
34+}
35+
36+func TestHintOffStartsNoHintGoroutine(t *testing.T) {
37+ t.Setenv("XDG_RUNTIME_DIR", t.TempDir())
38+ ctx, cancel := context.WithCancel(context.Background())
39+ defer cancel()
40+ done := make(chan error, 1)
41+ go func() {
42+ done <- Run(ctx, Options{Backend: "headless", NoXwayland: true, Config: config.Defaults(), NoTerminal: true, Timeout: 800 * time.Millisecond})
43+ }()
44+ time.Sleep(400 * time.Millisecond)
45+ buf := make([]byte, 1<<20)
46+ dump := string(buf[:runtime.Stack(buf, true)])
47+ require.NotContains(t, dump, "adapters/hints.")
48+ require.NotContains(t, dump, "adapters/atspi.")
49+ require.NotContains(t, dump, "routeCaptured")
50+ select {
51+ case <-done:
52+ case <-time.After(3 * time.Second):
53+ t.Fatal("session did not stop")
54+ }
55+}
56+
57+func TestRelayConfigKeepsStartupHintEnable(t *testing.T) {
58+ ctx, cancel := context.WithCancel(context.Background())
59+ defer cancel()
60+ cur := config.Defaults()
61+ in := make(chan ports.ConfigChanged, 1)
62+ out := make(chan ports.ConfigChanged, 1)
63+ go relayConfig(ctx, cur, in, out, make(chan *xkb.Keymap, 1), make(chan ports.InputDevicesConfig, 1), newCursors(clock.System{}, 0), make(chan ports.ClientCommand, 1), logging.For(ctx, "config"))
64+ next := config.Defaults()
65+ next.Hint.Enable = true
66+ in <- ports.ConfigChanged{Config: next}
67+ require.False(t, (<-out).Config.Hint.Enable)
68+}
@@ -0,0 +1,68 @@
1+package app
2+
3+import (
4+ "context"
5+ "runtime"
6+ "testing"
7+ "time"
8+
9+ "github.com/stretchr/testify/require"
10+
11+ "github.com/bnema/neferwl/internal/adapters/clock"
12+ "github.com/bnema/neferwl/internal/adapters/config"
13+ "github.com/bnema/neferwl/internal/adapters/logging"
14+ "github.com/bnema/neferwl/internal/adapters/xkb"
15+ "github.com/bnema/neferwl/internal/ports"
16+)
17+
18+func TestHintPipesOffByDefault(t *testing.T) {
19+ require.False(t, config.Defaults().Hint.Enable)
20+ off := newPipes(false, false)
21+ require.Nil(t, off.hintScans)
22+ require.Nil(t, off.hintResults)
23+ require.Nil(t, off.hintCaptured)
24+ // No demultiplexing: the outputs reply on the wayland channel itself.
25+ require.Equal(t, off.captured, off.capturedAll)
26+ ch := off.core()
27+ require.Nil(t, ch.HintScans)
28+ require.Nil(t, ch.Hints)
29+ on := newPipes(false, true)
30+ require.NotNil(t, on.hintScans)
31+ require.NotNil(t, on.hintResults)
32+ require.NotNil(t, on.hintCaptured)
33+ require.NotEqual(t, on.captured, on.capturedAll)
34+}
35+
36+func TestHintOffStartsNoHintGoroutine(t *testing.T) {
37+ t.Setenv("XDG_RUNTIME_DIR", t.TempDir())
38+ ctx, cancel := context.WithCancel(context.Background())
39+ defer cancel()
40+ done := make(chan error, 1)
41+ go func() {
42+ done <- Run(ctx, Options{Backend: "headless", NoXwayland: true, Config: config.Defaults(), NoTerminal: true, Timeout: 800 * time.Millisecond})
43+ }()
44+ time.Sleep(400 * time.Millisecond)
45+ buf := make([]byte, 1<<20)
46+ dump := string(buf[:runtime.Stack(buf, true)])
47+ require.NotContains(t, dump, "adapters/hints.")
48+ require.NotContains(t, dump, "adapters/atspi.")
49+ require.NotContains(t, dump, "routeCaptured")
50+ select {
51+ case <-done:
52+ case <-time.After(3 * time.Second):
53+ t.Fatal("session did not stop")
54+ }
55+}
56+
57+func TestRelayConfigKeepsStartupHintEnable(t *testing.T) {
58+ ctx, cancel := context.WithCancel(context.Background())
59+ defer cancel()
60+ cur := config.Defaults()
61+ in := make(chan ports.ConfigChanged, 1)
62+ out := make(chan ports.ConfigChanged, 1)
63+ go relayConfig(ctx, cur, in, out, make(chan *xkb.Keymap, 1), make(chan ports.InputDevicesConfig, 1), newCursors(clock.System{}, 0), make(chan ports.ClientCommand, 1), logging.For(ctx, "config"))
64+ next := config.Defaults()
65+ next.Hint.Enable = true
66+ in <- ports.ConfigChanged{Config: next}
67+ require.False(t, (<-out).Config.Hint.Enable)
68+}
M internal/app/session.go +17 -2
@@ -14,6 +14,7 @@ import (
1414 "github.com/bnema/neferwl/internal/adapters/config"
1515 "github.com/bnema/neferwl/internal/adapters/drm"
1616 "github.com/bnema/neferwl/internal/adapters/headlessinput"
17+ "github.com/bnema/neferwl/internal/adapters/hints"
1718 "github.com/bnema/neferwl/internal/adapters/launcher"
1819 "github.com/bnema/neferwl/internal/adapters/libinput"
1920 "github.com/bnema/neferwl/internal/adapters/logging"
@@ -51,6 +52,7 @@ type session struct {
5152 km *xkb.Keymap
5253 core *core.Core
5354 dmabuf ports.DMABufSupport
55+ hintOpts hints.Options
5456 wsIDs *workspaceid.IDs
5557 captureAllow *captureallow.Store
5658 cast *screencast.Service // nil without the D-Bus services
@@ -90,16 +92,20 @@ func assembleSession(ctx context.Context, opts Options) (s *session, err error)
9092 return s, err
9193 }
9294 }
93- s.p = newPipes(s.hw != nil)
95+ s.p = newPipes(s.hw != nil, opts.Config.Hint.Enable)
9496 // core() carries channels only; the gate, clock and terminal flag are
9597 // explicit core.Options.
96- if s.core, err = core.New(opts.Config, s.p.core(), core.Options{Security: s.security, Clock: s.clock, Terminal: !opts.NoTerminal}); err != nil {
98+ if s.core, err = core.New(opts.Config, s.p.core(), core.Options{Security: s.security, Clock: s.clock, Terminal: !opts.NoTerminal, HintDisabled: hintDisabledNote(ctx)}); err != nil {
9799 return s, err
98100 }
99101 if s.km, err = xkb.New(xkb.RMLVO{Layout: opts.Config.Keyboard.Layout, Variant: opts.Config.Keyboard.Variant, Options: opts.Config.Keyboard.Options}); err != nil {
100102 return s, err
101103 }
102104 runtimeDir := os.Getenv("XDG_RUNTIME_DIR")
105+ // Hint mode off: no accessibility connection, no debug env, nothing.
106+ if opts.Config.Hint.Enable {
107+ s.hintOpts = hintOptions(ctx)
108+ }
103109 // Every output renders on the same GPU: its formats are the clients'.
104110 s.dmabuf = vulkan.Probe()
105111 log.Info().Int("formats", len(s.dmabuf.Formats)).Msg("dmabuf")
@@ -340,6 +346,15 @@ func (s *session) run(ctx context.Context, inject func(chan<- ports.InputEvent))
340346 log.Warn().Err(err).Msg("capture allowlist not watched, restart to apply changes")
341347 }
342348 }()
349+ if p.hintScans != nil {
350+ workers.Add(2)
351+ go func() { defer workers.Done(); routeCaptured(ctx, p.capturedAll, p.captured, p.internalCaptured()) }()
352+ go func() {
353+ defer workers.Done()
354+ // Run only returns when ctx is done: nothing to report.
355+ _ = hints.Run(ctx, p.hintScans, p.captures, p.hintCaptured, p.hintResults, s.hintOpts)
356+ }()
357+ }
343358 go func() { defer workers.Done(); done <- child.Run(ctx, p.spawn) }()
344359 go func() { defer workers.Done(); done <- c.Run(ctx) }()
345360 workers.Add(1)
@@ -14,6 +14,7 @@ import (
14 "github.com/bnema/neferwl/internal/adapters/config"14 "github.com/bnema/neferwl/internal/adapters/config"
15 "github.com/bnema/neferwl/internal/adapters/drm"15 "github.com/bnema/neferwl/internal/adapters/drm"
16 "github.com/bnema/neferwl/internal/adapters/headlessinput"16 "github.com/bnema/neferwl/internal/adapters/headlessinput"
17+ "github.com/bnema/neferwl/internal/adapters/hints"
17 "github.com/bnema/neferwl/internal/adapters/launcher"18 "github.com/bnema/neferwl/internal/adapters/launcher"
18 "github.com/bnema/neferwl/internal/adapters/libinput"19 "github.com/bnema/neferwl/internal/adapters/libinput"
19 "github.com/bnema/neferwl/internal/adapters/logging"20 "github.com/bnema/neferwl/internal/adapters/logging"
@@ -51,6 +52,7 @@ type session struct {
51 km *xkb.Keymap52 km *xkb.Keymap
52 core *core.Core53 core *core.Core
53 dmabuf ports.DMABufSupport54 dmabuf ports.DMABufSupport
55+ hintOpts hints.Options
54 wsIDs *workspaceid.IDs56 wsIDs *workspaceid.IDs
55 captureAllow *captureallow.Store57 captureAllow *captureallow.Store
56 cast *screencast.Service // nil without the D-Bus services58 cast *screencast.Service // nil without the D-Bus services
@@ -90,16 +92,20 @@ func assembleSession(ctx context.Context, opts Options) (s *session, err error)
90 return s, err92 return s, err
91 }93 }
92 }94 }
93- s.p = newPipes(s.hw != nil)95+ s.p = newPipes(s.hw != nil, opts.Config.Hint.Enable)
94 // core() carries channels only; the gate, clock and terminal flag are96 // core() carries channels only; the gate, clock and terminal flag are
95 // explicit core.Options.97 // explicit core.Options.
96- if s.core, err = core.New(opts.Config, s.p.core(), core.Options{Security: s.security, Clock: s.clock, Terminal: !opts.NoTerminal}); err != nil {98+ if s.core, err = core.New(opts.Config, s.p.core(), core.Options{Security: s.security, Clock: s.clock, Terminal: !opts.NoTerminal, HintDisabled: hintDisabledNote(ctx)}); err != nil {
97 return s, err99 return s, err
98 }100 }
99 if s.km, err = xkb.New(xkb.RMLVO{Layout: opts.Config.Keyboard.Layout, Variant: opts.Config.Keyboard.Variant, Options: opts.Config.Keyboard.Options}); err != nil {101 if s.km, err = xkb.New(xkb.RMLVO{Layout: opts.Config.Keyboard.Layout, Variant: opts.Config.Keyboard.Variant, Options: opts.Config.Keyboard.Options}); err != nil {
100 return s, err102 return s, err
101 }103 }
102 runtimeDir := os.Getenv("XDG_RUNTIME_DIR")104 runtimeDir := os.Getenv("XDG_RUNTIME_DIR")
105+ // Hint mode off: no accessibility connection, no debug env, nothing.
106+ if opts.Config.Hint.Enable {
107+ s.hintOpts = hintOptions(ctx)
108+ }
103 // Every output renders on the same GPU: its formats are the clients'.109 // Every output renders on the same GPU: its formats are the clients'.
104 s.dmabuf = vulkan.Probe()110 s.dmabuf = vulkan.Probe()
105 log.Info().Int("formats", len(s.dmabuf.Formats)).Msg("dmabuf")111 log.Info().Int("formats", len(s.dmabuf.Formats)).Msg("dmabuf")
@@ -340,6 +346,15 @@ func (s *session) run(ctx context.Context, inject func(chan<- ports.InputEvent))
340 log.Warn().Err(err).Msg("capture allowlist not watched, restart to apply changes")346 log.Warn().Err(err).Msg("capture allowlist not watched, restart to apply changes")
341 }347 }
342 }()348 }()
349+ if p.hintScans != nil {
350+ workers.Add(2)
351+ go func() { defer workers.Done(); routeCaptured(ctx, p.capturedAll, p.captured, p.internalCaptured()) }()
352+ go func() {
353+ defer workers.Done()
354+ // Run only returns when ctx is done: nothing to report.
355+ _ = hints.Run(ctx, p.hintScans, p.captures, p.hintCaptured, p.hintResults, s.hintOpts)
356+ }()
357+ }
343 go func() { defer workers.Done(); done <- child.Run(ctx, p.spawn) }()358 go func() { defer workers.Done(); done <- child.Run(ctx, p.spawn) }()
344 go func() { defer workers.Done(); done <- c.Run(ctx) }()359 go func() { defer workers.Done(); done <- c.Run(ctx) }()
345 workers.Add(1)360 workers.Add(1)
M internal/app/wiring.go +34 -5
@@ -46,10 +46,21 @@ type pipes struct {
4646 // move running slides.
4747 frames chan ports.OutputFrame
4848
49+ // Hint mode: core asks the hints adapter for a scan and gets the result.
50+ hintScans chan ports.HintScanRequest
51+ hintResults chan ports.HintScanResult
52+
4953 captures chan ports.CaptureRequest
50- // Every admitted capture owns one reply slot until wayland consumes it.
51- // Keep this capacity tied to admission so output routing never waits.
52- captured chan ports.CaptureDone
54+ // Every admitted capture owns one reply slot until routeCaptured takes
55+ // it. Keep this capacity tied to admission so output routing never waits.
56+ // With hint mode on, the outputs reply on capturedAll and routeCaptured splits the replies by
57+ // ports.CaptureIDInternal into captured (the wayland consumer) and
58+ // hintCaptured (internal consumers). With hint mode off, capturedAll is
59+ // captured itself and there is no routeCaptured.
60+ capturedAll chan ports.CaptureDone
61+ captured chan ports.CaptureDone
62+ // hintCaptured carries internal capture replies to the hints adapter.
63+ hintCaptured chan ports.CaptureDone
5364
5465 outputFormats chan ports.OutputFormats
5566 outputHeads chan ports.OutputHeads
@@ -69,7 +80,10 @@ type pipes struct {
6980 castCommands chan ports.CastCommand
7081 }
7182
72-func newPipes(realSession bool) *pipes {
83+// newPipes makes the channels. The hint channels exist only with hint set
84+// (hint.enable = on): without them core's hint action is a no-op and nothing
85+// of hint mode is allocated.
86+func newPipes(realSession, hint bool) *pipes {
7387 p := &pipes{
7488 securityChanges: make(chan ports.SecurityState, 8),
7589 securityEvents: make(chan ports.SecurityBackendEvent, 64),
@@ -112,6 +126,15 @@ func newPipes(realSession bool) *pipes {
112126
113127 castCommands: make(chan ports.CastCommand, 64),
114128 }
129+ // Without hint mode every reply is for the wayland consumer: the outputs
130+ // reply on its channel directly and nothing demultiplexes.
131+ p.capturedAll = p.captured
132+ if hint {
133+ p.capturedAll = make(chan ports.CaptureDone, ports.MaxCaptureInflight)
134+ p.hintScans = make(chan ports.HintScanRequest, 1)
135+ p.hintResults = make(chan ports.HintScanResult, 1)
136+ p.hintCaptured = make(chan ports.CaptureDone, 1)
137+ }
115138 if realSession {
116139 p.scales = make(chan ports.ScaleChanged, 8)
117140 }
@@ -136,9 +159,15 @@ func (p *pipes) core() core.Channels {
136159 ConfigErrors: p.configErrors,
137160 Scales: p.scales,
138161 Frames: p.frames,
162+ HintScans: p.hintScans,
163+ Hints: p.hintResults,
139164 }
140165 }
141166
167+// internalCaptured is the channel internal capture replies are routed to: the
168+// hints adapter reads it.
169+func (p *pipes) internalCaptured() chan<- ports.CaptureDone { return p.hintCaptured }
170+
142171 // wayland is the wayland server's side.
143172 func (p *pipes) wayland() wayland.Channels {
144173 return wayland.Channels{
@@ -175,7 +204,7 @@ func (p *pipes) outputs(security ports.SessionSecurity, presented ports.Presenta
175204 content: content,
176205 frames: p.frames,
177206 captures: p.captures,
178- captured: p.captured,
207+ captured: p.capturedAll,
179208 formats: p.outputFormats,
180209 heads: p.outputHeads,
181210 leaseRequests: p.leaseRequests,
@@ -46,10 +46,21 @@ type pipes struct {
46 // move running slides.46 // move running slides.
47 frames chan ports.OutputFrame47 frames chan ports.OutputFrame
48 48
49+ // Hint mode: core asks the hints adapter for a scan and gets the result.
50+ hintScans chan ports.HintScanRequest
51+ hintResults chan ports.HintScanResult
52+
49 captures chan ports.CaptureRequest53 captures chan ports.CaptureRequest
50- // Every admitted capture owns one reply slot until wayland consumes it.54+ // Every admitted capture owns one reply slot until routeCaptured takes
51- // Keep this capacity tied to admission so output routing never waits.55+ // it. Keep this capacity tied to admission so output routing never waits.
52- captured chan ports.CaptureDone56+ // With hint mode on, the outputs reply on capturedAll and routeCaptured splits the replies by
57+ // ports.CaptureIDInternal into captured (the wayland consumer) and
58+ // hintCaptured (internal consumers). With hint mode off, capturedAll is
59+ // captured itself and there is no routeCaptured.
60+ capturedAll chan ports.CaptureDone
61+ captured chan ports.CaptureDone
62+ // hintCaptured carries internal capture replies to the hints adapter.
63+ hintCaptured chan ports.CaptureDone
53 64
54 outputFormats chan ports.OutputFormats65 outputFormats chan ports.OutputFormats
55 outputHeads chan ports.OutputHeads66 outputHeads chan ports.OutputHeads
@@ -69,7 +80,10 @@ type pipes struct {
69 castCommands chan ports.CastCommand80 castCommands chan ports.CastCommand
70 }81 }
71 82
72-func newPipes(realSession bool) *pipes {83+// newPipes makes the channels. The hint channels exist only with hint set
84+// (hint.enable = on): without them core's hint action is a no-op and nothing
85+// of hint mode is allocated.
86+func newPipes(realSession, hint bool) *pipes {
73 p := &pipes{87 p := &pipes{
74 securityChanges: make(chan ports.SecurityState, 8),88 securityChanges: make(chan ports.SecurityState, 8),
75 securityEvents: make(chan ports.SecurityBackendEvent, 64),89 securityEvents: make(chan ports.SecurityBackendEvent, 64),
@@ -112,6 +126,15 @@ func newPipes(realSession bool) *pipes {
112 126
113 castCommands: make(chan ports.CastCommand, 64),127 castCommands: make(chan ports.CastCommand, 64),
114 }128 }
129+ // Without hint mode every reply is for the wayland consumer: the outputs
130+ // reply on its channel directly and nothing demultiplexes.
131+ p.capturedAll = p.captured
132+ if hint {
133+ p.capturedAll = make(chan ports.CaptureDone, ports.MaxCaptureInflight)
134+ p.hintScans = make(chan ports.HintScanRequest, 1)
135+ p.hintResults = make(chan ports.HintScanResult, 1)
136+ p.hintCaptured = make(chan ports.CaptureDone, 1)
137+ }
115 if realSession {138 if realSession {
116 p.scales = make(chan ports.ScaleChanged, 8)139 p.scales = make(chan ports.ScaleChanged, 8)
117 }140 }
@@ -136,9 +159,15 @@ func (p *pipes) core() core.Channels {
136 ConfigErrors: p.configErrors,159 ConfigErrors: p.configErrors,
137 Scales: p.scales,160 Scales: p.scales,
138 Frames: p.frames,161 Frames: p.frames,
162+ HintScans: p.hintScans,
163+ Hints: p.hintResults,
139 }164 }
140 }165 }
141 166
167+// internalCaptured is the channel internal capture replies are routed to: the
168+// hints adapter reads it.
169+func (p *pipes) internalCaptured() chan<- ports.CaptureDone { return p.hintCaptured }
170+
142 // wayland is the wayland server's side.171 // wayland is the wayland server's side.
143 func (p *pipes) wayland() wayland.Channels {172 func (p *pipes) wayland() wayland.Channels {
144 return wayland.Channels{173 return wayland.Channels{
@@ -175,7 +204,7 @@ func (p *pipes) outputs(security ports.SessionSecurity, presented ports.Presenta
175 content: content,204 content: content,
176 frames: p.frames,205 frames: p.frames,
177 captures: p.captures,206 captures: p.captures,
178- captured: p.captured,207+ captured: p.capturedAll,
179 formats: p.outputFormats,208 formats: p.outputFormats,
180 heads: p.outputHeads,209 heads: p.outputHeads,
181 leaseRequests: p.leaseRequests,210 leaseRequests: p.leaseRequests,
M internal/core/action.go +6 -0
@@ -63,6 +63,9 @@ const (
6363 // command key is held (switcher.go).
6464 ActionSwitchColumnNext Action = "switch-column-next"
6565 ActionSwitchColumnPrev Action = "switch-column-prev"
66+ // Show keyboard hint labels over the links and buttons of the focused
67+ // output (hint.go).
68+ ActionHint Action = "hint"
6669 )
6770
6871 // monitorDirection returns the direction of the neighbor screen targeted by
@@ -303,6 +306,9 @@ func (c *Core) dispatchAction(a Action) Effect {
303306 return Effect{}
304307 }
305308 switch a {
309+ case ActionHint:
310+ c.hintRequest()
311+ return Effect{}
306312 case ActionFocusMonitorLeft, ActionFocusMonitorRight, ActionFocusMonitorUp, ActionFocusMonitorDown:
307313 if i := c.neighbor(monitorDirection(a)); i >= 0 {
308314 c.focusScreen = i
@@ -63,6 +63,9 @@ const (
63 // command key is held (switcher.go).63 // command key is held (switcher.go).
64 ActionSwitchColumnNext Action = "switch-column-next"64 ActionSwitchColumnNext Action = "switch-column-next"
65 ActionSwitchColumnPrev Action = "switch-column-prev"65 ActionSwitchColumnPrev Action = "switch-column-prev"
66+ // Show keyboard hint labels over the links and buttons of the focused
67+ // output (hint.go).
68+ ActionHint Action = "hint"
66 )69 )
67 70
68 // monitorDirection returns the direction of the neighbor screen targeted by71 // monitorDirection returns the direction of the neighbor screen targeted by
@@ -303,6 +306,9 @@ func (c *Core) dispatchAction(a Action) Effect {
303 return Effect{}306 return Effect{}
304 }307 }
305 switch a {308 switch a {
309+ case ActionHint:
310+ c.hintRequest()
311+ return Effect{}
306 case ActionFocusMonitorLeft, ActionFocusMonitorRight, ActionFocusMonitorUp, ActionFocusMonitorDown:312 case ActionFocusMonitorLeft, ActionFocusMonitorRight, ActionFocusMonitorUp, ActionFocusMonitorDown:
307 if i := c.neighbor(monitorDirection(a)); i >= 0 {313 if i := c.neighbor(monitorDirection(a)); i >= 0 {
308 c.focusScreen = i314 c.focusScreen = i
M internal/core/core.go +30 -1
@@ -47,6 +47,12 @@ type Channels struct {
4747 // animating scene (Scene.Animating): a running slide moves one step per
4848 // report. Without them it moves on a timer.
4949 Frames <-chan ports.OutputFrame
50+ // HintScans, when set, receives the scan request of each hint action
51+ // (capacity 1; core never waits: a full channel cancels the hint).
52+ // Hints carries the results back. With HintScans nil the hint action
53+ // does nothing.
54+ HintScans chan<- ports.HintScanRequest
55+ Hints <-chan ports.HintScanResult
5056 }
5157
5258 // Options are the services and policy core is constructed with.
@@ -57,6 +63,9 @@ type Options struct {
5763 Clock ports.Clock
5864 // Terminal enables automatic terminal opening according to terminal.auto-open.
5965 Terminal bool
66+ // HintDisabled, when set, is called the first time the hint action runs
67+ // without a scanner (hint mode off). It must not block.
68+ HintDisabled func()
6069 }
6170
6271 // fullscreenGrace is how long after mapping a window's fullscreen request
@@ -157,6 +166,14 @@ type Core struct {
157166 // capt is the capture state (capture.go, capindicator.go).
158167 capt captureState
159168 flash focusFlash
169+ // hint is keyboard hint mode (hint.go).
170+ hint hintState
171+ // keyState is the real modifier state of the latest key event, swallowed
172+ // keys included.
173+ keyState ports.ModState
174+ // synthetic is set while a hint click runs through the pointer handlers:
175+ // compositor modifier+button actions (window drag) do not apply to it.
176+ synthetic bool
160177 // switcher is the column switcher in progress: the screen it is on and
161178 // the timer that shows its cards (switcher.go).
162179 switcher struct {
@@ -275,7 +292,7 @@ func (c *Core) apply(cfg ports.Config) error {
275292 }
276293 switch Action(a) {
277294 case "none", ActionSpawnTerminal, ActionFocusColumnLeft, ActionFocusColumnRight, ActionFocusWindowUp, ActionFocusWindowDown, ActionMoveColumnLeft, ActionMoveColumnRight, ActionCycleColumnWidth, ActionMaximizeColumn, ActionToggleFullscreen, ActionToggleWindowStash, ActionToggleStashVisible, ActionToggleOverview, ActionCloseWindow, ActionQuit, ActionFocusWorkspacePrev, ActionFocusWorkspaceNext, ActionMoveColumnToWorkspacePrev, ActionMoveColumnToWorkspaceNext, ActionMoveWindowToWorkspacePrev, ActionMoveWindowToWorkspaceNext, ActionFocusMonitorLeft, ActionFocusMonitorRight, ActionFocusMonitorUp, ActionFocusMonitorDown, ActionMoveWorkspaceLeft, ActionMoveWorkspaceRight, ActionMoveWorkspaceToMonitorUp, ActionMoveWorkspaceToMonitorDown, ActionConsumeOrExpelLeft, ActionConsumeOrExpelRight,
278- ActionMoveWindowUp, ActionMoveWindowDown, ActionMoveWorkspacePrev, ActionMoveWorkspaceNext, ActionToggleFloating, ActionSwitchColumnNext, ActionSwitchColumnPrev:
295+ ActionMoveWindowUp, ActionMoveWindowDown, ActionMoveWorkspacePrev, ActionMoveWorkspaceNext, ActionToggleFloating, ActionSwitchColumnNext, ActionSwitchColumnPrev, ActionHint:
279296 default:
280297 return fmt.Errorf("invalid action %q", a)
281298 }
@@ -634,6 +651,7 @@ func (c *Core) publishFrame(ctx context.Context, only *screen) error {
634651 return errSecurityChanged
635652 }
636653 c.syncSecurity()
654+ c.hintValidate()
637655 if c.security.Protected {
638656 return c.publishProtected(ctx)
639657 }
@@ -1206,6 +1224,17 @@ func (c *Core) Run(ctx context.Context) error {
12061224 return nil
12071225 }
12081226 continue
1227+ case res, ok := <-c.ch.Hints:
1228+ if !ok {
1229+ c.ch.Hints = nil
1230+ continue
1231+ }
1232+ if c.securityCheckpoint(ctx) != nil {
1233+ return nil
1234+ }
1235+ if c.hintResult(res) && c.publish(ctx) != nil {
1236+ return nil
1237+ }
12091238 case ev, ok := <-c.ch.Output:
12101239 if !ok {
12111240 c.ch.Output = nil
@@ -47,6 +47,12 @@ type Channels struct {
47 // animating scene (Scene.Animating): a running slide moves one step per47 // animating scene (Scene.Animating): a running slide moves one step per
48 // report. Without them it moves on a timer.48 // report. Without them it moves on a timer.
49 Frames <-chan ports.OutputFrame49 Frames <-chan ports.OutputFrame
50+ // HintScans, when set, receives the scan request of each hint action
51+ // (capacity 1; core never waits: a full channel cancels the hint).
52+ // Hints carries the results back. With HintScans nil the hint action
53+ // does nothing.
54+ HintScans chan<- ports.HintScanRequest
55+ Hints <-chan ports.HintScanResult
50 }56 }
51 57
52 // Options are the services and policy core is constructed with.58 // Options are the services and policy core is constructed with.
@@ -57,6 +63,9 @@ type Options struct {
57 Clock ports.Clock63 Clock ports.Clock
58 // Terminal enables automatic terminal opening according to terminal.auto-open.64 // Terminal enables automatic terminal opening according to terminal.auto-open.
59 Terminal bool65 Terminal bool
66+ // HintDisabled, when set, is called the first time the hint action runs
67+ // without a scanner (hint mode off). It must not block.
68+ HintDisabled func()
60 }69 }
61 70
62 // fullscreenGrace is how long after mapping a window's fullscreen request71 // fullscreenGrace is how long after mapping a window's fullscreen request
@@ -157,6 +166,14 @@ type Core struct {
157 // capt is the capture state (capture.go, capindicator.go).166 // capt is the capture state (capture.go, capindicator.go).
158 capt captureState167 capt captureState
159 flash focusFlash168 flash focusFlash
169+ // hint is keyboard hint mode (hint.go).
170+ hint hintState
171+ // keyState is the real modifier state of the latest key event, swallowed
172+ // keys included.
173+ keyState ports.ModState
174+ // synthetic is set while a hint click runs through the pointer handlers:
175+ // compositor modifier+button actions (window drag) do not apply to it.
176+ synthetic bool
160 // switcher is the column switcher in progress: the screen it is on and177 // switcher is the column switcher in progress: the screen it is on and
161 // the timer that shows its cards (switcher.go).178 // the timer that shows its cards (switcher.go).
162 switcher struct {179 switcher struct {
@@ -275,7 +292,7 @@ func (c *Core) apply(cfg ports.Config) error {
275 }292 }
276 switch Action(a) {293 switch Action(a) {
277 case "none", ActionSpawnTerminal, ActionFocusColumnLeft, ActionFocusColumnRight, ActionFocusWindowUp, ActionFocusWindowDown, ActionMoveColumnLeft, ActionMoveColumnRight, ActionCycleColumnWidth, ActionMaximizeColumn, ActionToggleFullscreen, ActionToggleWindowStash, ActionToggleStashVisible, ActionToggleOverview, ActionCloseWindow, ActionQuit, ActionFocusWorkspacePrev, ActionFocusWorkspaceNext, ActionMoveColumnToWorkspacePrev, ActionMoveColumnToWorkspaceNext, ActionMoveWindowToWorkspacePrev, ActionMoveWindowToWorkspaceNext, ActionFocusMonitorLeft, ActionFocusMonitorRight, ActionFocusMonitorUp, ActionFocusMonitorDown, ActionMoveWorkspaceLeft, ActionMoveWorkspaceRight, ActionMoveWorkspaceToMonitorUp, ActionMoveWorkspaceToMonitorDown, ActionConsumeOrExpelLeft, ActionConsumeOrExpelRight,294 case "none", ActionSpawnTerminal, ActionFocusColumnLeft, ActionFocusColumnRight, ActionFocusWindowUp, ActionFocusWindowDown, ActionMoveColumnLeft, ActionMoveColumnRight, ActionCycleColumnWidth, ActionMaximizeColumn, ActionToggleFullscreen, ActionToggleWindowStash, ActionToggleStashVisible, ActionToggleOverview, ActionCloseWindow, ActionQuit, ActionFocusWorkspacePrev, ActionFocusWorkspaceNext, ActionMoveColumnToWorkspacePrev, ActionMoveColumnToWorkspaceNext, ActionMoveWindowToWorkspacePrev, ActionMoveWindowToWorkspaceNext, ActionFocusMonitorLeft, ActionFocusMonitorRight, ActionFocusMonitorUp, ActionFocusMonitorDown, ActionMoveWorkspaceLeft, ActionMoveWorkspaceRight, ActionMoveWorkspaceToMonitorUp, ActionMoveWorkspaceToMonitorDown, ActionConsumeOrExpelLeft, ActionConsumeOrExpelRight,
278- ActionMoveWindowUp, ActionMoveWindowDown, ActionMoveWorkspacePrev, ActionMoveWorkspaceNext, ActionToggleFloating, ActionSwitchColumnNext, ActionSwitchColumnPrev:295+ ActionMoveWindowUp, ActionMoveWindowDown, ActionMoveWorkspacePrev, ActionMoveWorkspaceNext, ActionToggleFloating, ActionSwitchColumnNext, ActionSwitchColumnPrev, ActionHint:
279 default:296 default:
280 return fmt.Errorf("invalid action %q", a)297 return fmt.Errorf("invalid action %q", a)
281 }298 }
@@ -634,6 +651,7 @@ func (c *Core) publishFrame(ctx context.Context, only *screen) error {
634 return errSecurityChanged651 return errSecurityChanged
635 }652 }
636 c.syncSecurity()653 c.syncSecurity()
654+ c.hintValidate()
637 if c.security.Protected {655 if c.security.Protected {
638 return c.publishProtected(ctx)656 return c.publishProtected(ctx)
639 }657 }
@@ -1206,6 +1224,17 @@ func (c *Core) Run(ctx context.Context) error {
1206 return nil1224 return nil
1207 }1225 }
1208 continue1226 continue
1227+ case res, ok := <-c.ch.Hints:
1228+ if !ok {
1229+ c.ch.Hints = nil
1230+ continue
1231+ }
1232+ if c.securityCheckpoint(ctx) != nil {
1233+ return nil
1234+ }
1235+ if c.hintResult(res) && c.publish(ctx) != nil {
1236+ return nil
1237+ }
1209 case ev, ok := <-c.ch.Output:1238 case ev, ok := <-c.ch.Output:
1210 if !ok {1239 if !ok {
1211 c.ch.Output = nil1240 c.ch.Output = nil
M internal/core/drag.go +1 -1
@@ -90,7 +90,7 @@ func (c *Core) draggable(id WindowID) (*screen, *Workspace, bool) {
9090 // startDrag starts a cmd + button drag on the window under the pointer.
9191 // It reports whether it did: the press is then not forwarded.
9292 func (c *Core) startDrag(ctx context.Context, v ports.PointerButton) (bool, error) {
93- if !v.Pressed || c.drag != nil || c.held() != 0 || c.grab != 0 || c.cmdMod == 0 || c.mods&c.cmdMod == 0 {
93+ if c.synthetic || !v.Pressed || c.drag != nil || c.held() != 0 || c.grab != 0 || c.cmdMod == 0 || c.mods&c.cmdMod == 0 {
9494 return false, nil
9595 }
9696 if v.Button != btnLeft && v.Button != btnRight {
@@ -90,7 +90,7 @@ func (c *Core) draggable(id WindowID) (*screen, *Workspace, bool) {
90 // startDrag starts a cmd + button drag on the window under the pointer.90 // startDrag starts a cmd + button drag on the window under the pointer.
91 // It reports whether it did: the press is then not forwarded.91 // It reports whether it did: the press is then not forwarded.
92 func (c *Core) startDrag(ctx context.Context, v ports.PointerButton) (bool, error) {92 func (c *Core) startDrag(ctx context.Context, v ports.PointerButton) (bool, error) {
93- if !v.Pressed || c.drag != nil || c.held() != 0 || c.grab != 0 || c.cmdMod == 0 || c.mods&c.cmdMod == 0 {93+ if c.synthetic || !v.Pressed || c.drag != nil || c.held() != 0 || c.grab != 0 || c.cmdMod == 0 || c.mods&c.cmdMod == 0 {
94 return false, nil94 return false, nil
95 }95 }
96 if v.Button != btnLeft && v.Button != btnRight {96 if v.Button != btnLeft && v.Button != btnRight {
A internal/core/hint.go +361 -0
@@ -0,0 +1,361 @@
1+package core
2+
3+import (
4+ "cmp"
5+ "context"
6+ "slices"
7+
8+ "github.com/bnema/neferwl/internal/ports"
9+)
10+
11+// Hint mode: the hint action asks the scanner (HintScans) for the clickable
12+// targets in the last frame of the focused output, labels the ones that sit
13+// over a managed window, grabs the keyboard for the label letters, and clicks
14+// the centre of the target whose label was typed. Everything here runs on the
15+// owner goroutine.
16+
17+// hintAlphabet orders the label letters: home row first.
18+const hintAlphabet = "fjdkslaghrueiwoqptyvbcnxmz"
19+
20+// hintMaxTargets is the most targets labelled: every two-letter label.
21+const hintMaxTargets = len(hintAlphabet) * len(hintAlphabet)
22+
23+type hintPhase uint8
24+
25+const (
26+ hintIdle hintPhase = iota
27+ hintPending
28+ hintShown
29+)
30+
31+// hintTarget is a labelled click target. x and y are global logical
32+// coordinates of its centre.
33+type hintTarget struct {
34+ x, y float64
35+ window WindowID
36+ label string
37+ rect ports.Rect
38+ text bool // found by OCR
39+}
40+
41+type hintState struct {
42+ phase hintPhase
43+ seq uint64 // last request sequence number
44+ output string
45+ // ws is the workspace on the hint output when hint mode began: another
46+ // one on screen ends it.
47+ ws *Workspace
48+ geom hintGeom
49+ windows []ports.HintWindow // visible windows at request time
50+ targets []hintTarget
51+ typed string
52+}
53+
54+// hintGeom is the output geometry a scan was requested on: a result or a
55+// label set is only good for the same one.
56+type hintGeom struct {
57+ x, y int
58+ w, h int // logical size
59+ width, height int // buffer size, physical pixels
60+ scale float64
61+ transform ports.BufferTransform
62+}
63+
64+func (sc *screen) hintGeom() hintGeom {
65+ r := sc.rect()
66+ return hintGeom{x: sc.x, y: sc.y, w: r.W, h: r.H, width: sc.info.Width, height: sc.info.Height, scale: sc.scale, transform: sc.transform}
67+}
68+
69+// hintRequest starts hint mode on the focused screen. It does nothing
70+// (reporting once through Options.HintDisabled) without a scanner, while the session is protected, while hint mode is
71+// active, or on an output with a non-normal transform.
72+func (c *Core) hintRequest() {
73+ if c.ch.HintScans == nil {
74+ if c.opts.HintDisabled != nil {
75+ c.opts.HintDisabled()
76+ c.opts.HintDisabled = nil
77+ }
78+ return
79+ }
80+ if c.security.Protected || c.hint.phase != hintIdle || c.hintPointerBusy() {
81+ return
82+ }
83+ sc := c.cur()
84+ if sc.name() == "" || sc.transform != 0 || sc.mon.ov.open || sc.info.Width <= 0 || sc.info.Height <= 0 {
85+ return
86+ }
87+ wins := c.hintWindows(sc)
88+ req := ports.HintScanRequest{Seq: c.hint.seq + 1, Output: sc.name(), Width: sc.info.Width, Height: sc.info.Height, Scale: sc.scale, LinksOnly: c.cfg.Hint.LinksOnly, Windows: wins}
89+ select {
90+ case c.ch.HintScans <- req:
91+ default:
92+ return
93+ }
94+ c.hint = hintState{phase: hintPending, seq: req.Seq, output: sc.name(), ws: sc.mon.Current(), geom: sc.hintGeom(), windows: slices.Clone(wins)}
95+}
96+
97+// hintWindows lists the windows drawn on sc, bottom to top, with their client
98+// area (placements are output-local).
99+func (c *Core) hintWindows(sc *screen) []ports.HintWindow {
100+ view := sc.mon.viewport()
101+ var out []ports.HintWindow
102+ for _, p := range sc.shownLayout() {
103+ if p.Hidden || p.Preview > 0 || !view.shows(p) {
104+ continue
105+ }
106+ r := c.clientRect(p)
107+ if r.W <= 0 || r.H <= 0 {
108+ continue
109+ }
110+ w := ports.HintWindow{ID: p.ID, Rect: r, Focused: p.Focused}
111+ rec := c.windows.lookup(p.ID)
112+ w.PID, w.AppID = rec.client.PID, rec.client.AppID
113+ out = append(out, w)
114+ }
115+ return out
116+}
117+
118+// hintPointerBusy reports whether the pointer is in use by something a
119+// synthetic click would disturb: a pointer constraint, a grab, a held
120+// button (swallowed ones included: they stay in c.buttons), a window or data
121+// drag, or the column switcher.
122+func (c *Core) hintPointerBusy() bool {
123+ return c.constraint.Mode != ports.ConstraintNone || c.constrained.Mode != ports.ConstraintNone ||
124+ c.grab != 0 || len(c.buttons) > 0 || c.drag != nil || c.dataDrag != nil || c.switching()
125+}
126+
127+// hintResult handles a scanner result and reports whether to publish.
128+func (c *Core) hintResult(res ports.HintScanResult) bool {
129+ h := &c.hint
130+ if h.phase != hintPending || res.Seq != h.seq || res.Output != h.output {
131+ return false
132+ }
133+ if res.Err != nil || c.security.Protected {
134+ c.hintCancel()
135+ return true
136+ }
137+ i := c.screenIndex(h.output)
138+ if i < 0 {
139+ c.hintCancel()
140+ return true
141+ }
142+ sc := c.screens[i]
143+ if sc.hintGeom() != h.geom {
144+ c.hintCancel()
145+ return true
146+ }
147+ out := sc.rect()
148+ var kept []hintTarget
149+ for _, t := range res.Targets {
150+ x := float64(sc.x) + float64(t.Rect.X) + float64(t.Rect.W)/2
151+ y := float64(sc.y) + float64(t.Rect.Y) + float64(t.Rect.H)/2
152+ if x < float64(out.X) || y < float64(out.Y) || x >= float64(out.X+out.W) || y >= float64(out.Y+out.H) {
153+ continue
154+ }
155+ id, _, _ := c.hit(x, y)
156+ if id == 0 {
157+ continue
158+ }
159+ root := c.popupRoot(id)
160+ if s, _ := c.screenOf(root); s != sc {
161+ // A layer surface, or a window of another output.
162+ continue
163+ }
164+ if !c.hintOwned(sc, t, root) {
165+ continue
166+ }
167+ kept = append(kept, hintTarget{x: x, y: y, window: id, rect: t.Rect, text: t.Text})
168+ }
169+ if len(kept) == 0 {
170+ c.hintCancel()
171+ return true
172+ }
173+ slices.SortStableFunc(kept, func(a, b hintTarget) int {
174+ return cmp.Or(cmp.Compare(a.rect.Y, b.rect.Y), cmp.Compare(a.rect.X, b.rect.X))
175+ })
176+ if len(kept) > hintMaxTargets {
177+ kept = kept[:hintMaxTargets]
178+ }
179+ n := len(hintAlphabet)
180+ for i := range kept {
181+ if len(kept) <= n {
182+ kept[i].label = string(hintAlphabet[i])
183+ } else {
184+ kept[i].label = string([]byte{hintAlphabet[i/n], hintAlphabet[i%n]})
185+ }
186+ }
187+ h.phase, h.targets, h.typed = hintShown, kept, ""
188+ return true
189+}
190+
191+// hintOwned reports whether the window under a target is the one it was
192+// scanned on. An accessibility target names its window; an OCR target only
193+// counts over a window that was visible when the scan was requested and has
194+// not moved or resized since.
195+func (c *Core) hintOwned(sc *screen, t ports.HintTarget, root WindowID) bool {
196+ if t.Window != 0 {
197+ return root == t.Window
198+ }
199+ for _, w := range c.hint.windows {
200+ if w.ID != root {
201+ continue
202+ }
203+ for _, p := range sc.shownLayout() {
204+ if p.ID == root {
205+ return c.clientRect(p) == w.Rect
206+ }
207+ }
208+ }
209+ return false
210+}
211+
212+// hintCancel ends hint mode.
213+func (c *Core) hintCancel() { c.hint = hintState{seq: c.hint.seq} }
214+
215+// hintValidate ends hint mode when what it was started on is gone: the
216+// session became protected, the output went or changed geometry, another workspace is on screen
217+// there, or the overview opened. It runs at the start of every publish.
218+func (c *Core) hintValidate() {
219+ h := &c.hint
220+ if h.phase == hintIdle {
221+ return
222+ }
223+ i := c.screenIndex(h.output)
224+ if c.security.Protected || i < 0 || c.screens[i].mon.ov.open || c.screens[i].mon.Current() != h.ws || c.cur().mon.ov.open {
225+ c.hintCancel()
226+ return
227+ }
228+ if c.screens[i].hintGeom() != h.geom {
229+ c.hintCancel()
230+ }
231+}
232+
233+// hintKey handles a key while hint mode is active: the key never goes on.
234+// A press is the whole job; releases take the usual swallow path.
235+func (c *Core) hintKey(ctx context.Context, key ports.KeyEvent) (bool, error) {
236+ if c.hint.phase == hintIdle || !key.Pressed {
237+ return false, nil
238+ }
239+ c.pressed[heldKey(key)] = true
240+ h := &c.hint
241+ if h.phase == hintPending || keyName(key.Keysym) == "Escape" {
242+ c.hintCancel()
243+ return true, c.publish(ctx)
244+ }
245+ if keyName(key.Keysym) == "BackSpace" {
246+ if h.typed != "" {
247+ h.typed = h.typed[:len(h.typed)-1]
248+ }
249+ return true, c.publish(ctx)
250+ }
251+ name := key.Base
252+ if name == "" {
253+ name = key.Keysym
254+ }
255+ name = keyName(name)
256+ if len(name) != 1 || name[0] < 'a' || name[0] > 'z' {
257+ c.hintCancel()
258+ return true, c.publish(ctx)
259+ }
260+ typed := h.typed + name
261+ var match *hintTarget
262+ any := false
263+ for i := range h.targets {
264+ if len(h.targets[i].label) >= len(typed) && h.targets[i].label[:len(typed)] == typed {
265+ any = true
266+ if h.targets[i].label == typed {
267+ match = &h.targets[i]
268+ }
269+ }
270+ }
271+ if !any {
272+ return true, nil
273+ }
274+ if match == nil {
275+ h.typed = typed
276+ return true, c.publish(ctx)
277+ }
278+ return true, c.hintClick(ctx, *match)
279+}
280+
281+// hintClick ends hint mode and left-clicks the target's centre, provided the
282+// pointer is free and the same window is still under it. The click is
283+// synthetic: c.synthetic keeps the physical modifiers (the bind's Cmd may
284+// still be down) from turning it into a window drag, without touching c.mods.
285+//
286+// A target found by OCR is clicked with hint.text-click held (terminals open
287+// a link on a modified click). Every click gives the window the keyboard
288+// first, then sends exactly the modifiers to hold (none for other targets),
289+// and the real modifier state is always sent back afterwards.
290+func (c *Core) hintClick(ctx context.Context, t hintTarget) (err error) {
291+ c.hintCancel()
292+ if err := c.publish(ctx); err != nil {
293+ return err
294+ }
295+ if !c.hintClickable(t) {
296+ return nil
297+ }
298+ // Every click states its modifiers (possibly none): the bind's Cmd may
299+ // still be down and the client must not see it on a plain click.
300+ if err := c.clickFocus(ctx, t.window); err != nil {
301+ return err
302+ }
303+ if err := c.command(ctx, ports.SetModifiers{ID: t.window, Mods: c.hintMods(t)}); err != nil {
304+ return err
305+ }
306+ defer func() {
307+ if e := c.command(ctx, ports.SetModifiers{ID: t.window, Restore: true, State: c.keyState}); err == nil {
308+ err = e
309+ }
310+ }()
311+ c.synthetic = true
312+ defer func() { c.synthetic = false }()
313+ if err := c.pointerMotion(ctx, ports.PointerMotion{X: t.x, Y: t.y, Time: c.motionTime}); err != nil {
314+ return err
315+ }
316+ // The motion may have been bent (constraint) or the layout changed:
317+ // press only where the target is.
318+ if c.cursorX != t.x || c.cursorY != t.y || !c.hintClickable(t) {
319+ return nil
320+ }
321+ if err := c.pointerButton(ctx, ports.PointerButton{Button: btnLeft, Pressed: true, Time: c.motionTime}); err != nil {
322+ return err
323+ }
324+ return c.pointerButton(ctx, ports.PointerButton{Button: btnLeft, Time: c.motionTime})
325+}
326+
327+// hintMods are the modifiers to hold for a click on t, read at click time so
328+// a reloaded configuration applies.
329+func (c *Core) hintMods(t hintTarget) ports.Mods {
330+ if !t.text {
331+ return 0
332+ }
333+ return c.cfg.Hint.TextClick
334+}
335+
336+// hintClickable reports whether a click on t may go out now: unprotected,
337+// pointer free, and the selected window is still under the target's centre.
338+func (c *Core) hintClickable(t hintTarget) bool {
339+ if c.security.Protected || c.hintPointerBusy() {
340+ return false
341+ }
342+ id, _, _ := c.hit(t.x, t.y)
343+ return id == t.window
344+}
345+
346+// hintLabels are the labels to draw on sc: those still matching what was
347+// typed. Nil unless sc is the hint output.
348+func (c *Core) hintLabels(sc *screen) []ports.HintLabel {
349+ h := &c.hint
350+ if h.phase != hintShown || sc.name() != h.output {
351+ return nil
352+ }
353+ var out []ports.HintLabel
354+ for _, t := range h.targets {
355+ if len(t.label) < len(h.typed) || t.label[:len(h.typed)] != h.typed {
356+ continue
357+ }
358+ out = append(out, ports.HintLabel{Rect: ports.Rect{X: t.rect.X, Y: t.rect.Y}, Text: t.label, Typed: len(h.typed)})
359+ }
360+ return out
361+}
@@ -0,0 +1,361 @@
1+package core
2+
3+import (
4+ "cmp"
5+ "context"
6+ "slices"
7+
8+ "github.com/bnema/neferwl/internal/ports"
9+)
10+
11+// Hint mode: the hint action asks the scanner (HintScans) for the clickable
12+// targets in the last frame of the focused output, labels the ones that sit
13+// over a managed window, grabs the keyboard for the label letters, and clicks
14+// the centre of the target whose label was typed. Everything here runs on the
15+// owner goroutine.
16+
17+// hintAlphabet orders the label letters: home row first.
18+const hintAlphabet = "fjdkslaghrueiwoqptyvbcnxmz"
19+
20+// hintMaxTargets is the most targets labelled: every two-letter label.
21+const hintMaxTargets = len(hintAlphabet) * len(hintAlphabet)
22+
23+type hintPhase uint8
24+
25+const (
26+ hintIdle hintPhase = iota
27+ hintPending
28+ hintShown
29+)
30+
31+// hintTarget is a labelled click target. x and y are global logical
32+// coordinates of its centre.
33+type hintTarget struct {
34+ x, y float64
35+ window WindowID
36+ label string
37+ rect ports.Rect
38+ text bool // found by OCR
39+}
40+
41+type hintState struct {
42+ phase hintPhase
43+ seq uint64 // last request sequence number
44+ output string
45+ // ws is the workspace on the hint output when hint mode began: another
46+ // one on screen ends it.
47+ ws *Workspace
48+ geom hintGeom
49+ windows []ports.HintWindow // visible windows at request time
50+ targets []hintTarget
51+ typed string
52+}
53+
54+// hintGeom is the output geometry a scan was requested on: a result or a
55+// label set is only good for the same one.
56+type hintGeom struct {
57+ x, y int
58+ w, h int // logical size
59+ width, height int // buffer size, physical pixels
60+ scale float64
61+ transform ports.BufferTransform
62+}
63+
64+func (sc *screen) hintGeom() hintGeom {
65+ r := sc.rect()
66+ return hintGeom{x: sc.x, y: sc.y, w: r.W, h: r.H, width: sc.info.Width, height: sc.info.Height, scale: sc.scale, transform: sc.transform}
67+}
68+
69+// hintRequest starts hint mode on the focused screen. It does nothing
70+// (reporting once through Options.HintDisabled) without a scanner, while the session is protected, while hint mode is
71+// active, or on an output with a non-normal transform.
72+func (c *Core) hintRequest() {
73+ if c.ch.HintScans == nil {
74+ if c.opts.HintDisabled != nil {
75+ c.opts.HintDisabled()
76+ c.opts.HintDisabled = nil
77+ }
78+ return
79+ }
80+ if c.security.Protected || c.hint.phase != hintIdle || c.hintPointerBusy() {
81+ return
82+ }
83+ sc := c.cur()
84+ if sc.name() == "" || sc.transform != 0 || sc.mon.ov.open || sc.info.Width <= 0 || sc.info.Height <= 0 {
85+ return
86+ }
87+ wins := c.hintWindows(sc)
88+ req := ports.HintScanRequest{Seq: c.hint.seq + 1, Output: sc.name(), Width: sc.info.Width, Height: sc.info.Height, Scale: sc.scale, LinksOnly: c.cfg.Hint.LinksOnly, Windows: wins}
89+ select {
90+ case c.ch.HintScans <- req:
91+ default:
92+ return
93+ }
94+ c.hint = hintState{phase: hintPending, seq: req.Seq, output: sc.name(), ws: sc.mon.Current(), geom: sc.hintGeom(), windows: slices.Clone(wins)}
95+}
96+
97+// hintWindows lists the windows drawn on sc, bottom to top, with their client
98+// area (placements are output-local).
99+func (c *Core) hintWindows(sc *screen) []ports.HintWindow {
100+ view := sc.mon.viewport()
101+ var out []ports.HintWindow
102+ for _, p := range sc.shownLayout() {
103+ if p.Hidden || p.Preview > 0 || !view.shows(p) {
104+ continue
105+ }
106+ r := c.clientRect(p)
107+ if r.W <= 0 || r.H <= 0 {
108+ continue
109+ }
110+ w := ports.HintWindow{ID: p.ID, Rect: r, Focused: p.Focused}
111+ rec := c.windows.lookup(p.ID)
112+ w.PID, w.AppID = rec.client.PID, rec.client.AppID
113+ out = append(out, w)
114+ }
115+ return out
116+}
117+
118+// hintPointerBusy reports whether the pointer is in use by something a
119+// synthetic click would disturb: a pointer constraint, a grab, a held
120+// button (swallowed ones included: they stay in c.buttons), a window or data
121+// drag, or the column switcher.
122+func (c *Core) hintPointerBusy() bool {
123+ return c.constraint.Mode != ports.ConstraintNone || c.constrained.Mode != ports.ConstraintNone ||
124+ c.grab != 0 || len(c.buttons) > 0 || c.drag != nil || c.dataDrag != nil || c.switching()
125+}
126+
127+// hintResult handles a scanner result and reports whether to publish.
128+func (c *Core) hintResult(res ports.HintScanResult) bool {
129+ h := &c.hint
130+ if h.phase != hintPending || res.Seq != h.seq || res.Output != h.output {
131+ return false
132+ }
133+ if res.Err != nil || c.security.Protected {
134+ c.hintCancel()
135+ return true
136+ }
137+ i := c.screenIndex(h.output)
138+ if i < 0 {
139+ c.hintCancel()
140+ return true
141+ }
142+ sc := c.screens[i]
143+ if sc.hintGeom() != h.geom {
144+ c.hintCancel()
145+ return true
146+ }
147+ out := sc.rect()
148+ var kept []hintTarget
149+ for _, t := range res.Targets {
150+ x := float64(sc.x) + float64(t.Rect.X) + float64(t.Rect.W)/2
151+ y := float64(sc.y) + float64(t.Rect.Y) + float64(t.Rect.H)/2
152+ if x < float64(out.X) || y < float64(out.Y) || x >= float64(out.X+out.W) || y >= float64(out.Y+out.H) {
153+ continue
154+ }
155+ id, _, _ := c.hit(x, y)
156+ if id == 0 {
157+ continue
158+ }
159+ root := c.popupRoot(id)
160+ if s, _ := c.screenOf(root); s != sc {
161+ // A layer surface, or a window of another output.
162+ continue
163+ }
164+ if !c.hintOwned(sc, t, root) {
165+ continue
166+ }
167+ kept = append(kept, hintTarget{x: x, y: y, window: id, rect: t.Rect, text: t.Text})
168+ }
169+ if len(kept) == 0 {
170+ c.hintCancel()
171+ return true
172+ }
173+ slices.SortStableFunc(kept, func(a, b hintTarget) int {
174+ return cmp.Or(cmp.Compare(a.rect.Y, b.rect.Y), cmp.Compare(a.rect.X, b.rect.X))
175+ })
176+ if len(kept) > hintMaxTargets {
177+ kept = kept[:hintMaxTargets]
178+ }
179+ n := len(hintAlphabet)
180+ for i := range kept {
181+ if len(kept) <= n {
182+ kept[i].label = string(hintAlphabet[i])
183+ } else {
184+ kept[i].label = string([]byte{hintAlphabet[i/n], hintAlphabet[i%n]})
185+ }
186+ }
187+ h.phase, h.targets, h.typed = hintShown, kept, ""
188+ return true
189+}
190+
191+// hintOwned reports whether the window under a target is the one it was
192+// scanned on. An accessibility target names its window; an OCR target only
193+// counts over a window that was visible when the scan was requested and has
194+// not moved or resized since.
195+func (c *Core) hintOwned(sc *screen, t ports.HintTarget, root WindowID) bool {
196+ if t.Window != 0 {
197+ return root == t.Window
198+ }
199+ for _, w := range c.hint.windows {
200+ if w.ID != root {
201+ continue
202+ }
203+ for _, p := range sc.shownLayout() {
204+ if p.ID == root {
205+ return c.clientRect(p) == w.Rect
206+ }
207+ }
208+ }
209+ return false
210+}
211+
212+// hintCancel ends hint mode.
213+func (c *Core) hintCancel() { c.hint = hintState{seq: c.hint.seq} }
214+
215+// hintValidate ends hint mode when what it was started on is gone: the
216+// session became protected, the output went or changed geometry, another workspace is on screen
217+// there, or the overview opened. It runs at the start of every publish.
218+func (c *Core) hintValidate() {
219+ h := &c.hint
220+ if h.phase == hintIdle {
221+ return
222+ }
223+ i := c.screenIndex(h.output)
224+ if c.security.Protected || i < 0 || c.screens[i].mon.ov.open || c.screens[i].mon.Current() != h.ws || c.cur().mon.ov.open {
225+ c.hintCancel()
226+ return
227+ }
228+ if c.screens[i].hintGeom() != h.geom {
229+ c.hintCancel()
230+ }
231+}
232+
233+// hintKey handles a key while hint mode is active: the key never goes on.
234+// A press is the whole job; releases take the usual swallow path.
235+func (c *Core) hintKey(ctx context.Context, key ports.KeyEvent) (bool, error) {
236+ if c.hint.phase == hintIdle || !key.Pressed {
237+ return false, nil
238+ }
239+ c.pressed[heldKey(key)] = true
240+ h := &c.hint
241+ if h.phase == hintPending || keyName(key.Keysym) == "Escape" {
242+ c.hintCancel()
243+ return true, c.publish(ctx)
244+ }
245+ if keyName(key.Keysym) == "BackSpace" {
246+ if h.typed != "" {
247+ h.typed = h.typed[:len(h.typed)-1]
248+ }
249+ return true, c.publish(ctx)
250+ }
251+ name := key.Base
252+ if name == "" {
253+ name = key.Keysym
254+ }
255+ name = keyName(name)
256+ if len(name) != 1 || name[0] < 'a' || name[0] > 'z' {
257+ c.hintCancel()
258+ return true, c.publish(ctx)
259+ }
260+ typed := h.typed + name
261+ var match *hintTarget
262+ any := false
263+ for i := range h.targets {
264+ if len(h.targets[i].label) >= len(typed) && h.targets[i].label[:len(typed)] == typed {
265+ any = true
266+ if h.targets[i].label == typed {
267+ match = &h.targets[i]
268+ }
269+ }
270+ }
271+ if !any {
272+ return true, nil
273+ }
274+ if match == nil {
275+ h.typed = typed
276+ return true, c.publish(ctx)
277+ }
278+ return true, c.hintClick(ctx, *match)
279+}
280+
281+// hintClick ends hint mode and left-clicks the target's centre, provided the
282+// pointer is free and the same window is still under it. The click is
283+// synthetic: c.synthetic keeps the physical modifiers (the bind's Cmd may
284+// still be down) from turning it into a window drag, without touching c.mods.
285+//
286+// A target found by OCR is clicked with hint.text-click held (terminals open
287+// a link on a modified click). Every click gives the window the keyboard
288+// first, then sends exactly the modifiers to hold (none for other targets),
289+// and the real modifier state is always sent back afterwards.
290+func (c *Core) hintClick(ctx context.Context, t hintTarget) (err error) {
291+ c.hintCancel()
292+ if err := c.publish(ctx); err != nil {
293+ return err
294+ }
295+ if !c.hintClickable(t) {
296+ return nil
297+ }
298+ // Every click states its modifiers (possibly none): the bind's Cmd may
299+ // still be down and the client must not see it on a plain click.
300+ if err := c.clickFocus(ctx, t.window); err != nil {
301+ return err
302+ }
303+ if err := c.command(ctx, ports.SetModifiers{ID: t.window, Mods: c.hintMods(t)}); err != nil {
304+ return err
305+ }
306+ defer func() {
307+ if e := c.command(ctx, ports.SetModifiers{ID: t.window, Restore: true, State: c.keyState}); err == nil {
308+ err = e
309+ }
310+ }()
311+ c.synthetic = true
312+ defer func() { c.synthetic = false }()
313+ if err := c.pointerMotion(ctx, ports.PointerMotion{X: t.x, Y: t.y, Time: c.motionTime}); err != nil {
314+ return err
315+ }
316+ // The motion may have been bent (constraint) or the layout changed:
317+ // press only where the target is.
318+ if c.cursorX != t.x || c.cursorY != t.y || !c.hintClickable(t) {
319+ return nil
320+ }
321+ if err := c.pointerButton(ctx, ports.PointerButton{Button: btnLeft, Pressed: true, Time: c.motionTime}); err != nil {
322+ return err
323+ }
324+ return c.pointerButton(ctx, ports.PointerButton{Button: btnLeft, Time: c.motionTime})
325+}
326+
327+// hintMods are the modifiers to hold for a click on t, read at click time so
328+// a reloaded configuration applies.
329+func (c *Core) hintMods(t hintTarget) ports.Mods {
330+ if !t.text {
331+ return 0
332+ }
333+ return c.cfg.Hint.TextClick
334+}
335+
336+// hintClickable reports whether a click on t may go out now: unprotected,
337+// pointer free, and the selected window is still under the target's centre.
338+func (c *Core) hintClickable(t hintTarget) bool {
339+ if c.security.Protected || c.hintPointerBusy() {
340+ return false
341+ }
342+ id, _, _ := c.hit(t.x, t.y)
343+ return id == t.window
344+}
345+
346+// hintLabels are the labels to draw on sc: those still matching what was
347+// typed. Nil unless sc is the hint output.
348+func (c *Core) hintLabels(sc *screen) []ports.HintLabel {
349+ h := &c.hint
350+ if h.phase != hintShown || sc.name() != h.output {
351+ return nil
352+ }
353+ var out []ports.HintLabel
354+ for _, t := range h.targets {
355+ if len(t.label) < len(h.typed) || t.label[:len(h.typed)] != h.typed {
356+ continue
357+ }
358+ out = append(out, ports.HintLabel{Rect: ports.Rect{X: t.rect.X, Y: t.rect.Y}, Text: t.label, Typed: len(h.typed)})
359+ }
360+ return out
361+}
A internal/core/hint_test.go +834 -0
@@ -0,0 +1,834 @@
1+package core_test
2+
3+import (
4+ "context"
5+ "fmt"
6+ "slices"
7+ "sync/atomic"
8+ "testing"
9+ "time"
10+
11+ "github.com/bnema/neferwl/internal/core"
12+ portsmocks "github.com/bnema/neferwl/internal/mocks/ports"
13+ "github.com/bnema/neferwl/internal/ports"
14+)
15+
16+type hintHarness struct {
17+ t *testing.T
18+ input chan ports.InputEvent
19+ client chan ports.ClientEvent
20+ output chan ports.OutputEvent
21+ commands chan ports.ClientCommand
22+ scenes chan []ports.Scene
23+ scans chan ports.HintScanRequest
24+ hints chan ports.HintScanResult
25+ win ports.Rect // window 1, output-local
26+ // state is the security state input is stamped with.
27+ state ports.SecurityState
28+}
29+
30+// newHint starts a core with one 1000x800 output "OUT-1" and window 1 mapped.
31+func newHint(t *testing.T, opts core.Options) *hintHarness {
32+ t.Helper()
33+ return newHintMods(t, opts, 0)
34+}
35+
36+// newHintMods is newHint with hint.text-click set to textClick.
37+func newHintMods(t *testing.T, opts core.Options, textClick ports.Mods) *hintHarness {
38+ t.Helper()
39+ cfg := scrollDefaults()
40+ cfg.Hint.TextClick = textClick
41+ cfg.Border.Width = 0
42+ cfg.Layout.Gaps = 20
43+ cfg.Binds["Cmd+g"] = "hint"
44+ h := &hintHarness{
45+ t: t,
46+ input: make(chan ports.InputEvent, 32),
47+ client: make(chan ports.ClientEvent, 32),
48+ output: make(chan ports.OutputEvent, 8),
49+ commands: make(chan ports.ClientCommand, 256),
50+ scenes: make(chan []ports.Scene, 1),
51+ scans: make(chan ports.HintScanRequest, 1),
52+ // Unbuffered: a send returns once core took the result.
53+ hints: make(chan ports.HintScanResult),
54+ }
55+ ch := core.Channels{Client: h.client, Input: h.input, Output: h.output, Commands: h.commands, Scenes: h.scenes, HintScans: h.scans, Hints: h.hints}
56+ c, err := core.New(cfg, ch, opts)
57+ if err != nil {
58+ t.Fatal(err)
59+ }
60+ ctx, cancel := context.WithCancel(context.Background())
61+ done := make(chan error, 1)
62+ go func() { done <- c.Run(ctx) }()
63+ t.Cleanup(func() { cancel(); receive(t, done) })
64+ h.output <- ports.OutputAdded{Info: ports.OutputInfo{Name: "OUT-1", Width: 1000, Height: 800}}
65+ scene(t, h.scenes)
66+ h.client <- ports.WindowMapped{ID: 1, AppID: "org.test.app", PID: 4242}
67+ s := scene(t, h.scenes)
68+ h.win = s.Windows[0].Rect
69+ h.drain()
70+ return h
71+}
72+
73+func (h *hintHarness) drain() {
74+ for {
75+ select {
76+ case <-h.commands:
77+ default:
78+ return
79+ }
80+ }
81+}
82+
83+// press and release send one key event.
84+func (h *hintHarness) press(sym string, mods ports.Mods) {
85+ h.send(ports.KeyEvent{Keysym: sym, Base: sym, Mods: mods, Pressed: true})
86+}
87+
88+func (h *hintHarness) release(sym string, mods ports.Mods) {
89+ h.send(ports.KeyEvent{Keysym: sym, Base: sym, Mods: mods})
90+}
91+
92+// send gives core an input event stamped with the current security state.
93+func (h *hintHarness) send(ev ports.InputEvent) {
94+ h.input <- ports.SecurityInput{State: h.state, Event: ev}
95+}
96+
97+// start presses the hint bind and returns the request.
98+func (h *hintHarness) start() ports.HintScanRequest {
99+ h.t.Helper()
100+ h.press("g", ports.ModSuper)
101+ req := receive(h.t, h.scans)
102+ h.release("g", 0) // the user lets go of the whole chord
103+ return req
104+}
105+
106+// show starts hint mode and answers with targets, returning the next scene.
107+func (h *hintHarness) show(targets ...ports.HintTarget) ports.Scene {
108+ h.t.Helper()
109+ req := h.start()
110+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: targets}
111+ return h.labelled(len(targets), 0)
112+}
113+
114+// until returns the first scene, from the next one on, that satisfies ok:
115+// the bind press and other input publish scenes before the one waited for.
116+func (h *hintHarness) until(ok func(ports.Scene) bool) ports.Scene {
117+ h.t.Helper()
118+ for {
119+ if s := scene(h.t, h.scenes); ok(s) {
120+ return s
121+ }
122+ }
123+}
124+
125+// labelled waits for a scene with n labels, all typed with typed letters.
126+func (h *hintHarness) labelled(n, typed int) ports.Scene {
127+ h.t.Helper()
128+ return h.until(func(s ports.Scene) bool {
129+ return len(s.HintLabels) == n && (n == 0 || s.HintLabels[0].Typed == typed)
130+ })
131+}
132+
133+// inside is a target of the given size inside window 1.
134+func (h *hintHarness) inside(i int) ports.HintTarget {
135+ return ports.HintTarget{Rect: ports.Rect{X: h.win.X + 10 + i, Y: h.win.Y + 10 + 12*i, W: 40, H: 10}}
136+}
137+
138+func TestHintRequestOnBind(t *testing.T) {
139+ h := newHint(t, core.Options{})
140+ req := h.start()
141+ if req.Output != "OUT-1" || req.Width != 1000 || req.Height != 800 || req.Scale != 1 || req.LinksOnly || req.Seq == 0 {
142+ t.Fatalf("request %+v", req)
143+ }
144+ want := []ports.HintWindow{{ID: 1, PID: 4242, AppID: "org.test.app", Rect: h.win, Focused: true}}
145+ if !slices.Equal(req.Windows, want) {
146+ t.Fatalf("windows %+v, want %+v", req.Windows, want)
147+ }
148+ // A second bind while pending does not ask again; the pending phase
149+ // cancels on any key.
150+ h.press("g", ports.ModSuper)
151+ select {
152+ case r := <-h.scans:
153+ t.Fatalf("second request %+v", r)
154+ case <-time.After(50 * time.Millisecond):
155+ }
156+}
157+
158+func TestHintRequestNilScansIsNoop(t *testing.T) {
159+ cfg := scrollDefaults()
160+ cfg.Binds["Cmd+g"] = "hint"
161+ input := make(chan ports.InputEvent, 4)
162+ client := make(chan ports.ClientEvent, 4)
163+ output := make(chan ports.OutputEvent, 4)
164+ commands := make(chan ports.ClientCommand, 32)
165+ scenes := make(chan []ports.Scene, 1)
166+ c, err := core.New(cfg, core.Channels{Client: client, Input: input, Output: output, Commands: commands, Scenes: scenes}, core.Options{})
167+ if err != nil {
168+ t.Fatal(err)
169+ }
170+ ctx, cancel := context.WithCancel(context.Background())
171+ done := make(chan error, 1)
172+ go func() { done <- c.Run(ctx) }()
173+ defer func() { cancel(); receive(t, done) }()
174+ output <- ports.OutputAdded{Info: ports.OutputInfo{Name: "OUT-1", Width: 1000, Height: 800}}
175+ scene(t, scenes)
176+ input <- ports.KeyEvent{Keysym: "g", Base: "g", Mods: ports.ModSuper, Pressed: true}
177+ if s := scene(t, scenes); len(s.HintLabels) != 0 {
178+ t.Fatal(s.HintLabels)
179+ }
180+}
181+
182+func TestHintRefusedWhenProtected(t *testing.T) {
183+ var state atomic.Pointer[ports.SecurityState]
184+ state.Store(&ports.SecurityState{})
185+ gate := portsmocks.NewMockSessionSecurity(t)
186+ gate.EXPECT().Snapshot().RunAndReturn(func() ports.SecurityState { return *state.Load() }).Maybe()
187+ h := newHint(t, core.Options{Security: gate})
188+ // Hint mode shown, then the session locks: the labels go away.
189+ h.show(h.inside(0))
190+ state.Store(&ports.SecurityState{Generation: 2, Protected: true})
191+ h.state = *state.Load()
192+ h.client <- ports.WindowMapped{ID: 2}
193+ s := h.until(func(s ports.Scene) bool { return s.Security.Protected })
194+ if len(s.HintLabels) != 0 {
195+ t.Fatalf("labels while protected %+v", s.HintLabels)
196+ }
197+ // And a new request is refused while it lasts.
198+ h.press("g", ports.ModSuper)
199+ select {
200+ case r := <-h.scans:
201+ t.Fatalf("request while protected %+v", r)
202+ case <-time.After(100 * time.Millisecond):
203+ }
204+}
205+
206+func TestHintResultShowsLabels(t *testing.T) {
207+ h := newHint(t, core.Options{})
208+ var targets []ports.HintTarget
209+ for i := 0; i < 26; i++ {
210+ targets = append(targets, h.inside(i))
211+ }
212+ s := h.show(targets...)
213+ if len(s.HintLabels) != 26 {
214+ t.Fatalf("%d labels", len(s.HintLabels))
215+ }
216+ seen := map[string]bool{}
217+ for _, l := range s.HintLabels {
218+ if len(l.Text) != 1 || l.Typed != 0 || seen[l.Text] {
219+ t.Fatalf("label %+v", l)
220+ }
221+ seen[l.Text] = true
222+ }
223+ // Sorted by position: the first target gets the first letter.
224+ if s.HintLabels[0].Text != "f" || s.HintLabels[0].Rect.X != targets[0].Rect.X || s.HintLabels[0].Rect.Y != targets[0].Rect.Y {
225+ t.Fatalf("first %+v", s.HintLabels[0])
226+ }
227+}
228+
229+func TestHintTwoLetterLabels(t *testing.T) {
230+ h := newHint(t, core.Options{})
231+ var targets []ports.HintTarget
232+ for i := 0; i < 27; i++ {
233+ targets = append(targets, ports.HintTarget{Rect: ports.Rect{X: h.win.X + 5, Y: h.win.Y + 5 + 14*i, W: 30, H: 10}})
234+ }
235+ s := h.show(targets...)
236+ if len(s.HintLabels) != 27 {
237+ t.Fatalf("%d labels", len(s.HintLabels))
238+ }
239+ seen := map[string]bool{}
240+ for _, l := range s.HintLabels {
241+ if len(l.Text) != 2 || seen[l.Text] {
242+ t.Fatalf("label %+v", l)
243+ }
244+ seen[l.Text] = true
245+ }
246+ if s.HintLabels[0].Text != "ff" || s.HintLabels[1].Text != "fj" {
247+ t.Fatalf("labels %v %v", s.HintLabels[0].Text, s.HintLabels[1].Text)
248+ }
249+}
250+
251+func TestHintDropsTargetsOutsideWindows(t *testing.T) {
252+ h := newHint(t, core.Options{})
253+ in := h.inside(0)
254+ // In the gap around the window, over the background.
255+ out := ports.HintTarget{Rect: ports.Rect{X: 2, Y: 2, W: 4, H: 4}}
256+ if h.win.X < 8 || h.win.Y < 8 {
257+ t.Fatalf("window %+v leaves no gap", h.win)
258+ }
259+ req := h.start()
260+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{out, in}}
261+ s := h.labelled(1, 0)
262+ if len(s.HintLabels) != 1 || s.HintLabels[0].Rect.X != in.Rect.X {
263+ t.Fatalf("labels %+v (window %+v)", s.HintLabels, h.win)
264+ }
265+}
266+
267+func TestHintTypingFilters(t *testing.T) {
268+ h := newHint(t, core.Options{})
269+ var targets []ports.HintTarget
270+ for i := 0; i < 27; i++ {
271+ targets = append(targets, ports.HintTarget{Rect: ports.Rect{X: h.win.X + 5, Y: h.win.Y + 5 + 14*i, W: 30, H: 10}})
272+ }
273+ h.show(targets...)
274+ h.press("f", 0)
275+ s := h.labelled(26, 1)
276+ if len(s.HintLabels) != 26 {
277+ t.Fatalf("%d labels after f", len(s.HintLabels))
278+ }
279+ for _, l := range s.HintLabels {
280+ if l.Typed != 1 || l.Text[0] != 'f' {
281+ t.Fatalf("label %+v", l)
282+ }
283+ }
284+ h.press("BackSpace", 0)
285+ h.labelled(27, 0)
286+ // Only the 27th target starts with j: its label is "jf".
287+ h.press("j", 0)
288+ s = h.labelled(1, 1)
289+ if len(s.HintLabels) != 1 || s.HintLabels[0].Text != "jf" {
290+ t.Fatalf("after j %+v", s.HintLabels)
291+ }
292+}
293+
294+func TestHintClick(t *testing.T) {
295+ h := newHint(t, core.Options{})
296+ tg := h.inside(0)
297+ h.show(tg)
298+ h.drain()
299+ h.press("f", 0)
300+ s := h.labelled(0, 0)
301+ if len(s.HintLabels) != 0 {
302+ t.Fatalf("labels after click %+v", s.HintLabels)
303+ }
304+ cx := float64(tg.Rect.X) + float64(tg.Rect.W)/2 - float64(h.win.X)
305+ cy := float64(tg.Rect.Y) + float64(tg.Rect.H)/2 - float64(h.win.Y)
306+ var gotMotion, gotPress, gotRelease, gotFocus bool
307+ for len(h.commands) > 0 || !gotRelease {
308+ var cmd ports.ClientCommand
309+ select {
310+ case cmd = <-h.commands:
311+ case <-time.After(time.Second):
312+ t.Fatalf("missing commands: focus %v motion %v press %v release %v", gotFocus, gotMotion, gotPress, gotRelease)
313+ }
314+ switch v := cmd.(type) {
315+ case ports.PointerFocus:
316+ if v.ID == 1 && v.X == cx && v.Y == cy {
317+ gotFocus = true
318+ }
319+ case ports.PointerMotionTo:
320+ if v.ID == 1 && v.X == cx && v.Y == cy {
321+ gotMotion = true
322+ }
323+ case ports.PointerButtonTo:
324+ if v.ID != 1 || v.Button != 0x110 {
325+ t.Fatalf("button %+v", v)
326+ }
327+ if v.Pressed {
328+ gotPress = true
329+ } else {
330+ gotRelease = gotPress
331+ }
332+ }
333+ }
334+ if !gotFocus || !gotMotion {
335+ t.Fatalf("focus %v motion %v", gotFocus, gotMotion)
336+ }
337+}
338+
339+func TestHintEscapeCancels(t *testing.T) {
340+ h := newHint(t, core.Options{})
341+ h.show(h.inside(0))
342+ h.drain()
343+ h.press("Escape", 0)
344+ s := h.labelled(0, 0)
345+ if len(s.HintLabels) != 0 {
346+ t.Fatalf("labels %+v", s.HintLabels)
347+ }
348+ h.release("Escape", 0)
349+ // A later key goes to the window again: the Escape release did not.
350+ h.press("x", 0)
351+ cmd := command(t, h.commands)
352+ if v, ok := cmd.(ports.ForwardKey); !ok || v.Key.Keysym != "x" {
353+ t.Fatalf("command %#v", cmd)
354+ }
355+ for len(h.commands) > 0 {
356+ if _, ok := (<-h.commands).(ports.PointerButtonTo); ok {
357+ t.Fatal("button sent")
358+ }
359+ }
360+}
361+
362+func TestHintKeysNotForwarded(t *testing.T) {
363+ h := newHint(t, core.Options{})
364+ h.show(h.inside(0), h.inside(1))
365+ h.drain()
366+ // A letter that matches nothing is swallowed and does not cancel.
367+ h.press("q", 0)
368+ h.release("q", 0)
369+ // Any other key cancels and is swallowed.
370+ h.press("Return", 0)
371+ s := h.labelled(0, 0)
372+ if len(s.HintLabels) != 0 {
373+ t.Fatalf("labels %+v", s.HintLabels)
374+ }
375+ h.release("Return", 0)
376+ h.press("x", 0)
377+ cmd := command(t, h.commands)
378+ if v, ok := cmd.(ports.ForwardKey); !ok || v.Key.Keysym != "x" {
379+ t.Fatalf("first forwarded command %#v", cmd)
380+ }
381+}
382+
383+func TestHintStaleSeqIgnored(t *testing.T) {
384+ h := newHint(t, core.Options{})
385+ req := h.start()
386+ h.hints <- ports.HintScanResult{Seq: req.Seq + 7, Output: req.Output, Targets: []ports.HintTarget{h.inside(0)}}
387+ // A matching result afterwards shows: the stale one changed nothing.
388+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{h.inside(0), h.inside(1)}}
389+ s := h.labelled(2, 0)
390+ if len(s.HintLabels) != 2 {
391+ t.Fatalf("labels %+v", s.HintLabels)
392+ }
393+}
394+
395+func TestHintErrorCancels(t *testing.T) {
396+ h := newHint(t, core.Options{})
397+ req := h.start()
398+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Err: fmt.Errorf("boom")}
399+ h.press("x", 0)
400+ cmd := command(t, h.commands)
401+ if v, ok := cmd.(ports.ForwardKey); !ok || v.Key.Keysym != "x" {
402+ t.Fatalf("command %#v", cmd)
403+ }
404+}
405+
406+func TestHintWindowMovedAborts(t *testing.T) {
407+ h := newHint(t, core.Options{})
408+ tg := h.inside(0)
409+ h.show(tg)
410+ // The window closes before the label is typed: the click must not go
411+ // anywhere.
412+ h.client <- ports.WindowUnmapped{ID: 1}
413+ h.until(func(s ports.Scene) bool { return len(s.Windows) == 0 })
414+ h.drain()
415+ h.press("f", 0)
416+ // With no window left a key goes nowhere; the next bind press, answered
417+ // by a scan request, shows core handled the label key before it.
418+ h.press("g", ports.ModSuper)
419+ receive(t, h.scans)
420+ for len(h.commands) > 0 {
421+ if _, ok := (<-h.commands).(ports.PointerButtonTo); ok {
422+ t.Fatal("button sent to a moved window")
423+ }
424+ }
425+}
426+
427+// barrier returns the commands core sent for everything given to it so far:
428+// a marker key goes through the same input channel and owner, and is
429+// forwarded to the focused window after the commands of the earlier events.
430+func (h *hintHarness) barrier() []ports.ClientCommand {
431+ h.t.Helper()
432+ h.press("marker", 0)
433+ var out []ports.ClientCommand
434+ for {
435+ cmd := command(h.t, h.commands)
436+ if k, ok := cmd.(ports.ForwardKey); ok && k.Key.Keysym == "marker" {
437+ return out
438+ }
439+ out = append(out, cmd)
440+ }
441+}
442+
443+// buttonsSent drains the commands and counts pointer button commands.
444+func (h *hintHarness) buttonsSent() (presses, releases int) {
445+ for _, cmd := range h.barrier() {
446+ if b, ok := cmd.(ports.PointerButtonTo); ok {
447+ if b.Pressed {
448+ presses++
449+ } else {
450+ releases++
451+ }
452+ }
453+ }
454+ return presses, releases
455+}
456+
457+// The bind's modifier is still held while the label is typed: the synthetic
458+// click must reach the client, not start a Cmd+left-button window drag.
459+func TestHintClickWithBindModifierHeld(t *testing.T) {
460+ h := newHint(t, core.Options{})
461+ tg := h.inside(0)
462+ h.press("g", ports.ModSuper)
463+ req := receive(t, h.scans)
464+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{tg}}
465+ h.labelled(1, 0)
466+ h.drain()
467+ h.press("f", ports.ModSuper) // Super is still down
468+ h.labelled(0, 0)
469+ presses, releases := h.buttonsSent()
470+ if presses != 1 || releases != 1 {
471+ t.Fatalf("client got %d presses, %d releases", presses, releases)
472+ }
473+ // No drag is left behind: a later plain motion and a click work as usual.
474+ h.release("f", ports.ModSuper)
475+ h.release("Super_L", 0)
476+ h.send(ports.PointerMotion{X: float64(tg.Rect.X), Y: float64(tg.Rect.Y)})
477+ h.send(ports.PointerButton{Button: 0x110, Pressed: true})
478+ h.send(ports.PointerButton{Button: 0x110})
479+ if presses, releases = h.buttonsSent(); presses != 1 || releases != 1 {
480+ t.Fatalf("after: %d presses, %d releases (drag left active?)", presses, releases)
481+ }
482+}
483+
484+// constrain gives window 1 a pointer constraint and waits until core applied it.
485+func (h *hintHarness) constrain(mode ports.ConstraintMode, r ports.Rect) {
486+ h.client <- ports.PointerConstrained{ID: 1, PointerConstraint: ports.PointerConstraint{Mode: mode, Rect: r}}
487+ h.client <- ports.WindowMapped{ID: 1} // a publish after the event
488+ h.until(func(ports.Scene) bool { return true })
489+}
490+
491+func TestHintRefusedWithPointerConstraint(t *testing.T) {
492+ for name, c := range map[string]ports.PointerConstraint{
493+ "lock": {Mode: ports.ConstraintLock},
494+ "confine": {Mode: ports.ConstraintConfine, Rect: ports.Rect{X: 0, Y: 0, W: 5, H: 5}},
495+ } {
496+ t.Run(name, func(t *testing.T) {
497+ h := newHint(t, core.Options{})
498+ h.constrain(c.Mode, c.Rect)
499+ h.press("g", ports.ModSuper)
500+ select {
501+ case r := <-h.scans:
502+ t.Fatalf("request under a %s: %+v", name, r)
503+ case <-time.After(100 * time.Millisecond):
504+ }
505+ })
506+ }
507+}
508+
509+// A constraint that appears while the labels are shown (the pointer would be
510+// held or bent away from the target) cancels the click.
511+func TestHintNoClickWhenConstraintAppears(t *testing.T) {
512+ for name, c := range map[string]ports.PointerConstraint{
513+ "lock": {Mode: ports.ConstraintLock},
514+ "confine": {Mode: ports.ConstraintConfine, Rect: ports.Rect{X: 0, Y: 0, W: 5, H: 5}},
515+ } {
516+ t.Run(name, func(t *testing.T) {
517+ h := newHint(t, core.Options{})
518+ h.show(h.inside(0))
519+ h.constrain(c.Mode, c.Rect)
520+ h.drain()
521+ h.press("f", 0)
522+ if p, r := h.buttonsSent(); p+r != 0 {
523+ t.Fatalf("%d presses, %d releases under a %s", p, r, name)
524+ }
525+ })
526+ }
527+}
528+
529+// pressReal sends a real left press over window 1 and waits for the client to
530+// see it: the window then holds the pointer grab.
531+func (h *hintHarness) pressReal() {
532+ h.t.Helper()
533+ h.send(ports.PointerMotion{X: float64(h.win.X + 1), Y: float64(h.win.Y + 1)})
534+ h.send(ports.PointerButton{Button: 0x110, Pressed: true})
535+ for {
536+ cmd := command(h.t, h.commands)
537+ if b, ok := cmd.(ports.PointerButtonTo); ok && b.Pressed {
538+ return
539+ }
540+ }
541+}
542+
543+func TestHintRefusedWhileButtonHeld(t *testing.T) {
544+ h := newHint(t, core.Options{})
545+ h.pressReal()
546+ h.press("g", ports.ModSuper)
547+ select {
548+ case r := <-h.scans:
549+ t.Fatalf("request with a button held: %+v", r)
550+ case <-time.After(100 * time.Millisecond):
551+ }
552+}
553+
554+// A button pressed (and so a grab taken) after the labels appeared: the label
555+// is typed, no synthetic button goes out and the real press stays held.
556+func TestHintNoClickWhileGrabbed(t *testing.T) {
557+ h := newHint(t, core.Options{})
558+ h.show(h.inside(0))
559+ h.pressReal()
560+ h.press("f", 0)
561+ if p, r := h.buttonsSent(); p+r != 0 {
562+ t.Fatalf("synthetic click during a grab: %d presses, %d releases", p, r)
563+ }
564+ // The real release still reaches the client: c.buttons was not touched.
565+ h.send(ports.PointerButton{Button: 0x110})
566+ if _, r := h.buttonsSent(); r != 1 {
567+ t.Fatalf("real release delivered %d times", r)
568+ }
569+}
570+
571+// The window stays mapped but moves away from the target's centre.
572+func TestHintMappedWindowMovedAborts(t *testing.T) {
573+ h := newHint(t, core.Options{})
574+ // A target at the right edge of window 1.
575+ tg := ports.HintTarget{Rect: ports.Rect{X: h.win.X + h.win.W - 50, Y: h.win.Y + 10, W: 40, H: 10}}
576+ h.show(tg)
577+ h.client <- ports.WindowMapped{ID: 2}
578+ s := h.until(func(s ports.Scene) bool { return len(s.Windows) == 2 && len(s.HintLabels) <= 1 })
579+ if s.Windows[0].Rect == h.win {
580+ t.Fatalf("window 1 did not move: %+v", s.Windows)
581+ }
582+ h.drain()
583+ h.press("f", 0)
584+ if p, r := h.buttonsSent(); p+r != 0 {
585+ t.Fatalf("click went to %d presses %d releases after the window moved", p, r)
586+ }
587+}
588+
589+// commandsSent drains the commands up to a barrier, keeping the ones that
590+// matter for a click: modifiers, focus, motion and buttons.
591+func (h *hintHarness) commandsSent() []ports.ClientCommand {
592+ var out []ports.ClientCommand
593+ for _, cmd := range h.barrier() {
594+ switch cmd.(type) {
595+ case ports.SetModifiers, ports.FocusWindow, ports.PointerMotionTo, ports.PointerButtonTo:
596+ out = append(out, cmd)
597+ }
598+ }
599+ return out
600+}
601+
602+func commandKinds(cmds []ports.ClientCommand) []string {
603+ var out []string
604+ for _, c := range cmds {
605+ switch v := c.(type) {
606+ case ports.SetModifiers:
607+ if v.Restore {
608+ out = append(out, "restore")
609+ } else {
610+ out = append(out, fmt.Sprintf("mods:%d", v.Mods))
611+ }
612+ case ports.FocusWindow:
613+ out = append(out, fmt.Sprintf("focus:%d", v.ID))
614+ case ports.PointerMotionTo:
615+ out = append(out, "motion")
616+ case ports.PointerButtonTo:
617+ if v.Pressed {
618+ out = append(out, "press")
619+ } else {
620+ out = append(out, "release")
621+ }
622+ }
623+ }
624+ return out
625+}
626+
627+// An OCR target is clicked with the configured modifiers held: they go out
628+// before the motion and are restored after the release.
629+func TestHintTextClickHoldsModifiers(t *testing.T) {
630+ mods := ports.ModCtrl | ports.ModShift
631+ h := newHintMods(t, core.Options{}, mods)
632+ tg := h.inside(0)
633+ tg.Text = true
634+ h.show(tg)
635+ h.drain()
636+ h.press("f", 0)
637+ got := commandKinds(h.commandsSent())
638+ want := []string{fmt.Sprintf("mods:%d", mods), "motion", "press", "release", "restore"}
639+ if !slices.Equal(got, want) {
640+ t.Fatalf("commands %v, want %v", got, want)
641+ }
642+}
643+
644+// Without text targets or without modifiers the click states no modifiers
645+// (depressed 0) rather than leave the held bind modifier to the client.
646+func TestHintClickWithoutModifiers(t *testing.T) {
647+ for name, c := range map[string]struct {
648+ mods ports.Mods
649+ text bool
650+ }{"accessibility target": {ports.ModCtrl, false}, "none configured": {0, true}} {
651+ t.Run(name, func(t *testing.T) {
652+ h := newHintMods(t, core.Options{}, c.mods)
653+ tg := h.inside(0)
654+ tg.Text = c.text
655+ h.show(tg)
656+ h.drain()
657+ h.press("f", 0)
658+ got := commandKinds(h.commandsSent())
659+ if want := []string{"mods:0", "motion", "press", "release", "restore"}; !slices.Equal(got, want) {
660+ t.Fatalf("commands %v, want %v", got, want)
661+ }
662+ })
663+ }
664+}
665+
666+// The bind's Super is still physically held: an accessibility click states
667+// depressed 0, and the restore carries the real state of the latest key,
668+// swallowed ones included (a latch a label key consumed).
669+func TestHintClickHeldBindModifierAndLatchRestore(t *testing.T) {
670+ h := newHint(t, core.Options{})
671+ tg := h.inside(0)
672+ h.press("g", ports.ModSuper)
673+ req := receive(t, h.scans)
674+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{tg, h.inside(1)}}
675+ h.labelled(2, 0)
676+ h.drain()
677+ real := ports.ModState{Depressed: 64, Latched: 0, Locked: 2}
678+ h.send(ports.KeyEvent{Keysym: "f", Base: "f", Mods: ports.ModSuper, Pressed: true, State: real})
679+ cmds := h.commandsSent()
680+ var restore *ports.SetModifiers
681+ for i, c := range cmds {
682+ if v, ok := c.(ports.SetModifiers); ok {
683+ if i == 0 && (v.Restore || v.Mods != 0) {
684+ t.Fatalf("first modifiers %+v, want depressed 0", v)
685+ }
686+ if v.Restore {
687+ restore = &v
688+ }
689+ }
690+ }
691+ if restore == nil || restore.State != real {
692+ t.Fatalf("restore %+v, want state %+v (commands %v)", restore, real, commandKinds(cmds))
693+ }
694+}
695+
696+// A result is kept only for the window it was scanned on.
697+func TestHintResultWindowChecks(t *testing.T) {
698+ h := newHint(t, core.Options{})
699+ req := h.start()
700+ // Accessibility target naming another window: dropped; naming window 1: kept.
701+ ok := h.inside(1)
702+ ok.Window = 1
703+ bad := h.inside(0)
704+ bad.Window = 7
705+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{bad, ok}}
706+ s := h.labelled(1, 0)
707+ if len(s.HintLabels) != 1 || s.HintLabels[0].Rect.Y != ok.Rect.Y {
708+ t.Fatalf("labels %+v", s.HintLabels)
709+ }
710+}
711+
712+// A window that moved while the scan ran: an OCR target over it is dropped.
713+func TestHintOCRResultDroppedWhenWindowMoved(t *testing.T) {
714+ h := newHint(t, core.Options{})
715+ req := h.start()
716+ tg := h.inside(0)
717+ tg.Text = true
718+ h.client <- ports.WindowMapped{ID: 2}
719+ h.until(func(s ports.Scene) bool { return len(s.Windows) == 2 })
720+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{tg}}
721+ // Whatever window now covers the target was not scanned: no labels.
722+ if s := h.labelled(0, 0); len(s.HintLabels) != 0 {
723+ t.Fatalf("labels %+v", s.HintLabels)
724+ }
725+}
726+
727+// A new window covering the target (not in the snapshot) drops it as well.
728+func TestHintOCRResultDroppedForUnseenWindow(t *testing.T) {
729+ h := newHint(t, core.Options{})
730+ req := h.start()
731+ h.client <- ports.WindowMapped{ID: 2}
732+ s := h.until(func(s ports.Scene) bool { return len(s.Windows) == 2 })
733+ tg := ports.HintTarget{Text: true}
734+ for _, w := range s.Windows {
735+ if w.ID == 2 {
736+ tg.Rect = ports.Rect{X: w.Rect.X + 5, Y: w.Rect.Y + 5, W: 20, H: 10}
737+ }
738+ }
739+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{tg}}
740+ if s := h.labelled(0, 0); len(s.HintLabels) != 0 {
741+ t.Fatalf("labels %+v", s.HintLabels)
742+ }
743+}
744+
745+// Targets outside the output are dropped.
746+func TestHintDropsTargetsOutsideOutput(t *testing.T) {
747+ h := newHint(t, core.Options{})
748+ req := h.start()
749+ out := ports.HintTarget{Rect: ports.Rect{X: 2000, Y: 10, W: 10, H: 10}}
750+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{out, h.inside(0)}}
751+ if s := h.labelled(1, 0); len(s.HintLabels) != 1 {
752+ t.Fatalf("labels %+v", s.HintLabels)
753+ }
754+}
755+
756+// The output changing size while labels are shown ends hint mode.
757+func TestHintCancelsWhenOutputChanges(t *testing.T) {
758+ h := newHint(t, core.Options{})
759+ h.show(h.inside(0))
760+ h.output <- ports.OutputAdded{Info: ports.OutputInfo{Name: "OUT-1", Width: 1200, Height: 800}}
761+ if s := h.labelled(0, 0); len(s.HintLabels) != 0 {
762+ t.Fatalf("labels %+v", s.HintLabels)
763+ }
764+}
765+
766+// A window that does not have the keyboard gets it before the modifiers.
767+func TestHintTextClickFocusesWindowFirst(t *testing.T) {
768+ h := newHintMods(t, core.Options{}, ports.ModCtrl)
769+ h.client <- ports.WindowMapped{ID: 2}
770+ s := h.until(func(s ports.Scene) bool { return len(s.Windows) == 2 })
771+ var r ports.Rect
772+ for _, w := range s.Windows {
773+ if w.ID == 1 {
774+ r = w.Rect
775+ }
776+ }
777+ h.win = r
778+ tg := h.inside(0)
779+ tg.Text = true
780+ h.show(tg)
781+ h.drain()
782+ h.press("f", 0)
783+ got := commandKinds(h.commandsSent())
784+ want := []string{"focus:1", fmt.Sprintf("mods:%d", ports.ModCtrl), "motion", "press", "release", "restore"}
785+ if !slices.Equal(got, want) {
786+ t.Fatalf("commands %v, want %v", got, want)
787+ }
788+}
789+
790+// A click that is no longer allowed sends no modifiers at all.
791+func TestHintTextClickNoModifiersWhenNotClickable(t *testing.T) {
792+ h := newHintMods(t, core.Options{}, ports.ModCtrl)
793+ tg := h.inside(0)
794+ tg.Text = true
795+ h.show(tg)
796+ h.constrain(ports.ConstraintLock, ports.Rect{})
797+ h.drain()
798+ h.press("f", 0)
799+ if got := commandKinds(h.commandsSent()); len(got) != 0 {
800+ t.Fatalf("commands %v under a constraint", got)
801+ }
802+}
803+
804+func TestHintDisabledNotesOnce(t *testing.T) {
805+ cfg := scrollDefaults()
806+ cfg.Binds["Cmd+g"] = "hint"
807+ var noted atomic.Int32
808+ h := &hintHarness{
809+ input: make(chan ports.InputEvent, 32),
810+ client: make(chan ports.ClientEvent, 32),
811+ output: make(chan ports.OutputEvent, 8),
812+ commands: make(chan ports.ClientCommand, 256),
813+ scenes: make(chan []ports.Scene, 1),
814+ }
815+ ch := core.Channels{Client: h.client, Input: h.input, Output: h.output, Commands: h.commands, Scenes: h.scenes}
816+ c, err := core.New(cfg, ch, core.Options{HintDisabled: func() { noted.Add(1) }})
817+ if err != nil {
818+ t.Fatal(err)
819+ }
820+ ctx, cancel := context.WithCancel(context.Background())
821+ done := make(chan error, 1)
822+ go func() { done <- c.Run(ctx) }()
823+ t.Cleanup(func() { cancel(); receive(t, done) })
824+ h.output <- ports.OutputAdded{Info: ports.OutputInfo{Name: "OUT-1", Width: 1000, Height: 800}}
825+ scene(t, h.scenes)
826+ for range 3 {
827+ h.press("g", ports.ModSuper)
828+ h.release("g", 0)
829+ }
830+ c.Ping(ctx)
831+ if got := noted.Load(); got != 1 {
832+ t.Fatalf("disabled note ran %d times, want 1", got)
833+ }
834+}
@@ -0,0 +1,834 @@
1+package core_test
2+
3+import (
4+ "context"
5+ "fmt"
6+ "slices"
7+ "sync/atomic"
8+ "testing"
9+ "time"
10+
11+ "github.com/bnema/neferwl/internal/core"
12+ portsmocks "github.com/bnema/neferwl/internal/mocks/ports"
13+ "github.com/bnema/neferwl/internal/ports"
14+)
15+
16+type hintHarness struct {
17+ t *testing.T
18+ input chan ports.InputEvent
19+ client chan ports.ClientEvent
20+ output chan ports.OutputEvent
21+ commands chan ports.ClientCommand
22+ scenes chan []ports.Scene
23+ scans chan ports.HintScanRequest
24+ hints chan ports.HintScanResult
25+ win ports.Rect // window 1, output-local
26+ // state is the security state input is stamped with.
27+ state ports.SecurityState
28+}
29+
30+// newHint starts a core with one 1000x800 output "OUT-1" and window 1 mapped.
31+func newHint(t *testing.T, opts core.Options) *hintHarness {
32+ t.Helper()
33+ return newHintMods(t, opts, 0)
34+}
35+
36+// newHintMods is newHint with hint.text-click set to textClick.
37+func newHintMods(t *testing.T, opts core.Options, textClick ports.Mods) *hintHarness {
38+ t.Helper()
39+ cfg := scrollDefaults()
40+ cfg.Hint.TextClick = textClick
41+ cfg.Border.Width = 0
42+ cfg.Layout.Gaps = 20
43+ cfg.Binds["Cmd+g"] = "hint"
44+ h := &hintHarness{
45+ t: t,
46+ input: make(chan ports.InputEvent, 32),
47+ client: make(chan ports.ClientEvent, 32),
48+ output: make(chan ports.OutputEvent, 8),
49+ commands: make(chan ports.ClientCommand, 256),
50+ scenes: make(chan []ports.Scene, 1),
51+ scans: make(chan ports.HintScanRequest, 1),
52+ // Unbuffered: a send returns once core took the result.
53+ hints: make(chan ports.HintScanResult),
54+ }
55+ ch := core.Channels{Client: h.client, Input: h.input, Output: h.output, Commands: h.commands, Scenes: h.scenes, HintScans: h.scans, Hints: h.hints}
56+ c, err := core.New(cfg, ch, opts)
57+ if err != nil {
58+ t.Fatal(err)
59+ }
60+ ctx, cancel := context.WithCancel(context.Background())
61+ done := make(chan error, 1)
62+ go func() { done <- c.Run(ctx) }()
63+ t.Cleanup(func() { cancel(); receive(t, done) })
64+ h.output <- ports.OutputAdded{Info: ports.OutputInfo{Name: "OUT-1", Width: 1000, Height: 800}}
65+ scene(t, h.scenes)
66+ h.client <- ports.WindowMapped{ID: 1, AppID: "org.test.app", PID: 4242}
67+ s := scene(t, h.scenes)
68+ h.win = s.Windows[0].Rect
69+ h.drain()
70+ return h
71+}
72+
73+func (h *hintHarness) drain() {
74+ for {
75+ select {
76+ case <-h.commands:
77+ default:
78+ return
79+ }
80+ }
81+}
82+
83+// press and release send one key event.
84+func (h *hintHarness) press(sym string, mods ports.Mods) {
85+ h.send(ports.KeyEvent{Keysym: sym, Base: sym, Mods: mods, Pressed: true})
86+}
87+
88+func (h *hintHarness) release(sym string, mods ports.Mods) {
89+ h.send(ports.KeyEvent{Keysym: sym, Base: sym, Mods: mods})
90+}
91+
92+// send gives core an input event stamped with the current security state.
93+func (h *hintHarness) send(ev ports.InputEvent) {
94+ h.input <- ports.SecurityInput{State: h.state, Event: ev}
95+}
96+
97+// start presses the hint bind and returns the request.
98+func (h *hintHarness) start() ports.HintScanRequest {
99+ h.t.Helper()
100+ h.press("g", ports.ModSuper)
101+ req := receive(h.t, h.scans)
102+ h.release("g", 0) // the user lets go of the whole chord
103+ return req
104+}
105+
106+// show starts hint mode and answers with targets, returning the next scene.
107+func (h *hintHarness) show(targets ...ports.HintTarget) ports.Scene {
108+ h.t.Helper()
109+ req := h.start()
110+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: targets}
111+ return h.labelled(len(targets), 0)
112+}
113+
114+// until returns the first scene, from the next one on, that satisfies ok:
115+// the bind press and other input publish scenes before the one waited for.
116+func (h *hintHarness) until(ok func(ports.Scene) bool) ports.Scene {
117+ h.t.Helper()
118+ for {
119+ if s := scene(h.t, h.scenes); ok(s) {
120+ return s
121+ }
122+ }
123+}
124+
125+// labelled waits for a scene with n labels, all typed with typed letters.
126+func (h *hintHarness) labelled(n, typed int) ports.Scene {
127+ h.t.Helper()
128+ return h.until(func(s ports.Scene) bool {
129+ return len(s.HintLabels) == n && (n == 0 || s.HintLabels[0].Typed == typed)
130+ })
131+}
132+
133+// inside is a target of the given size inside window 1.
134+func (h *hintHarness) inside(i int) ports.HintTarget {
135+ return ports.HintTarget{Rect: ports.Rect{X: h.win.X + 10 + i, Y: h.win.Y + 10 + 12*i, W: 40, H: 10}}
136+}
137+
138+func TestHintRequestOnBind(t *testing.T) {
139+ h := newHint(t, core.Options{})
140+ req := h.start()
141+ if req.Output != "OUT-1" || req.Width != 1000 || req.Height != 800 || req.Scale != 1 || req.LinksOnly || req.Seq == 0 {
142+ t.Fatalf("request %+v", req)
143+ }
144+ want := []ports.HintWindow{{ID: 1, PID: 4242, AppID: "org.test.app", Rect: h.win, Focused: true}}
145+ if !slices.Equal(req.Windows, want) {
146+ t.Fatalf("windows %+v, want %+v", req.Windows, want)
147+ }
148+ // A second bind while pending does not ask again; the pending phase
149+ // cancels on any key.
150+ h.press("g", ports.ModSuper)
151+ select {
152+ case r := <-h.scans:
153+ t.Fatalf("second request %+v", r)
154+ case <-time.After(50 * time.Millisecond):
155+ }
156+}
157+
158+func TestHintRequestNilScansIsNoop(t *testing.T) {
159+ cfg := scrollDefaults()
160+ cfg.Binds["Cmd+g"] = "hint"
161+ input := make(chan ports.InputEvent, 4)
162+ client := make(chan ports.ClientEvent, 4)
163+ output := make(chan ports.OutputEvent, 4)
164+ commands := make(chan ports.ClientCommand, 32)
165+ scenes := make(chan []ports.Scene, 1)
166+ c, err := core.New(cfg, core.Channels{Client: client, Input: input, Output: output, Commands: commands, Scenes: scenes}, core.Options{})
167+ if err != nil {
168+ t.Fatal(err)
169+ }
170+ ctx, cancel := context.WithCancel(context.Background())
171+ done := make(chan error, 1)
172+ go func() { done <- c.Run(ctx) }()
173+ defer func() { cancel(); receive(t, done) }()
174+ output <- ports.OutputAdded{Info: ports.OutputInfo{Name: "OUT-1", Width: 1000, Height: 800}}
175+ scene(t, scenes)
176+ input <- ports.KeyEvent{Keysym: "g", Base: "g", Mods: ports.ModSuper, Pressed: true}
177+ if s := scene(t, scenes); len(s.HintLabels) != 0 {
178+ t.Fatal(s.HintLabels)
179+ }
180+}
181+
182+func TestHintRefusedWhenProtected(t *testing.T) {
183+ var state atomic.Pointer[ports.SecurityState]
184+ state.Store(&ports.SecurityState{})
185+ gate := portsmocks.NewMockSessionSecurity(t)
186+ gate.EXPECT().Snapshot().RunAndReturn(func() ports.SecurityState { return *state.Load() }).Maybe()
187+ h := newHint(t, core.Options{Security: gate})
188+ // Hint mode shown, then the session locks: the labels go away.
189+ h.show(h.inside(0))
190+ state.Store(&ports.SecurityState{Generation: 2, Protected: true})
191+ h.state = *state.Load()
192+ h.client <- ports.WindowMapped{ID: 2}
193+ s := h.until(func(s ports.Scene) bool { return s.Security.Protected })
194+ if len(s.HintLabels) != 0 {
195+ t.Fatalf("labels while protected %+v", s.HintLabels)
196+ }
197+ // And a new request is refused while it lasts.
198+ h.press("g", ports.ModSuper)
199+ select {
200+ case r := <-h.scans:
201+ t.Fatalf("request while protected %+v", r)
202+ case <-time.After(100 * time.Millisecond):
203+ }
204+}
205+
206+func TestHintResultShowsLabels(t *testing.T) {
207+ h := newHint(t, core.Options{})
208+ var targets []ports.HintTarget
209+ for i := 0; i < 26; i++ {
210+ targets = append(targets, h.inside(i))
211+ }
212+ s := h.show(targets...)
213+ if len(s.HintLabels) != 26 {
214+ t.Fatalf("%d labels", len(s.HintLabels))
215+ }
216+ seen := map[string]bool{}
217+ for _, l := range s.HintLabels {
218+ if len(l.Text) != 1 || l.Typed != 0 || seen[l.Text] {
219+ t.Fatalf("label %+v", l)
220+ }
221+ seen[l.Text] = true
222+ }
223+ // Sorted by position: the first target gets the first letter.
224+ if s.HintLabels[0].Text != "f" || s.HintLabels[0].Rect.X != targets[0].Rect.X || s.HintLabels[0].Rect.Y != targets[0].Rect.Y {
225+ t.Fatalf("first %+v", s.HintLabels[0])
226+ }
227+}
228+
229+func TestHintTwoLetterLabels(t *testing.T) {
230+ h := newHint(t, core.Options{})
231+ var targets []ports.HintTarget
232+ for i := 0; i < 27; i++ {
233+ targets = append(targets, ports.HintTarget{Rect: ports.Rect{X: h.win.X + 5, Y: h.win.Y + 5 + 14*i, W: 30, H: 10}})
234+ }
235+ s := h.show(targets...)
236+ if len(s.HintLabels) != 27 {
237+ t.Fatalf("%d labels", len(s.HintLabels))
238+ }
239+ seen := map[string]bool{}
240+ for _, l := range s.HintLabels {
241+ if len(l.Text) != 2 || seen[l.Text] {
242+ t.Fatalf("label %+v", l)
243+ }
244+ seen[l.Text] = true
245+ }
246+ if s.HintLabels[0].Text != "ff" || s.HintLabels[1].Text != "fj" {
247+ t.Fatalf("labels %v %v", s.HintLabels[0].Text, s.HintLabels[1].Text)
248+ }
249+}
250+
251+func TestHintDropsTargetsOutsideWindows(t *testing.T) {
252+ h := newHint(t, core.Options{})
253+ in := h.inside(0)
254+ // In the gap around the window, over the background.
255+ out := ports.HintTarget{Rect: ports.Rect{X: 2, Y: 2, W: 4, H: 4}}
256+ if h.win.X < 8 || h.win.Y < 8 {
257+ t.Fatalf("window %+v leaves no gap", h.win)
258+ }
259+ req := h.start()
260+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{out, in}}
261+ s := h.labelled(1, 0)
262+ if len(s.HintLabels) != 1 || s.HintLabels[0].Rect.X != in.Rect.X {
263+ t.Fatalf("labels %+v (window %+v)", s.HintLabels, h.win)
264+ }
265+}
266+
267+func TestHintTypingFilters(t *testing.T) {
268+ h := newHint(t, core.Options{})
269+ var targets []ports.HintTarget
270+ for i := 0; i < 27; i++ {
271+ targets = append(targets, ports.HintTarget{Rect: ports.Rect{X: h.win.X + 5, Y: h.win.Y + 5 + 14*i, W: 30, H: 10}})
272+ }
273+ h.show(targets...)
274+ h.press("f", 0)
275+ s := h.labelled(26, 1)
276+ if len(s.HintLabels) != 26 {
277+ t.Fatalf("%d labels after f", len(s.HintLabels))
278+ }
279+ for _, l := range s.HintLabels {
280+ if l.Typed != 1 || l.Text[0] != 'f' {
281+ t.Fatalf("label %+v", l)
282+ }
283+ }
284+ h.press("BackSpace", 0)
285+ h.labelled(27, 0)
286+ // Only the 27th target starts with j: its label is "jf".
287+ h.press("j", 0)
288+ s = h.labelled(1, 1)
289+ if len(s.HintLabels) != 1 || s.HintLabels[0].Text != "jf" {
290+ t.Fatalf("after j %+v", s.HintLabels)
291+ }
292+}
293+
294+func TestHintClick(t *testing.T) {
295+ h := newHint(t, core.Options{})
296+ tg := h.inside(0)
297+ h.show(tg)
298+ h.drain()
299+ h.press("f", 0)
300+ s := h.labelled(0, 0)
301+ if len(s.HintLabels) != 0 {
302+ t.Fatalf("labels after click %+v", s.HintLabels)
303+ }
304+ cx := float64(tg.Rect.X) + float64(tg.Rect.W)/2 - float64(h.win.X)
305+ cy := float64(tg.Rect.Y) + float64(tg.Rect.H)/2 - float64(h.win.Y)
306+ var gotMotion, gotPress, gotRelease, gotFocus bool
307+ for len(h.commands) > 0 || !gotRelease {
308+ var cmd ports.ClientCommand
309+ select {
310+ case cmd = <-h.commands:
311+ case <-time.After(time.Second):
312+ t.Fatalf("missing commands: focus %v motion %v press %v release %v", gotFocus, gotMotion, gotPress, gotRelease)
313+ }
314+ switch v := cmd.(type) {
315+ case ports.PointerFocus:
316+ if v.ID == 1 && v.X == cx && v.Y == cy {
317+ gotFocus = true
318+ }
319+ case ports.PointerMotionTo:
320+ if v.ID == 1 && v.X == cx && v.Y == cy {
321+ gotMotion = true
322+ }
323+ case ports.PointerButtonTo:
324+ if v.ID != 1 || v.Button != 0x110 {
325+ t.Fatalf("button %+v", v)
326+ }
327+ if v.Pressed {
328+ gotPress = true
329+ } else {
330+ gotRelease = gotPress
331+ }
332+ }
333+ }
334+ if !gotFocus || !gotMotion {
335+ t.Fatalf("focus %v motion %v", gotFocus, gotMotion)
336+ }
337+}
338+
339+func TestHintEscapeCancels(t *testing.T) {
340+ h := newHint(t, core.Options{})
341+ h.show(h.inside(0))
342+ h.drain()
343+ h.press("Escape", 0)
344+ s := h.labelled(0, 0)
345+ if len(s.HintLabels) != 0 {
346+ t.Fatalf("labels %+v", s.HintLabels)
347+ }
348+ h.release("Escape", 0)
349+ // A later key goes to the window again: the Escape release did not.
350+ h.press("x", 0)
351+ cmd := command(t, h.commands)
352+ if v, ok := cmd.(ports.ForwardKey); !ok || v.Key.Keysym != "x" {
353+ t.Fatalf("command %#v", cmd)
354+ }
355+ for len(h.commands) > 0 {
356+ if _, ok := (<-h.commands).(ports.PointerButtonTo); ok {
357+ t.Fatal("button sent")
358+ }
359+ }
360+}
361+
362+func TestHintKeysNotForwarded(t *testing.T) {
363+ h := newHint(t, core.Options{})
364+ h.show(h.inside(0), h.inside(1))
365+ h.drain()
366+ // A letter that matches nothing is swallowed and does not cancel.
367+ h.press("q", 0)
368+ h.release("q", 0)
369+ // Any other key cancels and is swallowed.
370+ h.press("Return", 0)
371+ s := h.labelled(0, 0)
372+ if len(s.HintLabels) != 0 {
373+ t.Fatalf("labels %+v", s.HintLabels)
374+ }
375+ h.release("Return", 0)
376+ h.press("x", 0)
377+ cmd := command(t, h.commands)
378+ if v, ok := cmd.(ports.ForwardKey); !ok || v.Key.Keysym != "x" {
379+ t.Fatalf("first forwarded command %#v", cmd)
380+ }
381+}
382+
383+func TestHintStaleSeqIgnored(t *testing.T) {
384+ h := newHint(t, core.Options{})
385+ req := h.start()
386+ h.hints <- ports.HintScanResult{Seq: req.Seq + 7, Output: req.Output, Targets: []ports.HintTarget{h.inside(0)}}
387+ // A matching result afterwards shows: the stale one changed nothing.
388+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{h.inside(0), h.inside(1)}}
389+ s := h.labelled(2, 0)
390+ if len(s.HintLabels) != 2 {
391+ t.Fatalf("labels %+v", s.HintLabels)
392+ }
393+}
394+
395+func TestHintErrorCancels(t *testing.T) {
396+ h := newHint(t, core.Options{})
397+ req := h.start()
398+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Err: fmt.Errorf("boom")}
399+ h.press("x", 0)
400+ cmd := command(t, h.commands)
401+ if v, ok := cmd.(ports.ForwardKey); !ok || v.Key.Keysym != "x" {
402+ t.Fatalf("command %#v", cmd)
403+ }
404+}
405+
406+func TestHintWindowMovedAborts(t *testing.T) {
407+ h := newHint(t, core.Options{})
408+ tg := h.inside(0)
409+ h.show(tg)
410+ // The window closes before the label is typed: the click must not go
411+ // anywhere.
412+ h.client <- ports.WindowUnmapped{ID: 1}
413+ h.until(func(s ports.Scene) bool { return len(s.Windows) == 0 })
414+ h.drain()
415+ h.press("f", 0)
416+ // With no window left a key goes nowhere; the next bind press, answered
417+ // by a scan request, shows core handled the label key before it.
418+ h.press("g", ports.ModSuper)
419+ receive(t, h.scans)
420+ for len(h.commands) > 0 {
421+ if _, ok := (<-h.commands).(ports.PointerButtonTo); ok {
422+ t.Fatal("button sent to a moved window")
423+ }
424+ }
425+}
426+
427+// barrier returns the commands core sent for everything given to it so far:
428+// a marker key goes through the same input channel and owner, and is
429+// forwarded to the focused window after the commands of the earlier events.
430+func (h *hintHarness) barrier() []ports.ClientCommand {
431+ h.t.Helper()
432+ h.press("marker", 0)
433+ var out []ports.ClientCommand
434+ for {
435+ cmd := command(h.t, h.commands)
436+ if k, ok := cmd.(ports.ForwardKey); ok && k.Key.Keysym == "marker" {
437+ return out
438+ }
439+ out = append(out, cmd)
440+ }
441+}
442+
443+// buttonsSent drains the commands and counts pointer button commands.
444+func (h *hintHarness) buttonsSent() (presses, releases int) {
445+ for _, cmd := range h.barrier() {
446+ if b, ok := cmd.(ports.PointerButtonTo); ok {
447+ if b.Pressed {
448+ presses++
449+ } else {
450+ releases++
451+ }
452+ }
453+ }
454+ return presses, releases
455+}
456+
457+// The bind's modifier is still held while the label is typed: the synthetic
458+// click must reach the client, not start a Cmd+left-button window drag.
459+func TestHintClickWithBindModifierHeld(t *testing.T) {
460+ h := newHint(t, core.Options{})
461+ tg := h.inside(0)
462+ h.press("g", ports.ModSuper)
463+ req := receive(t, h.scans)
464+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{tg}}
465+ h.labelled(1, 0)
466+ h.drain()
467+ h.press("f", ports.ModSuper) // Super is still down
468+ h.labelled(0, 0)
469+ presses, releases := h.buttonsSent()
470+ if presses != 1 || releases != 1 {
471+ t.Fatalf("client got %d presses, %d releases", presses, releases)
472+ }
473+ // No drag is left behind: a later plain motion and a click work as usual.
474+ h.release("f", ports.ModSuper)
475+ h.release("Super_L", 0)
476+ h.send(ports.PointerMotion{X: float64(tg.Rect.X), Y: float64(tg.Rect.Y)})
477+ h.send(ports.PointerButton{Button: 0x110, Pressed: true})
478+ h.send(ports.PointerButton{Button: 0x110})
479+ if presses, releases = h.buttonsSent(); presses != 1 || releases != 1 {
480+ t.Fatalf("after: %d presses, %d releases (drag left active?)", presses, releases)
481+ }
482+}
483+
484+// constrain gives window 1 a pointer constraint and waits until core applied it.
485+func (h *hintHarness) constrain(mode ports.ConstraintMode, r ports.Rect) {
486+ h.client <- ports.PointerConstrained{ID: 1, PointerConstraint: ports.PointerConstraint{Mode: mode, Rect: r}}
487+ h.client <- ports.WindowMapped{ID: 1} // a publish after the event
488+ h.until(func(ports.Scene) bool { return true })
489+}
490+
491+func TestHintRefusedWithPointerConstraint(t *testing.T) {
492+ for name, c := range map[string]ports.PointerConstraint{
493+ "lock": {Mode: ports.ConstraintLock},
494+ "confine": {Mode: ports.ConstraintConfine, Rect: ports.Rect{X: 0, Y: 0, W: 5, H: 5}},
495+ } {
496+ t.Run(name, func(t *testing.T) {
497+ h := newHint(t, core.Options{})
498+ h.constrain(c.Mode, c.Rect)
499+ h.press("g", ports.ModSuper)
500+ select {
501+ case r := <-h.scans:
502+ t.Fatalf("request under a %s: %+v", name, r)
503+ case <-time.After(100 * time.Millisecond):
504+ }
505+ })
506+ }
507+}
508+
509+// A constraint that appears while the labels are shown (the pointer would be
510+// held or bent away from the target) cancels the click.
511+func TestHintNoClickWhenConstraintAppears(t *testing.T) {
512+ for name, c := range map[string]ports.PointerConstraint{
513+ "lock": {Mode: ports.ConstraintLock},
514+ "confine": {Mode: ports.ConstraintConfine, Rect: ports.Rect{X: 0, Y: 0, W: 5, H: 5}},
515+ } {
516+ t.Run(name, func(t *testing.T) {
517+ h := newHint(t, core.Options{})
518+ h.show(h.inside(0))
519+ h.constrain(c.Mode, c.Rect)
520+ h.drain()
521+ h.press("f", 0)
522+ if p, r := h.buttonsSent(); p+r != 0 {
523+ t.Fatalf("%d presses, %d releases under a %s", p, r, name)
524+ }
525+ })
526+ }
527+}
528+
529+// pressReal sends a real left press over window 1 and waits for the client to
530+// see it: the window then holds the pointer grab.
531+func (h *hintHarness) pressReal() {
532+ h.t.Helper()
533+ h.send(ports.PointerMotion{X: float64(h.win.X + 1), Y: float64(h.win.Y + 1)})
534+ h.send(ports.PointerButton{Button: 0x110, Pressed: true})
535+ for {
536+ cmd := command(h.t, h.commands)
537+ if b, ok := cmd.(ports.PointerButtonTo); ok && b.Pressed {
538+ return
539+ }
540+ }
541+}
542+
543+func TestHintRefusedWhileButtonHeld(t *testing.T) {
544+ h := newHint(t, core.Options{})
545+ h.pressReal()
546+ h.press("g", ports.ModSuper)
547+ select {
548+ case r := <-h.scans:
549+ t.Fatalf("request with a button held: %+v", r)
550+ case <-time.After(100 * time.Millisecond):
551+ }
552+}
553+
554+// A button pressed (and so a grab taken) after the labels appeared: the label
555+// is typed, no synthetic button goes out and the real press stays held.
556+func TestHintNoClickWhileGrabbed(t *testing.T) {
557+ h := newHint(t, core.Options{})
558+ h.show(h.inside(0))
559+ h.pressReal()
560+ h.press("f", 0)
561+ if p, r := h.buttonsSent(); p+r != 0 {
562+ t.Fatalf("synthetic click during a grab: %d presses, %d releases", p, r)
563+ }
564+ // The real release still reaches the client: c.buttons was not touched.
565+ h.send(ports.PointerButton{Button: 0x110})
566+ if _, r := h.buttonsSent(); r != 1 {
567+ t.Fatalf("real release delivered %d times", r)
568+ }
569+}
570+
571+// The window stays mapped but moves away from the target's centre.
572+func TestHintMappedWindowMovedAborts(t *testing.T) {
573+ h := newHint(t, core.Options{})
574+ // A target at the right edge of window 1.
575+ tg := ports.HintTarget{Rect: ports.Rect{X: h.win.X + h.win.W - 50, Y: h.win.Y + 10, W: 40, H: 10}}
576+ h.show(tg)
577+ h.client <- ports.WindowMapped{ID: 2}
578+ s := h.until(func(s ports.Scene) bool { return len(s.Windows) == 2 && len(s.HintLabels) <= 1 })
579+ if s.Windows[0].Rect == h.win {
580+ t.Fatalf("window 1 did not move: %+v", s.Windows)
581+ }
582+ h.drain()
583+ h.press("f", 0)
584+ if p, r := h.buttonsSent(); p+r != 0 {
585+ t.Fatalf("click went to %d presses %d releases after the window moved", p, r)
586+ }
587+}
588+
589+// commandsSent drains the commands up to a barrier, keeping the ones that
590+// matter for a click: modifiers, focus, motion and buttons.
591+func (h *hintHarness) commandsSent() []ports.ClientCommand {
592+ var out []ports.ClientCommand
593+ for _, cmd := range h.barrier() {
594+ switch cmd.(type) {
595+ case ports.SetModifiers, ports.FocusWindow, ports.PointerMotionTo, ports.PointerButtonTo:
596+ out = append(out, cmd)
597+ }
598+ }
599+ return out
600+}
601+
602+func commandKinds(cmds []ports.ClientCommand) []string {
603+ var out []string
604+ for _, c := range cmds {
605+ switch v := c.(type) {
606+ case ports.SetModifiers:
607+ if v.Restore {
608+ out = append(out, "restore")
609+ } else {
610+ out = append(out, fmt.Sprintf("mods:%d", v.Mods))
611+ }
612+ case ports.FocusWindow:
613+ out = append(out, fmt.Sprintf("focus:%d", v.ID))
614+ case ports.PointerMotionTo:
615+ out = append(out, "motion")
616+ case ports.PointerButtonTo:
617+ if v.Pressed {
618+ out = append(out, "press")
619+ } else {
620+ out = append(out, "release")
621+ }
622+ }
623+ }
624+ return out
625+}
626+
627+// An OCR target is clicked with the configured modifiers held: they go out
628+// before the motion and are restored after the release.
629+func TestHintTextClickHoldsModifiers(t *testing.T) {
630+ mods := ports.ModCtrl | ports.ModShift
631+ h := newHintMods(t, core.Options{}, mods)
632+ tg := h.inside(0)
633+ tg.Text = true
634+ h.show(tg)
635+ h.drain()
636+ h.press("f", 0)
637+ got := commandKinds(h.commandsSent())
638+ want := []string{fmt.Sprintf("mods:%d", mods), "motion", "press", "release", "restore"}
639+ if !slices.Equal(got, want) {
640+ t.Fatalf("commands %v, want %v", got, want)
641+ }
642+}
643+
644+// Without text targets or without modifiers the click states no modifiers
645+// (depressed 0) rather than leave the held bind modifier to the client.
646+func TestHintClickWithoutModifiers(t *testing.T) {
647+ for name, c := range map[string]struct {
648+ mods ports.Mods
649+ text bool
650+ }{"accessibility target": {ports.ModCtrl, false}, "none configured": {0, true}} {
651+ t.Run(name, func(t *testing.T) {
652+ h := newHintMods(t, core.Options{}, c.mods)
653+ tg := h.inside(0)
654+ tg.Text = c.text
655+ h.show(tg)
656+ h.drain()
657+ h.press("f", 0)
658+ got := commandKinds(h.commandsSent())
659+ if want := []string{"mods:0", "motion", "press", "release", "restore"}; !slices.Equal(got, want) {
660+ t.Fatalf("commands %v, want %v", got, want)
661+ }
662+ })
663+ }
664+}
665+
666+// The bind's Super is still physically held: an accessibility click states
667+// depressed 0, and the restore carries the real state of the latest key,
668+// swallowed ones included (a latch a label key consumed).
669+func TestHintClickHeldBindModifierAndLatchRestore(t *testing.T) {
670+ h := newHint(t, core.Options{})
671+ tg := h.inside(0)
672+ h.press("g", ports.ModSuper)
673+ req := receive(t, h.scans)
674+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{tg, h.inside(1)}}
675+ h.labelled(2, 0)
676+ h.drain()
677+ real := ports.ModState{Depressed: 64, Latched: 0, Locked: 2}
678+ h.send(ports.KeyEvent{Keysym: "f", Base: "f", Mods: ports.ModSuper, Pressed: true, State: real})
679+ cmds := h.commandsSent()
680+ var restore *ports.SetModifiers
681+ for i, c := range cmds {
682+ if v, ok := c.(ports.SetModifiers); ok {
683+ if i == 0 && (v.Restore || v.Mods != 0) {
684+ t.Fatalf("first modifiers %+v, want depressed 0", v)
685+ }
686+ if v.Restore {
687+ restore = &v
688+ }
689+ }
690+ }
691+ if restore == nil || restore.State != real {
692+ t.Fatalf("restore %+v, want state %+v (commands %v)", restore, real, commandKinds(cmds))
693+ }
694+}
695+
696+// A result is kept only for the window it was scanned on.
697+func TestHintResultWindowChecks(t *testing.T) {
698+ h := newHint(t, core.Options{})
699+ req := h.start()
700+ // Accessibility target naming another window: dropped; naming window 1: kept.
701+ ok := h.inside(1)
702+ ok.Window = 1
703+ bad := h.inside(0)
704+ bad.Window = 7
705+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{bad, ok}}
706+ s := h.labelled(1, 0)
707+ if len(s.HintLabels) != 1 || s.HintLabels[0].Rect.Y != ok.Rect.Y {
708+ t.Fatalf("labels %+v", s.HintLabels)
709+ }
710+}
711+
712+// A window that moved while the scan ran: an OCR target over it is dropped.
713+func TestHintOCRResultDroppedWhenWindowMoved(t *testing.T) {
714+ h := newHint(t, core.Options{})
715+ req := h.start()
716+ tg := h.inside(0)
717+ tg.Text = true
718+ h.client <- ports.WindowMapped{ID: 2}
719+ h.until(func(s ports.Scene) bool { return len(s.Windows) == 2 })
720+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{tg}}
721+ // Whatever window now covers the target was not scanned: no labels.
722+ if s := h.labelled(0, 0); len(s.HintLabels) != 0 {
723+ t.Fatalf("labels %+v", s.HintLabels)
724+ }
725+}
726+
727+// A new window covering the target (not in the snapshot) drops it as well.
728+func TestHintOCRResultDroppedForUnseenWindow(t *testing.T) {
729+ h := newHint(t, core.Options{})
730+ req := h.start()
731+ h.client <- ports.WindowMapped{ID: 2}
732+ s := h.until(func(s ports.Scene) bool { return len(s.Windows) == 2 })
733+ tg := ports.HintTarget{Text: true}
734+ for _, w := range s.Windows {
735+ if w.ID == 2 {
736+ tg.Rect = ports.Rect{X: w.Rect.X + 5, Y: w.Rect.Y + 5, W: 20, H: 10}
737+ }
738+ }
739+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{tg}}
740+ if s := h.labelled(0, 0); len(s.HintLabels) != 0 {
741+ t.Fatalf("labels %+v", s.HintLabels)
742+ }
743+}
744+
745+// Targets outside the output are dropped.
746+func TestHintDropsTargetsOutsideOutput(t *testing.T) {
747+ h := newHint(t, core.Options{})
748+ req := h.start()
749+ out := ports.HintTarget{Rect: ports.Rect{X: 2000, Y: 10, W: 10, H: 10}}
750+ h.hints <- ports.HintScanResult{Seq: req.Seq, Output: req.Output, Targets: []ports.HintTarget{out, h.inside(0)}}
751+ if s := h.labelled(1, 0); len(s.HintLabels) != 1 {
752+ t.Fatalf("labels %+v", s.HintLabels)
753+ }
754+}
755+
756+// The output changing size while labels are shown ends hint mode.
757+func TestHintCancelsWhenOutputChanges(t *testing.T) {
758+ h := newHint(t, core.Options{})
759+ h.show(h.inside(0))
760+ h.output <- ports.OutputAdded{Info: ports.OutputInfo{Name: "OUT-1", Width: 1200, Height: 800}}
761+ if s := h.labelled(0, 0); len(s.HintLabels) != 0 {
762+ t.Fatalf("labels %+v", s.HintLabels)
763+ }
764+}
765+
766+// A window that does not have the keyboard gets it before the modifiers.
767+func TestHintTextClickFocusesWindowFirst(t *testing.T) {
768+ h := newHintMods(t, core.Options{}, ports.ModCtrl)
769+ h.client <- ports.WindowMapped{ID: 2}
770+ s := h.until(func(s ports.Scene) bool { return len(s.Windows) == 2 })
771+ var r ports.Rect
772+ for _, w := range s.Windows {
773+ if w.ID == 1 {
774+ r = w.Rect
775+ }
776+ }
777+ h.win = r
778+ tg := h.inside(0)
779+ tg.Text = true
780+ h.show(tg)
781+ h.drain()
782+ h.press("f", 0)
783+ got := commandKinds(h.commandsSent())
784+ want := []string{"focus:1", fmt.Sprintf("mods:%d", ports.ModCtrl), "motion", "press", "release", "restore"}
785+ if !slices.Equal(got, want) {
786+ t.Fatalf("commands %v, want %v", got, want)
787+ }
788+}
789+
790+// A click that is no longer allowed sends no modifiers at all.
791+func TestHintTextClickNoModifiersWhenNotClickable(t *testing.T) {
792+ h := newHintMods(t, core.Options{}, ports.ModCtrl)
793+ tg := h.inside(0)
794+ tg.Text = true
795+ h.show(tg)
796+ h.constrain(ports.ConstraintLock, ports.Rect{})
797+ h.drain()
798+ h.press("f", 0)
799+ if got := commandKinds(h.commandsSent()); len(got) != 0 {
800+ t.Fatalf("commands %v under a constraint", got)
801+ }
802+}
803+
804+func TestHintDisabledNotesOnce(t *testing.T) {
805+ cfg := scrollDefaults()
806+ cfg.Binds["Cmd+g"] = "hint"
807+ var noted atomic.Int32
808+ h := &hintHarness{
809+ input: make(chan ports.InputEvent, 32),
810+ client: make(chan ports.ClientEvent, 32),
811+ output: make(chan ports.OutputEvent, 8),
812+ commands: make(chan ports.ClientCommand, 256),
813+ scenes: make(chan []ports.Scene, 1),
814+ }
815+ ch := core.Channels{Client: h.client, Input: h.input, Output: h.output, Commands: h.commands, Scenes: h.scenes}
816+ c, err := core.New(cfg, ch, core.Options{HintDisabled: func() { noted.Add(1) }})
817+ if err != nil {
818+ t.Fatal(err)
819+ }
820+ ctx, cancel := context.WithCancel(context.Background())
821+ done := make(chan error, 1)
822+ go func() { done <- c.Run(ctx) }()
823+ t.Cleanup(func() { cancel(); receive(t, done) })
824+ h.output <- ports.OutputAdded{Info: ports.OutputInfo{Name: "OUT-1", Width: 1000, Height: 800}}
825+ scene(t, h.scenes)
826+ for range 3 {
827+ h.press("g", ports.ModSuper)
828+ h.release("g", 0)
829+ }
830+ c.Ping(ctx)
831+ if got := noted.Load(); got != 1 {
832+ t.Fatalf("disabled note ran %d times, want 1", got)
833+ }
834+}
M internal/core/input.go +22 -8
@@ -124,6 +124,21 @@ func (c *Core) pointerMotion(ctx context.Context, v ports.PointerMotion) error {
124124 return nil
125125 }
126126
127+// clickFocus focuses the window id and its output the way a click on it
128+// does, unless it already has the keyboard.
129+func (c *Core) clickFocus(ctx context.Context, id WindowID) error {
130+ s, w := c.screenOf(id)
131+ if s == nil || w != s.mon.Current() || (c.keyboard.sent == id && s == c.cur()) {
132+ return nil
133+ }
134+ now := c.now()
135+ before := c.snapshot(now)
136+ w.Click(id)
137+ c.focusScreen = c.screenIndex(s.name())
138+ c.transition(before, now)
139+ return c.publish(ctx)
140+}
141+
127142 func (c *Core) pointerButton(ctx context.Context, v ports.PointerButton) error {
128143 // Buttons a drag took stay in c.buttons until released, so the
129144 // security gate admits their release; no client sees it.
@@ -199,14 +214,8 @@ func (c *Core) pointerButton(ctx context.Context, v ports.PointerButton) error {
199214 }
200215 }
201216 // A click focuses the window and its output.
202- s, w := c.screenOf(id)
203- if v.Pressed && s != nil && w == s.mon.Current() && (c.keyboard.sent != id || s != c.cur()) {
204- now := c.now()
205- before := c.snapshot(now)
206- w.Click(id)
207- c.focusScreen = c.screenIndex(s.name())
208- c.transition(before, now)
209- if err := c.publish(ctx); err != nil {
217+ if v.Pressed {
218+ if err := c.clickFocus(ctx, id); err != nil {
210219 return err
211220 }
212221 }
@@ -249,6 +258,7 @@ func (c *Core) pointerAxis(ctx context.Context, v ports.PointerAxis) error {
249258
250259 func (c *Core) keyEvent(ctx context.Context, key ports.KeyEvent) error {
251260 c.mods = key.Mods
261+ c.keyState = key.State
252262 c.repeatKey(key)
253263 // Releasing the command key commits the switcher; the release still
254264 // goes on to the bind lookup and the clients.
@@ -277,6 +287,10 @@ func (c *Core) keyEvent(ctx context.Context, key ports.KeyEvent) error {
277287 }
278288 return c.rehit(ctx)
279289 }
290+ // Hint mode takes every key press before binds and windows.
291+ if handled, err := c.hintKey(ctx, key); handled || err != nil {
292+ return err
293+ }
280294 // The overview takes its keys before any window; others still
281295 // run binds, and are not forwarded.
282296 // A launcher or a menu holding the keyboard gets them first.
@@ -124,6 +124,21 @@ func (c *Core) pointerMotion(ctx context.Context, v ports.PointerMotion) error {
124 return nil124 return nil
125 }125 }
126 126
127+// clickFocus focuses the window id and its output the way a click on it
128+// does, unless it already has the keyboard.
129+func (c *Core) clickFocus(ctx context.Context, id WindowID) error {
130+ s, w := c.screenOf(id)
131+ if s == nil || w != s.mon.Current() || (c.keyboard.sent == id && s == c.cur()) {
132+ return nil
133+ }
134+ now := c.now()
135+ before := c.snapshot(now)
136+ w.Click(id)
137+ c.focusScreen = c.screenIndex(s.name())
138+ c.transition(before, now)
139+ return c.publish(ctx)
140+}
141+
127 func (c *Core) pointerButton(ctx context.Context, v ports.PointerButton) error {142 func (c *Core) pointerButton(ctx context.Context, v ports.PointerButton) error {
128 // Buttons a drag took stay in c.buttons until released, so the143 // Buttons a drag took stay in c.buttons until released, so the
129 // security gate admits their release; no client sees it.144 // security gate admits their release; no client sees it.
@@ -199,14 +214,8 @@ func (c *Core) pointerButton(ctx context.Context, v ports.PointerButton) error {
199 }214 }
200 }215 }
201 // A click focuses the window and its output.216 // A click focuses the window and its output.
202- s, w := c.screenOf(id)217+ if v.Pressed {
203- if v.Pressed && s != nil && w == s.mon.Current() && (c.keyboard.sent != id || s != c.cur()) {218+ if err := c.clickFocus(ctx, id); err != nil {
204- now := c.now()
205- before := c.snapshot(now)
206- w.Click(id)
207- c.focusScreen = c.screenIndex(s.name())
208- c.transition(before, now)
209- if err := c.publish(ctx); err != nil {
210 return err219 return err
211 }220 }
212 }221 }
@@ -249,6 +258,7 @@ func (c *Core) pointerAxis(ctx context.Context, v ports.PointerAxis) error {
249 258
250 func (c *Core) keyEvent(ctx context.Context, key ports.KeyEvent) error {259 func (c *Core) keyEvent(ctx context.Context, key ports.KeyEvent) error {
251 c.mods = key.Mods260 c.mods = key.Mods
261+ c.keyState = key.State
252 c.repeatKey(key)262 c.repeatKey(key)
253 // Releasing the command key commits the switcher; the release still263 // Releasing the command key commits the switcher; the release still
254 // goes on to the bind lookup and the clients.264 // goes on to the bind lookup and the clients.
@@ -277,6 +287,10 @@ func (c *Core) keyEvent(ctx context.Context, key ports.KeyEvent) error {
277 }287 }
278 return c.rehit(ctx)288 return c.rehit(ctx)
279 }289 }
290+ // Hint mode takes every key press before binds and windows.
291+ if handled, err := c.hintKey(ctx, key); handled || err != nil {
292+ return err
293+ }
280 // The overview takes its keys before any window; others still294 // The overview takes its keys before any window; others still
281 // run binds, and are not forwarded.295 // run binds, and are not forwarded.
282 // A launcher or a menu holding the keyboard gets them first.296 // A launcher or a menu holding the keyboard gets them first.
M internal/core/scene.go +2 -0
@@ -123,6 +123,8 @@ func (c *Core) sceneFor(ctx context.Context, i int, sc *screen, only *screen, ca
123123 }
124124 scene.Flash = c.flashFor(sc)
125125 scene.CaptureIndicators = c.captureIndicators(sc)
126+ scene.HintLabels = c.hintLabels(sc)
127+ scene.HintStyle = ports.HintStyle{Size: c.cfg.Hint.Size, Background: c.cfg.Hint.Background, Foreground: c.cfg.Hint.Foreground, Width: sc.info.Width, Height: sc.info.Height, PhysicalW: sc.info.PhysicalW, PhysicalH: sc.info.PhysicalH}
126128 scene.Capture = c.captureSceneFor(sc, capture)
127129 // The springs of this frame are advanced: a scene shown while one still
128130 // runs wakes core when it flips; the one a spring settles on does not.
@@ -123,6 +123,8 @@ func (c *Core) sceneFor(ctx context.Context, i int, sc *screen, only *screen, ca
123 }123 }
124 scene.Flash = c.flashFor(sc)124 scene.Flash = c.flashFor(sc)
125 scene.CaptureIndicators = c.captureIndicators(sc)125 scene.CaptureIndicators = c.captureIndicators(sc)
126+ scene.HintLabels = c.hintLabels(sc)
127+ scene.HintStyle = ports.HintStyle{Size: c.cfg.Hint.Size, Background: c.cfg.Hint.Background, Foreground: c.cfg.Hint.Foreground, Width: sc.info.Width, Height: sc.info.Height, PhysicalW: sc.info.PhysicalW, PhysicalH: sc.info.PhysicalH}
126 scene.Capture = c.captureSceneFor(sc, capture)128 scene.Capture = c.captureSceneFor(sc, capture)
127 // The springs of this frame are advanced: a scene shown while one still129 // The springs of this frame are advanced: a scene shown while one still
128 // runs wakes core when it flips; the one a spring settles on does not.130 // runs wakes core when it flips; the one a spring settles on does not.
M internal/ports/capture_messages.go +6 -0
@@ -65,3 +65,9 @@ type CaptureDone struct {
6565 Err error
6666 Time time.Time
6767 }
68+
69+// CaptureIDInternal is the bit set in the ID of captures requested by the
70+// compositor itself (hint scanning) rather than by a Wayland client. Wayland
71+// IDs start at 1 and increment, so they never set it. Replies carrying it are
72+// demultiplexed away from the Wayland consumer.
73+const CaptureIDInternal uint64 = 1 << 63
@@ -65,3 +65,9 @@ type CaptureDone struct {
65 Err error65 Err error
66 Time time.Time66 Time time.Time
67 }67 }
68+
69+// CaptureIDInternal is the bit set in the ID of captures requested by the
70+// compositor itself (hint scanning) rather than by a Wayland client. Wayland
71+// IDs start at 1 and increment, so they never set it. Replies carrying it are
72+// demultiplexed away from the Wayland consumer.
73+const CaptureIDInternal uint64 = 1 << 63
M internal/ports/client_commands.go +14 -0
@@ -289,3 +289,17 @@ type SpawnRequest struct {
289289 // Env adds KEY=value entries to the child environment.
290290 Env []string
291291 }
292+
293+// SetModifiers carries core → wayland a temporary modifier override for the
294+// focused window's keyboards: Mods are sent as depressed modifiers (mapped
295+// to the current keymap's masks), latched/locked/group kept from the real
296+// state. Mods 0 with Restore true sends the real modifier state again: State
297+// is that state as core last saw it, which the seat adopts.
298+type SetModifiers struct {
299+ ID WindowID
300+ Mods Mods
301+ Restore bool
302+ State ModState
303+}
304+
305+func (SetModifiers) clientCommand() {}
@@ -289,3 +289,17 @@ type SpawnRequest struct {
289 // Env adds KEY=value entries to the child environment.289 // Env adds KEY=value entries to the child environment.
290 Env []string290 Env []string
291 }291 }
292+
293+// SetModifiers carries core → wayland a temporary modifier override for the
294+// focused window's keyboards: Mods are sent as depressed modifiers (mapped
295+// to the current keymap's masks), latched/locked/group kept from the real
296+// state. Mods 0 with Restore true sends the real modifier state again: State
297+// is that state as core last saw it, which the seat adopts.
298+type SetModifiers struct {
299+ ID WindowID
300+ Mods Mods
301+ Restore bool
302+ State ModState
303+}
304+
305+func (SetModifiers) clientCommand() {}
M internal/ports/config.go +14 -0
@@ -161,6 +161,20 @@ type Config struct {
161161 FlashDuration time.Duration
162162 FlashColor string
163163 }
164+ // Hint configures keyboard hint mode (action hint).
165+ Hint struct {
166+ // Enable turns the experimental hint mode on (hint.enable). It is
167+ // read at startup: a reload keeps the startup value.
168+ Enable bool
169+ // LinksOnly limits the targets to links and URLs; false also
170+ // includes buttons (hint.targets = clickable).
171+ LinksOnly bool
172+ // TextClick are the modifiers held for a click on a link found by
173+ // OCR (hint.text-click); zero sends a plain click.
174+ TextClick Mods
175+ Size float64 // logical pixels; zero selects automatic density sizing
176+ Background, Foreground string // empty derives from Border.Active
177+ }
164178 // Workspaces are declared with workspace.<name>.* keys, in first-seen order.
165179 Workspaces []WorkspaceConfig
166180 // Rules are the window rules (rule.<name>.* keys), in first-seen order.
@@ -161,6 +161,20 @@ type Config struct {
161 FlashDuration time.Duration161 FlashDuration time.Duration
162 FlashColor string162 FlashColor string
163 }163 }
164+ // Hint configures keyboard hint mode (action hint).
165+ Hint struct {
166+ // Enable turns the experimental hint mode on (hint.enable). It is
167+ // read at startup: a reload keeps the startup value.
168+ Enable bool
169+ // LinksOnly limits the targets to links and URLs; false also
170+ // includes buttons (hint.targets = clickable).
171+ LinksOnly bool
172+ // TextClick are the modifiers held for a click on a link found by
173+ // OCR (hint.text-click); zero sends a plain click.
174+ TextClick Mods
175+ Size float64 // logical pixels; zero selects automatic density sizing
176+ Background, Foreground string // empty derives from Border.Active
177+ }
164 // Workspaces are declared with workspace.<name>.* keys, in first-seen order.178 // Workspaces are declared with workspace.<name>.* keys, in first-seen order.
165 Workspaces []WorkspaceConfig179 Workspaces []WorkspaceConfig
166 // Rules are the window rules (rule.<name>.* keys), in first-seen order.180 // Rules are the window rules (rule.<name>.* keys), in first-seen order.
A internal/ports/hint_messages.go +67 -0
@@ -0,0 +1,67 @@
1+package ports
2+
3+// HintKind is the kind of a clickable hint target.
4+type HintKind uint8
5+
6+const (
7+ // HintLink is a link or URL.
8+ HintLink HintKind = iota
9+ // HintButton is a button or control.
10+ HintButton
11+)
12+
13+// HintScanRequest asks the hint scanner to find clickable targets in the last
14+// frame of Output. Core sends it and expects one HintScanResult echoing Seq.
15+type HintScanRequest struct {
16+ Seq uint64 // core-chosen, echoed back
17+ Output string
18+ // Width and Height are the output buffer size in physical pixels
19+ // (OutputInfo.Width/Height).
20+ Width, Height int
21+ Scale float64 // screen scale
22+ LinksOnly bool
23+ // Windows are the windows visible on Output, for sources that read a
24+ // toolkit's accessibility tree. Core owns the slice per request.
25+ Windows []HintWindow
26+}
27+
28+// HintWindow is a window visible on the hint output.
29+type HintWindow struct {
30+ ID WindowID
31+ PID int
32+ AppID string
33+ // Rect is the client area, logical output-local.
34+ Rect Rect
35+ Focused bool
36+}
37+
38+// HintTarget is a clickable region found in a frame.
39+type HintTarget struct {
40+ Rect Rect // logical, output-local
41+ Kind HintKind
42+ // Text is set for targets found by OCR (read from pixels, not from an
43+ // accessibility tree).
44+ Text bool
45+ // Window is the source window of an accessibility target, 0 for OCR.
46+ Window WindowID
47+}
48+
49+// HintScanResult answers a HintScanRequest. Core owns Targets.
50+type HintScanResult struct {
51+ Seq uint64
52+ Output string
53+ Targets []HintTarget
54+ Err error
55+}
56+
57+// HintLabel is a label drawn over a hint target.
58+type HintLabel struct {
59+ // Rect is the label box in logical output-local coordinates; X and Y are
60+ // the target's top-left corner, which the renderer anchors the label
61+ // outside of. W and H may be 0: the renderer sizes the box from Text.
62+ Rect Rect
63+ // Text is the full label, lowercase a–z.
64+ Text string
65+ // Typed is the number of leading characters already typed (drawn dimmed).
66+ Typed int
67+}
@@ -0,0 +1,67 @@
1+package ports
2+
3+// HintKind is the kind of a clickable hint target.
4+type HintKind uint8
5+
6+const (
7+ // HintLink is a link or URL.
8+ HintLink HintKind = iota
9+ // HintButton is a button or control.
10+ HintButton
11+)
12+
13+// HintScanRequest asks the hint scanner to find clickable targets in the last
14+// frame of Output. Core sends it and expects one HintScanResult echoing Seq.
15+type HintScanRequest struct {
16+ Seq uint64 // core-chosen, echoed back
17+ Output string
18+ // Width and Height are the output buffer size in physical pixels
19+ // (OutputInfo.Width/Height).
20+ Width, Height int
21+ Scale float64 // screen scale
22+ LinksOnly bool
23+ // Windows are the windows visible on Output, for sources that read a
24+ // toolkit's accessibility tree. Core owns the slice per request.
25+ Windows []HintWindow
26+}
27+
28+// HintWindow is a window visible on the hint output.
29+type HintWindow struct {
30+ ID WindowID
31+ PID int
32+ AppID string
33+ // Rect is the client area, logical output-local.
34+ Rect Rect
35+ Focused bool
36+}
37+
38+// HintTarget is a clickable region found in a frame.
39+type HintTarget struct {
40+ Rect Rect // logical, output-local
41+ Kind HintKind
42+ // Text is set for targets found by OCR (read from pixels, not from an
43+ // accessibility tree).
44+ Text bool
45+ // Window is the source window of an accessibility target, 0 for OCR.
46+ Window WindowID
47+}
48+
49+// HintScanResult answers a HintScanRequest. Core owns Targets.
50+type HintScanResult struct {
51+ Seq uint64
52+ Output string
53+ Targets []HintTarget
54+ Err error
55+}
56+
57+// HintLabel is a label drawn over a hint target.
58+type HintLabel struct {
59+ // Rect is the label box in logical output-local coordinates; X and Y are
60+ // the target's top-left corner, which the renderer anchors the label
61+ // outside of. W and H may be 0: the renderer sizes the box from Text.
62+ Rect Rect
63+ // Text is the full label, lowercase a–z.
64+ Text string
65+ // Typed is the number of leading characters already typed (drawn dimmed).
66+ Typed int
67+}
M internal/ports/scene.go +14 -1
@@ -69,6 +69,11 @@ type Scene struct {
6969 // drawn over everything; nil when none. Captures never hold them: the
7070 // capture pipeline serves requests from a scene without them.
7171 CaptureIndicators []CaptureIndicator
72+ // HintLabels are the keyboard hint labels of this output, drawn over
73+ // everything, after CaptureIndicators; nil when hint mode is not shown
74+ // here. Captures never contain them.
75+ HintLabels []HintLabel
76+ HintStyle HintStyle
7277 // Flash is the focus flash drawn over everything of this output; nil
7378 // draws nothing.
7479 Flash *FocusFlash
@@ -112,7 +117,8 @@ func (s Scene) SameAs(o Scene) bool {
112117 }
113118 return slices.Equal(s.Windows, o.Windows) && slices.Equal(s.Separators, o.Separators) &&
114119 slices.Equal(s.DropHints, o.DropHints) && slices.Equal(s.Layers, o.Layers) &&
115- slices.Equal(s.CaptureIndicators, o.CaptureIndicators)
120+ slices.Equal(s.CaptureIndicators, o.CaptureIndicators) &&
121+ s.HintStyle == o.HintStyle && slices.Equal(s.HintLabels, o.HintLabels)
116122 }
117123
118124 // Shows reports whether the scene draws the surface of id: only its
@@ -233,3 +239,10 @@ type SceneWindow struct {
233239 func (w SceneWindow) Tile() bool {
234240 return !w.Floating && !w.Fullscreen && !w.Popup && w.Preview == 0
235241 }
242+
243+// HintStyle carries reloadable label settings and untransformed output density.
244+type HintStyle struct {
245+ Size float64
246+ Background, Foreground string
247+ Width, Height, PhysicalW, PhysicalH int
248+}
@@ -69,6 +69,11 @@ type Scene struct {
69 // drawn over everything; nil when none. Captures never hold them: the69 // drawn over everything; nil when none. Captures never hold them: the
70 // capture pipeline serves requests from a scene without them.70 // capture pipeline serves requests from a scene without them.
71 CaptureIndicators []CaptureIndicator71 CaptureIndicators []CaptureIndicator
72+ // HintLabels are the keyboard hint labels of this output, drawn over
73+ // everything, after CaptureIndicators; nil when hint mode is not shown
74+ // here. Captures never contain them.
75+ HintLabels []HintLabel
76+ HintStyle HintStyle
72 // Flash is the focus flash drawn over everything of this output; nil77 // Flash is the focus flash drawn over everything of this output; nil
73 // draws nothing.78 // draws nothing.
74 Flash *FocusFlash79 Flash *FocusFlash
@@ -112,7 +117,8 @@ func (s Scene) SameAs(o Scene) bool {
112 }117 }
113 return slices.Equal(s.Windows, o.Windows) && slices.Equal(s.Separators, o.Separators) &&118 return slices.Equal(s.Windows, o.Windows) && slices.Equal(s.Separators, o.Separators) &&
114 slices.Equal(s.DropHints, o.DropHints) && slices.Equal(s.Layers, o.Layers) &&119 slices.Equal(s.DropHints, o.DropHints) && slices.Equal(s.Layers, o.Layers) &&
115- slices.Equal(s.CaptureIndicators, o.CaptureIndicators)120+ slices.Equal(s.CaptureIndicators, o.CaptureIndicators) &&
121+ s.HintStyle == o.HintStyle && slices.Equal(s.HintLabels, o.HintLabels)
116 }122 }
117 123
118 // Shows reports whether the scene draws the surface of id: only its124 // Shows reports whether the scene draws the surface of id: only its
@@ -233,3 +239,10 @@ type SceneWindow struct {
233 func (w SceneWindow) Tile() bool {239 func (w SceneWindow) Tile() bool {
234 return !w.Floating && !w.Fullscreen && !w.Popup && w.Preview == 0240 return !w.Floating && !w.Fullscreen && !w.Popup && w.Preview == 0
235 }241 }
242+
243+// HintStyle carries reloadable label settings and untransformed output density.
244+type HintStyle struct {
245+ Size float64
246+ Background, Foreground string
247+ Width, Height, PhysicalW, PhysicalH int
248+}
M internal/ports/scene_same_test.go +4 -2
@@ -25,6 +25,7 @@ func sameSceneBase() Scene {
2525
2626 Flash: &FocusFlash{Rect: Rect{W: 20, H: 10}, Width: 2, Color: "#ff0000", Alpha: 0.6},
2727 CaptureIndicators: []CaptureIndicator{{Rect: Rect{W: 4, H: 4}}},
28+ HintLabels: []HintLabel{{Rect: Rect{X: 3, Y: 4}, Text: "fj", Typed: 1}},
2829 }
2930 }
3031
@@ -61,6 +62,7 @@ func TestSceneSameAs(t *testing.T) {
6162 "drop hint": func(s *Scene) { s.DropHints[0].W++ },
6263 "layer": func(s *Scene) { s.Layers[0].Rect.H++ },
6364 "indicator": func(s *Scene) { s.CaptureIndicators[0].Pill = true },
65+ "hint style": func(s *Scene) { s.HintStyle.Size = 24 },
6466 "excluded": func(s *Scene) { s.Capture.Excluded[0] = 4 },
6567 "transform": func(s *Scene) { s.Transform = 1 },
6668 "capture shown": func(s *Scene) { s.Capture.Shown++ },
@@ -89,9 +91,9 @@ func TestSceneSameAs(t *testing.T) {
8991 // TestSceneSameAsCoversEveryField fails when Scene gains or loses a field:
9092 // update Scene.SameAs, then this count.
9193 func TestSceneSameAsCoversEveryField(t *testing.T) {
92- // 23 with Animating, which SameAs intentionally does not compare: it
94+ // 25 with Animating, which SameAs intentionally does not compare: it
9395 // only schedules the next frame, it draws nothing.
94- const fields = 23
96+ const fields = 25
9597 assert.Equal(t, fields, len(reflect.VisibleFields(reflect.TypeOf(Scene{}))),
9698 "Scene fields changed: update Scene.SameAs (and TestSceneSameAs), then this count")
9799 }
@@ -25,6 +25,7 @@ func sameSceneBase() Scene {
25 25
26 Flash: &FocusFlash{Rect: Rect{W: 20, H: 10}, Width: 2, Color: "#ff0000", Alpha: 0.6},26 Flash: &FocusFlash{Rect: Rect{W: 20, H: 10}, Width: 2, Color: "#ff0000", Alpha: 0.6},
27 CaptureIndicators: []CaptureIndicator{{Rect: Rect{W: 4, H: 4}}},27 CaptureIndicators: []CaptureIndicator{{Rect: Rect{W: 4, H: 4}}},
28+ HintLabels: []HintLabel{{Rect: Rect{X: 3, Y: 4}, Text: "fj", Typed: 1}},
28 }29 }
29 }30 }
30 31
@@ -61,6 +62,7 @@ func TestSceneSameAs(t *testing.T) {
61 "drop hint": func(s *Scene) { s.DropHints[0].W++ },62 "drop hint": func(s *Scene) { s.DropHints[0].W++ },
62 "layer": func(s *Scene) { s.Layers[0].Rect.H++ },63 "layer": func(s *Scene) { s.Layers[0].Rect.H++ },
63 "indicator": func(s *Scene) { s.CaptureIndicators[0].Pill = true },64 "indicator": func(s *Scene) { s.CaptureIndicators[0].Pill = true },
65+ "hint style": func(s *Scene) { s.HintStyle.Size = 24 },
64 "excluded": func(s *Scene) { s.Capture.Excluded[0] = 4 },66 "excluded": func(s *Scene) { s.Capture.Excluded[0] = 4 },
65 "transform": func(s *Scene) { s.Transform = 1 },67 "transform": func(s *Scene) { s.Transform = 1 },
66 "capture shown": func(s *Scene) { s.Capture.Shown++ },68 "capture shown": func(s *Scene) { s.Capture.Shown++ },
@@ -89,9 +91,9 @@ func TestSceneSameAs(t *testing.T) {
89 // TestSceneSameAsCoversEveryField fails when Scene gains or loses a field:91 // TestSceneSameAsCoversEveryField fails when Scene gains or loses a field:
90 // update Scene.SameAs, then this count.92 // update Scene.SameAs, then this count.
91 func TestSceneSameAsCoversEveryField(t *testing.T) {93 func TestSceneSameAsCoversEveryField(t *testing.T) {
92- // 23 with Animating, which SameAs intentionally does not compare: it94+ // 25 with Animating, which SameAs intentionally does not compare: it
93 // only schedules the next frame, it draws nothing.95 // only schedules the next frame, it draws nothing.
94- const fields = 2396+ const fields = 25
95 assert.Equal(t, fields, len(reflect.VisibleFields(reflect.TypeOf(Scene{}))),97 assert.Equal(t, fields, len(reflect.VisibleFields(reflect.TypeOf(Scene{}))),
96 "Scene fields changed: update Scene.SameAs (and TestSceneSameAs), then this count")98 "Scene fields changed: update Scene.SameAs (and TestSceneSameAs), then this count")
97 }99 }

Comments0 open / 0

no comments yet.

Keyboard shortcuts

Single-key shortcuts are on. ctrl k and esc work whatever this says.

ctrl k :
go to a section, branch, tag, file or repository
?
this list
j k
move the row cursor down / up
enter
open the row under the cursor; on Files, a directory opens or closes in place
↑ ↓ ← → home end
in the file tree (focus on a row): previous / next row, close / open a directory or go to its parent / first entry, first / last row. They never open a file
l h
on Files: open a directory or a file / close a directory or go up to its parent, then to the parent page. Elsewhere: into the side panel (on a merge request, its files) / back to the content; there j k move between its controls
g g G
first / last row
/
filter the list; on Files, find a file anywhere in the repository (esc clears, enter opens the best match)
b
switch branch or tag
g s g f g c g b g m g i g r g a
tabs: summary, files, commits, branches & tags, merge requests, issues, releases, actions (when available)
g h
home
w
wide mode: use the whole width of the screen, or not (remembered)
y
copy a permalink pinned to the commit
d u x
diff: split / unified / collapse or expand all
e E
diff: the whole file in view / every file, with the lines the hunks hide
n
merge request: next note in the diff
l
merge request: the list of files (j k move, enter opens, esc back)
M
merge request: the merge menu; there 1–3 pick the mode, D deletes the branch, enter merges
R
merge request: mark the head read
esc
close a dialog, leave a field

Branches 25

Tags 10

↑↓ moveenter openesc close