docs(config): describe hybrid hint target sources
- commit
- 386d4456c5f492f0cb3f323127f9ba6960104ac3
- author
- brice · · 2026-10-19 13:26 UTC
- parent
- e9ffd04
- signature
- GPG Verified
CD88F9E564E0102646FB6077A1A07C7C7C3E349D· arch-gpg Verified for the committer email
1 file · +25 -12
M docs/config.md +25 -12
| @@ -298,19 +298,32 @@ With `layout.overflow = cascade`, workspaces sit side by side, each band of colu | |||
| 298 | 298 | ||
| 299 | 299 | Otherwise, left from the first column unfolds the stash. Up/down walk the cards, the focus-workspace binds change workspace, and right goes back to the columns. | |
| 300 | 300 | ||
| 301 | - | ### Hint mode | |
| 301 | + | ## Hint mode | |
| 302 | 302 | ||
| 303 | - | `hint` labels the links, URLs and buttons visible on the focused screen, so you can click one from the keyboard. Targets come from the accessibility tree when the app exposes it, and from the pixels of the screen otherwise. No bind is set by default; for example `bind.cmd+g = hint`. | |
| 303 | + | `hint` labels the links and buttons on the focused output. Type a label to | |
| 304 | + | click its target; BackSpace erases a letter, Escape cancels. | |
| 304 | 305 | ||
| 305 | - | - Each target gets a label of lowercase letters. With up to 26 targets the labels are one letter, with more they are two. | |
| 306 | - | - Type a label to click its target: the pointer moves to the center of the target and presses the left button. With two-letter labels, the labels that do not match what you typed disappear and the typed letters are dimmed. | |
| 307 | - | - `BackSpace` removes the last typed letter. `Escape` cancels, and so does any other key, which no window receives. | |
| 308 | - | - Only the visible area of windows is scanned: a target covered by another window or outside the screen has no label. Targets on other screens are not labeled. | |
| 309 | - | - A window whose app exposes its accessibility tree gets exactly the links and controls the app reports; the other windows are scanned from pixels. NeferWL turns accessibility on when it starts. GTK apps and Firefox work once it is on. Chromium and Electron apps need `--force-renderer-accessibility` (some Electron apps need `ACCESSIBILITY_ENABLED=1` instead), and Qt apps need `QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1`. Without the accessibility bus, every window is scanned from pixels. | |
| 310 | - | - `hint.targets = links` labels links and URLs only, not buttons. | |
| 311 | - | - `hint` does nothing while the session is locked, on a rotated or flipped screen, while the overview is open, while a pointer button, drag or pointer constraint is active, or while hint mode is already active. Hint mode ends when the workspace on that screen changes or the overview opens. | |
| 312 | - | - No capture indicator appears: the scan happens inside NeferWL and no client receives the pixels. Screen captures do not contain the labels. | |
| 306 | + | ### Where targets come from | |
| 313 | 307 | ||
| 314 | - | #### Debugging hint mode | |
| 308 | + | | Source | Apps | Accuracy | Speed | | |
| 309 | + | |---|---|---|---| | |
| 310 | + | | Accessibility tree (AT-SPI) | Firefox, Chromium, GTK, Qt, Electron | Exact: the app reports its own links and buttons | A few milliseconds | | |
| 311 | + | | Text recognition (OCR) | Everything else: terminals, apps without accessibility | Best effort: only URLs and domains written on screen | 150 ms or more, more on large screens | | |
| 315 | 312 | ||
| 316 | - | With `NEFERWL_HINT_DEBUG=1` set when NeferWL starts (for example `NEFERWL_HINT_DEBUG=1 make tty`, or the variable in the launch environment), every successful scan writes three files to `$XDG_STATE_HOME/neferwl/hints/` (`~/.local/state/neferwl/hints/` when unset): `<time>-<output>.png`, the frame as scanned; `<time>-<output>.annotated.png`, the same with each target outlined (links green, buttons blue, text grey, targets from the accessibility tree magenta); and `<time>-<output>.json`, the targets (with their `source`, `atspi` or `pixels`) and timings. NeferWL keeps the 50 most recent scans and logs a warning at startup. The files contain the screen in clear: delete them after use. | |
| 313 | + | NeferWL asks the accessibility tree first. Windows it does not cover are read | |
| 314 | + | with OCR, which finds visible URLs, domains and email addresses. OCR cannot | |
| 315 | + | see a link whose address is hidden behind its text, and it can misread small | |
| 316 | + | or low-contrast text. | |
| 317 | + | ||
| 318 | + | ### Requirements | |
| 319 | + | ||
| 320 | + | - Accessibility: NeferWL turns it on at startup. Some apps need a flag: | |
| 321 | + | - Chromium: `--force-renderer-accessibility` | |
| 322 | + | - Electron: `ACCESSIBILITY_ENABLED=1` | |
| 323 | + | - Qt: `QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1` | |
| 324 | + | - OCR: `tesseract` and `tesseract-data-eng`. Without them, only apps with | |
| 325 | + | accessibility get labels. | |
| 326 | + | ||
| 327 | + | ### Debugging hint mode | |
| 328 | + | ||
| 329 | + | With `NEFERWL_HINT_DEBUG=1` set when NeferWL starts (for example `NEFERWL_HINT_DEBUG=1 make tty`, or the variable in the launch environment), every successful scan writes three files to `$XDG_STATE_HOME/neferwl/hints/` (`~/.local/state/neferwl/hints/` when unset): `<time>-<output>.png`, the frame as scanned; `<time>-<output>.annotated.png`, the same with each target outlined (OCR links green, accessibility targets magenta, OCR window crops grey); and `<time>-<output>.json`, the targets (with their `source`, `atspi` or `ocr`) and timings. NeferWL keeps the 50 most recent scans and logs a warning at startup. The files contain the screen in clear: delete them after use. | |
| @@ -298,19 +298,32 @@ With `layout.overflow = cascade`, workspaces sit side by side, each band of colu | |||||
| 298 | 298 | ||||
| 299 | Otherwise, left from the first column unfolds the stash. Up/down walk the cards, the focus-workspace binds change workspace, and right goes back to the columns. | 299 | Otherwise, left from the first column unfolds the stash. Up/down walk the cards, the focus-workspace binds change workspace, and right goes back to the columns. | ||
| 300 | 300 | ||||
| 301 | - | ### Hint mode | 301 | + | ## Hint mode |
| 302 | 302 | ||||
| 303 | - | `hint` labels the links, URLs and buttons visible on the focused screen, so you can click one from the keyboard. Targets come from the accessibility tree when the app exposes it, and from the pixels of the screen otherwise. No bind is set by default; for example `bind.cmd+g = hint`. | 303 | + | `hint` labels the links and buttons on the focused output. Type a label to |
| 304 | + | click its target; BackSpace erases a letter, Escape cancels. | |||
| 304 | 305 | ||||
| 305 | - | - Each target gets a label of lowercase letters. With up to 26 targets the labels are one letter, with more they are two. | 306 | + | ### Where targets come from |
| 306 | - | - Type a label to click its target: the pointer moves to the center of the target and presses the left button. With two-letter labels, the labels that do not match what you typed disappear and the typed letters are dimmed. | |||
| 307 | - | - `BackSpace` removes the last typed letter. `Escape` cancels, and so does any other key, which no window receives. | |||
| 308 | - | - Only the visible area of windows is scanned: a target covered by another window or outside the screen has no label. Targets on other screens are not labeled. | |||
| 309 | - | - A window whose app exposes its accessibility tree gets exactly the links and controls the app reports; the other windows are scanned from pixels. NeferWL turns accessibility on when it starts. GTK apps and Firefox work once it is on. Chromium and Electron apps need `--force-renderer-accessibility` (some Electron apps need `ACCESSIBILITY_ENABLED=1` instead), and Qt apps need `QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1`. Without the accessibility bus, every window is scanned from pixels. | |||
| 310 | - | - `hint.targets = links` labels links and URLs only, not buttons. | |||
| 311 | - | - `hint` does nothing while the session is locked, on a rotated or flipped screen, while the overview is open, while a pointer button, drag or pointer constraint is active, or while hint mode is already active. Hint mode ends when the workspace on that screen changes or the overview opens. | |||
| 312 | - | - No capture indicator appears: the scan happens inside NeferWL and no client receives the pixels. Screen captures do not contain the labels. | |||
| 313 | 307 | ||||
| 314 | - | #### Debugging hint mode | 308 | + | | Source | Apps | Accuracy | Speed | |
| 309 | + | |---|---|---|---| | |||
| 310 | + | | Accessibility tree (AT-SPI) | Firefox, Chromium, GTK, Qt, Electron | Exact: the app reports its own links and buttons | A few milliseconds | | |||
| 311 | + | | Text recognition (OCR) | Everything else: terminals, apps without accessibility | Best effort: only URLs and domains written on screen | 150 ms or more, more on large screens | | |||
| 315 | 312 | ||||
| 316 | - | With `NEFERWL_HINT_DEBUG=1` set when NeferWL starts (for example `NEFERWL_HINT_DEBUG=1 make tty`, or the variable in the launch environment), every successful scan writes three files to `$XDG_STATE_HOME/neferwl/hints/` (`~/.local/state/neferwl/hints/` when unset): `<time>-<output>.png`, the frame as scanned; `<time>-<output>.annotated.png`, the same with each target outlined (links green, buttons blue, text grey, targets from the accessibility tree magenta); and `<time>-<output>.json`, the targets (with their `source`, `atspi` or `pixels`) and timings. NeferWL keeps the 50 most recent scans and logs a warning at startup. The files contain the screen in clear: delete them after use. | 313 | + | NeferWL asks the accessibility tree first. Windows it does not cover are read |
| 314 | + | with OCR, which finds visible URLs, domains and email addresses. OCR cannot | |||
| 315 | + | see a link whose address is hidden behind its text, and it can misread small | |||
| 316 | + | or low-contrast text. | |||
| 317 | + | ||||
| 318 | + | ### Requirements | |||
| 319 | + | ||||
| 320 | + | - Accessibility: NeferWL turns it on at startup. Some apps need a flag: | |||
| 321 | + | - Chromium: `--force-renderer-accessibility` | |||
| 322 | + | - Electron: `ACCESSIBILITY_ENABLED=1` | |||
| 323 | + | - Qt: `QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1` | |||
| 324 | + | - OCR: `tesseract` and `tesseract-data-eng`. Without them, only apps with | |||
| 325 | + | accessibility get labels. | |||
| 326 | + | ||||
| 327 | + | ### Debugging hint mode | |||
| 328 | + | ||||
| 329 | + | With `NEFERWL_HINT_DEBUG=1` set when NeferWL starts (for example `NEFERWL_HINT_DEBUG=1 make tty`, or the variable in the launch environment), every successful scan writes three files to `$XDG_STATE_HOME/neferwl/hints/` (`~/.local/state/neferwl/hints/` when unset): `<time>-<output>.png`, the frame as scanned; `<time>-<output>.annotated.png`, the same with each target outlined (OCR links green, accessibility targets magenta, OCR window crops grey); and `<time>-<output>.json`, the targets (with their `source`, `atspi` or `ocr`) and timings. NeferWL keeps the 50 most recent scans and logs a warning at startup. The files contain the screen in clear: delete them after use. | |||