#124 perf(capture): reuse cursor and DMA-BUF capture resources

state
merged · 1c8b56b · merge commit
branch
perf/capture-cursor → main
opened
bnema ·
updated

Cuts allocations and GPU work in screen capture and screencast.

  • Cursor capture reuses its argument scratch and skips idle cursor snapshots and pixel clones.
  • Cast images are retained for the buffer pool lifetime; DMA copy command buffers and sync objects are recycled.
  • Idle capture resources are reclaimed during trim.
  • perf-check covers the cursor and DMA-BUF copy budgets.

Review: go-reviewer approved. Tests: race, mocks, staticcheck, perf-check budgets.

CommitSubjectAuthorDateSignatureStatus
4351adbMerge remote-tracking branch 'origin/main' into perf/capture-cursorbrice GPG Verified
8379177docs(vulkan): describe retained cast image ownershipbrice GPG Verified
9f3fcf0perf(vulkan): reclaim idle capture resources during trimbrice GPG Verified
316c010test(capture): attribute gated DMA copies and budget sync wrappersbrice GPG Verified
dec5201perf(vulkan): recycle capture DMA copy command and sync objectsbrice GPG Verified
1cc00fbperf(vulkan): retain cast images for buffer pool lifetimebrice GPG Verified
63c0c77perf(capture): skip idle cursor snapshots and pixel clonesbrice GPG Verified
316d8e9perf(vulkan): reuse cursor capture argument scratchbrice GPG Verified

Checks 1

26 files · +952 -148
1/1
M Makefile +3 -1
@@ -64,8 +64,10 @@ arch: adapter-imports
6464 adapter-imports:
6565 CGO_ENABLED=0 go test . -run '^TestAdapterImports$$' -count=1
6666 # Fast, mandatory guards for tiled content publications and SHM copies.
67+# GPU capture budgets follow the package tests: run on available Vulkan GPUs,
68+# otherwise skip via their existing capability gates; idle budgets need no GPU.
6769 perf-check:
68- CGO_ENABLED=0 go test ./internal/adapters/outputkit/presented ./internal/core ./internal/app ./internal/adapters/drm ./internal/adapters/headless ./internal/adapters/vulkan ./internal/adapters/wayland ./internal/adapters/outputkit/capture ./internal/adapters/outputkit/routing ./internal/adapters/screencast ./internal/adapters/wayland/xwm -run '^(TestLineageObserverDormantAndAllocations|TestWaitNilFencesAllocations|TestAnimateAllocations|TestPublishAllocations|TestPublishExpansionAllocations|TestPublishSubscribersAllocations|TestPublishStashSlideAllocations|TestPublishOverviewAllocations|TestPublishLeavingAllocations|TestOutputRoutingAllocations|TestReportSeenAllocations|TestFrameLifecycleTransitionsAllocations|TestFullscreenShownAllocations|TestFrameDecisionAllocations|TestFrameDecisionExpansionAllocations|TestOverlayExpansionAllocations|TestCommitFrameAllocations|TestColorPipelineAllocations|TestShownBySnapshotAllocations|TestFlipShowsAllocations|TestMRUNoteFocusDoesNotAllocateWhenSettled|TestDueFramesAllocations|TestRenderSteadyStateAllocations|TestRenderExpansionAllocations|TestRenderSteadyStateDMABufAllocations|TestSceneOpaqueRegionAllocations|TestSceneDeltaAllocations|TestAccountFlipAllocations|TestFlipDoneAllocations|TestSceneWalkUnchangedTiledSHM|TestTiledCommitPublishAllocations|TestCapturedCommitApplyAllocations|TestCapturedViewportApplyAllocations|TestEffectiveInputEmptyTreeAllocations|TestHeadlessTiledSHMCallbackAndCopyBudget|TestReleaseHeldAllocations|TestCommitDamageAllocations|TestEmitContentAllocations|TestPresentationPushAllocations|TestPresentationSnapshotAllocations|TestAnimatingFrameSendAllocations|TestContentBatchCycleAllocations|TestWriteMappedAllocations|TestCapturedForwardAllocations|TestCommandForwardAllocations|TestOutboxQueuedDeliveryAllocations|TestCastFrameAllocations|TestCastProducerAllocations|TestCaptureCycleAllocations|TestLogScenesAllocations|TestOverviewNamedRowsAllocations|TestOverviewWorkspaceSwitchAllocations|TestBufferContentAllocations|TestDueFramesFireScratch|TestRetireSemaphoreAllocatesNothing|TestTableLookupAllocations|TestTableInsertRemoveAllocations|TestPropertyUnchangedAllocations|TestDispatchAllocations|TestConfigureFlushAllocations|TestFocusAllocations|TestSpaceAllocations|TestUnmanagedGeometryAllocations|TestFocusInUnmanagedAllocations|TestUnmanagedPublishAllocations|TestUnmanagedHitAllocations|TestOverrideMovedUnchangedAllocations|TestFullscreenConfigureAllocations|TestKeyboardGrabSyncAllocations|TestSelectionCopyAllocations|TestSelectionINCRChunkAllocations|TestSelectionCopyAllocationsWaylandToX|TestXSettingsNoChangeAllocations|TestSettleLeftAllocs)$$' -count=1
70+ CGO_ENABLED=0 go test ./internal/adapters/outputkit/presented ./internal/core ./internal/app ./internal/adapters/drm ./internal/adapters/headless ./internal/adapters/vulkan ./internal/adapters/wayland ./internal/adapters/outputkit/capture ./internal/adapters/outputkit/routing ./internal/adapters/screencast ./internal/adapters/wayland/xwm -run '^(TestLineageObserverDormantAndAllocations|TestWaitNilFencesAllocations|TestAnimateAllocations|TestPublishAllocations|TestPublishExpansionAllocations|TestPublishSubscribersAllocations|TestPublishStashSlideAllocations|TestPublishOverviewAllocations|TestPublishLeavingAllocations|TestOutputRoutingAllocations|TestReportSeenAllocations|TestFrameLifecycleTransitionsAllocations|TestFullscreenShownAllocations|TestFrameDecisionAllocations|TestFrameDecisionExpansionAllocations|TestOverlayExpansionAllocations|TestCommitFrameAllocations|TestColorPipelineAllocations|TestShownBySnapshotAllocations|TestFlipShowsAllocations|TestMRUNoteFocusDoesNotAllocateWhenSettled|TestDueFramesAllocations|TestRenderSteadyStateAllocations|TestRenderExpansionAllocations|TestRenderSteadyStateDMABufAllocations|TestSceneOpaqueRegionAllocations|TestSceneDeltaAllocations|TestAccountFlipAllocations|TestFlipDoneAllocations|TestSceneWalkUnchangedTiledSHM|TestTiledCommitPublishAllocations|TestCapturedCommitApplyAllocations|TestCapturedViewportApplyAllocations|TestEffectiveInputEmptyTreeAllocations|TestHeadlessTiledSHMCallbackAndCopyBudget|TestReleaseHeldAllocations|TestCommitDamageAllocations|TestEmitContentAllocations|TestPresentationPushAllocations|TestPresentationSnapshotAllocations|TestAnimatingFrameSendAllocations|TestContentBatchCycleAllocations|TestWriteMappedAllocations|TestCapturedForwardAllocations|TestCommandForwardAllocations|TestOutboxQueuedDeliveryAllocations|TestCastFrameAllocations|TestCastProducerAllocations|TestCaptureCycleAllocations|TestCaptureCursorCycleAllocations|TestCaptureCursorIdleAllocations|TestCaptureDMABufCopyAllocations|TestLogScenesAllocations|TestOverviewNamedRowsAllocations|TestOverviewWorkspaceSwitchAllocations|TestBufferContentAllocations|TestDueFramesFireScratch|TestRetireSemaphoreAllocatesNothing|TestTableLookupAllocations|TestTableInsertRemoveAllocations|TestPropertyUnchangedAllocations|TestDispatchAllocations|TestConfigureFlushAllocations|TestFocusAllocations|TestSpaceAllocations|TestUnmanagedGeometryAllocations|TestFocusInUnmanagedAllocations|TestUnmanagedPublishAllocations|TestUnmanagedHitAllocations|TestOverrideMovedUnchangedAllocations|TestFullscreenConfigureAllocations|TestKeyboardGrabSyncAllocations|TestSelectionCopyAllocations|TestSelectionINCRChunkAllocations|TestSelectionCopyAllocationsWaylandToX|TestXSettingsNoChangeAllocations|TestSettleLeftAllocs)$$' -count=1
6971 CGO_ENABLED=0 go test -tags xturbo_poison ./internal/adapters/wayland/xwm -count=1
7072 check: vet test arch fakes-check log-check perf-check
7173
@@ -64,8 +64,10 @@ arch: adapter-imports
64 adapter-imports:64 adapter-imports:
65 CGO_ENABLED=0 go test . -run '^TestAdapterImports$$' -count=165 CGO_ENABLED=0 go test . -run '^TestAdapterImports$$' -count=1
66 # Fast, mandatory guards for tiled content publications and SHM copies.66 # Fast, mandatory guards for tiled content publications and SHM copies.
67+# GPU capture budgets follow the package tests: run on available Vulkan GPUs,
68+# otherwise skip via their existing capability gates; idle budgets need no GPU.
67 perf-check:69 perf-check:
68- CGO_ENABLED=0 go test ./internal/adapters/outputkit/presented ./internal/core ./internal/app ./internal/adapters/drm ./internal/adapters/headless ./internal/adapters/vulkan ./internal/adapters/wayland ./internal/adapters/outputkit/capture ./internal/adapters/outputkit/routing ./internal/adapters/screencast ./internal/adapters/wayland/xwm -run '^(TestLineageObserverDormantAndAllocations|TestWaitNilFencesAllocations|TestAnimateAllocations|TestPublishAllocations|TestPublishExpansionAllocations|TestPublishSubscribersAllocations|TestPublishStashSlideAllocations|TestPublishOverviewAllocations|TestPublishLeavingAllocations|TestOutputRoutingAllocations|TestReportSeenAllocations|TestFrameLifecycleTransitionsAllocations|TestFullscreenShownAllocations|TestFrameDecisionAllocations|TestFrameDecisionExpansionAllocations|TestOverlayExpansionAllocations|TestCommitFrameAllocations|TestColorPipelineAllocations|TestShownBySnapshotAllocations|TestFlipShowsAllocations|TestMRUNoteFocusDoesNotAllocateWhenSettled|TestDueFramesAllocations|TestRenderSteadyStateAllocations|TestRenderExpansionAllocations|TestRenderSteadyStateDMABufAllocations|TestSceneOpaqueRegionAllocations|TestSceneDeltaAllocations|TestAccountFlipAllocations|TestFlipDoneAllocations|TestSceneWalkUnchangedTiledSHM|TestTiledCommitPublishAllocations|TestCapturedCommitApplyAllocations|TestCapturedViewportApplyAllocations|TestEffectiveInputEmptyTreeAllocations|TestHeadlessTiledSHMCallbackAndCopyBudget|TestReleaseHeldAllocations|TestCommitDamageAllocations|TestEmitContentAllocations|TestPresentationPushAllocations|TestPresentationSnapshotAllocations|TestAnimatingFrameSendAllocations|TestContentBatchCycleAllocations|TestWriteMappedAllocations|TestCapturedForwardAllocations|TestCommandForwardAllocations|TestOutboxQueuedDeliveryAllocations|TestCastFrameAllocations|TestCastProducerAllocations|TestCaptureCycleAllocations|TestLogScenesAllocations|TestOverviewNamedRowsAllocations|TestOverviewWorkspaceSwitchAllocations|TestBufferContentAllocations|TestDueFramesFireScratch|TestRetireSemaphoreAllocatesNothing|TestTableLookupAllocations|TestTableInsertRemoveAllocations|TestPropertyUnchangedAllocations|TestDispatchAllocations|TestConfigureFlushAllocations|TestFocusAllocations|TestSpaceAllocations|TestUnmanagedGeometryAllocations|TestFocusInUnmanagedAllocations|TestUnmanagedPublishAllocations|TestUnmanagedHitAllocations|TestOverrideMovedUnchangedAllocations|TestFullscreenConfigureAllocations|TestKeyboardGrabSyncAllocations|TestSelectionCopyAllocations|TestSelectionINCRChunkAllocations|TestSelectionCopyAllocationsWaylandToX|TestXSettingsNoChangeAllocations|TestSettleLeftAllocs)$$' -count=170+ CGO_ENABLED=0 go test ./internal/adapters/outputkit/presented ./internal/core ./internal/app ./internal/adapters/drm ./internal/adapters/headless ./internal/adapters/vulkan ./internal/adapters/wayland ./internal/adapters/outputkit/capture ./internal/adapters/outputkit/routing ./internal/adapters/screencast ./internal/adapters/wayland/xwm -run '^(TestLineageObserverDormantAndAllocations|TestWaitNilFencesAllocations|TestAnimateAllocations|TestPublishAllocations|TestPublishExpansionAllocations|TestPublishSubscribersAllocations|TestPublishStashSlideAllocations|TestPublishOverviewAllocations|TestPublishLeavingAllocations|TestOutputRoutingAllocations|TestReportSeenAllocations|TestFrameLifecycleTransitionsAllocations|TestFullscreenShownAllocations|TestFrameDecisionAllocations|TestFrameDecisionExpansionAllocations|TestOverlayExpansionAllocations|TestCommitFrameAllocations|TestColorPipelineAllocations|TestShownBySnapshotAllocations|TestFlipShowsAllocations|TestMRUNoteFocusDoesNotAllocateWhenSettled|TestDueFramesAllocations|TestRenderSteadyStateAllocations|TestRenderExpansionAllocations|TestRenderSteadyStateDMABufAllocations|TestSceneOpaqueRegionAllocations|TestSceneDeltaAllocations|TestAccountFlipAllocations|TestFlipDoneAllocations|TestSceneWalkUnchangedTiledSHM|TestTiledCommitPublishAllocations|TestCapturedCommitApplyAllocations|TestCapturedViewportApplyAllocations|TestEffectiveInputEmptyTreeAllocations|TestHeadlessTiledSHMCallbackAndCopyBudget|TestReleaseHeldAllocations|TestCommitDamageAllocations|TestEmitContentAllocations|TestPresentationPushAllocations|TestPresentationSnapshotAllocations|TestAnimatingFrameSendAllocations|TestContentBatchCycleAllocations|TestWriteMappedAllocations|TestCapturedForwardAllocations|TestCommandForwardAllocations|TestOutboxQueuedDeliveryAllocations|TestCastFrameAllocations|TestCastProducerAllocations|TestCaptureCycleAllocations|TestCaptureCursorCycleAllocations|TestCaptureCursorIdleAllocations|TestCaptureDMABufCopyAllocations|TestLogScenesAllocations|TestOverviewNamedRowsAllocations|TestOverviewWorkspaceSwitchAllocations|TestBufferContentAllocations|TestDueFramesFireScratch|TestRetireSemaphoreAllocatesNothing|TestTableLookupAllocations|TestTableInsertRemoveAllocations|TestPropertyUnchangedAllocations|TestDispatchAllocations|TestConfigureFlushAllocations|TestFocusAllocations|TestSpaceAllocations|TestUnmanagedGeometryAllocations|TestFocusInUnmanagedAllocations|TestUnmanagedPublishAllocations|TestUnmanagedHitAllocations|TestOverrideMovedUnchangedAllocations|TestFullscreenConfigureAllocations|TestKeyboardGrabSyncAllocations|TestSelectionCopyAllocations|TestSelectionINCRChunkAllocations|TestSelectionCopyAllocationsWaylandToX|TestXSettingsNoChangeAllocations|TestSettleLeftAllocs)$$' -count=1
69 CGO_ENABLED=0 go test -tags xturbo_poison ./internal/adapters/wayland/xwm -count=171 CGO_ENABLED=0 go test -tags xturbo_poison ./internal/adapters/wayland/xwm -count=1
70 check: vet test arch fakes-check log-check perf-check72 check: vet test arch fakes-check log-check perf-check
71 73
M internal/adapters/drm/capture_cursor_test.go +52 -3
@@ -6,21 +6,70 @@ import (
66 )
77
88 func TestCaptureCursorSnapshot(t *testing.T) {
9+ var storage domain.CaptureCursor
910 c := newCursor(nil, 64)
1011 c.image = true
1112 c.captureImage = domain.CursorImage{W: 1, H: 1, HotX: 2, Pixels: []byte{1, 2, 3, 255}}
1213 c.Move(10, 20)
13- s := c.snapshot()
14+ s := c.snapshot(&storage)
1415 if s == nil || s.X != 10 || s.Y != 20 || s.Image.HotX != 2 || s.Image.Pixels[2] != 3 {
1516 t.Fatalf("snapshot: %+v", s)
1617 }
1718 c.Hide()
18- if c.snapshot() != nil {
19+ if c.snapshot(&storage) != nil {
1920 t.Fatal("away cursor captured")
2021 }
2122 c.Move(30, 40)
2223 c.image = false
23- if c.snapshot() != nil {
24+ if c.snapshot(&storage) != nil {
2425 t.Fatal("security-invalidated cursor captured")
2526 }
2627 }
28+
29+func TestCaptureCursorIdleAllocations(t *testing.T) {
30+ c := newCursor(nil, 64)
31+ c.image = true
32+ c.captureImage = domain.CursorImage{W: 1, H: 1, Pixels: []byte{0, 0, 255, 255}}
33+ var storage domain.CaptureCursor
34+ noCursor := []domain.CaptureRequest{{Cursor: false}}
35+ for _, tc := range []struct {
36+ name string
37+ normal, excluded []domain.CaptureRequest
38+ }{
39+ {name: "idle"}, {name: "cursor-free", normal: noCursor, excluded: noCursor},
40+ } {
41+ t.Run(tc.name, func(t *testing.T) {
42+ storage.X = -123
43+ // Holding the motion lock proves that the idle path does not snapshot.
44+ c.mu.Lock()
45+ cycle := func() {
46+ if got := c.captureCursor(&storage, tc.normal, tc.excluded); got != nil {
47+ t.Fatal("unexpected cursor")
48+ }
49+ }
50+ allocs := testing.AllocsPerRun(100, cycle)
51+ c.mu.Unlock()
52+ if allocs != 0 || storage.X != -123 {
53+ t.Fatalf("allocs=%v storage=%+v", allocs, storage)
54+ }
55+ result := testing.Benchmark(func(b *testing.B) {
56+ for b.Loop() {
57+ cycle()
58+ }
59+ })
60+ t.Logf("%s %s", result.String(), result.MemString())
61+ })
62+ }
63+ for _, excluded := range []bool{false, true} {
64+ requests := []domain.CaptureRequest{{Cursor: true}}
65+ var normal, scoped []domain.CaptureRequest
66+ if excluded {
67+ scoped = requests
68+ } else {
69+ normal = requests
70+ }
71+ if got := c.captureCursor(&storage, normal, scoped); got != &storage || got.Image.W != 1 {
72+ t.Fatalf("owner snapshot: %+v", got)
73+ }
74+ }
75+}
@@ -6,21 +6,70 @@ import (
6 )6 )
7 7
8 func TestCaptureCursorSnapshot(t *testing.T) {8 func TestCaptureCursorSnapshot(t *testing.T) {
9+ var storage domain.CaptureCursor
9 c := newCursor(nil, 64)10 c := newCursor(nil, 64)
10 c.image = true11 c.image = true
11 c.captureImage = domain.CursorImage{W: 1, H: 1, HotX: 2, Pixels: []byte{1, 2, 3, 255}}12 c.captureImage = domain.CursorImage{W: 1, H: 1, HotX: 2, Pixels: []byte{1, 2, 3, 255}}
12 c.Move(10, 20)13 c.Move(10, 20)
13- s := c.snapshot()14+ s := c.snapshot(&storage)
14 if s == nil || s.X != 10 || s.Y != 20 || s.Image.HotX != 2 || s.Image.Pixels[2] != 3 {15 if s == nil || s.X != 10 || s.Y != 20 || s.Image.HotX != 2 || s.Image.Pixels[2] != 3 {
15 t.Fatalf("snapshot: %+v", s)16 t.Fatalf("snapshot: %+v", s)
16 }17 }
17 c.Hide()18 c.Hide()
18- if c.snapshot() != nil {19+ if c.snapshot(&storage) != nil {
19 t.Fatal("away cursor captured")20 t.Fatal("away cursor captured")
20 }21 }
21 c.Move(30, 40)22 c.Move(30, 40)
22 c.image = false23 c.image = false
23- if c.snapshot() != nil {24+ if c.snapshot(&storage) != nil {
24 t.Fatal("security-invalidated cursor captured")25 t.Fatal("security-invalidated cursor captured")
25 }26 }
26 }27 }
28+
29+func TestCaptureCursorIdleAllocations(t *testing.T) {
30+ c := newCursor(nil, 64)
31+ c.image = true
32+ c.captureImage = domain.CursorImage{W: 1, H: 1, Pixels: []byte{0, 0, 255, 255}}
33+ var storage domain.CaptureCursor
34+ noCursor := []domain.CaptureRequest{{Cursor: false}}
35+ for _, tc := range []struct {
36+ name string
37+ normal, excluded []domain.CaptureRequest
38+ }{
39+ {name: "idle"}, {name: "cursor-free", normal: noCursor, excluded: noCursor},
40+ } {
41+ t.Run(tc.name, func(t *testing.T) {
42+ storage.X = -123
43+ // Holding the motion lock proves that the idle path does not snapshot.
44+ c.mu.Lock()
45+ cycle := func() {
46+ if got := c.captureCursor(&storage, tc.normal, tc.excluded); got != nil {
47+ t.Fatal("unexpected cursor")
48+ }
49+ }
50+ allocs := testing.AllocsPerRun(100, cycle)
51+ c.mu.Unlock()
52+ if allocs != 0 || storage.X != -123 {
53+ t.Fatalf("allocs=%v storage=%+v", allocs, storage)
54+ }
55+ result := testing.Benchmark(func(b *testing.B) {
56+ for b.Loop() {
57+ cycle()
58+ }
59+ })
60+ t.Logf("%s %s", result.String(), result.MemString())
61+ })
62+ }
63+ for _, excluded := range []bool{false, true} {
64+ requests := []domain.CaptureRequest{{Cursor: true}}
65+ var normal, scoped []domain.CaptureRequest
66+ if excluded {
67+ scoped = requests
68+ } else {
69+ normal = requests
70+ }
71+ if got := c.captureCursor(&storage, normal, scoped); got != &storage || got.Image.W != 1 {
72+ t.Fatalf("owner snapshot: %+v", got)
73+ }
74+ }
75+}
M internal/adapters/drm/cursor.go +25 -3
@@ -96,14 +96,15 @@ func (c *Cursor) TakeStats() CursorStats {
9696
9797 // snapshot samples physical hotspot motion under its lock; the SDR image is
9898 // owned by this output goroutine, not by the input goroutine or KMS plane.
99-func (c *Cursor) snapshot() *domain.CaptureCursor {
99+func (c *Cursor) snapshot(dst *domain.CaptureCursor) *domain.CaptureCursor {
100100 c.mu.Lock()
101101 x, y, away := c.x, c.y, c.away
102102 c.mu.Unlock()
103103 if away || !c.image || c.captureImage.W == 0 || c.captureImage.H == 0 {
104104 return nil
105105 }
106- return &domain.CaptureCursor{Image: c.captureImage, X: x, Y: y}
106+ *dst = domain.CaptureCursor{Image: c.captureImage, X: x, Y: y}
107+ return dst
107108 }
108109
109110 // Limit is the largest image side the cursor plane accepts.
@@ -197,7 +198,10 @@ func (c *Cursor) useFallback(hdrNits float64) (bool, error) {
197198 func (c *Cursor) setImage(pixels []byte, w, h, hotX, hotY int, hdrNits float64) error {
198199 c.captureImage = domain.CursorImage{}
199200 if w > 0 && h > 0 && len(pixels) >= w*h*4 {
200- c.captureImage = domain.CursorImage{W: w, H: h, HotX: hotX, HotY: hotY, Pixels: slices.Clone(pixels[:w*h*4])}
201+ // Loader images are immutable (also shared by the headless cursor).
202+ // Prefer our replay/queued copy below when one is required for KMS.
203+ // Snapshots are uploaded synchronously before the owner changes images.
204+ c.captureImage = domain.CursorImage{W: w, H: h, HotX: hotX, HotY: hotY, Pixels: pixels[:w*h*4]}
201205 }
202206 c.later = nil
203207 if w <= 0 || h <= 0 || len(pixels) < w*h*4 {
@@ -211,6 +215,7 @@ func (c *Cursor) setImage(pixels []byte, w, h, hotX, hotY int, hdrNits float64)
211215 next := c.freeSlot()
212216 if next < 0 {
213217 c.later = &laterImage{pixels: slices.Clone(pixels[:w*h*4]), w: w, h: h, hotX: hotX, hotY: hotY}
218+ c.captureImage.Pixels = c.later.pixels
214219 return nil
215220 }
216221 if err := c.imgs.write(next, pixels, w, h, hdrNits); err != nil {
@@ -219,6 +224,7 @@ func (c *Cursor) setImage(pixels []byte, w, h, hotX, hotY int, hdrNits float64)
219224 c.cur, c.image, c.hotX, c.hotY = next, true, hotX, hotY
220225 if c.alloc == allocDumb && !c.off {
221226 c.last = laterImage{pixels: append(c.last.pixels[:0], pixels[:w*h*4]...), w: w, h: h, hotX: hotX, hotY: hotY}
227+ c.captureImage.Pixels = c.last.pixels
222228 }
223229 return nil
224230 }
@@ -308,3 +314,19 @@ func (c *Cursor) free() {
308314 c.last = laterImage{}
309315 c.screen, c.flying = 0, false
310316 }
317+
318+// captureCursor samples only admitted displayed-frame requests. Hidden children
319+// have no output cursor. dst belongs to the output owner, not to a frame lease.
320+func (c *Cursor) captureCursor(dst *domain.CaptureCursor, normal, excluded []domain.CaptureRequest) *domain.CaptureCursor {
321+ if c == nil {
322+ return nil
323+ }
324+ for _, group := range [2][]domain.CaptureRequest{normal, excluded} {
325+ for _, q := range group {
326+ if q.Cursor {
327+ return c.snapshot(dst)
328+ }
329+ }
330+ }
331+ return nil
332+}
@@ -96,14 +96,15 @@ func (c *Cursor) TakeStats() CursorStats {
96 96
97 // snapshot samples physical hotspot motion under its lock; the SDR image is97 // snapshot samples physical hotspot motion under its lock; the SDR image is
98 // owned by this output goroutine, not by the input goroutine or KMS plane.98 // owned by this output goroutine, not by the input goroutine or KMS plane.
99-func (c *Cursor) snapshot() *domain.CaptureCursor {99+func (c *Cursor) snapshot(dst *domain.CaptureCursor) *domain.CaptureCursor {
100 c.mu.Lock()100 c.mu.Lock()
101 x, y, away := c.x, c.y, c.away101 x, y, away := c.x, c.y, c.away
102 c.mu.Unlock()102 c.mu.Unlock()
103 if away || !c.image || c.captureImage.W == 0 || c.captureImage.H == 0 {103 if away || !c.image || c.captureImage.W == 0 || c.captureImage.H == 0 {
104 return nil104 return nil
105 }105 }
106- return &domain.CaptureCursor{Image: c.captureImage, X: x, Y: y}106+ *dst = domain.CaptureCursor{Image: c.captureImage, X: x, Y: y}
107+ return dst
107 }108 }
108 109
109 // Limit is the largest image side the cursor plane accepts.110 // Limit is the largest image side the cursor plane accepts.
@@ -197,7 +198,10 @@ func (c *Cursor) useFallback(hdrNits float64) (bool, error) {
197 func (c *Cursor) setImage(pixels []byte, w, h, hotX, hotY int, hdrNits float64) error {198 func (c *Cursor) setImage(pixels []byte, w, h, hotX, hotY int, hdrNits float64) error {
198 c.captureImage = domain.CursorImage{}199 c.captureImage = domain.CursorImage{}
199 if w > 0 && h > 0 && len(pixels) >= w*h*4 {200 if w > 0 && h > 0 && len(pixels) >= w*h*4 {
200- c.captureImage = domain.CursorImage{W: w, H: h, HotX: hotX, HotY: hotY, Pixels: slices.Clone(pixels[:w*h*4])}201+ // Loader images are immutable (also shared by the headless cursor).
202+ // Prefer our replay/queued copy below when one is required for KMS.
203+ // Snapshots are uploaded synchronously before the owner changes images.
204+ c.captureImage = domain.CursorImage{W: w, H: h, HotX: hotX, HotY: hotY, Pixels: pixels[:w*h*4]}
201 }205 }
202 c.later = nil206 c.later = nil
203 if w <= 0 || h <= 0 || len(pixels) < w*h*4 {207 if w <= 0 || h <= 0 || len(pixels) < w*h*4 {
@@ -211,6 +215,7 @@ func (c *Cursor) setImage(pixels []byte, w, h, hotX, hotY int, hdrNits float64)
211 next := c.freeSlot()215 next := c.freeSlot()
212 if next < 0 {216 if next < 0 {
213 c.later = &laterImage{pixels: slices.Clone(pixels[:w*h*4]), w: w, h: h, hotX: hotX, hotY: hotY}217 c.later = &laterImage{pixels: slices.Clone(pixels[:w*h*4]), w: w, h: h, hotX: hotX, hotY: hotY}
218+ c.captureImage.Pixels = c.later.pixels
214 return nil219 return nil
215 }220 }
216 if err := c.imgs.write(next, pixels, w, h, hdrNits); err != nil {221 if err := c.imgs.write(next, pixels, w, h, hdrNits); err != nil {
@@ -219,6 +224,7 @@ func (c *Cursor) setImage(pixels []byte, w, h, hotX, hotY int, hdrNits float64)
219 c.cur, c.image, c.hotX, c.hotY = next, true, hotX, hotY224 c.cur, c.image, c.hotX, c.hotY = next, true, hotX, hotY
220 if c.alloc == allocDumb && !c.off {225 if c.alloc == allocDumb && !c.off {
221 c.last = laterImage{pixels: append(c.last.pixels[:0], pixels[:w*h*4]...), w: w, h: h, hotX: hotX, hotY: hotY}226 c.last = laterImage{pixels: append(c.last.pixels[:0], pixels[:w*h*4]...), w: w, h: h, hotX: hotX, hotY: hotY}
227+ c.captureImage.Pixels = c.last.pixels
222 }228 }
223 return nil229 return nil
224 }230 }
@@ -308,3 +314,19 @@ func (c *Cursor) free() {
308 c.last = laterImage{}314 c.last = laterImage{}
309 c.screen, c.flying = 0, false315 c.screen, c.flying = 0, false
310 }316 }
317+
318+// captureCursor samples only admitted displayed-frame requests. Hidden children
319+// have no output cursor. dst belongs to the output owner, not to a frame lease.
320+func (c *Cursor) captureCursor(dst *domain.CaptureCursor, normal, excluded []domain.CaptureRequest) *domain.CaptureCursor {
321+ if c == nil {
322+ return nil
323+ }
324+ for _, group := range [2][]domain.CaptureRequest{normal, excluded} {
325+ for _, q := range group {
326+ if q.Cursor {
327+ return c.snapshot(dst)
328+ }
329+ }
330+ }
331+ return nil
332+}
M internal/adapters/drm/frame_submit.go +3 -3
@@ -44,9 +44,6 @@ func (o *Output) submitFrame(ctx context.Context, r ports.Renderer, scene domain
4444 // (the caller keeps it, see capture.Waiting).
4545 if pipeline != nil {
4646 pipeline.Cursor = nil
47- if o.cursor != nil {
48- pipeline.Cursor = o.cursor.snapshot()
49- }
5047 }
5148 ready, _ := pipeline.Hold(scene, requests, time.Now())
5249 if len(scene.CaptureIndicators) > 0 {
@@ -62,6 +59,9 @@ func (o *Output) submitFrame(ctx context.Context, r ports.Renderer, scene domain
6259 }()
6360 }
6461 normal, excluded, hidden := pipeline.Split(scene, ready)
62+ if pipeline != nil {
63+ pipeline.Cursor = o.cursor.captureCursor(&o.captureCursor, normal, excluded)
64+ }
6565 pipeline.Retire(scene)
6666 if len(hidden) > 0 {
6767 if !o.sceneCurrent(scene) || o.protected {
@@ -44,9 +44,6 @@ func (o *Output) submitFrame(ctx context.Context, r ports.Renderer, scene domain
44 // (the caller keeps it, see capture.Waiting).44 // (the caller keeps it, see capture.Waiting).
45 if pipeline != nil {45 if pipeline != nil {
46 pipeline.Cursor = nil46 pipeline.Cursor = nil
47- if o.cursor != nil {
48- pipeline.Cursor = o.cursor.snapshot()
49- }
50 }47 }
51 ready, _ := pipeline.Hold(scene, requests, time.Now())48 ready, _ := pipeline.Hold(scene, requests, time.Now())
52 if len(scene.CaptureIndicators) > 0 {49 if len(scene.CaptureIndicators) > 0 {
@@ -62,6 +59,9 @@ func (o *Output) submitFrame(ctx context.Context, r ports.Renderer, scene domain
62 }()59 }()
63 }60 }
64 normal, excluded, hidden := pipeline.Split(scene, ready)61 normal, excluded, hidden := pipeline.Split(scene, ready)
62+ if pipeline != nil {
63+ pipeline.Cursor = o.cursor.captureCursor(&o.captureCursor, normal, excluded)
64+ }
65 pipeline.Retire(scene)65 pipeline.Retire(scene)
66 if len(hidden) > 0 {66 if len(hidden) > 0 {
67 if !o.sceneCurrent(scene) || o.protected {67 if !o.sceneCurrent(scene) || o.protected {
M internal/adapters/drm/output.go +4 -3
@@ -71,9 +71,10 @@ type Output struct {
7171 // primary shows frames; cursor is the cursor plane (nil: none);
7272 // stray are other planes that can show on this CRTC, turned off at
7373 // every modeset when a previous master left them on it.
74- primary *plane
75- cursor *Cursor
76- stray []*plane
74+ primary *plane
75+ cursor *Cursor
76+ captureCursor domain.CaptureCursor
77+ stray []*plane
7778 // overlay shows one window's buffer above the composed frame
7879 // (overlay.go); overlayOn is the buffer on it in the pending or last
7980 // frame (0: off), overlayReason why it is off.
@@ -71,9 +71,10 @@ type Output struct {
71 // primary shows frames; cursor is the cursor plane (nil: none);71 // primary shows frames; cursor is the cursor plane (nil: none);
72 // stray are other planes that can show on this CRTC, turned off at72 // stray are other planes that can show on this CRTC, turned off at
73 // every modeset when a previous master left them on it.73 // every modeset when a previous master left them on it.
74- primary *plane74+ primary *plane
75- cursor *Cursor75+ cursor *Cursor
76- stray []*plane76+ captureCursor domain.CaptureCursor
77+ stray []*plane
77 // overlay shows one window's buffer above the composed frame78 // overlay shows one window's buffer above the composed frame
78 // (overlay.go); overlayOn is the buffer on it in the pending or last79 // (overlay.go); overlayOn is the buffer on it in the pending or last
79 // frame (0: off), overlayReason why it is off.80 // frame (0: off), overlayReason why it is off.
M internal/adapters/headless/cursor.go +19 -2
@@ -38,13 +38,14 @@ func (c *Cursor) set(img domain.CursorImage) {
3838
3939 // snapshot is called by the output owner; set replaces rather than mutates
4040 // image pixels, so the returned image stays immutable through the GPU upload.
41-func (c *Cursor) snapshot() *domain.CaptureCursor {
41+func (c *Cursor) snapshot(dst *domain.CaptureCursor) *domain.CaptureCursor {
4242 c.mu.Lock()
4343 defer c.mu.Unlock()
4444 if c.hidden || c.img.W == 0 || c.img.H == 0 {
4545 return nil
4646 }
47- return &domain.CaptureCursor{Image: c.img, X: int(c.x), Y: int(c.y)}
47+ *dst = domain.CaptureCursor{Image: c.img, X: int(c.x), Y: int(c.y)}
48+ return dst
4849 }
4950
5051 // draw blends the premultiplied cursor over dst with its hotspot at (x, y).
@@ -81,3 +82,19 @@ func (c *Cursor) drawSecure(dst *image.RGBA, security ports.SessionSecurity) {
8182 }
8283 }
8384 }
85+
86+// captureCursor samples only admitted displayed-frame requests. Hidden children
87+// have no output cursor. dst belongs to the output owner, not to a frame lease.
88+func (c *Cursor) captureCursor(dst *domain.CaptureCursor, normal, excluded []domain.CaptureRequest) *domain.CaptureCursor {
89+ if c == nil {
90+ return nil
91+ }
92+ for _, group := range [2][]domain.CaptureRequest{normal, excluded} {
93+ for _, q := range group {
94+ if q.Cursor {
95+ return c.snapshot(dst)
96+ }
97+ }
98+ }
99+ return nil
100+}
@@ -38,13 +38,14 @@ func (c *Cursor) set(img domain.CursorImage) {
38 38
39 // snapshot is called by the output owner; set replaces rather than mutates39 // snapshot is called by the output owner; set replaces rather than mutates
40 // image pixels, so the returned image stays immutable through the GPU upload.40 // image pixels, so the returned image stays immutable through the GPU upload.
41-func (c *Cursor) snapshot() *domain.CaptureCursor {41+func (c *Cursor) snapshot(dst *domain.CaptureCursor) *domain.CaptureCursor {
42 c.mu.Lock()42 c.mu.Lock()
43 defer c.mu.Unlock()43 defer c.mu.Unlock()
44 if c.hidden || c.img.W == 0 || c.img.H == 0 {44 if c.hidden || c.img.W == 0 || c.img.H == 0 {
45 return nil45 return nil
46 }46 }
47- return &domain.CaptureCursor{Image: c.img, X: int(c.x), Y: int(c.y)}47+ *dst = domain.CaptureCursor{Image: c.img, X: int(c.x), Y: int(c.y)}
48+ return dst
48 }49 }
49 50
50 // draw blends the premultiplied cursor over dst with its hotspot at (x, y).51 // draw blends the premultiplied cursor over dst with its hotspot at (x, y).
@@ -81,3 +82,19 @@ func (c *Cursor) drawSecure(dst *image.RGBA, security ports.SessionSecurity) {
81 }82 }
82 }83 }
83 }84 }
85+
86+// captureCursor samples only admitted displayed-frame requests. Hidden children
87+// have no output cursor. dst belongs to the output owner, not to a frame lease.
88+func (c *Cursor) captureCursor(dst *domain.CaptureCursor, normal, excluded []domain.CaptureRequest) *domain.CaptureCursor {
89+ if c == nil {
90+ return nil
91+ }
92+ for _, group := range [2][]domain.CaptureRequest{normal, excluded} {
93+ for _, q := range group {
94+ if q.Cursor {
95+ return c.snapshot(dst)
96+ }
97+ }
98+ }
99+ return nil
100+}
M internal/adapters/headless/cursor_test.go +53 -5
@@ -31,26 +31,74 @@ func TestCursorDraw(t *testing.T) {
3131 }
3232
3333 func TestCaptureCursorSnapshotVisibility(t *testing.T) {
34+ var storage domain.CaptureCursor
3435 c := &Cursor{}
35- if c.snapshot() != nil {
36+ if c.snapshot(&storage) != nil {
3637 t.Fatal("empty cursor visible")
3738 }
3839 c.set(domain.CursorImage{W: 1, H: 1, HotX: 1, Pixels: []byte{0, 0, 255, 255}})
3940 c.Move(12, 34)
40- got := c.snapshot()
41+ got := c.snapshot(&storage)
4142 if got == nil || got.X != 12 || got.Y != 34 || got.Image.HotX != 1 {
4243 t.Fatalf("snapshot: %+v", got)
4344 }
4445 c.Hide()
45- if c.snapshot() != nil {
46+ if c.snapshot(&storage) != nil {
4647 t.Fatal("hidden cursor visible")
4748 }
4849 c.Move(20, 30)
49- if c.snapshot() == nil {
50+ if c.snapshot(&storage) == nil {
5051 t.Fatal("motion did not show cursor")
5152 }
5253 c.set(domain.CursorImage{})
53- if c.snapshot() != nil {
54+ if c.snapshot(&storage) != nil {
5455 t.Fatal("client-hidden cursor visible")
5556 }
5657 }
58+
59+func TestCaptureCursorIdleAllocations(t *testing.T) {
60+ c := &Cursor{}
61+ c.set(domain.CursorImage{W: 1, H: 1, Pixels: []byte{0, 0, 255, 255}})
62+ var storage domain.CaptureCursor
63+ noCursor := []domain.CaptureRequest{{Cursor: false}}
64+ for _, tc := range []struct {
65+ name string
66+ normal, excluded []domain.CaptureRequest
67+ }{
68+ {name: "idle"}, {name: "cursor-free", normal: noCursor, excluded: noCursor},
69+ } {
70+ t.Run(tc.name, func(t *testing.T) {
71+ storage.X = -123
72+ // Holding the motion lock proves that the idle path does not snapshot.
73+ c.mu.Lock()
74+ cycle := func() {
75+ if got := c.captureCursor(&storage, tc.normal, tc.excluded); got != nil {
76+ t.Fatal("unexpected cursor")
77+ }
78+ }
79+ allocs := testing.AllocsPerRun(100, cycle)
80+ c.mu.Unlock()
81+ if allocs != 0 || storage.X != -123 {
82+ t.Fatalf("allocs=%v storage=%+v", allocs, storage)
83+ }
84+ result := testing.Benchmark(func(b *testing.B) {
85+ for b.Loop() {
86+ cycle()
87+ }
88+ })
89+ t.Logf("%s %s", result.String(), result.MemString())
90+ })
91+ }
92+ for _, excluded := range []bool{false, true} {
93+ requests := []domain.CaptureRequest{{Cursor: true}}
94+ var normal, scoped []domain.CaptureRequest
95+ if excluded {
96+ scoped = requests
97+ } else {
98+ normal = requests
99+ }
100+ if got := c.captureCursor(&storage, normal, scoped); got != &storage || got.Image.W != 1 {
101+ t.Fatalf("owner snapshot: %+v", got)
102+ }
103+ }
104+}
@@ -31,26 +31,74 @@ func TestCursorDraw(t *testing.T) {
31 }31 }
32 32
33 func TestCaptureCursorSnapshotVisibility(t *testing.T) {33 func TestCaptureCursorSnapshotVisibility(t *testing.T) {
34+ var storage domain.CaptureCursor
34 c := &Cursor{}35 c := &Cursor{}
35- if c.snapshot() != nil {36+ if c.snapshot(&storage) != nil {
36 t.Fatal("empty cursor visible")37 t.Fatal("empty cursor visible")
37 }38 }
38 c.set(domain.CursorImage{W: 1, H: 1, HotX: 1, Pixels: []byte{0, 0, 255, 255}})39 c.set(domain.CursorImage{W: 1, H: 1, HotX: 1, Pixels: []byte{0, 0, 255, 255}})
39 c.Move(12, 34)40 c.Move(12, 34)
40- got := c.snapshot()41+ got := c.snapshot(&storage)
41 if got == nil || got.X != 12 || got.Y != 34 || got.Image.HotX != 1 {42 if got == nil || got.X != 12 || got.Y != 34 || got.Image.HotX != 1 {
42 t.Fatalf("snapshot: %+v", got)43 t.Fatalf("snapshot: %+v", got)
43 }44 }
44 c.Hide()45 c.Hide()
45- if c.snapshot() != nil {46+ if c.snapshot(&storage) != nil {
46 t.Fatal("hidden cursor visible")47 t.Fatal("hidden cursor visible")
47 }48 }
48 c.Move(20, 30)49 c.Move(20, 30)
49- if c.snapshot() == nil {50+ if c.snapshot(&storage) == nil {
50 t.Fatal("motion did not show cursor")51 t.Fatal("motion did not show cursor")
51 }52 }
52 c.set(domain.CursorImage{})53 c.set(domain.CursorImage{})
53- if c.snapshot() != nil {54+ if c.snapshot(&storage) != nil {
54 t.Fatal("client-hidden cursor visible")55 t.Fatal("client-hidden cursor visible")
55 }56 }
56 }57 }
58+
59+func TestCaptureCursorIdleAllocations(t *testing.T) {
60+ c := &Cursor{}
61+ c.set(domain.CursorImage{W: 1, H: 1, Pixels: []byte{0, 0, 255, 255}})
62+ var storage domain.CaptureCursor
63+ noCursor := []domain.CaptureRequest{{Cursor: false}}
64+ for _, tc := range []struct {
65+ name string
66+ normal, excluded []domain.CaptureRequest
67+ }{
68+ {name: "idle"}, {name: "cursor-free", normal: noCursor, excluded: noCursor},
69+ } {
70+ t.Run(tc.name, func(t *testing.T) {
71+ storage.X = -123
72+ // Holding the motion lock proves that the idle path does not snapshot.
73+ c.mu.Lock()
74+ cycle := func() {
75+ if got := c.captureCursor(&storage, tc.normal, tc.excluded); got != nil {
76+ t.Fatal("unexpected cursor")
77+ }
78+ }
79+ allocs := testing.AllocsPerRun(100, cycle)
80+ c.mu.Unlock()
81+ if allocs != 0 || storage.X != -123 {
82+ t.Fatalf("allocs=%v storage=%+v", allocs, storage)
83+ }
84+ result := testing.Benchmark(func(b *testing.B) {
85+ for b.Loop() {
86+ cycle()
87+ }
88+ })
89+ t.Logf("%s %s", result.String(), result.MemString())
90+ })
91+ }
92+ for _, excluded := range []bool{false, true} {
93+ requests := []domain.CaptureRequest{{Cursor: true}}
94+ var normal, scoped []domain.CaptureRequest
95+ if excluded {
96+ scoped = requests
97+ } else {
98+ normal = requests
99+ }
100+ if got := c.captureCursor(&storage, normal, scoped); got != &storage || got.Image.W != 1 {
101+ t.Fatalf("owner snapshot: %+v", got)
102+ }
103+ }
104+}
M internal/adapters/headless/output.go +4 -3
@@ -219,6 +219,7 @@ func RunOutput(ctx context.Context, opts Options, scenes <-chan domain.Scene, co
219219 ctx, cancelCaptures := context.WithCancel(ctx)
220220 requests.Init(ctx, opts.Captured)
221221 pipeline := capture.NewPipeline(ctx, opts.Captured)
222+ var captureCursor domain.CaptureCursor
222223 pipeline.Security = opts.Security
223224 if opts.NewCaptureRenderer != nil {
224225 pipeline.EnableOffscreen(opts.NewCaptureRenderer)
@@ -682,9 +683,6 @@ func RunOutput(ctx context.Context, opts Options, scenes <-chan domain.Scene, co
682683 // workspace that moved) are failed here; see capture.Pipeline.Split.
683684 // A request whose indicator the scene does not show yet stays held.
684685 pipeline.Cursor = nil
685- if opts.Cursor != nil && !opts.protected() {
686- pipeline.Cursor = opts.Cursor.snapshot()
687- }
688686 ready, _ := pipeline.Hold(scene, requests.List, time.Now())
689687 if len(scene.CaptureIndicators) > 0 {
690688 // The captures of this frame are delivered only once its frame,
@@ -692,6 +690,9 @@ func RunOutput(ctx context.Context, opts Options, scenes <-chan domain.Scene, co
692690 pipeline.BeginGate()
693691 }
694692 normal, excluded, hidden := pipeline.Split(scene, ready)
693+ if !opts.protected() {
694+ pipeline.Cursor = opts.Cursor.captureCursor(&captureCursor, normal, excluded)
695+ }
695696 pipeline.Retire(scene)
696697 if len(hidden) > 0 {
697698 // A hidden workspace: a child renderer draws it. The display's
@@ -219,6 +219,7 @@ func RunOutput(ctx context.Context, opts Options, scenes <-chan domain.Scene, co
219 ctx, cancelCaptures := context.WithCancel(ctx)219 ctx, cancelCaptures := context.WithCancel(ctx)
220 requests.Init(ctx, opts.Captured)220 requests.Init(ctx, opts.Captured)
221 pipeline := capture.NewPipeline(ctx, opts.Captured)221 pipeline := capture.NewPipeline(ctx, opts.Captured)
222+ var captureCursor domain.CaptureCursor
222 pipeline.Security = opts.Security223 pipeline.Security = opts.Security
223 if opts.NewCaptureRenderer != nil {224 if opts.NewCaptureRenderer != nil {
224 pipeline.EnableOffscreen(opts.NewCaptureRenderer)225 pipeline.EnableOffscreen(opts.NewCaptureRenderer)
@@ -682,9 +683,6 @@ func RunOutput(ctx context.Context, opts Options, scenes <-chan domain.Scene, co
682 // workspace that moved) are failed here; see capture.Pipeline.Split.683 // workspace that moved) are failed here; see capture.Pipeline.Split.
683 // A request whose indicator the scene does not show yet stays held.684 // A request whose indicator the scene does not show yet stays held.
684 pipeline.Cursor = nil685 pipeline.Cursor = nil
685- if opts.Cursor != nil && !opts.protected() {
686- pipeline.Cursor = opts.Cursor.snapshot()
687- }
688 ready, _ := pipeline.Hold(scene, requests.List, time.Now())686 ready, _ := pipeline.Hold(scene, requests.List, time.Now())
689 if len(scene.CaptureIndicators) > 0 {687 if len(scene.CaptureIndicators) > 0 {
690 // The captures of this frame are delivered only once its frame,688 // The captures of this frame are delivered only once its frame,
@@ -692,6 +690,9 @@ func RunOutput(ctx context.Context, opts Options, scenes <-chan domain.Scene, co
692 pipeline.BeginGate()690 pipeline.BeginGate()
693 }691 }
694 normal, excluded, hidden := pipeline.Split(scene, ready)692 normal, excluded, hidden := pipeline.Split(scene, ready)
693+ if !opts.protected() {
694+ pipeline.Cursor = opts.Cursor.captureCursor(&captureCursor, normal, excluded)
695+ }
695 pipeline.Retire(scene)696 pipeline.Retire(scene)
696 if len(hidden) > 0 {697 if len(hidden) > 0 {
697 // A hidden workspace: a child renderer draws it. The display's698 // A hidden workspace: a child renderer draws it. The display's
M internal/adapters/screencast/allocator.go +16 -8
@@ -91,7 +91,7 @@ func (a *castAllocator) Fixate(f pipewire.VideoFormat, w, h int, candidates []ui
9191 continue
9292 }
9393 n := len(buf.Planes)
94- closePlanes(&buf)
94+ a.release(&buf)
9595 return m, n, nil
9696 }
9797 if last == nil {
@@ -109,25 +109,25 @@ func (a *castAllocator) Alloc(idx int, f pipewire.VideoFormat, w, h int, modifie
109109 if !ok {
110110 return fmt.Errorf("screencast: video format %d is not a cast format", f)
111111 }
112- closePlanes(&a.bufs[idx])
112+ a.release(&a.bufs[idx])
113113 buf, err := a.dev.AllocCast(fc, w, h, modifier)
114114 if err != nil {
115115 return err
116116 }
117117 if len(buf.Planes) < 1 || len(buf.Planes) > len(out.P) {
118- closePlanes(&buf)
118+ a.release(&buf)
119119 return fmt.Errorf("screencast: cast buffer with %d planes", len(buf.Planes))
120120 }
121121 // The planes share one object: its size bounds every plane.
122122 size, err := unix.Seek(int(buf.Planes[0].File.Fd()), 0, unix.SEEK_END)
123123 if err != nil || size <= 0 || size > 1<<32-1 {
124- closePlanes(&buf)
124+ a.release(&buf)
125125 return fmt.Errorf("screencast: cast buffer size %d: %v", size, err)
126126 }
127127 *out = pipewire.VideoPlanes{N: len(buf.Planes), Size: uint32(size)}
128128 for i, p := range buf.Planes {
129129 if int64(p.Offset) >= size {
130- closePlanes(&buf)
130+ a.release(&buf)
131131 *out = pipewire.VideoPlanes{}
132132 return fmt.Errorf("screencast: plane %d offset %d past the buffer", i, p.Offset)
133133 }
@@ -137,10 +137,10 @@ func (a *castAllocator) Alloc(idx int, f pipewire.VideoFormat, w, h int, modifie
137137 return nil
138138 }
139139
140-// Free closes buffer idx's files: nothing else holds its memory.
140+// Free releases buffer idx's retained allocation before closing its files.
141141 func (a *castAllocator) Free(idx int) {
142142 if idx >= 0 && idx < len(a.bufs) {
143- closePlanes(&a.bufs[idx])
143+ a.release(&a.bufs[idx])
144144 }
145145 }
146146
@@ -155,7 +155,7 @@ func (a *castAllocator) buffer(b *pipewire.VideoBuffer) *domain.DMABuf {
155155 // freeAll closes every buffer still held: the source is gone.
156156 func (a *castAllocator) freeAll() {
157157 for i := range a.bufs {
158- closePlanes(&a.bufs[i])
158+ a.release(&a.bufs[i])
159159 }
160160 }
161161
@@ -168,3 +168,11 @@ func closePlanes(b *domain.DMABuf) {
168168 }
169169 *b = domain.DMABuf{}
170170 }
171+
172+// release runs for remove-buffer, replacement, failed allocation and shutdown.
173+func (a *castAllocator) release(b *domain.DMABuf) {
174+ if len(b.Planes) > 0 {
175+ a.dev.ReleaseCast(*b)
176+ }
177+ closePlanes(b)
178+}
@@ -91,7 +91,7 @@ func (a *castAllocator) Fixate(f pipewire.VideoFormat, w, h int, candidates []ui
91 continue91 continue
92 }92 }
93 n := len(buf.Planes)93 n := len(buf.Planes)
94- closePlanes(&buf)94+ a.release(&buf)
95 return m, n, nil95 return m, n, nil
96 }96 }
97 if last == nil {97 if last == nil {
@@ -109,25 +109,25 @@ func (a *castAllocator) Alloc(idx int, f pipewire.VideoFormat, w, h int, modifie
109 if !ok {109 if !ok {
110 return fmt.Errorf("screencast: video format %d is not a cast format", f)110 return fmt.Errorf("screencast: video format %d is not a cast format", f)
111 }111 }
112- closePlanes(&a.bufs[idx])112+ a.release(&a.bufs[idx])
113 buf, err := a.dev.AllocCast(fc, w, h, modifier)113 buf, err := a.dev.AllocCast(fc, w, h, modifier)
114 if err != nil {114 if err != nil {
115 return err115 return err
116 }116 }
117 if len(buf.Planes) < 1 || len(buf.Planes) > len(out.P) {117 if len(buf.Planes) < 1 || len(buf.Planes) > len(out.P) {
118- closePlanes(&buf)118+ a.release(&buf)
119 return fmt.Errorf("screencast: cast buffer with %d planes", len(buf.Planes))119 return fmt.Errorf("screencast: cast buffer with %d planes", len(buf.Planes))
120 }120 }
121 // The planes share one object: its size bounds every plane.121 // The planes share one object: its size bounds every plane.
122 size, err := unix.Seek(int(buf.Planes[0].File.Fd()), 0, unix.SEEK_END)122 size, err := unix.Seek(int(buf.Planes[0].File.Fd()), 0, unix.SEEK_END)
123 if err != nil || size <= 0 || size > 1<<32-1 {123 if err != nil || size <= 0 || size > 1<<32-1 {
124- closePlanes(&buf)124+ a.release(&buf)
125 return fmt.Errorf("screencast: cast buffer size %d: %v", size, err)125 return fmt.Errorf("screencast: cast buffer size %d: %v", size, err)
126 }126 }
127 *out = pipewire.VideoPlanes{N: len(buf.Planes), Size: uint32(size)}127 *out = pipewire.VideoPlanes{N: len(buf.Planes), Size: uint32(size)}
128 for i, p := range buf.Planes {128 for i, p := range buf.Planes {
129 if int64(p.Offset) >= size {129 if int64(p.Offset) >= size {
130- closePlanes(&buf)130+ a.release(&buf)
131 *out = pipewire.VideoPlanes{}131 *out = pipewire.VideoPlanes{}
132 return fmt.Errorf("screencast: plane %d offset %d past the buffer", i, p.Offset)132 return fmt.Errorf("screencast: plane %d offset %d past the buffer", i, p.Offset)
133 }133 }
@@ -137,10 +137,10 @@ func (a *castAllocator) Alloc(idx int, f pipewire.VideoFormat, w, h int, modifie
137 return nil137 return nil
138 }138 }
139 139
140-// Free closes buffer idx's files: nothing else holds its memory.140+// Free releases buffer idx's retained allocation before closing its files.
141 func (a *castAllocator) Free(idx int) {141 func (a *castAllocator) Free(idx int) {
142 if idx >= 0 && idx < len(a.bufs) {142 if idx >= 0 && idx < len(a.bufs) {
143- closePlanes(&a.bufs[idx])143+ a.release(&a.bufs[idx])
144 }144 }
145 }145 }
146 146
@@ -155,7 +155,7 @@ func (a *castAllocator) buffer(b *pipewire.VideoBuffer) *domain.DMABuf {
155 // freeAll closes every buffer still held: the source is gone.155 // freeAll closes every buffer still held: the source is gone.
156 func (a *castAllocator) freeAll() {156 func (a *castAllocator) freeAll() {
157 for i := range a.bufs {157 for i := range a.bufs {
158- closePlanes(&a.bufs[i])158+ a.release(&a.bufs[i])
159 }159 }
160 }160 }
161 161
@@ -168,3 +168,11 @@ func closePlanes(b *domain.DMABuf) {
168 }168 }
169 *b = domain.DMABuf{}169 *b = domain.DMABuf{}
170 }170 }
171+
172+// release runs for remove-buffer, replacement, failed allocation and shutdown.
173+func (a *castAllocator) release(b *domain.DMABuf) {
174+ if len(b.Planes) > 0 {
175+ a.dev.ReleaseCast(*b)
176+ }
177+ closePlanes(b)
178+}
M internal/adapters/screencast/allocator_test.go +7 -0
@@ -60,6 +60,7 @@ func TestDMABufFormatsGroupsByCastFormat(t *testing.T) {
6060 // its plane count; nothing allocated for the trial is kept.
6161 func TestCastAllocatorFixate(t *testing.T) {
6262 gpu := portsmocks.NewMockCastBufferAllocator(t)
63+ gpu.EXPECT().ReleaseCast(mock.Anything).Maybe()
6364 gpu.EXPECT().CastFormats().Return([]domain.DMABufFormat{{Format: domain.CastXRGB8888, Modifier: 7}, {Format: domain.CastXRGB8888, Modifier: 9}})
6465 trial, f := tempCast(t, domain.CastXRGB8888, 64, 32, 2, 1<<16)
6566 gpu.EXPECT().AllocCast(domain.CastXRGB8888, 64, 32, uint64(9)).Return(domain.DMABuf{}, errors.New("no memory")).Once()
@@ -97,6 +98,11 @@ func TestCastAllocatorAllocFree(t *testing.T) {
9798 if a.buffer(&pipewire.VideoBuffer{Index: 4}) != nil {
9899 t.Fatal("memfd buffer has a dmabuf")
99100 }
101+ gpu.EXPECT().ReleaseCast(buf).Run(func(b domain.DMABuf) {
102+ if closed(b.Planes[0].File) {
103+ t.Fatal("released after close")
104+ }
105+ }).Once()
100106 a.Free(4)
101107 if !closed(f) || a.buffer(&pipewire.VideoBuffer{Index: 4, DMABuf: true}) != nil {
102108 t.Fatal("buffer not freed")
@@ -104,6 +110,7 @@ func TestCastAllocatorAllocFree(t *testing.T) {
104110
105111 // A plane past the object: the allocation is refused and closed.
106112 bad, g := tempCast(t, domain.CastXRGB8888, 64, 32, 2, 512)
113+ gpu.EXPECT().ReleaseCast(bad).Once()
107114 gpu.EXPECT().AllocCast(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(bad, nil).Once()
108115 if err := a.Alloc(5, pipewire.VideoFormatBGRx, 64, 32, 7, &out); err == nil {
109116 t.Fatal("bad planes accepted")
@@ -60,6 +60,7 @@ func TestDMABufFormatsGroupsByCastFormat(t *testing.T) {
60 // its plane count; nothing allocated for the trial is kept.60 // its plane count; nothing allocated for the trial is kept.
61 func TestCastAllocatorFixate(t *testing.T) {61 func TestCastAllocatorFixate(t *testing.T) {
62 gpu := portsmocks.NewMockCastBufferAllocator(t)62 gpu := portsmocks.NewMockCastBufferAllocator(t)
63+ gpu.EXPECT().ReleaseCast(mock.Anything).Maybe()
63 gpu.EXPECT().CastFormats().Return([]domain.DMABufFormat{{Format: domain.CastXRGB8888, Modifier: 7}, {Format: domain.CastXRGB8888, Modifier: 9}})64 gpu.EXPECT().CastFormats().Return([]domain.DMABufFormat{{Format: domain.CastXRGB8888, Modifier: 7}, {Format: domain.CastXRGB8888, Modifier: 9}})
64 trial, f := tempCast(t, domain.CastXRGB8888, 64, 32, 2, 1<<16)65 trial, f := tempCast(t, domain.CastXRGB8888, 64, 32, 2, 1<<16)
65 gpu.EXPECT().AllocCast(domain.CastXRGB8888, 64, 32, uint64(9)).Return(domain.DMABuf{}, errors.New("no memory")).Once()66 gpu.EXPECT().AllocCast(domain.CastXRGB8888, 64, 32, uint64(9)).Return(domain.DMABuf{}, errors.New("no memory")).Once()
@@ -97,6 +98,11 @@ func TestCastAllocatorAllocFree(t *testing.T) {
97 if a.buffer(&pipewire.VideoBuffer{Index: 4}) != nil {98 if a.buffer(&pipewire.VideoBuffer{Index: 4}) != nil {
98 t.Fatal("memfd buffer has a dmabuf")99 t.Fatal("memfd buffer has a dmabuf")
99 }100 }
101+ gpu.EXPECT().ReleaseCast(buf).Run(func(b domain.DMABuf) {
102+ if closed(b.Planes[0].File) {
103+ t.Fatal("released after close")
104+ }
105+ }).Once()
100 a.Free(4)106 a.Free(4)
101 if !closed(f) || a.buffer(&pipewire.VideoBuffer{Index: 4, DMABuf: true}) != nil {107 if !closed(f) || a.buffer(&pipewire.VideoBuffer{Index: 4, DMABuf: true}) != nil {
102 t.Fatal("buffer not freed")108 t.Fatal("buffer not freed")
@@ -104,6 +110,7 @@ func TestCastAllocatorAllocFree(t *testing.T) {
104 110
105 // A plane past the object: the allocation is refused and closed.111 // A plane past the object: the allocation is refused and closed.
106 bad, g := tempCast(t, domain.CastXRGB8888, 64, 32, 2, 512)112 bad, g := tempCast(t, domain.CastXRGB8888, 64, 32, 2, 512)
113+ gpu.EXPECT().ReleaseCast(bad).Once()
107 gpu.EXPECT().AllocCast(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(bad, nil).Once()114 gpu.EXPECT().AllocCast(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(bad, nil).Once()
108 if err := a.Alloc(5, pipewire.VideoFormatBGRx, 64, 32, 7, &out); err == nil {115 if err := a.Alloc(5, pipewire.VideoFormatBGRx, 64, 32, 7, &out); err == nil {
109 t.Fatal("bad planes accepted")116 t.Fatal("bad planes accepted")
M internal/adapters/screencast/harness_test.go +3 -2
@@ -12,11 +12,11 @@ import (
1212 "testing"
1313 "time"
1414
15+ "bnema.dev/zerobus"
16+ "bnema.dev/zerowrap"
1517 "github.com/bnema/neferwl/internal/adapters/captureallow"
1618 "github.com/bnema/neferwl/internal/domain"
1719 portsmocks "github.com/bnema/neferwl/internal/mocks/ports"
18- "bnema.dev/zerobus"
19- "bnema.dev/zerowrap"
2020 "github.com/stretchr/testify/mock"
2121 "golang.org/x/sys/unix"
2222 )
@@ -175,6 +175,7 @@ func newSvcHarness(t *testing.T, o harnessOpts) *svcHarness {
175175 h.svc.address = h.addr
176176 // A GPU exporting linear XR24 cast buffers, each a temporary file.
177177 gpu := portsmocks.NewMockCastBufferAllocator(t)
178+ gpu.EXPECT().ReleaseCast(mock.Anything).Maybe()
178179 gpu.EXPECT().CastFormats().Return([]domain.DMABufFormat{{Format: domain.CastXRGB8888}}).Maybe()
179180 gpu.EXPECT().AllocCast(domain.CastXRGB8888, mock.Anything, mock.Anything, uint64(0)).RunAndReturn(
180181 func(format uint32, w, hh int, _ uint64) (domain.DMABuf, error) {
@@ -12,11 +12,11 @@ import (
12 "testing"12 "testing"
13 "time"13 "time"
14 14
15+ "bnema.dev/zerobus"
16+ "bnema.dev/zerowrap"
15 "github.com/bnema/neferwl/internal/adapters/captureallow"17 "github.com/bnema/neferwl/internal/adapters/captureallow"
16 "github.com/bnema/neferwl/internal/domain"18 "github.com/bnema/neferwl/internal/domain"
17 portsmocks "github.com/bnema/neferwl/internal/mocks/ports"19 portsmocks "github.com/bnema/neferwl/internal/mocks/ports"
18- "bnema.dev/zerobus"
19- "bnema.dev/zerowrap"
20 "github.com/stretchr/testify/mock"20 "github.com/stretchr/testify/mock"
21 "golang.org/x/sys/unix"21 "golang.org/x/sys/unix"
22 )22 )
@@ -175,6 +175,7 @@ func newSvcHarness(t *testing.T, o harnessOpts) *svcHarness {
175 h.svc.address = h.addr175 h.svc.address = h.addr
176 // A GPU exporting linear XR24 cast buffers, each a temporary file.176 // A GPU exporting linear XR24 cast buffers, each a temporary file.
177 gpu := portsmocks.NewMockCastBufferAllocator(t)177 gpu := portsmocks.NewMockCastBufferAllocator(t)
178+ gpu.EXPECT().ReleaseCast(mock.Anything).Maybe()
178 gpu.EXPECT().CastFormats().Return([]domain.DMABufFormat{{Format: domain.CastXRGB8888}}).Maybe()179 gpu.EXPECT().CastFormats().Return([]domain.DMABufFormat{{Format: domain.CastXRGB8888}}).Maybe()
179 gpu.EXPECT().AllocCast(domain.CastXRGB8888, mock.Anything, mock.Anything, uint64(0)).RunAndReturn(180 gpu.EXPECT().AllocCast(domain.CastXRGB8888, mock.Anything, mock.Anything, uint64(0)).RunAndReturn(
180 func(format uint32, w, hh int, _ uint64) (domain.DMABuf, error) {181 func(format uint32, w, hh int, _ uint64) (domain.DMABuf, error) {
M internal/adapters/screencast/producer_test.go +1 -0
@@ -67,6 +67,7 @@ func newProducer(t *testing.T) *producer {
6767 p.s.pwc = p.core
6868 p.buf = &pipewire.VideoBuffer{Index: 3, Width: 64, Height: 32, Format: pipewire.VideoFormatBGRx, DMABuf: true}
6969 p.gpu = portsmocks.NewMockCastBufferAllocator(t)
70+ p.gpu.EXPECT().ReleaseCast(mock.Anything).Maybe()
7071 p.gpu.EXPECT().CastFormats().Return(testCastFormats).Maybe()
7172 p.s.SetCastBuffers(p.gpu)
7273
@@ -67,6 +67,7 @@ func newProducer(t *testing.T) *producer {
67 p.s.pwc = p.core67 p.s.pwc = p.core
68 p.buf = &pipewire.VideoBuffer{Index: 3, Width: 64, Height: 32, Format: pipewire.VideoFormatBGRx, DMABuf: true}68 p.buf = &pipewire.VideoBuffer{Index: 3, Width: 64, Height: 32, Format: pipewire.VideoFormatBGRx, DMABuf: true}
69 p.gpu = portsmocks.NewMockCastBufferAllocator(t)69 p.gpu = portsmocks.NewMockCastBufferAllocator(t)
70+ p.gpu.EXPECT().ReleaseCast(mock.Anything).Maybe()
70 p.gpu.EXPECT().CastFormats().Return(testCastFormats).Maybe()71 p.gpu.EXPECT().CastFormats().Return(testCastFormats).Maybe()
71 p.s.SetCastBuffers(p.gpu)72 p.s.SetCastBuffers(p.gpu)
72 73
M internal/adapters/vulkan/capture.go +11 -5
@@ -121,9 +121,11 @@ type captureScratch struct {
121121 begin vk.CommandBufferBeginInfo
122122 submit vk.SubmitInfo
123123 // The copy (recordCaptureCopy).
124- region vk.BufferImageCopy
125- hb vk.BufferMemoryBarrier
126- b vk.ImageMemoryBarrier
124+ region vk.BufferImageCopy
125+ blit vk.ImageBlit
126+ cursorParams captureCursorParams
127+ hb vk.BufferMemoryBarrier
128+ b vk.ImageMemoryBarrier
127129 // The HDR conversion pass (recordCaptureHDR).
128130 hdrIn, hdrB vk.ImageMemoryBarrier
129131 attachment vk.RenderingAttachmentInfo
@@ -362,7 +364,7 @@ func (r *Renderer) captureSync() captureSync {
362364 return r.sync
363365 }
364366
365-// freeCaptureSlot destroys a slot; the device must be idle for it.
367+// freeCaptureSlot destroys a slot; its capture and DMA-copy fences must be complete.
366368 func (r *Renderer) freeCaptureSlot(s *captureSlot) {
367369 r.freeCaptureCursor(&s.cursor)
368370 for _, c := range s.copies {
@@ -395,12 +397,16 @@ func (r *Renderer) freeCaptureSlot(s *captureSlot) {
395397 }
396398
397399 // destroyCaptures frees the slots and the HDR conversion pass. Every
398-// lease is invalid afterwards; the device is idle (Close).
400+// lease is invalid afterwards; all capture and DMA-copy fences are complete.
399401 func (r *Renderer) destroyCaptures() {
400402 for _, s := range r.captures {
401403 r.freeCaptureSlot(s)
402404 }
403405 r.captures = nil
406+ for _, c := range r.dmaIdle {
407+ r.freeDMACopy(c)
408+ }
409+ r.dmaIdle = nil
404410 r.destroyCapturePass()
405411 r.freeTarget(&r.captureSDR)
406412 }
@@ -121,9 +121,11 @@ type captureScratch struct {
121 begin vk.CommandBufferBeginInfo121 begin vk.CommandBufferBeginInfo
122 submit vk.SubmitInfo122 submit vk.SubmitInfo
123 // The copy (recordCaptureCopy).123 // The copy (recordCaptureCopy).
124- region vk.BufferImageCopy124+ region vk.BufferImageCopy
125- hb vk.BufferMemoryBarrier125+ blit vk.ImageBlit
126- b vk.ImageMemoryBarrier126+ cursorParams captureCursorParams
127+ hb vk.BufferMemoryBarrier
128+ b vk.ImageMemoryBarrier
127 // The HDR conversion pass (recordCaptureHDR).129 // The HDR conversion pass (recordCaptureHDR).
128 hdrIn, hdrB vk.ImageMemoryBarrier130 hdrIn, hdrB vk.ImageMemoryBarrier
129 attachment vk.RenderingAttachmentInfo131 attachment vk.RenderingAttachmentInfo
@@ -362,7 +364,7 @@ func (r *Renderer) captureSync() captureSync {
362 return r.sync364 return r.sync
363 }365 }
364 366
365-// freeCaptureSlot destroys a slot; the device must be idle for it.367+// freeCaptureSlot destroys a slot; its capture and DMA-copy fences must be complete.
366 func (r *Renderer) freeCaptureSlot(s *captureSlot) {368 func (r *Renderer) freeCaptureSlot(s *captureSlot) {
367 r.freeCaptureCursor(&s.cursor)369 r.freeCaptureCursor(&s.cursor)
368 for _, c := range s.copies {370 for _, c := range s.copies {
@@ -395,12 +397,16 @@ func (r *Renderer) freeCaptureSlot(s *captureSlot) {
395 }397 }
396 398
397 // destroyCaptures frees the slots and the HDR conversion pass. Every399 // destroyCaptures frees the slots and the HDR conversion pass. Every
398-// lease is invalid afterwards; the device is idle (Close).400+// lease is invalid afterwards; all capture and DMA-copy fences are complete.
399 func (r *Renderer) destroyCaptures() {401 func (r *Renderer) destroyCaptures() {
400 for _, s := range r.captures {402 for _, s := range r.captures {
401 r.freeCaptureSlot(s)403 r.freeCaptureSlot(s)
402 }404 }
403 r.captures = nil405 r.captures = nil
406+ for _, c := range r.dmaIdle {
407+ r.freeDMACopy(c)
408+ }
409+ r.dmaIdle = nil
404 r.destroyCapturePass()410 r.destroyCapturePass()
405 r.freeTarget(&r.captureSDR)411 r.freeTarget(&r.captureSDR)
406 }412 }
M internal/adapters/vulkan/capture_cursor.go +24 -21
@@ -149,45 +149,48 @@ func (r *Renderer) prepareCaptureCursor(s *captureSlot, c *domain.CaptureCursor,
149149 return nil
150150 }
151151
152+type captureCursorParams struct {
153+ Rect [4]int32
154+ Mode int32
155+ Nits float32
156+}
157+
152158 func (r *Renderer) recordCaptureCursor(cmd vk.CommandBuffer, s *captureSlot, c *domain.CaptureCursor, pq bool) {
153159 d, p, t := r.gpu.dd, &s.cursor, r.last
160+ sc := &r.capCall
154161 barrier := func(img vk.Image, old, new vk.ImageLayout, src, dst vk.AccessFlags, ss, ds vk.PipelineStageFlags) {
155- b := vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, SrcAccessMask: src, DstAccessMask: dst, OldLayout: old, NewLayout: new, SrcQueueFamilyIndex: queueFamilyIgnored, DstQueueFamilyIndex: queueFamilyIgnored, Image: img, SubresourceRange: colorRange}
156- d.CmdPipelineBarrier(cmd, ss, ds, 0, 0, nil, 0, nil, 1, &b)
162+ sc.b = vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, SrcAccessMask: src, DstAccessMask: dst, OldLayout: old, NewLayout: new, SrcQueueFamilyIndex: queueFamilyIgnored, DstQueueFamilyIndex: queueFamilyIgnored, Image: img, SubresourceRange: colorRange}
163+ d.CmdPipelineBarrier(cmd, ss, ds, 0, 0, nil, 0, nil, 1, &sc.b)
157164 }
158165 if t.exported {
159- b := vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, DstAccessMask: vk.AccessTransferReadBit, OldLayout: t.layout, NewLayout: t.layout, SrcQueueFamilyIndex: vk.QueueFamilyForeignEXT, DstQueueFamilyIndex: r.gpu.family, Image: t.image, SubresourceRange: colorRange}
160- d.CmdPipelineBarrier(cmd, vk.PipelineStageAllCommandsBit, vk.PipelineStageTransferBit, 0, 0, nil, 0, nil, 1, &b)
166+ sc.b = vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, DstAccessMask: vk.AccessTransferReadBit, OldLayout: t.layout, NewLayout: t.layout, SrcQueueFamilyIndex: vk.QueueFamilyForeignEXT, DstQueueFamilyIndex: r.gpu.family, Image: t.image, SubresourceRange: colorRange}
167+ d.CmdPipelineBarrier(cmd, vk.PipelineStageAllCommandsBit, vk.PipelineStageTransferBit, 0, 0, nil, 0, nil, 1, &sc.b)
161168 }
162169 barrier(p.source.image, vk.ImageLayoutUndefined, vk.ImageLayoutTransferDstOptimal, 0, vk.AccessTransferWriteBit, vk.PipelineStageTopOfPipeBit, vk.PipelineStageTransferBit)
163170 offsets := [2]vk.Offset3D{{}, {X: int32(r.width), Y: int32(r.height), Z: 1}}
164- region := vk.ImageBlit{SrcSubresource: vk.ImageSubresourceLayers{AspectMask: vk.ImageAspectColorBit, LayerCount: 1}, DstSubresource: vk.ImageSubresourceLayers{AspectMask: vk.ImageAspectColorBit, LayerCount: 1}, SrcOffsets: offsets, DstOffsets: offsets}
165- d.CmdBlitImage(cmd, t.image, t.layout, p.source.image, vk.ImageLayoutTransferDstOptimal, 1, &region, vk.FilterNearest)
171+ sc.blit = vk.ImageBlit{SrcSubresource: vk.ImageSubresourceLayers{AspectMask: vk.ImageAspectColorBit, LayerCount: 1}, DstSubresource: vk.ImageSubresourceLayers{AspectMask: vk.ImageAspectColorBit, LayerCount: 1}, SrcOffsets: offsets, DstOffsets: offsets}
172+ d.CmdBlitImage(cmd, t.image, t.layout, p.source.image, vk.ImageLayoutTransferDstOptimal, 1, &sc.blit, vk.FilterNearest)
166173 if t.exported {
167- b := vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, SrcAccessMask: vk.AccessTransferReadBit, OldLayout: t.layout, NewLayout: t.layout, SrcQueueFamilyIndex: r.gpu.family, DstQueueFamilyIndex: vk.QueueFamilyForeignEXT, Image: t.image, SubresourceRange: colorRange}
168- d.CmdPipelineBarrier(cmd, vk.PipelineStageTransferBit, vk.PipelineStageBottomOfPipeBit, 0, 0, nil, 0, nil, 1, &b)
174+ sc.b = vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, SrcAccessMask: vk.AccessTransferReadBit, OldLayout: t.layout, NewLayout: t.layout, SrcQueueFamilyIndex: r.gpu.family, DstQueueFamilyIndex: vk.QueueFamilyForeignEXT, Image: t.image, SubresourceRange: colorRange}
175+ d.CmdPipelineBarrier(cmd, vk.PipelineStageTransferBit, vk.PipelineStageBottomOfPipeBit, 0, 0, nil, 0, nil, 1, &sc.b)
169176 }
170177 barrier(p.source.image, vk.ImageLayoutTransferDstOptimal, vk.ImageLayoutShaderReadOnlyOptimal, vk.AccessTransferWriteBit, vk.AccessShaderReadBit, vk.PipelineStageTransferBit, vk.PipelineStageFragmentShaderBit)
171- hb := vk.BufferMemoryBarrier{SType: vk.StructureTypeBufferMemoryBarrier, SrcAccessMask: vk.AccessHostWriteBit, DstAccessMask: vk.AccessShaderReadBit, SrcQueueFamilyIndex: queueFamilyIgnored, DstQueueFamilyIndex: queueFamilyIgnored, Buffer: p.upload.buffer, Size: wholeSize}
172- d.CmdPipelineBarrier(cmd, vk.PipelineStageHostBit, vk.PipelineStageFragmentShaderBit, 0, 0, nil, 1, &hb, 0, nil)
178+ sc.hb = vk.BufferMemoryBarrier{SType: vk.StructureTypeBufferMemoryBarrier, SrcAccessMask: vk.AccessHostWriteBit, DstAccessMask: vk.AccessShaderReadBit, SrcQueueFamilyIndex: queueFamilyIgnored, DstQueueFamilyIndex: queueFamilyIgnored, Buffer: p.upload.buffer, Size: wholeSize}
179+ d.CmdPipelineBarrier(cmd, vk.PipelineStageHostBit, vk.PipelineStageFragmentShaderBit, 0, 0, nil, 1, &sc.hb, 0, nil)
173180 barrier(p.out.image, vk.ImageLayoutUndefined, vk.ImageLayoutColorAttachmentOptimal, 0, vk.AccessColorAttachmentWriteBit, vk.PipelineStageTopOfPipeBit, vk.PipelineStageColorAttachmentOutputBit)
174- attachment := vk.RenderingAttachmentInfo{SType: vk.StructureTypeRenderingAttachmentInfo, ImageView: p.out.view, ImageLayout: vk.ImageLayoutColorAttachmentOptimal, LoadOp: vk.AttachmentLoadOpDontCare, StoreOp: vk.AttachmentStoreOpStore}
175- info := vk.RenderingInfo{SType: vk.StructureTypeRenderingInfo, RenderArea: vk.Rect2D{Extent: vk.Extent2D{Width: uint32(r.width), Height: uint32(r.height)}}, LayerCount: 1, ColorAttachmentCount: 1, ColorAttachments: &attachment}
176- d.CmdBeginRendering(cmd, &info)
181+ sc.attachment = vk.RenderingAttachmentInfo{SType: vk.StructureTypeRenderingAttachmentInfo, ImageView: p.out.view, ImageLayout: vk.ImageLayoutColorAttachmentOptimal, LoadOp: vk.AttachmentLoadOpDontCare, StoreOp: vk.AttachmentStoreOpStore}
182+ sc.info = vk.RenderingInfo{SType: vk.StructureTypeRenderingInfo, RenderArea: vk.Rect2D{Extent: vk.Extent2D{Width: uint32(r.width), Height: uint32(r.height)}}, LayerCount: 1, ColorAttachmentCount: 1, ColorAttachments: &sc.attachment}
183+ d.CmdBeginRendering(cmd, &sc.info)
177184 d.CmdBindPipeline(cmd, vk.PipelineBindPointGraphics, p.pass.pipeline)
178185 d.CmdBindDescriptorSets(cmd, vk.PipelineBindPointGraphics, p.pass.layout, 0, 1, &p.pass.set, 0, nil)
179- params := struct {
180- Rect [4]int32
181- Mode int32
182- Nits float32
183- }{Rect: [4]int32{int32(c.X - c.Image.HotX), int32(c.Y - c.Image.HotY), int32(c.Image.W), int32(c.Image.H)}, Nits: float32(r.hdrNits)}
186+ sc.cursorParams = captureCursorParams{Rect: [4]int32{int32(c.X - c.Image.HotX), int32(c.Y - c.Image.HotY), int32(c.Image.W), int32(c.Image.H)}, Nits: float32(r.hdrNits)}
184187 if t == &r.hdrOwn {
185- params.Mode = 1
188+ sc.cursorParams.Mode = 1
186189 }
187190 if pq {
188- params.Mode = 2
191+ sc.cursorParams.Mode = 2
189192 }
190- d.CmdPushConstants(cmd, p.pass.layout, vk.ShaderStageFragmentBit, 0, 24, unsafe.Pointer(&params))
193+ d.CmdPushConstants(cmd, p.pass.layout, vk.ShaderStageFragmentBit, 0, 24, unsafe.Pointer(&sc.cursorParams))
191194 d.CmdDraw(cmd, 3, 1, 0, 0)
192195 d.CmdEndRendering(cmd)
193196 barrier(p.out.image, vk.ImageLayoutColorAttachmentOptimal, vk.ImageLayoutTransferSrcOptimal, vk.AccessColorAttachmentWriteBit, vk.AccessTransferReadBit, vk.PipelineStageColorAttachmentOutputBit, vk.PipelineStageTransferBit)
@@ -149,45 +149,48 @@ func (r *Renderer) prepareCaptureCursor(s *captureSlot, c *domain.CaptureCursor,
149 return nil149 return nil
150 }150 }
151 151
152+type captureCursorParams struct {
153+ Rect [4]int32
154+ Mode int32
155+ Nits float32
156+}
157+
152 func (r *Renderer) recordCaptureCursor(cmd vk.CommandBuffer, s *captureSlot, c *domain.CaptureCursor, pq bool) {158 func (r *Renderer) recordCaptureCursor(cmd vk.CommandBuffer, s *captureSlot, c *domain.CaptureCursor, pq bool) {
153 d, p, t := r.gpu.dd, &s.cursor, r.last159 d, p, t := r.gpu.dd, &s.cursor, r.last
160+ sc := &r.capCall
154 barrier := func(img vk.Image, old, new vk.ImageLayout, src, dst vk.AccessFlags, ss, ds vk.PipelineStageFlags) {161 barrier := func(img vk.Image, old, new vk.ImageLayout, src, dst vk.AccessFlags, ss, ds vk.PipelineStageFlags) {
155- b := vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, SrcAccessMask: src, DstAccessMask: dst, OldLayout: old, NewLayout: new, SrcQueueFamilyIndex: queueFamilyIgnored, DstQueueFamilyIndex: queueFamilyIgnored, Image: img, SubresourceRange: colorRange}162+ sc.b = vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, SrcAccessMask: src, DstAccessMask: dst, OldLayout: old, NewLayout: new, SrcQueueFamilyIndex: queueFamilyIgnored, DstQueueFamilyIndex: queueFamilyIgnored, Image: img, SubresourceRange: colorRange}
156- d.CmdPipelineBarrier(cmd, ss, ds, 0, 0, nil, 0, nil, 1, &b)163+ d.CmdPipelineBarrier(cmd, ss, ds, 0, 0, nil, 0, nil, 1, &sc.b)
157 }164 }
158 if t.exported {165 if t.exported {
159- b := vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, DstAccessMask: vk.AccessTransferReadBit, OldLayout: t.layout, NewLayout: t.layout, SrcQueueFamilyIndex: vk.QueueFamilyForeignEXT, DstQueueFamilyIndex: r.gpu.family, Image: t.image, SubresourceRange: colorRange}166+ sc.b = vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, DstAccessMask: vk.AccessTransferReadBit, OldLayout: t.layout, NewLayout: t.layout, SrcQueueFamilyIndex: vk.QueueFamilyForeignEXT, DstQueueFamilyIndex: r.gpu.family, Image: t.image, SubresourceRange: colorRange}
160- d.CmdPipelineBarrier(cmd, vk.PipelineStageAllCommandsBit, vk.PipelineStageTransferBit, 0, 0, nil, 0, nil, 1, &b)167+ d.CmdPipelineBarrier(cmd, vk.PipelineStageAllCommandsBit, vk.PipelineStageTransferBit, 0, 0, nil, 0, nil, 1, &sc.b)
161 }168 }
162 barrier(p.source.image, vk.ImageLayoutUndefined, vk.ImageLayoutTransferDstOptimal, 0, vk.AccessTransferWriteBit, vk.PipelineStageTopOfPipeBit, vk.PipelineStageTransferBit)169 barrier(p.source.image, vk.ImageLayoutUndefined, vk.ImageLayoutTransferDstOptimal, 0, vk.AccessTransferWriteBit, vk.PipelineStageTopOfPipeBit, vk.PipelineStageTransferBit)
163 offsets := [2]vk.Offset3D{{}, {X: int32(r.width), Y: int32(r.height), Z: 1}}170 offsets := [2]vk.Offset3D{{}, {X: int32(r.width), Y: int32(r.height), Z: 1}}
164- region := vk.ImageBlit{SrcSubresource: vk.ImageSubresourceLayers{AspectMask: vk.ImageAspectColorBit, LayerCount: 1}, DstSubresource: vk.ImageSubresourceLayers{AspectMask: vk.ImageAspectColorBit, LayerCount: 1}, SrcOffsets: offsets, DstOffsets: offsets}171+ sc.blit = vk.ImageBlit{SrcSubresource: vk.ImageSubresourceLayers{AspectMask: vk.ImageAspectColorBit, LayerCount: 1}, DstSubresource: vk.ImageSubresourceLayers{AspectMask: vk.ImageAspectColorBit, LayerCount: 1}, SrcOffsets: offsets, DstOffsets: offsets}
165- d.CmdBlitImage(cmd, t.image, t.layout, p.source.image, vk.ImageLayoutTransferDstOptimal, 1, &region, vk.FilterNearest)172+ d.CmdBlitImage(cmd, t.image, t.layout, p.source.image, vk.ImageLayoutTransferDstOptimal, 1, &sc.blit, vk.FilterNearest)
166 if t.exported {173 if t.exported {
167- b := vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, SrcAccessMask: vk.AccessTransferReadBit, OldLayout: t.layout, NewLayout: t.layout, SrcQueueFamilyIndex: r.gpu.family, DstQueueFamilyIndex: vk.QueueFamilyForeignEXT, Image: t.image, SubresourceRange: colorRange}174+ sc.b = vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, SrcAccessMask: vk.AccessTransferReadBit, OldLayout: t.layout, NewLayout: t.layout, SrcQueueFamilyIndex: r.gpu.family, DstQueueFamilyIndex: vk.QueueFamilyForeignEXT, Image: t.image, SubresourceRange: colorRange}
168- d.CmdPipelineBarrier(cmd, vk.PipelineStageTransferBit, vk.PipelineStageBottomOfPipeBit, 0, 0, nil, 0, nil, 1, &b)175+ d.CmdPipelineBarrier(cmd, vk.PipelineStageTransferBit, vk.PipelineStageBottomOfPipeBit, 0, 0, nil, 0, nil, 1, &sc.b)
169 }176 }
170 barrier(p.source.image, vk.ImageLayoutTransferDstOptimal, vk.ImageLayoutShaderReadOnlyOptimal, vk.AccessTransferWriteBit, vk.AccessShaderReadBit, vk.PipelineStageTransferBit, vk.PipelineStageFragmentShaderBit)177 barrier(p.source.image, vk.ImageLayoutTransferDstOptimal, vk.ImageLayoutShaderReadOnlyOptimal, vk.AccessTransferWriteBit, vk.AccessShaderReadBit, vk.PipelineStageTransferBit, vk.PipelineStageFragmentShaderBit)
171- hb := vk.BufferMemoryBarrier{SType: vk.StructureTypeBufferMemoryBarrier, SrcAccessMask: vk.AccessHostWriteBit, DstAccessMask: vk.AccessShaderReadBit, SrcQueueFamilyIndex: queueFamilyIgnored, DstQueueFamilyIndex: queueFamilyIgnored, Buffer: p.upload.buffer, Size: wholeSize}178+ sc.hb = vk.BufferMemoryBarrier{SType: vk.StructureTypeBufferMemoryBarrier, SrcAccessMask: vk.AccessHostWriteBit, DstAccessMask: vk.AccessShaderReadBit, SrcQueueFamilyIndex: queueFamilyIgnored, DstQueueFamilyIndex: queueFamilyIgnored, Buffer: p.upload.buffer, Size: wholeSize}
172- d.CmdPipelineBarrier(cmd, vk.PipelineStageHostBit, vk.PipelineStageFragmentShaderBit, 0, 0, nil, 1, &hb, 0, nil)179+ d.CmdPipelineBarrier(cmd, vk.PipelineStageHostBit, vk.PipelineStageFragmentShaderBit, 0, 0, nil, 1, &sc.hb, 0, nil)
173 barrier(p.out.image, vk.ImageLayoutUndefined, vk.ImageLayoutColorAttachmentOptimal, 0, vk.AccessColorAttachmentWriteBit, vk.PipelineStageTopOfPipeBit, vk.PipelineStageColorAttachmentOutputBit)180 barrier(p.out.image, vk.ImageLayoutUndefined, vk.ImageLayoutColorAttachmentOptimal, 0, vk.AccessColorAttachmentWriteBit, vk.PipelineStageTopOfPipeBit, vk.PipelineStageColorAttachmentOutputBit)
174- attachment := vk.RenderingAttachmentInfo{SType: vk.StructureTypeRenderingAttachmentInfo, ImageView: p.out.view, ImageLayout: vk.ImageLayoutColorAttachmentOptimal, LoadOp: vk.AttachmentLoadOpDontCare, StoreOp: vk.AttachmentStoreOpStore}181+ sc.attachment = vk.RenderingAttachmentInfo{SType: vk.StructureTypeRenderingAttachmentInfo, ImageView: p.out.view, ImageLayout: vk.ImageLayoutColorAttachmentOptimal, LoadOp: vk.AttachmentLoadOpDontCare, StoreOp: vk.AttachmentStoreOpStore}
175- info := vk.RenderingInfo{SType: vk.StructureTypeRenderingInfo, RenderArea: vk.Rect2D{Extent: vk.Extent2D{Width: uint32(r.width), Height: uint32(r.height)}}, LayerCount: 1, ColorAttachmentCount: 1, ColorAttachments: &attachment}182+ sc.info = vk.RenderingInfo{SType: vk.StructureTypeRenderingInfo, RenderArea: vk.Rect2D{Extent: vk.Extent2D{Width: uint32(r.width), Height: uint32(r.height)}}, LayerCount: 1, ColorAttachmentCount: 1, ColorAttachments: &sc.attachment}
176- d.CmdBeginRendering(cmd, &info)183+ d.CmdBeginRendering(cmd, &sc.info)
177 d.CmdBindPipeline(cmd, vk.PipelineBindPointGraphics, p.pass.pipeline)184 d.CmdBindPipeline(cmd, vk.PipelineBindPointGraphics, p.pass.pipeline)
178 d.CmdBindDescriptorSets(cmd, vk.PipelineBindPointGraphics, p.pass.layout, 0, 1, &p.pass.set, 0, nil)185 d.CmdBindDescriptorSets(cmd, vk.PipelineBindPointGraphics, p.pass.layout, 0, 1, &p.pass.set, 0, nil)
179- params := struct {186+ sc.cursorParams = captureCursorParams{Rect: [4]int32{int32(c.X - c.Image.HotX), int32(c.Y - c.Image.HotY), int32(c.Image.W), int32(c.Image.H)}, Nits: float32(r.hdrNits)}
180- Rect [4]int32
181- Mode int32
182- Nits float32
183- }{Rect: [4]int32{int32(c.X - c.Image.HotX), int32(c.Y - c.Image.HotY), int32(c.Image.W), int32(c.Image.H)}, Nits: float32(r.hdrNits)}
184 if t == &r.hdrOwn {187 if t == &r.hdrOwn {
185- params.Mode = 1188+ sc.cursorParams.Mode = 1
186 }189 }
187 if pq {190 if pq {
188- params.Mode = 2191+ sc.cursorParams.Mode = 2
189 }192 }
190- d.CmdPushConstants(cmd, p.pass.layout, vk.ShaderStageFragmentBit, 0, 24, unsafe.Pointer(&params))193+ d.CmdPushConstants(cmd, p.pass.layout, vk.ShaderStageFragmentBit, 0, 24, unsafe.Pointer(&sc.cursorParams))
191 d.CmdDraw(cmd, 3, 1, 0, 0)194 d.CmdDraw(cmd, 3, 1, 0, 0)
192 d.CmdEndRendering(cmd)195 d.CmdEndRendering(cmd)
193 barrier(p.out.image, vk.ImageLayoutColorAttachmentOptimal, vk.ImageLayoutTransferSrcOptimal, vk.AccessColorAttachmentWriteBit, vk.AccessTransferReadBit, vk.PipelineStageColorAttachmentOutputBit, vk.PipelineStageTransferBit)196 barrier(p.out.image, vk.ImageLayoutColorAttachmentOptimal, vk.ImageLayoutTransferSrcOptimal, vk.AccessColorAttachmentWriteBit, vk.AccessTransferReadBit, vk.PipelineStageColorAttachmentOutputBit, vk.PipelineStageTransferBit)
M internal/adapters/vulkan/capture_cursor_test.go +77 -0
@@ -164,3 +164,80 @@ func TestCaptureCursorCopyTo(t *testing.T) {
164164 t.Fatalf("dmabuf pixels: %v", raw)
165165 }
166166 }
167+
168+func TestCaptureCursorCycleAllocations(t *testing.T) {
169+ if raceEnabled {
170+ t.Skip("allocation budgets are not stable under the race runtime")
171+ }
172+ for _, tc := range []struct {
173+ name string
174+ pq bool
175+ new func(*testing.T) *Renderer
176+ }{
177+ {"sdr", false, func(t *testing.T) *Renderer {
178+ r := newTestRenderer(t, 16, 8)
179+ t.Cleanup(r.Close)
180+ return r
181+ }},
182+ {"hdr", false, hdrTestRenderer},
183+ {"pq", true, hdrTestRenderer},
184+ } {
185+ t.Run(tc.name, func(t *testing.T) {
186+ r := tc.new(t)
187+ requireCaptureSync(t, r)
188+ if err := render(r, domain.Scene{Seq: 1, Background: "#3060c0"}, nil); err != nil {
189+ t.Fatal(err)
190+ }
191+ dst := make([]byte, 16*8*4)
192+ fds := make([]unix.PollFd, 1)
193+ region := image.Rect(0, 0, 16, 8)
194+ cursor := &domain.CaptureCursor{X: 3, Y: 4, Image: domain.CursorImage{W: 1, H: 1, Pixels: []byte{0, 0, 255, 255}}}
195+ var cycleErr error
196+ cycle := func() {
197+ cf, err := r.BeginCaptureCursor(cursor, tc.pq)
198+ if err != nil {
199+ cycleErr = err
200+ return
201+ }
202+ if done := cf.Done(); done != nil {
203+ if err := pollSyncFile(done, fds); err != nil {
204+ cycleErr = err
205+ }
206+ }
207+ if err := cf.Read(region, dst, 16*4); err != nil {
208+ cycleErr = err
209+ }
210+ r.EndCapture(cf)
211+ }
212+ cycle() // creates the slots, the capture pass and the sync files
213+ cycle()
214+ if cycleErr != nil {
215+ t.Fatal(cycleErr)
216+ }
217+ fdsBefore := openFDs(t)
218+ // The cycle crosses into Vulkan through purego, whose argument pool
219+ // drops items under the race detector; the zero count is only
220+ // meaningful without it.
221+ allocs := testing.AllocsPerRun(50, cycle)
222+ t.Logf("%.1f allocs/cycle", allocs)
223+ result := testing.Benchmark(func(b *testing.B) {
224+ for b.Loop() {
225+ cycle()
226+ }
227+ })
228+ t.Logf("%s %s", result.String(), result.MemString())
229+ if allocs != 0 && !raceEnabled {
230+ t.Errorf("%.1f allocations per capture cycle, want 0", allocs)
231+ }
232+ if cycleErr != nil {
233+ t.Fatal(cycleErr)
234+ }
235+ if after := openFDs(t); after != fdsBefore {
236+ t.Errorf("%d descriptors open after the cycles, %d before: a sync file leaked", after, fdsBefore)
237+ }
238+ if dst[0] == 0 && dst[1] == 0 && dst[2] == 0 {
239+ t.Error("capture read back black")
240+ }
241+ })
242+ }
243+}
@@ -164,3 +164,80 @@ func TestCaptureCursorCopyTo(t *testing.T) {
164 t.Fatalf("dmabuf pixels: %v", raw)164 t.Fatalf("dmabuf pixels: %v", raw)
165 }165 }
166 }166 }
167+
168+func TestCaptureCursorCycleAllocations(t *testing.T) {
169+ if raceEnabled {
170+ t.Skip("allocation budgets are not stable under the race runtime")
171+ }
172+ for _, tc := range []struct {
173+ name string
174+ pq bool
175+ new func(*testing.T) *Renderer
176+ }{
177+ {"sdr", false, func(t *testing.T) *Renderer {
178+ r := newTestRenderer(t, 16, 8)
179+ t.Cleanup(r.Close)
180+ return r
181+ }},
182+ {"hdr", false, hdrTestRenderer},
183+ {"pq", true, hdrTestRenderer},
184+ } {
185+ t.Run(tc.name, func(t *testing.T) {
186+ r := tc.new(t)
187+ requireCaptureSync(t, r)
188+ if err := render(r, domain.Scene{Seq: 1, Background: "#3060c0"}, nil); err != nil {
189+ t.Fatal(err)
190+ }
191+ dst := make([]byte, 16*8*4)
192+ fds := make([]unix.PollFd, 1)
193+ region := image.Rect(0, 0, 16, 8)
194+ cursor := &domain.CaptureCursor{X: 3, Y: 4, Image: domain.CursorImage{W: 1, H: 1, Pixels: []byte{0, 0, 255, 255}}}
195+ var cycleErr error
196+ cycle := func() {
197+ cf, err := r.BeginCaptureCursor(cursor, tc.pq)
198+ if err != nil {
199+ cycleErr = err
200+ return
201+ }
202+ if done := cf.Done(); done != nil {
203+ if err := pollSyncFile(done, fds); err != nil {
204+ cycleErr = err
205+ }
206+ }
207+ if err := cf.Read(region, dst, 16*4); err != nil {
208+ cycleErr = err
209+ }
210+ r.EndCapture(cf)
211+ }
212+ cycle() // creates the slots, the capture pass and the sync files
213+ cycle()
214+ if cycleErr != nil {
215+ t.Fatal(cycleErr)
216+ }
217+ fdsBefore := openFDs(t)
218+ // The cycle crosses into Vulkan through purego, whose argument pool
219+ // drops items under the race detector; the zero count is only
220+ // meaningful without it.
221+ allocs := testing.AllocsPerRun(50, cycle)
222+ t.Logf("%.1f allocs/cycle", allocs)
223+ result := testing.Benchmark(func(b *testing.B) {
224+ for b.Loop() {
225+ cycle()
226+ }
227+ })
228+ t.Logf("%s %s", result.String(), result.MemString())
229+ if allocs != 0 && !raceEnabled {
230+ t.Errorf("%.1f allocations per capture cycle, want 0", allocs)
231+ }
232+ if cycleErr != nil {
233+ t.Fatal(cycleErr)
234+ }
235+ if after := openFDs(t); after != fdsBefore {
236+ t.Errorf("%d descriptors open after the cycles, %d before: a sync file leaked", after, fdsBefore)
237+ }
238+ if dst[0] == 0 && dst[1] == 0 && dst[2] == 0 {
239+ t.Error("capture read back black")
240+ }
241+ })
242+ }
243+}
M internal/adapters/vulkan/capture_dmabuf.go +127 -70
@@ -19,8 +19,8 @@ import (
1919 // compose pipelines never write alpha over the clear's 1; the HDR capture
2020 // pass writes 1), so a copy is that buffer's region uploaded
2121 // into the imported client image, on the frame queue after the readback.
22-// Each copy has its own command buffer, fence and exported semaphore; the
23-// slot keeps it until its fence polls signalled, and is not reused before.
22+// Each copy borrows command/sync objects from the renderer owner pool. The
23+// slot keeps them until its fence polls signalled, then they can be reused.
2424
2525 // VK_FORMAT_FEATURE_TRANSFER_DST_BIT (Vulkan 1.1), missing from the
2626 // generated bindings.
@@ -34,11 +34,22 @@ var captureFourccs = [...]uint32{fourcc('X', 'R', '2', '4'), fourcc('A', 'R', '2
3434
3535 // dmaCopy is one submitted copy into a client dmabuf.
3636 type dmaCopy struct {
37- cmd vk.CommandBuffer
38- fence vk.Fence
39- done vk.Semaphore
40- wait vk.Semaphore // the destination's implicit fence, 0 when none
41- img *target
37+ cmd vk.CommandBuffer
38+ fence vk.Fence
39+ done vk.Semaphore
40+ wait vk.Semaphore // the destination's implicit fence, 0 when none
41+ cached *castImage
42+ img *target
43+ exported bool
44+ waitUsed bool
45+ begin vk.CommandBufferBeginInfo
46+ hb vk.BufferMemoryBarrier
47+ ib vk.ImageMemoryBarrier
48+ region vk.BufferImageCopy
49+ submit vk.SubmitInfo
50+ waitStage vk.PipelineStageFlags
51+ get vk.SemaphoreGetFdInfoKHR
52+ fd int32
4253 }
4354
4455 // CaptureFormats lists the single-plane XRGB8888/ARGB8888 modifiers a
@@ -94,11 +105,20 @@ func (f *captureFrame) CopyTo(region image.Rectangle, dst domain.DMABuf) (*os.Fi
94105 if len(s.copies) >= maxSlotCopies {
95106 return nil, ports.ErrCaptureBusy
96107 }
97- img, err := r.gpu.importBlitImage(&dst, dst.Width, dst.Height, vk.ImageUsageTransferDstBit, formatFeatureTransferDst)
108+ img, cached, err := r.gpu.acquireCastImage(&dst)
98109 if err != nil {
99110 return nil, err
100111 }
101- c := &dmaCopy{img: img}
112+ var c *dmaCopy
113+ if n := len(r.dmaIdle); n > 0 {
114+ c = r.dmaIdle[n-1]
115+ r.dmaIdle = r.dmaIdle[:n-1]
116+ } else {
117+ c = &dmaCopy{}
118+ }
119+ c.img, c.cached = img, cached
120+ c.exported = false
121+ c.waitUsed = false
102122 submitted := false
103123 defer func() {
104124 if !submitted {
@@ -108,64 +128,74 @@ func (f *captureFrame) CopyTo(region image.Rectangle, dst domain.DMABuf) (*os.Fi
108128 // Implicit sync with the buffer's other users: the copy waits for what
109129 // they still do with it (a consumer reading the previous frame). When
110130 // that cannot be established nothing is written (fail closed).
111- if c.wait, err = r.dmabufBusy(dst.Planes[0].File); err != nil {
131+ if err = r.prepareDMAWait(c, dst.Planes[0].File); err != nil {
112132 return nil, err
113133 }
114134 d, dev := r.gpu.dd, r.gpu.device
115- ai := vk.CommandBufferAllocateInfo{SType: vk.StructureTypeCommandBufferAllocateInfo, CommandPool: r.pool, Level: vk.CommandBufferLevelPrimary, CommandBufferCount: 1}
116- if err := checked("vkAllocateCommandBuffers(capture dmabuf)", d.AllocateCommandBuffers(dev, &ai, &c.cmd)); err != nil {
117- return nil, err
135+ if c.cmd == 0 {
136+ ai := vk.CommandBufferAllocateInfo{SType: vk.StructureTypeCommandBufferAllocateInfo, CommandPool: r.pool, Level: vk.CommandBufferLevelPrimary, CommandBufferCount: 1}
137+ if err := checked("vkAllocateCommandBuffers(capture dmabuf)", d.AllocateCommandBuffers(dev, &ai, &c.cmd)); err != nil {
138+ return nil, err
139+ }
118140 }
119- fi := vk.FenceCreateInfo{SType: vk.StructureTypeFenceCreateInfo}
120- if err := checked("vkCreateFence(capture dmabuf)", d.CreateFence(dev, &fi, nil, &c.fence)); err != nil {
121- return nil, err
141+ if c.fence == 0 {
142+ fi := vk.FenceCreateInfo{SType: vk.StructureTypeFenceCreateInfo}
143+ if err := checked("vkCreateFence(capture dmabuf)", d.CreateFence(dev, &fi, nil, &c.fence)); err != nil {
144+ return nil, err
145+ }
122146 }
123- if c.done, err = (deviceSync{r}).newSemaphore(); err != nil {
124- return nil, err
147+ if c.done == 0 {
148+ if c.done, err = (deviceSync{r}).newSemaphore(); err != nil {
149+ return nil, err
150+ }
125151 }
126- begin := vk.CommandBufferBeginInfo{SType: vk.StructureTypeCommandBufferBeginInfo, Flags: vk.CommandBufferUsageOneTimeSubmitBit}
127- if err := checked("vkBeginCommandBuffer(capture dmabuf)", d.BeginCommandBuffer(c.cmd, &begin)); err != nil {
152+ c.begin = vk.CommandBufferBeginInfo{SType: vk.StructureTypeCommandBufferBeginInfo, Flags: vk.CommandBufferUsageOneTimeSubmitBit}
153+ if err := checked("vkBeginCommandBuffer(capture dmabuf)", d.BeginCommandBuffer(c.cmd, &c.begin)); err != nil {
128154 return nil, err
129155 }
130156 // The readback, submitted earlier on this queue, wrote the buffer.
131- hb := vk.BufferMemoryBarrier{SType: vk.StructureTypeBufferMemoryBarrier, SrcAccessMask: vk.AccessTransferWriteBit, DstAccessMask: vk.AccessTransferReadBit, SrcQueueFamilyIndex: queueFamilyIgnored, DstQueueFamilyIndex: queueFamilyIgnored, Buffer: s.buffer, Size: wholeSize}
157+ c.hb = vk.BufferMemoryBarrier{SType: vk.StructureTypeBufferMemoryBarrier, SrcAccessMask: vk.AccessTransferWriteBit, DstAccessMask: vk.AccessTransferReadBit, SrcQueueFamilyIndex: queueFamilyIgnored, DstQueueFamilyIndex: queueFamilyIgnored, Buffer: s.buffer, Size: wholeSize}
132158 // Acquire the client image from the foreign family; its content is replaced.
133- ib := vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, DstAccessMask: vk.AccessTransferWriteBit, OldLayout: vk.ImageLayoutUndefined, NewLayout: vk.ImageLayoutTransferDstOptimal, SrcQueueFamilyIndex: vk.QueueFamilyForeignEXT, DstQueueFamilyIndex: r.gpu.family, Image: img.image, SubresourceRange: colorRange}
134- d.CmdPipelineBarrier(c.cmd, vk.PipelineStageTransferBit, vk.PipelineStageTransferBit, 0, 0, nil, 1, &hb, 1, &ib)
135- region0 := vk.BufferImageCopy{
159+ c.ib = vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, DstAccessMask: vk.AccessTransferWriteBit, OldLayout: vk.ImageLayoutUndefined, NewLayout: vk.ImageLayoutTransferDstOptimal, SrcQueueFamilyIndex: vk.QueueFamilyForeignEXT, DstQueueFamilyIndex: r.gpu.family, Image: img.image, SubresourceRange: colorRange}
160+ d.CmdPipelineBarrier(c.cmd, vk.PipelineStageTransferBit, vk.PipelineStageTransferBit, 0, 0, nil, 1, &c.hb, 1, &c.ib)
161+ c.region = vk.BufferImageCopy{
136162 BufferOffset: vk.DeviceSize((region.Min.Y*f.w + region.Min.X) * captureBytesPerPixel),
137163 BufferRowLength: uint32(f.w),
138164 ImageSubresource: vk.ImageSubresourceLayers{AspectMask: vk.ImageAspectColorBit, LayerCount: 1},
139165 ImageExtent: vk.Extent3D{Width: uint32(dst.Width), Height: uint32(dst.Height), Depth: 1},
140166 }
141- d.CmdCopyBufferToImage(c.cmd, s.buffer, img.image, vk.ImageLayoutTransferDstOptimal, 1, &region0)
142- ib.SrcAccessMask, ib.DstAccessMask = vk.AccessTransferWriteBit, 0
143- ib.OldLayout, ib.NewLayout = vk.ImageLayoutTransferDstOptimal, vk.ImageLayoutGeneral
144- ib.SrcQueueFamilyIndex, ib.DstQueueFamilyIndex = r.gpu.family, vk.QueueFamilyForeignEXT
145- d.CmdPipelineBarrier(c.cmd, vk.PipelineStageTransferBit, vk.PipelineStageBottomOfPipeBit, 0, 0, nil, 0, nil, 1, &ib)
167+ d.CmdCopyBufferToImage(c.cmd, s.buffer, img.image, vk.ImageLayoutTransferDstOptimal, 1, &c.region)
168+ c.ib.SrcAccessMask, c.ib.DstAccessMask = vk.AccessTransferWriteBit, 0
169+ c.ib.OldLayout, c.ib.NewLayout = vk.ImageLayoutTransferDstOptimal, vk.ImageLayoutGeneral
170+ c.ib.SrcQueueFamilyIndex, c.ib.DstQueueFamilyIndex = r.gpu.family, vk.QueueFamilyForeignEXT
171+ d.CmdPipelineBarrier(c.cmd, vk.PipelineStageTransferBit, vk.PipelineStageBottomOfPipeBit, 0, 0, nil, 0, nil, 1, &c.ib)
146172 if err := checked("vkEndCommandBuffer(capture dmabuf)", d.EndCommandBuffer(c.cmd)); err != nil {
147173 return nil, err
148174 }
149- submit := vk.SubmitInfo{SType: vk.StructureTypeSubmitInfo, CommandBufferCount: 1, CommandBuffers: &c.cmd, SignalSemaphoreCount: 1, SignalSemaphores: &c.done}
150- waitStage := vk.PipelineStageFlags(vk.PipelineStageTransferBit)
151- if c.wait != 0 {
152- submit.WaitSemaphoreCount, submit.WaitSemaphores, submit.WaitDstStageMask = 1, &c.wait, &waitStage
175+ c.submit = vk.SubmitInfo{SType: vk.StructureTypeSubmitInfo, CommandBufferCount: 1, CommandBuffers: &c.cmd, SignalSemaphoreCount: 1, SignalSemaphores: &c.done}
176+ c.waitStage = vk.PipelineStageFlags(vk.PipelineStageTransferBit)
177+ if c.waitUsed {
178+ c.submit.WaitSemaphoreCount, c.submit.WaitSemaphores, c.submit.WaitDstStageMask = 1, &c.wait, &c.waitStage
153179 }
154- if err := r.q.submit(1, &submit, c.fence); err != nil {
180+ if err := r.q.submit(1, &c.submit, c.fence); err != nil {
155181 return nil, err
156182 }
157183 // From here the copy is tracked by its fence, whatever happens.
158184 submitted = true
159185 s.copies = append(s.copies, c)
160- get := vk.SemaphoreGetFdInfoKHR{SType: vk.StructureTypeSemaphoreGetFDInfoKHR, Semaphore: c.done, HandleType: vk.ExternalSemaphoreHandleTypeSyncFDBit}
161- fd := int32(-1)
162- if err := checked("vkGetSemaphoreFdKHR(capture dmabuf)", d.GetSemaphoreFdKHR(dev, &get, &fd)); err != nil {
186+ c.get = vk.SemaphoreGetFdInfoKHR{SType: vk.StructureTypeSemaphoreGetFDInfoKHR, Semaphore: c.done, HandleType: vk.ExternalSemaphoreHandleTypeSyncFDBit}
187+ c.fd = -1
188+ if err := checked("vkGetSemaphoreFdKHR(capture dmabuf)", d.GetSemaphoreFdKHR(dev, &c.get, &c.fd)); err != nil {
163189 return nil, err
164190 }
165- if fd < 0 {
191+ c.exported = true
192+ if c.fd < 0 {
166193 return nil, nil
167194 }
168- done := os.NewFile(uintptr(fd), "capture-dmabuf-done")
195+ // CopyTo transfers this wrapper to its caller, which may still be using it
196+ // after the GPU fence signals. No return notification exists in this API;
197+ // recycling it here would race the pipeline worker's wait/Close.
198+ done := os.NewFile(uintptr(c.fd), "capture-dmabuf-done")
169199 // Implicit sync the other way: users that do not wait on done (a
170200 // PipeWire consumer) wait on the buffer's own fence.
171201 attachWriteFence(dst.Planes[0].File, done)
@@ -206,8 +236,8 @@ func (d deviceFences) createSemaphore() (vk.Semaphore, error) {
206236 }
207237
208238 func (d deviceFences) importSyncFD(s vk.Semaphore, fd int32) vk.Result {
209- imp := vk.ImportSemaphoreFdInfoKHR{SType: vk.StructureTypeImportSemaphoreFDInfoKHR, Semaphore: s, Flags: vk.SemaphoreImportTemporaryBit, HandleType: vk.ExternalSemaphoreHandleTypeSyncFDBit, Fd: fd}
210- return d.r.gpu.dd.ImportSemaphoreFdKHR(d.r.gpu.device, &imp)
239+ d.r.dmaImport = vk.ImportSemaphoreFdInfoKHR{SType: vk.StructureTypeImportSemaphoreFDInfoKHR, Semaphore: s, Flags: vk.SemaphoreImportTemporaryBit, HandleType: vk.ExternalSemaphoreHandleTypeSyncFDBit, Fd: fd}
240+ return d.r.gpu.dd.ImportSemaphoreFdKHR(d.r.gpu.device, &d.r.dmaImport)
211241 }
212242
213243 func (d deviceFences) destroySemaphore(s vk.Semaphore) {
@@ -222,33 +252,6 @@ func (r *Renderer) fences() dmabufFences {
222252 return r.dmaFences
223253 }
224254
225-// dmabufBusy makes a wait semaphore of every fence pending on the dmabuf
226-// of f (DMA_BUF_IOCTL_EXPORT_SYNC_FILE for writing: readers and writers).
227-// 0 with no error only when the kernel reports no fence; any failure to
228-// export or import is an error, so the caller writes nothing. The exported
229-// sync file is closed on every path but a successful import.
230-func (r *Renderer) dmabufBusy(f *os.File) (vk.Semaphore, error) {
231- fs := r.fences()
232- fd, err := fs.exportSyncFile(f)
233- if err != nil {
234- return 0, err
235- }
236- if fd < 0 {
237- return 0, nil
238- }
239- sem, err := fs.createSemaphore()
240- if err != nil {
241- _ = unix.Close(int(fd))
242- return 0, err
243- }
244- if err := checked("vkImportSemaphoreFdKHR(dmabuf wait)", fs.importSyncFD(sem, fd)); err != nil {
245- _ = unix.Close(int(fd))
246- fs.destroySemaphore(sem)
247- return 0, err
248- }
249- return sem, nil
250-}
251-
252255 // attachWriteFence adds the sync file done to the dmabuf of f as a write
253256 // fence (DMA_BUF_IOCTL_IMPORT_SYNC_FILE): implicit-sync readers wait for it.
254257 // Best effort: an older kernel leaves the reader to explicit sync.
@@ -288,7 +291,7 @@ func (r *Renderer) reapCopies(s *captureSlot) error {
288291 for _, c := range s.copies {
289292 switch res := r.gpu.dd.WaitForFences(r.gpu.device, 1, &c.fence, 1, 0); res {
290293 case vk.Success:
291- r.freeDMACopy(c)
294+ r.recycleDMACopy(c)
292295 case vk.Timeout, vk.NotReady:
293296 kept = append(kept, c)
294297 default:
@@ -318,8 +321,62 @@ func (r *Renderer) freeDMACopy(c *dmaCopy) {
318321 if c.cmd != 0 {
319322 d.FreeCommandBuffers(dev, r.pool, 1, &c.cmd)
320323 }
321- if c.img != nil {
324+ if c.cached != nil {
325+ r.gpu.releaseCastImage(c.cached)
326+ } else if c.img != nil {
322327 r.gpu.freeImage(c.img)
323328 }
324329 *c = dmaCopy{}
325330 }
331+
332+// prepareDMAWait fails closed unless all implicit fences can be exported
333+// and imported. It imports a fresh temporary payload each frame. A completed
334+// queue wait consumed the previous payload before this object entered dmaIdle.
335+func (r *Renderer) prepareDMAWait(c *dmaCopy, f *os.File) error {
336+ fs := r.fences()
337+ fd, err := fs.exportSyncFile(f)
338+ if err != nil {
339+ return err
340+ }
341+ if fd < 0 {
342+ return nil
343+ }
344+ if c.wait == 0 {
345+ c.wait, err = fs.createSemaphore()
346+ if err != nil {
347+ unix.Close(int(fd))
348+ return err
349+ }
350+ }
351+ if err = checked("vkImportSemaphoreFdKHR(dmabuf wait)", fs.importSyncFD(c.wait, fd)); err != nil {
352+ unix.Close(int(fd))
353+ return err
354+ }
355+ c.waitUsed = true
356+ return nil
357+}
358+
359+func (r *Renderer) recycleDMACopy(c *dmaCopy) {
360+ // Failed sync-file export leaves a signalled binary semaphore payload;
361+ // it must be destroyed, not signalled a second time.
362+ if !c.exported {
363+ r.freeDMACopy(c)
364+ return
365+ }
366+ if c.cached != nil {
367+ r.gpu.releaseCastImage(c.cached)
368+ } else if c.img != nil {
369+ r.gpu.freeImage(c.img)
370+ }
371+ c.cached, c.img = nil, nil
372+ d := r.gpu.dd
373+ if d.ResetFences(r.gpu.device, 1, &c.fence) != vk.Success || d.ResetCommandBuffer(c.cmd, 0) != vk.Success {
374+ r.freeDMACopy(c)
375+ return
376+ }
377+ if len(r.dmaIdle) >= captureSlots*maxSlotCopies {
378+ r.freeDMACopy(c)
379+ return
380+ }
381+ r.dmaIdle = append(r.dmaIdle, c)
382+}
@@ -19,8 +19,8 @@ import (
19 // compose pipelines never write alpha over the clear's 1; the HDR capture19 // compose pipelines never write alpha over the clear's 1; the HDR capture
20 // pass writes 1), so a copy is that buffer's region uploaded20 // pass writes 1), so a copy is that buffer's region uploaded
21 // into the imported client image, on the frame queue after the readback.21 // into the imported client image, on the frame queue after the readback.
22-// Each copy has its own command buffer, fence and exported semaphore; the22+// Each copy borrows command/sync objects from the renderer owner pool. The
23-// slot keeps it until its fence polls signalled, and is not reused before.23+// slot keeps them until its fence polls signalled, then they can be reused.
24 24
25 // VK_FORMAT_FEATURE_TRANSFER_DST_BIT (Vulkan 1.1), missing from the25 // VK_FORMAT_FEATURE_TRANSFER_DST_BIT (Vulkan 1.1), missing from the
26 // generated bindings.26 // generated bindings.
@@ -34,11 +34,22 @@ var captureFourccs = [...]uint32{fourcc('X', 'R', '2', '4'), fourcc('A', 'R', '2
34 34
35 // dmaCopy is one submitted copy into a client dmabuf.35 // dmaCopy is one submitted copy into a client dmabuf.
36 type dmaCopy struct {36 type dmaCopy struct {
37- cmd vk.CommandBuffer37+ cmd vk.CommandBuffer
38- fence vk.Fence38+ fence vk.Fence
39- done vk.Semaphore39+ done vk.Semaphore
40- wait vk.Semaphore // the destination's implicit fence, 0 when none40+ wait vk.Semaphore // the destination's implicit fence, 0 when none
41- img *target41+ cached *castImage
42+ img *target
43+ exported bool
44+ waitUsed bool
45+ begin vk.CommandBufferBeginInfo
46+ hb vk.BufferMemoryBarrier
47+ ib vk.ImageMemoryBarrier
48+ region vk.BufferImageCopy
49+ submit vk.SubmitInfo
50+ waitStage vk.PipelineStageFlags
51+ get vk.SemaphoreGetFdInfoKHR
52+ fd int32
42 }53 }
43 54
44 // CaptureFormats lists the single-plane XRGB8888/ARGB8888 modifiers a55 // CaptureFormats lists the single-plane XRGB8888/ARGB8888 modifiers a
@@ -94,11 +105,20 @@ func (f *captureFrame) CopyTo(region image.Rectangle, dst domain.DMABuf) (*os.Fi
94 if len(s.copies) >= maxSlotCopies {105 if len(s.copies) >= maxSlotCopies {
95 return nil, ports.ErrCaptureBusy106 return nil, ports.ErrCaptureBusy
96 }107 }
97- img, err := r.gpu.importBlitImage(&dst, dst.Width, dst.Height, vk.ImageUsageTransferDstBit, formatFeatureTransferDst)108+ img, cached, err := r.gpu.acquireCastImage(&dst)
98 if err != nil {109 if err != nil {
99 return nil, err110 return nil, err
100 }111 }
101- c := &dmaCopy{img: img}112+ var c *dmaCopy
113+ if n := len(r.dmaIdle); n > 0 {
114+ c = r.dmaIdle[n-1]
115+ r.dmaIdle = r.dmaIdle[:n-1]
116+ } else {
117+ c = &dmaCopy{}
118+ }
119+ c.img, c.cached = img, cached
120+ c.exported = false
121+ c.waitUsed = false
102 submitted := false122 submitted := false
103 defer func() {123 defer func() {
104 if !submitted {124 if !submitted {
@@ -108,64 +128,74 @@ func (f *captureFrame) CopyTo(region image.Rectangle, dst domain.DMABuf) (*os.Fi
108 // Implicit sync with the buffer's other users: the copy waits for what128 // Implicit sync with the buffer's other users: the copy waits for what
109 // they still do with it (a consumer reading the previous frame). When129 // they still do with it (a consumer reading the previous frame). When
110 // that cannot be established nothing is written (fail closed).130 // that cannot be established nothing is written (fail closed).
111- if c.wait, err = r.dmabufBusy(dst.Planes[0].File); err != nil {131+ if err = r.prepareDMAWait(c, dst.Planes[0].File); err != nil {
112 return nil, err132 return nil, err
113 }133 }
114 d, dev := r.gpu.dd, r.gpu.device134 d, dev := r.gpu.dd, r.gpu.device
115- ai := vk.CommandBufferAllocateInfo{SType: vk.StructureTypeCommandBufferAllocateInfo, CommandPool: r.pool, Level: vk.CommandBufferLevelPrimary, CommandBufferCount: 1}135+ if c.cmd == 0 {
116- if err := checked("vkAllocateCommandBuffers(capture dmabuf)", d.AllocateCommandBuffers(dev, &ai, &c.cmd)); err != nil {136+ ai := vk.CommandBufferAllocateInfo{SType: vk.StructureTypeCommandBufferAllocateInfo, CommandPool: r.pool, Level: vk.CommandBufferLevelPrimary, CommandBufferCount: 1}
117- return nil, err137+ if err := checked("vkAllocateCommandBuffers(capture dmabuf)", d.AllocateCommandBuffers(dev, &ai, &c.cmd)); err != nil {
138+ return nil, err
139+ }
118 }140 }
119- fi := vk.FenceCreateInfo{SType: vk.StructureTypeFenceCreateInfo}141+ if c.fence == 0 {
120- if err := checked("vkCreateFence(capture dmabuf)", d.CreateFence(dev, &fi, nil, &c.fence)); err != nil {142+ fi := vk.FenceCreateInfo{SType: vk.StructureTypeFenceCreateInfo}
121- return nil, err143+ if err := checked("vkCreateFence(capture dmabuf)", d.CreateFence(dev, &fi, nil, &c.fence)); err != nil {
144+ return nil, err
145+ }
122 }146 }
123- if c.done, err = (deviceSync{r}).newSemaphore(); err != nil {147+ if c.done == 0 {
124- return nil, err148+ if c.done, err = (deviceSync{r}).newSemaphore(); err != nil {
149+ return nil, err
150+ }
125 }151 }
126- begin := vk.CommandBufferBeginInfo{SType: vk.StructureTypeCommandBufferBeginInfo, Flags: vk.CommandBufferUsageOneTimeSubmitBit}152+ c.begin = vk.CommandBufferBeginInfo{SType: vk.StructureTypeCommandBufferBeginInfo, Flags: vk.CommandBufferUsageOneTimeSubmitBit}
127- if err := checked("vkBeginCommandBuffer(capture dmabuf)", d.BeginCommandBuffer(c.cmd, &begin)); err != nil {153+ if err := checked("vkBeginCommandBuffer(capture dmabuf)", d.BeginCommandBuffer(c.cmd, &c.begin)); err != nil {
128 return nil, err154 return nil, err
129 }155 }
130 // The readback, submitted earlier on this queue, wrote the buffer.156 // The readback, submitted earlier on this queue, wrote the buffer.
131- hb := vk.BufferMemoryBarrier{SType: vk.StructureTypeBufferMemoryBarrier, SrcAccessMask: vk.AccessTransferWriteBit, DstAccessMask: vk.AccessTransferReadBit, SrcQueueFamilyIndex: queueFamilyIgnored, DstQueueFamilyIndex: queueFamilyIgnored, Buffer: s.buffer, Size: wholeSize}157+ c.hb = vk.BufferMemoryBarrier{SType: vk.StructureTypeBufferMemoryBarrier, SrcAccessMask: vk.AccessTransferWriteBit, DstAccessMask: vk.AccessTransferReadBit, SrcQueueFamilyIndex: queueFamilyIgnored, DstQueueFamilyIndex: queueFamilyIgnored, Buffer: s.buffer, Size: wholeSize}
132 // Acquire the client image from the foreign family; its content is replaced.158 // Acquire the client image from the foreign family; its content is replaced.
133- ib := vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, DstAccessMask: vk.AccessTransferWriteBit, OldLayout: vk.ImageLayoutUndefined, NewLayout: vk.ImageLayoutTransferDstOptimal, SrcQueueFamilyIndex: vk.QueueFamilyForeignEXT, DstQueueFamilyIndex: r.gpu.family, Image: img.image, SubresourceRange: colorRange}159+ c.ib = vk.ImageMemoryBarrier{SType: vk.StructureTypeImageMemoryBarrier, DstAccessMask: vk.AccessTransferWriteBit, OldLayout: vk.ImageLayoutUndefined, NewLayout: vk.ImageLayoutTransferDstOptimal, SrcQueueFamilyIndex: vk.QueueFamilyForeignEXT, DstQueueFamilyIndex: r.gpu.family, Image: img.image, SubresourceRange: colorRange}
134- d.CmdPipelineBarrier(c.cmd, vk.PipelineStageTransferBit, vk.PipelineStageTransferBit, 0, 0, nil, 1, &hb, 1, &ib)160+ d.CmdPipelineBarrier(c.cmd, vk.PipelineStageTransferBit, vk.PipelineStageTransferBit, 0, 0, nil, 1, &c.hb, 1, &c.ib)
135- region0 := vk.BufferImageCopy{161+ c.region = vk.BufferImageCopy{
136 BufferOffset: vk.DeviceSize((region.Min.Y*f.w + region.Min.X) * captureBytesPerPixel),162 BufferOffset: vk.DeviceSize((region.Min.Y*f.w + region.Min.X) * captureBytesPerPixel),
137 BufferRowLength: uint32(f.w),163 BufferRowLength: uint32(f.w),
138 ImageSubresource: vk.ImageSubresourceLayers{AspectMask: vk.ImageAspectColorBit, LayerCount: 1},164 ImageSubresource: vk.ImageSubresourceLayers{AspectMask: vk.ImageAspectColorBit, LayerCount: 1},
139 ImageExtent: vk.Extent3D{Width: uint32(dst.Width), Height: uint32(dst.Height), Depth: 1},165 ImageExtent: vk.Extent3D{Width: uint32(dst.Width), Height: uint32(dst.Height), Depth: 1},
140 }166 }
141- d.CmdCopyBufferToImage(c.cmd, s.buffer, img.image, vk.ImageLayoutTransferDstOptimal, 1, &region0)167+ d.CmdCopyBufferToImage(c.cmd, s.buffer, img.image, vk.ImageLayoutTransferDstOptimal, 1, &c.region)
142- ib.SrcAccessMask, ib.DstAccessMask = vk.AccessTransferWriteBit, 0168+ c.ib.SrcAccessMask, c.ib.DstAccessMask = vk.AccessTransferWriteBit, 0
143- ib.OldLayout, ib.NewLayout = vk.ImageLayoutTransferDstOptimal, vk.ImageLayoutGeneral169+ c.ib.OldLayout, c.ib.NewLayout = vk.ImageLayoutTransferDstOptimal, vk.ImageLayoutGeneral
144- ib.SrcQueueFamilyIndex, ib.DstQueueFamilyIndex = r.gpu.family, vk.QueueFamilyForeignEXT170+ c.ib.SrcQueueFamilyIndex, c.ib.DstQueueFamilyIndex = r.gpu.family, vk.QueueFamilyForeignEXT
145- d.CmdPipelineBarrier(c.cmd, vk.PipelineStageTransferBit, vk.PipelineStageBottomOfPipeBit, 0, 0, nil, 0, nil, 1, &ib)171+ d.CmdPipelineBarrier(c.cmd, vk.PipelineStageTransferBit, vk.PipelineStageBottomOfPipeBit, 0, 0, nil, 0, nil, 1, &c.ib)
146 if err := checked("vkEndCommandBuffer(capture dmabuf)", d.EndCommandBuffer(c.cmd)); err != nil {172 if err := checked("vkEndCommandBuffer(capture dmabuf)", d.EndCommandBuffer(c.cmd)); err != nil {
147 return nil, err173 return nil, err
148 }174 }
149- submit := vk.SubmitInfo{SType: vk.StructureTypeSubmitInfo, CommandBufferCount: 1, CommandBuffers: &c.cmd, SignalSemaphoreCount: 1, SignalSemaphores: &c.done}175+ c.submit = vk.SubmitInfo{SType: vk.StructureTypeSubmitInfo, CommandBufferCount: 1, CommandBuffers: &c.cmd, SignalSemaphoreCount: 1, SignalSemaphores: &c.done}
150- waitStage := vk.PipelineStageFlags(vk.PipelineStageTransferBit)176+ c.waitStage = vk.PipelineStageFlags(vk.PipelineStageTransferBit)
151- if c.wait != 0 {177+ if c.waitUsed {
152- submit.WaitSemaphoreCount, submit.WaitSemaphores, submit.WaitDstStageMask = 1, &c.wait, &waitStage178+ c.submit.WaitSemaphoreCount, c.submit.WaitSemaphores, c.submit.WaitDstStageMask = 1, &c.wait, &c.waitStage
153 }179 }
154- if err := r.q.submit(1, &submit, c.fence); err != nil {180+ if err := r.q.submit(1, &c.submit, c.fence); err != nil {
155 return nil, err181 return nil, err
156 }182 }
157 // From here the copy is tracked by its fence, whatever happens.183 // From here the copy is tracked by its fence, whatever happens.
158 submitted = true184 submitted = true
159 s.copies = append(s.copies, c)185 s.copies = append(s.copies, c)
160- get := vk.SemaphoreGetFdInfoKHR{SType: vk.StructureTypeSemaphoreGetFDInfoKHR, Semaphore: c.done, HandleType: vk.ExternalSemaphoreHandleTypeSyncFDBit}186+ c.get = vk.SemaphoreGetFdInfoKHR{SType: vk.StructureTypeSemaphoreGetFDInfoKHR, Semaphore: c.done, HandleType: vk.ExternalSemaphoreHandleTypeSyncFDBit}
161- fd := int32(-1)187+ c.fd = -1
162- if err := checked("vkGetSemaphoreFdKHR(capture dmabuf)", d.GetSemaphoreFdKHR(dev, &get, &fd)); err != nil {188+ if err := checked("vkGetSemaphoreFdKHR(capture dmabuf)", d.GetSemaphoreFdKHR(dev, &c.get, &c.fd)); err != nil {
163 return nil, err189 return nil, err
164 }190 }
165- if fd < 0 {191+ c.exported = true
192+ if c.fd < 0 {
166 return nil, nil193 return nil, nil
167 }194 }
168- done := os.NewFile(uintptr(fd), "capture-dmabuf-done")195+ // CopyTo transfers this wrapper to its caller, which may still be using it
196+ // after the GPU fence signals. No return notification exists in this API;
197+ // recycling it here would race the pipeline worker's wait/Close.
198+ done := os.NewFile(uintptr(c.fd), "capture-dmabuf-done")
169 // Implicit sync the other way: users that do not wait on done (a199 // Implicit sync the other way: users that do not wait on done (a
170 // PipeWire consumer) wait on the buffer's own fence.200 // PipeWire consumer) wait on the buffer's own fence.
171 attachWriteFence(dst.Planes[0].File, done)201 attachWriteFence(dst.Planes[0].File, done)
@@ -206,8 +236,8 @@ func (d deviceFences) createSemaphore() (vk.Semaphore, error) {
206 }236 }
207 237
208 func (d deviceFences) importSyncFD(s vk.Semaphore, fd int32) vk.Result {238 func (d deviceFences) importSyncFD(s vk.Semaphore, fd int32) vk.Result {
209- imp := vk.ImportSemaphoreFdInfoKHR{SType: vk.StructureTypeImportSemaphoreFDInfoKHR, Semaphore: s, Flags: vk.SemaphoreImportTemporaryBit, HandleType: vk.ExternalSemaphoreHandleTypeSyncFDBit, Fd: fd}239+ d.r.dmaImport = vk.ImportSemaphoreFdInfoKHR{SType: vk.StructureTypeImportSemaphoreFDInfoKHR, Semaphore: s, Flags: vk.SemaphoreImportTemporaryBit, HandleType: vk.ExternalSemaphoreHandleTypeSyncFDBit, Fd: fd}
210- return d.r.gpu.dd.ImportSemaphoreFdKHR(d.r.gpu.device, &imp)240+ return d.r.gpu.dd.ImportSemaphoreFdKHR(d.r.gpu.device, &d.r.dmaImport)
211 }241 }
212 242
213 func (d deviceFences) destroySemaphore(s vk.Semaphore) {243 func (d deviceFences) destroySemaphore(s vk.Semaphore) {
@@ -222,33 +252,6 @@ func (r *Renderer) fences() dmabufFences {
222 return r.dmaFences252 return r.dmaFences
223 }253 }
224 254
225-// dmabufBusy makes a wait semaphore of every fence pending on the dmabuf
226-// of f (DMA_BUF_IOCTL_EXPORT_SYNC_FILE for writing: readers and writers).
227-// 0 with no error only when the kernel reports no fence; any failure to
228-// export or import is an error, so the caller writes nothing. The exported
229-// sync file is closed on every path but a successful import.
230-func (r *Renderer) dmabufBusy(f *os.File) (vk.Semaphore, error) {
231- fs := r.fences()
232- fd, err := fs.exportSyncFile(f)
233- if err != nil {
234- return 0, err
235- }
236- if fd < 0 {
237- return 0, nil
238- }
239- sem, err := fs.createSemaphore()
240- if err != nil {
241- _ = unix.Close(int(fd))
242- return 0, err
243- }
244- if err := checked("vkImportSemaphoreFdKHR(dmabuf wait)", fs.importSyncFD(sem, fd)); err != nil {
245- _ = unix.Close(int(fd))
246- fs.destroySemaphore(sem)
247- return 0, err
248- }
249- return sem, nil
250-}
251-
252 // attachWriteFence adds the sync file done to the dmabuf of f as a write255 // attachWriteFence adds the sync file done to the dmabuf of f as a write
253 // fence (DMA_BUF_IOCTL_IMPORT_SYNC_FILE): implicit-sync readers wait for it.256 // fence (DMA_BUF_IOCTL_IMPORT_SYNC_FILE): implicit-sync readers wait for it.
254 // Best effort: an older kernel leaves the reader to explicit sync.257 // Best effort: an older kernel leaves the reader to explicit sync.
@@ -288,7 +291,7 @@ func (r *Renderer) reapCopies(s *captureSlot) error {
288 for _, c := range s.copies {291 for _, c := range s.copies {
289 switch res := r.gpu.dd.WaitForFences(r.gpu.device, 1, &c.fence, 1, 0); res {292 switch res := r.gpu.dd.WaitForFences(r.gpu.device, 1, &c.fence, 1, 0); res {
290 case vk.Success:293 case vk.Success:
291- r.freeDMACopy(c)294+ r.recycleDMACopy(c)
292 case vk.Timeout, vk.NotReady:295 case vk.Timeout, vk.NotReady:
293 kept = append(kept, c)296 kept = append(kept, c)
294 default:297 default:
@@ -318,8 +321,62 @@ func (r *Renderer) freeDMACopy(c *dmaCopy) {
318 if c.cmd != 0 {321 if c.cmd != 0 {
319 d.FreeCommandBuffers(dev, r.pool, 1, &c.cmd)322 d.FreeCommandBuffers(dev, r.pool, 1, &c.cmd)
320 }323 }
321- if c.img != nil {324+ if c.cached != nil {
325+ r.gpu.releaseCastImage(c.cached)
326+ } else if c.img != nil {
322 r.gpu.freeImage(c.img)327 r.gpu.freeImage(c.img)
323 }328 }
324 *c = dmaCopy{}329 *c = dmaCopy{}
325 }330 }
331+
332+// prepareDMAWait fails closed unless all implicit fences can be exported
333+// and imported. It imports a fresh temporary payload each frame. A completed
334+// queue wait consumed the previous payload before this object entered dmaIdle.
335+func (r *Renderer) prepareDMAWait(c *dmaCopy, f *os.File) error {
336+ fs := r.fences()
337+ fd, err := fs.exportSyncFile(f)
338+ if err != nil {
339+ return err
340+ }
341+ if fd < 0 {
342+ return nil
343+ }
344+ if c.wait == 0 {
345+ c.wait, err = fs.createSemaphore()
346+ if err != nil {
347+ unix.Close(int(fd))
348+ return err
349+ }
350+ }
351+ if err = checked("vkImportSemaphoreFdKHR(dmabuf wait)", fs.importSyncFD(c.wait, fd)); err != nil {
352+ unix.Close(int(fd))
353+ return err
354+ }
355+ c.waitUsed = true
356+ return nil
357+}
358+
359+func (r *Renderer) recycleDMACopy(c *dmaCopy) {
360+ // Failed sync-file export leaves a signalled binary semaphore payload;
361+ // it must be destroyed, not signalled a second time.
362+ if !c.exported {
363+ r.freeDMACopy(c)
364+ return
365+ }
366+ if c.cached != nil {
367+ r.gpu.releaseCastImage(c.cached)
368+ } else if c.img != nil {
369+ r.gpu.freeImage(c.img)
370+ }
371+ c.cached, c.img = nil, nil
372+ d := r.gpu.dd
373+ if d.ResetFences(r.gpu.device, 1, &c.fence) != vk.Success || d.ResetCommandBuffer(c.cmd, 0) != vk.Success {
374+ r.freeDMACopy(c)
375+ return
376+ }
377+ if len(r.dmaIdle) >= captureSlots*maxSlotCopies {
378+ r.freeDMACopy(c)
379+ return
380+ }
381+ r.dmaIdle = append(r.dmaIdle, c)
382+}
A internal/adapters/vulkan/capture_import_test.go +141 -0
@@ -0,0 +1,141 @@
1+package vulkan
2+
3+import (
4+ "context"
5+ "github.com/bnema/neferwl/internal/adapters/syncfile"
6+ "github.com/bnema/neferwl/internal/domain"
7+ "image"
8+ "testing"
9+)
10+
11+func TestCaptureDMABufCopyAllocations(t *testing.T) {
12+ if raceEnabled {
13+ t.Skip("allocation measurements require non-race runtime")
14+ }
15+ r := newTestRenderer(t, 16, 8)
16+ defer r.Close()
17+ requireCaptureSync(t, r)
18+ buf := castLinear(t, r.gpu, domain.CastXRGB8888, 16, 8)
19+ if err := render(r, domain.Scene{Background: "#3060c0"}, nil); err != nil {
20+ t.Fatal(err)
21+ }
22+ cf := captureWait(t, r)
23+ defer r.EndCapture(cf)
24+ cycle := func() {
25+ castSubmit(t, cf, image.Rect(0, 0, 16, 8), buf)
26+ if err := r.reapCopies(cf.(*captureFrame).slot); err != nil {
27+ t.Fatal(err)
28+ }
29+ }
30+ cycle()
31+ // os.NewFile allocates its public wrapper and internal file (2 total).
32+ // The caller owns Close; GPU completion cannot authorize wrapper reuse.
33+ if allocs := testing.AllocsPerRun(100, cycle); allocs > 2 {
34+ t.Fatalf("%.1f allocs/copy, budget 2 caller-owned file allocations", allocs)
35+ }
36+ before := openFDs(t)
37+ result := testing.Benchmark(func(b *testing.B) {
38+ for b.Loop() {
39+ cycle()
40+ }
41+ })
42+ t.Logf("%s %s", result.String(), result.MemString())
43+ if after := openFDs(t); after != before {
44+ t.Fatalf("fd count %d -> %d", before, after)
45+ }
46+}
47+
48+func TestCaptureRetainedCastImage(t *testing.T) {
49+ r := newTestRenderer(t, 16, 8)
50+ defer r.Close()
51+ requireCaptureSync(t, r)
52+ buf := castLinear(t, r.gpu, domain.CastXRGB8888, 16, 8)
53+ allocationFDs := openFDs(t)
54+ e := r.gpu.castImages[buf.ID]
55+ if buf.ID == 0 || e == nil || e.img == nil {
56+ t.Fatal("allocation not retained")
57+ }
58+ original := e.img
59+ if err := render(r, domain.Scene{Background: "#3060c0"}, nil); err != nil {
60+ t.Fatal(err)
61+ }
62+ cf := captureWait(t, r)
63+ defer r.EndCapture(cf)
64+ slot := cf.(*captureFrame).slot
65+ for range 20 {
66+ castSubmit(t, cf, image.Rect(0, 0, 16, 8), buf)
67+ if len(slot.copies) != 1 || slot.copies[0].img != original || slot.copies[0].cached != e {
68+ t.Fatal("copy imported instead of using exporting image")
69+ }
70+ if err := r.reapCopies(slot); err != nil {
71+ t.Fatal(err)
72+ }
73+ }
74+ warmed := openFDs(t)
75+ // Keep a submitted copy reference across removal, even if this fast GPU
76+ // already completed it. Only fence reclamation may release that reference.
77+ done, err := cf.CopyTo(image.Rect(0, 0, 16, 8), buf)
78+ if err != nil {
79+ t.Fatal(err)
80+ }
81+ r.gpu.ReleaseCast(buf)
82+ if r.gpu.castImages[buf.ID] != nil || !e.released || e.img != original || e.refs != 1 {
83+ t.Fatal("release destroyed referenced image")
84+ }
85+ if done != nil {
86+ if err := syncfile.Wait(context.Background(), done); err != nil {
87+ t.Fatal(err)
88+ }
89+ done.Close()
90+ }
91+ if err := r.reapCopies(slot); err != nil {
92+ t.Fatal(err)
93+ }
94+ if e.refs != 0 || e.img != nil || original.image != 0 || original.memory != 0 {
95+ t.Fatal("completed removal retained resources")
96+ }
97+ for _, p := range buf.Planes {
98+ p.File.Close()
99+ }
100+ r.EndCapture(cf)
101+ r.Close()
102+ if after := openFDs(t); after >= allocationFDs || after >= warmed {
103+ t.Fatalf("fds allocated=%d warmed=%d released=%d", allocationFDs, warmed, after)
104+ }
105+}
106+
107+func TestCaptureDMACopyObjectReuse(t *testing.T) {
108+ r := newTestRenderer(t, 16, 8)
109+ defer r.Close()
110+ requireCaptureSync(t, r)
111+ buf := castLinear(t, r.gpu, domain.CastXRGB8888, 16, 8)
112+ if err := render(r, domain.Scene{Background: "#3060c0"}, nil); err != nil {
113+ t.Fatal(err)
114+ }
115+ cf := captureWait(t, r)
116+ defer r.EndCapture(cf)
117+ slot := cf.(*captureFrame).slot
118+ castSubmit(t, cf, image.Rect(0, 0, 16, 8), buf)
119+ first := slot.copies[0]
120+ cmd, fence, done, wait := first.cmd, first.fence, first.done, first.wait
121+ if err := r.reapCopies(slot); err != nil {
122+ t.Fatal(err)
123+ }
124+ for range 20 {
125+ castSubmit(t, cf, image.Rect(0, 0, 16, 8), buf)
126+ c := slot.copies[0]
127+ if c != first || c.cmd != cmd || c.fence != fence || c.done != done || c.wait != wait {
128+ t.Fatal("completed copy objects not reused")
129+ }
130+ if err := r.reapCopies(slot); err != nil {
131+ t.Fatal(err)
132+ }
133+ }
134+ // Export failure leaves a binary signal unconsumed: discard that object.
135+ first.exported = false
136+ r.dmaIdle = nil
137+ r.recycleDMACopy(first)
138+ if first.cmd != 0 || first.done != 0 || len(r.dmaIdle) != 0 {
139+ t.Fatal("unconsumed signal recycled")
140+ }
141+}
@@ -0,0 +1,141 @@
1+package vulkan
2+
3+import (
4+ "context"
5+ "github.com/bnema/neferwl/internal/adapters/syncfile"
6+ "github.com/bnema/neferwl/internal/domain"
7+ "image"
8+ "testing"
9+)
10+
11+func TestCaptureDMABufCopyAllocations(t *testing.T) {
12+ if raceEnabled {
13+ t.Skip("allocation measurements require non-race runtime")
14+ }
15+ r := newTestRenderer(t, 16, 8)
16+ defer r.Close()
17+ requireCaptureSync(t, r)
18+ buf := castLinear(t, r.gpu, domain.CastXRGB8888, 16, 8)
19+ if err := render(r, domain.Scene{Background: "#3060c0"}, nil); err != nil {
20+ t.Fatal(err)
21+ }
22+ cf := captureWait(t, r)
23+ defer r.EndCapture(cf)
24+ cycle := func() {
25+ castSubmit(t, cf, image.Rect(0, 0, 16, 8), buf)
26+ if err := r.reapCopies(cf.(*captureFrame).slot); err != nil {
27+ t.Fatal(err)
28+ }
29+ }
30+ cycle()
31+ // os.NewFile allocates its public wrapper and internal file (2 total).
32+ // The caller owns Close; GPU completion cannot authorize wrapper reuse.
33+ if allocs := testing.AllocsPerRun(100, cycle); allocs > 2 {
34+ t.Fatalf("%.1f allocs/copy, budget 2 caller-owned file allocations", allocs)
35+ }
36+ before := openFDs(t)
37+ result := testing.Benchmark(func(b *testing.B) {
38+ for b.Loop() {
39+ cycle()
40+ }
41+ })
42+ t.Logf("%s %s", result.String(), result.MemString())
43+ if after := openFDs(t); after != before {
44+ t.Fatalf("fd count %d -> %d", before, after)
45+ }
46+}
47+
48+func TestCaptureRetainedCastImage(t *testing.T) {
49+ r := newTestRenderer(t, 16, 8)
50+ defer r.Close()
51+ requireCaptureSync(t, r)
52+ buf := castLinear(t, r.gpu, domain.CastXRGB8888, 16, 8)
53+ allocationFDs := openFDs(t)
54+ e := r.gpu.castImages[buf.ID]
55+ if buf.ID == 0 || e == nil || e.img == nil {
56+ t.Fatal("allocation not retained")
57+ }
58+ original := e.img
59+ if err := render(r, domain.Scene{Background: "#3060c0"}, nil); err != nil {
60+ t.Fatal(err)
61+ }
62+ cf := captureWait(t, r)
63+ defer r.EndCapture(cf)
64+ slot := cf.(*captureFrame).slot
65+ for range 20 {
66+ castSubmit(t, cf, image.Rect(0, 0, 16, 8), buf)
67+ if len(slot.copies) != 1 || slot.copies[0].img != original || slot.copies[0].cached != e {
68+ t.Fatal("copy imported instead of using exporting image")
69+ }
70+ if err := r.reapCopies(slot); err != nil {
71+ t.Fatal(err)
72+ }
73+ }
74+ warmed := openFDs(t)
75+ // Keep a submitted copy reference across removal, even if this fast GPU
76+ // already completed it. Only fence reclamation may release that reference.
77+ done, err := cf.CopyTo(image.Rect(0, 0, 16, 8), buf)
78+ if err != nil {
79+ t.Fatal(err)
80+ }
81+ r.gpu.ReleaseCast(buf)
82+ if r.gpu.castImages[buf.ID] != nil || !e.released || e.img != original || e.refs != 1 {
83+ t.Fatal("release destroyed referenced image")
84+ }
85+ if done != nil {
86+ if err := syncfile.Wait(context.Background(), done); err != nil {
87+ t.Fatal(err)
88+ }
89+ done.Close()
90+ }
91+ if err := r.reapCopies(slot); err != nil {
92+ t.Fatal(err)
93+ }
94+ if e.refs != 0 || e.img != nil || original.image != 0 || original.memory != 0 {
95+ t.Fatal("completed removal retained resources")
96+ }
97+ for _, p := range buf.Planes {
98+ p.File.Close()
99+ }
100+ r.EndCapture(cf)
101+ r.Close()
102+ if after := openFDs(t); after >= allocationFDs || after >= warmed {
103+ t.Fatalf("fds allocated=%d warmed=%d released=%d", allocationFDs, warmed, after)
104+ }
105+}
106+
107+func TestCaptureDMACopyObjectReuse(t *testing.T) {
108+ r := newTestRenderer(t, 16, 8)
109+ defer r.Close()
110+ requireCaptureSync(t, r)
111+ buf := castLinear(t, r.gpu, domain.CastXRGB8888, 16, 8)
112+ if err := render(r, domain.Scene{Background: "#3060c0"}, nil); err != nil {
113+ t.Fatal(err)
114+ }
115+ cf := captureWait(t, r)
116+ defer r.EndCapture(cf)
117+ slot := cf.(*captureFrame).slot
118+ castSubmit(t, cf, image.Rect(0, 0, 16, 8), buf)
119+ first := slot.copies[0]
120+ cmd, fence, done, wait := first.cmd, first.fence, first.done, first.wait
121+ if err := r.reapCopies(slot); err != nil {
122+ t.Fatal(err)
123+ }
124+ for range 20 {
125+ castSubmit(t, cf, image.Rect(0, 0, 16, 8), buf)
126+ c := slot.copies[0]
127+ if c != first || c.cmd != cmd || c.fence != fence || c.done != done || c.wait != wait {
128+ t.Fatal("completed copy objects not reused")
129+ }
130+ if err := r.reapCopies(slot); err != nil {
131+ t.Fatal(err)
132+ }
133+ }
134+ // Export failure leaves a binary signal unconsumed: discard that object.
135+ first.exported = false
136+ r.dmaIdle = nil
137+ r.recycleDMACopy(first)
138+ if first.cmd != 0 || first.done != 0 || len(r.dmaIdle) != 0 {
139+ t.Fatal("unconsumed signal recycled")
140+ }
141+}
A internal/adapters/vulkan/capture_trim_test.go +119 -0
@@ -0,0 +1,119 @@
1+package vulkan
2+
3+import (
4+ vk "bnema.dev/purego-vulkan/vulkan"
5+ "image"
6+ "testing"
7+ "time"
8+
9+ "github.com/bnema/neferwl/internal/domain"
10+)
11+
12+func TestTrimCaptureResources(t *testing.T) {
13+ for _, mode := range []string{"sdr", "cursor", "hdr", "pq", "pq-cursor"} {
14+ t.Run(mode, func(t *testing.T) {
15+ var r *Renderer
16+ if mode == "hdr" || mode == "pq" || mode == "pq-cursor" {
17+ r = hdrTestRenderer(t)
18+ } else {
19+ r = newTestRenderer(t, 16, 8)
20+ }
21+ requireCaptureSync(t, r)
22+ if err := render(r, domain.Scene{Background: "#3060c0"}, nil); err != nil {
23+ t.Fatal(err)
24+ }
25+ trimSettled(t, r, time.Time{})
26+ baseline := openFDs(t)
27+ pq := mode == "pq" || mode == "pq-cursor"
28+ var cursor *domain.CaptureCursor
29+ if mode == "cursor" || mode == "pq-cursor" {
30+ cursor = &domain.CaptureCursor{X: 3, Y: 4, Image: domain.CursorImage{W: 1, H: 1, Pixels: []byte{0, 0, 255, 255}}}
31+ }
32+ for range 2 { // A capture immediately after trim recreates everything lazily.
33+ f, err := r.beginCaptureCursor(false, pq, cursor)
34+ if err != nil {
35+ t.Fatal(err)
36+ }
37+ waitCaptureSlot(t, r, f)
38+ if err := r.Trim(time.Time{}); err != nil {
39+ t.Fatal(err)
40+ }
41+ if len(r.captures) == 0 {
42+ t.Fatal("trim destroyed a leased capture")
43+ }
44+ dst := make([]byte, 16*8*4)
45+ if err := f.Read(image.Rect(0, 0, 16, 8), dst, 16*4); err != nil {
46+ t.Fatal(err)
47+ }
48+ r.EndCapture(f)
49+ if err := r.Trim(time.Time{}); err != nil {
50+ t.Fatal(err)
51+ }
52+ if len(r.captures) != 0 || len(r.dmaIdle) != 0 || r.capturePass.pipeline != 0 || r.captureSDR.image != 0 || r.capturePQ.pipeline != 0 || r.capturePQ.out.image != 0 {
53+ t.Fatal("capture resources survived trim")
54+ }
55+ if after := openFDs(t); after != baseline {
56+ t.Fatalf("fds baseline=%d after trim=%d", baseline, after)
57+ }
58+ }
59+ })
60+ }
61+}
62+
63+func TestTrimCaptureDMACopies(t *testing.T) {
64+ r := newTestRenderer(t, 16, 8)
65+ requireCaptureSync(t, r)
66+ if err := render(r, domain.Scene{Background: "#3060c0"}, nil); err != nil {
67+ t.Fatal(err)
68+ }
69+ trimSettled(t, r, time.Time{})
70+ baseline := openFDs(t)
71+ buf := castLinear(t, r.gpu, domain.CastXRGB8888, 16, 8)
72+ retained := r.gpu.castImages[buf.ID]
73+ f := captureWait(t, r).(*captureFrame)
74+ castSubmit(t, f, image.Rect(0, 0, 16, 8), buf)
75+ r.gpu.ReleaseCast(buf)
76+ for _, p := range buf.Planes {
77+ p.File.Close()
78+ }
79+ r.EndCapture(f)
80+ if err := r.Trim(time.Time{}); err != nil {
81+ t.Fatal(err)
82+ }
83+ if len(r.captures) != 0 || len(r.dmaIdle) != 0 || retained.img != nil || retained.refs != 0 {
84+ t.Fatal("trim retained released cast or copy resources")
85+ }
86+ if after := openFDs(t); after != baseline {
87+ t.Fatalf("fds baseline=%d after trim=%d", baseline, after)
88+ }
89+}
90+
91+func TestTrimCapturePendingFence(t *testing.T) {
92+ r := newTestRenderer(t, 16, 8)
93+ if err := render(r, domain.Scene{Background: "#3060c0"}, nil); err != nil {
94+ t.Fatal(err)
95+ }
96+ f, err := r.beginCaptureAs(true, false)
97+ if err != nil {
98+ t.Fatal(err)
99+ }
100+ waitCaptureSlot(t, r, f)
101+ r.EndCapture(f)
102+ saved := r.sync
103+ m := newMockcaptureSync(t)
104+ m.EXPECT().fenceStatus(f.slot.fence).Return(vk.Timeout).Once()
105+ r.sync = m
106+ if err := r.Trim(time.Time{}); err != nil {
107+ t.Fatal(err)
108+ }
109+ if len(r.captures) == 0 {
110+ t.Fatal("trim destroyed pending capture")
111+ }
112+ r.sync = saved
113+ if err := r.Trim(time.Time{}); err != nil {
114+ t.Fatal(err)
115+ }
116+ if len(r.captures) != 0 {
117+ t.Fatal("trim retained completed capture")
118+ }
119+}
@@ -0,0 +1,119 @@
1+package vulkan
2+
3+import (
4+ vk "bnema.dev/purego-vulkan/vulkan"
5+ "image"
6+ "testing"
7+ "time"
8+
9+ "github.com/bnema/neferwl/internal/domain"
10+)
11+
12+func TestTrimCaptureResources(t *testing.T) {
13+ for _, mode := range []string{"sdr", "cursor", "hdr", "pq", "pq-cursor"} {
14+ t.Run(mode, func(t *testing.T) {
15+ var r *Renderer
16+ if mode == "hdr" || mode == "pq" || mode == "pq-cursor" {
17+ r = hdrTestRenderer(t)
18+ } else {
19+ r = newTestRenderer(t, 16, 8)
20+ }
21+ requireCaptureSync(t, r)
22+ if err := render(r, domain.Scene{Background: "#3060c0"}, nil); err != nil {
23+ t.Fatal(err)
24+ }
25+ trimSettled(t, r, time.Time{})
26+ baseline := openFDs(t)
27+ pq := mode == "pq" || mode == "pq-cursor"
28+ var cursor *domain.CaptureCursor
29+ if mode == "cursor" || mode == "pq-cursor" {
30+ cursor = &domain.CaptureCursor{X: 3, Y: 4, Image: domain.CursorImage{W: 1, H: 1, Pixels: []byte{0, 0, 255, 255}}}
31+ }
32+ for range 2 { // A capture immediately after trim recreates everything lazily.
33+ f, err := r.beginCaptureCursor(false, pq, cursor)
34+ if err != nil {
35+ t.Fatal(err)
36+ }
37+ waitCaptureSlot(t, r, f)
38+ if err := r.Trim(time.Time{}); err != nil {
39+ t.Fatal(err)
40+ }
41+ if len(r.captures) == 0 {
42+ t.Fatal("trim destroyed a leased capture")
43+ }
44+ dst := make([]byte, 16*8*4)
45+ if err := f.Read(image.Rect(0, 0, 16, 8), dst, 16*4); err != nil {
46+ t.Fatal(err)
47+ }
48+ r.EndCapture(f)
49+ if err := r.Trim(time.Time{}); err != nil {
50+ t.Fatal(err)
51+ }
52+ if len(r.captures) != 0 || len(r.dmaIdle) != 0 || r.capturePass.pipeline != 0 || r.captureSDR.image != 0 || r.capturePQ.pipeline != 0 || r.capturePQ.out.image != 0 {
53+ t.Fatal("capture resources survived trim")
54+ }
55+ if after := openFDs(t); after != baseline {
56+ t.Fatalf("fds baseline=%d after trim=%d", baseline, after)
57+ }
58+ }
59+ })
60+ }
61+}
62+
63+func TestTrimCaptureDMACopies(t *testing.T) {
64+ r := newTestRenderer(t, 16, 8)
65+ requireCaptureSync(t, r)
66+ if err := render(r, domain.Scene{Background: "#3060c0"}, nil); err != nil {
67+ t.Fatal(err)
68+ }
69+ trimSettled(t, r, time.Time{})
70+ baseline := openFDs(t)
71+ buf := castLinear(t, r.gpu, domain.CastXRGB8888, 16, 8)
72+ retained := r.gpu.castImages[buf.ID]
73+ f := captureWait(t, r).(*captureFrame)
74+ castSubmit(t, f, image.Rect(0, 0, 16, 8), buf)
75+ r.gpu.ReleaseCast(buf)
76+ for _, p := range buf.Planes {
77+ p.File.Close()
78+ }
79+ r.EndCapture(f)
80+ if err := r.Trim(time.Time{}); err != nil {
81+ t.Fatal(err)
82+ }
83+ if len(r.captures) != 0 || len(r.dmaIdle) != 0 || retained.img != nil || retained.refs != 0 {
84+ t.Fatal("trim retained released cast or copy resources")
85+ }
86+ if after := openFDs(t); after != baseline {
87+ t.Fatalf("fds baseline=%d after trim=%d", baseline, after)
88+ }
89+}
90+
91+func TestTrimCapturePendingFence(t *testing.T) {
92+ r := newTestRenderer(t, 16, 8)
93+ if err := render(r, domain.Scene{Background: "#3060c0"}, nil); err != nil {
94+ t.Fatal(err)
95+ }
96+ f, err := r.beginCaptureAs(true, false)
97+ if err != nil {
98+ t.Fatal(err)
99+ }
100+ waitCaptureSlot(t, r, f)
101+ r.EndCapture(f)
102+ saved := r.sync
103+ m := newMockcaptureSync(t)
104+ m.EXPECT().fenceStatus(f.slot.fence).Return(vk.Timeout).Once()
105+ r.sync = m
106+ if err := r.Trim(time.Time{}); err != nil {
107+ t.Fatal(err)
108+ }
109+ if len(r.captures) == 0 {
110+ t.Fatal("trim destroyed pending capture")
111+ }
112+ r.sync = saved
113+ if err := r.Trim(time.Time{}); err != nil {
114+ t.Fatal(err)
115+ }
116+ if len(r.captures) != 0 {
117+ t.Fatal("trim retained completed capture")
118+ }
119+}
M internal/adapters/vulkan/cast_alloc.go +80 -7
@@ -6,6 +6,7 @@ import (
66 "math"
77 "os"
88 "slices"
9+ "sync/atomic"
910 "unsafe"
1011
1112 "github.com/bnema/neferwl/internal/domain"
@@ -16,9 +17,11 @@ import (
1617 // Screencast buffers (ports.CastBufferAllocator). The PipeWire producer
1718 // offers only DMA-BUF buffers, allocated here: one exported image per
1819 // buffer, of a modifier the device can export and copy captures into
19-// (CaptureFrame.CopyTo, a transfer destination). The exported file keeps
20-// the memory alive, so the image and memory objects are released at once:
21-// a buffer is nothing but its plane files, closed by the caller.
20+// (CaptureFrame.CopyTo, a transfer destination). The GPU keeps the
21+// exporting image and memory, keyed by the buffer ID, so CopyTo writes
22+// into them without importing the planes again. The caller calls
23+// ReleaseCast before closing the plane files; outstanding copies defer the
24+// destruction of the image and memory until their fences are reclaimed.
2225
2326 // castFourccs are the cast formats, preferred first within an offer:
2427 // XBGR2101010 carries the PQ words of an HDR frame, the 8-bit formats the
@@ -86,7 +89,8 @@ var errCastClosed = errors.New("vulkan: GPU closed")
8689
8790 // AllocCast exports a w×h image of a cast format with modifier, one listed
8891 // by CastFormats, and returns it as a dmabuf whose plane files are the
89-// caller's (every plane a file of the one memory object). It may run on
92+// caller's (every plane a file of the one memory object). ReleaseCast must
93+// precede closing them. It may run on
9094 // any goroutine.
9195 func (g *GPU) AllocCast(format uint32, w, h int, modifier uint64) (domain.DMABuf, error) {
9296 if w <= 0 || h <= 0 || w > math.MaxUint16 || h > math.MaxUint16 {
@@ -108,11 +112,16 @@ func (g *GPU) AllocCast(format uint32, w, h int, modifier uint64) (domain.DMABuf
108112 return g.exportCast(vf, format, w, h, modifier, planes)
109113 }
110114
111-// exportCast creates, exports and releases one cast image. g.poolMu is held.
115+// exportCast creates and retains one cast image until ReleaseCast. g.poolMu is held.
112116 func (g *GPU) exportCast(vf vk.Format, format uint32, w, h int, modifier uint64, n uint32) (domain.DMABuf, error) {
113117 d, dev := g.dd, g.device
114118 t := &target{}
115- defer g.freeImage(t)
119+ retained := false
120+ defer func() {
121+ if !retained {
122+ g.freeImage(t)
123+ }
124+ }()
116125 list := vk.ImageDrmFormatModifierListCreateInfoEXT{SType: vk.StructureTypeImageDRMFormatModifierListCreateInfoEXT, DrmFormatModifierCount: 1, DrmFormatModifiers: &modifier}
117126 external := vk.ExternalMemoryImageCreateInfo{SType: vk.StructureTypeExternalMemoryImageCreateInfo, Next: unsafe.Pointer(&list), HandleTypes: vk.ExternalMemoryHandleTypeDMABUFBitEXT}
118127 ii := vk.ImageCreateInfo{SType: vk.StructureTypeImageCreateInfo, Next: unsafe.Pointer(&external), ImageType: vk.ImageType2d, Format: vf, Extent: vk.Extent3D{Width: uint32(w), Height: uint32(h), Depth: 1}, MipLevels: 1, ArrayLayers: 1, Samples: vk.SampleCount1Bit, Tiling: vk.ImageTilingDRMFormatModifierEXT, Usage: castUsage, SharingMode: vk.SharingModeExclusive, InitialLayout: vk.ImageLayoutUndefined}
@@ -167,5 +176,69 @@ func (g *GPU) exportCast(vf vk.Format, format uint32, w, h int, modifier uint64,
167176 }
168177 planes = append(planes, domain.DMABufPlane{File: os.NewFile(uintptr(dup), "neferwl-cast"), Offset: uint32(layouts[i].Offset), Stride: uint32(layouts[i].RowPitch)})
169178 }
170- return domain.DMABuf{Width: w, Height: h, Format: format, Modifier: modifier, Planes: planes}, nil
179+ buf := domain.DMABuf{ID: castBufferID.Add(1), Width: w, Height: h, Format: format, Modifier: modifier, Planes: planes}
180+ if g.castImages == nil {
181+ g.castImages = make(map[uint64]*castImage)
182+ }
183+ g.castImages[buf.ID] = &castImage{buffer: buf, img: t}
184+ retained = true
185+ return buf, nil
186+}
187+
188+// castBufferID is process-wide, so cast identities cannot collide across GPUs.
189+var castBufferID atomic.Uint64
190+
191+type castImage struct {
192+ buffer domain.DMABuf
193+ img *target
194+ refs int
195+ released bool
196+}
197+
198+// ReleaseCast is safe on the PipeWire owner while output owners finish copies.
199+func (g *GPU) ReleaseCast(buf domain.DMABuf) {
200+ g.poolMu.Lock()
201+ defer g.poolMu.Unlock()
202+ e := g.castImages[buf.ID]
203+ if e == nil {
204+ return
205+ }
206+ delete(g.castImages, buf.ID)
207+ e.released = true
208+ if e.refs == 0 && e.img != nil {
209+ g.freeImage(e.img)
210+ e.img = nil
211+ }
212+}
213+
214+func (g *GPU) acquireCastImage(buf *domain.DMABuf) (*target, *castImage, error) {
215+ g.poolMu.Lock()
216+ defer g.poolMu.Unlock()
217+ e := g.castImages[buf.ID]
218+ if e == nil {
219+ img, err := g.importBlitImage(buf, buf.Width, buf.Height, vk.ImageUsageTransferDstBit, formatFeatureTransferDst)
220+ return img, nil, err
221+ }
222+ // Registered IDs name an immutable allocation, not arbitrary request geometry.
223+ if buf.Width != e.buffer.Width || buf.Height != e.buffer.Height || buf.Format != e.buffer.Format || buf.Modifier != e.buffer.Modifier || len(buf.Planes) != len(e.buffer.Planes) {
224+ return nil, nil, errImageGeometry
225+ }
226+ for i, p := range buf.Planes {
227+ if p != e.buffer.Planes[i] {
228+ return nil, nil, errImagePlanes
229+ }
230+ }
231+
232+ e.refs++
233+ return e.img, e, nil
234+}
235+
236+func (g *GPU) releaseCastImage(e *castImage) {
237+ g.poolMu.Lock()
238+ defer g.poolMu.Unlock()
239+ e.refs--
240+ if e.released && e.refs == 0 && e.img != nil {
241+ g.freeImage(e.img)
242+ e.img = nil
243+ }
171244 }
@@ -6,6 +6,7 @@ import (
6 "math"6 "math"
7 "os"7 "os"
8 "slices"8 "slices"
9+ "sync/atomic"
9 "unsafe"10 "unsafe"
10 11
11 "github.com/bnema/neferwl/internal/domain"12 "github.com/bnema/neferwl/internal/domain"
@@ -16,9 +17,11 @@ import (
16 // Screencast buffers (ports.CastBufferAllocator). The PipeWire producer17 // Screencast buffers (ports.CastBufferAllocator). The PipeWire producer
17 // offers only DMA-BUF buffers, allocated here: one exported image per18 // offers only DMA-BUF buffers, allocated here: one exported image per
18 // buffer, of a modifier the device can export and copy captures into19 // buffer, of a modifier the device can export and copy captures into
19-// (CaptureFrame.CopyTo, a transfer destination). The exported file keeps20+// (CaptureFrame.CopyTo, a transfer destination). The GPU keeps the
20-// the memory alive, so the image and memory objects are released at once:21+// exporting image and memory, keyed by the buffer ID, so CopyTo writes
21-// a buffer is nothing but its plane files, closed by the caller.22+// into them without importing the planes again. The caller calls
23+// ReleaseCast before closing the plane files; outstanding copies defer the
24+// destruction of the image and memory until their fences are reclaimed.
22 25
23 // castFourccs are the cast formats, preferred first within an offer:26 // castFourccs are the cast formats, preferred first within an offer:
24 // XBGR2101010 carries the PQ words of an HDR frame, the 8-bit formats the27 // XBGR2101010 carries the PQ words of an HDR frame, the 8-bit formats the
@@ -86,7 +89,8 @@ var errCastClosed = errors.New("vulkan: GPU closed")
86 89
87 // AllocCast exports a w×h image of a cast format with modifier, one listed90 // AllocCast exports a w×h image of a cast format with modifier, one listed
88 // by CastFormats, and returns it as a dmabuf whose plane files are the91 // by CastFormats, and returns it as a dmabuf whose plane files are the
89-// caller's (every plane a file of the one memory object). It may run on92+// caller's (every plane a file of the one memory object). ReleaseCast must
93+// precede closing them. It may run on
90 // any goroutine.94 // any goroutine.
91 func (g *GPU) AllocCast(format uint32, w, h int, modifier uint64) (domain.DMABuf, error) {95 func (g *GPU) AllocCast(format uint32, w, h int, modifier uint64) (domain.DMABuf, error) {
92 if w <= 0 || h <= 0 || w > math.MaxUint16 || h > math.MaxUint16 {96 if w <= 0 || h <= 0 || w > math.MaxUint16 || h > math.MaxUint16 {
@@ -108,11 +112,16 @@ func (g *GPU) AllocCast(format uint32, w, h int, modifier uint64) (domain.DMABuf
108 return g.exportCast(vf, format, w, h, modifier, planes)112 return g.exportCast(vf, format, w, h, modifier, planes)
109 }113 }
110 114
111-// exportCast creates, exports and releases one cast image. g.poolMu is held.115+// exportCast creates and retains one cast image until ReleaseCast. g.poolMu is held.
112 func (g *GPU) exportCast(vf vk.Format, format uint32, w, h int, modifier uint64, n uint32) (domain.DMABuf, error) {116 func (g *GPU) exportCast(vf vk.Format, format uint32, w, h int, modifier uint64, n uint32) (domain.DMABuf, error) {
113 d, dev := g.dd, g.device117 d, dev := g.dd, g.device
114 t := &target{}118 t := &target{}
115- defer g.freeImage(t)119+ retained := false
120+ defer func() {
121+ if !retained {
122+ g.freeImage(t)
123+ }
124+ }()
116 list := vk.ImageDrmFormatModifierListCreateInfoEXT{SType: vk.StructureTypeImageDRMFormatModifierListCreateInfoEXT, DrmFormatModifierCount: 1, DrmFormatModifiers: &modifier}125 list := vk.ImageDrmFormatModifierListCreateInfoEXT{SType: vk.StructureTypeImageDRMFormatModifierListCreateInfoEXT, DrmFormatModifierCount: 1, DrmFormatModifiers: &modifier}
117 external := vk.ExternalMemoryImageCreateInfo{SType: vk.StructureTypeExternalMemoryImageCreateInfo, Next: unsafe.Pointer(&list), HandleTypes: vk.ExternalMemoryHandleTypeDMABUFBitEXT}126 external := vk.ExternalMemoryImageCreateInfo{SType: vk.StructureTypeExternalMemoryImageCreateInfo, Next: unsafe.Pointer(&list), HandleTypes: vk.ExternalMemoryHandleTypeDMABUFBitEXT}
118 ii := vk.ImageCreateInfo{SType: vk.StructureTypeImageCreateInfo, Next: unsafe.Pointer(&external), ImageType: vk.ImageType2d, Format: vf, Extent: vk.Extent3D{Width: uint32(w), Height: uint32(h), Depth: 1}, MipLevels: 1, ArrayLayers: 1, Samples: vk.SampleCount1Bit, Tiling: vk.ImageTilingDRMFormatModifierEXT, Usage: castUsage, SharingMode: vk.SharingModeExclusive, InitialLayout: vk.ImageLayoutUndefined}127 ii := vk.ImageCreateInfo{SType: vk.StructureTypeImageCreateInfo, Next: unsafe.Pointer(&external), ImageType: vk.ImageType2d, Format: vf, Extent: vk.Extent3D{Width: uint32(w), Height: uint32(h), Depth: 1}, MipLevels: 1, ArrayLayers: 1, Samples: vk.SampleCount1Bit, Tiling: vk.ImageTilingDRMFormatModifierEXT, Usage: castUsage, SharingMode: vk.SharingModeExclusive, InitialLayout: vk.ImageLayoutUndefined}
@@ -167,5 +176,69 @@ func (g *GPU) exportCast(vf vk.Format, format uint32, w, h int, modifier uint64,
167 }176 }
168 planes = append(planes, domain.DMABufPlane{File: os.NewFile(uintptr(dup), "neferwl-cast"), Offset: uint32(layouts[i].Offset), Stride: uint32(layouts[i].RowPitch)})177 planes = append(planes, domain.DMABufPlane{File: os.NewFile(uintptr(dup), "neferwl-cast"), Offset: uint32(layouts[i].Offset), Stride: uint32(layouts[i].RowPitch)})
169 }178 }
170- return domain.DMABuf{Width: w, Height: h, Format: format, Modifier: modifier, Planes: planes}, nil179+ buf := domain.DMABuf{ID: castBufferID.Add(1), Width: w, Height: h, Format: format, Modifier: modifier, Planes: planes}
180+ if g.castImages == nil {
181+ g.castImages = make(map[uint64]*castImage)
182+ }
183+ g.castImages[buf.ID] = &castImage{buffer: buf, img: t}
184+ retained = true
185+ return buf, nil
186+}
187+
188+// castBufferID is process-wide, so cast identities cannot collide across GPUs.
189+var castBufferID atomic.Uint64
190+
191+type castImage struct {
192+ buffer domain.DMABuf
193+ img *target
194+ refs int
195+ released bool
196+}
197+
198+// ReleaseCast is safe on the PipeWire owner while output owners finish copies.
199+func (g *GPU) ReleaseCast(buf domain.DMABuf) {
200+ g.poolMu.Lock()
201+ defer g.poolMu.Unlock()
202+ e := g.castImages[buf.ID]
203+ if e == nil {
204+ return
205+ }
206+ delete(g.castImages, buf.ID)
207+ e.released = true
208+ if e.refs == 0 && e.img != nil {
209+ g.freeImage(e.img)
210+ e.img = nil
211+ }
212+}
213+
214+func (g *GPU) acquireCastImage(buf *domain.DMABuf) (*target, *castImage, error) {
215+ g.poolMu.Lock()
216+ defer g.poolMu.Unlock()
217+ e := g.castImages[buf.ID]
218+ if e == nil {
219+ img, err := g.importBlitImage(buf, buf.Width, buf.Height, vk.ImageUsageTransferDstBit, formatFeatureTransferDst)
220+ return img, nil, err
221+ }
222+ // Registered IDs name an immutable allocation, not arbitrary request geometry.
223+ if buf.Width != e.buffer.Width || buf.Height != e.buffer.Height || buf.Format != e.buffer.Format || buf.Modifier != e.buffer.Modifier || len(buf.Planes) != len(e.buffer.Planes) {
224+ return nil, nil, errImageGeometry
225+ }
226+ for i, p := range buf.Planes {
227+ if p != e.buffer.Planes[i] {
228+ return nil, nil, errImagePlanes
229+ }
230+ }
231+
232+ e.refs++
233+ return e.img, e, nil
234+}
235+
236+func (g *GPU) releaseCastImage(e *castImage) {
237+ g.poolMu.Lock()
238+ defer g.poolMu.Unlock()
239+ e.refs--
240+ if e.released && e.refs == 0 && e.img != nil {
241+ g.freeImage(e.img)
242+ e.img = nil
243+ }
171 }244 }
M internal/adapters/vulkan/cast_alloc_test.go +1 -0
@@ -26,6 +26,7 @@ func castLinear(t *testing.T, g *GPU, fc uint32, w, h int) domain.DMABuf {
2626 t.Fatal(err)
2727 }
2828 t.Cleanup(func() {
29+ g.ReleaseCast(buf)
2930 for _, p := range buf.Planes {
3031 _ = p.File.Close()
3132 }
@@ -26,6 +26,7 @@ func castLinear(t *testing.T, g *GPU, fc uint32, w, h int) domain.DMABuf {
26 t.Fatal(err)26 t.Fatal(err)
27 }27 }
28 t.Cleanup(func() {28 t.Cleanup(func() {
29+ g.ReleaseCast(buf)
29 for _, p := range buf.Planes {30 for _, p := range buf.Planes {
30 _ = p.File.Close()31 _ = p.File.Close()
31 }32 }
M internal/adapters/vulkan/gpu.go +8 -1
@@ -3,9 +3,9 @@ package vulkan
33 import (
44 "fmt"
55
6+ "bnema.dev/purego"
67 "github.com/bnema/neferwl/internal/domain"
78 "github.com/bnema/neferwl/internal/ports"
8- "bnema.dev/purego"
99
1010 "runtime"
1111 "sync"
@@ -34,6 +34,7 @@ type GPU struct {
3434 hdrReadbackMods []uint64 // Immutable transfer-source export list, computed at open.
3535 modPlanes map[uint64]uint32
3636 importMods map[importKey][]uint64
37+ castImages map[uint64]*castImage // retained exports, protected by poolMu
3738 poolMu sync.Mutex
3839 syncFD bool
3940 resources uint32
@@ -229,6 +230,12 @@ func (g *GPU) Close() {
229230 if err := checked("vkDeviceWaitIdle", g.dd.DeviceWaitIdle(g.device)); err != nil {
230231 g.log.Error().Err(err).Msg("wait for GPU idle")
231232 }
233+ for id, entry := range g.castImages {
234+ if entry.img != nil {
235+ g.freeImage(entry.img)
236+ }
237+ delete(g.castImages, id)
238+ }
232239 g.dd.DestroyDevice(g.device, nil)
233240 g.device = 0
234241 g.queues = nil
@@ -3,9 +3,9 @@ package vulkan
3 import (3 import (
4 "fmt"4 "fmt"
5 5
6+ "bnema.dev/purego"
6 "github.com/bnema/neferwl/internal/domain"7 "github.com/bnema/neferwl/internal/domain"
7 "github.com/bnema/neferwl/internal/ports"8 "github.com/bnema/neferwl/internal/ports"
8- "bnema.dev/purego"
9 9
10 "runtime"10 "runtime"
11 "sync"11 "sync"
@@ -34,6 +34,7 @@ type GPU struct {
34 hdrReadbackMods []uint64 // Immutable transfer-source export list, computed at open.34 hdrReadbackMods []uint64 // Immutable transfer-source export list, computed at open.
35 modPlanes map[uint64]uint3235 modPlanes map[uint64]uint32
36 importMods map[importKey][]uint6436 importMods map[importKey][]uint64
37+ castImages map[uint64]*castImage // retained exports, protected by poolMu
37 poolMu sync.Mutex38 poolMu sync.Mutex
38 syncFD bool39 syncFD bool
39 resources uint3240 resources uint32
@@ -229,6 +230,12 @@ func (g *GPU) Close() {
229 if err := checked("vkDeviceWaitIdle", g.dd.DeviceWaitIdle(g.device)); err != nil {230 if err := checked("vkDeviceWaitIdle", g.dd.DeviceWaitIdle(g.device)); err != nil {
230 g.log.Error().Err(err).Msg("wait for GPU idle")231 g.log.Error().Err(err).Msg("wait for GPU idle")
231 }232 }
233+ for id, entry := range g.castImages {
234+ if entry.img != nil {
235+ g.freeImage(entry.img)
236+ }
237+ delete(g.castImages, id)
238+ }
232 g.dd.DestroyDevice(g.device, nil)239 g.dd.DestroyDevice(g.device, nil)
233 g.device = 0240 g.device = 0
234 g.queues = nil241 g.queues = nil
M internal/adapters/vulkan/renderer.go +3 -1
@@ -60,7 +60,9 @@ type Renderer struct {
6060 // capturePQ is the PQ readback pass of BeginCapturePQ (capture_pq.go).
6161 capturePQ capturePQPass
6262 sync captureSync
63- dmaFences dmabufFences // CopyTo's implicit sync (capture_dmabuf.go)
63+ dmaImport vk.ImportSemaphoreFdInfoKHR // owner scratch for temporary wait payload
64+ dmaIdle []*dmaCopy // owner-only copies recycled after fence completion
65+ dmaFences dmabufFences // CopyTo's implicit sync (capture_dmabuf.go)
6466 pool vk.CommandPool
6567 // Frames in flight (frame.go): frame is the last frame recorded,
6668 // submitted the last submitted, completed the last known finished.
@@ -60,7 +60,9 @@ type Renderer struct {
60 // capturePQ is the PQ readback pass of BeginCapturePQ (capture_pq.go).60 // capturePQ is the PQ readback pass of BeginCapturePQ (capture_pq.go).
61 capturePQ capturePQPass61 capturePQ capturePQPass
62 sync captureSync62 sync captureSync
63- dmaFences dmabufFences // CopyTo's implicit sync (capture_dmabuf.go)63+ dmaImport vk.ImportSemaphoreFdInfoKHR // owner scratch for temporary wait payload
64+ dmaIdle []*dmaCopy // owner-only copies recycled after fence completion
65+ dmaFences dmabufFences // CopyTo's implicit sync (capture_dmabuf.go)
64 pool vk.CommandPool66 pool vk.CommandPool
65 // Frames in flight (frame.go): frame is the last frame recorded,67 // Frames in flight (frame.go): frame is the last frame recorded,
66 // submitted the last submitted, completed the last known finished.68 // submitted the last submitted, completed the last known finished.
M internal/adapters/vulkan/trim.go +40 -0
@@ -78,6 +78,9 @@ func (r *Renderer) Trim(now time.Time) error {
7878 }
7979 }
8080 }
81+ if err := r.trimCaptures(); err != nil {
82+ return err
83+ }
8184 if err := r.poll(); err != nil {
8285 return err
8386 }
@@ -132,3 +135,40 @@ func (r *Renderer) noteHidden(s domain.Scene) {
132135 }
133136 }
134137 }
138+
139+// trimCaptures reclaims capture-only storage without waiting on the device.
140+// Output owners call Trim periodically; there is no per-capture timer. A live
141+// lease (including one held by the pipeline worker) pins the entire capture
142+// pass. Returned leases can still have GPU copies running, so poll all fences
143+// before destroying shared SDR/PQ passes and slot-owned cursor resources.
144+func (r *Renderer) trimCaptures() error {
145+ for _, s := range r.captures {
146+ if s.leased {
147+ return nil
148+ }
149+ }
150+ pending := false
151+ for _, s := range r.captures {
152+ if err := r.reapCopies(s); err != nil {
153+ return err
154+ }
155+ if len(s.copies) > 0 {
156+ pending = true
157+ }
158+ if s.pending {
159+ switch res := r.captureSync().fenceStatus(s.fence); res {
160+ case vk.Success:
161+ case vk.Timeout, vk.NotReady:
162+ pending = true
163+ default:
164+ return checked("vkWaitForFences(capture trim)", res)
165+ }
166+ }
167+ }
168+ if pending {
169+ return nil
170+ }
171+ r.destroyCaptures()
172+ r.destroyCapturePQ()
173+ return nil
174+}
@@ -78,6 +78,9 @@ func (r *Renderer) Trim(now time.Time) error {
78 }78 }
79 }79 }
80 }80 }
81+ if err := r.trimCaptures(); err != nil {
82+ return err
83+ }
81 if err := r.poll(); err != nil {84 if err := r.poll(); err != nil {
82 return err85 return err
83 }86 }
@@ -132,3 +135,40 @@ func (r *Renderer) noteHidden(s domain.Scene) {
132 }135 }
133 }136 }
134 }137 }
138+
139+// trimCaptures reclaims capture-only storage without waiting on the device.
140+// Output owners call Trim periodically; there is no per-capture timer. A live
141+// lease (including one held by the pipeline worker) pins the entire capture
142+// pass. Returned leases can still have GPU copies running, so poll all fences
143+// before destroying shared SDR/PQ passes and slot-owned cursor resources.
144+func (r *Renderer) trimCaptures() error {
145+ for _, s := range r.captures {
146+ if s.leased {
147+ return nil
148+ }
149+ }
150+ pending := false
151+ for _, s := range r.captures {
152+ if err := r.reapCopies(s); err != nil {
153+ return err
154+ }
155+ if len(s.copies) > 0 {
156+ pending = true
157+ }
158+ if s.pending {
159+ switch res := r.captureSync().fenceStatus(s.fence); res {
160+ case vk.Success:
161+ case vk.Timeout, vk.NotReady:
162+ pending = true
163+ default:
164+ return checked("vkWaitForFences(capture trim)", res)
165+ }
166+ }
167+ }
168+ if pending {
169+ return nil
170+ }
171+ r.destroyCaptures()
172+ r.destroyCapturePQ()
173+ return nil
174+}
M internal/app/screencast_test.go +47 -9
@@ -9,6 +9,7 @@ import (
99 "os/exec"
1010 "path/filepath"
1111 "runtime"
12+ "runtime/debug"
1213 "strconv"
1314 "strings"
1415 "sync"
@@ -16,10 +17,10 @@ import (
1617 "testing"
1718 "time"
1819
20+ "bnema.dev/zerobus"
1921 "github.com/bnema/neferwl/internal/adapters/config"
2022 "github.com/bnema/neferwl/internal/domain"
2123 "github.com/bnema/neferwl/internal/ports"
22- "bnema.dev/zerobus"
2324 )
2425
2526 // The screencast tests need a session bus of their own (run them under
@@ -300,7 +301,10 @@ func TestScreencastStreamsMonitor(t *testing.T) {
300301 // Steady state: measure 150 frames after 60 warm-up frames. Every allocation is
301302 // recorded (MemProfileRate 1) and attributed by its stack: the cast path
302303 // (screencast, wayland casts, purego-pipewire, zerobus) must allocate
303- // less than once per frame, and so must the renderer capture path it
304+ // less than once per frame, excluding a named allowance of two allocations
305+ // for the caller-owned CopyTo sync-file wrapper (os.newFile). That wrapper
306+ // cannot be recycled on GPU completion while the worker still owns it.
307+ // The same budget covers the renderer capture path it
304308 // shares with ext-image-copy-capture (BeginCapture, EndCapture and the
305309 // pipeline worker). The headless output's own display frame (Render's
306310 // sync file, the flip report, injected input) allocates whether or not
@@ -320,11 +324,16 @@ func TestScreencastStreamsMonitor(t *testing.T) {
320324 if n < 60 {
321325 t.Errorf("only %d frames in measurement interval", n)
322326 } else {
323- cast, other := after.sub(before)
327+ cast, other, wrappers := after.sub(before)
328+ wrapperPer := float64(wrappers) / float64(n)
324329 per := float64(cast) / float64(n)
325330 t.Logf("%d frames: cast path %.3f mallocs per frame, whole process %.2f (renderer and runtime included; %d other profiled)",
326331 n, per, float64(m1.Mallocs-m0.Mallocs)/float64(n), other)
327- if per >= 1 {
332+ t.Logf("sync-file wrapper allowance: %.3f mallocs/frame (budget 2); total cast path %.3f", wrapperPer, per+wrapperPer)
333+ if wrapperPer > 2 && !castRaceEnabled() {
334+ t.Errorf("sync-file wrappers: %.3f mallocs/frame, budget 2", wrapperPer)
335+ }
336+ if per >= 1 && !castRaceEnabled() {
328337 t.Errorf("cast path: %.2f mallocs per frame over %d frames, want < 1", per, n)
329338 }
330339 }
@@ -417,6 +426,8 @@ var castPath = []string{
417426 // screencopy: lease, copy, read, return.
418427 "vulkan.(*Renderer).BeginCapture",
419428 "vulkan.(*Renderer).EndCapture",
429+ "vulkan.(*captureFrame).CopyTo",
430+ "outputkit/capture.(*batch).submitCopies",
420431 "outputkit/capture.(*Pipeline).SubmitScoped",
421432 "outputkit/capture.(*Pipeline).process",
422433 "outputkit/capture.(*Pipeline).Recycle",
@@ -425,7 +436,8 @@ var castPath = []string{
425436 // allocs counts profiled allocation objects by stack, for the cast path and
426437 // for the rest.
427438 type allocs struct {
428- cast, other map[string]int64
439+ cast, other map[string]int64
440+ syncFileWrappers map[string]int64
429441 }
430442
431443 func allocSnapshot() allocs {
@@ -444,14 +456,21 @@ func allocSnapshot() allocs {
444456 break
445457 }
446458 }
447- a := allocs{cast: map[string]int64{}, other: map[string]int64{}}
459+ a := allocs{cast: map[string]int64{}, other: map[string]int64{}, syncFileWrappers: map[string]int64{}}
448460 for i := range recs {
449461 pcs := recs[i].Stack()
450462 frames := runtime.CallersFrames(pcs)
451463 var key strings.Builder
452464 isCast := false
465+ isCopy, isNewFile := false, false
453466 for {
454467 f, more := frames.Next()
468+ if strings.Contains(f.Function, "vulkan.(*captureFrame).CopyTo") {
469+ isCopy = true
470+ }
471+ if f.Function == "os.newFile" {
472+ isNewFile = true
473+ }
455474 key.WriteString(f.Function)
456475 key.WriteByte(';')
457476 for _, p := range castPath {
@@ -463,7 +482,9 @@ func allocSnapshot() allocs {
463482 break
464483 }
465484 }
466- if isCast {
485+ if isCopy && isNewFile {
486+ a.syncFileWrappers[key.String()] += recs[i].AllocObjects
487+ } else if isCast {
467488 a.cast[key.String()] += recs[i].AllocObjects
468489 } else {
469490 a.other[key.String()] += recs[i].AllocObjects
@@ -473,14 +494,17 @@ func allocSnapshot() allocs {
473494 }
474495
475496 // sub returns the objects allocated between two snapshots.
476-func (a allocs) sub(b allocs) (cast, other int64) {
497+func (a allocs) sub(b allocs) (cast, other, syncFileWrappers int64) {
498+ for k, v := range a.syncFileWrappers {
499+ syncFileWrappers += v - b.syncFileWrappers[k]
500+ }
477501 for k, v := range a.cast {
478502 cast += v - b.cast[k]
479503 }
480504 for k, v := range a.other {
481505 other += v - b.other[k]
482506 }
483- return cast, other
507+ return cast, other, syncFileWrappers
484508 }
485509
486510 //testpolicy:realtime observes frames from the real GStreamer/PipeWire consumer
@@ -498,3 +522,17 @@ func waitFrames(t *testing.T, frames *atomic.Int64, changed <-chan struct{}, exi
498522 }
499523 }
500524 }
525+
526+// castRaceEnabled keeps the real-I/O lifecycle running with -race, but disables
527+// allocation assertions: purego's sync.Pool drops argument storage under race.
528+func castRaceEnabled() bool {
529+ info, ok := debug.ReadBuildInfo()
530+ if ok {
531+ for _, setting := range info.Settings {
532+ if setting.Key == "-race" {
533+ return setting.Value == "true"
534+ }
535+ }
536+ }
537+ return false
538+}
@@ -9,6 +9,7 @@ import (
9 "os/exec"9 "os/exec"
10 "path/filepath"10 "path/filepath"
11 "runtime"11 "runtime"
12+ "runtime/debug"
12 "strconv"13 "strconv"
13 "strings"14 "strings"
14 "sync"15 "sync"
@@ -16,10 +17,10 @@ import (
16 "testing"17 "testing"
17 "time"18 "time"
18 19
20+ "bnema.dev/zerobus"
19 "github.com/bnema/neferwl/internal/adapters/config"21 "github.com/bnema/neferwl/internal/adapters/config"
20 "github.com/bnema/neferwl/internal/domain"22 "github.com/bnema/neferwl/internal/domain"
21 "github.com/bnema/neferwl/internal/ports"23 "github.com/bnema/neferwl/internal/ports"
22- "bnema.dev/zerobus"
23 )24 )
24 25
25 // The screencast tests need a session bus of their own (run them under26 // The screencast tests need a session bus of their own (run them under
@@ -300,7 +301,10 @@ func TestScreencastStreamsMonitor(t *testing.T) {
300 // Steady state: measure 150 frames after 60 warm-up frames. Every allocation is301 // Steady state: measure 150 frames after 60 warm-up frames. Every allocation is
301 // recorded (MemProfileRate 1) and attributed by its stack: the cast path302 // recorded (MemProfileRate 1) and attributed by its stack: the cast path
302 // (screencast, wayland casts, purego-pipewire, zerobus) must allocate303 // (screencast, wayland casts, purego-pipewire, zerobus) must allocate
303- // less than once per frame, and so must the renderer capture path it304+ // less than once per frame, excluding a named allowance of two allocations
305+ // for the caller-owned CopyTo sync-file wrapper (os.newFile). That wrapper
306+ // cannot be recycled on GPU completion while the worker still owns it.
307+ // The same budget covers the renderer capture path it
304 // shares with ext-image-copy-capture (BeginCapture, EndCapture and the308 // shares with ext-image-copy-capture (BeginCapture, EndCapture and the
305 // pipeline worker). The headless output's own display frame (Render's309 // pipeline worker). The headless output's own display frame (Render's
306 // sync file, the flip report, injected input) allocates whether or not310 // sync file, the flip report, injected input) allocates whether or not
@@ -320,11 +324,16 @@ func TestScreencastStreamsMonitor(t *testing.T) {
320 if n < 60 {324 if n < 60 {
321 t.Errorf("only %d frames in measurement interval", n)325 t.Errorf("only %d frames in measurement interval", n)
322 } else {326 } else {
323- cast, other := after.sub(before)327+ cast, other, wrappers := after.sub(before)
328+ wrapperPer := float64(wrappers) / float64(n)
324 per := float64(cast) / float64(n)329 per := float64(cast) / float64(n)
325 t.Logf("%d frames: cast path %.3f mallocs per frame, whole process %.2f (renderer and runtime included; %d other profiled)",330 t.Logf("%d frames: cast path %.3f mallocs per frame, whole process %.2f (renderer and runtime included; %d other profiled)",
326 n, per, float64(m1.Mallocs-m0.Mallocs)/float64(n), other)331 n, per, float64(m1.Mallocs-m0.Mallocs)/float64(n), other)
327- if per >= 1 {332+ t.Logf("sync-file wrapper allowance: %.3f mallocs/frame (budget 2); total cast path %.3f", wrapperPer, per+wrapperPer)
333+ if wrapperPer > 2 && !castRaceEnabled() {
334+ t.Errorf("sync-file wrappers: %.3f mallocs/frame, budget 2", wrapperPer)
335+ }
336+ if per >= 1 && !castRaceEnabled() {
328 t.Errorf("cast path: %.2f mallocs per frame over %d frames, want < 1", per, n)337 t.Errorf("cast path: %.2f mallocs per frame over %d frames, want < 1", per, n)
329 }338 }
330 }339 }
@@ -417,6 +426,8 @@ var castPath = []string{
417 // screencopy: lease, copy, read, return.426 // screencopy: lease, copy, read, return.
418 "vulkan.(*Renderer).BeginCapture",427 "vulkan.(*Renderer).BeginCapture",
419 "vulkan.(*Renderer).EndCapture",428 "vulkan.(*Renderer).EndCapture",
429+ "vulkan.(*captureFrame).CopyTo",
430+ "outputkit/capture.(*batch).submitCopies",
420 "outputkit/capture.(*Pipeline).SubmitScoped",431 "outputkit/capture.(*Pipeline).SubmitScoped",
421 "outputkit/capture.(*Pipeline).process",432 "outputkit/capture.(*Pipeline).process",
422 "outputkit/capture.(*Pipeline).Recycle",433 "outputkit/capture.(*Pipeline).Recycle",
@@ -425,7 +436,8 @@ var castPath = []string{
425 // allocs counts profiled allocation objects by stack, for the cast path and436 // allocs counts profiled allocation objects by stack, for the cast path and
426 // for the rest.437 // for the rest.
427 type allocs struct {438 type allocs struct {
428- cast, other map[string]int64439+ cast, other map[string]int64
440+ syncFileWrappers map[string]int64
429 }441 }
430 442
431 func allocSnapshot() allocs {443 func allocSnapshot() allocs {
@@ -444,14 +456,21 @@ func allocSnapshot() allocs {
444 break456 break
445 }457 }
446 }458 }
447- a := allocs{cast: map[string]int64{}, other: map[string]int64{}}459+ a := allocs{cast: map[string]int64{}, other: map[string]int64{}, syncFileWrappers: map[string]int64{}}
448 for i := range recs {460 for i := range recs {
449 pcs := recs[i].Stack()461 pcs := recs[i].Stack()
450 frames := runtime.CallersFrames(pcs)462 frames := runtime.CallersFrames(pcs)
451 var key strings.Builder463 var key strings.Builder
452 isCast := false464 isCast := false
465+ isCopy, isNewFile := false, false
453 for {466 for {
454 f, more := frames.Next()467 f, more := frames.Next()
468+ if strings.Contains(f.Function, "vulkan.(*captureFrame).CopyTo") {
469+ isCopy = true
470+ }
471+ if f.Function == "os.newFile" {
472+ isNewFile = true
473+ }
455 key.WriteString(f.Function)474 key.WriteString(f.Function)
456 key.WriteByte(';')475 key.WriteByte(';')
457 for _, p := range castPath {476 for _, p := range castPath {
@@ -463,7 +482,9 @@ func allocSnapshot() allocs {
463 break482 break
464 }483 }
465 }484 }
466- if isCast {485+ if isCopy && isNewFile {
486+ a.syncFileWrappers[key.String()] += recs[i].AllocObjects
487+ } else if isCast {
467 a.cast[key.String()] += recs[i].AllocObjects488 a.cast[key.String()] += recs[i].AllocObjects
468 } else {489 } else {
469 a.other[key.String()] += recs[i].AllocObjects490 a.other[key.String()] += recs[i].AllocObjects
@@ -473,14 +494,17 @@ func allocSnapshot() allocs {
473 }494 }
474 495
475 // sub returns the objects allocated between two snapshots.496 // sub returns the objects allocated between two snapshots.
476-func (a allocs) sub(b allocs) (cast, other int64) {497+func (a allocs) sub(b allocs) (cast, other, syncFileWrappers int64) {
498+ for k, v := range a.syncFileWrappers {
499+ syncFileWrappers += v - b.syncFileWrappers[k]
500+ }
477 for k, v := range a.cast {501 for k, v := range a.cast {
478 cast += v - b.cast[k]502 cast += v - b.cast[k]
479 }503 }
480 for k, v := range a.other {504 for k, v := range a.other {
481 other += v - b.other[k]505 other += v - b.other[k]
482 }506 }
483- return cast, other507+ return cast, other, syncFileWrappers
484 }508 }
485 509
486 //testpolicy:realtime observes frames from the real GStreamer/PipeWire consumer510 //testpolicy:realtime observes frames from the real GStreamer/PipeWire consumer
@@ -498,3 +522,17 @@ func waitFrames(t *testing.T, frames *atomic.Int64, changed <-chan struct{}, exi
498 }522 }
499 }523 }
500 }524 }
525+
526+// castRaceEnabled keeps the real-I/O lifecycle running with -race, but disables
527+// allocation assertions: purego's sync.Pool drops argument storage under race.
528+func castRaceEnabled() bool {
529+ info, ok := debug.ReadBuildInfo()
530+ if ok {
531+ for _, setting := range info.Settings {
532+ if setting.Key == "-race" {
533+ return setting.Value == "true"
534+ }
535+ }
536+ }
537+ return false
538+}
M internal/mocks/ports/devices_mock.go +80 -0
@@ -1818,6 +1818,46 @@ func (_c *MockRenderDevice_Node_Call) RunAndReturn(run func() (uint64, uint64))
18181818 return _c
18191819 }
18201820
1821+// ReleaseCast provides a mock function for the type MockRenderDevice
1822+func (_mock *MockRenderDevice) ReleaseCast(buffer domain.DMABuf) {
1823+ _mock.Called(buffer)
1824+ return
1825+}
1826+
1827+// MockRenderDevice_ReleaseCast_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'ReleaseCast'
1828+type MockRenderDevice_ReleaseCast_Call struct {
1829+ *mock.Call
1830+}
1831+
1832+// ReleaseCast is a helper method to define mock.On call
1833+// - buffer domain.DMABuf
1834+func (_e *MockRenderDevice_Expecter) ReleaseCast(buffer any) *MockRenderDevice_ReleaseCast_Call {
1835+ return &MockRenderDevice_ReleaseCast_Call{Call: _e.mock.On("ReleaseCast", buffer)}
1836+}
1837+
1838+func (_c *MockRenderDevice_ReleaseCast_Call) Run(run func(buffer domain.DMABuf)) *MockRenderDevice_ReleaseCast_Call {
1839+ _c.Call.Run(func(args mock.Arguments) {
1840+ var arg0 domain.DMABuf
1841+ if args[0] != nil {
1842+ arg0 = args[0].(domain.DMABuf)
1843+ }
1844+ run(
1845+ arg0,
1846+ )
1847+ })
1848+ return _c
1849+}
1850+
1851+func (_c *MockRenderDevice_ReleaseCast_Call) Return() *MockRenderDevice_ReleaseCast_Call {
1852+ _c.Call.Return()
1853+ return _c
1854+}
1855+
1856+func (_c *MockRenderDevice_ReleaseCast_Call) RunAndReturn(run func(buffer domain.DMABuf)) *MockRenderDevice_ReleaseCast_Call {
1857+ _c.Run(run)
1858+ return _c
1859+}
1860+
18211861 // NewMockCastBufferAllocator creates a new instance of MockCastBufferAllocator. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
18221862 // The first argument is typically a *testing.T value.
18231863 func NewMockCastBufferAllocator(t interface {
@@ -1978,6 +2018,46 @@ func (_c *MockCastBufferAllocator_CastFormats_Call) RunAndReturn(run func() []do
19782018 return _c
19792019 }
19802020
2021+// ReleaseCast provides a mock function for the type MockCastBufferAllocator
2022+func (_mock *MockCastBufferAllocator) ReleaseCast(buffer domain.DMABuf) {
2023+ _mock.Called(buffer)
2024+ return
2025+}
2026+
2027+// MockCastBufferAllocator_ReleaseCast_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'ReleaseCast'
2028+type MockCastBufferAllocator_ReleaseCast_Call struct {
2029+ *mock.Call
2030+}
2031+
2032+// ReleaseCast is a helper method to define mock.On call
2033+// - buffer domain.DMABuf
2034+func (_e *MockCastBufferAllocator_Expecter) ReleaseCast(buffer any) *MockCastBufferAllocator_ReleaseCast_Call {
2035+ return &MockCastBufferAllocator_ReleaseCast_Call{Call: _e.mock.On("ReleaseCast", buffer)}
2036+}
2037+
2038+func (_c *MockCastBufferAllocator_ReleaseCast_Call) Run(run func(buffer domain.DMABuf)) *MockCastBufferAllocator_ReleaseCast_Call {
2039+ _c.Call.Run(func(args mock.Arguments) {
2040+ var arg0 domain.DMABuf
2041+ if args[0] != nil {
2042+ arg0 = args[0].(domain.DMABuf)
2043+ }
2044+ run(
2045+ arg0,
2046+ )
2047+ })
2048+ return _c
2049+}
2050+
2051+func (_c *MockCastBufferAllocator_ReleaseCast_Call) Return() *MockCastBufferAllocator_ReleaseCast_Call {
2052+ _c.Call.Return()
2053+ return _c
2054+}
2055+
2056+func (_c *MockCastBufferAllocator_ReleaseCast_Call) RunAndReturn(run func(buffer domain.DMABuf)) *MockCastBufferAllocator_ReleaseCast_Call {
2057+ _c.Run(run)
2058+ return _c
2059+}
2060+
19812061 // NewMockRenderDeviceOpener creates a new instance of MockRenderDeviceOpener. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
19822062 // The first argument is typically a *testing.T value.
19832063 func NewMockRenderDeviceOpener(t interface {
@@ -1818,6 +1818,46 @@ func (_c *MockRenderDevice_Node_Call) RunAndReturn(run func() (uint64, uint64))
1818 return _c1818 return _c
1819 }1819 }
1820 1820
1821+// ReleaseCast provides a mock function for the type MockRenderDevice
1822+func (_mock *MockRenderDevice) ReleaseCast(buffer domain.DMABuf) {
1823+ _mock.Called(buffer)
1824+ return
1825+}
1826+
1827+// MockRenderDevice_ReleaseCast_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'ReleaseCast'
1828+type MockRenderDevice_ReleaseCast_Call struct {
1829+ *mock.Call
1830+}
1831+
1832+// ReleaseCast is a helper method to define mock.On call
1833+// - buffer domain.DMABuf
1834+func (_e *MockRenderDevice_Expecter) ReleaseCast(buffer any) *MockRenderDevice_ReleaseCast_Call {
1835+ return &MockRenderDevice_ReleaseCast_Call{Call: _e.mock.On("ReleaseCast", buffer)}
1836+}
1837+
1838+func (_c *MockRenderDevice_ReleaseCast_Call) Run(run func(buffer domain.DMABuf)) *MockRenderDevice_ReleaseCast_Call {
1839+ _c.Call.Run(func(args mock.Arguments) {
1840+ var arg0 domain.DMABuf
1841+ if args[0] != nil {
1842+ arg0 = args[0].(domain.DMABuf)
1843+ }
1844+ run(
1845+ arg0,
1846+ )
1847+ })
1848+ return _c
1849+}
1850+
1851+func (_c *MockRenderDevice_ReleaseCast_Call) Return() *MockRenderDevice_ReleaseCast_Call {
1852+ _c.Call.Return()
1853+ return _c
1854+}
1855+
1856+func (_c *MockRenderDevice_ReleaseCast_Call) RunAndReturn(run func(buffer domain.DMABuf)) *MockRenderDevice_ReleaseCast_Call {
1857+ _c.Run(run)
1858+ return _c
1859+}
1860+
1821 // NewMockCastBufferAllocator creates a new instance of MockCastBufferAllocator. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.1861 // NewMockCastBufferAllocator creates a new instance of MockCastBufferAllocator. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
1822 // The first argument is typically a *testing.T value.1862 // The first argument is typically a *testing.T value.
1823 func NewMockCastBufferAllocator(t interface {1863 func NewMockCastBufferAllocator(t interface {
@@ -1978,6 +2018,46 @@ func (_c *MockCastBufferAllocator_CastFormats_Call) RunAndReturn(run func() []do
1978 return _c2018 return _c
1979 }2019 }
1980 2020
2021+// ReleaseCast provides a mock function for the type MockCastBufferAllocator
2022+func (_mock *MockCastBufferAllocator) ReleaseCast(buffer domain.DMABuf) {
2023+ _mock.Called(buffer)
2024+ return
2025+}
2026+
2027+// MockCastBufferAllocator_ReleaseCast_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'ReleaseCast'
2028+type MockCastBufferAllocator_ReleaseCast_Call struct {
2029+ *mock.Call
2030+}
2031+
2032+// ReleaseCast is a helper method to define mock.On call
2033+// - buffer domain.DMABuf
2034+func (_e *MockCastBufferAllocator_Expecter) ReleaseCast(buffer any) *MockCastBufferAllocator_ReleaseCast_Call {
2035+ return &MockCastBufferAllocator_ReleaseCast_Call{Call: _e.mock.On("ReleaseCast", buffer)}
2036+}
2037+
2038+func (_c *MockCastBufferAllocator_ReleaseCast_Call) Run(run func(buffer domain.DMABuf)) *MockCastBufferAllocator_ReleaseCast_Call {
2039+ _c.Call.Run(func(args mock.Arguments) {
2040+ var arg0 domain.DMABuf
2041+ if args[0] != nil {
2042+ arg0 = args[0].(domain.DMABuf)
2043+ }
2044+ run(
2045+ arg0,
2046+ )
2047+ })
2048+ return _c
2049+}
2050+
2051+func (_c *MockCastBufferAllocator_ReleaseCast_Call) Return() *MockCastBufferAllocator_ReleaseCast_Call {
2052+ _c.Call.Return()
2053+ return _c
2054+}
2055+
2056+func (_c *MockCastBufferAllocator_ReleaseCast_Call) RunAndReturn(run func(buffer domain.DMABuf)) *MockCastBufferAllocator_ReleaseCast_Call {
2057+ _c.Run(run)
2058+ return _c
2059+}
2060+
1981 // NewMockRenderDeviceOpener creates a new instance of MockRenderDeviceOpener. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.2061 // NewMockRenderDeviceOpener creates a new instance of MockRenderDeviceOpener. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
1982 // The first argument is typically a *testing.T value.2062 // The first argument is typically a *testing.T value.
1983 func NewMockRenderDeviceOpener(t interface {2063 func NewMockRenderDeviceOpener(t interface {
M internal/ports/devices.go +4 -1

Comments00 unresolved

    no comments yet.

    Keyboard shortcuts

    Single-key shortcuts are on. ctrl k and esc work whatever this says.

    ctrl k :
    go to a section, branch, tag, file or repository
    ?
    this list
    j k
    move the row cursor down / up
    enter
    open the row under the cursor; on Files, a directory opens or closes in place
    ↑ ↓ ← → home end
    in the file tree (focus on a row): previous / next row, close / open a directory or go to its parent / first entry, first / last row. They never open a file
    l h
    on Files: open a directory or a file / close a directory or go up to its parent, then to the parent page. On pages with a side panel: into the panel (on a merge request, its files) / back to the content; there j k move between its controls
    g g G
    first / last row
    /
    filter the list; on Files, find a file anywhere in the repository (esc clears, enter opens the best match)
    b
    switch branch or tag
    g s g f g c g b g m g i g r g a
    tabs: summary, files, commits, branches & tags, merge requests, issues, releases, actions (when available)
    g h
    home
    w
    wide mode: use the whole width of the screen, or not (remembered)
    y
    copy a permalink pinned to the commit; in a file, click a line number to select it, shift-click for a range
    d u x
    diff: split / unified / collapse or expand all
    e E
    diff: the whole file in view / every file, with the lines the hunks hide
    n
    merge request: next note in the diff
    l
    merge request: the list of files (j k move, enter opens, esc back)
    M
    merge request: the merge menu; there 1–3 pick the mode, D deletes the branch, enter merges
    R
    merge request: mark the head read
    esc
    close a dialog, leave a field

    Branches 33

    Tags 15

    ↑↓ moveenter openesc close