detached d60aa9f

refactor(traffic): route websecure through TLS mux

commit
d60aa9f98edb6d7a104f4eae2b4ebe3b3cea613c
author
brice · · 2026-06-30 05:33 UTC
committer
brice · · 2026-06-30 05:38 UTC
parent
78a4426
signature
GPG VerifiedCD88F9E564E0102646FB6077A1A07C7C7C3E349D · arch-gpg Verified for the committer email
17 files · +681 -213
M docs/config/traffic.md +14 -4
@@ -4,7 +4,7 @@ Gordon models network listeners as a traffic graph: entrypoints receive connecti
44
55 ## Entrypoints
66
7-`server.port` creates the default HTTP entrypoint `web`. `server.tls_port` creates `websecure`, the default TLS entrypoint used by Gordon's legacy HTTPS termination path. L4 TLS passthrough must use a custom traffic-manager-owned `tls_mux` entrypoint instead of `websecure`.
7+`server.port` creates the default HTTP entrypoint `web`. `server.tls_port` creates `websecure`, Gordon's default TLS mux entrypoint. `websecure` serves normal HTTPS routes and can also route TLS passthrough by ClientHello SNI. Use a custom `tls_mux` entrypoint when you want an additional port or isolation from the default HTTPS listener.
88
99 Custom entrypoints use the top-level `entrypoints` table:
1010
@@ -78,7 +78,17 @@ UDP sessions are keyed by client address and expire after `idle_timeout`. If `ma
7878
7979 ## TLS Passthrough
8080
81-TLS passthrough routers run on traffic-manager-owned `tls_mux` entrypoints and route by ClientHello SNI without terminating TLS. The `websecure` entrypoint created by `server.tls_port` is reserved for Gordon's legacy HTTPS listener until full HTTP/HTTPS traffic-manager ownership is enabled; use a custom `tls_mux` entrypoint for passthrough.
81+TLS passthrough routers run on `tls_mux` entrypoints and route by ClientHello SNI without terminating TLS. The default `websecure` entrypoint can host normal HTTPS routes and TLS passthrough routes on the same port as long as their hosts do not conflict.
82+
83+```toml
84+[[traffic.tls.routers]]
85+name = "raw-tls"
86+entrypoint = "websecure"
87+sni = "raw.example.com"
88+service = "network_service:raw:tls"
89+```
90+
91+Custom `tls_mux` entrypoints are useful for additional ports:
8292
8393 ```toml
8494 [entrypoints.rawtls]
@@ -86,9 +96,9 @@ address = "0.0.0.0:9443"
8696 protocol = "tls_mux"
8797
8898 [[traffic.tls.routers]]
89-name = "raw-tls"
99+name = "raw-tls-alt"
90100 entrypoint = "rawtls"
91-sni = "raw.example.com"
101+sni = "raw-alt.example.com"
92102 service = "network_service:raw:tls"
93103 ```
94104
@@ -4,7 +4,7 @@ Gordon models network listeners as a traffic graph: entrypoints receive connecti
4 4
5 ## Entrypoints5 ## Entrypoints
6 6
7-`server.port` creates the default HTTP entrypoint `web`. `server.tls_port` creates `websecure`, the default TLS entrypoint used by Gordon's legacy HTTPS termination path. L4 TLS passthrough must use a custom traffic-manager-owned `tls_mux` entrypoint instead of `websecure`.7+`server.port` creates the default HTTP entrypoint `web`. `server.tls_port` creates `websecure`, Gordon's default TLS mux entrypoint. `websecure` serves normal HTTPS routes and can also route TLS passthrough by ClientHello SNI. Use a custom `tls_mux` entrypoint when you want an additional port or isolation from the default HTTPS listener.
8 8
9 Custom entrypoints use the top-level `entrypoints` table:9 Custom entrypoints use the top-level `entrypoints` table:
10 10
@@ -78,7 +78,17 @@ UDP sessions are keyed by client address and expire after `idle_timeout`. If `ma
78 78
79 ## TLS Passthrough79 ## TLS Passthrough
80 80
81-TLS passthrough routers run on traffic-manager-owned `tls_mux` entrypoints and route by ClientHello SNI without terminating TLS. The `websecure` entrypoint created by `server.tls_port` is reserved for Gordon's legacy HTTPS listener until full HTTP/HTTPS traffic-manager ownership is enabled; use a custom `tls_mux` entrypoint for passthrough.81+TLS passthrough routers run on `tls_mux` entrypoints and route by ClientHello SNI without terminating TLS. The default `websecure` entrypoint can host normal HTTPS routes and TLS passthrough routes on the same port as long as their hosts do not conflict.
82+
83+```toml
84+[[traffic.tls.routers]]
85+name = "raw-tls"
86+entrypoint = "websecure"
87+sni = "raw.example.com"
88+service = "network_service:raw:tls"
89+```
90+
91+Custom `tls_mux` entrypoints are useful for additional ports:
82 92
83 ```toml93 ```toml
84 [entrypoints.rawtls]94 [entrypoints.rawtls]
@@ -86,9 +96,9 @@ address = "0.0.0.0:9443"
86 protocol = "tls_mux"96 protocol = "tls_mux"
87 97
88 [[traffic.tls.routers]]98 [[traffic.tls.routers]]
89-name = "raw-tls"99+name = "raw-tls-alt"
90 entrypoint = "rawtls"100 entrypoint = "rawtls"
91-sni = "raw.example.com"101+sni = "raw-alt.example.com"
92 service = "network_service:raw:tls"102 service = "network_service:raw:tls"
93 ```103 ```
94 104
M internal/adapters/in/http/admin/handler_traffic.go +1 -1
@@ -18,7 +18,7 @@ func (h *Handler) handleTrafficStatus(w http.ResponseWriter, r *http.Request) {
1818 return
1919 }
2020 if h.trafficSvc == nil {
21- h.sendJSON(w, http.StatusOK, dto.TrafficStatusResponse{LastReloadStatus: "unavailable"})
21+ h.sendJSON(w, http.StatusOK, dto.TrafficStatusFromDomain(domain.TrafficStatus{LastReloadStatus: "unavailable"}))
2222 return
2323 }
2424 h.sendJSON(w, http.StatusOK, dto.TrafficStatusFromDomain(h.trafficSvc.Status()))
@@ -18,7 +18,7 @@ func (h *Handler) handleTrafficStatus(w http.ResponseWriter, r *http.Request) {
18 return18 return
19 }19 }
20 if h.trafficSvc == nil {20 if h.trafficSvc == nil {
21- h.sendJSON(w, http.StatusOK, dto.TrafficStatusResponse{LastReloadStatus: "unavailable"})21+ h.sendJSON(w, http.StatusOK, dto.TrafficStatusFromDomain(domain.TrafficStatus{LastReloadStatus: "unavailable"}))
22 return22 return
23 }23 }
24 h.sendJSON(w, http.StatusOK, dto.TrafficStatusFromDomain(h.trafficSvc.Status()))24 h.sendJSON(w, http.StatusOK, dto.TrafficStatusFromDomain(h.trafficSvc.Status()))
M internal/adapters/in/http/admin/handler_traffic_test.go +3 -0
@@ -65,4 +65,7 @@ func TestHandler_TrafficStatusUnavailableWithoutService(t *testing.T) {
6565 var body dto.TrafficStatusResponse
6666 require.NoError(t, json.NewDecoder(resp.Body).Decode(&body))
6767 assert.Equal(t, "unavailable", body.LastReloadStatus)
68+ assert.NotNil(t, body.EntryPoints)
69+ assert.NotNil(t, body.Routers)
70+ assert.NotNil(t, body.Services)
6871 }
@@ -65,4 +65,7 @@ func TestHandler_TrafficStatusUnavailableWithoutService(t *testing.T) {
65 var body dto.TrafficStatusResponse65 var body dto.TrafficStatusResponse
66 require.NoError(t, json.NewDecoder(resp.Body).Decode(&body))66 require.NoError(t, json.NewDecoder(resp.Body).Decode(&body))
67 assert.Equal(t, "unavailable", body.LastReloadStatus)67 assert.Equal(t, "unavailable", body.LastReloadStatus)
68+ assert.NotNil(t, body.EntryPoints)
69+ assert.NotNil(t, body.Routers)
70+ assert.NotNil(t, body.Services)
68 }71 }
M internal/adapters/in/traffic/manager.go +7 -14
@@ -4,6 +4,7 @@ package traffic
44 import (
55 "context"
66 "fmt"
7+ "maps"
78 "net"
89 "sort"
910 "strings"
@@ -53,7 +54,7 @@ type Manager struct {
5354
5455 lastReloadStatus string
5556 lastReloadError string
56- tlsFallbacks atomic.Value
57+ tlsHTTPServers atomic.Value
5758 }
5859
5960 // NewManager creates a traffic manager.
@@ -63,7 +64,7 @@ func NewManager() *Manager {
6364 udpListeners: map[string]*udpEntryPointRuntime{},
6465 lastReloadStatus: reloadStatusOK,
6566 }
66- manager.tlsFallbacks.Store(tlsFallbacks{})
67+ manager.tlsHTTPServers.Store(tlsHTTPServers{})
6768 return manager
6869 }
6970
@@ -220,13 +221,9 @@ func (m *Manager) Shutdown(ctx context.Context) error {
220221 func (m *Manager) Status() domain.TrafficStatus {
221222 m.mu.Lock()
222223 listeners := make(map[string]*entryPointRuntime, len(m.listeners))
223- for name, runtime := range m.listeners {
224- listeners[name] = runtime
225- }
224+ maps.Copy(listeners, m.listeners)
226225 udpListeners := make(map[string]*udpEntryPointRuntime, len(m.udpListeners))
227- for name, runtime := range m.udpListeners {
228- udpListeners[name] = runtime
229- }
226+ maps.Copy(udpListeners, m.udpListeners)
230227 status := domain.TrafficStatus{LastReloadStatus: m.lastReloadStatus, LastReloadError: m.lastReloadError}
231228 graph := m.snapshot.Load()
232229 m.mu.Unlock()
@@ -265,9 +262,7 @@ type udpRuntimeUpdate struct {
265262
266263 func (m *Manager) prepareTCPListeners(ctx context.Context, graph *domain.TrafficGraph) (map[string]*entryPointRuntime, []*entryPointRuntime, []tcpRuntimeUpdate, error) {
267264 current := make(map[string]*entryPointRuntime, len(m.listeners))
268- for name, runtime := range m.listeners {
269- current[name] = runtime
270- }
265+ maps.Copy(current, m.listeners)
271266
272267 next := make(map[string]*entryPointRuntime, len(current))
273268 created := []*entryPointRuntime{}
@@ -346,9 +341,7 @@ func (m *Manager) bindTCPEntryPoint(ctx context.Context, entryPoint domain.Entry
346341
347342 func (m *Manager) prepareUDPListeners(ctx context.Context, graph *domain.TrafficGraph) (map[string]*udpEntryPointRuntime, []*udpEntryPointRuntime, []udpRuntimeUpdate, error) {
348343 current := make(map[string]*udpEntryPointRuntime, len(m.udpListeners))
349- for name, runtime := range m.udpListeners {
350- current[name] = runtime
351- }
344+ maps.Copy(current, m.udpListeners)
352345
353346 next := make(map[string]*udpEntryPointRuntime, len(current))
354347 created := []*udpEntryPointRuntime{}
@@ -4,6 +4,7 @@ package traffic
4 import (4 import (
5 "context"5 "context"
6 "fmt"6 "fmt"
7+ "maps"
7 "net"8 "net"
8 "sort"9 "sort"
9 "strings"10 "strings"
@@ -53,7 +54,7 @@ type Manager struct {
53 54
54 lastReloadStatus string55 lastReloadStatus string
55 lastReloadError string56 lastReloadError string
56- tlsFallbacks atomic.Value57+ tlsHTTPServers atomic.Value
57 }58 }
58 59
59 // NewManager creates a traffic manager.60 // NewManager creates a traffic manager.
@@ -63,7 +64,7 @@ func NewManager() *Manager {
63 udpListeners: map[string]*udpEntryPointRuntime{},64 udpListeners: map[string]*udpEntryPointRuntime{},
64 lastReloadStatus: reloadStatusOK,65 lastReloadStatus: reloadStatusOK,
65 }66 }
66- manager.tlsFallbacks.Store(tlsFallbacks{})67+ manager.tlsHTTPServers.Store(tlsHTTPServers{})
67 return manager68 return manager
68 }69 }
69 70
@@ -220,13 +221,9 @@ func (m *Manager) Shutdown(ctx context.Context) error {
220 func (m *Manager) Status() domain.TrafficStatus {221 func (m *Manager) Status() domain.TrafficStatus {
221 m.mu.Lock()222 m.mu.Lock()
222 listeners := make(map[string]*entryPointRuntime, len(m.listeners))223 listeners := make(map[string]*entryPointRuntime, len(m.listeners))
223- for name, runtime := range m.listeners {224+ maps.Copy(listeners, m.listeners)
224- listeners[name] = runtime
225- }
226 udpListeners := make(map[string]*udpEntryPointRuntime, len(m.udpListeners))225 udpListeners := make(map[string]*udpEntryPointRuntime, len(m.udpListeners))
227- for name, runtime := range m.udpListeners {226+ maps.Copy(udpListeners, m.udpListeners)
228- udpListeners[name] = runtime
229- }
230 status := domain.TrafficStatus{LastReloadStatus: m.lastReloadStatus, LastReloadError: m.lastReloadError}227 status := domain.TrafficStatus{LastReloadStatus: m.lastReloadStatus, LastReloadError: m.lastReloadError}
231 graph := m.snapshot.Load()228 graph := m.snapshot.Load()
232 m.mu.Unlock()229 m.mu.Unlock()
@@ -265,9 +262,7 @@ type udpRuntimeUpdate struct {
265 262
266 func (m *Manager) prepareTCPListeners(ctx context.Context, graph *domain.TrafficGraph) (map[string]*entryPointRuntime, []*entryPointRuntime, []tcpRuntimeUpdate, error) {263 func (m *Manager) prepareTCPListeners(ctx context.Context, graph *domain.TrafficGraph) (map[string]*entryPointRuntime, []*entryPointRuntime, []tcpRuntimeUpdate, error) {
267 current := make(map[string]*entryPointRuntime, len(m.listeners))264 current := make(map[string]*entryPointRuntime, len(m.listeners))
268- for name, runtime := range m.listeners {265+ maps.Copy(current, m.listeners)
269- current[name] = runtime
270- }
271 266
272 next := make(map[string]*entryPointRuntime, len(current))267 next := make(map[string]*entryPointRuntime, len(current))
273 created := []*entryPointRuntime{}268 created := []*entryPointRuntime{}
@@ -346,9 +341,7 @@ func (m *Manager) bindTCPEntryPoint(ctx context.Context, entryPoint domain.Entry
346 341
347 func (m *Manager) prepareUDPListeners(ctx context.Context, graph *domain.TrafficGraph) (map[string]*udpEntryPointRuntime, []*udpEntryPointRuntime, []udpRuntimeUpdate, error) {342 func (m *Manager) prepareUDPListeners(ctx context.Context, graph *domain.TrafficGraph) (map[string]*udpEntryPointRuntime, []*udpEntryPointRuntime, []udpRuntimeUpdate, error) {
348 current := make(map[string]*udpEntryPointRuntime, len(m.udpListeners))343 current := make(map[string]*udpEntryPointRuntime, len(m.udpListeners))
349- for name, runtime := range m.udpListeners {344+ maps.Copy(current, m.udpListeners)
350- current[name] = runtime
351- }
352 345
353 next := make(map[string]*udpEntryPointRuntime, len(current))346 next := make(map[string]*udpEntryPointRuntime, len(current))
354 created := []*udpEntryPointRuntime{}347 created := []*udpEntryPointRuntime{}
M internal/adapters/in/traffic/tcp.go +109 -2
@@ -6,6 +6,7 @@ import (
66 "fmt"
77 "io"
88 "net"
9+ "net/http"
910 "strconv"
1011 "strings"
1112 "sync"
@@ -33,6 +34,10 @@ type entryPointRuntime struct {
3334
3435 mu sync.Mutex
3536 activeConns map[*trackedTCPConn]struct{}
37+
38+ tlsHTTPListener *tlsHTTPListener
39+ tlsHTTPServer *http.Server
40+ tlsHTTPDone chan struct{}
3641 }
3742
3843 type trackedTCPConn struct {
@@ -59,10 +64,64 @@ func (r *entryPointRuntime) start() {
5964 return
6065 }
6166 entryPoint := r.entryPointSnapshot()
67+ r.startTLSHTTPServer(entryPoint)
6268 trafficInfo(r.ctx).Str("entrypoint", entryPoint.Name).Str("address", entryPoint.Address).Str("protocol", string(entryPoint.Protocol)).Msg("started tcp traffic entrypoint")
6369 go r.acceptLoop()
6470 }
6571
72+func (r *entryPointRuntime) startTLSHTTPServer(entryPoint domain.EntryPoint) {
73+ if entryPoint.Protocol != domain.EntryPointProtocolTLSMux {
74+ return
75+ }
76+ config, ok := r.manager.tlsHTTPServer(entryPoint.Name)
77+ if !ok || config.Handler == nil || config.TLSConfig == nil {
78+ return
79+ }
80+ r.mu.Lock()
81+ alreadyStarted := r.tlsHTTPServer != nil
82+ r.mu.Unlock()
83+ if alreadyStarted {
84+ return
85+ }
86+ listener := newTLSHTTPListener(r.listener.Addr())
87+ server := &http.Server{
88+ Handler: config.Handler,
89+ TLSConfig: config.TLSConfig.Clone(),
90+ ReadHeaderTimeout: 5 * time.Second,
91+ ReadTimeout: 10 * time.Second,
92+ WriteTimeout: 10 * time.Second,
93+ IdleTimeout: 30 * time.Second,
94+ }
95+ done := make(chan struct{})
96+
97+ r.mu.Lock()
98+ r.tlsHTTPListener = listener
99+ r.tlsHTTPServer = server
100+ r.tlsHTTPDone = done
101+ r.mu.Unlock()
102+
103+ go func() {
104+ defer close(done)
105+ if err := server.ServeTLS(listener, "", ""); err != nil && !errors.Is(err, http.ErrServerClosed) && !errors.Is(err, net.ErrClosed) {
106+ trafficWarn(r.ctx).Err(err).Str("entrypoint", entryPoint.Name).Msg("tls mux https server stopped with error")
107+ }
108+ }()
109+}
110+
111+func (r *entryPointRuntime) routeToHTTPS(conn net.Conn) bool {
112+ r.mu.Lock()
113+ listener := r.tlsHTTPListener
114+ r.mu.Unlock()
115+ if listener == nil {
116+ return false
117+ }
118+ if listener.serve(conn) {
119+ return true
120+ }
121+ _ = conn.Close()
122+ return true
123+}
124+
66125 func (r *entryPointRuntime) acceptLoop() {
67126 defer r.closeAcceptDone()
68127 for {
@@ -141,9 +200,8 @@ func (r *entryPointRuntime) handleTLSMux(tracked *trackedTCPConn, options domain
141200 r.proxyToBackend(tracked, peeked.conn, backend, options)
142201 return
143202 }
144- if fallback := r.manager.tlsFallback(r.entryPointSnapshot().Name); fallback != nil {
203+ if r.routeToHTTPS(peeked.conn) {
145204 r.counters.totalAccepted.Add(1)
146- fallback(r.ctx, peeked.conn)
147205 return
148206 }
149207 r.counters.totalRefused.Add(1)
@@ -398,6 +456,7 @@ func (r *entryPointRuntime) stop(ctx context.Context, drainTimeout time.Duration
398456 case <-ctx.Done():
399457 return
400458 }
459+ r.stopTLSHTTPServer(ctx, drainTimeout)
401460 if r.waitActive(ctx, drainTimeout) {
402461 trafficInfo(ctx).Str("entrypoint", r.entryPointSnapshot().Name).Msg("stopped tcp traffic entrypoint")
403462 return
@@ -411,6 +470,35 @@ func (r *entryPointRuntime) stop(ctx context.Context, drainTimeout time.Duration
411470 }
412471 }
413472
473+func (r *entryPointRuntime) stopTLSHTTPServer(ctx context.Context, drainTimeout time.Duration) {
474+ r.mu.Lock()
475+ listener := r.tlsHTTPListener
476+ server := r.tlsHTTPServer
477+ done := r.tlsHTTPDone
478+ r.tlsHTTPListener = nil
479+ r.tlsHTTPServer = nil
480+ r.tlsHTTPDone = nil
481+ r.mu.Unlock()
482+
483+ if listener == nil || server == nil || done == nil {
484+ return
485+ }
486+ _ = listener.Close()
487+ if drainTimeout <= 0 {
488+ drainTimeout = defaultTCPOptions().DrainTimeout
489+ }
490+ shutdownCtx, cancel := context.WithTimeout(ctx, drainTimeout)
491+ defer cancel()
492+ if err := server.Shutdown(shutdownCtx); err != nil {
493+ trafficDebug(ctx).Err(err).Str("entrypoint", r.entryPointSnapshot().Name).Msg("forcing tls mux https server close")
494+ _ = server.Close()
495+ }
496+ select {
497+ case <-done:
498+ case <-ctx.Done():
499+ }
500+}
501+
414502 func (r *entryPointRuntime) waitActive(ctx context.Context, drainTimeout time.Duration) bool {
415503 if drainTimeout <= 0 {
416504 drainTimeout = defaultTCPOptions().DrainTimeout
@@ -460,9 +548,28 @@ func (r *entryPointRuntime) sameAddress(entryPoint domain.EntryPoint) bool {
460548
461549 func (r *entryPointRuntime) updateEntryPoint(entryPoint domain.EntryPoint, trusted []*net.IPNet) {
462550 r.mu.Lock()
551+ previousProtocol := r.entryPoint.Protocol
463552 r.entryPoint = entryPoint
464553 r.trusted = trusted
554+ stale := make([]*trackedTCPConn, 0)
555+ for conn := range r.activeConns {
556+ if !trustedRemoteAddr(trusted, conn.client.RemoteAddr()) {
557+ stale = append(stale, conn)
558+ }
559+ }
465560 r.mu.Unlock()
561+ for _, conn := range stale {
562+ _ = conn.client.Close()
563+ if conn.backend != nil {
564+ _ = conn.backend.Close()
565+ }
566+ }
567+ if previousProtocol == domain.EntryPointProtocolTLSMux && entryPoint.Protocol != domain.EntryPointProtocolTLSMux {
568+ r.stopTLSHTTPServer(r.ctx, 0)
569+ }
570+ if entryPoint.Protocol == domain.EntryPointProtocolTLSMux {
571+ r.startTLSHTTPServer(entryPoint)
572+ }
466573 }
467574
468575 func (r *entryPointRuntime) entryPointSnapshot() domain.EntryPoint {
@@ -6,6 +6,7 @@ import (
6 "fmt"6 "fmt"
7 "io"7 "io"
8 "net"8 "net"
9+ "net/http"
9 "strconv"10 "strconv"
10 "strings"11 "strings"
11 "sync"12 "sync"
@@ -33,6 +34,10 @@ type entryPointRuntime struct {
33 34
34 mu sync.Mutex35 mu sync.Mutex
35 activeConns map[*trackedTCPConn]struct{}36 activeConns map[*trackedTCPConn]struct{}
37+
38+ tlsHTTPListener *tlsHTTPListener
39+ tlsHTTPServer *http.Server
40+ tlsHTTPDone chan struct{}
36 }41 }
37 42
38 type trackedTCPConn struct {43 type trackedTCPConn struct {
@@ -59,10 +64,64 @@ func (r *entryPointRuntime) start() {
59 return64 return
60 }65 }
61 entryPoint := r.entryPointSnapshot()66 entryPoint := r.entryPointSnapshot()
67+ r.startTLSHTTPServer(entryPoint)
62 trafficInfo(r.ctx).Str("entrypoint", entryPoint.Name).Str("address", entryPoint.Address).Str("protocol", string(entryPoint.Protocol)).Msg("started tcp traffic entrypoint")68 trafficInfo(r.ctx).Str("entrypoint", entryPoint.Name).Str("address", entryPoint.Address).Str("protocol", string(entryPoint.Protocol)).Msg("started tcp traffic entrypoint")
63 go r.acceptLoop()69 go r.acceptLoop()
64 }70 }
65 71
72+func (r *entryPointRuntime) startTLSHTTPServer(entryPoint domain.EntryPoint) {
73+ if entryPoint.Protocol != domain.EntryPointProtocolTLSMux {
74+ return
75+ }
76+ config, ok := r.manager.tlsHTTPServer(entryPoint.Name)
77+ if !ok || config.Handler == nil || config.TLSConfig == nil {
78+ return
79+ }
80+ r.mu.Lock()
81+ alreadyStarted := r.tlsHTTPServer != nil
82+ r.mu.Unlock()
83+ if alreadyStarted {
84+ return
85+ }
86+ listener := newTLSHTTPListener(r.listener.Addr())
87+ server := &http.Server{
88+ Handler: config.Handler,
89+ TLSConfig: config.TLSConfig.Clone(),
90+ ReadHeaderTimeout: 5 * time.Second,
91+ ReadTimeout: 10 * time.Second,
92+ WriteTimeout: 10 * time.Second,
93+ IdleTimeout: 30 * time.Second,
94+ }
95+ done := make(chan struct{})
96+
97+ r.mu.Lock()
98+ r.tlsHTTPListener = listener
99+ r.tlsHTTPServer = server
100+ r.tlsHTTPDone = done
101+ r.mu.Unlock()
102+
103+ go func() {
104+ defer close(done)
105+ if err := server.ServeTLS(listener, "", ""); err != nil && !errors.Is(err, http.ErrServerClosed) && !errors.Is(err, net.ErrClosed) {
106+ trafficWarn(r.ctx).Err(err).Str("entrypoint", entryPoint.Name).Msg("tls mux https server stopped with error")
107+ }
108+ }()
109+}
110+
111+func (r *entryPointRuntime) routeToHTTPS(conn net.Conn) bool {
112+ r.mu.Lock()
113+ listener := r.tlsHTTPListener
114+ r.mu.Unlock()
115+ if listener == nil {
116+ return false
117+ }
118+ if listener.serve(conn) {
119+ return true
120+ }
121+ _ = conn.Close()
122+ return true
123+}
124+
66 func (r *entryPointRuntime) acceptLoop() {125 func (r *entryPointRuntime) acceptLoop() {
67 defer r.closeAcceptDone()126 defer r.closeAcceptDone()
68 for {127 for {
@@ -141,9 +200,8 @@ func (r *entryPointRuntime) handleTLSMux(tracked *trackedTCPConn, options domain
141 r.proxyToBackend(tracked, peeked.conn, backend, options)200 r.proxyToBackend(tracked, peeked.conn, backend, options)
142 return201 return
143 }202 }
144- if fallback := r.manager.tlsFallback(r.entryPointSnapshot().Name); fallback != nil {203+ if r.routeToHTTPS(peeked.conn) {
145 r.counters.totalAccepted.Add(1)204 r.counters.totalAccepted.Add(1)
146- fallback(r.ctx, peeked.conn)
147 return205 return
148 }206 }
149 r.counters.totalRefused.Add(1)207 r.counters.totalRefused.Add(1)
@@ -398,6 +456,7 @@ func (r *entryPointRuntime) stop(ctx context.Context, drainTimeout time.Duration
398 case <-ctx.Done():456 case <-ctx.Done():
399 return457 return
400 }458 }
459+ r.stopTLSHTTPServer(ctx, drainTimeout)
401 if r.waitActive(ctx, drainTimeout) {460 if r.waitActive(ctx, drainTimeout) {
402 trafficInfo(ctx).Str("entrypoint", r.entryPointSnapshot().Name).Msg("stopped tcp traffic entrypoint")461 trafficInfo(ctx).Str("entrypoint", r.entryPointSnapshot().Name).Msg("stopped tcp traffic entrypoint")
403 return462 return
@@ -411,6 +470,35 @@ func (r *entryPointRuntime) stop(ctx context.Context, drainTimeout time.Duration
411 }470 }
412 }471 }
413 472
473+func (r *entryPointRuntime) stopTLSHTTPServer(ctx context.Context, drainTimeout time.Duration) {
474+ r.mu.Lock()
475+ listener := r.tlsHTTPListener
476+ server := r.tlsHTTPServer
477+ done := r.tlsHTTPDone
478+ r.tlsHTTPListener = nil
479+ r.tlsHTTPServer = nil
480+ r.tlsHTTPDone = nil
481+ r.mu.Unlock()
482+
483+ if listener == nil || server == nil || done == nil {
484+ return
485+ }
486+ _ = listener.Close()
487+ if drainTimeout <= 0 {
488+ drainTimeout = defaultTCPOptions().DrainTimeout
489+ }
490+ shutdownCtx, cancel := context.WithTimeout(ctx, drainTimeout)
491+ defer cancel()
492+ if err := server.Shutdown(shutdownCtx); err != nil {
493+ trafficDebug(ctx).Err(err).Str("entrypoint", r.entryPointSnapshot().Name).Msg("forcing tls mux https server close")
494+ _ = server.Close()
495+ }
496+ select {
497+ case <-done:
498+ case <-ctx.Done():
499+ }
500+}
501+
414 func (r *entryPointRuntime) waitActive(ctx context.Context, drainTimeout time.Duration) bool {502 func (r *entryPointRuntime) waitActive(ctx context.Context, drainTimeout time.Duration) bool {
415 if drainTimeout <= 0 {503 if drainTimeout <= 0 {
416 drainTimeout = defaultTCPOptions().DrainTimeout504 drainTimeout = defaultTCPOptions().DrainTimeout
@@ -460,9 +548,28 @@ func (r *entryPointRuntime) sameAddress(entryPoint domain.EntryPoint) bool {
460 548
461 func (r *entryPointRuntime) updateEntryPoint(entryPoint domain.EntryPoint, trusted []*net.IPNet) {549 func (r *entryPointRuntime) updateEntryPoint(entryPoint domain.EntryPoint, trusted []*net.IPNet) {
462 r.mu.Lock()550 r.mu.Lock()
551+ previousProtocol := r.entryPoint.Protocol
463 r.entryPoint = entryPoint552 r.entryPoint = entryPoint
464 r.trusted = trusted553 r.trusted = trusted
554+ stale := make([]*trackedTCPConn, 0)
555+ for conn := range r.activeConns {
556+ if !trustedRemoteAddr(trusted, conn.client.RemoteAddr()) {
557+ stale = append(stale, conn)
558+ }
559+ }
465 r.mu.Unlock()560 r.mu.Unlock()
561+ for _, conn := range stale {
562+ _ = conn.client.Close()
563+ if conn.backend != nil {
564+ _ = conn.backend.Close()
565+ }
566+ }
567+ if previousProtocol == domain.EntryPointProtocolTLSMux && entryPoint.Protocol != domain.EntryPointProtocolTLSMux {
568+ r.stopTLSHTTPServer(r.ctx, 0)
569+ }
570+ if entryPoint.Protocol == domain.EntryPointProtocolTLSMux {
571+ r.startTLSHTTPServer(entryPoint)
572+ }
466 }573 }
467 574
468 func (r *entryPointRuntime) entryPointSnapshot() domain.EntryPoint {575 func (r *entryPointRuntime) entryPointSnapshot() domain.EntryPoint {
M internal/adapters/in/traffic/tcp_test.go +69 -0
@@ -3,9 +3,11 @@ package traffic
33 import (
44 "bufio"
55 "context"
6+ "crypto/tls"
67 "errors"
78 "io"
89 "net"
10+ "net/http"
911 "testing"
1012 "time"
1113
@@ -81,6 +83,46 @@ func TestTCPManagerReplacesSameAddressEntrypoint(t *testing.T) {
8183 assert.Equal(t, domain.EntryPointProtocolTLSMux, status.EntryPoints[0].Protocol)
8284 }
8385
86+func TestTCPManagerSameAddressProtocolChangeStartsTLSHTTPServer(t *testing.T) {
87+ backend := startTCPEchoServer(t, 0)
88+ graph := tcpGraph(t, freeTCPAddress(t), backend.address)
89+ manager := NewManager()
90+ require.NoError(t, manager.Apply(context.Background(), &graph))
91+ defer shutdownManager(t, manager)
92+
93+ manager.SetTLSHTTPServer("websecure", http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
94+ _, _ = w.Write([]byte("secure"))
95+ }), testTLSConfig(t, "app.example.com"))
96+ replacement := domain.TrafficGraph{
97+ Options: graph.Options,
98+ EntryPoints: []domain.EntryPoint{{Name: "websecure", Address: graph.EntryPoints[0].Address, Protocol: domain.EntryPointProtocolTLSMux}},
99+ }
100+ require.NoError(t, manager.Apply(context.Background(), &replacement))
101+
102+ assertHTTPSBody(t, graph.EntryPoints[0].Address, "app.example.com", "secure")
103+}
104+
105+func TestTCPManagerSameAddressProtocolChangeStopsTLSHTTPServer(t *testing.T) {
106+ backend := startTCPEchoServer(t, 0)
107+ graph := domain.TrafficGraph{
108+ Options: domain.TrafficOptions{TCP: domain.TCPOptions{DialTimeout: time.Second, IdleTimeout: time.Minute, DrainTimeout: time.Second}},
109+ EntryPoints: []domain.EntryPoint{{Name: "websecure", Address: freeTCPAddress(t), Protocol: domain.EntryPointProtocolTLSMux}},
110+ }
111+ manager := NewManager()
112+ manager.SetTLSHTTPServer("websecure", http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
113+ _, _ = w.Write([]byte("secure"))
114+ }), testTLSConfig(t, "app.example.com"))
115+ require.NoError(t, manager.Apply(context.Background(), &graph))
116+ defer shutdownManager(t, manager)
117+ assertHTTPSBody(t, graph.EntryPoints[0].Address, "app.example.com", "secure")
118+
119+ replacement := tcpGraph(t, graph.EntryPoints[0].Address, backend.address)
120+ require.NoError(t, manager.Apply(context.Background(), &replacement))
121+
122+ _, err := tls.DialWithDialer(&net.Dialer{Timeout: time.Second}, "tcp", graph.EntryPoints[0].Address, &tls.Config{ServerName: "app.example.com", InsecureSkipVerify: true})
123+ require.Error(t, err)
124+}
125+
84126 func TestTCPPassthroughRejectsUntrustedCIDR(t *testing.T) {
85127 backend := startTCPEchoServer(t, 0)
86128 graph := tcpGraph(t, freeTCPAddress(t), backend.address)
@@ -110,6 +152,33 @@ func TestTCPPassthroughReloadAppliesTrustedCIDRChange(t *testing.T) {
110152 assertTCPRejected(t, manager, graph.EntryPoints[0].Address, 1)
111153 }
112154
155+func TestTCPPassthroughReloadClosesActiveUntrustedConnections(t *testing.T) {
156+ backend := startTCPEchoServer(t, 0)
157+ graph := tcpGraph(t, freeTCPAddress(t), backend.address)
158+ graph.EntryPoints[0].TrustedCIDRs = []string{"127.0.0.0/8"}
159+ manager := NewManager()
160+ require.NoError(t, manager.Apply(context.Background(), &graph))
161+ defer shutdownManager(t, manager)
162+
163+ conn := dialTCP(t, graph.EntryPoints[0].Address)
164+ defer conn.Close()
165+ assertRoundTrip(t, conn, "before")
166+
167+ updated := graph
168+ updated.EntryPoints[0].TrustedCIDRs = []string{"192.0.2.0/24"}
169+ require.NoError(t, manager.Apply(context.Background(), &updated))
170+
171+ assert.Eventually(t, func() bool {
172+ _, writeErr := conn.Write([]byte("after"))
173+ if writeErr != nil {
174+ return true
175+ }
176+ _ = conn.SetReadDeadline(time.Now().Add(10 * time.Millisecond))
177+ _, readErr := bufio.NewReader(conn).ReadByte()
178+ return readErr != nil
179+ }, time.Second, 10*time.Millisecond)
180+}
181+
113182 func assertTCPRejected(t *testing.T, manager *Manager, address string, refused int64) {
114183 t.Helper()
115184 conn := dialTCP(t, address)
@@ -3,9 +3,11 @@ package traffic
3 import (3 import (
4 "bufio"4 "bufio"
5 "context"5 "context"
6+ "crypto/tls"
6 "errors"7 "errors"
7 "io"8 "io"
8 "net"9 "net"
10+ "net/http"
9 "testing"11 "testing"
10 "time"12 "time"
11 13
@@ -81,6 +83,46 @@ func TestTCPManagerReplacesSameAddressEntrypoint(t *testing.T) {
81 assert.Equal(t, domain.EntryPointProtocolTLSMux, status.EntryPoints[0].Protocol)83 assert.Equal(t, domain.EntryPointProtocolTLSMux, status.EntryPoints[0].Protocol)
82 }84 }
83 85
86+func TestTCPManagerSameAddressProtocolChangeStartsTLSHTTPServer(t *testing.T) {
87+ backend := startTCPEchoServer(t, 0)
88+ graph := tcpGraph(t, freeTCPAddress(t), backend.address)
89+ manager := NewManager()
90+ require.NoError(t, manager.Apply(context.Background(), &graph))
91+ defer shutdownManager(t, manager)
92+
93+ manager.SetTLSHTTPServer("websecure", http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
94+ _, _ = w.Write([]byte("secure"))
95+ }), testTLSConfig(t, "app.example.com"))
96+ replacement := domain.TrafficGraph{
97+ Options: graph.Options,
98+ EntryPoints: []domain.EntryPoint{{Name: "websecure", Address: graph.EntryPoints[0].Address, Protocol: domain.EntryPointProtocolTLSMux}},
99+ }
100+ require.NoError(t, manager.Apply(context.Background(), &replacement))
101+
102+ assertHTTPSBody(t, graph.EntryPoints[0].Address, "app.example.com", "secure")
103+}
104+
105+func TestTCPManagerSameAddressProtocolChangeStopsTLSHTTPServer(t *testing.T) {
106+ backend := startTCPEchoServer(t, 0)
107+ graph := domain.TrafficGraph{
108+ Options: domain.TrafficOptions{TCP: domain.TCPOptions{DialTimeout: time.Second, IdleTimeout: time.Minute, DrainTimeout: time.Second}},
109+ EntryPoints: []domain.EntryPoint{{Name: "websecure", Address: freeTCPAddress(t), Protocol: domain.EntryPointProtocolTLSMux}},
110+ }
111+ manager := NewManager()
112+ manager.SetTLSHTTPServer("websecure", http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
113+ _, _ = w.Write([]byte("secure"))
114+ }), testTLSConfig(t, "app.example.com"))
115+ require.NoError(t, manager.Apply(context.Background(), &graph))
116+ defer shutdownManager(t, manager)
117+ assertHTTPSBody(t, graph.EntryPoints[0].Address, "app.example.com", "secure")
118+
119+ replacement := tcpGraph(t, graph.EntryPoints[0].Address, backend.address)
120+ require.NoError(t, manager.Apply(context.Background(), &replacement))
121+
122+ _, err := tls.DialWithDialer(&net.Dialer{Timeout: time.Second}, "tcp", graph.EntryPoints[0].Address, &tls.Config{ServerName: "app.example.com", InsecureSkipVerify: true})
123+ require.Error(t, err)
124+}
125+
84 func TestTCPPassthroughRejectsUntrustedCIDR(t *testing.T) {126 func TestTCPPassthroughRejectsUntrustedCIDR(t *testing.T) {
85 backend := startTCPEchoServer(t, 0)127 backend := startTCPEchoServer(t, 0)
86 graph := tcpGraph(t, freeTCPAddress(t), backend.address)128 graph := tcpGraph(t, freeTCPAddress(t), backend.address)
@@ -110,6 +152,33 @@ func TestTCPPassthroughReloadAppliesTrustedCIDRChange(t *testing.T) {
110 assertTCPRejected(t, manager, graph.EntryPoints[0].Address, 1)152 assertTCPRejected(t, manager, graph.EntryPoints[0].Address, 1)
111 }153 }
112 154
155+func TestTCPPassthroughReloadClosesActiveUntrustedConnections(t *testing.T) {
156+ backend := startTCPEchoServer(t, 0)
157+ graph := tcpGraph(t, freeTCPAddress(t), backend.address)
158+ graph.EntryPoints[0].TrustedCIDRs = []string{"127.0.0.0/8"}
159+ manager := NewManager()
160+ require.NoError(t, manager.Apply(context.Background(), &graph))
161+ defer shutdownManager(t, manager)
162+
163+ conn := dialTCP(t, graph.EntryPoints[0].Address)
164+ defer conn.Close()
165+ assertRoundTrip(t, conn, "before")
166+
167+ updated := graph
168+ updated.EntryPoints[0].TrustedCIDRs = []string{"192.0.2.0/24"}
169+ require.NoError(t, manager.Apply(context.Background(), &updated))
170+
171+ assert.Eventually(t, func() bool {
172+ _, writeErr := conn.Write([]byte("after"))
173+ if writeErr != nil {
174+ return true
175+ }
176+ _ = conn.SetReadDeadline(time.Now().Add(10 * time.Millisecond))
177+ _, readErr := bufio.NewReader(conn).ReadByte()
178+ return readErr != nil
179+ }, time.Second, 10*time.Millisecond)
180+}
181+
113 func assertTCPRejected(t *testing.T, manager *Manager, address string, refused int64) {182 func assertTCPRejected(t *testing.T, manager *Manager, address string, refused int64) {
114 t.Helper()183 t.Helper()
115 conn := dialTCP(t, address)184 conn := dialTCP(t, address)
M internal/adapters/in/traffic/tls_mux.go +45 -86
@@ -2,52 +2,52 @@ package traffic
22
33 import (
44 "bytes"
5- "context"
65 "crypto/tls"
7- "errors"
86 "fmt"
7+ "maps"
98 "net"
109 "net/http"
1110 "strings"
1211 "sync"
13- "time"
1412 )
1513
1614 const maxClientHelloBytes = 64 << 10
1715
18-// TLSFallback handles tls_mux connections that do not match a passthrough router.
19-type TLSFallback func(context.Context, net.Conn)
16+// TLSHTTPServerConfig describes the HTTPS server attached to a tls_mux entrypoint.
17+type TLSHTTPServerConfig struct {
18+ Handler http.Handler
19+ TLSConfig *tls.Config
20+}
2021
21-type tlsFallbacks map[string]TLSFallback
22+type tlsHTTPServers map[string]TLSHTTPServerConfig
2223
23-// SetTLSFallback installs or removes an HTTPS fallback for a tls_mux entrypoint.
24-func (m *Manager) SetTLSFallback(entryPoint string, fallback TLSFallback) {
24+// SetTLSHTTPServer installs or removes the HTTPS server for a tls_mux entrypoint.
25+func (m *Manager) SetTLSHTTPServer(entryPoint string, handler http.Handler, tlsConfig *tls.Config) {
2526 m.mu.Lock()
2627 defer m.mu.Unlock()
2728
28- current := m.loadTLSFallbacks()
29- next := make(tlsFallbacks, len(current)+1)
30- for name, value := range current {
31- next[name] = value
32- }
33- if fallback == nil {
29+ current := m.loadTLSHTTPServers()
30+ next := make(tlsHTTPServers, len(current)+1)
31+ maps.Copy(next, current)
32+ if handler == nil || tlsConfig == nil {
3433 delete(next, entryPoint)
3534 } else {
36- next[entryPoint] = fallback
35+ next[entryPoint] = TLSHTTPServerConfig{Handler: handler, TLSConfig: tlsConfig.Clone()}
3736 }
38- m.tlsFallbacks.Store(next)
37+ m.tlsHTTPServers.Store(next)
3938 }
4039
41-func (m *Manager) tlsFallback(entryPoint string) TLSFallback {
42- return m.loadTLSFallbacks()[entryPoint]
40+func (m *Manager) tlsHTTPServer(entryPoint string) (TLSHTTPServerConfig, bool) {
41+ config, ok := m.loadTLSHTTPServers()[entryPoint]
42+ return config, ok
4343 }
4444
45-func (m *Manager) loadTLSFallbacks() tlsFallbacks {
46- value := m.tlsFallbacks.Load()
45+func (m *Manager) loadTLSHTTPServers() tlsHTTPServers {
46+ value := m.tlsHTTPServers.Load()
4747 if value == nil {
48- return tlsFallbacks{}
48+ return tlsHTTPServers{}
4949 }
50- return value.(tlsFallbacks)
50+ return value.(tlsHTTPServers)
5151 }
5252
5353 func peekClientHelloSNI(conn net.Conn) (string, net.Conn, error) {
@@ -234,7 +234,7 @@ func readLength(reader *bytes.Reader, size int) (int, error) {
234234 return 0, fmt.Errorf("client hello length is truncated")
235235 }
236236 value := 0
237- for i := 0; i < size; i++ {
237+ for range size {
238238 b, _ := reader.ReadByte()
239239 value = value<<8 | int(b)
240240 }
@@ -261,79 +261,38 @@ func (c replayConn) Read(p []byte) (int, error) {
261261 return c.Conn.Read(p)
262262 }
263263
264-// ServeHTTPSFallback serves a single TLS connection with the provided HTTP handler.
265-func ServeHTTPSFallback(ctx context.Context, conn net.Conn, handler http.Handler, tlsConfig *tls.Config) error {
266- if handler == nil {
267- return fmt.Errorf("https fallback handler is required")
268- }
269- if tlsConfig == nil {
270- return fmt.Errorf("https fallback tls config is required")
271- }
272- tracked := newTrackedCloseConn(tls.Server(conn, tlsConfig))
273- listener := newSingleConnListener(tracked)
274- server := &http.Server{Handler: handler, ReadHeaderTimeout: 5 * time.Second, ReadTimeout: 10 * time.Second, WriteTimeout: 10 * time.Second, IdleTimeout: 30 * time.Second}
275- errCh := make(chan error, 1)
276- go func() { errCh <- server.Serve(listener) }()
277-
278- select {
279- case <-tracked.done:
280- _ = server.Close()
281- err := <-errCh
282- if errors.Is(err, net.ErrClosed) || errors.Is(err, http.ErrServerClosed) {
283- return nil
284- }
285- return err
286- case <-ctx.Done():
287- _ = server.Close()
288- _ = tracked.Close()
289- <-errCh
290- return ctx.Err()
291- }
292-}
293-
294-type singleConnListener struct {
295- conn net.Conn
296- once sync.Once
297- done chan struct{}
298-}
299-
300-func newSingleConnListener(conn net.Conn) *singleConnListener {
301- return &singleConnListener{conn: conn, done: make(chan struct{})}
264+type tlsHTTPListener struct {
265+ addr net.Addr
266+ conns chan net.Conn
267+ done chan struct{}
268+ once sync.Once
302269 }
303270
304-func (l *singleConnListener) Accept() (net.Conn, error) {
305- var conn net.Conn
306- l.once.Do(func() { conn = l.conn })
307- if conn != nil {
308- return conn, nil
309- }
310- <-l.done
311- return nil, net.ErrClosed
271+func newTLSHTTPListener(addr net.Addr) *tlsHTTPListener {
272+ return &tlsHTTPListener{addr: addr, conns: make(chan net.Conn, 128), done: make(chan struct{})}
312273 }
313274
314-func (l *singleConnListener) Close() error {
275+func (l *tlsHTTPListener) Accept() (net.Conn, error) {
315276 select {
277+ case conn := <-l.conns:
278+ return conn, nil
316279 case <-l.done:
317- default:
318- close(l.done)
280+ return nil, net.ErrClosed
319281 }
320- return nil
321282 }
322283
323-func (l *singleConnListener) Addr() net.Addr { return l.conn.LocalAddr() }
324-
325-type trackedCloseConn struct {
326- net.Conn
327- done chan struct{}
328- once sync.Once
284+func (l *tlsHTTPListener) Close() error {
285+ l.once.Do(func() { close(l.done) })
286+ return nil
329287 }
330288
331-func newTrackedCloseConn(conn net.Conn) *trackedCloseConn {
332- return &trackedCloseConn{Conn: conn, done: make(chan struct{})}
333-}
289+func (l *tlsHTTPListener) Addr() net.Addr { return l.addr }
334290
335-func (c *trackedCloseConn) Close() error {
336- err := c.Conn.Close()
337- c.once.Do(func() { close(c.done) })
338- return err
291+func (l *tlsHTTPListener) serve(conn net.Conn) bool {
292+ select {
293+ case l.conns <- conn:
294+ return true
295+ case <-l.done:
296+ return false
297+ }
339298 }
@@ -2,52 +2,52 @@ package traffic
2 2
3 import (3 import (
4 "bytes"4 "bytes"
5- "context"
6 "crypto/tls"5 "crypto/tls"
7- "errors"
8 "fmt"6 "fmt"
7+ "maps"
9 "net"8 "net"
10 "net/http"9 "net/http"
11 "strings"10 "strings"
12 "sync"11 "sync"
13- "time"
14 )12 )
15 13
16 const maxClientHelloBytes = 64 << 1014 const maxClientHelloBytes = 64 << 10
17 15
18-// TLSFallback handles tls_mux connections that do not match a passthrough router.16+// TLSHTTPServerConfig describes the HTTPS server attached to a tls_mux entrypoint.
19-type TLSFallback func(context.Context, net.Conn)17+type TLSHTTPServerConfig struct {
18+ Handler http.Handler
19+ TLSConfig *tls.Config
20+}
20 21
21-type tlsFallbacks map[string]TLSFallback22+type tlsHTTPServers map[string]TLSHTTPServerConfig
22 23
23-// SetTLSFallback installs or removes an HTTPS fallback for a tls_mux entrypoint.24+// SetTLSHTTPServer installs or removes the HTTPS server for a tls_mux entrypoint.
24-func (m *Manager) SetTLSFallback(entryPoint string, fallback TLSFallback) {25+func (m *Manager) SetTLSHTTPServer(entryPoint string, handler http.Handler, tlsConfig *tls.Config) {
25 m.mu.Lock()26 m.mu.Lock()
26 defer m.mu.Unlock()27 defer m.mu.Unlock()
27 28
28- current := m.loadTLSFallbacks()29+ current := m.loadTLSHTTPServers()
29- next := make(tlsFallbacks, len(current)+1)30+ next := make(tlsHTTPServers, len(current)+1)
30- for name, value := range current {31+ maps.Copy(next, current)
31- next[name] = value32+ if handler == nil || tlsConfig == nil {
32- }
33- if fallback == nil {
34 delete(next, entryPoint)33 delete(next, entryPoint)
35 } else {34 } else {
36- next[entryPoint] = fallback35+ next[entryPoint] = TLSHTTPServerConfig{Handler: handler, TLSConfig: tlsConfig.Clone()}
37 }36 }
38- m.tlsFallbacks.Store(next)37+ m.tlsHTTPServers.Store(next)
39 }38 }
40 39
41-func (m *Manager) tlsFallback(entryPoint string) TLSFallback {40+func (m *Manager) tlsHTTPServer(entryPoint string) (TLSHTTPServerConfig, bool) {
42- return m.loadTLSFallbacks()[entryPoint]41+ config, ok := m.loadTLSHTTPServers()[entryPoint]
42+ return config, ok
43 }43 }
44 44
45-func (m *Manager) loadTLSFallbacks() tlsFallbacks {45+func (m *Manager) loadTLSHTTPServers() tlsHTTPServers {
46- value := m.tlsFallbacks.Load()46+ value := m.tlsHTTPServers.Load()
47 if value == nil {47 if value == nil {
48- return tlsFallbacks{}48+ return tlsHTTPServers{}
49 }49 }
50- return value.(tlsFallbacks)50+ return value.(tlsHTTPServers)
51 }51 }
52 52
53 func peekClientHelloSNI(conn net.Conn) (string, net.Conn, error) {53 func peekClientHelloSNI(conn net.Conn) (string, net.Conn, error) {
@@ -234,7 +234,7 @@ func readLength(reader *bytes.Reader, size int) (int, error) {
234 return 0, fmt.Errorf("client hello length is truncated")234 return 0, fmt.Errorf("client hello length is truncated")
235 }235 }
236 value := 0236 value := 0
237- for i := 0; i < size; i++ {237+ for range size {
238 b, _ := reader.ReadByte()238 b, _ := reader.ReadByte()
239 value = value<<8 | int(b)239 value = value<<8 | int(b)
240 }240 }
@@ -261,79 +261,38 @@ func (c replayConn) Read(p []byte) (int, error) {
261 return c.Conn.Read(p)261 return c.Conn.Read(p)
262 }262 }
263 263
264-// ServeHTTPSFallback serves a single TLS connection with the provided HTTP handler.264+type tlsHTTPListener struct {
265-func ServeHTTPSFallback(ctx context.Context, conn net.Conn, handler http.Handler, tlsConfig *tls.Config) error {265+ addr net.Addr
266- if handler == nil {266+ conns chan net.Conn
267- return fmt.Errorf("https fallback handler is required")267+ done chan struct{}
268- }268+ once sync.Once
269- if tlsConfig == nil {
270- return fmt.Errorf("https fallback tls config is required")
271- }
272- tracked := newTrackedCloseConn(tls.Server(conn, tlsConfig))
273- listener := newSingleConnListener(tracked)
274- server := &http.Server{Handler: handler, ReadHeaderTimeout: 5 * time.Second, ReadTimeout: 10 * time.Second, WriteTimeout: 10 * time.Second, IdleTimeout: 30 * time.Second}
275- errCh := make(chan error, 1)
276- go func() { errCh <- server.Serve(listener) }()
277-
278- select {
279- case <-tracked.done:
280- _ = server.Close()
281- err := <-errCh
282- if errors.Is(err, net.ErrClosed) || errors.Is(err, http.ErrServerClosed) {
283- return nil
284- }
285- return err
286- case <-ctx.Done():
287- _ = server.Close()
288- _ = tracked.Close()
289- <-errCh
290- return ctx.Err()
291- }
292-}
293-
294-type singleConnListener struct {
295- conn net.Conn
296- once sync.Once
297- done chan struct{}
298-}
299-
300-func newSingleConnListener(conn net.Conn) *singleConnListener {
301- return &singleConnListener{conn: conn, done: make(chan struct{})}
302 }269 }
303 270
304-func (l *singleConnListener) Accept() (net.Conn, error) {271+func newTLSHTTPListener(addr net.Addr) *tlsHTTPListener {
305- var conn net.Conn272+ return &tlsHTTPListener{addr: addr, conns: make(chan net.Conn, 128), done: make(chan struct{})}
306- l.once.Do(func() { conn = l.conn })
307- if conn != nil {
308- return conn, nil
309- }
310- <-l.done
311- return nil, net.ErrClosed
312 }273 }
313 274
314-func (l *singleConnListener) Close() error {275+func (l *tlsHTTPListener) Accept() (net.Conn, error) {
315 select {276 select {
277+ case conn := <-l.conns:
278+ return conn, nil
316 case <-l.done:279 case <-l.done:
317- default:280+ return nil, net.ErrClosed
318- close(l.done)
319 }281 }
320- return nil
321 }282 }
322 283
323-func (l *singleConnListener) Addr() net.Addr { return l.conn.LocalAddr() }284+func (l *tlsHTTPListener) Close() error {
324-285+ l.once.Do(func() { close(l.done) })
325-type trackedCloseConn struct {286+ return nil
326- net.Conn
327- done chan struct{}
328- once sync.Once
329 }287 }
330 288
331-func newTrackedCloseConn(conn net.Conn) *trackedCloseConn {289+func (l *tlsHTTPListener) Addr() net.Addr { return l.addr }
332- return &trackedCloseConn{Conn: conn, done: make(chan struct{})}
333-}
334 290
335-func (c *trackedCloseConn) Close() error {291+func (l *tlsHTTPListener) serve(conn net.Conn) bool {
336- err := c.Conn.Close()292+ select {
337- c.once.Do(func() { close(c.done) })293+ case l.conns <- conn:
338- return err294+ return true
295+ case <-l.done:
296+ return false
297+ }
339 }298 }
M internal/adapters/in/traffic/tls_mux_test.go +43 -12
@@ -137,7 +137,7 @@ func TestTLSPassthrough(t *testing.T) {
137137 assertTLSGreeting(t, graph.EntryPoints[0].Address, "api.example.com", "backend-exact\n")
138138 }
139139
140-func TestTLSPassthroughUnknownCloses(t *testing.T) {
140+func TestTLSMuxUnknownClosesWithoutHTTPSRoute(t *testing.T) {
141141 backend := startTLSGreetingServer(t, "backend", "backend\n")
142142 graph := tlsGraph(t, freeTCPAddress(t), []tlsRoute{{name: "raw", sni: "raw.example.com", service: "raw", backend: backend.address}})
143143 manager := NewManager()
@@ -169,22 +169,19 @@ func TestTLSMuxIncompleteClientHelloDoesNotHangShutdown(t *testing.T) {
169169 _ = conn.Close()
170170 }
171171
172-func TestTLSMuxHTTPSFallback(t *testing.T) {
172+func TestTLSMuxHTTPSRoute(t *testing.T) {
173173 backend := startTLSGreetingServer(t, "raw-backend", "raw\n")
174174 graph := tlsGraph(t, freeTCPAddress(t), []tlsRoute{{name: "raw", sni: "raw.example.com", service: "raw", backend: backend.address}})
175175 manager := NewManager()
176- fallbackCert := testTLSConfig(t, "fallback.example.com")
177- manager.SetTLSFallback("websecure", func(ctx context.Context, conn net.Conn) {
178- _ = ServeHTTPSFallback(ctx, conn, http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
179- _, _ = w.Write([]byte("fallback"))
180- }), fallbackCert)
181- })
176+ manager.SetTLSHTTPServer("websecure", http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
177+ _, _ = w.Write([]byte("https route"))
178+ }), testTLSConfig(t, "app.example.com"))
182179 require.NoError(t, manager.Apply(context.Background(), &graph))
183180 defer shutdownManager(t, manager)
184181
185182 assertTLSGreeting(t, graph.EntryPoints[0].Address, "raw.example.com", "raw\n")
186- assertHTTPSFallback(t, graph.EntryPoints[0].Address, "app.example.com", "fallback")
187- assertHTTPSFallback(t, graph.EntryPoints[0].Address, "", "fallback")
183+ assertHTTPSBody(t, graph.EntryPoints[0].Address, "app.example.com", "https route")
184+ assertHTTPSBodyNoSNI(t, graph.EntryPoints[0].Address, "https route")
188185
189186 conn := dialTCP(t, graph.EntryPoints[0].Address)
190187 _, err := conn.Write([]byte("malformed"))
@@ -193,6 +190,20 @@ func TestTLSMuxHTTPSFallback(t *testing.T) {
193190 assert.Eventually(t, func() bool { return manager.Status().Counters.TotalErrors >= 1 }, time.Second, 10*time.Millisecond)
194191 }
195192
193+func TestTLSMuxHTTP2Route(t *testing.T) {
194+ graph := tlsGraph(t, freeTCPAddress(t), nil)
195+ manager := NewManager()
196+ tlsConfig := testTLSConfig(t, "app.example.com")
197+ tlsConfig.NextProtos = []string{"h2", "http/1.1"}
198+ manager.SetTLSHTTPServer("websecure", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
199+ _, _ = w.Write([]byte(r.Proto))
200+ }), tlsConfig)
201+ require.NoError(t, manager.Apply(context.Background(), &graph))
202+ defer shutdownManager(t, manager)
203+
204+ assertHTTPSBody(t, graph.EntryPoints[0].Address, "app.example.com", "HTTP/2.0")
205+}
206+
196207 func peekHelloFromTLSClient(t *testing.T, serverName string) (string, net.Conn) {
197208 t.Helper()
198209 client, server := net.Pipe()
@@ -254,9 +265,29 @@ func assertTLSGreeting(t *testing.T, address string, serverName string, want str
254265 assert.Equal(t, want, line)
255266 }
256267
257-func assertHTTPSFallback(t *testing.T, address string, serverName string, wantBody string) {
268+func assertHTTPSBody(t *testing.T, address string, serverName string, wantBody string) {
258269 t.Helper()
259- conn, err := tls.DialWithDialer(&net.Dialer{Timeout: time.Second}, "tcp", address, &tls.Config{ServerName: serverName, InsecureSkipVerify: true})
270+ transport := &http.Transport{
271+ TLSClientConfig: &tls.Config{ServerName: serverName, InsecureSkipVerify: true},
272+ ForceAttemptHTTP2: true,
273+ }
274+ client := &http.Client{Transport: transport, Timeout: time.Second}
275+ defer transport.CloseIdleConnections()
276+
277+ req, err := http.NewRequest(http.MethodGet, "https://"+address+"/", nil)
278+ require.NoError(t, err)
279+ req.Host = serverName
280+ resp, err := client.Do(req)
281+ require.NoError(t, err)
282+ defer resp.Body.Close()
283+ body, err := io.ReadAll(resp.Body)
284+ require.NoError(t, err)
285+ assert.Contains(t, string(body), wantBody)
286+}
287+
288+func assertHTTPSBodyNoSNI(t *testing.T, address string, wantBody string) {
289+ t.Helper()
290+ conn, err := tls.DialWithDialer(&net.Dialer{Timeout: time.Second}, "tcp", address, &tls.Config{InsecureSkipVerify: true})
260291 require.NoError(t, err)
261292 defer conn.Close()
262293 _, err = conn.Write([]byte("GET / HTTP/1.1\r\nHost: app.example.com\r\nConnection: close\r\n\r\n"))
@@ -137,7 +137,7 @@ func TestTLSPassthrough(t *testing.T) {
137 assertTLSGreeting(t, graph.EntryPoints[0].Address, "api.example.com", "backend-exact\n")137 assertTLSGreeting(t, graph.EntryPoints[0].Address, "api.example.com", "backend-exact\n")
138 }138 }
139 139
140-func TestTLSPassthroughUnknownCloses(t *testing.T) {140+func TestTLSMuxUnknownClosesWithoutHTTPSRoute(t *testing.T) {
141 backend := startTLSGreetingServer(t, "backend", "backend\n")141 backend := startTLSGreetingServer(t, "backend", "backend\n")
142 graph := tlsGraph(t, freeTCPAddress(t), []tlsRoute{{name: "raw", sni: "raw.example.com", service: "raw", backend: backend.address}})142 graph := tlsGraph(t, freeTCPAddress(t), []tlsRoute{{name: "raw", sni: "raw.example.com", service: "raw", backend: backend.address}})
143 manager := NewManager()143 manager := NewManager()
@@ -169,22 +169,19 @@ func TestTLSMuxIncompleteClientHelloDoesNotHangShutdown(t *testing.T) {
169 _ = conn.Close()169 _ = conn.Close()
170 }170 }
171 171
172-func TestTLSMuxHTTPSFallback(t *testing.T) {172+func TestTLSMuxHTTPSRoute(t *testing.T) {
173 backend := startTLSGreetingServer(t, "raw-backend", "raw\n")173 backend := startTLSGreetingServer(t, "raw-backend", "raw\n")
174 graph := tlsGraph(t, freeTCPAddress(t), []tlsRoute{{name: "raw", sni: "raw.example.com", service: "raw", backend: backend.address}})174 graph := tlsGraph(t, freeTCPAddress(t), []tlsRoute{{name: "raw", sni: "raw.example.com", service: "raw", backend: backend.address}})
175 manager := NewManager()175 manager := NewManager()
176- fallbackCert := testTLSConfig(t, "fallback.example.com")176+ manager.SetTLSHTTPServer("websecure", http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
177- manager.SetTLSFallback("websecure", func(ctx context.Context, conn net.Conn) {177+ _, _ = w.Write([]byte("https route"))
178- _ = ServeHTTPSFallback(ctx, conn, http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {178+ }), testTLSConfig(t, "app.example.com"))
179- _, _ = w.Write([]byte("fallback"))
180- }), fallbackCert)
181- })
182 require.NoError(t, manager.Apply(context.Background(), &graph))179 require.NoError(t, manager.Apply(context.Background(), &graph))
183 defer shutdownManager(t, manager)180 defer shutdownManager(t, manager)
184 181
185 assertTLSGreeting(t, graph.EntryPoints[0].Address, "raw.example.com", "raw\n")182 assertTLSGreeting(t, graph.EntryPoints[0].Address, "raw.example.com", "raw\n")
186- assertHTTPSFallback(t, graph.EntryPoints[0].Address, "app.example.com", "fallback")183+ assertHTTPSBody(t, graph.EntryPoints[0].Address, "app.example.com", "https route")
187- assertHTTPSFallback(t, graph.EntryPoints[0].Address, "", "fallback")184+ assertHTTPSBodyNoSNI(t, graph.EntryPoints[0].Address, "https route")
188 185
189 conn := dialTCP(t, graph.EntryPoints[0].Address)186 conn := dialTCP(t, graph.EntryPoints[0].Address)
190 _, err := conn.Write([]byte("malformed"))187 _, err := conn.Write([]byte("malformed"))
@@ -193,6 +190,20 @@ func TestTLSMuxHTTPSFallback(t *testing.T) {
193 assert.Eventually(t, func() bool { return manager.Status().Counters.TotalErrors >= 1 }, time.Second, 10*time.Millisecond)190 assert.Eventually(t, func() bool { return manager.Status().Counters.TotalErrors >= 1 }, time.Second, 10*time.Millisecond)
194 }191 }
195 192
193+func TestTLSMuxHTTP2Route(t *testing.T) {
194+ graph := tlsGraph(t, freeTCPAddress(t), nil)
195+ manager := NewManager()
196+ tlsConfig := testTLSConfig(t, "app.example.com")
197+ tlsConfig.NextProtos = []string{"h2", "http/1.1"}
198+ manager.SetTLSHTTPServer("websecure", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
199+ _, _ = w.Write([]byte(r.Proto))
200+ }), tlsConfig)
201+ require.NoError(t, manager.Apply(context.Background(), &graph))
202+ defer shutdownManager(t, manager)
203+
204+ assertHTTPSBody(t, graph.EntryPoints[0].Address, "app.example.com", "HTTP/2.0")
205+}
206+
196 func peekHelloFromTLSClient(t *testing.T, serverName string) (string, net.Conn) {207 func peekHelloFromTLSClient(t *testing.T, serverName string) (string, net.Conn) {
197 t.Helper()208 t.Helper()
198 client, server := net.Pipe()209 client, server := net.Pipe()
@@ -254,9 +265,29 @@ func assertTLSGreeting(t *testing.T, address string, serverName string, want str
254 assert.Equal(t, want, line)265 assert.Equal(t, want, line)
255 }266 }
256 267
257-func assertHTTPSFallback(t *testing.T, address string, serverName string, wantBody string) {268+func assertHTTPSBody(t *testing.T, address string, serverName string, wantBody string) {
258 t.Helper()269 t.Helper()
259- conn, err := tls.DialWithDialer(&net.Dialer{Timeout: time.Second}, "tcp", address, &tls.Config{ServerName: serverName, InsecureSkipVerify: true})270+ transport := &http.Transport{
271+ TLSClientConfig: &tls.Config{ServerName: serverName, InsecureSkipVerify: true},
272+ ForceAttemptHTTP2: true,
273+ }
274+ client := &http.Client{Transport: transport, Timeout: time.Second}
275+ defer transport.CloseIdleConnections()
276+
277+ req, err := http.NewRequest(http.MethodGet, "https://"+address+"/", nil)
278+ require.NoError(t, err)
279+ req.Host = serverName
280+ resp, err := client.Do(req)
281+ require.NoError(t, err)
282+ defer resp.Body.Close()
283+ body, err := io.ReadAll(resp.Body)
284+ require.NoError(t, err)
285+ assert.Contains(t, string(body), wantBody)
286+}
287+
288+func assertHTTPSBodyNoSNI(t *testing.T, address string, wantBody string) {
289+ t.Helper()
290+ conn, err := tls.DialWithDialer(&net.Dialer{Timeout: time.Second}, "tcp", address, &tls.Config{InsecureSkipVerify: true})
260 require.NoError(t, err)291 require.NoError(t, err)
261 defer conn.Close()292 defer conn.Close()
262 _, err = conn.Write([]byte("GET / HTTP/1.1\r\nHost: app.example.com\r\nConnection: close\r\n\r\n"))293 _, err = conn.Write([]byte("GET / HTTP/1.1\r\nHost: app.example.com\r\nConnection: close\r\n\r\n"))
M internal/adapters/in/traffic/udp.go +55 -11
@@ -47,6 +47,7 @@ type udpDatagram struct {
4747 type udpSession struct {
4848 clientAddr net.Addr
4949 backend net.Conn
50+ backendRef domain.TrafficBackend
5051 lastSeen atomic.Int64
5152 done chan struct{}
5253 once sync.Once
@@ -73,7 +74,7 @@ func (r *udpEntryPointRuntime) start() {
7374 }
7475 entryPoint := r.entryPointSnapshot()
7576 trafficInfo(r.ctx).Str("entrypoint", entryPoint.Name).Str("address", entryPoint.Address).Str("protocol", string(entryPoint.Protocol)).Msg("started udp traffic entrypoint")
76- for i := 0; i < udpWorkerCount; i++ {
77+ for range udpWorkerCount {
7778 go r.datagramWorker()
7879 }
7980 go r.readLoop()
@@ -133,24 +134,34 @@ func (r *udpEntryPointRuntime) handleDatagram(clientAddr net.Addr, packet []byte
133134 }
134135
135136 func (r *udpEntryPointRuntime) session(key string, clientAddr net.Addr, options domain.UDPOptions) (*udpSession, bool) {
137+ backend, ok := r.resolveUDPBackend()
138+ if !ok {
139+ r.counters.totalRefused.Add(1)
140+ return nil, false
141+ }
142+
136143 r.mu.Lock()
137144 if session := r.sessions[key]; session != nil {
138- session.touch()
145+ if trafficBackendEqual(session.backendRef, backend) {
146+ session.touch()
147+ r.mu.Unlock()
148+ return session, true
149+ }
150+ delete(r.sessions, key)
151+ r.mu.Unlock()
152+ session.close()
153+ r.counters.activeUDPSessions.Add(-1)
154+ } else {
139155 r.mu.Unlock()
140- return session, true
141156 }
157+
158+ r.mu.Lock()
142159 if options.MaxSessions > 0 && len(r.sessions) >= options.MaxSessions {
143160 r.mu.Unlock()
144161 r.counters.totalRefused.Add(1)
145162 return nil, false
146163 }
147164 r.mu.Unlock()
148-
149- backend, ok := r.resolveUDPBackend()
150- if !ok {
151- r.counters.totalRefused.Add(1)
152- return nil, false
153- }
154165 dialCtx, cancel := context.WithTimeout(r.ctx, udpDialTimeout(options))
155166 backendConn, err := (&net.Dialer{}).DialContext(dialCtx, "udp", net.JoinHostPort(backend.Host, strconv.Itoa(backend.Port)))
156167 cancel()
@@ -158,15 +169,29 @@ func (r *udpEntryPointRuntime) session(key string, clientAddr net.Addr, options
158169 r.counters.totalErrors.Add(1)
159170 return nil, false
160171 }
161- session := &udpSession{clientAddr: clientAddr, backend: backendConn, done: make(chan struct{})}
172+ session := &udpSession{clientAddr: clientAddr, backend: backendConn, backendRef: backend, done: make(chan struct{})}
162173 session.touch()
163174
164175 r.mu.Lock()
165176 if existing := r.sessions[key]; existing != nil {
177+ if trafficBackendEqual(existing.backendRef, backend) {
178+ r.mu.Unlock()
179+ _ = backendConn.Close()
180+ return existing, true
181+ }
182+ delete(r.sessions, key)
183+ r.mu.Unlock()
184+ existing.close()
185+ r.counters.activeUDPSessions.Add(-1)
186+ } else {
166187 r.mu.Unlock()
188+ }
189+ if options.MaxSessions > 0 && r.sessionCount() >= options.MaxSessions {
167190 _ = backendConn.Close()
168- return existing, true
191+ r.counters.totalRefused.Add(1)
192+ return nil, false
169193 }
194+ r.mu.Lock()
170195 if options.MaxSessions > 0 && len(r.sessions) >= options.MaxSessions {
171196 r.mu.Unlock()
172197 _ = backendConn.Close()
@@ -182,6 +207,16 @@ func (r *udpEntryPointRuntime) session(key string, clientAddr net.Addr, options
182207 return session, true
183208 }
184209
210+func (r *udpEntryPointRuntime) sessionCount() int {
211+ r.mu.Lock()
212+ defer r.mu.Unlock()
213+ return len(r.sessions)
214+}
215+
216+func trafficBackendEqual(left domain.TrafficBackend, right domain.TrafficBackend) bool {
217+ return left.Name == right.Name && left.Host == right.Host && left.Port == right.Port && left.Protocol == right.Protocol
218+}
219+
185220 func udpDialTimeout(options domain.UDPOptions) time.Duration {
186221 if options.IdleTimeout > 0 && options.IdleTimeout < 5*time.Second {
187222 return options.IdleTimeout
@@ -387,7 +422,16 @@ func (r *udpEntryPointRuntime) updateEntryPoint(entryPoint domain.EntryPoint, tr
387422 r.mu.Lock()
388423 r.entryPoint = entryPoint
389424 r.trusted = trusted
425+ stale := make([]string, 0)
426+ for key, session := range r.sessions {
427+ if !trustedRemoteAddr(trusted, session.clientAddr) {
428+ stale = append(stale, key)
429+ }
430+ }
390431 r.mu.Unlock()
432+ for _, key := range stale {
433+ r.removeSession(key)
434+ }
391435 }
392436
393437 func (r *udpEntryPointRuntime) entryPointSnapshot() domain.EntryPoint {
@@ -47,6 +47,7 @@ type udpDatagram struct {
47 type udpSession struct {47 type udpSession struct {
48 clientAddr net.Addr48 clientAddr net.Addr
49 backend net.Conn49 backend net.Conn
50+ backendRef domain.TrafficBackend
50 lastSeen atomic.Int6451 lastSeen atomic.Int64
51 done chan struct{}52 done chan struct{}
52 once sync.Once53 once sync.Once
@@ -73,7 +74,7 @@ func (r *udpEntryPointRuntime) start() {
73 }74 }
74 entryPoint := r.entryPointSnapshot()75 entryPoint := r.entryPointSnapshot()
75 trafficInfo(r.ctx).Str("entrypoint", entryPoint.Name).Str("address", entryPoint.Address).Str("protocol", string(entryPoint.Protocol)).Msg("started udp traffic entrypoint")76 trafficInfo(r.ctx).Str("entrypoint", entryPoint.Name).Str("address", entryPoint.Address).Str("protocol", string(entryPoint.Protocol)).Msg("started udp traffic entrypoint")
76- for i := 0; i < udpWorkerCount; i++ {77+ for range udpWorkerCount {
77 go r.datagramWorker()78 go r.datagramWorker()
78 }79 }
79 go r.readLoop()80 go r.readLoop()
@@ -133,24 +134,34 @@ func (r *udpEntryPointRuntime) handleDatagram(clientAddr net.Addr, packet []byte
133 }134 }
134 135
135 func (r *udpEntryPointRuntime) session(key string, clientAddr net.Addr, options domain.UDPOptions) (*udpSession, bool) {136 func (r *udpEntryPointRuntime) session(key string, clientAddr net.Addr, options domain.UDPOptions) (*udpSession, bool) {
137+ backend, ok := r.resolveUDPBackend()
138+ if !ok {
139+ r.counters.totalRefused.Add(1)
140+ return nil, false
141+ }
142+
136 r.mu.Lock()143 r.mu.Lock()
137 if session := r.sessions[key]; session != nil {144 if session := r.sessions[key]; session != nil {
138- session.touch()145+ if trafficBackendEqual(session.backendRef, backend) {
146+ session.touch()
147+ r.mu.Unlock()
148+ return session, true
149+ }
150+ delete(r.sessions, key)
151+ r.mu.Unlock()
152+ session.close()
153+ r.counters.activeUDPSessions.Add(-1)
154+ } else {
139 r.mu.Unlock()155 r.mu.Unlock()
140- return session, true
141 }156 }
157+
158+ r.mu.Lock()
142 if options.MaxSessions > 0 && len(r.sessions) >= options.MaxSessions {159 if options.MaxSessions > 0 && len(r.sessions) >= options.MaxSessions {
143 r.mu.Unlock()160 r.mu.Unlock()
144 r.counters.totalRefused.Add(1)161 r.counters.totalRefused.Add(1)
145 return nil, false162 return nil, false
146 }163 }
147 r.mu.Unlock()164 r.mu.Unlock()
148-
149- backend, ok := r.resolveUDPBackend()
150- if !ok {
151- r.counters.totalRefused.Add(1)
152- return nil, false
153- }
154 dialCtx, cancel := context.WithTimeout(r.ctx, udpDialTimeout(options))165 dialCtx, cancel := context.WithTimeout(r.ctx, udpDialTimeout(options))
155 backendConn, err := (&net.Dialer{}).DialContext(dialCtx, "udp", net.JoinHostPort(backend.Host, strconv.Itoa(backend.Port)))166 backendConn, err := (&net.Dialer{}).DialContext(dialCtx, "udp", net.JoinHostPort(backend.Host, strconv.Itoa(backend.Port)))
156 cancel()167 cancel()
@@ -158,15 +169,29 @@ func (r *udpEntryPointRuntime) session(key string, clientAddr net.Addr, options
158 r.counters.totalErrors.Add(1)169 r.counters.totalErrors.Add(1)
159 return nil, false170 return nil, false
160 }171 }
161- session := &udpSession{clientAddr: clientAddr, backend: backendConn, done: make(chan struct{})}172+ session := &udpSession{clientAddr: clientAddr, backend: backendConn, backendRef: backend, done: make(chan struct{})}
162 session.touch()173 session.touch()
163 174
164 r.mu.Lock()175 r.mu.Lock()
165 if existing := r.sessions[key]; existing != nil {176 if existing := r.sessions[key]; existing != nil {
177+ if trafficBackendEqual(existing.backendRef, backend) {
178+ r.mu.Unlock()
179+ _ = backendConn.Close()
180+ return existing, true
181+ }
182+ delete(r.sessions, key)
183+ r.mu.Unlock()
184+ existing.close()
185+ r.counters.activeUDPSessions.Add(-1)
186+ } else {
166 r.mu.Unlock()187 r.mu.Unlock()
188+ }
189+ if options.MaxSessions > 0 && r.sessionCount() >= options.MaxSessions {
167 _ = backendConn.Close()190 _ = backendConn.Close()
168- return existing, true191+ r.counters.totalRefused.Add(1)
192+ return nil, false
169 }193 }
194+ r.mu.Lock()
170 if options.MaxSessions > 0 && len(r.sessions) >= options.MaxSessions {195 if options.MaxSessions > 0 && len(r.sessions) >= options.MaxSessions {
171 r.mu.Unlock()196 r.mu.Unlock()
172 _ = backendConn.Close()197 _ = backendConn.Close()
@@ -182,6 +207,16 @@ func (r *udpEntryPointRuntime) session(key string, clientAddr net.Addr, options
182 return session, true207 return session, true
183 }208 }
184 209
210+func (r *udpEntryPointRuntime) sessionCount() int {
211+ r.mu.Lock()
212+ defer r.mu.Unlock()
213+ return len(r.sessions)
214+}
215+
216+func trafficBackendEqual(left domain.TrafficBackend, right domain.TrafficBackend) bool {
217+ return left.Name == right.Name && left.Host == right.Host && left.Port == right.Port && left.Protocol == right.Protocol
218+}
219+
185 func udpDialTimeout(options domain.UDPOptions) time.Duration {220 func udpDialTimeout(options domain.UDPOptions) time.Duration {
186 if options.IdleTimeout > 0 && options.IdleTimeout < 5*time.Second {221 if options.IdleTimeout > 0 && options.IdleTimeout < 5*time.Second {
187 return options.IdleTimeout222 return options.IdleTimeout
@@ -387,7 +422,16 @@ func (r *udpEntryPointRuntime) updateEntryPoint(entryPoint domain.EntryPoint, tr
387 r.mu.Lock()422 r.mu.Lock()
388 r.entryPoint = entryPoint423 r.entryPoint = entryPoint
389 r.trusted = trusted424 r.trusted = trusted
425+ stale := make([]string, 0)
426+ for key, session := range r.sessions {
427+ if !trustedRemoteAddr(trusted, session.clientAddr) {
428+ stale = append(stale, key)
429+ }
430+ }
390 r.mu.Unlock()431 r.mu.Unlock()
432+ for _, key := range stale {
433+ r.removeSession(key)
434+ }
391 }435 }
392 436
393 func (r *udpEntryPointRuntime) entryPointSnapshot() domain.EntryPoint {437 func (r *udpEntryPointRuntime) entryPointSnapshot() domain.EntryPoint {
M internal/adapters/in/traffic/udp_test.go +60 -2
@@ -53,6 +53,54 @@ func TestUDPReloadAppliesTrustedCIDRChange(t *testing.T) {
5353 assertUDPRejected(t, manager, graph.EntryPoints[0].Address, 1)
5454 }
5555
56+func TestUDPReloadClosesActiveUntrustedSessions(t *testing.T) {
57+ backend := startUDPEchoServer(t)
58+ graph := udpGraph(t, freeUDPAddress(t), backend.address)
59+ graph.EntryPoints[0].TrustedCIDRs = []string{"127.0.0.0/8"}
60+ manager := NewManager()
61+ require.NoError(t, manager.Apply(context.Background(), &graph))
62+ defer shutdownManager(t, manager)
63+
64+ conn := dialUDP(t, graph.EntryPoints[0].Address)
65+ defer conn.Close()
66+ assertUDPRoundTrip(t, conn, "before")
67+ require.Eventually(t, func() bool { return manager.Status().Counters.ActiveUDPSessions == 1 }, time.Second, 10*time.Millisecond)
68+
69+ updated := graph
70+ updated.EntryPoints[0].TrustedCIDRs = []string{"192.0.2.0/24"}
71+ require.NoError(t, manager.Apply(context.Background(), &updated))
72+
73+ assert.Eventually(t, func() bool { return manager.Status().Counters.ActiveUDPSessions == 0 }, time.Second, 10*time.Millisecond)
74+ _, err := conn.Write([]byte("after"))
75+ require.NoError(t, err)
76+ _ = conn.SetReadDeadline(time.Now().Add(80 * time.Millisecond))
77+ buf := make([]byte, 32)
78+ _, err = conn.Read(buf)
79+ require.Error(t, err)
80+}
81+
82+func TestUDPReloadReplacesSessionWhenBackendChanges(t *testing.T) {
83+ backendA := startUDPEchoServerWithPrefix(t, "a:")
84+ backendB := startUDPEchoServerWithPrefix(t, "b:")
85+ graph := udpGraph(t, freeUDPAddress(t), backendA.address)
86+ manager := NewManager()
87+ require.NoError(t, manager.Apply(context.Background(), &graph))
88+ defer shutdownManager(t, manager)
89+
90+ conn := dialUDP(t, graph.EntryPoints[0].Address)
91+ defer conn.Close()
92+ assertUDPRoundTripWant(t, conn, "first", "a:first")
93+
94+ updated := graph
95+ backend, err := backendFromAddress("echo:udp", backendB.address)
96+ require.NoError(t, err)
97+ backend.Protocol = domain.NetworkProtocolUDP
98+ updated.Services[0].Backends = []domain.TrafficBackend{backend}
99+ require.NoError(t, manager.Apply(context.Background(), &updated))
100+
101+ assertUDPRoundTripWant(t, conn, "second", "b:second")
102+}
103+
56104 func assertUDPRejected(t *testing.T, manager *Manager, address string, refused int64) {
57105 t.Helper()
58106 conn := dialUDP(t, address)
@@ -237,6 +285,11 @@ func dialUDP(t *testing.T, address string) *net.UDPConn {
237285 }
238286
239287 func assertUDPRoundTrip(t *testing.T, conn *net.UDPConn, message string) {
288+ t.Helper()
289+ assertUDPRoundTripWant(t, conn, message, message)
290+}
291+
292+func assertUDPRoundTripWant(t *testing.T, conn *net.UDPConn, message string, want string) {
240293 t.Helper()
241294 _, err := conn.Write([]byte(message))
242295 require.NoError(t, err)
@@ -244,12 +297,17 @@ func assertUDPRoundTrip(t *testing.T, conn *net.UDPConn, message string) {
244297 buf := make([]byte, 128)
245298 n, err := conn.Read(buf)
246299 require.NoError(t, err)
247- assert.Equal(t, message, string(buf[:n]))
300+ assert.Equal(t, want, string(buf[:n]))
248301 }
249302
250303 type udpEchoServer struct{ address string }
251304
252305 func startUDPEchoServer(t *testing.T) udpEchoServer {
306+ t.Helper()
307+ return startUDPEchoServerWithPrefix(t, "")
308+}
309+
310+func startUDPEchoServerWithPrefix(t *testing.T, prefix string) udpEchoServer {
253311 t.Helper()
254312 conn, err := net.ListenPacket("udp", "127.0.0.1:0")
255313 require.NoError(t, err)
@@ -262,7 +320,7 @@ func startUDPEchoServer(t *testing.T) udpEchoServer {
262320 if err != nil {
263321 return
264322 }
265- _, _ = conn.WriteTo(buf[:n], addr)
323+ _, _ = conn.WriteTo(append([]byte(prefix), buf[:n]...), addr)
266324 }
267325 }()
268326 t.Cleanup(func() {
@@ -53,6 +53,54 @@ func TestUDPReloadAppliesTrustedCIDRChange(t *testing.T) {
53 assertUDPRejected(t, manager, graph.EntryPoints[0].Address, 1)53 assertUDPRejected(t, manager, graph.EntryPoints[0].Address, 1)
54 }54 }
55 55
56+func TestUDPReloadClosesActiveUntrustedSessions(t *testing.T) {
57+ backend := startUDPEchoServer(t)
58+ graph := udpGraph(t, freeUDPAddress(t), backend.address)
59+ graph.EntryPoints[0].TrustedCIDRs = []string{"127.0.0.0/8"}
60+ manager := NewManager()
61+ require.NoError(t, manager.Apply(context.Background(), &graph))
62+ defer shutdownManager(t, manager)
63+
64+ conn := dialUDP(t, graph.EntryPoints[0].Address)
65+ defer conn.Close()
66+ assertUDPRoundTrip(t, conn, "before")
67+ require.Eventually(t, func() bool { return manager.Status().Counters.ActiveUDPSessions == 1 }, time.Second, 10*time.Millisecond)
68+
69+ updated := graph
70+ updated.EntryPoints[0].TrustedCIDRs = []string{"192.0.2.0/24"}
71+ require.NoError(t, manager.Apply(context.Background(), &updated))
72+
73+ assert.Eventually(t, func() bool { return manager.Status().Counters.ActiveUDPSessions == 0 }, time.Second, 10*time.Millisecond)
74+ _, err := conn.Write([]byte("after"))
75+ require.NoError(t, err)
76+ _ = conn.SetReadDeadline(time.Now().Add(80 * time.Millisecond))
77+ buf := make([]byte, 32)
78+ _, err = conn.Read(buf)
79+ require.Error(t, err)
80+}
81+
82+func TestUDPReloadReplacesSessionWhenBackendChanges(t *testing.T) {
83+ backendA := startUDPEchoServerWithPrefix(t, "a:")
84+ backendB := startUDPEchoServerWithPrefix(t, "b:")
85+ graph := udpGraph(t, freeUDPAddress(t), backendA.address)
86+ manager := NewManager()
87+ require.NoError(t, manager.Apply(context.Background(), &graph))
88+ defer shutdownManager(t, manager)
89+
90+ conn := dialUDP(t, graph.EntryPoints[0].Address)
91+ defer conn.Close()
92+ assertUDPRoundTripWant(t, conn, "first", "a:first")
93+
94+ updated := graph
95+ backend, err := backendFromAddress("echo:udp", backendB.address)
96+ require.NoError(t, err)
97+ backend.Protocol = domain.NetworkProtocolUDP
98+ updated.Services[0].Backends = []domain.TrafficBackend{backend}
99+ require.NoError(t, manager.Apply(context.Background(), &updated))
100+
101+ assertUDPRoundTripWant(t, conn, "second", "b:second")
102+}
103+
56 func assertUDPRejected(t *testing.T, manager *Manager, address string, refused int64) {104 func assertUDPRejected(t *testing.T, manager *Manager, address string, refused int64) {
57 t.Helper()105 t.Helper()
58 conn := dialUDP(t, address)106 conn := dialUDP(t, address)
@@ -237,6 +285,11 @@ func dialUDP(t *testing.T, address string) *net.UDPConn {
237 }285 }
238 286
239 func assertUDPRoundTrip(t *testing.T, conn *net.UDPConn, message string) {287 func assertUDPRoundTrip(t *testing.T, conn *net.UDPConn, message string) {
288+ t.Helper()
289+ assertUDPRoundTripWant(t, conn, message, message)
290+}
291+
292+func assertUDPRoundTripWant(t *testing.T, conn *net.UDPConn, message string, want string) {
240 t.Helper()293 t.Helper()
241 _, err := conn.Write([]byte(message))294 _, err := conn.Write([]byte(message))
242 require.NoError(t, err)295 require.NoError(t, err)
@@ -244,12 +297,17 @@ func assertUDPRoundTrip(t *testing.T, conn *net.UDPConn, message string) {
244 buf := make([]byte, 128)297 buf := make([]byte, 128)
245 n, err := conn.Read(buf)298 n, err := conn.Read(buf)
246 require.NoError(t, err)299 require.NoError(t, err)
247- assert.Equal(t, message, string(buf[:n]))300+ assert.Equal(t, want, string(buf[:n]))
248 }301 }
249 302
250 type udpEchoServer struct{ address string }303 type udpEchoServer struct{ address string }
251 304
252 func startUDPEchoServer(t *testing.T) udpEchoServer {305 func startUDPEchoServer(t *testing.T) udpEchoServer {
306+ t.Helper()
307+ return startUDPEchoServerWithPrefix(t, "")
308+}
309+
310+func startUDPEchoServerWithPrefix(t *testing.T, prefix string) udpEchoServer {
253 t.Helper()311 t.Helper()
254 conn, err := net.ListenPacket("udp", "127.0.0.1:0")312 conn, err := net.ListenPacket("udp", "127.0.0.1:0")
255 require.NoError(t, err)313 require.NoError(t, err)
@@ -262,7 +320,7 @@ func startUDPEchoServer(t *testing.T) udpEchoServer {
262 if err != nil {320 if err != nil {
263 return321 return
264 }322 }
265- _, _ = conn.WriteTo(buf[:n], addr)323+ _, _ = conn.WriteTo(append([]byte(prefix), buf[:n]...), addr)
266 }324 }
267 }()325 }()
268 t.Cleanup(func() {326 t.Cleanup(func() {
M internal/app/run.go +32 -67
@@ -717,9 +717,6 @@ func (si *serviceInit) initRuntimeProxyAndTraffic() error {
717717 return err
718718 }
719719 si.svc.trafficManager = trafficadapter.NewManager()
720- if err := applyTrafficRuntimeConfig(si.ctx, si.svc.trafficManager, si.cfg, si.svc.configSvc); err != nil {
721- return si.log.WrapErr(err, "failed to apply traffic configuration")
722- }
723720 return nil
724721 }
725722
@@ -2823,33 +2820,23 @@ func runServers(ctx context.Context, v *viper.Viper, cfg Config, svc *services,
28232820 }
28242821 }
28252822
2826- proxySrv, proxyReady, tlsSrv, tlsReady, err := startProxyServers(cfg, httpProxyHandler, httpsProxyHandler, svc.pkiSvc, svc.publicTLSSvc, svc.trafficManager, errChan, log)
2823+ proxySrv, proxyReady, tlsSrv, _, err := startProxyServers(cfg, httpProxyHandler, httpsProxyHandler, svc.pkiSvc, svc.publicTLSSvc, svc.trafficManager, errChan, log)
28272824 if err != nil {
28282825 closeStarted(registrySrv)
28292826 return err
28302827 }
28312828
2832- // Wait for all servers to bind their ports before auto-starting containers.
2833- // This prevents the race where auto-start pulls from the registry before it's listening.
2834- if err := waitForServerReady(registryReady, errChan); err != nil {
2829+ // Wait for the registry and HTTP proxy to bind before applying the traffic graph.
2830+ // This prevents auto-start races while keeping the TLS mux under one owner.
2831+ if err := waitForCoreProxyReadyAndApplyTraffic(ctx, cfg, svc, registryReady, proxyReady, errChan); err != nil {
28352832 closeStarted(registrySrv, proxySrv, tlsSrv)
28362833 return err
28372834 }
2838- if err := waitForServerReady(proxyReady, errChan); err != nil {
2839- closeStarted(registrySrv, proxySrv, tlsSrv)
2840- return err
2841- }
2842- if tlsReady != nil {
2843- if err := waitForServerReady(tlsReady, errChan); err != nil {
2844- closeStarted(registrySrv, proxySrv, tlsSrv)
2845- return err
2846- }
2847- }
28482835
28492836 logEvent := log.Info().
28502837 Int("proxy_port", cfg.Server.Port).
28512838 Int("registry_port", cfg.Server.RegistryPort)
2852- if tlsSrv != nil {
2839+ if cfg.Server.TLSPort != 0 {
28532840 logEvent = logEvent.Int("tls_port", cfg.Server.TLSPort)
28542841 }
28552842 logEvent.Msg("Gordon is running")
@@ -2858,6 +2845,8 @@ func runServers(ctx context.Context, v *viper.Viper, cfg Config, svc *services,
28582845
28592846 schedulerCleanup, err := startOptionalSchedulers(ctx, cfg, svc, log, v)
28602847 if err != nil {
2848+ shutdownTrafficManagerForStartupCleanup(svc.trafficManager, log)
2849+ closeStarted(registrySrv, proxySrv, tlsSrv)
28612850 return err
28622851 }
28632852 if schedulerCleanup != nil {
@@ -2879,6 +2868,27 @@ func startPublicTLSRuntimeWithWarning(ctx context.Context, svc publicTLSRuntime,
28792868 }
28802869 }
28812870
2871+func waitForCoreProxyReadyAndApplyTraffic(ctx context.Context, cfg Config, svc *services, registryReady <-chan struct{}, proxyReady <-chan struct{}, errChan <-chan error) error {
2872+ if err := waitForServerReady(registryReady, errChan); err != nil {
2873+ return err
2874+ }
2875+ if err := waitForServerReady(proxyReady, errChan); err != nil {
2876+ return err
2877+ }
2878+ return applyTrafficRuntimeConfig(ctx, svc.trafficManager, cfg, svc.configSvc)
2879+}
2880+
2881+func shutdownTrafficManagerForStartupCleanup(manager *trafficadapter.Manager, log zerowrap.Logger) {
2882+ if manager == nil {
2883+ return
2884+ }
2885+ shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 5*time.Second)
2886+ defer shutdownCancel()
2887+ if err := manager.Shutdown(shutdownCtx); err != nil {
2888+ log.Error().Err(err).Msg("failed to shut down traffic manager during startup cleanup")
2889+ }
2890+}
2891+
28822892 func waitForServerReady(ready <-chan struct{}, errChan <-chan error) error {
28832893 select {
28842894 case <-ready:
@@ -3311,24 +3321,12 @@ func startProxyServers(cfg Config, httpHandler, httpsHandler http.Handler, pkiSv
33113321 GetCertificate: selector.GetCertificate,
33123322 NextProtos: []string{"h2", "http/1.1"},
33133323 }
3314- if trafficManager != nil {
3315- trafficManager.SetTLSFallback(traffic.DefaultTLSEntryPointName, func(ctx context.Context, conn net.Conn) {
3316- if err := trafficadapter.ServeHTTPSFallback(ctx, conn, httpsHandler, tlsConfig.Clone()); err != nil {
3317- log.Warn().Err(err).Msg("https fallback connection failed")
3318- }
3319- })
3324+ if trafficManager == nil {
3325+ return nil, nil, nil, nil, fmt.Errorf("traffic manager is required when server.tls_port is enabled")
33203326 }
3327+ trafficManager.SetTLSHTTPServer(traffic.DefaultTLSEntryPointName, httpsHandler, tlsConfig)
33213328
3322- tlsSrv, tlsReady := startTLSServerWithConfig(
3323- fmt.Sprintf(":%d", cfg.Server.TLSPort),
3324- httpsHandler,
3325- "proxy-tls",
3326- tlsConfig,
3327- errChan,
3328- log,
3329- )
3330-
3331- return httpSrv, httpReady, tlsSrv, tlsReady, nil
3329+ return httpSrv, httpReady, nil, nil, nil
33323330 }
33333331
33343332 // startServer starts an HTTP server, returning the server instance and a channel
@@ -3368,39 +3366,6 @@ func startServer(addr string, handler http.Handler, name string, protocols *http
33683366 return server, ready
33693367 }
33703368
3371-// startTLSServerWithConfig starts an HTTPS server using a pre-built tls.Config
3372-// (e.g. with GetCertificate for on-demand issuance via the internal CA).
3373-// Returns the server instance and a channel that closes once the port is bound.
3374-func startTLSServerWithConfig(addr string, handler http.Handler, name string, tlsCfg *tls.Config, errChan chan<- error, log zerowrap.Logger) (*http.Server, <-chan struct{}) {
3375- srv := &http.Server{
3376- Addr: addr,
3377- Handler: handler,
3378- TLSConfig: tlsCfg,
3379- ReadHeaderTimeout: 10 * time.Second,
3380- ReadTimeout: 5 * time.Minute,
3381- WriteTimeout: 5 * time.Minute,
3382- IdleTimeout: 120 * time.Second,
3383- MaxHeaderBytes: 1 << 20,
3384- }
3385-
3386- ready := make(chan struct{})
3387- go func() {
3388- ln, err := net.Listen("tcp", addr)
3389- if err != nil {
3390- errChan <- fmt.Errorf("%s listen: %w", name, err)
3391- return
3392- }
3393- tlsLn := tls.NewListener(ln, tlsCfg)
3394- log.Info().Str("addr", addr).Str("name", name).Msg("TLS server starting")
3395- close(ready)
3396- if err := srv.Serve(tlsLn); err != nil && err != http.ErrServerClosed {
3397- errChan <- fmt.Errorf("%s: %w", name, err)
3398- }
3399- }()
3400-
3401- return srv, ready
3402-}
3403-
34043369 // SendReloadSignal sends SIGUSR1 to the running Gordon process.
34053370 func SendReloadSignal() error {
34063371 process, _, err := findRunningProcess()
@@ -717,9 +717,6 @@ func (si *serviceInit) initRuntimeProxyAndTraffic() error {
717 return err717 return err
718 }718 }
719 si.svc.trafficManager = trafficadapter.NewManager()719 si.svc.trafficManager = trafficadapter.NewManager()
720- if err := applyTrafficRuntimeConfig(si.ctx, si.svc.trafficManager, si.cfg, si.svc.configSvc); err != nil {
721- return si.log.WrapErr(err, "failed to apply traffic configuration")
722- }
723 return nil720 return nil
724 }721 }
725 722
@@ -2823,33 +2820,23 @@ func runServers(ctx context.Context, v *viper.Viper, cfg Config, svc *services,
2823 }2820 }
2824 }2821 }
2825 2822
2826- proxySrv, proxyReady, tlsSrv, tlsReady, err := startProxyServers(cfg, httpProxyHandler, httpsProxyHandler, svc.pkiSvc, svc.publicTLSSvc, svc.trafficManager, errChan, log)2823+ proxySrv, proxyReady, tlsSrv, _, err := startProxyServers(cfg, httpProxyHandler, httpsProxyHandler, svc.pkiSvc, svc.publicTLSSvc, svc.trafficManager, errChan, log)
2827 if err != nil {2824 if err != nil {
2828 closeStarted(registrySrv)2825 closeStarted(registrySrv)
2829 return err2826 return err
2830 }2827 }
2831 2828
2832- // Wait for all servers to bind their ports before auto-starting containers.2829+ // Wait for the registry and HTTP proxy to bind before applying the traffic graph.
2833- // This prevents the race where auto-start pulls from the registry before it's listening.2830+ // This prevents auto-start races while keeping the TLS mux under one owner.
2834- if err := waitForServerReady(registryReady, errChan); err != nil {2831+ if err := waitForCoreProxyReadyAndApplyTraffic(ctx, cfg, svc, registryReady, proxyReady, errChan); err != nil {
2835 closeStarted(registrySrv, proxySrv, tlsSrv)2832 closeStarted(registrySrv, proxySrv, tlsSrv)
2836 return err2833 return err
2837 }2834 }
2838- if err := waitForServerReady(proxyReady, errChan); err != nil {
2839- closeStarted(registrySrv, proxySrv, tlsSrv)
2840- return err
2841- }
2842- if tlsReady != nil {
2843- if err := waitForServerReady(tlsReady, errChan); err != nil {
2844- closeStarted(registrySrv, proxySrv, tlsSrv)
2845- return err
2846- }
2847- }
2848 2835
2849 logEvent := log.Info().2836 logEvent := log.Info().
2850 Int("proxy_port", cfg.Server.Port).2837 Int("proxy_port", cfg.Server.Port).
2851 Int("registry_port", cfg.Server.RegistryPort)2838 Int("registry_port", cfg.Server.RegistryPort)
2852- if tlsSrv != nil {2839+ if cfg.Server.TLSPort != 0 {
2853 logEvent = logEvent.Int("tls_port", cfg.Server.TLSPort)2840 logEvent = logEvent.Int("tls_port", cfg.Server.TLSPort)
2854 }2841 }
2855 logEvent.Msg("Gordon is running")2842 logEvent.Msg("Gordon is running")
@@ -2858,6 +2845,8 @@ func runServers(ctx context.Context, v *viper.Viper, cfg Config, svc *services,
2858 2845
2859 schedulerCleanup, err := startOptionalSchedulers(ctx, cfg, svc, log, v)2846 schedulerCleanup, err := startOptionalSchedulers(ctx, cfg, svc, log, v)
2860 if err != nil {2847 if err != nil {
2848+ shutdownTrafficManagerForStartupCleanup(svc.trafficManager, log)
2849+ closeStarted(registrySrv, proxySrv, tlsSrv)
2861 return err2850 return err
2862 }2851 }
2863 if schedulerCleanup != nil {2852 if schedulerCleanup != nil {
@@ -2879,6 +2868,27 @@ func startPublicTLSRuntimeWithWarning(ctx context.Context, svc publicTLSRuntime,
2879 }2868 }
2880 }2869 }
2881 2870
2871+func waitForCoreProxyReadyAndApplyTraffic(ctx context.Context, cfg Config, svc *services, registryReady <-chan struct{}, proxyReady <-chan struct{}, errChan <-chan error) error {
2872+ if err := waitForServerReady(registryReady, errChan); err != nil {
2873+ return err
2874+ }
2875+ if err := waitForServerReady(proxyReady, errChan); err != nil {
2876+ return err
2877+ }
2878+ return applyTrafficRuntimeConfig(ctx, svc.trafficManager, cfg, svc.configSvc)
2879+}
2880+
2881+func shutdownTrafficManagerForStartupCleanup(manager *trafficadapter.Manager, log zerowrap.Logger) {
2882+ if manager == nil {
2883+ return
2884+ }
2885+ shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 5*time.Second)
2886+ defer shutdownCancel()
2887+ if err := manager.Shutdown(shutdownCtx); err != nil {
2888+ log.Error().Err(err).Msg("failed to shut down traffic manager during startup cleanup")
2889+ }
2890+}
2891+
2882 func waitForServerReady(ready <-chan struct{}, errChan <-chan error) error {2892 func waitForServerReady(ready <-chan struct{}, errChan <-chan error) error {
2883 select {2893 select {
2884 case <-ready:2894 case <-ready:
@@ -3311,24 +3321,12 @@ func startProxyServers(cfg Config, httpHandler, httpsHandler http.Handler, pkiSv
3311 GetCertificate: selector.GetCertificate,3321 GetCertificate: selector.GetCertificate,
3312 NextProtos: []string{"h2", "http/1.1"},3322 NextProtos: []string{"h2", "http/1.1"},
3313 }3323 }
3314- if trafficManager != nil {3324+ if trafficManager == nil {
3315- trafficManager.SetTLSFallback(traffic.DefaultTLSEntryPointName, func(ctx context.Context, conn net.Conn) {3325+ return nil, nil, nil, nil, fmt.Errorf("traffic manager is required when server.tls_port is enabled")
3316- if err := trafficadapter.ServeHTTPSFallback(ctx, conn, httpsHandler, tlsConfig.Clone()); err != nil {
3317- log.Warn().Err(err).Msg("https fallback connection failed")
3318- }
3319- })
3320 }3326 }
3327+ trafficManager.SetTLSHTTPServer(traffic.DefaultTLSEntryPointName, httpsHandler, tlsConfig)
3321 3328
3322- tlsSrv, tlsReady := startTLSServerWithConfig(3329+ return httpSrv, httpReady, nil, nil, nil
3323- fmt.Sprintf(":%d", cfg.Server.TLSPort),
3324- httpsHandler,
3325- "proxy-tls",
3326- tlsConfig,
3327- errChan,
3328- log,
3329- )
3330-
3331- return httpSrv, httpReady, tlsSrv, tlsReady, nil
3332 }3330 }
3333 3331
3334 // startServer starts an HTTP server, returning the server instance and a channel3332 // startServer starts an HTTP server, returning the server instance and a channel
@@ -3368,39 +3366,6 @@ func startServer(addr string, handler http.Handler, name string, protocols *http
3368 return server, ready3366 return server, ready
3369 }3367 }
3370 3368
3371-// startTLSServerWithConfig starts an HTTPS server using a pre-built tls.Config
3372-// (e.g. with GetCertificate for on-demand issuance via the internal CA).
3373-// Returns the server instance and a channel that closes once the port is bound.
3374-func startTLSServerWithConfig(addr string, handler http.Handler, name string, tlsCfg *tls.Config, errChan chan<- error, log zerowrap.Logger) (*http.Server, <-chan struct{}) {
3375- srv := &http.Server{
3376- Addr: addr,
3377- Handler: handler,
3378- TLSConfig: tlsCfg,
3379- ReadHeaderTimeout: 10 * time.Second,
3380- ReadTimeout: 5 * time.Minute,
3381- WriteTimeout: 5 * time.Minute,
3382- IdleTimeout: 120 * time.Second,
3383- MaxHeaderBytes: 1 << 20,
3384- }
3385-
3386- ready := make(chan struct{})
3387- go func() {
3388- ln, err := net.Listen("tcp", addr)
3389- if err != nil {
3390- errChan <- fmt.Errorf("%s listen: %w", name, err)
3391- return
3392- }
3393- tlsLn := tls.NewListener(ln, tlsCfg)
3394- log.Info().Str("addr", addr).Str("name", name).Msg("TLS server starting")
3395- close(ready)
3396- if err := srv.Serve(tlsLn); err != nil && err != http.ErrServerClosed {
3397- errChan <- fmt.Errorf("%s: %w", name, err)
3398- }
3399- }()
3400-
3401- return srv, ready
3402-}
3403-
3404 // SendReloadSignal sends SIGUSR1 to the running Gordon process.3369 // SendReloadSignal sends SIGUSR1 to the running Gordon process.
3405 func SendReloadSignal() error {3370 func SendReloadSignal() error {
3406 process, _, err := findRunningProcess()3371 process, _, err := findRunningProcess()
A internal/app/start_proxy_servers_test.go +145 -0
@@ -0,0 +1,145 @@
1+package app
2+
3+import (
4+ "context"
5+ "crypto/tls"
6+ "fmt"
7+ "io"
8+ "net"
9+ "net/http"
10+ "strconv"
11+ "testing"
12+ "time"
13+
14+ "github.com/bnema/zerowrap"
15+ "github.com/stretchr/testify/mock"
16+ "github.com/stretchr/testify/require"
17+
18+ trafficadapter "github.com/bnema/gordon/internal/adapters/in/traffic"
19+ pkiadapter "github.com/bnema/gordon/internal/adapters/out/pki"
20+ inmocks "github.com/bnema/gordon/internal/boundaries/in/mocks"
21+ outmocks "github.com/bnema/gordon/internal/boundaries/out/mocks"
22+ "github.com/bnema/gordon/internal/domain"
23+ pkiusecase "github.com/bnema/gordon/internal/usecase/pki"
24+)
25+
26+func TestStartProxyServers_ConfiguresTrafficManagerHTTPSRoute(t *testing.T) {
27+ cfg := Config{}
28+ cfg.Server.Port = freeTCPPort(t)
29+ cfg.Server.TLSPort = freeTCPPort(t)
30+
31+ manager := trafficadapter.NewManager()
32+ defer func() { require.NoError(t, manager.Shutdown(context.Background())) }()
33+ pkiSvc := newTestPKIService(t)
34+ errChan := make(chan error, 4)
35+
36+ httpSrv, httpReady, tlsSrv, tlsReady, err := startProxyServers(
37+ cfg,
38+ http.NotFoundHandler(),
39+ http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write([]byte("secure")) }),
40+ pkiSvc,
41+ nil,
42+ manager,
43+ errChan,
44+ zerowrap.Default(),
45+ )
46+ require.NoError(t, err)
47+ defer shutdownHTTPServer(t, httpSrv)
48+ require.Nil(t, tlsSrv)
49+ require.Nil(t, tlsReady)
50+ waitReadyForTest(t, httpReady, errChan)
51+
52+ configSvc := inmocks.NewMockConfigService(t)
53+ configSvc.EXPECT().GetRoutes(context.Background()).Return([]domain.Route{{Domain: "app.example.com", HTTPS: true}})
54+ configSvc.EXPECT().GetExternalRoutes().Return(map[string]string{})
55+ require.NoError(t, applyTrafficRuntimeConfig(context.Background(), manager, cfg, configSvc))
56+
57+ body := httpsGetForTest(t, cfg.Server.TLSPort, "app.example.com")
58+ require.Contains(t, body, "secure")
59+}
60+
61+func TestStartProxyServers_TLSRequiresTrafficManager(t *testing.T) {
62+ cfg := Config{}
63+ cfg.Server.Port = freeTCPPort(t)
64+ cfg.Server.TLSPort = freeTCPPort(t)
65+
66+ httpSrv, _, tlsSrv, _, err := startProxyServers(
67+ cfg,
68+ http.NotFoundHandler(),
69+ http.NotFoundHandler(),
70+ newTestPKIService(t),
71+ nil,
72+ nil,
73+ make(chan error, 2),
74+ zerowrap.Default(),
75+ )
76+ if httpSrv != nil {
77+ shutdownHTTPServer(t, httpSrv)
78+ }
79+ if tlsSrv != nil {
80+ shutdownHTTPServer(t, tlsSrv)
81+ }
82+ require.ErrorContains(t, err, "traffic manager is required")
83+}
84+
85+func newTestPKIService(t *testing.T) *pkiusecase.Service {
86+ t.Helper()
87+ routes := outmocks.NewMockRouteChecker(t)
88+ routes.EXPECT().GetRoutes(mock.Anything).Return([]domain.Route{{Domain: "app.example.com", HTTPS: true}}).Maybe()
89+ routes.EXPECT().GetExternalRoutes().Return(map[string]string{}).Maybe()
90+ ca, err := pkiadapter.NewCA(t.TempDir(), zerowrap.Default())
91+ require.NoError(t, err)
92+ pkiSvc := pkiusecase.NewService(context.Background(), ca, routes, zerowrap.Default())
93+ t.Cleanup(pkiSvc.Stop)
94+ return pkiSvc
95+}
96+
97+func freeTCPPort(t *testing.T) int {
98+ t.Helper()
99+ listener, err := net.Listen("tcp", "127.0.0.1:0")
100+ require.NoError(t, err)
101+ defer listener.Close()
102+ _, portValue, err := net.SplitHostPort(listener.Addr().String())
103+ require.NoError(t, err)
104+ port, err := strconv.Atoi(portValue)
105+ require.NoError(t, err)
106+ return port
107+}
108+
109+func waitReadyForTest(t *testing.T, ready <-chan struct{}, errChan <-chan error) {
110+ t.Helper()
111+ select {
112+ case <-ready:
113+ case err := <-errChan:
114+ require.NoError(t, err)
115+ case <-time.After(2 * time.Second):
116+ t.Fatal("server did not become ready")
117+ }
118+}
119+
120+func shutdownHTTPServer(t *testing.T, srv *http.Server) {
121+ t.Helper()
122+ ctx, cancel := context.WithTimeout(context.Background(), time.Second)
123+ defer cancel()
124+ require.NoError(t, srv.Shutdown(ctx))
125+}
126+
127+func httpsGetForTest(t *testing.T, port int, serverName string) string {
128+ t.Helper()
129+ transport := &http.Transport{
130+ TLSClientConfig: &tls.Config{ServerName: serverName, InsecureSkipVerify: true},
131+ ForceAttemptHTTP2: true,
132+ }
133+ client := &http.Client{Transport: transport, Timeout: 2 * time.Second}
134+ defer transport.CloseIdleConnections()
135+
136+ req, err := http.NewRequest(http.MethodGet, fmt.Sprintf("https://127.0.0.1:%d/", port), nil)
137+ require.NoError(t, err)
138+ req.Host = serverName
139+ resp, err := client.Do(req)
140+ require.NoError(t, err)
141+ defer resp.Body.Close()
142+ body, err := io.ReadAll(resp.Body)
143+ require.NoError(t, err)
144+ return string(body)
145+}
@@ -0,0 +1,145 @@
1+package app
2+
3+import (
4+ "context"
5+ "crypto/tls"
6+ "fmt"
7+ "io"
8+ "net"
9+ "net/http"
10+ "strconv"
11+ "testing"
12+ "time"
13+
14+ "github.com/bnema/zerowrap"
15+ "github.com/stretchr/testify/mock"
16+ "github.com/stretchr/testify/require"
17+
18+ trafficadapter "github.com/bnema/gordon/internal/adapters/in/traffic"
19+ pkiadapter "github.com/bnema/gordon/internal/adapters/out/pki"
20+ inmocks "github.com/bnema/gordon/internal/boundaries/in/mocks"
21+ outmocks "github.com/bnema/gordon/internal/boundaries/out/mocks"
22+ "github.com/bnema/gordon/internal/domain"
23+ pkiusecase "github.com/bnema/gordon/internal/usecase/pki"
24+)
25+
26+func TestStartProxyServers_ConfiguresTrafficManagerHTTPSRoute(t *testing.T) {
27+ cfg := Config{}
28+ cfg.Server.Port = freeTCPPort(t)
29+ cfg.Server.TLSPort = freeTCPPort(t)
30+
31+ manager := trafficadapter.NewManager()
32+ defer func() { require.NoError(t, manager.Shutdown(context.Background())) }()
33+ pkiSvc := newTestPKIService(t)
34+ errChan := make(chan error, 4)
35+
36+ httpSrv, httpReady, tlsSrv, tlsReady, err := startProxyServers(
37+ cfg,
38+ http.NotFoundHandler(),
39+ http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write([]byte("secure")) }),
40+ pkiSvc,
41+ nil,
42+ manager,
43+ errChan,
44+ zerowrap.Default(),
45+ )
46+ require.NoError(t, err)
47+ defer shutdownHTTPServer(t, httpSrv)
48+ require.Nil(t, tlsSrv)
49+ require.Nil(t, tlsReady)
50+ waitReadyForTest(t, httpReady, errChan)
51+
52+ configSvc := inmocks.NewMockConfigService(t)
53+ configSvc.EXPECT().GetRoutes(context.Background()).Return([]domain.Route{{Domain: "app.example.com", HTTPS: true}})
54+ configSvc.EXPECT().GetExternalRoutes().Return(map[string]string{})
55+ require.NoError(t, applyTrafficRuntimeConfig(context.Background(), manager, cfg, configSvc))
56+
57+ body := httpsGetForTest(t, cfg.Server.TLSPort, "app.example.com")
58+ require.Contains(t, body, "secure")
59+}
60+
61+func TestStartProxyServers_TLSRequiresTrafficManager(t *testing.T) {
62+ cfg := Config{}
63+ cfg.Server.Port = freeTCPPort(t)
64+ cfg.Server.TLSPort = freeTCPPort(t)
65+
66+ httpSrv, _, tlsSrv, _, err := startProxyServers(
67+ cfg,
68+ http.NotFoundHandler(),
69+ http.NotFoundHandler(),
70+ newTestPKIService(t),
71+ nil,
72+ nil,
73+ make(chan error, 2),
74+ zerowrap.Default(),
75+ )
76+ if httpSrv != nil {
77+ shutdownHTTPServer(t, httpSrv)
78+ }
79+ if tlsSrv != nil {
80+ shutdownHTTPServer(t, tlsSrv)
81+ }
82+ require.ErrorContains(t, err, "traffic manager is required")
83+}
84+
85+func newTestPKIService(t *testing.T) *pkiusecase.Service {
86+ t.Helper()
87+ routes := outmocks.NewMockRouteChecker(t)
88+ routes.EXPECT().GetRoutes(mock.Anything).Return([]domain.Route{{Domain: "app.example.com", HTTPS: true}}).Maybe()
89+ routes.EXPECT().GetExternalRoutes().Return(map[string]string{}).Maybe()
90+ ca, err := pkiadapter.NewCA(t.TempDir(), zerowrap.Default())
91+ require.NoError(t, err)
92+ pkiSvc := pkiusecase.NewService(context.Background(), ca, routes, zerowrap.Default())
93+ t.Cleanup(pkiSvc.Stop)
94+ return pkiSvc
95+}
96+
97+func freeTCPPort(t *testing.T) int {
98+ t.Helper()
99+ listener, err := net.Listen("tcp", "127.0.0.1:0")
100+ require.NoError(t, err)
101+ defer listener.Close()
102+ _, portValue, err := net.SplitHostPort(listener.Addr().String())
103+ require.NoError(t, err)
104+ port, err := strconv.Atoi(portValue)
105+ require.NoError(t, err)
106+ return port
107+}
108+
109+func waitReadyForTest(t *testing.T, ready <-chan struct{}, errChan <-chan error) {
110+ t.Helper()
111+ select {
112+ case <-ready:
113+ case err := <-errChan:
114+ require.NoError(t, err)
115+ case <-time.After(2 * time.Second):
116+ t.Fatal("server did not become ready")
117+ }
118+}
119+
120+func shutdownHTTPServer(t *testing.T, srv *http.Server) {
121+ t.Helper()
122+ ctx, cancel := context.WithTimeout(context.Background(), time.Second)
123+ defer cancel()
124+ require.NoError(t, srv.Shutdown(ctx))
125+}
126+
127+func httpsGetForTest(t *testing.T, port int, serverName string) string {
128+ t.Helper()
129+ transport := &http.Transport{
130+ TLSClientConfig: &tls.Config{ServerName: serverName, InsecureSkipVerify: true},
131+ ForceAttemptHTTP2: true,
132+ }
133+ client := &http.Client{Transport: transport, Timeout: 2 * time.Second}
134+ defer transport.CloseIdleConnections()
135+
136+ req, err := http.NewRequest(http.MethodGet, fmt.Sprintf("https://127.0.0.1:%d/", port), nil)
137+ require.NoError(t, err)
138+ req.Host = serverName
139+ resp, err := client.Do(req)
140+ require.NoError(t, err)
141+ defer resp.Body.Close()
142+ body, err := io.ReadAll(resp.Body)
143+ require.NoError(t, err)
144+ return string(body)
145+}
M internal/app/traffic.go +1 -3
@@ -78,10 +78,8 @@ func trafficManagerRouterProtocol(protocol domain.RouterProtocol) bool {
7878
7979 func trafficManagerOwnsEntryPoint(entryPoint domain.EntryPoint) bool {
8080 switch entryPoint.Protocol {
81- case domain.EntryPointProtocolTCP, domain.EntryPointProtocolUDP:
81+ case domain.EntryPointProtocolTLSMux, domain.EntryPointProtocolTCP, domain.EntryPointProtocolUDP:
8282 return true
83- case domain.EntryPointProtocolTLSMux:
84- return entryPoint.Name != trafficbuilder.DefaultTLSEntryPointName
8583 default:
8684 return false
8785 }
@@ -78,10 +78,8 @@ func trafficManagerRouterProtocol(protocol domain.RouterProtocol) bool {
78 78
79 func trafficManagerOwnsEntryPoint(entryPoint domain.EntryPoint) bool {79 func trafficManagerOwnsEntryPoint(entryPoint domain.EntryPoint) bool {
80 switch entryPoint.Protocol {80 switch entryPoint.Protocol {
81- case domain.EntryPointProtocolTCP, domain.EntryPointProtocolUDP:81+ case domain.EntryPointProtocolTLSMux, domain.EntryPointProtocolTCP, domain.EntryPointProtocolUDP:
82 return true82 return true
83- case domain.EntryPointProtocolTLSMux:
84- return entryPoint.Name != trafficbuilder.DefaultTLSEntryPointName
85 default:83 default:
86 return false84 return false
87 }85 }
M internal/app/traffic_test.go +36 -10
@@ -14,7 +14,7 @@ import (
1414 "github.com/bnema/gordon/internal/usecase/traffic"
1515 )
1616
17-func TestTrafficRuntimeGraphFiltersLegacyHTTPEntrypoints(t *testing.T) {
17+func TestTrafficRuntimeGraphKeepsDefaultTLSEntrypoint(t *testing.T) {
1818 graph := domain.TrafficGraph{
1919 EntryPoints: []domain.EntryPoint{
2020 {Name: traffic.DefaultHTTPEntryPointName, Address: "127.0.0.1:8080", Protocol: domain.EntryPointProtocolHTTP},
@@ -23,6 +23,7 @@ func TestTrafficRuntimeGraphFiltersLegacyHTTPEntrypoints(t *testing.T) {
2323 },
2424 Routers: []domain.TrafficRouter{
2525 {Name: "app", EntryPoint: traffic.DefaultHTTPEntryPointName, Protocol: domain.RouterProtocolHTTP, Service: "route:app.example.com"},
26+ {Name: "app-secure", EntryPoint: traffic.DefaultTLSEntryPointName, Protocol: domain.RouterProtocolHTTP, Rule: domain.TrafficRule{Host: "app.example.com"}, Service: "route:app.example.com"},
2627 {Name: "postgres", EntryPoint: "postgres", Protocol: domain.RouterProtocolTCP, Service: "network_service:postgres:db"},
2728 },
2829 Services: []domain.TrafficService{
@@ -33,23 +34,24 @@ func TestTrafficRuntimeGraphFiltersLegacyHTTPEntrypoints(t *testing.T) {
3334
3435 filtered, err := trafficRuntimeGraph(graph)
3536 require.NoError(t, err)
36- require.Len(t, filtered.EntryPoints, 1)
37- assert.Equal(t, "postgres", filtered.EntryPoints[0].Name)
38- require.Len(t, filtered.Routers, 1)
39- assert.Equal(t, "postgres", filtered.Routers[0].Name)
40- require.Len(t, filtered.Services, 1)
41- assert.Equal(t, "network_service:postgres:db", filtered.Services[0].Name)
37+ assert.ElementsMatch(t, []string{traffic.DefaultTLSEntryPointName, "postgres"}, entryPointNames(filtered.EntryPoints))
38+ assert.ElementsMatch(t, []string{"app-secure", "postgres"}, routerNames(filtered.Routers))
39+ assert.ElementsMatch(t, []string{"route:app.example.com", "network_service:postgres:db"}, serviceNames(filtered.Services))
4240 }
4341
44-func TestTrafficRuntimeGraphRejectsL4RouterOnLegacyEntrypoint(t *testing.T) {
42+func TestTrafficRuntimeGraphAllowsTLSPassthroughOnDefaultTLS(t *testing.T) {
4543 graph := domain.TrafficGraph{
4644 EntryPoints: []domain.EntryPoint{{Name: traffic.DefaultTLSEntryPointName, Address: "127.0.0.1:8443", Protocol: domain.EntryPointProtocolTLSMux}},
4745 Routers: []domain.TrafficRouter{{Name: "raw", EntryPoint: traffic.DefaultTLSEntryPointName, Protocol: domain.RouterProtocolTLSPassthrough, Rule: domain.TrafficRule{SNI: "raw.example.com"}, Service: "network_service:raw:tls"}},
4846 Services: []domain.TrafficService{{Name: "network_service:raw:tls", Backends: []domain.TrafficBackend{{Name: "raw:tls", Host: "127.0.0.1", Port: 443, Protocol: domain.NetworkProtocolTCP}}}},
4947 }
5048
51- _, err := trafficRuntimeGraph(graph)
52- require.ErrorContains(t, err, "not owned by the traffic manager")
49+ filtered, err := trafficRuntimeGraph(graph)
50+ require.NoError(t, err)
51+ require.Len(t, filtered.EntryPoints, 1)
52+ assert.Equal(t, traffic.DefaultTLSEntryPointName, filtered.EntryPoints[0].Name)
53+ require.Len(t, filtered.Routers, 1)
54+ assert.Equal(t, "raw", filtered.Routers[0].Name)
5355 }
5456
5557 func TestApplyTrafficRuntimeConfigAppliesCustomL4Entrypoint(t *testing.T) {
@@ -77,6 +79,30 @@ func TestApplyTrafficRuntimeConfigAppliesCustomL4Entrypoint(t *testing.T) {
7779 assert.True(t, status.EntryPoints[0].Active)
7880 }
7981
82+func entryPointNames(entries []domain.EntryPoint) []string {
83+ names := make([]string, 0, len(entries))
84+ for _, entry := range entries {
85+ names = append(names, entry.Name)
86+ }
87+ return names
88+}
89+
90+func routerNames(routers []domain.TrafficRouter) []string {
91+ names := make([]string, 0, len(routers))
92+ for _, router := range routers {
93+ names = append(names, router.Name)
94+ }
95+ return names
96+}
97+
98+func serviceNames(services []domain.TrafficService) []string {
99+ names := make([]string, 0, len(services))
100+ for _, service := range services {
101+ names = append(names, service.Name)
102+ }
103+ return names
104+}
105+
80106 func freeTCPAddress(t *testing.T) string {
81107 t.Helper()
82108 listener, err := net.Listen("tcp", "127.0.0.1:0")
@@ -14,7 +14,7 @@ import (
14 "github.com/bnema/gordon/internal/usecase/traffic"14 "github.com/bnema/gordon/internal/usecase/traffic"
15 )15 )
16 16
17-func TestTrafficRuntimeGraphFiltersLegacyHTTPEntrypoints(t *testing.T) {17+func TestTrafficRuntimeGraphKeepsDefaultTLSEntrypoint(t *testing.T) {
18 graph := domain.TrafficGraph{18 graph := domain.TrafficGraph{
19 EntryPoints: []domain.EntryPoint{19 EntryPoints: []domain.EntryPoint{
20 {Name: traffic.DefaultHTTPEntryPointName, Address: "127.0.0.1:8080", Protocol: domain.EntryPointProtocolHTTP},20 {Name: traffic.DefaultHTTPEntryPointName, Address: "127.0.0.1:8080", Protocol: domain.EntryPointProtocolHTTP},
@@ -23,6 +23,7 @@ func TestTrafficRuntimeGraphFiltersLegacyHTTPEntrypoints(t *testing.T) {
23 },23 },
24 Routers: []domain.TrafficRouter{24 Routers: []domain.TrafficRouter{
25 {Name: "app", EntryPoint: traffic.DefaultHTTPEntryPointName, Protocol: domain.RouterProtocolHTTP, Service: "route:app.example.com"},25 {Name: "app", EntryPoint: traffic.DefaultHTTPEntryPointName, Protocol: domain.RouterProtocolHTTP, Service: "route:app.example.com"},
26+ {Name: "app-secure", EntryPoint: traffic.DefaultTLSEntryPointName, Protocol: domain.RouterProtocolHTTP, Rule: domain.TrafficRule{Host: "app.example.com"}, Service: "route:app.example.com"},
26 {Name: "postgres", EntryPoint: "postgres", Protocol: domain.RouterProtocolTCP, Service: "network_service:postgres:db"},27 {Name: "postgres", EntryPoint: "postgres", Protocol: domain.RouterProtocolTCP, Service: "network_service:postgres:db"},
27 },28 },
28 Services: []domain.TrafficService{29 Services: []domain.TrafficService{
@@ -33,23 +34,24 @@ func TestTrafficRuntimeGraphFiltersLegacyHTTPEntrypoints(t *testing.T) {
33 34
34 filtered, err := trafficRuntimeGraph(graph)35 filtered, err := trafficRuntimeGraph(graph)
35 require.NoError(t, err)36 require.NoError(t, err)
36- require.Len(t, filtered.EntryPoints, 1)37+ assert.ElementsMatch(t, []string{traffic.DefaultTLSEntryPointName, "postgres"}, entryPointNames(filtered.EntryPoints))
37- assert.Equal(t, "postgres", filtered.EntryPoints[0].Name)38+ assert.ElementsMatch(t, []string{"app-secure", "postgres"}, routerNames(filtered.Routers))
38- require.Len(t, filtered.Routers, 1)39+ assert.ElementsMatch(t, []string{"route:app.example.com", "network_service:postgres:db"}, serviceNames(filtered.Services))
39- assert.Equal(t, "postgres", filtered.Routers[0].Name)
40- require.Len(t, filtered.Services, 1)
41- assert.Equal(t, "network_service:postgres:db", filtered.Services[0].Name)
42 }40 }
43 41
44-func TestTrafficRuntimeGraphRejectsL4RouterOnLegacyEntrypoint(t *testing.T) {42+func TestTrafficRuntimeGraphAllowsTLSPassthroughOnDefaultTLS(t *testing.T) {
45 graph := domain.TrafficGraph{43 graph := domain.TrafficGraph{
46 EntryPoints: []domain.EntryPoint{{Name: traffic.DefaultTLSEntryPointName, Address: "127.0.0.1:8443", Protocol: domain.EntryPointProtocolTLSMux}},44 EntryPoints: []domain.EntryPoint{{Name: traffic.DefaultTLSEntryPointName, Address: "127.0.0.1:8443", Protocol: domain.EntryPointProtocolTLSMux}},
47 Routers: []domain.TrafficRouter{{Name: "raw", EntryPoint: traffic.DefaultTLSEntryPointName, Protocol: domain.RouterProtocolTLSPassthrough, Rule: domain.TrafficRule{SNI: "raw.example.com"}, Service: "network_service:raw:tls"}},45 Routers: []domain.TrafficRouter{{Name: "raw", EntryPoint: traffic.DefaultTLSEntryPointName, Protocol: domain.RouterProtocolTLSPassthrough, Rule: domain.TrafficRule{SNI: "raw.example.com"}, Service: "network_service:raw:tls"}},
48 Services: []domain.TrafficService{{Name: "network_service:raw:tls", Backends: []domain.TrafficBackend{{Name: "raw:tls", Host: "127.0.0.1", Port: 443, Protocol: domain.NetworkProtocolTCP}}}},46 Services: []domain.TrafficService{{Name: "network_service:raw:tls", Backends: []domain.TrafficBackend{{Name: "raw:tls", Host: "127.0.0.1", Port: 443, Protocol: domain.NetworkProtocolTCP}}}},
49 }47 }
50 48
51- _, err := trafficRuntimeGraph(graph)49+ filtered, err := trafficRuntimeGraph(graph)
52- require.ErrorContains(t, err, "not owned by the traffic manager")50+ require.NoError(t, err)
51+ require.Len(t, filtered.EntryPoints, 1)
52+ assert.Equal(t, traffic.DefaultTLSEntryPointName, filtered.EntryPoints[0].Name)
53+ require.Len(t, filtered.Routers, 1)
54+ assert.Equal(t, "raw", filtered.Routers[0].Name)
53 }55 }
54 56
55 func TestApplyTrafficRuntimeConfigAppliesCustomL4Entrypoint(t *testing.T) {57 func TestApplyTrafficRuntimeConfigAppliesCustomL4Entrypoint(t *testing.T) {
@@ -77,6 +79,30 @@ func TestApplyTrafficRuntimeConfigAppliesCustomL4Entrypoint(t *testing.T) {
77 assert.True(t, status.EntryPoints[0].Active)79 assert.True(t, status.EntryPoints[0].Active)
78 }80 }
79 81
82+func entryPointNames(entries []domain.EntryPoint) []string {
83+ names := make([]string, 0, len(entries))
84+ for _, entry := range entries {
85+ names = append(names, entry.Name)
86+ }
87+ return names
88+}
89+
90+func routerNames(routers []domain.TrafficRouter) []string {
91+ names := make([]string, 0, len(routers))
92+ for _, router := range routers {
93+ names = append(names, router.Name)
94+ }
95+ return names
96+}
97+
98+func serviceNames(services []domain.TrafficService) []string {
99+ names := make([]string, 0, len(services))
100+ for _, service := range services {
101+ names = append(names, service.Name)
102+ }
103+ return names
104+}
105+
80 func freeTCPAddress(t *testing.T) string {106 func freeTCPAddress(t *testing.T) string {
81 t.Helper()107 t.Helper()
82 listener, err := net.Listen("tcp", "127.0.0.1:0")108 listener, err := net.Listen("tcp", "127.0.0.1:0")
M internal/boundaries/in/mocks/mock_traffic_status_service.go +2 -1
@@ -5,8 +5,9 @@
55 package mocks
66
77 import (
8- "github.com/bnema/gordon/internal/domain"
98 mock "github.com/stretchr/testify/mock"
9+
10+ "github.com/bnema/gordon/internal/domain"
1011 )
1112
1213 // NewMockTrafficStatusService creates a new instance of MockTrafficStatusService. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
@@ -5,8 +5,9 @@
5 package mocks5 package mocks
6 6
7 import (7 import (
8- "github.com/bnema/gordon/internal/domain"
9 mock "github.com/stretchr/testify/mock"8 mock "github.com/stretchr/testify/mock"
9+
10+ "github.com/bnema/gordon/internal/domain"
10 )11 )
11 12
12 // NewMockTrafficStatusService creates a new instance of MockTrafficStatusService. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.13 // NewMockTrafficStatusService creates a new instance of MockTrafficStatusService. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
M internal/domain/traffic.go +35 -0
@@ -187,6 +187,9 @@ func ParseTrafficServiceRef(value string) (TrafficServiceRef, error) {
187187 }
188188
189189 func (g TrafficGraph) Validate() error {
190+ if err := validateTrafficOptions(g.Options); err != nil {
191+ return err
192+ }
190193 entryPoints, err := validateTrafficEntryPoints(g.EntryPoints)
191194 if err != nil {
192195 return err
@@ -198,6 +201,38 @@ func (g TrafficGraph) Validate() error {
198201 return g.validateRouters(entryPoints, services)
199202 }
200203
204+func validateTrafficOptions(options TrafficOptions) error {
205+ if err := validateOptionalPositiveDuration(options.TCP.DialTimeout, "traffic.tcp.dial_timeout"); err != nil {
206+ return err
207+ }
208+ if err := validateOptionalPositiveDuration(options.TCP.IdleTimeout, "traffic.tcp.idle_timeout"); err != nil {
209+ return err
210+ }
211+ if err := validateOptionalPositiveDuration(options.TCP.DrainTimeout, "traffic.tcp.drain_timeout"); err != nil {
212+ return err
213+ }
214+ if options.TCP.MaxConnections < 0 {
215+ return fmt.Errorf("traffic.tcp.max_connections must be non-negative")
216+ }
217+ if err := validateOptionalPositiveDuration(options.UDP.IdleTimeout, "traffic.udp.idle_timeout"); err != nil {
218+ return err
219+ }
220+ if err := validateOptionalPositiveDuration(options.UDP.DrainTimeout, "traffic.udp.drain_timeout"); err != nil {
221+ return err
222+ }
223+ if options.UDP.MaxSessions < 0 {
224+ return fmt.Errorf("traffic.udp.max_sessions must be non-negative")
225+ }
226+ return nil
227+}
228+
229+func validateOptionalPositiveDuration(value time.Duration, name string) error {
230+ if value < 0 {
231+ return fmt.Errorf("%s must be positive", name)
232+ }
233+ return nil
234+}
235+
201236 func validateTrafficEntryPoints(entries []EntryPoint) (map[string]EntryPoint, error) {
202237 entryPoints := make(map[string]EntryPoint, len(entries))
203238 listenAddresses := newProtocolListenAddressSet()
@@ -187,6 +187,9 @@ func ParseTrafficServiceRef(value string) (TrafficServiceRef, error) {
187 }187 }
188 188
189 func (g TrafficGraph) Validate() error {189 func (g TrafficGraph) Validate() error {
190+ if err := validateTrafficOptions(g.Options); err != nil {
191+ return err
192+ }
190 entryPoints, err := validateTrafficEntryPoints(g.EntryPoints)193 entryPoints, err := validateTrafficEntryPoints(g.EntryPoints)
191 if err != nil {194 if err != nil {
192 return err195 return err
@@ -198,6 +201,38 @@ func (g TrafficGraph) Validate() error {
198 return g.validateRouters(entryPoints, services)201 return g.validateRouters(entryPoints, services)
199 }202 }
200 203
204+func validateTrafficOptions(options TrafficOptions) error {
205+ if err := validateOptionalPositiveDuration(options.TCP.DialTimeout, "traffic.tcp.dial_timeout"); err != nil {
206+ return err
207+ }
208+ if err := validateOptionalPositiveDuration(options.TCP.IdleTimeout, "traffic.tcp.idle_timeout"); err != nil {
209+ return err
210+ }
211+ if err := validateOptionalPositiveDuration(options.TCP.DrainTimeout, "traffic.tcp.drain_timeout"); err != nil {
212+ return err
213+ }
214+ if options.TCP.MaxConnections < 0 {
215+ return fmt.Errorf("traffic.tcp.max_connections must be non-negative")
216+ }
217+ if err := validateOptionalPositiveDuration(options.UDP.IdleTimeout, "traffic.udp.idle_timeout"); err != nil {
218+ return err
219+ }
220+ if err := validateOptionalPositiveDuration(options.UDP.DrainTimeout, "traffic.udp.drain_timeout"); err != nil {
221+ return err
222+ }
223+ if options.UDP.MaxSessions < 0 {
224+ return fmt.Errorf("traffic.udp.max_sessions must be non-negative")
225+ }
226+ return nil
227+}
228+
229+func validateOptionalPositiveDuration(value time.Duration, name string) error {
230+ if value < 0 {
231+ return fmt.Errorf("%s must be positive", name)
232+ }
233+ return nil
234+}
235+
201 func validateTrafficEntryPoints(entries []EntryPoint) (map[string]EntryPoint, error) {236 func validateTrafficEntryPoints(entries []EntryPoint) (map[string]EntryPoint, error) {
202 entryPoints := make(map[string]EntryPoint, len(entries))237 entryPoints := make(map[string]EntryPoint, len(entries))
203 listenAddresses := newProtocolListenAddressSet()238 listenAddresses := newProtocolListenAddressSet()
M internal/domain/traffic_test.go +24 -0
@@ -507,6 +507,30 @@ func TestParseTrafficServiceRef(t *testing.T) {
507507 }
508508 }
509509
510+func TestTrafficGraphValidateRejectsInvalidOptions(t *testing.T) {
511+ base := TrafficGraph{}
512+ tests := []struct {
513+ name string
514+ mutate func(*TrafficGraph)
515+ wantErr string
516+ }{
517+ {name: "negative tcp dial timeout", mutate: func(g *TrafficGraph) { g.Options.TCP.DialTimeout = -time.Second }, wantErr: "traffic.tcp.dial_timeout must be positive"},
518+ {name: "negative tcp idle timeout", mutate: func(g *TrafficGraph) { g.Options.TCP.IdleTimeout = -time.Second }, wantErr: "traffic.tcp.idle_timeout must be positive"},
519+ {name: "negative tcp drain timeout", mutate: func(g *TrafficGraph) { g.Options.TCP.DrainTimeout = -time.Second }, wantErr: "traffic.tcp.drain_timeout must be positive"},
520+ {name: "negative tcp max connections", mutate: func(g *TrafficGraph) { g.Options.TCP.MaxConnections = -1 }, wantErr: "traffic.tcp.max_connections must be non-negative"},
521+ {name: "negative udp idle timeout", mutate: func(g *TrafficGraph) { g.Options.UDP.IdleTimeout = -time.Second }, wantErr: "traffic.udp.idle_timeout must be positive"},
522+ {name: "negative udp drain timeout", mutate: func(g *TrafficGraph) { g.Options.UDP.DrainTimeout = -time.Second }, wantErr: "traffic.udp.drain_timeout must be positive"},
523+ {name: "negative udp max sessions", mutate: func(g *TrafficGraph) { g.Options.UDP.MaxSessions = -1 }, wantErr: "traffic.udp.max_sessions must be non-negative"},
524+ }
525+ for _, tt := range tests {
526+ t.Run(tt.name, func(t *testing.T) {
527+ graph := base
528+ tt.mutate(&graph)
529+ require.ErrorContains(t, graph.Validate(), tt.wantErr)
530+ })
531+ }
532+}
533+
510534 func TestTrafficDomainTypesMatchSpecFields(t *testing.T) {
511535 graph := TrafficGraph{
512536 Options: TrafficOptions{
@@ -507,6 +507,30 @@ func TestParseTrafficServiceRef(t *testing.T) {
507 }507 }
508 }508 }
509 509
510+func TestTrafficGraphValidateRejectsInvalidOptions(t *testing.T) {
511+ base := TrafficGraph{}
512+ tests := []struct {
513+ name string
514+ mutate func(*TrafficGraph)
515+ wantErr string
516+ }{
517+ {name: "negative tcp dial timeout", mutate: func(g *TrafficGraph) { g.Options.TCP.DialTimeout = -time.Second }, wantErr: "traffic.tcp.dial_timeout must be positive"},
518+ {name: "negative tcp idle timeout", mutate: func(g *TrafficGraph) { g.Options.TCP.IdleTimeout = -time.Second }, wantErr: "traffic.tcp.idle_timeout must be positive"},
519+ {name: "negative tcp drain timeout", mutate: func(g *TrafficGraph) { g.Options.TCP.DrainTimeout = -time.Second }, wantErr: "traffic.tcp.drain_timeout must be positive"},
520+ {name: "negative tcp max connections", mutate: func(g *TrafficGraph) { g.Options.TCP.MaxConnections = -1 }, wantErr: "traffic.tcp.max_connections must be non-negative"},
521+ {name: "negative udp idle timeout", mutate: func(g *TrafficGraph) { g.Options.UDP.IdleTimeout = -time.Second }, wantErr: "traffic.udp.idle_timeout must be positive"},
522+ {name: "negative udp drain timeout", mutate: func(g *TrafficGraph) { g.Options.UDP.DrainTimeout = -time.Second }, wantErr: "traffic.udp.drain_timeout must be positive"},
523+ {name: "negative udp max sessions", mutate: func(g *TrafficGraph) { g.Options.UDP.MaxSessions = -1 }, wantErr: "traffic.udp.max_sessions must be non-negative"},
524+ }
525+ for _, tt := range tests {
526+ t.Run(tt.name, func(t *testing.T) {
527+ graph := base
528+ tt.mutate(&graph)
529+ require.ErrorContains(t, graph.Validate(), tt.wantErr)
530+ })
531+ }
532+}
533+
510 func TestTrafficDomainTypesMatchSpecFields(t *testing.T) {534 func TestTrafficDomainTypesMatchSpecFields(t *testing.T) {
511 graph := TrafficGraph{535 graph := TrafficGraph{
512 Options: TrafficOptions{536 Options: TrafficOptions{

Keyboard shortcuts

Single-key shortcuts are on. ctrl k and esc work whatever this says.

ctrl k :
go to a section, branch, tag, file or repository
?
this list
j k
move the row cursor down / up
enter
open the row under the cursor; on Files, a directory opens or closes in place
↑ ↓ ← → home end
in the file tree (focus on a row): previous / next row, close / open a directory or go to its parent / first entry, first / last row. They never open a file
l h
on Files: open a directory or a file / close a directory or go up to its parent, then to the parent page. Elsewhere: into the side panel (on a merge request, its files) / back to the content; there j k move between its controls
g g G
first / last row
/
filter the list; on Files, find a file anywhere in the repository (esc clears, enter opens the best match)
b
switch branch or tag
g s g f g c g b g m g i g r g a
tabs: summary, files, commits, branches & tags, merge requests, issues, releases, actions (when available)
g h
home
w
wide mode: use the whole width of the screen, or not (remembered)
y
copy a permalink pinned to the commit
d u x
diff: split / unified / collapse or expand all
e E
diff: the whole file in view / every file, with the lines the hunks hide
n
merge request: next note in the diff
esc
close a dialog, leave a field

Branches 7

Tags 106

↑↓ moveenter openesc close