Telemetry Configuration
Export traces, metrics, and logs from Gordon via OpenTelemetry (OTLP/HTTP).
When enabled, Gordon ships all three observability signals to an OTLP-compatible backend such as OpenObserve, Jaeger, or Grafana Cloud. Existing zerolog output continues unchanged; the OTel log bridge runs alongside it.
Configuration
[telemetry]
enabled = false
endpoint = ""
auth_token = ""
traces = true
metrics = true
logs = true
trace_sample_rate = 1.0
Options
| Option |
Type |
Default |
Description |
enabled |
bool |
false |
Enable OTLP telemetry export |
endpoint |
string |
"" |
OTLP HTTP endpoint URL (e.g. http://localhost:4318/api/default) |
auth_token |
string |
"" |
Base64-encoded user:password for Basic auth |
traces |
bool |
true |
Export distributed traces |
metrics |
bool |
true |
Export metrics (deploy counters, container lifecycle, registry, events) |
logs |
bool |
true |
Bridge zerolog output to OTLP logs |
trace_sample_rate |
float |
1.0 |
Fraction of traces to sample (0.0 = none, 1.0 = all) |
How It Works
Gordon initializes an OTel provider at startup. When telemetry is enabled:
- Traces -- Spans wrap critical operations: container deploy, image pull, registry manifest push, and proxy target resolution. The
otelhttp middleware adds a span to every HTTP request on both the proxy and registry servers.
- Metrics -- Gordon records custom counters and histograms for deploys, container restarts, crash loops, managed container count, registry pushes, and event bus throughput.
- Logs -- A zerowrap/otel hook bridges all structured log output to the OTLP log pipeline. Every log line automatically carries
trace_id and span_id when emitted inside a traced request.
When telemetry is disabled (the default), all OTel instruments are noop -- zero overhead.
Endpoint URL
The endpoint field accepts a full URL. Gordon parses it to extract:
- Host and port for the OTLP exporter
- Base path appended with
/v1/traces, /v1/metrics, or /v1/logs
- Scheme --
http:// disables TLS; https:// enables it
For OpenObserve, the endpoint typically includes the organization path:
http://localhost:5080/api/default
This produces export URLs like http://localhost:5080/api/default/v1/traces.
Authentication
Set auth_token to the Base64-encoded user:password string. Gordon sends it as a Basic authorization header on every OTLP export request.
For OpenObserve, copy the token from Ingestion > OTLP in the web UI.
Since Gordon itself is the platform (not a managed container), it does not use gordon secrets set. Store the token with one of these methods:
| Method |
How |
| Environment variable |
GORDON_TELEMETRY_AUTH_TOKEN=<token> gordon serve |
| Plain text in config |
Set auth_token = "<token>" (restrict file permissions) |
Trace Sampling
| Value |
Behavior |
1.0 |
Sample every trace (default) |
0.5 |
Sample 50% of traces |
0.0 |
Drop all traces (tracing pipeline stays initialized) |
At low traffic volumes (< 1000 req/s), keep the rate at 1.0. Reduce it if storage costs or export bandwidth become a concern.
Metrics Reference
Gordon exports these custom metrics (all prefixed with gordon.):
Deployments
| Metric |
Type |
Unit |
Description |
gordon.deploy.total |
Counter |
- |
Total deployments attempted |
gordon.deploy.duration_seconds |
Histogram |
s |
Time from deploy start to completion |
gordon.deploy.errors |
Counter |
- |
Deployments that failed |
Attributes: domain, image
Container Lifecycle
| Metric |
Type |
Unit |
Description |
gordon.container.restarts |
Counter |
- |
Container restart count |
gordon.container.crash_loops |
Counter |
- |
Crash loop detections |
gordon.container.managed |
UpDownCounter |
- |
Currently tracked containers |
Attributes: source (restarts only: monitor or api); gordon.container.managed is a global gauge with no attributes
Registry
| Metric |
Type |
Unit |
Description |
gordon.registry.push.total |
Counter |
- |
Manifest pushes received |
gordon.registry.push.bytes |
Counter |
By |
Total manifest bytes pushed |
Attributes: name, reference
Event Bus
| Metric |
Type |
Unit |
Description |
gordon.events.processed |
Counter |
- |
Events handled successfully |
gordon.events.dropped |
Counter |
- |
Events dropped (channel full) |
Attributes: event_type
HTTP (via otelhttp)
The otelhttp middleware automatically records standard HTTP server metrics on both the proxy and registry servers:
http.server.request.durationhttp.server.request.body.sizehttp.server.response.body.size
Trace Spans
| Span Name |
Package |
Description |
container.deploy |
container |
Full deploy lifecycle (root span) |
container.create_and_start |
container |
Container creation, start, and readiness |
container.ensure_image |
container |
Image pull and tagging |
registry.put_manifest |
registry |
Manifest storage and event publish |
proxy.get_target |
proxy |
Proxy target resolution and caching |
The otelhttp middleware adds an HTTP-level span to every request on both servers.
Examples
OpenObserve (self-hosted)
[telemetry]
enabled = true
endpoint = "http://localhost:5080/api/default"
auth_token = "YWRtaW5AZXhhbXBsZS5jb206c2VjcmV0"
traces = true
metrics = true
logs = true
trace_sample_rate = 1.0
Grafana Cloud (OTLP)
[telemetry]
enabled = true
endpoint = "https://otlp-gateway-prod-us-east-0.grafana.net/otlp"
auth_token = "<instance-id>:<api-key>"
traces = true
metrics = true
logs = true
trace_sample_rate = 0.1
Traces Only (minimal overhead)
[telemetry]
enabled = true
endpoint = "http://localhost:4318"
traces = true
metrics = false
logs = false
trace_sample_rate = 0.5
Environment Variable Override
GORDON_TELEMETRY_ENABLED=true
GORDON_TELEMETRY_ENDPOINT=http://localhost:5080/api/default
GORDON_TELEMETRY_AUTH_TOKEN=YWRtaW5AZXhhbXBsZS5jb206c2VjcmV0
GORDON_TELEMETRY_TRACES=true
GORDON_TELEMETRY_METRICS=true
GORDON_TELEMETRY_LOGS=true
GORDON_TELEMETRY_TRACE_SAMPLE_RATE=1.0
source
1# Telemetry Configuration
2
3Export traces, metrics, and logs from Gordon via OpenTelemetry (OTLP/HTTP).
4
5When enabled, Gordon ships all three observability signals to an OTLP-compatible backend such as OpenObserve, Jaeger, or Grafana Cloud. Existing zerolog output continues unchanged; the OTel log bridge runs alongside it.
6
7## Configuration
8
9```toml
10[telemetry]
11enabled = false
12endpoint = ""
13auth_token = ""
14traces = true
15metrics = true
16logs = true
17trace_sample_rate = 1.0
18```
19
20## Options
21
22| Option | Type | Default | Description |
23|--------|------|---------|-------------|
24| `enabled` | bool | `false` | Enable OTLP telemetry export |
25| `endpoint` | string | `""` | OTLP HTTP endpoint URL (e.g. `http://localhost:4318/api/default`) |
26| `auth_token` | string | `""` | Base64-encoded `user:password` for Basic auth |
27| `traces` | bool | `true` | Export distributed traces |
28| `metrics` | bool | `true` | Export metrics (deploy counters, container lifecycle, registry, events) |
29| `logs` | bool | `true` | Bridge zerolog output to OTLP logs |
30| `trace_sample_rate` | float | `1.0` | Fraction of traces to sample (`0.0` = none, `1.0` = all) |
31
32## How It Works
33
34Gordon initializes an OTel provider at startup. When telemetry is enabled:
35
361. **Traces** -- Spans wrap critical operations: container deploy, image pull, registry manifest push, and proxy target resolution. The `otelhttp` middleware adds a span to every HTTP request on both the proxy and registry servers.
372. **Metrics** -- Gordon records custom counters and histograms for deploys, container restarts, crash loops, managed container count, registry pushes, and event bus throughput.
383. **Logs** -- A zerowrap/otel hook bridges all structured log output to the OTLP log pipeline. Every log line automatically carries `trace_id` and `span_id` when emitted inside a traced request.
39
40When telemetry is disabled (the default), all OTel instruments are noop -- zero overhead.
41
42## Endpoint URL
43
44The `endpoint` field accepts a full URL. Gordon parses it to extract:
45
46- **Host and port** for the OTLP exporter
47- **Base path** appended with `/v1/traces`, `/v1/metrics`, or `/v1/logs`
48- **Scheme** -- `http://` disables TLS; `https://` enables it
49
50For OpenObserve, the endpoint typically includes the organization path:
51
52```text
53http://localhost:5080/api/default
54```
55
56This produces export URLs like `http://localhost:5080/api/default/v1/traces`.
57
58## Authentication
59
60Set `auth_token` to the Base64-encoded `user:password` string. Gordon sends it as a `Basic` authorization header on every OTLP export request.
61
62For OpenObserve, copy the token from **Ingestion > OTLP** in the web UI.
63
64Since Gordon itself is the platform (not a managed container), it does not use `gordon secrets set`. Store the token with one of these methods:
65
66| Method | How |
67|--------|-----|
68| Environment variable | `GORDON_TELEMETRY_AUTH_TOKEN=<token> gordon serve` |
69| Plain text in config | Set `auth_token = "<token>"` (restrict file permissions) |
70
71## Trace Sampling
72
73| Value | Behavior |
74|-------|----------|
75| `1.0` | Sample every trace (default) |
76| `0.5` | Sample 50% of traces |
77| `0.0` | Drop all traces (tracing pipeline stays initialized) |
78
79At low traffic volumes (< 1000 req/s), keep the rate at `1.0`. Reduce it if storage costs or export bandwidth become a concern.
80
81## Metrics Reference
82
83Gordon exports these custom metrics (all prefixed with `gordon.`):
84
85### Deployments
86
87| Metric | Type | Unit | Description |
88|--------|------|------|-------------|
89| `gordon.deploy.total` | Counter | - | Total deployments attempted |
90| `gordon.deploy.duration_seconds` | Histogram | s | Time from deploy start to completion |
91| `gordon.deploy.errors` | Counter | - | Deployments that failed |
92
93Attributes: `domain`, `image`
94
95### Container Lifecycle
96
97| Metric | Type | Unit | Description |
98|--------|------|------|-------------|
99| `gordon.container.restarts` | Counter | - | Container restart count |
100| `gordon.container.crash_loops` | Counter | - | Crash loop detections |
101| `gordon.container.managed` | UpDownCounter | - | Currently tracked containers |
102
103Attributes: `source` (restarts only: `monitor` or `api`); `gordon.container.managed` is a global gauge with no attributes
104
105### Registry
106
107| Metric | Type | Unit | Description |
108|--------|------|------|-------------|
109| `gordon.registry.push.total` | Counter | - | Manifest pushes received |
110| `gordon.registry.push.bytes` | Counter | By | Total manifest bytes pushed |
111
112Attributes: `name`, `reference`
113
114### Event Bus
115
116| Metric | Type | Unit | Description |
117|--------|------|------|-------------|
118| `gordon.events.processed` | Counter | - | Events handled successfully |
119| `gordon.events.dropped` | Counter | - | Events dropped (channel full) |
120
121Attributes: `event_type`
122
123### HTTP (via otelhttp)
124
125The `otelhttp` middleware automatically records standard HTTP server metrics on both the proxy and registry servers:
126
127- `http.server.request.duration`
128- `http.server.request.body.size`
129- `http.server.response.body.size`
130
131## Trace Spans
132
133| Span Name | Package | Description |
134|-----------|---------|-------------|
135| `container.deploy` | container | Full deploy lifecycle (root span) |
136| `container.create_and_start` | container | Container creation, start, and readiness |
137| `container.ensure_image` | container | Image pull and tagging |
138| `registry.put_manifest` | registry | Manifest storage and event publish |
139| `proxy.get_target` | proxy | Proxy target resolution and caching |
140
141The `otelhttp` middleware adds an HTTP-level span to every request on both servers.
142
143## Examples
144
145### OpenObserve (self-hosted)
146
147```toml
148[telemetry]
149enabled = true
150endpoint = "http://localhost:5080/api/default"
151auth_token = "YWRtaW5AZXhhbXBsZS5jb206c2VjcmV0"
152traces = true
153metrics = true
154logs = true
155trace_sample_rate = 1.0
156```
157
158### Grafana Cloud (OTLP)
159
160```toml
161[telemetry]
162enabled = true
163endpoint = "https://otlp-gateway-prod-us-east-0.grafana.net/otlp"
164auth_token = "<instance-id>:<api-key>"
165traces = true
166metrics = true
167logs = true
168trace_sample_rate = 0.1
169```
170
171### Traces Only (minimal overhead)
172
173```toml
174[telemetry]
175enabled = true
176endpoint = "http://localhost:4318"
177traces = true
178metrics = false
179logs = false
180trace_sample_rate = 0.5
181```
182
183## Environment Variable Override
184
185```bash
186GORDON_TELEMETRY_ENABLED=true
187GORDON_TELEMETRY_ENDPOINT=http://localhost:5080/api/default
188GORDON_TELEMETRY_AUTH_TOKEN=YWRtaW5AZXhhbXBsZS5jb206c2VjcmV0
189GORDON_TELEMETRY_TRACES=true
190GORDON_TELEMETRY_METRICS=true
191GORDON_TELEMETRY_LOGS=true
192GORDON_TELEMETRY_TRACE_SAMPLE_RATE=1.0
193```
194
195## Related
196
197- [Configuration Overview](./index.md)
198- [Logging](./logging.md)
199- [Configuration Reference](./reference.md)