v2.31.1 f37fe67

troubleshooting.md RawHistoryFolder

Troubleshooting

Common issues and solutions when using Gordon.

Registry Issues

"unauthorized: authentication required"

Cause: Registry authentication is enabled but credentials are missing or invalid.

Solutions:

  1. Login to registry:

    docker login registry.mydomain.com
    
  2. Check token is valid:

    gordon auth token list
    
  3. Generate new token:

    gordon auth token generate --subject myuser --expiry 0
    

"connection refused" on push

Cause: Gordon is not running or registry port is blocked.

Solutions:

  1. Check Gordon is running:

    systemctl --user status gordon
    # or
    pgrep -f "gordon serve"
    
  2. Check registry port is accessible:

    curl -v http://localhost:5000/v2/
    
  3. Check firewall:

    sudo firewall-cmd --list-ports
    

"server gave HTTP response to HTTPS client"

Cause: Docker/Podman defaults to HTTPS but your registry is serving plain HTTP (common with localhost:5000 or when TLS is terminated by a reverse proxy).

Solution — Docker:

Add the registry to /etc/docker/daemon.json:

{
  "insecure-registries": ["gordon.mydomain.com:5000"]
}

Then restart Docker:

sudo systemctl restart docker

Solution — Podman:

Create or edit ~/.config/containers/registries.conf.d/gordon.conf:

[[registry]]
location = "gordon.mydomain.com:5000"
insecure = true

No restart needed — Podman reads this on each pull/push.

Note: For localhost:5000, Docker already allows insecure access by default. This is only needed for remote or domain-based registry access over HTTP.

TLS certificate errors with the Gordon CLI

Cause: The Gordon server uses a self-signed certificate or TLS is terminated elsewhere.

Solutions (in priority order):

  1. Per-command flag:

    gordon push myapp:latest --insecure
    
  2. Environment variable:

    export GORDON_INSECURE=true
    
  3. Per-remote in ~/.config/gordon/remotes.toml:

    [remotes.my-server]
    url = "https://gordon.mydomain.com"
    token = "..."
    insecure_tls = true
    
  4. Global default in ~/.config/gordon/gordon.toml:

    [client]
    insecure_tls = true
    

"unknown: image not found"

Cause: Image was pushed but no route configured.

Solution: Add route to config:

[routes]
"app.mydomain.com" = "myapp:latest"

Then reload:

gordon reload

Deployment Issues

Container starts but app not accessible

Causes and solutions:

  1. Wrong port exposed: Check your Dockerfile exposes the correct port:

    EXPOSE 3000
    
  2. Multiple ports - wrong one selected: Add gordon.proxy.port label:

    LABEL gordon.proxy.port=3000
    
  3. App not listening on 0.0.0.0: Ensure app binds to 0.0.0.0, not 127.0.0.1:

    app.listen(3000, '0.0.0.0');
    

Container keeps restarting

Cause: Application crashing on startup.

Solutions:

  1. Check container logs:

    docker logs gordon-app-mydomain-com
    
  2. Check Gordon logs:

    gordon logs -f
    
  3. Run container manually to debug:

    docker run -it registry.mydomain.com/myapp:latest sh
    

Environment variables not loaded

Cause: Env file not found or wrong format.

Solutions:

  1. Check file exists with correct name:

    ls ~/.gordon/env/
    # Should show: app_mydomain_com.env (dots → underscores)
    
  2. Check file permissions:

    chmod 600 ~/.gordon/env/app_mydomain_com.env
    
  3. Check file format (no spaces around =):

    # Correct
    KEY=value
    
    # Wrong
    KEY = value
    

Secrets not resolved

Cause: Secret provider not available or secret not found.

Solutions:

  1. For pass backend:

    # Check pass is available
    which pass
    
    # Check secret exists
    pass show myapp/db-password
    
  2. For sops backend:

    # Check sops is available
    which sops
    
    # Check file can be decrypted
    sops -d secrets.yaml
    

Network Issues

Containers can't reach each other

Cause: Network isolation enabled but services not in same network.

Solutions:

  1. Check attachments are configured:

    [attachments]
    "app.mydomain.com" = ["postgres:latest"]
    
  2. Check containers are in same network:

    docker network inspect gordon-app-mydomain-com
    
  3. Use correct hostname (image name before colon):

    // Correct
    connect("postgresql://postgres:5432/mydb")
    
    // Wrong
    connect("postgresql://localhost:5432/mydb")
    

DNS resolution failing

Cause: Container can't resolve service names.

Solutions:

  1. Verify network isolation is enabled:

    [network_isolation]
    enabled = true
    
  2. Check both containers in same network:

    docker network inspect gordon-app-mydomain-com
    

Volume Issues

Data not persisting

Cause: Volume not configured or not preserved.

Solutions:

  1. Add VOLUME to Dockerfile:

    VOLUME ["/data"]
    
  2. Check volume preservation:

    [volumes]
    preserve = true  # default: true
    
  3. List volumes:

    docker volume ls | grep gordon
    

Disk full

Cause: Registry or logs consuming too much space.

Solutions:

  1. Check disk usage:

    du -sh ~/.gordon/*
    
  2. Prune unused images:

    docker image prune -a
    
  3. Configure log rotation:

    [logging.file]
    max_size = 100
    max_backups = 3
    

Configuration Issues

Config not reloading

Cause: Gordon not watching config file.

Solution: Manual reload:

gordon reload

Stale targets.toml in config directory

Cause: Gordon renamed targets.toml to remotes.toml in v2.9. The old file is ignored but may cause confusion.

Solution: Delete the old file and use remotes.toml:

rm ~/.config/gordon/targets.toml

If you had remotes configured in targets.toml, recreate them:

gordon remotes add my-server https://gordon.mydomain.com --token YOUR_TOKEN

"failed to read config file"

Cause: TOML syntax error.

Solution: Validate TOML:

# Check for syntax errors
cat ~/.config/gordon/gordon.toml | tomlv
# or use online validator

Logging Issues

No logs appearing

Cause: File logging not enabled.

Solution: Enable in config:

[logging.file]
enabled = true
path = "~/.gordon/logs/gordon.log"

Container logs missing

Cause: Container log collection disabled.

Solution:

[logging.container_logs]
enabled = true  # default: true

Diagnostic Commands

# Check Gordon status
systemctl --user status gordon

# View Gordon logs
gordon logs -f
journalctl --user -u gordon -f

# List containers
docker ps -f "label=gordon.managed=true"

# List networks
docker network ls | grep gordon

# List volumes
docker volume ls | grep gordon

# Check container logs
docker logs gordon-app-mydomain-com

# Inspect container
docker inspect gordon-app-mydomain-com

# Check connectivity
curl -v http://localhost:5000/v2/
curl -v http://localhost:8088/

Getting Help

source
1# Troubleshooting
2
3Common issues and solutions when using Gordon.
4
5## Registry Issues
6
7### "unauthorized: authentication required"
8
9**Cause:** Registry authentication is enabled but credentials are missing or invalid.
10
11**Solutions:**
12
131. Login to registry:
14   ```bash
15   docker login registry.mydomain.com
16   ```
17
182. Check token is valid:
19   ```bash
20   gordon auth token list
21   ```
22
233. Generate new token:
24   ```bash
25   gordon auth token generate --subject myuser --expiry 0
26   ```
27
28### "connection refused" on push
29
30**Cause:** Gordon is not running or registry port is blocked.
31
32**Solutions:**
33
341. Check Gordon is running:
35   ```bash
36   systemctl --user status gordon
37   # or
38   pgrep -f "gordon serve"
39   ```
40
412. Check registry port is accessible:
42   ```bash
43   curl -v http://localhost:5000/v2/
44   ```
45
463. Check firewall:
47   ```bash
48   sudo firewall-cmd --list-ports
49   ```
50
51### "server gave HTTP response to HTTPS client"
52
53**Cause:** Docker/Podman defaults to HTTPS but your registry is serving plain HTTP (common with `localhost:5000` or when TLS is terminated by a reverse proxy).
54
55**Solution — Docker:**
56
57Add the registry to `/etc/docker/daemon.json`:
58
59```json
60{
61  "insecure-registries": ["gordon.mydomain.com:5000"]
62}
63```
64
65Then restart Docker:
66
67```bash
68sudo systemctl restart docker
69```
70
71**Solution — Podman:**
72
73Create or edit `~/.config/containers/registries.conf.d/gordon.conf`:
74
75```toml
76[[registry]]
77location = "gordon.mydomain.com:5000"
78insecure = true
79```
80
81No restart needed — Podman reads this on each pull/push.
82
83> **Note:** For `localhost:5000`, Docker already allows insecure access by default. This is only needed for remote or domain-based registry access over HTTP.
84
85### TLS certificate errors with the Gordon CLI
86
87**Cause:** The Gordon server uses a self-signed certificate or TLS is terminated elsewhere.
88
89**Solutions (in priority order):**
90
911. Per-command flag:
92
93   ```bash
94   gordon push myapp:latest --insecure
95   ```
96
972. Environment variable:
98
99   ```bash
100   export GORDON_INSECURE=true
101   ```
102
1033. Per-remote in `~/.config/gordon/remotes.toml`:
104
105   ```toml
106   [remotes.my-server]
107   url = "https://gordon.mydomain.com"
108   token = "..."
109   insecure_tls = true
110   ```
111
1124. Global default in `~/.config/gordon/gordon.toml`:
113
114   ```toml
115   [client]
116   insecure_tls = true
117   ```
118
119### "unknown: image not found"
120
121**Cause:** Image was pushed but no route configured.
122
123**Solution:** Add route to config:
124```toml
125[routes]
126"app.mydomain.com" = "myapp:latest"
127```
128
129Then reload:
130```bash
131gordon reload
132```
133
134## Deployment Issues
135
136### Container starts but app not accessible
137
138**Causes and solutions:**
139
1401. **Wrong port exposed:**
141   Check your Dockerfile exposes the correct port:
142   ```dockerfile
143   EXPOSE 3000
144   ```
145
1462. **Multiple ports - wrong one selected:**
147   Add `gordon.proxy.port` label:
148   ```dockerfile
149   LABEL gordon.proxy.port=3000
150   ```
151
1523. **App not listening on 0.0.0.0:**
153   Ensure app binds to `0.0.0.0`, not `127.0.0.1`:
154   ```javascript
155   app.listen(3000, '0.0.0.0');
156   ```
157
158### Container keeps restarting
159
160**Cause:** Application crashing on startup.
161
162**Solutions:**
163
1641. Check container logs:
165   ```bash
166   docker logs gordon-app-mydomain-com
167   ```
168
1692. Check Gordon logs:
170   ```bash
171   gordon logs -f
172   ```
173
1743. Run container manually to debug:
175   ```bash
176   docker run -it registry.mydomain.com/myapp:latest sh
177   ```
178
179### Environment variables not loaded
180
181**Cause:** Env file not found or wrong format.
182
183**Solutions:**
184
1851. Check file exists with correct name:
186   ```bash
187   ls ~/.gordon/env/
188   # Should show: app_mydomain_com.env (dots → underscores)
189   ```
190
1912. Check file permissions:
192   ```bash
193   chmod 600 ~/.gordon/env/app_mydomain_com.env
194   ```
195
1963. Check file format (no spaces around `=`):
197   ```bash
198   # Correct
199   KEY=value
200
201   # Wrong
202   KEY = value
203   ```
204
205### Secrets not resolved
206
207**Cause:** Secret provider not available or secret not found.
208
209**Solutions:**
210
2111. For `pass` backend:
212   ```bash
213   # Check pass is available
214   which pass
215
216   # Check secret exists
217   pass show myapp/db-password
218   ```
219
2202. For `sops` backend:
221   ```bash
222   # Check sops is available
223   which sops
224
225   # Check file can be decrypted
226   sops -d secrets.yaml
227   ```
228
229## Network Issues
230
231### Containers can't reach each other
232
233**Cause:** Network isolation enabled but services not in same network.
234
235**Solutions:**
236
2371. Check attachments are configured:
238   ```toml
239   [attachments]
240   "app.mydomain.com" = ["postgres:latest"]
241   ```
242
2432. Check containers are in same network:
244   ```bash
245   docker network inspect gordon-app-mydomain-com
246   ```
247
2483. Use correct hostname (image name before colon):
249   ```javascript
250   // Correct
251   connect("postgresql://postgres:5432/mydb")
252
253   // Wrong
254   connect("postgresql://localhost:5432/mydb")
255   ```
256
257### DNS resolution failing
258
259**Cause:** Container can't resolve service names.
260
261**Solutions:**
262
2631. Verify network isolation is enabled:
264   ```toml
265   [network_isolation]
266   enabled = true
267   ```
268
2692. Check both containers in same network:
270   ```bash
271   docker network inspect gordon-app-mydomain-com
272   ```
273
274## Volume Issues
275
276### Data not persisting
277
278**Cause:** Volume not configured or not preserved.
279
280**Solutions:**
281
2821. Add VOLUME to Dockerfile:
283   ```dockerfile
284   VOLUME ["/data"]
285   ```
286
2872. Check volume preservation:
288   ```toml
289   [volumes]
290   preserve = true  # default: true
291   ```
292
2933. List volumes:
294   ```bash
295   docker volume ls | grep gordon
296   ```
297
298### Disk full
299
300**Cause:** Registry or logs consuming too much space.
301
302**Solutions:**
303
3041. Check disk usage:
305   ```bash
306   du -sh ~/.gordon/*
307   ```
308
3092. Prune unused images:
310   ```bash
311   docker image prune -a
312   ```
313
3143. Configure log rotation:
315   ```toml
316   [logging.file]
317   max_size = 100
318   max_backups = 3
319   ```
320
321## Configuration Issues
322
323### Config not reloading
324
325**Cause:** Gordon not watching config file.
326
327**Solution:** Manual reload:
328```bash
329gordon reload
330```
331
332### Stale `targets.toml` in config directory
333
334**Cause:** Gordon renamed `targets.toml` to `remotes.toml` in v2.9. The old file is ignored but may cause confusion.
335
336**Solution:** Delete the old file and use `remotes.toml`:
337
338```bash
339rm ~/.config/gordon/targets.toml
340```
341
342If you had remotes configured in `targets.toml`, recreate them:
343
344```bash
345gordon remotes add my-server https://gordon.mydomain.com --token YOUR_TOKEN
346```
347
348### "failed to read config file"
349
350**Cause:** TOML syntax error.
351
352**Solution:** Validate TOML:
353```bash
354# Check for syntax errors
355cat ~/.config/gordon/gordon.toml | tomlv
356# or use online validator
357```
358
359## Logging Issues
360
361### No logs appearing
362
363**Cause:** File logging not enabled.
364
365**Solution:** Enable in config:
366```toml
367[logging.file]
368enabled = true
369path = "~/.gordon/logs/gordon.log"
370```
371
372### Container logs missing
373
374**Cause:** Container log collection disabled.
375
376**Solution:**
377```toml
378[logging.container_logs]
379enabled = true  # default: true
380```
381
382## Diagnostic Commands
383
384```bash
385# Check Gordon status
386systemctl --user status gordon
387
388# View Gordon logs
389gordon logs -f
390journalctl --user -u gordon -f
391
392# List containers
393docker ps -f "label=gordon.managed=true"
394
395# List networks
396docker network ls | grep gordon
397
398# List volumes
399docker volume ls | grep gordon
400
401# Check container logs
402docker logs gordon-app-mydomain-com
403
404# Inspect container
405docker inspect gordon-app-mydomain-com
406
407# Check connectivity
408curl -v http://localhost:5000/v2/
409curl -v http://localhost:8088/
410```
411
412## Getting Help
413
414- [GitHub Issues](https://github.com/bnema/gordon/issues)
415- [GitHub Discussions](https://github.com/bnema/gordon/discussions)
416
417## Related
418
419- [Installation](../installation.md)
420- [Configuration](../config/index.md)
421- [CLI Commands](../cli/index.md)

Keyboard shortcuts

Single-key shortcuts are on. ctrl k and esc work whatever this says.

ctrl k :
go to a section, branch, tag, file or repository
?
this list
j k
move the row cursor down / up
enter
open the row under the cursor; on Files, a directory opens or closes in place
↑ ↓ ← → home end
in the file tree (focus on a row): previous / next row, close / open a directory or go to its parent / first entry, first / last row. They never open a file
l h
on Files: open a directory or a file / close a directory or go up to its parent, then to the parent page. Elsewhere: into the side panel (on a merge request, its files) / back to the content; there j k move between its controls
g g G
first / last row
/
filter the list; on Files, find a file anywhere in the repository (esc clears, enter opens the best match)
b
switch branch or tag
g s g f g c g b g m g i g r g a
tabs: summary, files, commits, branches & tags, merge requests, issues, releases, actions (when available)
g h
home
w
wide mode: use the whole width of the screen, or not (remembered)
y
copy a permalink pinned to the commit
d u x
diff: split / unified / collapse or expand all
e E
diff: the whole file in view / every file, with the lines the hunks hide
n
merge request: next note in the diff
esc
close a dialog, leave a field

Branches 7

Tags 106

↑↓ moveenter openesc close