# Troubleshooting

Common issues and solutions when using Gordon.

## Registry Issues

### "unauthorized: authentication required"

**Cause:** Registry authentication is enabled but credentials are missing or invalid.

**Solutions:**

1. Login to registry:
   ```bash
   docker login registry.mydomain.com
   ```

2. Check token is valid:
   ```bash
   gordon auth token list
   ```

3. Generate new token:
   ```bash
   gordon auth token generate --subject myuser --expiry 0
   ```

### "connection refused" on push

**Cause:** Gordon is not running or registry port is blocked.

**Solutions:**

1. Check Gordon is running:
   ```bash
   systemctl --user status gordon
   # or
   pgrep -f "gordon serve"
   ```

2. Check registry port is accessible:
   ```bash
   curl -v http://localhost:5000/v2/
   ```

3. Check firewall:
   ```bash
   sudo firewall-cmd --list-ports
   ```

### "server gave HTTP response to HTTPS client"

**Cause:** Docker/Podman defaults to HTTPS but your registry is serving plain HTTP (common with `localhost:5000` or when TLS is terminated by a reverse proxy).

**Solution — Docker:**

Add the registry to `/etc/docker/daemon.json`:

```json
{
  "insecure-registries": ["gordon.mydomain.com:5000"]
}
```

Then restart Docker:

```bash
sudo systemctl restart docker
```

**Solution — Podman:**

Create or edit `~/.config/containers/registries.conf.d/gordon.conf`:

```toml
[[registry]]
location = "gordon.mydomain.com:5000"
insecure = true
```

No restart needed — Podman reads this on each pull/push.

> **Note:** For `localhost:5000`, Docker already allows insecure access by default. This is only needed for remote or domain-based registry access over HTTP.

### TLS certificate errors with the Gordon CLI

**Cause:** The Gordon server uses a self-signed certificate or TLS is terminated elsewhere.

**Solutions (in priority order):**

1. Per-command flag:

   ```bash
   gordon images push myapp:latest --insecure
   ```

2. Environment variable:

   ```bash
   export GORDON_INSECURE=true
   ```

3. Per-remote in `~/.config/gordon/remotes.toml`:

   ```toml
   [remotes.my-server]
   url = "https://gordon.mydomain.com"
   token = "..."
   insecure_tls = true
   ```

4. Global default in `~/.config/gordon/gordon.toml`:

   ```toml
   [client]
   insecure_tls = true
   ```

### "unknown: image not found"

**Cause:** The app manifest references an image that is unavailable to the runtime.

**Solution:** Push the image to the configured registry, verify the service's `image` reference in the app manifest, then apply and deploy the accepted revision:

```bash
gordon apps apply --file app.toml
gordon apps deploy app
```

## Deployment Issues

### Container starts but app not accessible

**Causes and solutions:**

1. **Wrong port exposed:**
   Check your Dockerfile exposes the correct port:
   ```dockerfile
   EXPOSE 3000
   ```

2. **Multiple ports - wrong one selected:**
   Add `gordon.proxy.port` label:
   ```dockerfile
   LABEL gordon.proxy.port=3000
   ```

3. **App not listening on 0.0.0.0:**
   Ensure app binds to `0.0.0.0`, not `127.0.0.1`:
   ```javascript
   app.listen(3000, '0.0.0.0');
   ```

### Container keeps restarting

**Cause:** Application crashing on startup.

**Solutions:**

1. Check workload logs:
   ```bash
   gordon apps logs blog --service web
   ```

2. Check Gordon logs:
   ```bash
   gordon daemon logs -f
   ```

3. Run container manually to debug:
   ```bash
   docker run -it registry.mydomain.com/myapp:latest sh
   ```

### Environment variables not loaded

**Cause:** The variable is not declared in the app file, or its secret value was never set.

**Solutions:**

1. Declare app-wide public values under `[env]`, per-service public values under `[services.<name>.env]`, and sensitive values under `[services.<name>.secrets]` in the app file, then apply it:
   ```bash
   gordon apps apply --file ./blog.toml
   ```

2. Check which secret values are set (names only):
   ```bash
   gordon apps secrets list blog
   ```

3. Deploy or restart: values apply on the next deploy/restart, never to running containers.
   ```bash
   gordon apps restart blog
   ```

### Secrets not resolved

**Cause:** Secret provider not available or secret not found.

**Solutions:**

1. For `pass` backend:
   ```bash
   # Check pass is available
   which pass

   # Check secret exists
   pass show myapp/db-password
   ```

2. For `sops` backend:
   ```bash
   # Check sops is available
   which sops

   # Check file can be decrypted
   sops -d secrets.yaml
   ```

## Network Issues

### Containers can't reach each other

**Cause:** Network isolation enabled but services not in same network.

**Solutions:**

1. Check that both services declare the same shared network in the app manifest.

2. Check containers are attached to that network:
   ```bash
   docker network inspect NETWORK_NAME
   ```

3. Use the service name as the internal hostname:
   ```javascript
   // Correct
   connect("postgresql://postgres:5432/mydb")

   // Wrong
   connect("postgresql://localhost:5432/mydb")
   ```

### DNS resolution failing

**Cause:** Container can't resolve service names.

**Solutions:**

1. Verify network isolation is enabled:
   ```toml
   [network_isolation]
   enabled = true
   ```

2. Check both containers in same network:
   ```bash
   docker network inspect gordon-app-mydomain-com
   ```

## Volume Issues

### Data not persisting

**Cause:** Volume not configured or not preserved.

**Solutions:**

1. Add VOLUME to Dockerfile:
   ```dockerfile
   VOLUME ["/data"]
   ```

2. Check volume preservation:
   ```toml
   [volumes]
   preserve = true  # default: true
   ```

3. List volumes:
   ```bash
   docker volume ls | grep gordon
   ```

### Disk full

**Cause:** Registry or logs consuming too much space.

**Solutions:**

1. Check disk usage:
   ```bash
   du -sh ~/.gordon/*
   ```

2. Prune unused images:
   ```bash
   docker image prune -a
   ```

3. Configure log rotation:
   ```toml
   [logging.file]
   max_size = 100
   max_backups = 3
   ```

## Configuration Issues

### Config not reloading

**Cause:** Gordon not watching config file.

**Solution:** Manual reload:
```bash
gordon daemon reload
```

### Stale `targets.toml` in config directory

**Cause:** Gordon renamed `targets.toml` to `remotes.toml` in v2.9. The old file is ignored but may cause confusion.

**Solution:** Delete the old file and use `remotes.toml`:

```bash
rm ~/.config/gordon/targets.toml
```

If you had remotes configured in `targets.toml`, recreate them:

```bash
gordon remotes add my-server https://gordon.mydomain.com --token YOUR_TOKEN
```

### "failed to read config file"

**Cause:** TOML syntax error.

**Solution:** Validate TOML:
```bash
# Check for syntax errors
cat ~/.config/gordon/gordon.toml | tomlv
# or use online validator
```

## Logging Issues

### No logs appearing

**Cause:** File logging not enabled.

**Solution:** Enable in config:
```toml
[logging.file]
enabled = true
path = "~/.gordon/logs/gordon.log"
```

### Workload logs unavailable

`gordon apps logs APP --service SERVICE` reads directly from the container runtime. Confirm that the app has an active deployment, use the exact service name, and check the runtime's logging driver and retention settings. Gordon does not write workload logs to `logging.container_logs` files.

## Diagnostic Commands

```bash
# Check Gordon status
systemctl --user status gordon

# View Gordon logs
gordon daemon logs -f
journalctl --user -u gordon -f

# List containers
docker ps -f "label=gordon.managed=true"

# List networks
docker network ls | grep gordon

# List volumes
docker volume ls | grep gordon

# Check workload logs through Gordon
gordon apps logs blog --service web

# Inspect runtime containers when diagnosing locally
docker ps -f "label=gordon.app=blog"

# Check connectivity
curl -v http://localhost:5000/v2/
curl -v http://localhost:8088/
```

## Getting Help

- [GitHub Issues](https://github.com/bnema/gordon/issues)
- [GitHub Discussions](https://github.com/bnema/gordon/discussions)

## Related

- [Installation](../installation.md)
- [Configuration](../config/index.md)
- [CLI Commands](../cli/index.md)