secrets-sops.md Raw History Folder

Using SOPS for Secrets

Set up Mozilla SOPS as Gordon's secrets backend for encrypted file-based secret storage.

What You'll Learn

Installing SOPS and age (or GPG)
Creating encrypted secrets files
Configuring Gordon to use SOPS
Using secrets in environment files
Team workflows and key rotation

Prerequisites

Linux or macOS system
Basic understanding of encryption concepts

Why SOPS?

SOPS (Secrets OPerationS) encrypts specific values in YAML/JSON files while keeping keys readable:

Git-friendly: Encrypted files can be version controlled
Multiple backends: age, GPG, AWS KMS, GCP KMS, Azure Key Vault
Selective encryption: Only values are encrypted, keys remain visible
Team support: Multiple recipients can decrypt the same file

Installation

SOPS

# Ubuntu/Debian
curl -LO https://github.com/getsops/sops/releases/download/v3.9.0/sops-v3.9.0.linux.amd64
sudo mv sops-v3.9.0.linux.amd64 /usr/local/bin/sops
sudo chmod +x /usr/local/bin/sops

# macOS
brew install sops

# Arch Linux
sudo pacman -S sops

age (Recommended)

age is simpler than GPG and recommended for new setups:

# Ubuntu/Debian
sudo apt install age

# macOS
brew install age

# Arch Linux
sudo pacman -S age

GPG (Alternative)

# Ubuntu/Debian
sudo apt install gnupg

# macOS
brew install gnupg

Setup with age

1. Generate an age Key

# Generate key pair
age-keygen -o ~/.config/sops/age/keys.txt

# Output shows your public key:
# Public key: age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Note your public key - you'll need it for encryption.

2. Configure SOPS

Create .sops.yaml in your project or home directory:

creation_rules:
  - age: age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Or for multiple recipients:

creation_rules:
  - age: >-
      age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,
      age1yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

3. Set age Key Location

# Add to ~/.bashrc or ~/.zshrc
export SOPS_AGE_KEY_FILE=~/.config/sops/age/keys.txt

Setup with GPG

1. Generate or Use Existing GPG Key

# Generate new key
gpg --gen-key

# List keys to get fingerprint
gpg --list-keys

2. Configure SOPS

creation_rules:
  - pgp: ABCD1234EFGH5678...  # Your GPG fingerprint

Creating Encrypted Secrets

Create secrets.yaml

# Create and encrypt in one step
sops edit secrets.yaml

This opens your editor. Add secrets in YAML format:

auth:
  password_hash: "$2y$10$..."
  token_secret: "your-random-32-char-string-here"
database:
  password: "db-password-here"
api:
  key: "api-key-here"

Save and exit. SOPS encrypts the values automatically.

View Encrypted File

cat secrets.yaml

Output shows encrypted values:

auth:
    password_hash: ENC[AES256_GCM,data:...,iv:...,tag:...]
    token_secret: ENC[AES256_GCM,data:...,iv:...,tag:...]
database:
    password: ENC[AES256_GCM,data:...,iv:...,tag:...]
api:
    key: ENC[AES256_GCM,data:...,iv:...,tag:...]
sops:
    age:
        - recipient: age1xxx...
          enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
            ...
            -----END AGE ENCRYPTED FILE-----
    lastmodified: "2024-01-15T12:00:00Z"
    version: 3.9.0

Decrypt and View

# View entire file
sops decrypt secrets.yaml

# Extract specific value
sops decrypt --extract '["registry"]["token_secret"]' secrets.yaml

Edit Existing File

sops edit secrets.yaml

Gordon Configuration

Enable SOPS Backend

[auth]
enabled = true
secrets_backend = "sops"
token_secret = "secrets.yaml:auth.token_secret"
# Optional: enable password auth for interactive login
# username = "admin"
# password_hash = "secrets.yaml:auth.password_hash"

The path format is file:key.path where:

file is the SOPS-encrypted file path
key.path is dot-notation to the value

Route secrets remain in .env files with ${sops:...} references. The SOPS backend is used for auth secrets (like token_secret) and provider lookups.

Using Secrets in Environment Files

Reference SOPS secrets in your app's environment files:

# ~/.gordon/env/app_mydomain_com.env
DATABASE_URL=postgresql://user:${sops:secrets.yaml:database.password}@postgres:5432/app
API_KEY=${sops:secrets.yaml:api.key}

File Organization

Recommended structure:

project/
├── .sops.yaml              # SOPS configuration
├── secrets.yaml            # Encrypted secrets (commit this)
├── secrets.dev.yaml        # Dev environment secrets
├── secrets.prod.yaml       # Production secrets
└── gordon.toml             # Gordon config (references secrets)

Environment-Specific Rules

# .sops.yaml
creation_rules:
  # Dev secrets - developer keys
  - path_regex: \.dev\.yaml$
    age: age1devkey...

  # Prod secrets - production keys only
  - path_regex: \.prod\.yaml$
    age: age1prodkey...

  # Default
  - age: age1defaultkey...

Running in Containers

Custom Dockerfile

FROM ghcr.io/bnema/gordon:latest

USER root
RUN apk add --no-cache sops age
USER gordon

Mount Secrets and Keys

docker run -d \
  --name gordon \
  -p 80:8080 \
  -p 5000:5000 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v gordon-data:/data \
  -v $(pwd)/secrets.yaml:/app/secrets.yaml:ro \
  -v $(pwd)/.sops.yaml:/app/.sops.yaml:ro \
  -v ~/.config/sops/age/keys.txt:/home/gordon/.config/sops/age/keys.txt:ro \
  -e SOPS_AGE_KEY_FILE=/home/gordon/.config/sops/age/keys.txt \
  -v $(pwd)/gordon.toml:/etc/gordon/gordon.toml:ro \
  gordon-with-sops

Docker Compose

services:
  gordon:
    build:
      context: .
      dockerfile: Dockerfile.gordon
    container_name: gordon
    restart: unless-stopped
    ports:
      - "80:8080"
      - "5000:5000"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - gordon-data:/data
      - ./gordon.toml:/etc/gordon/gordon.toml:ro
      - ./secrets.yaml:/app/secrets.yaml:ro
      - ./.sops.yaml:/app/.sops.yaml:ro
      - ~/.config/sops/age/keys.txt:/home/gordon/.config/sops/age/keys.txt:ro
    environment:
      - SOPS_AGE_KEY_FILE=/home/gordon/.config/sops/age/keys.txt
      - GORDON_SECRETS_BACKEND=sops

volumes:
  gordon-data:

Team Workflows

Adding Team Members

Get their age public key
Add to .sops.yaml:

creation_rules:
  - age: >-
      age1yourkey...,
      age1teammemberkey...

Re-encrypt existing files:

sops updatekeys secrets.yaml

Key Rotation

Generate new key
Update .sops.yaml
Re-encrypt all files:

sops updatekeys secrets.yaml
sops updatekeys secrets.prod.yaml

Git Workflow

Encrypted files are safe to commit:

git add .sops.yaml secrets.yaml
git commit -m "Add encrypted secrets"
git push

Team members with valid keys can decrypt after pulling.

Troubleshooting

"failed to get the data key"

Your key isn't in the recipients list or key file is missing:

# Check if age key is set
echo $SOPS_AGE_KEY_FILE
cat $SOPS_AGE_KEY_FILE

# Check recipients in encrypted file
grep -A5 "age:" secrets.yaml

"sops: command not found"

Install SOPS:

# Check if installed
which sops

# Install if missing (see Installation section)

"could not decrypt data key"

Wrong key or corrupted file:

# Verify your public key matches a recipient
age-keygen -y ~/.config/sops/age/keys.txt
# Compare output to recipients in .sops.yaml

"no matching creation_rules"

Create or fix .sops.yaml:

# Check if .sops.yaml exists
cat .sops.yaml

# Ensure path_regex matches your file

Permission Denied in Container

Ensure key file is readable:

docker exec gordon cat /home/gordon/.config/sops/age/keys.txt

Check mount permissions:

ls -la ~/.config/sops/age/keys.txt
# Should be readable by container user

Security Best Practices

Never commit unencrypted secrets: Always use sops edit, never edit decrypted files directly
Protect age/GPG keys: Store private keys securely, never commit them
Use separate keys per environment: Production keys should be isolated
Rotate keys regularly: Update keys and re-encrypt periodically
Audit access: Track who has decryption keys for each environment
Use .sops.yaml: Enforce encryption rules via configuration

source

1# Using SOPS for Secrets
2
3Set up Mozilla SOPS as Gordon's secrets backend for encrypted file-based secret storage.
4
5## What You'll Learn
6
7- Installing SOPS and age (or GPG)
8- Creating encrypted secrets files
9- Configuring Gordon to use SOPS
10- Using secrets in environment files
11- Team workflows and key rotation
12
13## Prerequisites
14
15- Linux or macOS system
16- Basic understanding of encryption concepts
17
18## Why SOPS?
19
20SOPS (Secrets OPerationS) encrypts specific values in YAML/JSON files while keeping keys readable:
21
22- **Git-friendly**: Encrypted files can be version controlled
23- **Multiple backends**: age, GPG, AWS KMS, GCP KMS, Azure Key Vault
24- **Selective encryption**: Only values are encrypted, keys remain visible
25- **Team support**: Multiple recipients can decrypt the same file
26
27## Installation
28
29### SOPS
30
31```bash
32# Ubuntu/Debian
33curl -LO https://github.com/getsops/sops/releases/download/v3.9.0/sops-v3.9.0.linux.amd64
34sudo mv sops-v3.9.0.linux.amd64 /usr/local/bin/sops
35sudo chmod +x /usr/local/bin/sops
36
37# macOS
38brew install sops
39
40# Arch Linux
41sudo pacman -S sops
42```
43
44### age (Recommended)
45
46age is simpler than GPG and recommended for new setups:
47
48```bash
49# Ubuntu/Debian
50sudo apt install age
51
52# macOS
53brew install age
54
55# Arch Linux
56sudo pacman -S age
57```
58
59### GPG (Alternative)
60
61```bash
62# Ubuntu/Debian
63sudo apt install gnupg
64
65# macOS
66brew install gnupg
67```
68
69## Setup with age
70
71### 1. Generate an age Key
72
73```bash
74# Generate key pair
75age-keygen -o ~/.config/sops/age/keys.txt
76
77# Output shows your public key:
78# Public key: age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
79```
80
81Note your public key - you'll need it for encryption.
82
83### 2. Configure SOPS
84
85Create `.sops.yaml` in your project or home directory:
86
87```yaml
88creation_rules:
89  - age: age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
90```
91
92Or for multiple recipients:
93
94```yaml
95creation_rules:
96  - age: >-
97      age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,
98      age1yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
99```
100
101### 3. Set age Key Location
102
103```bash
104# Add to ~/.bashrc or ~/.zshrc
105export SOPS_AGE_KEY_FILE=~/.config/sops/age/keys.txt
106```
107
108## Setup with GPG
109
110### 1. Generate or Use Existing GPG Key
111
112```bash
113# Generate new key
114gpg --gen-key
115
116# List keys to get fingerprint
117gpg --list-keys
118```
119
120### 2. Configure SOPS
121
122```yaml
123creation_rules:
124  - pgp: ABCD1234EFGH5678...  # Your GPG fingerprint
125```
126
127## Creating Encrypted Secrets
128
129### Create secrets.yaml
130
131```bash
132# Create and encrypt in one step
133sops edit secrets.yaml
134```
135
136This opens your editor. Add secrets in YAML format:
137
138```yaml
139auth:
140  password_hash: "$2y$10$..."
141  token_secret: "your-random-32-char-string-here"
142database:
143  password: "db-password-here"
144api:
145  key: "api-key-here"
146```
147
148Save and exit. SOPS encrypts the values automatically.
149
150### View Encrypted File
151
152```bash
153cat secrets.yaml
154```
155
156Output shows encrypted values:
157
158```yaml
159auth:
160    password_hash: ENC[AES256_GCM,data:...,iv:...,tag:...]
161    token_secret: ENC[AES256_GCM,data:...,iv:...,tag:...]
162database:
163    password: ENC[AES256_GCM,data:...,iv:...,tag:...]
164api:
165    key: ENC[AES256_GCM,data:...,iv:...,tag:...]
166sops:
167    age:
168        - recipient: age1xxx...
169          enc: |
170            -----BEGIN AGE ENCRYPTED FILE-----
171            ...
172            -----END AGE ENCRYPTED FILE-----
173    lastmodified: "2024-01-15T12:00:00Z"
174    version: 3.9.0
175```
176
177### Decrypt and View
178
179```bash
180# View entire file
181sops decrypt secrets.yaml
182
183# Extract specific value
184sops decrypt --extract '["registry"]["token_secret"]' secrets.yaml
185```
186
187### Edit Existing File
188
189```bash
190sops edit secrets.yaml
191```
192
193## Gordon Configuration
194
195### Enable SOPS Backend
196
197```toml
198[auth]
199enabled = true
200secrets_backend = "sops"
201token_secret = "secrets.yaml:auth.token_secret"
202# Optional: enable password auth for interactive login
203# username = "admin"
204# password_hash = "secrets.yaml:auth.password_hash"
205```
206
207The path format is `file:key.path` where:
208- `file` is the SOPS-encrypted file path
209- `key.path` is dot-notation to the value
210
211Route secrets remain in `.env` files with `${sops:...}` references. The SOPS backend is used for auth secrets (like `token_secret`) and provider lookups.
212
213### Using Secrets in Environment Files
214
215Reference SOPS secrets in your app's environment files:
216
217```bash
218# ~/.gordon/env/app_mydomain_com.env
219DATABASE_URL=postgresql://user:${sops:secrets.yaml:database.password}@postgres:5432/app
220API_KEY=${sops:secrets.yaml:api.key}
221```
222
223## File Organization
224
225Recommended structure:
226
227```
228project/
229├── .sops.yaml              # SOPS configuration
230├── secrets.yaml            # Encrypted secrets (commit this)
231├── secrets.dev.yaml        # Dev environment secrets
232├── secrets.prod.yaml       # Production secrets
233└── gordon.toml             # Gordon config (references secrets)
234```
235
236### Environment-Specific Rules
237
238```yaml
239# .sops.yaml
240creation_rules:
241  # Dev secrets - developer keys
242  - path_regex: \.dev\.yaml$
243    age: age1devkey...
244
245  # Prod secrets - production keys only
246  - path_regex: \.prod\.yaml$
247    age: age1prodkey...
248
249  # Default
250  - age: age1defaultkey...
251```
252
253## Running in Containers
254
255### Custom Dockerfile
256
257```dockerfile
258FROM ghcr.io/bnema/gordon:latest
259
260USER root
261RUN apk add --no-cache sops age
262USER gordon
263```
264
265### Mount Secrets and Keys
266
267```bash
268docker run -d \
269  --name gordon \
270  -p 80:8080 \
271  -p 5000:5000 \
272  -v /var/run/docker.sock:/var/run/docker.sock \
273  -v gordon-data:/data \
274  -v $(pwd)/secrets.yaml:/app/secrets.yaml:ro \
275  -v $(pwd)/.sops.yaml:/app/.sops.yaml:ro \
276  -v ~/.config/sops/age/keys.txt:/home/gordon/.config/sops/age/keys.txt:ro \
277  -e SOPS_AGE_KEY_FILE=/home/gordon/.config/sops/age/keys.txt \
278  -v $(pwd)/gordon.toml:/etc/gordon/gordon.toml:ro \
279  gordon-with-sops
280```
281
282### Docker Compose
283
284```yaml
285services:
286  gordon:
287    build:
288      context: .
289      dockerfile: Dockerfile.gordon
290    container_name: gordon
291    restart: unless-stopped
292    ports:
293      - "80:8080"
294      - "5000:5000"
295    volumes:
296      - /var/run/docker.sock:/var/run/docker.sock
297      - gordon-data:/data
298      - ./gordon.toml:/etc/gordon/gordon.toml:ro
299      - ./secrets.yaml:/app/secrets.yaml:ro
300      - ./.sops.yaml:/app/.sops.yaml:ro
301      - ~/.config/sops/age/keys.txt:/home/gordon/.config/sops/age/keys.txt:ro
302    environment:
303      - SOPS_AGE_KEY_FILE=/home/gordon/.config/sops/age/keys.txt
304      - GORDON_SECRETS_BACKEND=sops
305
306volumes:
307  gordon-data:
308```
309
310## Team Workflows
311
312### Adding Team Members
313
3141. Get their age public key
3152. Add to `.sops.yaml`:
316
317```yaml
318creation_rules:
319  - age: >-
320      age1yourkey...,
321      age1teammemberkey...
322```
323
3243. Re-encrypt existing files:
325
326```bash
327sops updatekeys secrets.yaml
328```
329
330### Key Rotation
331
3321. Generate new key
3332. Update `.sops.yaml`
3343. Re-encrypt all files:
335
336```bash
337sops updatekeys secrets.yaml
338sops updatekeys secrets.prod.yaml
339```
340
341### Git Workflow
342
343Encrypted files are safe to commit:
344
345```bash
346git add .sops.yaml secrets.yaml
347git commit -m "Add encrypted secrets"
348git push
349```
350
351Team members with valid keys can decrypt after pulling.
352
353## Troubleshooting
354
355### "failed to get the data key"
356
357Your key isn't in the recipients list or key file is missing:
358
359```bash
360# Check if age key is set
361echo $SOPS_AGE_KEY_FILE
362cat $SOPS_AGE_KEY_FILE
363
364# Check recipients in encrypted file
365grep -A5 "age:" secrets.yaml
366```
367
368### "sops: command not found"
369
370Install SOPS:
371
372```bash
373# Check if installed
374which sops
375
376# Install if missing (see Installation section)
377```
378
379### "could not decrypt data key"
380
381Wrong key or corrupted file:
382
383```bash
384# Verify your public key matches a recipient
385age-keygen -y ~/.config/sops/age/keys.txt
386# Compare output to recipients in .sops.yaml
387```
388
389### "no matching creation_rules"
390
391Create or fix `.sops.yaml`:
392
393```bash
394# Check if .sops.yaml exists
395cat .sops.yaml
396
397# Ensure path_regex matches your file
398```
399
400### Permission Denied in Container
401
402Ensure key file is readable:
403
404```bash
405docker exec gordon cat /home/gordon/.config/sops/age/keys.txt
406```
407
408Check mount permissions:
409
410```bash
411ls -la ~/.config/sops/age/keys.txt
412# Should be readable by container user
413```
414
415## Security Best Practices
416
4171. **Never commit unencrypted secrets**: Always use `sops edit`, never edit decrypted files directly
4182. **Protect age/GPG keys**: Store private keys securely, never commit them
4193. **Use separate keys per environment**: Production keys should be isolated
4204. **Rotate keys regularly**: Update keys and re-encrypt periodically
4215. **Audit access**: Track who has decryption keys for each environment
4226. **Use .sops.yaml**: Enforce encryption rules via configuration
423
424## Related
425
426- [Secrets Configuration](/docs/config/secrets.md)
427- [Authentication](/docs/config/auth.md)
428- [Running in Containers](/wiki/guides/running-in-container.md)
429- [Using Pass for Secrets](./secrets-pass.md)

secrets-sops.md RawHistoryFolder

Using SOPS for Secrets

What You'll Learn

Prerequisites

Why SOPS?

Installation

SOPS

age (Recommended)

GPG (Alternative)

Setup with age

1. Generate an age Key

2. Configure SOPS

3. Set age Key Location

Setup with GPG

1. Generate or Use Existing GPG Key

2. Configure SOPS

Creating Encrypted Secrets

Create secrets.yaml

View Encrypted File

Decrypt and View

Edit Existing File

Gordon Configuration

Enable SOPS Backend

Using Secrets in Environment Files

File Organization

Environment-Specific Rules

Running in Containers

Custom Dockerfile

Mount Secrets and Keys

Docker Compose

Team Workflows

Adding Team Members

Key Rotation

Git Workflow

Troubleshooting

"failed to get the data key"

"sops: command not found"

"could not decrypt data key"

"no matching creation_rules"

Permission Denied in Container

Security Best Practices

Related

secrets-sops.md Raw History Folder