secrets-sops.md Raw History Folder

Using SOPS for Secrets

Set up Mozilla SOPS as Gordon's secrets backend for encrypted file-based secret storage.

What You'll Learn

Installing SOPS and age (or GPG)
Creating encrypted secrets files
Configuring Gordon to use SOPS
Using secrets in environment files
Team workflows and key rotation

Prerequisites

Linux or macOS system
Basic understanding of encryption concepts

Why SOPS?

SOPS (Secrets OPerationS) encrypts specific values in YAML/JSON files while keeping keys readable:

Git-friendly: Encrypted files can be version controlled
Multiple backends: age, GPG, AWS KMS, GCP KMS, Azure Key Vault
Selective encryption: Only values are encrypted, keys remain visible
Team support: Multiple recipients can decrypt the same file

Installation

SOPS

# Ubuntu/Debian
curl -LO https://github.com/getsops/sops/releases/download/v3.9.0/sops-v3.9.0.linux.amd64
sudo mv sops-v3.9.0.linux.amd64 /usr/local/bin/sops
sudo chmod +x /usr/local/bin/sops

# macOS
brew install sops

# Arch Linux
sudo pacman -S sops

age (Recommended)

age is simpler than GPG and recommended for new setups:

# Ubuntu/Debian
sudo apt install age

# macOS
brew install age

# Arch Linux
sudo pacman -S age

GPG (Alternative)

# Ubuntu/Debian
sudo apt install gnupg

# macOS
brew install gnupg

Setup with age

1. Generate an age Key

# Generate key pair
age-keygen -o ~/.config/sops/age/keys.txt

# Output shows your public key:
# Public key: age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Note your public key - you'll need it for encryption.

2. Configure SOPS

Create .sops.yaml in your project or home directory:

creation_rules:
  - age: age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Or for multiple recipients:

creation_rules:
  - age: >-
      age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,
      age1yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

3. Set age Key Location

# Add to ~/.bashrc or ~/.zshrc
export SOPS_AGE_KEY_FILE=~/.config/sops/age/keys.txt

Setup with GPG

1. Generate or Use Existing GPG Key

# Generate new key
gpg --gen-key

# List keys to get fingerprint
gpg --list-keys

2. Configure SOPS

creation_rules:
  - pgp: ABCD1234EFGH5678...  # Your GPG fingerprint

Creating Encrypted Secrets

Create secrets.yaml

# Create and encrypt in one step
sops edit secrets.yaml

This opens your editor. Add secrets in YAML format:

auth:
  password_hash: "$2y$10$..."
  token_secret: "your-random-32-char-string-here"
database:
  password: "db-password-here"
api:
  key: "api-key-here"

Save and exit. SOPS encrypts the values automatically.

View Encrypted File

cat secrets.yaml

Output shows encrypted values:

auth:
    password_hash: ENC[AES256_GCM,data:...,iv:...,tag:...]
    token_secret: ENC[AES256_GCM,data:...,iv:...,tag:...]
database:
    password: ENC[AES256_GCM,data:...,iv:...,tag:...]
api:
    key: ENC[AES256_GCM,data:...,iv:...,tag:...]
sops:
    age:
        - recipient: age1xxx...
          enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
            ...
            -----END AGE ENCRYPTED FILE-----
    lastmodified: "2024-01-15T12:00:00Z"
    version: 3.9.0

Decrypt and View

# View entire file
sops decrypt secrets.yaml

# Extract specific value
sops decrypt --extract '["registry"]["token_secret"]' secrets.yaml

Edit Existing File

sops edit secrets.yaml

Gordon Configuration

Enable SOPS Backend

[auth]
enabled = true
secrets_backend = "sops"
token_secret = "secrets.yaml:auth.token_secret"
# Optional: enable password auth for interactive login
# username = "admin"
# password_hash = "secrets.yaml:auth.password_hash"

The path format is file:key.path where:

file is the SOPS-encrypted file path
key.path is dot-notation to the value

Using Secrets in Environment Files

Reference SOPS secrets in your app's environment files:

# ~/.gordon/env/app_mydomain_com.env
DATABASE_URL=postgresql://user:${sops:secrets.yaml:database.password}@postgres:5432/app
API_KEY=${sops:secrets.yaml:api.key}

File Organization

Recommended structure:

project/
├── .sops.yaml              # SOPS configuration
├── secrets.yaml            # Encrypted secrets (commit this)
├── secrets.dev.yaml        # Dev environment secrets
├── secrets.prod.yaml       # Production secrets
└── gordon.toml             # Gordon config (references secrets)

Environment-Specific Rules

# .sops.yaml
creation_rules:
  # Dev secrets - developer keys
  - path_regex: \.dev\.yaml$
    age: age1devkey...

  # Prod secrets - production keys only
  - path_regex: \.prod\.yaml$
    age: age1prodkey...

  # Default
  - age: age1defaultkey...

Running in Containers

Custom Dockerfile

FROM ghcr.io/bnema/gordon:latest

USER root
RUN apk add --no-cache sops age
USER gordon

Mount Secrets and Keys

docker run -d \
  --name gordon \
  -p 80:8080 \
  -p 5000:5000 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v gordon-data:/data \
  -v $(pwd)/secrets.yaml:/app/secrets.yaml:ro \
  -v $(pwd)/.sops.yaml:/app/.sops.yaml:ro \
  -v ~/.config/sops/age/keys.txt:/home/gordon/.config/sops/age/keys.txt:ro \
  -e SOPS_AGE_KEY_FILE=/home/gordon/.config/sops/age/keys.txt \
  -v $(pwd)/gordon.toml:/etc/gordon/gordon.toml:ro \
  gordon-with-sops

Docker Compose

services:
  gordon:
    build:
      context: .
      dockerfile: Dockerfile.gordon
    container_name: gordon
    restart: unless-stopped
    ports:
      - "80:8080"
      - "5000:5000"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - gordon-data:/data
      - ./gordon.toml:/etc/gordon/gordon.toml:ro
      - ./secrets.yaml:/app/secrets.yaml:ro
      - ./.sops.yaml:/app/.sops.yaml:ro
      - ~/.config/sops/age/keys.txt:/home/gordon/.config/sops/age/keys.txt:ro
    environment:
      - SOPS_AGE_KEY_FILE=/home/gordon/.config/sops/age/keys.txt
      - GORDON_SECRETS_BACKEND=sops

volumes:
  gordon-data:

Team Workflows

Adding Team Members

Get their age public key
Add to .sops.yaml:

creation_rules:
  - age: >-
      age1yourkey...,
      age1teammemberkey...

Re-encrypt existing files:

sops updatekeys secrets.yaml

Key Rotation

Generate new key
Update .sops.yaml
Re-encrypt all files:

sops updatekeys secrets.yaml
sops updatekeys secrets.prod.yaml

Git Workflow

Encrypted files are safe to commit:

git add .sops.yaml secrets.yaml
git commit -m "Add encrypted secrets"
git push

Team members with valid keys can decrypt after pulling.

Troubleshooting

"failed to get the data key"

Your key isn't in the recipients list or key file is missing:

# Check if age key is set
echo $SOPS_AGE_KEY_FILE
cat $SOPS_AGE_KEY_FILE

# Check recipients in encrypted file
grep -A5 "age:" secrets.yaml

"sops: command not found"

Install SOPS:

# Check if installed
which sops

# Install if missing (see Installation section)

"could not decrypt data key"

Wrong key or corrupted file:

# Verify your public key matches a recipient
age-keygen -y ~/.config/sops/age/keys.txt
# Compare output to recipients in .sops.yaml

"no matching creation_rules"

Create or fix .sops.yaml:

# Check if .sops.yaml exists
cat .sops.yaml

# Ensure path_regex matches your file

Permission Denied in Container

Ensure key file is readable:

docker exec gordon cat /home/gordon/.config/sops/age/keys.txt

Check mount permissions:

ls -la ~/.config/sops/age/keys.txt
# Should be readable by container user

Security Best Practices

Never commit unencrypted secrets: Always use sops edit, never edit decrypted files directly
Protect age/GPG keys: Store private keys securely, never commit them
Use separate keys per environment: Production keys should be isolated
Rotate keys regularly: Update keys and re-encrypt periodically
Audit access: Track who has decryption keys for each environment
Use .sops.yaml: Enforce encryption rules via configuration

source

1# Using SOPS for Secrets
2
3Set up Mozilla SOPS as Gordon's secrets backend for encrypted file-based secret storage.
4
5## What You'll Learn
6
7- Installing SOPS and age (or GPG)
8- Creating encrypted secrets files
9- Configuring Gordon to use SOPS
10- Using secrets in environment files
11- Team workflows and key rotation
12
13## Prerequisites
14
15- Linux or macOS system
16- Basic understanding of encryption concepts
17
18## Why SOPS?
19
20SOPS (Secrets OPerationS) encrypts specific values in YAML/JSON files while keeping keys readable:
21
22- **Git-friendly**: Encrypted files can be version controlled
23- **Multiple backends**: age, GPG, AWS KMS, GCP KMS, Azure Key Vault
24- **Selective encryption**: Only values are encrypted, keys remain visible
25- **Team support**: Multiple recipients can decrypt the same file
26
27## Installation
28
29### SOPS
30
31```bash
32# Ubuntu/Debian
33curl -LO https://github.com/getsops/sops/releases/download/v3.9.0/sops-v3.9.0.linux.amd64
34sudo mv sops-v3.9.0.linux.amd64 /usr/local/bin/sops
35sudo chmod +x /usr/local/bin/sops
36
37# macOS
38brew install sops
39
40# Arch Linux
41sudo pacman -S sops
42```
43
44### age (Recommended)
45
46age is simpler than GPG and recommended for new setups:
47
48```bash
49# Ubuntu/Debian
50sudo apt install age
51
52# macOS
53brew install age
54
55# Arch Linux
56sudo pacman -S age
57```
58
59### GPG (Alternative)
60
61```bash
62# Ubuntu/Debian
63sudo apt install gnupg
64
65# macOS
66brew install gnupg
67```
68
69## Setup with age
70
71### 1. Generate an age Key
72
73```bash
74# Generate key pair
75age-keygen -o ~/.config/sops/age/keys.txt
76
77# Output shows your public key:
78# Public key: age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
79```
80
81Note your public key - you'll need it for encryption.
82
83### 2. Configure SOPS
84
85Create `.sops.yaml` in your project or home directory:
86
87```yaml
88creation_rules:
89  - age: age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
90```
91
92Or for multiple recipients:
93
94```yaml
95creation_rules:
96  - age: >-
97      age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,
98      age1yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
99```
100
101### 3. Set age Key Location
102
103```bash
104# Add to ~/.bashrc or ~/.zshrc
105export SOPS_AGE_KEY_FILE=~/.config/sops/age/keys.txt
106```
107
108## Setup with GPG
109
110### 1. Generate or Use Existing GPG Key
111
112```bash
113# Generate new key
114gpg --gen-key
115
116# List keys to get fingerprint
117gpg --list-keys
118```
119
120### 2. Configure SOPS
121
122```yaml
123creation_rules:
124  - pgp: ABCD1234EFGH5678...  # Your GPG fingerprint
125```
126
127## Creating Encrypted Secrets
128
129### Create secrets.yaml
130
131```bash
132# Create and encrypt in one step
133sops edit secrets.yaml
134```
135
136This opens your editor. Add secrets in YAML format:
137
138```yaml
139auth:
140  password_hash: "$2y$10$..."
141  token_secret: "your-random-32-char-string-here"
142database:
143  password: "db-password-here"
144api:
145  key: "api-key-here"
146```
147
148Save and exit. SOPS encrypts the values automatically.
149
150### View Encrypted File
151
152```bash
153cat secrets.yaml
154```
155
156Output shows encrypted values:
157
158```yaml
159auth:
160    password_hash: ENC[AES256_GCM,data:...,iv:...,tag:...]
161    token_secret: ENC[AES256_GCM,data:...,iv:...,tag:...]
162database:
163    password: ENC[AES256_GCM,data:...,iv:...,tag:...]
164api:
165    key: ENC[AES256_GCM,data:...,iv:...,tag:...]
166sops:
167    age:
168        - recipient: age1xxx...
169          enc: |
170            -----BEGIN AGE ENCRYPTED FILE-----
171            ...
172            -----END AGE ENCRYPTED FILE-----
173    lastmodified: "2024-01-15T12:00:00Z"
174    version: 3.9.0
175```
176
177### Decrypt and View
178
179```bash
180# View entire file
181sops decrypt secrets.yaml
182
183# Extract specific value
184sops decrypt --extract '["registry"]["token_secret"]' secrets.yaml
185```
186
187### Edit Existing File
188
189```bash
190sops edit secrets.yaml
191```
192
193## Gordon Configuration
194
195### Enable SOPS Backend
196
197```toml
198[auth]
199enabled = true
200secrets_backend = "sops"
201token_secret = "secrets.yaml:auth.token_secret"
202# Optional: enable password auth for interactive login
203# username = "admin"
204# password_hash = "secrets.yaml:auth.password_hash"
205```
206
207The path format is `file:key.path` where:
208- `file` is the SOPS-encrypted file path
209- `key.path` is dot-notation to the value
210
211### Using Secrets in Environment Files
212
213Reference SOPS secrets in your app's environment files:
214
215```bash
216# ~/.gordon/env/app_mydomain_com.env
217DATABASE_URL=postgresql://user:${sops:secrets.yaml:database.password}@postgres:5432/app
218API_KEY=${sops:secrets.yaml:api.key}
219```
220
221## File Organization
222
223Recommended structure:
224
225```
226project/
227├── .sops.yaml              # SOPS configuration
228├── secrets.yaml            # Encrypted secrets (commit this)
229├── secrets.dev.yaml        # Dev environment secrets
230├── secrets.prod.yaml       # Production secrets
231└── gordon.toml             # Gordon config (references secrets)
232```
233
234### Environment-Specific Rules
235
236```yaml
237# .sops.yaml
238creation_rules:
239  # Dev secrets - developer keys
240  - path_regex: \.dev\.yaml$
241    age: age1devkey...
242
243  # Prod secrets - production keys only
244  - path_regex: \.prod\.yaml$
245    age: age1prodkey...
246
247  # Default
248  - age: age1defaultkey...
249```
250
251## Running in Containers
252
253### Custom Dockerfile
254
255```dockerfile
256FROM ghcr.io/bnema/gordon:latest
257
258USER root
259RUN apk add --no-cache sops age
260USER gordon
261```
262
263### Mount Secrets and Keys
264
265```bash
266docker run -d \
267  --name gordon \
268  -p 80:8080 \
269  -p 5000:5000 \
270  -v /var/run/docker.sock:/var/run/docker.sock \
271  -v gordon-data:/data \
272  -v $(pwd)/secrets.yaml:/app/secrets.yaml:ro \
273  -v $(pwd)/.sops.yaml:/app/.sops.yaml:ro \
274  -v ~/.config/sops/age/keys.txt:/home/gordon/.config/sops/age/keys.txt:ro \
275  -e SOPS_AGE_KEY_FILE=/home/gordon/.config/sops/age/keys.txt \
276  -v $(pwd)/gordon.toml:/etc/gordon/gordon.toml:ro \
277  gordon-with-sops
278```
279
280### Docker Compose
281
282```yaml
283services:
284  gordon:
285    build:
286      context: .
287      dockerfile: Dockerfile.gordon
288    container_name: gordon
289    restart: unless-stopped
290    ports:
291      - "80:8080"
292      - "5000:5000"
293    volumes:
294      - /var/run/docker.sock:/var/run/docker.sock
295      - gordon-data:/data
296      - ./gordon.toml:/etc/gordon/gordon.toml:ro
297      - ./secrets.yaml:/app/secrets.yaml:ro
298      - ./.sops.yaml:/app/.sops.yaml:ro
299      - ~/.config/sops/age/keys.txt:/home/gordon/.config/sops/age/keys.txt:ro
300    environment:
301      - SOPS_AGE_KEY_FILE=/home/gordon/.config/sops/age/keys.txt
302      - GORDON_SECRETS_BACKEND=sops
303
304volumes:
305  gordon-data:
306```
307
308## Team Workflows
309
310### Adding Team Members
311
3121. Get their age public key
3132. Add to `.sops.yaml`:
314
315```yaml
316creation_rules:
317  - age: >-
318      age1yourkey...,
319      age1teammemberkey...
320```
321
3223. Re-encrypt existing files:
323
324```bash
325sops updatekeys secrets.yaml
326```
327
328### Key Rotation
329
3301. Generate new key
3312. Update `.sops.yaml`
3323. Re-encrypt all files:
333
334```bash
335sops updatekeys secrets.yaml
336sops updatekeys secrets.prod.yaml
337```
338
339### Git Workflow
340
341Encrypted files are safe to commit:
342
343```bash
344git add .sops.yaml secrets.yaml
345git commit -m "Add encrypted secrets"
346git push
347```
348
349Team members with valid keys can decrypt after pulling.
350
351## Troubleshooting
352
353### "failed to get the data key"
354
355Your key isn't in the recipients list or key file is missing:
356
357```bash
358# Check if age key is set
359echo $SOPS_AGE_KEY_FILE
360cat $SOPS_AGE_KEY_FILE
361
362# Check recipients in encrypted file
363grep -A5 "age:" secrets.yaml
364```
365
366### "sops: command not found"
367
368Install SOPS:
369
370```bash
371# Check if installed
372which sops
373
374# Install if missing (see Installation section)
375```
376
377### "could not decrypt data key"
378
379Wrong key or corrupted file:
380
381```bash
382# Verify your public key matches a recipient
383age-keygen -y ~/.config/sops/age/keys.txt
384# Compare output to recipients in .sops.yaml
385```
386
387### "no matching creation_rules"
388
389Create or fix `.sops.yaml`:
390
391```bash
392# Check if .sops.yaml exists
393cat .sops.yaml
394
395# Ensure path_regex matches your file
396```
397
398### Permission Denied in Container
399
400Ensure key file is readable:
401
402```bash
403docker exec gordon cat /home/gordon/.config/sops/age/keys.txt
404```
405
406Check mount permissions:
407
408```bash
409ls -la ~/.config/sops/age/keys.txt
410# Should be readable by container user
411```
412
413## Security Best Practices
414
4151. **Never commit unencrypted secrets**: Always use `sops edit`, never edit decrypted files directly
4162. **Protect age/GPG keys**: Store private keys securely, never commit them
4173. **Use separate keys per environment**: Production keys should be isolated
4184. **Rotate keys regularly**: Update keys and re-encrypt periodically
4195. **Audit access**: Track who has decryption keys for each environment
4206. **Use .sops.yaml**: Enforce encryption rules via configuration
421
422## Related
423
424- [Secrets Configuration](/docs/config/secrets.md)
425- [Authentication](/docs/config/auth.md)
426- [Running in Containers](/wiki/guides/running-in-container.md)
427- [Using Pass for Secrets](./secrets-pass.md)

secrets-sops.md RawHistoryFolder

Using SOPS for Secrets

What You'll Learn

Prerequisites

Why SOPS?

Installation

SOPS

age (Recommended)

GPG (Alternative)

Setup with age

1. Generate an age Key

2. Configure SOPS

3. Set age Key Location

Setup with GPG

1. Generate or Use Existing GPG Key

2. Configure SOPS

Creating Encrypted Secrets

Create secrets.yaml

View Encrypted File

Decrypt and View

Edit Existing File

Gordon Configuration

Enable SOPS Backend

Using Secrets in Environment Files

File Organization

Environment-Specific Rules

Running in Containers

Custom Dockerfile

Mount Secrets and Keys

Docker Compose

Team Workflows

Adding Team Members

Key Rotation

Git Workflow

Troubleshooting

"failed to get the data key"

"sops: command not found"

"could not decrypt data key"

"no matching creation_rules"

Permission Denied in Container

Security Best Practices

Related

secrets-sops.md Raw History Folder