v2.10.0 893d8fa

secure-vps-setup.md RawHistoryFolder

Secure VPS Setup

A production-ready VPS security configuration with Tailscale-only SSH access and CrowdSec intrusion prevention.

Overview

This guide configures three security layers:

Layer Protection
SSH Tailscale only (invisible to internet)
Firewall firewalld with DROP policy
IPS CrowdSec with community blocklists

Prerequisites

  • Fresh Ubuntu 24.04 VPS
  • Tailscale account
  • Initial SSH access via public IP (temporary)
  • Cloudflare account for HTTPS termination

Step 1: Initial System Hardening

Before anything else, enable automatic security updates and time sync:

# Automatic security updates
apt update && apt install -y unattended-upgrades
dpkg-reconfigure -plow unattended-upgrades

# Verify time sync (enabled by default on Ubuntu 24.04)
timedatectl set-ntp true
timedatectl status

# Basic kernel hardening
cat > /etc/sysctl.d/99-security.conf << 'EOF'
# SYN flood protection
net.ipv4.tcp_syncookies = 1

# Reverse path filtering (anti-spoofing)
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

# Disable ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv6.conf.all.accept_redirects = 0

# Ignore ICMP broadcasts
net.ipv4.icmp_echo_ignore_broadcasts = 1
EOF

sysctl --system

Step 2: Install Tailscale

Important: Use the official repository, NOT the Snap package (has AppArmor issues with Tailscale SSH on Ubuntu 24.04).

# Add official Tailscale repository
curl -fsSL https://pkgs.tailscale.com/stable/ubuntu/noble.noarmor.gpg | \
  tee /usr/share/keyrings/tailscale-archive-keyring.gpg >/dev/null
curl -fsSL https://pkgs.tailscale.com/stable/ubuntu/noble.tailscale-keyring.list | \
  tee /etc/apt/sources.list.d/tailscale.list

# Install
apt update
apt install -y tailscale

Step 3: Connect Tailscale with SSH

tailscale up --ssh

This outputs an authentication URL. Open it and authenticate with your Tailscale account.

Important: After connecting, go to https://login.tailscale.com/admin/machines and disable key expiry for this server to prevent lockouts.

Step 4: Configure firewalld

# Install firewalld
apt install -y firewalld

# Enable and start
systemctl enable --now firewalld

# Trust Tailscale interface (allows all traffic from your tailnet)
firewall-cmd --permanent --zone=trusted --add-interface=tailscale0

# Allow Tailscale UDP port (WireGuard)
firewall-cmd --permanent --add-port=41641/udp

# Allow web traffic (for Gordon reverse proxy)
firewall-cmd --permanent --add-service=http
firewall-cmd --permanent --add-service=https

# Apply changes
firewall-cmd --reload

Note: Gordon's registry is proxied through port 80/443 via gordon_domain. Port 5000 does NOT need to be exposed publicly.

Step 5: Verify Tailscale SSH Works

From another machine on your tailnet:

# Get the server's Tailscale hostname
tailscale status

# Connect from your local machine
ssh root@<tailscale-hostname>
# or
tailscale ssh root@<tailscale-hostname>

Step 6: Block Public SSH

Only proceed once Tailscale SSH is confirmed working.

# Remove SSH from public zone
firewall-cmd --permanent --remove-service=ssh

# Set default policy to DROP all unmatched traffic
firewall-cmd --permanent --zone=public --set-target=DROP

# Apply changes
firewall-cmd --reload

Verify public SSH is blocked:

# From another machine (not on tailnet)
ssh -o ConnectTimeout=5 root@<public-ip>  # Should timeout

Step 7: Install CrowdSec

# Add CrowdSec repo (NOT Ubuntu's outdated version)
curl -s https://install.crowdsec.net | bash

# Install CrowdSec
apt update
apt install -y crowdsec

Step 8: Install nftables Bouncer

Ubuntu 24.04 uses nftables by default:

apt install -y crowdsec-firewall-bouncer-nftables
systemctl enable --now crowdsec-firewall-bouncer

Step 9: Install Security Collections

# HTTP protection
cscli collections install crowdsecurity/http-cve
cscli collections install crowdsecurity/base-http-scenarios

# Reload to apply
systemctl reload crowdsec

Step 10: Whitelist Tailscale IPs

Prevent CrowdSec from blocking Tailscale traffic:

cat > /etc/crowdsec/parsers/s02-enrich/tailscale-whitelist.yaml << 'EOF'
name: tailscale-whitelist
description: "Whitelist Tailscale CGNAT range"
whitelist:
  reason: "Tailscale internal traffic"
  cidr:
    - "100.64.0.0/10"
EOF

systemctl reload crowdsec

Step 11: Enroll in CrowdSec Console

This enables community blocklists - IPs blocked across all CrowdSec users globally (15,000+ malicious IPs).

  1. Create account at https://app.crowdsec.net
  2. Go to Security Engines → Add Security Engine
  3. Copy enrollment key and run:
cscli console enroll <your-enrollment-key>
  1. Accept the enrollment in the CrowdSec console
  2. Enable console management:
systemctl restart crowdsec
cscli console enable console_management
systemctl reload crowdsec

Verification

# Firewall status
firewall-cmd --state
firewall-cmd --list-all
firewall-cmd --list-all --zone=trusted

# CrowdSec status
cscli metrics
cscli bouncers list
cscli console status

# Verify nftables has CrowdSec rules
nft list ruleset | grep -A5 'crowdsec'

# View blocked IPs count
cscli decisions list | wc -l

Port Summary

Port Protocol Status Purpose
22 TCP Closed SSH (use Tailscale instead)
80 TCP Open HTTP (Gordon proxy)
443 TCP Open HTTPS (Gordon proxy via Cloudflare)
5000 TCP Closed Registry (proxied through 80/443, containers bind to 127.0.0.1)
41641 UDP Open Tailscale WireGuard

Security Note: All managed containers bind to 127.0.0.1 by default, preventing direct access to their ports. The only entry point to your applications is Gordon's reverse proxy on ports 80/443, where authentication and rate limiting are enforced.


Troubleshooting

Can't SSH via Tailscale

Check for ACL issues:

journalctl -u tailscaled --since "10 minutes ago" | grep -i "rejected\|acl"

If you see rejected due to acl, check your Tailscale ACLs at https://login.tailscale.com/admin/acls

Locked out completely

Use your VPS provider's console access (e.g., Hetzner Cloud Console):

firewall-cmd --add-service=ssh  # Temporarily re-enable public SSH

CrowdSec not blocking

Verify bouncer is running:

systemctl status crowdsec-firewall-bouncer
cscli bouncers list  # Should show "Valid"

Check what's being blocked

cscli decisions list
cscli alerts list

Next Steps

References

source
1# Secure VPS Setup
2
3A production-ready VPS security configuration with Tailscale-only SSH access and CrowdSec intrusion prevention.
4
5## Overview
6
7This guide configures three security layers:
8
9| Layer | Protection |
10|-------|------------|
11| SSH | Tailscale only (invisible to internet) |
12| Firewall | firewalld with DROP policy |
13| IPS | CrowdSec with community blocklists |
14
15## Prerequisites
16
17- Fresh Ubuntu 24.04 VPS
18- Tailscale account
19- Initial SSH access via public IP (temporary)
20- Cloudflare account for HTTPS termination
21
22---
23
24## Step 1: Initial System Hardening
25
26Before anything else, enable automatic security updates and time sync:
27
28```bash
29# Automatic security updates
30apt update && apt install -y unattended-upgrades
31dpkg-reconfigure -plow unattended-upgrades
32
33# Verify time sync (enabled by default on Ubuntu 24.04)
34timedatectl set-ntp true
35timedatectl status
36
37# Basic kernel hardening
38cat > /etc/sysctl.d/99-security.conf << 'EOF'
39# SYN flood protection
40net.ipv4.tcp_syncookies = 1
41
42# Reverse path filtering (anti-spoofing)
43net.ipv4.conf.all.rp_filter = 1
44net.ipv4.conf.default.rp_filter = 1
45
46# Disable ICMP redirects
47net.ipv4.conf.all.accept_redirects = 0
48net.ipv4.conf.all.send_redirects = 0
49net.ipv6.conf.all.accept_redirects = 0
50
51# Ignore ICMP broadcasts
52net.ipv4.icmp_echo_ignore_broadcasts = 1
53EOF
54
55sysctl --system
56```
57
58## Step 2: Install Tailscale
59
60> **Important**: Use the official repository, NOT the Snap package (has AppArmor issues with Tailscale SSH on Ubuntu 24.04).
61
62```bash
63# Add official Tailscale repository
64curl -fsSL https://pkgs.tailscale.com/stable/ubuntu/noble.noarmor.gpg | \
65  tee /usr/share/keyrings/tailscale-archive-keyring.gpg >/dev/null
66curl -fsSL https://pkgs.tailscale.com/stable/ubuntu/noble.tailscale-keyring.list | \
67  tee /etc/apt/sources.list.d/tailscale.list
68
69# Install
70apt update
71apt install -y tailscale
72```
73
74## Step 3: Connect Tailscale with SSH
75
76```bash
77tailscale up --ssh
78```
79
80This outputs an authentication URL. Open it and authenticate with your Tailscale account.
81
82**Important**: After connecting, go to https://login.tailscale.com/admin/machines and **disable key expiry** for this server to prevent lockouts.
83
84## Step 4: Configure firewalld
85
86```bash
87# Install firewalld
88apt install -y firewalld
89
90# Enable and start
91systemctl enable --now firewalld
92
93# Trust Tailscale interface (allows all traffic from your tailnet)
94firewall-cmd --permanent --zone=trusted --add-interface=tailscale0
95
96# Allow Tailscale UDP port (WireGuard)
97firewall-cmd --permanent --add-port=41641/udp
98
99# Allow web traffic (for Gordon reverse proxy)
100firewall-cmd --permanent --add-service=http
101firewall-cmd --permanent --add-service=https
102
103# Apply changes
104firewall-cmd --reload
105```
106
107> **Note**: Gordon's registry is proxied through port 80/443 via `gordon_domain`. Port 5000 does NOT need to be exposed publicly.
108
109## Step 5: Verify Tailscale SSH Works
110
111From another machine on your tailnet:
112
113```bash
114# Get the server's Tailscale hostname
115tailscale status
116
117# Connect from your local machine
118ssh root@<tailscale-hostname>
119# or
120tailscale ssh root@<tailscale-hostname>
121```
122
123## Step 6: Block Public SSH
124
125**Only proceed once Tailscale SSH is confirmed working.**
126
127```bash
128# Remove SSH from public zone
129firewall-cmd --permanent --remove-service=ssh
130
131# Set default policy to DROP all unmatched traffic
132firewall-cmd --permanent --zone=public --set-target=DROP
133
134# Apply changes
135firewall-cmd --reload
136```
137
138Verify public SSH is blocked:
139
140```bash
141# From another machine (not on tailnet)
142ssh -o ConnectTimeout=5 root@<public-ip>  # Should timeout
143```
144
145---
146
147## Step 7: Install CrowdSec
148
149```bash
150# Add CrowdSec repo (NOT Ubuntu's outdated version)
151curl -s https://install.crowdsec.net | bash
152
153# Install CrowdSec
154apt update
155apt install -y crowdsec
156```
157
158## Step 8: Install nftables Bouncer
159
160Ubuntu 24.04 uses nftables by default:
161
162```bash
163apt install -y crowdsec-firewall-bouncer-nftables
164systemctl enable --now crowdsec-firewall-bouncer
165```
166
167## Step 9: Install Security Collections
168
169```bash
170# HTTP protection
171cscli collections install crowdsecurity/http-cve
172cscli collections install crowdsecurity/base-http-scenarios
173
174# Reload to apply
175systemctl reload crowdsec
176```
177
178## Step 10: Whitelist Tailscale IPs
179
180Prevent CrowdSec from blocking Tailscale traffic:
181
182```bash
183cat > /etc/crowdsec/parsers/s02-enrich/tailscale-whitelist.yaml << 'EOF'
184name: tailscale-whitelist
185description: "Whitelist Tailscale CGNAT range"
186whitelist:
187  reason: "Tailscale internal traffic"
188  cidr:
189    - "100.64.0.0/10"
190EOF
191
192systemctl reload crowdsec
193```
194
195## Step 11: Enroll in CrowdSec Console
196
197This enables community blocklists - IPs blocked across all CrowdSec users globally (15,000+ malicious IPs).
198
1991. Create account at https://app.crowdsec.net
2002. Go to **Security Engines** → **Add Security Engine**
2013. Copy enrollment key and run:
202
203```bash
204cscli console enroll <your-enrollment-key>
205```
206
2074. Accept the enrollment in the CrowdSec console
2085. Enable console management:
209
210```bash
211systemctl restart crowdsec
212cscli console enable console_management
213systemctl reload crowdsec
214```
215
216---
217
218## Verification
219
220```bash
221# Firewall status
222firewall-cmd --state
223firewall-cmd --list-all
224firewall-cmd --list-all --zone=trusted
225
226# CrowdSec status
227cscli metrics
228cscli bouncers list
229cscli console status
230
231# Verify nftables has CrowdSec rules
232nft list ruleset | grep -A5 'crowdsec'
233
234# View blocked IPs count
235cscli decisions list | wc -l
236```
237
238---
239
240## Port Summary
241
242| Port | Protocol | Status | Purpose |
243|------|----------|--------|---------|
244| 22 | TCP | Closed | SSH (use Tailscale instead) |
245| 80 | TCP | Open | HTTP (Gordon proxy) |
246| 443 | TCP | Open | HTTPS (Gordon proxy via Cloudflare) |
247| 5000 | TCP | Closed | Registry (proxied through 80/443, containers bind to 127.0.0.1) |
248| 41641 | UDP | Open | Tailscale WireGuard |
249
250> **Security Note:** All managed containers bind to `127.0.0.1` by default, preventing direct access to their ports. The only entry point to your applications is Gordon's reverse proxy on ports 80/443, where authentication and rate limiting are enforced.
251
252---
253
254## Troubleshooting
255
256### Can't SSH via Tailscale
257
258Check for ACL issues:
259
260```bash
261journalctl -u tailscaled --since "10 minutes ago" | grep -i "rejected\|acl"
262```
263
264If you see `rejected due to acl`, check your Tailscale ACLs at https://login.tailscale.com/admin/acls
265
266### Locked out completely
267
268Use your VPS provider's console access (e.g., Hetzner Cloud Console):
269
270```bash
271firewall-cmd --add-service=ssh  # Temporarily re-enable public SSH
272```
273
274### CrowdSec not blocking
275
276Verify bouncer is running:
277
278```bash
279systemctl status crowdsec-firewall-bouncer
280cscli bouncers list  # Should show "Valid"
281```
282
283### Check what's being blocked
284
285```bash
286cscli decisions list
287cscli alerts list
288```
289
290---
291
292## Next Steps
293
294- [Install Gordon](/docs/getting-started.md)
295- [Podman Rootless Setup](./podman-rootless.md) - Enhanced container isolation
296- [Cloudflare DNS & Proxy Setup](#) - HTTPS termination (coming soon)
297
298## References
299
300- [Tailscale SSH Documentation](https://tailscale.com/kb/1193/tailscale-ssh)
301- [firewalld Documentation](https://firewalld.org/documentation/)
302- [CrowdSec Documentation](https://docs.crowdsec.net/)
303- [CrowdSec Console](https://app.crowdsec.net)

Keyboard shortcuts

Single-key shortcuts are on. ctrl k and esc work whatever this says.

ctrl k :
go to a section, branch, tag, file or repository
?
this list
j k
move the row cursor down / up
enter
open the row under the cursor; on Files, a directory opens or closes in place
↑ ↓ ← → home end
in the file tree (focus on a row): previous / next row, close / open a directory or go to its parent / first entry, first / last row. They never open a file
l h
on Files: open a directory or a file / close a directory or go up to its parent, then to the parent page. Elsewhere: into the side panel (on a merge request, its files) / back to the content; there j k move between its controls
g g G
first / last row
/
filter the list; on Files, find a file anywhere in the repository (esc clears, enter opens the best match)
b
switch branch or tag
g s g f g c g b g m g i g r g a
tabs: summary, files, commits, branches & tags, merge requests, issues, releases, actions (when available)
g h
home
w
wide mode: use the whole width of the screen, or not (remembered)
y
copy a permalink pinned to the commit
d u x
diff: split / unified / collapse or expand all
e E
diff: the whole file in view / every file, with the lines the hunks hide
n
merge request: next note in the diff
esc
close a dialog, leave a field

Branches 7

Tags 106

↑↓ moveenter openesc close