detached 8d0d173

feat: Implement ACME certificate management

- Refactored token initialization to remove unnecessary logging.
- Updated database creation function to support transactions.
- Added a comprehensive ACME certificate management system, including user registration, certificate obtaining, and renewal.
- Enhanced database schema to accommodate new fields for ACME configuration in the Domain struct.
- Introduced a new file for ACME-related operations and integrated it with existing proxy logic.
- Implemented database update logic to ensure schema consistency with struct definitions.
- Added detailed logging for better traceability of ACME operations and database interactions.
commit
8d0d1736a2ed5c3df07fe57d5ca0f7d400287dc5
author
bamen · · 2025-03-29 08:05 UTC
parent
da25f67
signature
unsigned
26 files · +3530 -3073
Unified
UnifiedSplit
M go.mod +164 -4
@@ -13,6 +13,7 @@ require (
1313 github.com/docker/docker v28.0.1+incompatible
1414 github.com/docker/go-connections v0.5.0
1515 github.com/fatih/color v1.18.0
16+ github.com/go-acme/lego/v4 v4.22.2
1617 github.com/gorilla/sessions v1.4.0
1718 github.com/joho/godotenv v1.5.1
1819 github.com/labstack/echo-contrib v0.17.2
@@ -25,21 +26,180 @@ require (
2526 )
2627
2728 require (
29+ cloud.google.com/go/auth v0.13.0 // indirect
30+ cloud.google.com/go/auth/oauth2adapt v0.2.6 // indirect
31+ cloud.google.com/go/compute/metadata v0.6.0 // indirect
32+ github.com/AdamSLevy/jsonrpc2/v14 v14.1.0 // indirect
33+ github.com/Azure/azure-sdk-for-go v68.0.0+incompatible // indirect
34+ github.com/Azure/azure-sdk-for-go/sdk/azcore v1.16.0 // indirect
35+ github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.0 // indirect
36+ github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 // indirect
37+ github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/dns/armdns v1.2.0 // indirect
38+ github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/privatedns/armprivatedns v1.3.0 // indirect
39+ github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/resourcegraph/armresourcegraph v0.9.0 // indirect
40+ github.com/Azure/go-autorest v14.2.0+incompatible // indirect
41+ github.com/Azure/go-autorest/autorest v0.11.29 // indirect
42+ github.com/Azure/go-autorest/autorest/adal v0.9.22 // indirect
43+ github.com/Azure/go-autorest/autorest/azure/auth v0.5.13 // indirect
44+ github.com/Azure/go-autorest/autorest/azure/cli v0.4.6 // indirect
45+ github.com/Azure/go-autorest/autorest/date v0.3.0 // indirect
46+ github.com/Azure/go-autorest/autorest/to v0.4.0 // indirect
47+ github.com/Azure/go-autorest/logger v0.2.1 // indirect
48+ github.com/Azure/go-autorest/tracing v0.6.0 // indirect
49+ github.com/AzureAD/microsoft-authentication-library-for-go v1.2.2 // indirect
50+ github.com/OpenDNS/vegadns2client v0.0.0-20180418235048-a3fa4a771d87 // indirect
2851 github.com/a-h/parse v0.0.0-20250122154542-74294addb73e // indirect
52+ github.com/akamai/AkamaiOPEN-edgegrid-golang v1.2.2 // indirect
53+ github.com/aliyun/alibaba-cloud-sdk-go v1.63.72 // indirect
2954 github.com/andybalholm/brotli v1.1.0 // indirect
55+ github.com/aws/aws-sdk-go-v2 v1.32.7 // indirect
56+ github.com/aws/aws-sdk-go-v2/config v1.28.7 // indirect
57+ github.com/aws/aws-sdk-go-v2/credentials v1.17.48 // indirect
58+ github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.22 // indirect
59+ github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.26 // indirect
60+ github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.26 // indirect
61+ github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 // indirect
62+ github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.1 // indirect
63+ github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.7 // indirect
64+ github.com/aws/aws-sdk-go-v2/service/lightsail v1.42.8 // indirect
65+ github.com/aws/aws-sdk-go-v2/service/route53 v1.46.4 // indirect
66+ github.com/aws/aws-sdk-go-v2/service/sso v1.24.8 // indirect
67+ github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.7 // indirect
68+ github.com/aws/aws-sdk-go-v2/service/sts v1.33.3 // indirect
69+ github.com/aws/smithy-go v1.22.1 // indirect
70+ github.com/benbjohnson/clock v1.3.0 // indirect
71+ github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc // indirect
3072 github.com/cenkalti/backoff/v4 v4.3.0 // indirect
3173 github.com/cespare/xxhash/v2 v2.3.0 // indirect
74+ github.com/civo/civogo v0.3.11 // indirect
3275 github.com/cli/browser v1.3.0 // indirect
33- github.com/fsnotify/fsnotify v1.7.0 // indirect
76+ github.com/cloudflare/cloudflare-go v0.112.0 // indirect
77+ github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
78+ github.com/dimchansky/utfbom v1.1.1 // indirect
79+ github.com/dnsimple/dnsimple-go v1.7.0 // indirect
80+ github.com/exoscale/egoscale/v3 v3.1.7 // indirect
81+ github.com/fatih/structs v1.1.0 // indirect
82+ github.com/fsnotify/fsnotify v1.8.0 // indirect
83+ github.com/gabriel-vasile/mimetype v1.4.2 // indirect
84+ github.com/ghodss/yaml v1.0.0 // indirect
85+ github.com/go-errors/errors v1.0.1 // indirect
86+ github.com/go-jose/go-jose/v4 v4.0.4 // indirect
87+ github.com/go-playground/locales v0.14.1 // indirect
88+ github.com/go-playground/universal-translator v0.18.1 // indirect
89+ github.com/go-playground/validator/v10 v10.16.0 // indirect
90+ github.com/go-resty/resty/v2 v2.16.2 // indirect
91+ github.com/go-viper/mapstructure/v2 v2.2.1 // indirect
92+ github.com/goccy/go-json v0.10.4 // indirect
93+ github.com/gofrs/flock v0.12.1 // indirect
94+ github.com/golang-jwt/jwt/v4 v4.5.1 // indirect
95+ github.com/golang-jwt/jwt/v5 v5.2.1 // indirect
96+ github.com/google/go-querystring v1.1.0 // indirect
97+ github.com/google/s2a-go v0.1.8 // indirect
98+ github.com/googleapis/enterprise-certificate-proxy v0.3.4 // indirect
99+ github.com/googleapis/gax-go/v2 v2.14.0 // indirect
100+ github.com/gophercloud/gophercloud v1.14.1 // indirect
101+ github.com/gophercloud/utils v0.0.0-20231010081019-80377eca5d56 // indirect
102+ github.com/hashicorp/errwrap v1.0.0 // indirect
103+ github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
104+ github.com/hashicorp/go-multierror v1.1.1 // indirect
105+ github.com/hashicorp/go-retryablehttp v0.7.7 // indirect
106+ github.com/hashicorp/go-uuid v1.0.3 // indirect
107+ github.com/hashicorp/hcl v1.0.0 // indirect
108+ github.com/huaweicloud/huaweicloud-sdk-go-v3 v0.1.128 // indirect
109+ github.com/iij/doapi v0.0.0-20190504054126-0bbf12d6d7df // indirect
110+ github.com/infobloxopen/infoblox-go-client v1.1.1 // indirect
111+ github.com/jmespath/go-jmespath v0.4.0 // indirect
112+ github.com/json-iterator/go v1.1.12 // indirect
113+ github.com/k0kubun/go-ansi v0.0.0-20180517002512-3bf9e2903213 // indirect
34114 github.com/klauspost/compress v1.18.0 // indirect
35115 github.com/klauspost/cpuid/v2 v2.2.10 // indirect
116+ github.com/kolo/xmlrpc v0.0.0-20220921171641-a4b6fa1dd06b // indirect
117+ github.com/kylelemons/godebug v1.1.0 // indirect
118+ github.com/labbsr0x/bindman-dns-webhook v1.0.2 // indirect
119+ github.com/labbsr0x/goh v1.0.1 // indirect
120+ github.com/leodido/go-urn v1.2.4 // indirect
121+ github.com/linode/linodego v1.44.0 // indirect
122+ github.com/liquidweb/liquidweb-cli v0.6.9 // indirect
123+ github.com/liquidweb/liquidweb-go v1.6.4 // indirect
124+ github.com/magiconair/properties v1.8.7 // indirect
125+ github.com/miekg/dns v1.1.62 // indirect
126+ github.com/mimuret/golang-iij-dpf v0.9.1 // indirect
127+ github.com/mitchellh/go-homedir v1.1.0 // indirect
128+ github.com/mitchellh/mapstructure v1.5.0 // indirect
129+ github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
130+ github.com/modern-go/reflect2 v1.0.2 // indirect
131+ github.com/namedotcom/go v0.0.0-20180403034216-08470befbe04 // indirect
36132 github.com/natefinch/atomic v1.0.1 // indirect
133+ github.com/nrdcg/auroradns v1.1.0 // indirect
134+ github.com/nrdcg/bunny-go v0.0.0-20240207213615-dde5bf4577a3 // indirect
135+ github.com/nrdcg/desec v0.10.0 // indirect
136+ github.com/nrdcg/dnspod-go v0.4.0 // indirect
137+ github.com/nrdcg/freemyip v0.3.0 // indirect
138+ github.com/nrdcg/goacmedns v0.2.0 // indirect
139+ github.com/nrdcg/goinwx v0.10.0 // indirect
140+ github.com/nrdcg/mailinabox v0.2.0 // indirect
141+ github.com/nrdcg/namesilo v0.2.1 // indirect
142+ github.com/nrdcg/nodion v0.1.0 // indirect
143+ github.com/nrdcg/porkbun v0.4.0 // indirect
144+ github.com/nzdjb/go-metaname v1.0.0 // indirect
145+ github.com/opentracing/opentracing-go v1.2.1-0.20220228012449-10b1cf09e00b // indirect
146+ github.com/oracle/oci-go-sdk/v65 v65.81.1 // indirect
147+ github.com/ovh/go-ovh v1.6.0 // indirect
148+ github.com/patrickmn/go-cache v2.1.0+incompatible // indirect
149+ github.com/pelletier/go-toml/v2 v2.1.0 // indirect
150+ github.com/peterhellberg/link v1.2.0 // indirect
151+ github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
152+ github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
153+ github.com/pquerna/otp v1.4.0 // indirect
154+ github.com/regfish/regfish-dnsapi-go v0.1.1 // indirect
155+ github.com/sacloud/api-client-go v0.2.10 // indirect
156+ github.com/sacloud/go-http v0.1.8 // indirect
157+ github.com/sacloud/iaas-api-go v1.14.0 // indirect
158+ github.com/sacloud/packages-go v0.0.10 // indirect
159+ github.com/sagikazarmark/locafero v0.4.0 // indirect
160+ github.com/sagikazarmark/slog-shim v0.1.0 // indirect
161+ github.com/scaleway/scaleway-sdk-go v1.0.0-beta.30 // indirect
162+ github.com/selectel/domains-go v1.1.0 // indirect
163+ github.com/selectel/go-selvpcclient/v3 v3.2.1 // indirect
164+ github.com/shopspring/decimal v1.3.1 // indirect
165+ github.com/sirupsen/logrus v1.9.3 // indirect
166+ github.com/smartystreets/go-aws-auth v0.0.0-20180515143844-0c1422d1fdb9 // indirect
167+ github.com/softlayer/softlayer-go v1.1.7 // indirect
168+ github.com/softlayer/xmlrpc v0.0.0-20200409220501-5f089df7cb7e // indirect
169+ github.com/sony/gobreaker v0.5.0 // indirect
170+ github.com/sourcegraph/conc v0.3.0 // indirect
171+ github.com/spf13/afero v1.11.0 // indirect
172+ github.com/spf13/cast v1.6.0 // indirect
173+ github.com/spf13/viper v1.18.2 // indirect
174+ github.com/stretchr/testify v1.10.0 // indirect
175+ github.com/subosito/gotenv v1.6.0 // indirect
176+ github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.0.1065 // indirect
177+ github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/dnspod v1.0.1065 // indirect
178+ github.com/tjfoc/gmsm v1.4.1 // indirect
179+ github.com/transip/gotransip/v6 v6.26.0 // indirect
180+ github.com/ultradns/ultradns-go-sdk v1.8.0-20241010134910-243eeec // indirect
181+ github.com/vinyldns/go-vinyldns v0.9.16 // indirect
182+ github.com/volcengine/volc-sdk-golang v1.0.189 // indirect
183+ github.com/vultr/govultr/v3 v3.9.1 // indirect
184+ github.com/yandex-cloud/go-genproto v0.0.0-20241220122821-aeb3b05efd1c // indirect
185+ github.com/yandex-cloud/go-sdk v0.0.0-20241220131134-2393e243c134 // indirect
37186 github.com/zeebo/xxh3 v1.0.2 // indirect
38187 go.mongodb.org/mongo-driver v1.17.3 // indirect
39188 go.opentelemetry.io/auto/sdk v1.1.0 // indirect
189+ go.uber.org/atomic v1.11.0 // indirect
190+ go.uber.org/multierr v1.9.0 // indirect
191+ go.uber.org/ratelimit v0.3.0 // indirect
40192 golang.org/x/mod v0.24.0 // indirect
193+ golang.org/x/oauth2 v0.24.0 // indirect
41194 golang.org/x/tools v0.31.0 // indirect
42- google.golang.org/genproto/googleapis/rpc v0.0.0-20240521202816-d264139d666e // indirect
195+ google.golang.org/api v0.214.0 // indirect
196+ google.golang.org/genproto v0.0.0-20241021214115-324edc3d5d38 // indirect
197+ google.golang.org/genproto/googleapis/rpc v0.0.0-20241209162323-e6fa225c2576 // indirect
198+ google.golang.org/grpc v1.69.2 // indirect
199+ google.golang.org/protobuf v1.36.1 // indirect
200+ gopkg.in/ini.v1 v1.67.0 // indirect
201+ gopkg.in/ns1/ns1-go.v2 v2.13.0 // indirect
202+ gopkg.in/yaml.v2 v2.4.0 // indirect
43203 )
44204
45205 require (
@@ -78,7 +238,7 @@ require (
78238 golang.org/x/exp v0.0.0-20250305212735-054e65f0b394 // direct
79239 golang.org/x/sync v0.12.0 // indirect
80240 golang.org/x/term v0.30.0 // indirect
81- google.golang.org/genproto/googleapis/api v0.0.0-20240528184218-531527333157 // indirect
241+ google.golang.org/genproto/googleapis/api v0.0.0-20241118233622-e639e219e697 // indirect
82242 modernc.org/libc v1.61.13 // indirect
83243 modernc.org/mathutil v1.7.1 // indirect
84244 modernc.org/memory v1.8.2 // indirect
@@ -104,7 +264,7 @@ require (
104264 github.com/pkg/errors v0.9.1 // indirect
105265 github.com/valyala/bytebufferpool v1.0.0 // indirect
106266 github.com/valyala/fasttemplate v1.2.2 // indirect
107- golang.org/x/crypto v0.36.0
267+ golang.org/x/crypto v0.36.0 // indirect
108268 golang.org/x/net v0.37.0 // indirect
109269 golang.org/x/sys v0.31.0 // indirect
110270 golang.org/x/text v0.23.0 // indirect
M go.sum +1236 -8 generated file: collapsed
load diff
M internal/cli/cmd/proxy.go +6 -7
@@ -134,11 +134,8 @@ func NewProxyCommand(a *server.App) *cobra.Command {
134134 }
135135
136136 // List routes from the database
137- rows, err := a.DB.Query(`
138- SELECT d.name, pr.container_id, pr.container_ip, pr.container_port, pr.active
139- FROM proxy_route pr
140- JOIN domain d ON pr.domain_id = d.id
141- `)
137+ // Use the query from the instantiated proxy queries
138+ rows, err := a.DB.Query(p.Queries.GetAllRoutes)
142139 if err != nil {
143140 log.Fatal("Failed to query routes:", err)
144141 }
@@ -150,9 +147,11 @@ func NewProxyCommand(a *server.App) *cobra.Command {
150147
151148 var count int
152149 for rows.Next() {
153- var domain, containerID, containerIP, containerPort string
150+ // Adjust variables to match the columns selected by GetAllRoutes
151+ var id, domain, containerID, containerIP, containerPort, protocol, path string
154152 var active bool
155- if err := rows.Scan(&domain, &containerID, &containerIP, &containerPort, &active); err != nil {
153+ // Adjust scan targets
154+ if err := rows.Scan(&id, &domain, &containerID, &containerIP, &containerPort, &protocol, &path, &active); err != nil {
156155 log.Fatal("Failed to scan row:", err)
157156 }
158157
M internal/common/init_config.go +18 -14
@@ -74,20 +74,24 @@ type ContainerEngineConfig struct {
7474 }
7575
7676 type ReverseProxyConfig struct {
77- Enabled bool `yaml:"enabled"` // Whether the reverse proxy is enabled
78- Port string `yaml:"port"` // Port for the reverse proxy to listen on
79- HttpPort string `yaml:"httpPort"` // HTTP port (usually 80) for redirecting to HTTPS
80- CertDir string `yaml:"certDir"` // Directory to store Let's Encrypt certificates
81- AutoRenew bool `yaml:"autoRenew"` // Whether to automatically renew certificates
82- RenewBefore int `yaml:"renewBefore"` // Days before expiry to renew certificates
83- LetsEncryptMode string `yaml:"letsEncryptMode"` // "staging" or "production"
84- Email string `yaml:"email"` // Email for Let's Encrypt
85- CacheSize int `yaml:"cacheSize"` // Size of the certificate cache
86- GracePeriod int `yaml:"gracePeriod"` // Shutdown grace period in seconds
87- EnableHttpLogs bool `yaml:"enableHttpLogs"` // Whether to enable HTTP request logging (default: true)
88- EnableRateLimit bool `yaml:"enableRateLimit"` // Whether to enable rate limiting middleware (default: false)
89- DetectUpstreamProxy bool `yaml:"detectUpstreamProxy"` // Whether to detect and handle upstream TLS termination proxies (default: false)
90- SkipCertificates bool `yaml:"skipCertificates"` // Skip Let's Encrypt certificate acquisition when behind a TLS terminating proxy (default: false)
77+ Enabled bool `yaml:"enabled"` // Whether the reverse proxy is enabled
78+ Port string `yaml:"port"` // Port for the reverse proxy to listen on
79+ HttpPort string `yaml:"httpPort"` // HTTP port (usually 80) for redirecting to HTTPS
80+ CertDir string `yaml:"certDir"` // Directory to store Let's Encrypt certificates
81+ Email string `yaml:"email"` // Email for Let's Encrypt
82+ LetsEncryptMode string `yaml:"letsEncryptMode"` // staging or production
83+ SkipCertificates bool `yaml:"skipCertificates"` // Skip Let's Encrypt certificate acquisition when behind a TLS terminating proxy
84+ GracePeriod int `yaml:"gracePeriod"` // Shutdown grace period in seconds
85+ AutoRenew bool `yaml:"autoRenew"` // Whether to automatically renew certificates
86+ RenewBefore int `yaml:"renewBefore"` // Days before expiry to renew certificates
87+ CacheSize int `yaml:"cacheSize"` // Size of the certificate cache
88+ EnableHttpLogs bool `yaml:"enableHttpLogs"` // Whether to enable HTTP request logging
89+ EnableRateLimit bool `yaml:"enableRateLimit"` // Whether to enable rate limiting middleware
90+ DetectUpstreamProxy bool `yaml:"detectUpstreamProxy"` // Whether to detect and handle upstream TLS termination proxies
91+ DefaultChallengeType string `yaml:"defaultChallengeType"` // http-01 or dns-01
92+ DefaultHttpChallengePort string `yaml:"defaultHttpChallengePort"`
93+ DefaultDnsChallengePropagationTimeout int `yaml:"defaultDnsChallengePropagationTimeout"`
94+ DefaultDnsChallengePollingInterval int `yaml:"defaultDnsChallengePollingInterval"`
9195 }
9296
9397 // Default values
M internal/db/db_struct.go +22 -5
@@ -1,5 +1,9 @@
11 package db
22
3+import (
4+ "database/sql"
5+)
6+
37 // User struct model
48 type User struct {
59 ID string `sql:"id, primary_key"`
@@ -46,11 +50,15 @@ type Clients struct {
4650
4751 // Domain represents a domain configuration for the proxy
4852 type Domain struct {
49- ID string `sql:"id, primary_key"`
50- Name string `sql:"name"` // e.g. example.com
51- AccountID string `sql:"account_id, foreign_key=account.id"`
52- CreatedAt string `sql:"created_at"`
53- UpdatedAt string `sql:"updated_at"`
53+ ID string `sql:"id, primary_key"`
54+ Name string `sql:"name"` // e.g. example.com
55+ AccountID string `sql:"account_id, foreign_key=account.id"`
56+ CreatedAt string `sql:"created_at"`
57+ UpdatedAt string `sql:"updated_at"`
58+ AcmeEnabled bool `sql:"acme_enabled"` // Whether ACME is enabled for this domain
59+ AcmeChallengeType sql.NullString `sql:"acme_challenge_type"` // e.g. "http-01", "dns-01"
60+ AcmeDnsProvider sql.NullString `sql:"acme_dns_provider"` // DNS provider name if using DNS-01
61+ AcmeDnsCredentialsRef sql.NullString `sql:"acme_dns_credentials_ref"` // Reference to credentials (not raw credentials)
5462 }
5563
5664 // Certificate represents a TLS certificate for a domain
@@ -79,6 +87,15 @@ type ProxyRoute struct {
7987 UpdatedAt string `sql:"updated_at"`
8088 }
8189
90+// AcmeAccount represents the stored ACME account details
91+type AcmeAccount struct {
92+ Email string `sql:"email, primary_key"`
93+ PrivateKey string `sql:"private_key"` // PEM encoded private key
94+ RegistrationInfo sql.NullString `sql:"registration_info"` // JSON marshaled registration resource
95+ CreatedAt string `sql:"created_at"`
96+ UpdatedAt string `sql:"updated_at"`
97+}
98+
8299 // GithubUserInfo holds the essential information for a Github user
83100 type GithubUserInfo struct {
84101 Login string `json:"login"`
M internal/db/queries/account.go +14 -12
@@ -2,29 +2,31 @@ package queries
22
33 import (
44 "database/sql"
5+
56 "github.com/bnema/gordon/internal/db"
7+ "github.com/google/uuid"
68 )
79
810 // AccountQueries contains all SQL queries for account operations
911 type AccountQueries struct {
10- InsertAccount string
12+ InsertAccount string
1113 CheckAccountExists string
12- GetAccountByID string
14+ GetAccountByID string
1315 }
1416
1517 // NewAccountQueries returns a new instance of AccountQueries
1618 func NewAccountQueries() *AccountQueries {
1719 return &AccountQueries{
18- InsertAccount: "INSERT INTO account (id, user_id) VALUES (?, ?)",
20+ InsertAccount: "INSERT INTO account (id, user_id) VALUES (?, ?)",
1921 CheckAccountExists: "SELECT EXISTS(SELECT 1 FROM account WHERE id = ?)",
20- GetAccountByID: "SELECT id, user_id FROM account WHERE id = ?",
22+ GetAccountByID: "SELECT id, user_id FROM account WHERE id = ?",
2123 }
2224 }
2325
2426 // CreateDBAccount creates a new account in the database
2527 func CreateDBAccount(database *sql.DB, userID string) (*db.Account, error) {
2628 account := &db.Account{
27- ID: generateUUID(),
29+ ID: uuid.NewString(),
2830 UserID: userID,
2931 }
3032
@@ -32,23 +34,23 @@ func CreateDBAccount(database *sql.DB, userID string) (*db.Account, error) {
3234 NewAccountQueries().InsertAccount,
3335 account.ID, userID,
3436 )
35-
37+
3638 return account, err
3739 }
3840
3941 // CheckDBAccountExists checks if an account exists in the database
4042 func CheckDBAccountExists(database *sql.DB, accountID string) (bool, error) {
4143 var exists bool
42-
44+
4345 err := database.QueryRow(
4446 NewAccountQueries().CheckAccountExists,
4547 accountID,
4648 ).Scan(&exists)
47-
49+
4850 if err != nil {
4951 return false, err
5052 }
51-
53+
5254 return exists, nil
5355 }
5456
@@ -57,15 +59,15 @@ func GetDBAccountByID(database *sql.DB, accountID string) (*db.Account, error) {
5759 account := &db.Account{
5860 ID: accountID,
5961 }
60-
62+
6163 err := database.QueryRow(
6264 NewAccountQueries().GetAccountByID,
6365 accountID,
6466 ).Scan(&account.ID, &account.UserID)
65-
67+
6668 if err != nil {
6769 return nil, err
6870 }
69-
71+
7072 return account, nil
7173 }
M internal/db/queries/provider.go +94 -17
@@ -2,15 +2,20 @@ package queries
22
33 import (
44 "database/sql"
5+
56 "github.com/bnema/gordon/internal/db"
7+ "github.com/google/uuid"
68 )
79
810 // ProviderQueries contains all SQL queries for provider operations
911 type ProviderQueries struct {
10- InsertProvider string
11- PopulateProviderFromDB string
12- CheckProviderExists string
13- GetProviderByAccountLogin string
12+ InsertProvider string
13+ PopulateProviderFromDB string
14+ CheckProviderExists string
15+ GetProviderByAccountLogin string
16+ CountProvidersQuery string
17+ InsertProviderForAccountQuery string
18+ GetUserByProviderLoginQuery string
1419 }
1520
1621 // NewProviderQueries returns a new instance of ProviderQueries
@@ -20,13 +25,82 @@ func NewProviderQueries() *ProviderQueries {
2025 PopulateProviderFromDB: "SELECT id, account_id, name, login, avatar_url, profile_url, email FROM provider WHERE account_id = ?",
2126 CheckProviderExists: "SELECT EXISTS(SELECT 1 FROM provider WHERE login = ? AND email = ?)",
2227 GetProviderByAccountLogin: "SELECT id, account_id, name, login, avatar_url, profile_url, email FROM provider WHERE login = ? AND email = ?",
28+ CountProvidersQuery: "SELECT COUNT(*) FROM provider",
29+ InsertProviderForAccountQuery: `
30+ INSERT INTO provider (id, account_id, name, login, avatar_url, profile_url, email)
31+ VALUES (?, ?, ?, ?, ?, ?, ?)`,
32+ GetUserByProviderLoginQuery: `
33+ SELECT u.id, u.name, u.email, p.account_id
34+ FROM provider p
35+ JOIN account a ON p.account_id = a.id
36+ JOIN user u ON a.user_id = u.id
37+ WHERE p.name = ? AND p.login = ?`,
38+ }
39+}
40+
41+// CountProviders returns the total number of provider records in the database.
42+func CountProviders(database *sql.DB) (int, error) {
43+ var count int
44+ err := database.QueryRow(NewProviderQueries().CountProvidersQuery).Scan(&count)
45+ if err != nil {
46+ return 0, err
47+ }
48+ return count, nil
49+}
50+
51+// Inserts a new provider record linked to an existing account.
52+func InsertProviderForAccount(database *sql.DB, accountID string, providerName string, userInfo *db.GithubUserInfo) error {
53+ providerID := uuid.NewString()
54+ primaryEmail := ""
55+ if len(userInfo.Emails) > 0 {
56+ primaryEmail = userInfo.Emails[0]
57+ }
58+
59+ _, err := database.Exec(
60+ NewProviderQueries().InsertProviderForAccountQuery,
61+ providerID,
62+ accountID,
63+ providerName,
64+ userInfo.Login,
65+ userInfo.AvatarURL,
66+ userInfo.ProfileURL,
67+ primaryEmail,
68+ )
69+ return err
70+}
71+
72+// Retrieves the User and Account ID associated with a specific provider login.
73+func GetUserByProviderLogin(database *sql.DB, providerName string, providerLogin string) (*db.User, string, error) {
74+ user := &db.User{}
75+ var accountID string
76+
77+ err := database.QueryRow(
78+ NewProviderQueries().GetUserByProviderLoginQuery,
79+ providerName,
80+ providerLogin,
81+ ).Scan(
82+ &user.ID,
83+ &user.Name,
84+ &user.Email,
85+ &accountID,
86+ )
87+
88+ if err != nil {
89+ return nil, "", err
2390 }
91+
92+ return user, accountID, nil
2493 }
2594
2695 // CreateDBGitHubProvider creates a new GitHub provider in the database.
2796 func CreateDBGitHubProvider(database *sql.DB, accountID string, userInfo *db.GithubUserInfo) error {
28- providerID := generateUUID()
29-
97+ providerID := uuid.NewString()
98+
99+ primaryEmail := ""
100+ if len(userInfo.Emails) > 0 {
101+ primaryEmail = userInfo.Emails[0]
102+ }
103+
30104 _, err := database.Exec(
31105 NewProviderQueries().InsertProvider,
32106 providerID,
@@ -35,16 +109,16 @@ func CreateDBGitHubProvider(database *sql.DB, accountID string, userInfo *db.Git
35109 userInfo.Login,
36110 userInfo.AvatarURL,
37111 userInfo.ProfileURL,
38- userInfo.Emails[0],
112+ primaryEmail,
39113 )
40-
114+
41115 return err
42116 }
43117
44118 // PopulateDBProviderFromDB populates provider information from the database.
45119 func PopulateDBProviderFromDB(database *sql.DB, accountID string) (*db.Provider, error) {
46120 provider := &db.Provider{}
47-
121+
48122 err := database.QueryRow(
49123 NewProviderQueries().PopulateProviderFromDB,
50124 accountID,
@@ -57,35 +131,38 @@ func PopulateDBProviderFromDB(database *sql.DB, accountID string) (*db.Provider,
57131 &provider.ProfileURL,
58132 &provider.Email,
59133 )
60-
134+
61135 if err != nil {
62136 return nil, err
63137 }
64-
138+
65139 return provider, nil
66140 }
67141
68142 // CheckDBProviderExists checks if a provider exists in the database.
69143 func CheckDBProviderExists(database *sql.DB, login string, email string) (bool, error) {
70144 var exists bool
71-
145+
72146 err := database.QueryRow(
73147 NewProviderQueries().CheckProviderExists,
74148 login,
75149 email,
76150 ).Scan(&exists)
77-
151+
78152 if err != nil {
153+ if err == sql.ErrNoRows {
154+ return false, nil
155+ }
79156 return false, err
80157 }
81-
158+
82159 return exists, nil
83160 }
84161
85162 // GetProviderByLoginAndEmail returns a provider from the database by login and email.
86163 func GetProviderByLoginAndEmail(database *sql.DB, login string, email string) (*db.Provider, error) {
87164 provider := &db.Provider{}
88-
165+
89166 err := database.QueryRow(
90167 NewProviderQueries().GetProviderByAccountLogin,
91168 login,
@@ -99,10 +176,10 @@ func GetProviderByLoginAndEmail(database *sql.DB, login string, email string) (*
99176 &provider.ProfileURL,
100177 &provider.Email,
101178 )
102-
179+
103180 if err != nil {
104181 return nil, err
105182 }
106-
183+
107184 return provider, nil
108185 }
M internal/db/queries/proxy.go +103 -15
@@ -3,18 +3,36 @@ package queries
33 // ProxyQueries contains all SQL queries used by the proxy package
44 type ProxyQueries struct {
55 // Route queries
6- GetActiveRoutes string
7- GetRouteByDomain string
8- InsertDomain string
9- GetDomainByName string
10- GetFirstAccount string
11- UpdateRoute string
12- InsertRoute string
13- DeleteRouteByDomainID string
14- DeleteDomainByID string
15- UpdateRouteIP string
16- GetAllRoutes string
17- MarkRouteInactive string
6+ GetActiveRoutes string
7+ GetRouteByDomain string
8+ GetRouteIDAndContainerIDByDomainName string
9+ GetRouteIPByID string
10+ InsertDomain string
11+ GetDomainByName string
12+ GetFirstAccount string
13+ UpdateRoute string
14+ UpdateRecreatedRoute string
15+ InsertRoute string
16+ DeleteRouteByDomainID string
17+ DeleteDomainByID string
18+ UpdateRouteIP string
19+ GetAllRoutes string
20+ MarkRouteInactive string
21+
22+ // ACME queries
23+ GetDomainAcmeConfig string
24+ UpdateDomainAcmeConfig string
25+ GetCertificateByDomain string
26+ UpdateCertificate string
27+
28+ // Admin domain queries
29+ CheckDomainExists string
30+ CreateAdminDomain string
31+ UpdateAdminDomainAcme string
32+
33+ // ACME Account queries (NEW)
34+ GetAcmeAccountByEmail string
35+ InsertOrReplaceAcmeAccount string
1836 }
1937
2038 // NewProxyQueries returns a new instance of ProxyQueries
@@ -29,9 +47,20 @@ func NewProxyQueries() *ProxyQueries {
2947 `,
3048 GetRouteByDomain: `
3149 SELECT id, container_id, container_port
32- FROM proxy_route
50+ FROM proxy_route
3351 WHERE domain_id = ?
3452 `,
53+ GetRouteIDAndContainerIDByDomainName: `
54+ SELECT pr.id, pr.container_id
55+ FROM proxy_route pr
56+ JOIN domain d ON pr.domain_id = d.id
57+ WHERE d.name = ?
58+ `,
59+ GetRouteIPByID: `
60+ SELECT container_ip
61+ FROM proxy_route
62+ WHERE id = ?
63+ `,
3564 InsertDomain: `
3665 INSERT INTO domain (id, name, account_id, created_at, updated_at)
3766 VALUES (?, ?, ?, ?, ?)
@@ -53,6 +82,11 @@ func NewProxyQueries() *ProxyQueries {
5382 updated_at = ?
5483 WHERE id = ?
5584 `,
85+ UpdateRecreatedRoute: `
86+ UPDATE proxy_route
87+ SET container_id = ?, container_ip = ?, container_port = ?, updated_at = ?
88+ WHERE id = ?
89+ `,
5690 InsertRoute: `
5791 INSERT INTO proxy_route (
5892 id, domain_id, container_id, container_ip, container_port,
@@ -70,13 +104,67 @@ func NewProxyQueries() *ProxyQueries {
70104 WHERE domain_id = ? AND container_id = ?
71105 `,
72106 GetAllRoutes: `
73- SELECT d.name, pr.container_id, pr.container_ip, pr.container_port, pr.protocol, pr.path
107+ SELECT pr.id, d.name, pr.container_id, pr.container_ip, pr.container_port, pr.protocol, pr.path, pr.active
74108 FROM proxy_route pr
75109 JOIN domain d ON pr.domain_id = d.id
76- WHERE pr.active = 1
77110 `,
78111 MarkRouteInactive: `
79112 UPDATE proxy_route SET active = 0, updated_at = ? WHERE domain_id = ?
80113 `,
114+
115+ // ACME queries
116+ GetDomainAcmeConfig: `
117+ SELECT acme_enabled, acme_challenge_type, acme_dns_provider, acme_dns_credentials_ref
118+ FROM domain WHERE name = ?
119+ `,
120+ UpdateDomainAcmeConfig: `
121+ UPDATE domain SET
122+ acme_enabled = ?,
123+ acme_challenge_type = ?,
124+ acme_dns_provider = ?,
125+ acme_dns_credentials_ref = ?,
126+ updated_at = ?
127+ WHERE name = ?
128+ `,
129+ GetCertificateByDomain: `
130+ SELECT c.cert_file, c.key_file, c.issued_at, c.expires_at, c.issuer, c.status
131+ FROM certificate c
132+ JOIN domain d ON c.domain_id = d.id
133+ WHERE d.name = ?
134+ `,
135+ UpdateCertificate: `
136+ INSERT OR REPLACE INTO certificate (
137+ id, domain_id, cert_file, key_file, issued_at, expires_at, issuer, status
138+ ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
139+ `,
140+
141+ // Admin domain queries
142+ CheckDomainExists: `
143+ SELECT EXISTS(SELECT 1 FROM domain WHERE name = ?)
144+ `,
145+ CreateAdminDomain: `
146+ INSERT INTO domain (
147+ id, name, account_id, created_at, updated_at, acme_enabled, acme_challenge_type
148+ ) VALUES (?, ?, ?, ?, ?, ?, ?)
149+ `,
150+ UpdateAdminDomainAcme: `
151+ UPDATE domain SET
152+ acme_enabled = 1,
153+ acme_challenge_type = 'http-01',
154+ updated_at = ?
155+ WHERE name = ?
156+ `,
157+
158+ // ACME Account queries (NEW)
159+ GetAcmeAccountByEmail: `
160+ SELECT private_key, registration_info
161+ FROM acme_account
162+ WHERE email = ?
163+ `,
164+ InsertOrReplaceAcmeAccount: `
165+ INSERT OR REPLACE INTO acme_account (
166+ email, private_key, registration_info, created_at, updated_at
167+ ) VALUES (?, ?, ?, ?, ?)
168+ `,
81169 }
82170 }
M internal/db/queries/sessions.go +80 -32
@@ -7,6 +7,7 @@ import (
77
88 "github.com/bnema/gordon/internal/db"
99 "github.com/bnema/gordon/pkg/logger"
10+ "github.com/google/uuid"
1011 )
1112
1213 // SessionQueries contains all SQL queries for session operations
@@ -38,18 +39,18 @@ func NewSessionQueries() *SessionQueries {
3839 // CreateDBSession creates a session for the user.
3940 func CreateDBSession(database *sql.DB, browserInfo string, accessToken string, accountID string) (*db.Sessions, error) {
4041 expireTime := time.Now().Add(time.Hour * 24).Format(time.RFC3339)
41- sessionID := generateUUID()
42-
42+ sessionID := uuid.NewString()
43+
4344 _, err := database.Exec(
4445 NewSessionQueries().InsertSession,
4546 sessionID, accountID, accessToken, browserInfo, expireTime, true,
4647 )
47-
48+
4849 if err != nil {
4950 logger.Error("Failed to create session in database", "error", err, "session_id", sessionID, "account_id", accountID)
5051 return nil, err
5152 }
52-
53+
5354 // Create a Sessions struct to return
5455 session := &db.Sessions{
5556 ID: sessionID,
@@ -59,7 +60,7 @@ func CreateDBSession(database *sql.DB, browserInfo string, accessToken string, a
5960 Expires: expireTime,
6061 IsOnline: true,
6162 }
62-
63+
6364 logger.Debug("Session created in database", "session_id", sessionID, "account_id", accountID, "expires", expireTime)
6465 return session, nil
6566 }
@@ -97,51 +98,98 @@ func GetDBUserSession(database *sql.DB, accessToken string, browserInfo string)
9798 // UpdateDBSession updates an existing session with a new access token
9899 func UpdateDBSession(database *sql.DB, sessionID string, accessToken string) error {
99100 expireTime := time.Now().Add(time.Hour * 24).Format(time.RFC3339)
100-
101+
101102 _, err := database.Exec(
102103 NewSessionQueries().UpdateSessionToken,
103104 accessToken,
104105 expireTime,
105106 sessionID,
106107 )
107-
108+
108109 if err != nil {
109110 logger.Error("Failed to update session in database", "error", err, "session_id", sessionID)
110111 return err
111112 }
112-
113+
113114 logger.Debug("Session updated in database", "sessionID", sessionID, "expires", expireTime)
114115 return nil
115116 }
116117
117-// CreateOrUpdateDBSession creates a new session or updates an existing one
118-func CreateOrUpdateDBSession(database *sql.DB, accessToken string, browserInfo string, accountID string) error {
119- // Check if the session exists using the access token and browser info
120- session, err := GetDBUserSession(database, accessToken, browserInfo)
118+// CreateOrUpdateSession creates a new session or updates an existing one
119+func CreateOrUpdateSession(database *sql.DB, accountID string, accessToken string, browserInfo string) (*db.Sessions, error) {
120+ // Check if a session exists for this account with the same browser info
121+ // (Token might change, so check by account/browser)
122+ existingSession, err := GetSessionByAccountAndBrowser(database, accountID, browserInfo)
121123 if err != nil && err != sql.ErrNoRows {
122- logger.Warn("Failed to check for session", "error", err)
123- return fmt.Errorf("error checking for session: %w", err)
124+ logger.Error("Failed to check for existing session", "error", err, "accountID", accountID)
125+ return nil, fmt.Errorf("error checking for existing session: %w", err)
124126 }
125127
126- // If the session exists, update the session
127- if session != nil {
128- err := UpdateDBSession(database, session.ID, accessToken)
128+ expireTime := time.Now().Add(time.Hour * 24).Format(time.RFC3339)
129+
130+ // If the session exists, update its token and expiry
131+ if existingSession != nil {
132+ logger.Debug("Existing session found, updating token and expiry", "sessionID", existingSession.ID, "accountID", accountID)
133+ _, err := database.Exec(
134+ NewSessionQueries().UpdateSessionToken,
135+ accessToken,
136+ expireTime,
137+ existingSession.ID,
138+ )
129139 if err != nil {
130- logger.Warn("Failed to update session", "error", err)
131- return fmt.Errorf("error updating session: %w", err)
140+ logger.Error("Failed to update existing session", "error", err, "session_id", existingSession.ID)
141+ return nil, fmt.Errorf("error updating session: %w", err)
132142 }
133- logger.Debug("Session updated in database", "accountID", accountID, "sessionID", session.ID)
143+ // Update the struct we have in memory
144+ existingSession.AccessToken = accessToken
145+ existingSession.Expires = expireTime
146+ existingSession.IsOnline = true // Ensure it's marked online
147+ logger.Info("Session updated successfully", "sessionID", existingSession.ID)
148+ return existingSession, nil
134149 } else {
135- // If the session does not exist, create the session
136- _, err := CreateDBSession(database, browserInfo, accessToken, accountID)
150+ // If the session does not exist, create a new one
151+ logger.Debug("No existing session found, creating new session", "accountID", accountID)
152+ sessionID := uuid.NewString() // Use uuid directly
153+ _, err := database.Exec(
154+ NewSessionQueries().InsertSession,
155+ sessionID, accountID, accessToken, browserInfo, expireTime, true,
156+ )
137157 if err != nil {
138- logger.Warn("Failed to create session", "error", err)
139- return fmt.Errorf("error creating session: %w", err)
158+ logger.Error("Failed to create new session", "error", err, "accountID", accountID)
159+ return nil, fmt.Errorf("error creating session: %w", err)
160+ }
161+ // Create the session struct to return
162+ newSession := &db.Sessions{
163+ ID: sessionID,
164+ AccountID: accountID,
165+ AccessToken: accessToken,
166+ BrowserInfo: browserInfo,
167+ Expires: expireTime,
168+ IsOnline: true,
140169 }
141- logger.Debug("New session created in database", "accountID", accountID)
170+ logger.Info("New session created successfully", "sessionID", newSession.ID)
171+ return newSession, nil
142172 }
173+}
143174
144- return nil
175+// GetSessionByAccountAndBrowser finds a session by account ID and browser info
176+func GetSessionByAccountAndBrowser(database *sql.DB, accountID string, browserInfo string) (*db.Sessions, error) {
177+ session := &db.Sessions{}
178+ // Define the query inline or add to SessionQueries struct
179+ query := "SELECT id, account_id, access_token, browser_info, expires, is_online FROM sessions WHERE account_id = ? AND browser_info = ?"
180+ err := database.QueryRow(query, accountID, browserInfo).Scan(
181+ &session.ID,
182+ &session.AccountID,
183+ &session.AccessToken,
184+ &session.BrowserInfo,
185+ &session.Expires,
186+ &session.IsOnline,
187+ )
188+ if err != nil {
189+ // Return sql.ErrNoRows if not found
190+ return nil, err
191+ }
192+ return session, nil
145193 }
146194
147195 // DeleteDBSession deletes the session from the database.
@@ -151,12 +199,12 @@ func DeleteDBSession(database *sql.DB, accountID string, sessionID string) error
151199 accountID,
152200 sessionID,
153201 )
154-
202+
155203 if err != nil {
156204 logger.Error("Failed to delete session from database", "error", err, "account_id", accountID, "session_id", sessionID)
157205 return err
158206 }
159-
207+
160208 logger.Debug("Session deleted from database", "accountID", accountID, "sessionID", sessionID)
161209 return nil
162210 }
@@ -194,12 +242,12 @@ func ExtendSessionExpiration(database *sql.DB, accountID string, sessionID strin
194242 accountID,
195243 sessionID,
196244 )
197-
245+
198246 if err != nil {
199247 logger.Error("Failed to extend session expiration", "error", err, "account_id", accountID, "session_id", sessionID)
200248 return err
201249 }
202-
250+
203251 logger.Debug("Session expiration time extended", "accountID", accountID, "sessionID", sessionID, "newExpirationTime", newExpirationTime)
204252 return nil
205253 }
@@ -227,12 +275,12 @@ func InvalidateDBSession(database *sql.DB, sessionID string) error {
227275 false,
228276 sessionID,
229277 )
230-
278+
231279 if err != nil {
232280 logger.Error("Failed to invalidate session", "error", err, "session_id", sessionID)
233281 return err
234282 }
235-
283+
236284 logger.Debug("Session invalidated", "sessionID", sessionID)
237285 return nil
238286 }
M internal/db/queries/user.go +66 -69
@@ -3,35 +3,61 @@ package queries
33 import (
44 "database/sql"
55 "fmt"
6- "time"
76
87 "github.com/bnema/gordon/internal/db"
98 "github.com/bnema/gordon/pkg/logger"
109 "github.com/google/uuid"
1110 )
1211
13-func generateUUID() string {
14- return uuid.New().String()
15-}
16-
1712 // UserQueries contains all SQL queries for user operations
1813 type UserQueries struct {
19- InsertUser string
20- CheckUserExists string
21- GetUserByProvider string
22- GetAccountCount string
14+ InsertUser string
15+ CheckUserExists string
16+ GetUserByProvider string
17+ GetAccountCount string
18+ GetFirstUserAndAccountQuery string
19+ UpdateUserDetailsQuery string
2320 }
2421
2522 // NewUserQueries returns a new instance of UserQueries
2623 func NewUserQueries() *UserQueries {
2724 return &UserQueries{
2825 InsertUser: "INSERT INTO user (id, name, email) VALUES (?, ?, ?)",
29- CheckUserExists: "SELECT id FROM user",
26+ CheckUserExists: "SELECT EXISTS(SELECT 1 FROM user LIMIT 1)",
3027 GetUserByProvider: "SELECT user.id, user.name, user.email FROM user INNER JOIN account ON user.id = account.user_id INNER JOIN provider ON account.id = provider.account_id WHERE provider.login = ? AND provider.email = ?",
3128 GetAccountCount: "SELECT COUNT(*) FROM account",
29+ GetFirstUserAndAccountQuery: `
30+ SELECT u.id, u.name, u.email, a.id
31+ FROM user u
32+ JOIN account a ON u.id = a.user_id
33+ ORDER BY u.rowid ASC -- Rely on implicit rowid for 'first'
34+ LIMIT 1`,
35+ UpdateUserDetailsQuery: "UPDATE user SET name = ?, email = ? WHERE id = ?",
3236 }
3337 }
3438
39+// GetFirstUserAndAccount retrieves the first user and their associated account ID. Assumes this is the seeded admin.
40+func GetFirstUserAndAccount(database *sql.DB) (*db.User, string, error) {
41+ user := &db.User{}
42+ var accountID string
43+ err := database.QueryRow(NewUserQueries().GetFirstUserAndAccountQuery).Scan(
44+ &user.ID,
45+ &user.Name,
46+ &user.Email,
47+ &accountID,
48+ )
49+ if err != nil {
50+ return nil, "", err
51+ }
52+ return user, accountID, nil
53+}
54+
55+// UpdateUserDetails updates the name and email for a specific user ID.
56+func UpdateUserDetails(database *sql.DB, userID string, newName string, newEmail string) error {
57+ _, err := database.Exec(NewUserQueries().UpdateUserDetailsQuery, newName, newEmail, userID)
58+ return err
59+}
60+
3561 // CreateUser creates a new user along with the associated account, provider, and session.
3662 func CreateUser(database *sql.DB, accessToken string, browserInfo string, userInfo *db.GithubUserInfo) (*db.User, string, *db.Sessions, error) {
3763 // Check if a user already exists. If so, return an error.
@@ -58,7 +84,7 @@ func CreateUser(database *sql.DB, accessToken string, browserInfo string, userIn
5884 logger.Info("Database account created successfully", "account_id", account.ID)
5985
6086 logger.Debug("Creating GitHub provider for account", "account_id", account.ID)
61- err = CreateDBGitHubProvider(database, account.ID, userInfo)
87+ err = InsertProviderForAccount(database, account.ID, "github", userInfo)
6288 if err != nil {
6389 logger.Error("Failed to create GitHub provider", "error", err)
6490 return nil, "", nil, err
@@ -66,7 +92,7 @@ func CreateUser(database *sql.DB, accessToken string, browserInfo string, userIn
6692 logger.Info("GitHub provider created successfully")
6793
6894 logger.Debug("Creating session for account", "account_id", account.ID)
69- session, err := CreateDBSession(database, browserInfo, accessToken, account.ID)
95+ session, err := CreateOrUpdateSession(database, account.ID, accessToken, browserInfo)
7096 if err != nil {
7197 logger.Error("Failed to create session", "error", err)
7298 return nil, "", nil, err
@@ -78,10 +104,15 @@ func CreateUser(database *sql.DB, accessToken string, browserInfo string, userIn
78104
79105 // CreateDBUser creates a new user in the database
80106 func CreateDBUser(database *sql.DB, userInfo *db.GithubUserInfo) (*db.User, error) {
107+ primaryEmail := ""
108+ if len(userInfo.Emails) > 0 {
109+ primaryEmail = userInfo.Emails[0]
110+ }
111+
81112 user := &db.User{
82- ID: generateUUID(),
113+ ID: uuid.NewString(),
83114 Name: userInfo.Login,
84- Email: userInfo.Emails[0],
115+ Email: primaryEmail,
85116 }
86117
87118 _, err := database.Exec(
@@ -93,34 +124,29 @@ func CreateDBUser(database *sql.DB, userInfo *db.GithubUserInfo) (*db.User, erro
93124
94125 // CheckDBUserExists checks if any user exists in the database
95126 func CheckDBUserExists(database *sql.DB) (bool, error) {
96- var userID string
97- err := database.QueryRow(NewUserQueries().CheckUserExists).Scan(&userID)
127+ var exists bool
128+ err := database.QueryRow(NewUserQueries().CheckUserExists).Scan(&exists)
98129 if err != nil {
99- if err == sql.ErrNoRows {
100- return false, nil
101- }
102130 return false, err
103131 }
104-
105- if userID != "" {
106- return true, nil
107- }
108-
109- return false, nil
132+ return exists, nil
110133 }
111134
112135 // CheckDBUserIsGood checks if the user matches the provided GitHub info
113136 func CheckDBUserIsGood(database *sql.DB, userInfo *db.GithubUserInfo) (*db.User, bool, error) {
114137 login := userInfo.Login
115- email := userInfo.Emails[0]
138+ primaryEmail := ""
139+ if len(userInfo.Emails) > 0 {
140+ primaryEmail = userInfo.Emails[0]
141+ }
116142
117143 user := &db.User{}
118-
144+
119145 // Check if the user exists based on github login and email
120- err := database.QueryRow(NewUserQueries().GetUserByProvider, login, email).Scan(
146+ err := database.QueryRow(NewUserQueries().GetUserByProvider, login, primaryEmail).Scan(
121147 &user.ID, &user.Name, &user.Email,
122148 )
123-
149+
124150 if err != nil {
125151 if err == sql.ErrNoRows {
126152 return nil, false, nil
@@ -138,51 +164,22 @@ func CheckDBUserIsGood(database *sql.DB, userInfo *db.GithubUserInfo) (*db.User,
138164 // UpdateUser updates the user session with new access token
139165 func UpdateUser(database *sql.DB, accessToken string, browserInfo string, userInfo *db.GithubUserInfo) (*db.User, string, *db.Sessions, error) {
140166 // First, get the user info to get their account ID
141- user, isGood, err := CheckDBUserIsGood(database, userInfo)
167+ user, accountID, err := GetUserByProviderLogin(database, "github", userInfo.Login)
142168 if err != nil {
143- return nil, "", nil, fmt.Errorf("error checking user: %w", err)
144- }
145-
146- if !isGood {
147- return nil, "", nil, fmt.Errorf("user not found in database")
169+ if err == sql.ErrNoRows {
170+ return nil, "", nil, fmt.Errorf("user not found in database for provider login '%s'", userInfo.Login)
171+ }
172+ return nil, "", nil, fmt.Errorf("error checking user by provider login: %w", err)
148173 }
149-
150- // Get the account ID for this user
151- var accountID string
152- err = database.QueryRow("SELECT id FROM account WHERE user_id = ?", user.ID).Scan(&accountID)
174+
175+ // If user found, update/create their session
176+ session, err := CreateOrUpdateSession(database, accountID, accessToken, browserInfo)
153177 if err != nil {
154- return nil, "", nil, fmt.Errorf("error getting account ID: %w", err)
155- }
156-
157- // Check if the session exists using the access token and browser info
158- existingSession, err := GetDBUserSession(database, accessToken, browserInfo)
159- if err != nil && err != sql.ErrNoRows {
160- return nil, "", nil, fmt.Errorf("error checking for session: %w", err)
161- }
162-
163- var session *db.Sessions
164-
165- // If the session exists, update the session
166- if existingSession != nil {
167- expiresTime := time.Now().Add(time.Hour * 24).Format(time.RFC3339)
168- err := UpdateDBSession(database, existingSession.ID, accessToken)
169- if err != nil {
170- return nil, "", nil, fmt.Errorf("error updating session: %w", err)
171- }
172- // Update the session struct with new values
173- existingSession.AccessToken = accessToken
174- existingSession.Expires = expiresTime
175- session = existingSession
176- logger.Debug("Session updated in database", "accountID", accountID, "sessionID", session.ID)
177- } else {
178- // If the session does not exist, create the session
179- session, err = CreateDBSession(database, browserInfo, accessToken, accountID)
180- if err != nil {
181- return nil, "", nil, fmt.Errorf("could not create session: %w", err)
182- }
183- logger.Debug("New session created in database", "accountID", accountID, "sessionID", session.ID)
178+ return nil, "", nil, fmt.Errorf("error creating/updating session: %w", err)
184179 }
185180
181+ logger.Debug("Session updated/created in database", "accountID", accountID, "sessionID", session.ID)
182+
186183 return user, accountID, session, nil
187184 }
188185
M internal/httpserve/handlers/endpoint_deploy.go +1 -4
@@ -573,10 +573,7 @@ func AddProxyRoute(a *server.App, containerID, containerIP, containerPort, targe
573573
574574 // Verify the database update with a direct query
575575 var updatedContainerIP string
576- err = a.GetDB().QueryRow(`
577- SELECT container_ip
578- FROM proxy_route
579- WHERE id = ?`, existingRouteID).Scan(&updatedContainerIP)
576+ err = a.GetDB().QueryRow(p.Queries.GetRouteIPByID, existingRouteID).Scan(&updatedContainerIP)
580577
581578 if err == nil && updatedContainerIP != containerIP {
582579 logger.Warn("Database verification failed: container IP mismatch",
M internal/httpserve/handlers/login.go +87 -55
@@ -1,6 +1,7 @@
11 package handlers
22
33 import (
4+ "database/sql"
45 "encoding/base64"
56 "encoding/json"
67 "fmt"
@@ -257,88 +258,119 @@ func getSession(c echo.Context) (*sessions.Session, error) {
257258
258259 // handleUser handles the user creation or update
259260 func handleUser(c echo.Context, a *server.App, accessToken, browserInfo string, userInfo *db.GithubUserInfo) (*db.User, string, *db.Sessions, error) {
260- logger.Debug("Starting handleUser function", "github_login", userInfo.Login, "github_email", userInfo.Emails[0])
261- userExists, err := queries.CheckDBUserExists(a.DB)
261+ logger.Debug("Starting handleUser function", "github_login", userInfo.Login)
262+
263+ // --- Check provider count ---
264+ providerCount, err := queries.CountProviders(a.DB)
262265 if err != nil {
263- logger.Error("Error checking if user exists", "error", err)
264- return nil, "", nil, fmt.Errorf("could not check if user exists: %w", err)
266+ logger.Error("Error checking provider count", "error", err)
267+ return nil, "", nil, fmt.Errorf("could not check provider count: %w", err)
265268 }
269+ logger.Debug("Provider count check result", "count", providerCount)
266270
267- logger.Debug("User exists check result", "user_exists", userExists)
271+ // --- Case 1: First ever GitHub login ---
272+ if providerCount == 0 {
273+ logger.Info("First GitHub login detected. Linking to seeded admin account.")
268274
269- if !userExists {
270- // if it is a new user creation we compare the gordon token
271- logger.Debug("No user exists, attempting to create new user")
275+ // Compare the Gordon token (still need admin token for the *very first* setup)
272276 err := CompareGordonToken(c, a)
273277 if err != nil {
274- logger.Error("Token comparison failed", "error", err)
275- return nil, "", nil, echo.NewHTTPError(http.StatusUnauthorized, "Token is empty or not valid")
278+ logger.Error("Admin token comparison failed for first login", "error", err)
279+ return nil, "", nil, echo.NewHTTPError(http.StatusUnauthorized, "Admin token is empty or not valid for initial setup")
276280 }
277- logger.Debug("Token comparison successful, proceeding with user creation")
281+ logger.Debug("Admin token comparison successful for first login.")
278282
279- user, accountID, session, err := createUser(a, accessToken, browserInfo, userInfo)
283+ // --- Link to Seeded Admin ---
284+ seededUser, seededAccountID, session, err := linkProviderToSeededAdmin(a, accessToken, browserInfo, userInfo)
280285 if err != nil {
281- logger.Error("Failed to create user", "error", err)
286+ logger.Error("Failed to link provider to seeded admin", "error", err)
282287 return nil, "", nil, err
283288 }
284- logger.Info("Successfully created new user", "user_id", user.ID, "user_name", user.Name, "account_id", accountID)
285- return user, accountID, session, nil
289+ logger.Info("Successfully linked GitHub provider to seeded admin account", "user_id", seededUser.ID, "account_id", seededAccountID)
290+ return seededUser, seededAccountID, session, nil
286291 }
287292
288- if userExists {
289- // if it is an existing user we compare the login and email to see if it is the same user
290- logger.Debug("User exists, checking if GitHub info matches existing user")
291- _, isGoodUser, err := queries.CheckDBUserIsGood(a.DB, userInfo)
292- if err != nil {
293- logger.Error("Error checking if user is good", "error", err)
294- return nil, "", nil, fmt.Errorf("could not check if user is good: %w", err)
295- }
293+ // --- Case 2: Subsequent GitHub logins ---
294+ if providerCount > 0 {
295+ logger.Debug("Existing provider(s) found. Checking if incoming GitHub login matches.")
296296
297- logger.Debug("User check result", "is_good_user", isGoodUser)
298-
299- if !isGoodUser {
300- logger.Warn("Unauthorized user", "github_login", userInfo.Login)
301- return nil, "", nil, echo.NewHTTPError(http.StatusUnauthorized, "Unauthorized user")
297+ // --- Check if incoming GitHub user matches an existing provider ---
298+ existingUser, existingAccountID, err := queries.GetUserByProviderLogin(a.DB, "github", userInfo.Login)
299+ if err != nil {
300+ // Handle sql.ErrNoRows specifically: means this GitHub user is not linked yet.
301+ if err == sql.ErrNoRows {
302+ logger.Warn("Unauthorized GitHub user trying to log in (no matching provider found).", "github_login", userInfo.Login)
303+ // For now, deny access. Future: could allow multiple users if desired.
304+ return nil, "", nil, echo.NewHTTPError(http.StatusUnauthorized, "This GitHub account is not authorized.")
305+ }
306+ // Handle other potential database errors
307+ logger.Error("Error checking for existing provider by login", "error", err)
308+ return nil, "", nil, fmt.Errorf("could not check existing provider: %w", err)
302309 }
310+ logger.Debug("Incoming GitHub login matches existing user.", "github_login", userInfo.Login, "user_id", existingUser.ID, "account_id", existingAccountID)
303311
304- if isGoodUser {
305- // Update the user
306- logger.Debug("User is authorized, updating session")
307- updatedUser, accountID, session, err := updateUser(a, accessToken, browserInfo, userInfo)
308- if err != nil {
309- logger.Error("Could not update user", "error", err)
310- return nil, "", nil, fmt.Errorf("could not update user: %w", err)
311- }
312- logger.Info("Successfully updated user session", "user_id", updatedUser.ID, "account_id", accountID)
313- return updatedUser, accountID, session, nil
312+ // --- Update Session for Existing User ---
313+ // The user is valid and matches the existing provider, just update their session.
314+ session, err := queries.CreateOrUpdateSession(a.DB, existingAccountID, accessToken, browserInfo)
315+ if err != nil {
316+ logger.Error("Could not create/update session for existing user", "error", err)
317+ return nil, "", nil, fmt.Errorf("could not update session: %w", err)
314318 }
319+ logger.Info("Successfully created/updated session for existing user", "user_id", existingUser.ID, "account_id", existingAccountID)
320+ return existingUser, existingAccountID, session, nil
315321 }
316322
317- logger.Error("Could not handle user - reached end of function without returning")
323+ // Should not be reached if logic is correct
324+ logger.Error("Could not handle user - reached end of function unexpectedly", "provider_count", providerCount)
318325 return nil, "", nil, fmt.Errorf("could not handle user")
319326 }
320327
321-// createUser creates a new user in the database
322-func createUser(a *server.App, accessToken, browserInfo string, userInfo *db.GithubUserInfo) (*db.User, string, *db.Sessions, error) {
323- logger.Debug("Starting createUser function", "github_login", userInfo.Login)
324- user, accountID, session, err := queries.CreateUser(a.DB, accessToken, browserInfo, userInfo)
328+// --- NEW FUNCTION: linkProviderToSeededAdmin ---
329+// Finds the seeded admin, updates its details, creates the provider and session.
330+func linkProviderToSeededAdmin(a *server.App, accessToken, browserInfo string, userInfo *db.GithubUserInfo) (*db.User, string, *db.Sessions, error) {
331+ logger.Debug("Starting linkProviderToSeededAdmin", "github_login", userInfo.Login)
332+
333+ // 1. Get the seeded user and account ID (assuming it's the only one)
334+ seededUser, seededAccountID, err := queries.GetFirstUserAndAccount(a.DB)
325335 if err != nil {
326- logger.Error("Failed to create user in database", "error", err)
327- return nil, "", nil, fmt.Errorf("could not create user: %w", err)
336+ logger.Error("Failed to get seeded user/account", "error", err)
337+ return nil, "", nil, fmt.Errorf("critical: could not find seeded admin account: %w", err)
328338 }
329- logger.Info("User created successfully", "user_id", user.ID, "name", user.Name, "account_id", accountID)
330- return user, accountID, session, nil
331-}
339+ logger.Debug("Found seeded admin", "user_id", seededUser.ID, "account_id", seededAccountID, "original_email", seededUser.Email)
332340
333-// updateUser updates an existing user in the database
334-func updateUser(a *server.App, accessToken, browserInfo string, userInfo *db.GithubUserInfo) (*db.User, string, *db.Sessions, error) {
335- user, accountID, session, err := queries.UpdateUser(a.DB, accessToken, browserInfo, userInfo)
341+ // 2. Update the seeded user's details with GitHub info
342+ newEmail := seededUser.Email // Keep original unless GitHub provides one
343+ if len(userInfo.Emails) > 0 && userInfo.Emails[0] != "" {
344+ newEmail = userInfo.Emails[0]
345+ }
346+ newName := userInfo.Login
347+
348+ err = queries.UpdateUserDetails(a.DB, seededUser.ID, newName, newEmail)
349+ if err != nil {
350+ logger.Error("Failed to update seeded user details", "user_id", seededUser.ID, "error", err)
351+ return nil, "", nil, fmt.Errorf("could not update seeded user details: %w", err)
352+ }
353+ logger.Info("Updated seeded user details", "user_id", seededUser.ID, "new_name", newName, "new_email", newEmail)
354+ seededUser.Name = newName
355+ seededUser.Email = newEmail
356+
357+ // 3. Create the Provider record linked to the seeded account
358+ err = queries.InsertProviderForAccount(a.DB, seededAccountID, "github", userInfo)
336359 if err != nil {
337- logger.Error("Could not update user", "error", err)
338- return nil, "", nil, fmt.Errorf("could not update user: %w", err)
360+ logger.Error("Failed to insert provider record for seeded account", "account_id", seededAccountID, "error", err)
361+ return nil, "", nil, fmt.Errorf("could not insert provider record: %w", err)
339362 }
340- logger.Info("User updated successfully", "user_id", user.ID, "account_id", accountID)
341- return user, accountID, session, nil
363+ logger.Info("Inserted provider record", "account_id", seededAccountID, "provider", "github", "login", userInfo.Login)
364+
365+ // 4. Create/Update the Session record linked to the seeded account
366+ session, err := queries.CreateOrUpdateSession(a.DB, seededAccountID, accessToken, browserInfo)
367+ if err != nil {
368+ logger.Error("Failed to create/update session for seeded account", "account_id", seededAccountID, "error", err)
369+ return nil, "", nil, fmt.Errorf("could not create/update session: %w", err)
370+ }
371+ logger.Info("Created/Updated session record", "account_id", seededAccountID, "session_id", session.ID)
372+
373+ return seededUser, seededAccountID, session, nil
342374 }
343375
344376 // setSessionValues sets the session values for the user
M internal/httpserve/handlers/session_db.go +65 -27
@@ -1,6 +1,7 @@
11 package handlers
22
33 import (
4+ "database/sql"
45 "fmt"
56 "time"
67
@@ -16,67 +17,104 @@ func ValidateSessionAndUser(c echo.Context, a *server.App) error {
1617
1718 sess, err := session.Get("session", c)
1819 if err != nil {
19- logger.Warn("Failed to get session", "error", err)
20+ logger.Warn("Failed to get session from cookie store", "error", err)
2021 return fmt.Errorf("failed to get session: %w", err)
2122 }
2223
2324 accountID, ok := sess.Values["accountID"].(string)
24- if !ok {
25- logger.Warn("Account ID not found or invalid type")
26- return fmt.Errorf("accountID not found or invalid type")
25+ if !ok || accountID == "" {
26+ logger.Warn("Account ID not found in session or is empty")
27+ return fmt.Errorf("accountID not found or invalid")
2728 }
2829
2930 sessionID, ok := sess.Values["sessionID"].(string)
30- if !ok {
31- logger.Warn("Session ID not found or invalid type")
32- return fmt.Errorf("sessionID not found or invalid type")
31+ if !ok || sessionID == "" {
32+ logger.Warn("Session ID not found in session or is empty")
33+ return fmt.Errorf("sessionID not found or invalid")
34+ }
35+
36+ authenticated, ok := sess.Values["authenticated"].(bool)
37+ if !ok || !authenticated {
38+ logger.Warn("User not authenticated according to session cookie", "ok", ok, "authenticated_val", sess.Values["authenticated"])
39+ return fmt.Errorf("user not authenticated")
3340 }
3441
3542 accountExists, err := IsAccountIDInDB(a, accountID)
36- if err != nil || !accountExists {
37- logger.Warn("Account does not exist")
43+ if err != nil {
44+ logger.Error("Failed to check account existence", "error", err, "accountID", accountID)
45+ return fmt.Errorf("failed checking account existence: %w", err)
46+ }
47+ if !accountExists {
48+ logger.Warn("Account associated with session not found in DB", "accountID", accountID)
3849 return fmt.Errorf("account does not exist")
3950 }
4051
41- sessionExpired, err := IsSessionExpiredInDB(a, accountID, sessionID)
42- if err != nil || sessionExpired {
43- logger.Warn("Session is expired")
44- return fmt.Errorf("session is expired")
52+ isValidInDB, err := IsSessionValidInDB(a, sessionID)
53+ if err != nil {
54+ logger.Error("Failed to check session validity in DB", "error", err, "sessionID", sessionID)
55+ return fmt.Errorf("failed checking session validity in DB: %w", err)
56+ }
57+ if !isValidInDB {
58+ logger.Warn("Session is invalid in DB (expired or not online)", "sessionID", sessionID)
59+ return fmt.Errorf("session invalid or expired")
4560 }
4661
47- // Extend session expiration conditionally, e.g., if it's close to expiring
4862 err = queries.ExtendSessionExpiration(a.DB, accountID, sessionID, time.Now().Add(sessionExpiryTime))
4963 if err != nil {
50- logger.Warn("Failed to extend session", "error", err)
51- return fmt.Errorf("failed to extend session: %v", err)
64+ logger.Error("Failed to extend session in DB", "error", err, "accountID", accountID, "sessionID", sessionID)
65+ return fmt.Errorf("failed to extend session: %w", err)
66+ }
67+ sess.Values["expires"] = time.Now().Add(sessionExpiryTime).Format(time.RFC3339)
68+ if err := sess.Save(c.Request(), c.Response()); err != nil {
69+ logger.Error("Failed to save updated expiry to cookie session", "error", err, "sessionID", sessionID)
5270 }
5371
54- logger.Debug("Session extended successfully", "accountID", accountID, "sessionID", sessionID)
55-
72+ logger.Debug("Session validated and extended successfully", "accountID", accountID, "sessionID", sessionID)
5673 return nil
5774 }
5875
59-func IsSessionExpiredInDB(a *server.App, accountID string, sessionID string) (bool, error) {
60- currentTime := time.Now()
61- expirationTime, err := queries.GetSessionExpiration(a.DB, accountID, sessionID, currentTime)
76+func IsSessionValidInDB(a *server.App, sessionID string) (bool, error) {
77+ var dbExpiresStr string
78+ var isOnline bool
79+
80+ query := "SELECT expires, is_online FROM sessions WHERE id = ?"
81+ err := a.DB.QueryRow(query, sessionID).Scan(&dbExpiresStr, &isOnline)
82+
6283 if err != nil {
63- logger.Warn("Failed to get session expiration time", "error", err)
84+ if err == sql.ErrNoRows {
85+ logger.Warn("Session not found in DB", "sessionID", sessionID)
86+ return false, nil
87+ }
88+ logger.Error("Failed to query session validity from DB", "error", err, "sessionID", sessionID)
6489 return false, err
6590 }
6691
67- logger.Debug("Session expiration time for account", "accountID", accountID, "sessionID", sessionID, "expirationTime", expirationTime)
68- return currentTime.After(expirationTime), nil
92+ if !isOnline {
93+ logger.Warn("Session found in DB but marked as not online", "sessionID", sessionID)
94+ return false, nil
95+ }
96+
97+ expirationTime, err := time.Parse(time.RFC3339, dbExpiresStr)
98+ if err != nil {
99+ logger.Error("Failed to parse session expiration time from DB", "error", err, "sessionID", sessionID, "expires_str", dbExpiresStr)
100+ return false, fmt.Errorf("invalid expiry format in DB: %w", err)
101+ }
102+
103+ if time.Now().After(expirationTime) {
104+ logger.Warn("Session found in DB but expired", "sessionID", sessionID, "expires_at", expirationTime)
105+ return false, nil
106+ }
107+
108+ return true, nil
69109 }
70110
71111 func IsAccountIDInDB(a *server.App, accountID string) (bool, error) {
72112 accountExists, err := queries.CheckDBAccountExists(a.DB, accountID)
73113 if err != nil {
74- logger.Warn("Failed to check account existence", "error", err)
114+ logger.Error("Failed to check account existence", "error", err, "accountID", accountID)
75115 return false, err
76116 }
77-
78117 logger.Debug("Account existence check", "accountID", accountID, "exists", accountExists)
79-
80118 return accountExists, nil
81119 }
82120
A internal/proxy/acme.go +546 -0
@@ -0,0 +1,546 @@
1+package proxy
2+
3+import (
4+ "crypto"
5+ "crypto/tls"
6+ "database/sql"
7+ "encoding/json"
8+ "fmt"
9+ "os"
10+ "strings"
11+ "time"
12+
13+ "github.com/bnema/gordon/internal/db/queries"
14+ "github.com/bnema/gordon/internal/interfaces"
15+ "github.com/bnema/gordon/pkg/logger"
16+ "github.com/go-acme/lego/v4/certcrypto"
17+ "github.com/go-acme/lego/v4/certificate"
18+ "github.com/go-acme/lego/v4/challenge/http01"
19+ "github.com/go-acme/lego/v4/lego"
20+ "github.com/go-acme/lego/v4/providers/dns"
21+ "github.com/go-acme/lego/v4/registration"
22+)
23+
24+// CertManagerConfig holds configuration for the certificate manager
25+type CertManagerConfig struct {
26+ CertDir string
27+ Email string
28+ Mode string
29+ SkipCertificates bool
30+ BehindTLSProxy bool
31+ AdminDomain string
32+ RootDomain string
33+ HttpPort string
34+}
35+
36+// CertificateManager handles ACME certificate operations
37+type CertificateManager struct {
38+ config CertManagerConfig
39+ client *lego.Client
40+ user *AcmeUser
41+ db *sql.DB
42+ queries *queries.ProxyQueries
43+}
44+
45+// AcmeUser implements the registration.User interface
46+type AcmeUser struct {
47+ Email string `json:"email"`
48+ Registration *registration.Resource `json:"registration"`
49+ key crypto.PrivateKey `json:"-"` // Exclude key from JSON marshal
50+}
51+
52+// GetEmail returns the user's email
53+func (u *AcmeUser) GetEmail() string {
54+ return u.Email
55+}
56+
57+// GetRegistration returns the user's registration
58+func (u *AcmeUser) GetRegistration() *registration.Resource {
59+ return u.Registration
60+}
61+
62+// GetPrivateKey returns the user's private key
63+func (u *AcmeUser) GetPrivateKey() crypto.PrivateKey {
64+ return u.key
65+}
66+
67+// NewCertificateManager creates a new certificate manager
68+func NewCertificateManager(config CertManagerConfig, app interfaces.AppInterface, queries *queries.ProxyQueries) (*CertificateManager, error) {
69+ logger.Debug("Initializing certificate manager", "config", config)
70+
71+ db := app.GetDB() // Get DB connection
72+
73+ // --- User Loading/Creation ---
74+ user, err := loadAcmeUserFromDB(config.Email, db, queries)
75+
76+ if err != nil {
77+ // Check if the error is "user not found"
78+ if err == sql.ErrNoRows {
79+ logger.Info("No existing ACME user found in DB, creating new one.", "email", config.Email)
80+
81+ // Create a new key
82+ privateKey, keyErr := certcrypto.GeneratePrivateKey(certcrypto.RSA2048)
83+ if keyErr != nil {
84+ return nil, fmt.Errorf("failed to generate private key: %w", keyErr)
85+ }
86+
87+ user = &AcmeUser{
88+ Email: config.Email,
89+ key: privateKey,
90+ // Registration will be nil initially
91+ }
92+
93+ // Save the new user (key only for now, registration later)
94+ if saveErr := saveAcmeUserToDB(user, db, queries); saveErr != nil {
95+ // Log the error but proceed, registration might fix it or fail later
96+ logger.Error("Failed to save newly generated ACME user key to DB", "error", saveErr)
97+ // Depending on policy, might want to return error here:
98+ // return nil, fmt.Errorf("failed to save initial ACME user: %w", saveErr)
99+ }
100+ } else {
101+ // Handle other DB errors during load
102+ return nil, fmt.Errorf("failed to load ACME user from DB: %w", err)
103+ }
104+ } else {
105+ logger.Debug("Loaded existing ACME user from DB", "email", user.Email)
106+ }
107+ // --- End User Loading/Creation ---
108+
109+ // Create the lego config using the loaded/created user
110+ legoConfig := lego.NewConfig(user)
111+ legoConfig.Certificate.KeyType = certcrypto.RSA2048 // Match key type used
112+
113+ // Set the server based on mode
114+ if config.Mode == "staging" {
115+ logger.Info("Using Let's Encrypt staging environment")
116+ legoConfig.CADirURL = lego.LEDirectoryStaging
117+ } else {
118+ logger.Info("Using Let's Encrypt production environment")
119+ legoConfig.CADirURL = lego.LEDirectoryProduction
120+ }
121+
122+ // Create the lego client
123+ client, err := lego.NewClient(legoConfig)
124+ if err != nil {
125+ return nil, fmt.Errorf("failed to create lego client: %w", err)
126+ }
127+
128+ // --- ACME User Registration ---
129+ if user.Registration == nil {
130+ logger.Info("ACME user registration not found, attempting to register", "email", user.Email)
131+ reg, err := client.Registration.Register(registration.RegisterOptions{TermsOfServiceAgreed: true})
132+ if err != nil {
133+ // Check if it's an "already registered" error, which is fine
134+ // Note: Lego might handle this implicitly, but explicit check is safer.
135+ // The exact error type/message might vary depending on lego version and ACME server.
136+ // We might need to refine this error check based on observed behavior.
137+ if strings.Contains(err.Error(), "urn:ietf:params:acme:error:accountDoesNotExist") {
138+ // This might indicate a mismatch between loaded key and LE's record.
139+ // We might need to re-create the key and re-register, or query the account by key.
140+ // For now, just log and return error.
141+ logger.Error("ACME account registration failed: Account does not exist (key mismatch?)", "error", err)
142+ return nil, fmt.Errorf("failed to register ACME user (account not found): %w", err)
143+ } else if strings.Contains(err.Error(), "user already exists") { // Example, adjust as needed
144+ logger.Info("ACME user is already registered with the server.")
145+ // Try to resolve the account details if possible
146+ // reg, err = client.Registration.ResolveAccountByKey() // Requires lego client supporting this
147+ // if err != nil {
148+ // logger.Error("Failed to resolve existing ACME account by key", "error", err)
149+ // return nil, fmt.Errorf("failed to resolve existing ACME account: %w", err)
150+ // }
151+ // For now, let's assume if registration failed with "exists", we might need manual intervention
152+ // or a more sophisticated recovery mechanism. Returning error for safety.
153+ // If Obtain works despite this, we can reconsider.
154+ logger.Error("ACME user registration failed, but server indicates user exists. Manual check might be needed.", "error", err)
155+ return nil, fmt.Errorf("failed to register ACME user (already exists conflict): %w", err)
156+ } else {
157+ // Handle other registration errors
158+ logger.Error("ACME user registration failed", "error", err)
159+ return nil, fmt.Errorf("failed to register ACME user: %w", err)
160+ }
161+ } else {
162+ logger.Info("ACME user registered successfully", "uri", reg.URI)
163+ user.Registration = reg // Store the registration resource
164+
165+ // Save the user again, now with registration info
166+ if saveErr := saveAcmeUserToDB(user, db, queries); saveErr != nil {
167+ logger.Error("Failed to save ACME user registration info to DB", "error", saveErr)
168+ // Continue, as registration itself succeeded, but log the failure
169+ } else {
170+ logger.Debug("Saved ACME user registration info to DB")
171+ }
172+ }
173+ } else {
174+ logger.Debug("Existing ACME user registration loaded from DB", "uri", user.Registration.URI)
175+ // Optional: Could add a check here to ensure the registration is still valid
176+ }
177+ // --- End ACME User Registration ---
178+
179+ // Create the certificate manager struct
180+ cm := &CertificateManager{
181+ config: config,
182+ user: user,
183+ client: client,
184+ db: db,
185+ queries: queries,
186+ }
187+
188+ return cm, nil
189+}
190+
191+// loadAcmeUserFromDB loads the user's private key and registration info from the DB
192+func loadAcmeUserFromDB(email string, db *sql.DB, queries *queries.ProxyQueries) (*AcmeUser, error) {
193+ var privateKeyPEM string
194+ var registrationJSON sql.NullString
195+
196+ err := db.QueryRow(queries.GetAcmeAccountByEmail, email).Scan(&privateKeyPEM, &registrationJSON)
197+ if err != nil {
198+ if err == sql.ErrNoRows {
199+ logger.Debug("ACME user not found in DB", "email", email)
200+ return nil, sql.ErrNoRows // Return specific error
201+ }
202+ return nil, fmt.Errorf("failed to query ACME user: %w", err)
203+ }
204+
205+ // Parse the private key
206+ privateKey, err := certcrypto.ParsePEMPrivateKey([]byte(privateKeyPEM))
207+ if err != nil {
208+ return nil, fmt.Errorf("failed to parse private key from DB: %w", err)
209+ }
210+
211+ user := &AcmeUser{
212+ Email: email,
213+ key: privateKey,
214+ }
215+
216+ // Load the registration info if it exists
217+ if registrationJSON.Valid && registrationJSON.String != "" {
218+ // Need to unmarshal into a registration.Resource pointer
219+ reg := &registration.Resource{}
220+ err = json.Unmarshal([]byte(registrationJSON.String), reg)
221+ if err != nil {
222+ // Log the error but potentially continue without registration? Or return error?
223+ // Returning error is safer.
224+ logger.Error("Failed to unmarshal ACME registration info from DB", "email", email, "error", err)
225+ return nil, fmt.Errorf("failed to unmarshal registration info: %w", err)
226+ }
227+ user.Registration = reg // Assign the pointer
228+ logger.Debug("Loaded ACME user registration info from DB", "email", email, "uri", user.Registration.URI)
229+ } else {
230+ logger.Debug("ACME user registration info not found or null in DB", "email", email)
231+ }
232+
233+ return user, nil
234+}
235+
236+// saveAcmeUserToDB saves the user's private key and registration info to the DB
237+func saveAcmeUserToDB(user *AcmeUser, db *sql.DB, queries *queries.ProxyQueries) error {
238+ if user.key == nil {
239+ return fmt.Errorf("cannot save user without a private key")
240+ }
241+
242+ // Encode private key to PEM
243+ privateKeyPEM := certcrypto.PEMEncode(user.key)
244+
245+ // Marshal registration info to JSON (handle nil)
246+ var registrationJSON sql.NullString
247+ if user.Registration != nil {
248+ regBytes, err := json.Marshal(user.Registration)
249+ if err != nil {
250+ return fmt.Errorf("failed to marshal registration info: %w", err)
251+ }
252+ registrationJSON = sql.NullString{String: string(regBytes), Valid: true}
253+ }
254+
255+ now := time.Now().Format(time.RFC3339)
256+
257+ // Use InsertOrReplace query
258+ _, err := db.Exec(queries.InsertOrReplaceAcmeAccount,
259+ user.Email,
260+ string(privateKeyPEM),
261+ registrationJSON, // Pass the NullString directly
262+ now, // created_at (will be updated on replace)
263+ now, // updated_at
264+ )
265+
266+ if err != nil {
267+ return fmt.Errorf("failed to save ACME user to DB: %w", err)
268+ }
269+
270+ logger.Debug("Saved ACME user details to DB", "email", user.Email)
271+ return nil
272+}
273+
274+// SetupChallengeProvider sets up the appropriate challenge provider for a domain
275+func (cm *CertificateManager) SetupChallengeProvider(domainName string, proxy *Proxy) error {
276+ // Get domain configuration from database
277+ var domain struct {
278+ AcmeEnabled bool
279+ AcmeChallengeType sql.NullString
280+ AcmeDnsProvider sql.NullString
281+ AcmeDnsCredentialsRef sql.NullString
282+ }
283+
284+ err := proxy.app.GetDB().QueryRow(`
285+ SELECT acme_enabled, acme_challenge_type, acme_dns_provider, acme_dns_credentials_ref
286+ FROM domain WHERE name = ?
287+ `, domainName).Scan(&domain.AcmeEnabled, &domain.AcmeChallengeType, &domain.AcmeDnsProvider, &domain.AcmeDnsCredentialsRef)
288+
289+ if err != nil {
290+ return fmt.Errorf("failed to get domain configuration: %w", err)
291+ }
292+
293+ // Determine challenge type
294+ challengeType := domain.AcmeChallengeType.String
295+ if !domain.AcmeChallengeType.Valid {
296+ challengeType = proxy.config.DefaultChallengeType
297+ }
298+
299+ // Set up the appropriate challenge provider
300+ switch challengeType {
301+ case "http-01":
302+ // Use the configured HTTP challenge port
303+ port := proxy.config.DefaultHttpChallengePort
304+ if port == "" {
305+ port = "80"
306+ }
307+ provider := http01.NewProviderServer("", port)
308+ if err := cm.client.Challenge.SetHTTP01Provider(provider); err != nil {
309+ return fmt.Errorf("failed to set HTTP-01 provider: %w", err)
310+ }
311+
312+ case "dns-01":
313+ if domain.AcmeDnsProvider.String == "" {
314+ return fmt.Errorf("DNS provider not configured for domain %s", domainName)
315+ }
316+
317+ // Get DNS provider credentials from environment
318+ creds := cm.getDNSProviderCredentials(domain.AcmeDnsCredentialsRef.String)
319+ if creds == nil {
320+ return fmt.Errorf("DNS provider credentials not found for domain %s", domainName)
321+ }
322+
323+ // Create and configure the DNS provider
324+ provider, err := dns.NewDNSChallengeProviderByName(domain.AcmeDnsProvider.String)
325+ if err != nil {
326+ return fmt.Errorf("failed to create DNS provider: %w", err)
327+ }
328+
329+ // Configure the provider with credentials
330+ if err := provider.Present(domainName, "", ""); err != nil {
331+ return fmt.Errorf("failed to configure DNS provider: %w", err)
332+ }
333+
334+ // Set DNS timeouts
335+ if dnsProvider, ok := provider.(interface {
336+ SetPropagationTimeout(time.Duration)
337+ SetPollingInterval(time.Duration)
338+ }); ok {
339+ dnsProvider.SetPropagationTimeout(
340+ time.Duration(proxy.config.DefaultDnsChallengePropagationTimeout) * time.Second,
341+ )
342+ dnsProvider.SetPollingInterval(
343+ time.Duration(proxy.config.DefaultDnsChallengePollingInterval) * time.Second,
344+ )
345+ }
346+
347+ if err := cm.client.Challenge.SetDNS01Provider(provider); err != nil {
348+ return fmt.Errorf("failed to set DNS-01 provider: %w", err)
349+ }
350+
351+ default:
352+ return fmt.Errorf("unsupported challenge type: %s", challengeType)
353+ }
354+
355+ return nil
356+}
357+
358+// GetKeyAuthorization returns the key authorization for a token
359+func (cm *CertificateManager) GetKeyAuthorization(token string) (string, error) {
360+ // Create a new HTTP-01 provider server
361+ provider := http01.NewProviderServer("", cm.config.HttpPort)
362+
363+ // Present the challenge
364+ err := provider.Present("", token, "")
365+ if err != nil {
366+ return "", fmt.Errorf("failed to present challenge: %w", err)
367+ }
368+
369+ // Get the key authorization from the provider
370+ path := http01.ChallengePath(token)
371+ return path, nil
372+}
373+
374+// ObtainCertificate obtains a new certificate for a domain
375+func (cm *CertificateManager) ObtainCertificate(domainName string, proxy *Proxy) (*certificate.Resource, error) {
376+ logger.Debug("Obtaining certificate for domain", "domain", domainName)
377+
378+ // Set up the challenge provider
379+ if err := cm.SetupChallengeProvider(domainName, proxy); err != nil {
380+ return nil, fmt.Errorf("failed to set up challenge provider: %w", err)
381+ }
382+
383+ // Request the certificate
384+ request := certificate.ObtainRequest{
385+ Domains: []string{domainName},
386+ Bundle: true,
387+ }
388+
389+ cert, err := cm.client.Certificate.Obtain(request)
390+ if err != nil {
391+ return nil, fmt.Errorf("failed to obtain certificate: %w", err)
392+ }
393+
394+ // Update the certificate in the database
395+ if err := cm.updateCertificateInDB(domainName, cert, proxy); err != nil {
396+ logger.Error("Failed to update certificate in database", "error", err)
397+ }
398+
399+ return cert, nil
400+}
401+
402+// updateCertificateInDB updates the certificate information in the database
403+func (cm *CertificateManager) updateCertificateInDB(domainName string, cert *certificate.Resource, proxy *Proxy) error {
404+ // Get the domain ID
405+ var domainID string
406+ err := proxy.app.GetDB().QueryRow("SELECT id FROM domain WHERE name = ?", domainName).Scan(&domainID)
407+ if err != nil {
408+ return fmt.Errorf("failed to get domain ID: %w", err)
409+ }
410+
411+ // Parse the certificate to get expiry date
412+ x509Cert, err := certcrypto.ParsePEMCertificate(cert.Certificate)
413+ if err != nil {
414+ return fmt.Errorf("failed to parse certificate: %w", err)
415+ }
416+
417+ // Update or insert the certificate
418+ _, err = proxy.app.GetDB().Exec(`
419+ INSERT OR REPLACE INTO certificate (
420+ id, domain_id, cert_file, key_file, issued_at, expires_at, issuer, status
421+ ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
422+ `,
423+ cert.Domain,
424+ domainID,
425+ string(cert.Certificate),
426+ string(cert.PrivateKey),
427+ time.Now().Format(time.RFC3339),
428+ x509Cert.NotAfter.Format(time.RFC3339),
429+ "Let's Encrypt",
430+ "valid",
431+ )
432+
433+ if err != nil {
434+ return fmt.Errorf("failed to update certificate in database: %w", err)
435+ }
436+
437+ return nil
438+}
439+
440+// RenewCertificate renews an existing certificate
441+func (cm *CertificateManager) RenewCertificate(domainName string, proxy *Proxy) (*certificate.Resource, error) {
442+ logger.Debug("Renewing certificate for domain", "domain", domainName)
443+
444+ // Set up the challenge provider
445+ if err := cm.SetupChallengeProvider(domainName, proxy); err != nil {
446+ return nil, fmt.Errorf("failed to set up challenge provider: %w", err)
447+ }
448+
449+ // Get the existing certificate from the database
450+ var certFile, keyFile string
451+ err := proxy.app.GetDB().QueryRow(`
452+ SELECT cert_file, key_file FROM certificate c
453+ JOIN domain d ON c.domain_id = d.id
454+ WHERE d.name = ?
455+ `, domainName).Scan(&certFile, &keyFile)
456+ if err != nil {
457+ return nil, fmt.Errorf("failed to get existing certificate: %w", err)
458+ }
459+
460+ // Create certificate resource for renewal
461+ cert := &certificate.Resource{
462+ Domain: domainName,
463+ Certificate: []byte(certFile),
464+ PrivateKey: []byte(keyFile),
465+ }
466+
467+ // Renew the certificate
468+ _, err = cm.client.Certificate.Renew(*cert, true, false, "")
469+ if err != nil {
470+ return nil, fmt.Errorf("failed to renew certificate: %w", err)
471+ }
472+
473+ // Update the certificate in the database
474+ if err := cm.updateCertificateInDB(domainName, cert, proxy); err != nil {
475+ logger.Error("Failed to update certificate in database", "error", err)
476+ }
477+
478+ return cert, nil
479+}
480+
481+// GetCertificateForTLS returns a certificate for TLS handshake
482+func (cm *CertificateManager) GetCertificateForTLS(hello *tls.ClientHelloInfo) (*tls.Certificate, error) {
483+ logger.Debug("Getting certificate for TLS handshake", "server_name", hello.ServerName)
484+
485+ // Get the certificate resource
486+ cert, err := cm.GetCertificate(hello.ServerName)
487+ if err != nil {
488+ return nil, fmt.Errorf("failed to get certificate: %w", err)
489+ }
490+
491+ // Convert to tls.Certificate
492+ tlsCert, err := tls.X509KeyPair(cert.Certificate, cert.PrivateKey)
493+ if err != nil {
494+ return nil, fmt.Errorf("failed to parse certificate: %w", err)
495+ }
496+
497+ return &tlsCert, nil
498+}
499+
500+// GetCertificate retrieves a certificate from the database
501+func (cm *CertificateManager) GetCertificate(domainName string) (*certificate.Resource, error) {
502+ // Try to obtain the certificate if it doesn't exist
503+ request := certificate.ObtainRequest{
504+ Domains: []string{domainName},
505+ Bundle: true,
506+ }
507+ cert, err := cm.client.Certificate.Obtain(request)
508+ if err != nil {
509+ return nil, fmt.Errorf("failed to obtain certificate: %w", err)
510+ }
511+ return cert, nil
512+}
513+
514+// GetAutocertManager returns the autocert manager
515+func (cm *CertificateManager) GetAutocertManager() *tls.Config {
516+ return &tls.Config{
517+ GetCertificate: cm.GetCertificateForTLS,
518+ MinVersion: tls.VersionTLS12,
519+ }
520+}
521+
522+// getDNSProviderCredentials retrieves DNS provider credentials from environment variables
523+func (cm *CertificateManager) getDNSProviderCredentials(ref string) map[string]string {
524+ if ref == "" {
525+ return nil
526+ }
527+
528+ creds := make(map[string]string)
529+ prefix := fmt.Sprintf("GORDON_DNS_CRED_%s_", ref)
530+
531+ for _, env := range os.Environ() {
532+ if strings.HasPrefix(env, prefix) {
533+ parts := strings.SplitN(env, "=", 2)
534+ if len(parts) == 2 {
535+ key := strings.TrimPrefix(parts[0], prefix)
536+ creds[key] = parts[1]
537+ }
538+ }
539+ }
540+
541+ if len(creds) == 0 {
542+ return nil
543+ }
544+
545+ return creds
546+}
D internal/proxy/certificates.go +0 -2280
@@ -1,2280 +0,0 @@
1-package proxy
2-
3-import (
4- "context"
5- "crypto/ecdsa"
6- "crypto/elliptic"
7- "crypto/rand"
8- "crypto/tls"
9- "crypto/x509"
10- "crypto/x509/pkix"
11- "encoding/pem"
12- "fmt"
13- "math/big"
14- "net"
15- "net/http"
16- "os"
17- "path/filepath"
18- "regexp"
19- "strings"
20- "time"
21-
22- "github.com/bnema/gordon/pkg/logger"
23- "golang.org/x/crypto/acme"
24- "golang.org/x/crypto/acme/autocert"
25-)
26-
27-// Certificate naming patterns
28-const (
29- SelfSignedCertPattern = "%s_self-signed"
30- LetsEncryptStagingPattern = "%s_letsencrypt-staging"
31- LetsEncryptProdPattern = "%s_letsencrypt-production"
32-)
33-
34-// detectUpstreamProxy checks for signs of an upstream TLS-terminating proxy
35-func (p *Proxy) detectUpstreamProxy() bool {
36- // Check if detection is explicitly disabled
37- if !p.config.DetectUpstreamProxy {
38- return false
39- }
40-
41- // First check our cached detection status - if we already detected a proxy previously
42- p.upstreamProxyMu.RLock()
43- alreadyDetected := p.upstreamProxyDetected
44- p.upstreamProxyMu.RUnlock()
45-
46- if alreadyDetected {
47- return true
48- }
49-
50- // Check for known HTTP headers that indicate a TLS termination proxy
51- // This is a best-effort approach since we're analyzing HTTP headers from
52- // previous requests stored in our context (if available)
53-
54- // Look for common proxy headers in our recent requests
55- // This is a static check at startup; ideally we would check incoming
56- // requests dynamically, but for this implementation we're using
57- // known proxy environment indicators
58-
59- // Most common proxy headers: X-Forwarded-Proto, X-Forwarded-SSL, X-Forwarded-For
60- // Environment variables: HTTPS=on, HTTP_X_FORWARDED_PROTO=https
61-
62- // Check for common environment variables set by upstream proxies
63- if os.Getenv("HTTPS") == "on" ||
64- os.Getenv("HTTP_X_FORWARDED_PROTO") == "https" ||
65- os.Getenv("HTTP_X_FORWARDED_SSL") == "on" {
66- logger.Info("Upstream TLS-terminating proxy detected via environment variables",
67- "detection_method", "environment_variables",
68- "https", os.Getenv("HTTPS"),
69- "x_forwarded_proto", os.Getenv("HTTP_X_FORWARDED_PROTO"),
70- "x_forwarded_ssl", os.Getenv("HTTP_X_FORWARDED_SSL"))
71-
72- // Cache this result
73- p.upstreamProxyMu.Lock()
74- p.upstreamProxyDetected = true
75- p.upstreamProxyMu.Unlock()
76-
77- return true
78- }
79-
80- // Check if Cloudflare headers are present in environment
81- // Cloudflare sets environment variables like HTTP_CF_VISITOR={"scheme":"https"}
82- if cfVisitor := os.Getenv("HTTP_CF_VISITOR"); cfVisitor != "" {
83- if strings.Contains(cfVisitor, "\"scheme\":\"https\"") {
84- logger.Info("Cloudflare proxy detected via environment variables",
85- "detection_method", "cloudflare_headers",
86- "cf_visitor", cfVisitor)
87-
88- // Cache this result
89- p.upstreamProxyMu.Lock()
90- p.upstreamProxyDetected = true
91- p.upstreamProxyMu.Unlock()
92-
93- return true
94- }
95- }
96-
97- // Not detected, but log if explicit configuration is set
98- if p.config.SkipCertificates {
99- logger.Info("No upstream proxy detected automatically, but certificate acquisition will be skipped due to configuration",
100- "detection_result", "false",
101- "skip_certificates", "true (forced via config)")
102- }
103-
104- return false
105-}
106-
107-// setupCertManager configures the autocert manager for Let's Encrypt
108-func (p *Proxy) setupCertManager() {
109- // First check for upstream TLS-terminating proxies if detection is enabled
110- isProxied := p.detectUpstreamProxy()
111-
112- // Get admin domain and root domain
113- adminDomain := p.app.GetConfig().Http.FullDomain()
114- rootDomain := p.app.GetConfig().Http.Domain
115-
116- // Generate a fallback certificate for admin domain regardless of proxy status
117- var fallbackDomains []string
118- fallbackDomains = append(fallbackDomains, adminDomain)
119-
120- // Include root domain in fallback certificate if it's different from admin domain
121- if rootDomain != "" && rootDomain != adminDomain {
122- fallbackDomains = append(fallbackDomains, rootDomain)
123- }
124-
125- // Generate the fallback certificate
126- fallbackCert, err := generateFallbackCertificates(fallbackDomains)
127- if err != nil {
128- logger.Warn("Failed to generate fallback certificate", "error", err)
129- } else {
130- logger.Info("Generated fallback self-signed certificate for admin domain",
131- "domain", adminDomain,
132- "valid_until", time.Now().Add(24*time.Hour).Format("2006-01-02 15:04:05"))
133- // Store the fallback certificate
134- p.fallbackCert = fallbackCert
135- }
136-
137- // Skip certificate acquisition if we're behind a TLS-terminating proxy
138- // and the skip certificates option is enabled
139- if isProxied && p.config.SkipCertificates {
140- logger.Info("Upstream TLS-terminating proxy detected, skipping Let's Encrypt certificate acquisition",
141- "upstream_proxy", "detected",
142- "certificate_acquisition", "disabled")
143-
144- // We will still initialize the certificate manager but will use it in a more limited way
145- logger.Info("Initializing minimal Let's Encrypt certificate manager for fallback purposes")
146- }
147-
148- // Create a cache directory if it doesn't exist
149- dir := p.config.CertDir
150- if dir == "" {
151- dir = p.app.GetConfig().General.StorageDir + "/certs"
152- }
153-
154- // Ensure directory exists
155- if err := os.MkdirAll(dir, 0755); err != nil {
156- logger.Warn("Failed to create certificate cache directory",
157- "dir", dir,
158- "error", err)
159- }
160-
161- // Set up the certificate manager
162- certManager := &autocert.Manager{
163- Prompt: autocert.AcceptTOS,
164- Cache: autocert.DirCache(dir),
165- }
166-
167- // Configure the email if provided
168- if p.config.Email != "" {
169- certManager.Email = p.config.Email
170- }
171-
172- // Configure the Let's Encrypt client
173- if p.config.LetsEncryptMode == "staging" {
174- certManager.Client = &acme.Client{
175- DirectoryURL: "https://acme-staging-v02.api.letsencrypt.org/directory",
176- }
177- logger.Debug("Using Let's Encrypt staging environment",
178- "url", "https://acme-staging-v02.api.letsencrypt.org/directory")
179- } else {
180- // Explicitly set production URL when not in staging mode
181- certManager.Client = &acme.Client{
182- DirectoryURL: acme.LetsEncryptURL, // "https://acme-v02.api.letsencrypt.org/directory"
183- }
184- logger.Debug("Using Let's Encrypt production environment",
185- "url", acme.LetsEncryptURL)
186- }
187-
188- // Use a stricter hostpolicy that only allows domains in our routes
189- certManager.HostPolicy = func(_ context.Context, host string) error {
190- // Always allow the admin domain and root domain
191- if host == adminDomain || host == rootDomain {
192- return nil
193- }
194-
195- // For other domains, check if they are in our routes
196- p.mu.RLock()
197- defer p.mu.RUnlock()
198-
199- if _, ok := p.routes[host]; ok {
200- return nil
201- }
202-
203- // Log the attempt and reject it
204- logger.Warn("Rejecting certificate request for unknown host",
205- "host", host,
206- "adminDomain", adminDomain,
207- "allowed", "no")
208-
209- // Return an error for domains not in our routes
210- return fmt.Errorf("host %q not configured in routes", host)
211- }
212-
213- p.certManager = certManager
214- logger.Debug("Certificate manager setup completed",
215- "directory", dir,
216- "mode", p.config.LetsEncryptMode)
217-
218- // Preload certificates from disk into cache to avoid unnecessary Let's Encrypt requests
219- p.preloadCertificates()
220-
221- // Only request admin certificate during server startup, not during specific domain processing
222- if !p.processingSpecificDomain {
223- // Request the certificate for the admin domain
224- go p.requestAdminCertificate()
225- }
226-}
227-
228-// checkCertificateInCache checks if a valid certificate for the given domain exists in the cache
229-// Returns true if a valid certificate exists, false otherwise
230-func (p *Proxy) checkCertificateInCache(domain string) (bool, string) {
231- if p.certManager == nil || p.certManager.Cache == nil {
232- return false, ""
233- }
234-
235- // The cache key format used by autocert is "cert-" + domain
236- cacheKey := "cert-" + domain
237-
238- // Create a context with a short timeout for the cache check
239- ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
240- defer cancel()
241-
242- // Try to get the certificate from the cache
243- certData, err := p.certManager.Cache.Get(ctx, cacheKey)
244- if err != nil {
245- // ErrCacheMiss or other error means the certificate is not in the cache
246- logger.Debug("Certificate not found in cache",
247- "domain", domain,
248- "error", err)
249- return false, ""
250- }
251-
252- // Validate the certificate data
253- validCert, expiresInDays, certType, _ := isCertificateValid(certData, domain)
254- if !validCert {
255- return false, certType
256- }
257-
258- logger.Info("Valid certificate found in cache",
259- "domain", domain,
260- "expires_in", expiresInDays,
261- "days",
262- "type", certType)
263-
264- // Check if certificate matches current mode
265- if p.config.LetsEncryptMode == "production" && certType == "staging" {
266- logger.Info("Found staging certificate but production mode is enabled",
267- "domain", domain,
268- "action", "will request new production certificate")
269- return false, certType
270- } else if p.config.LetsEncryptMode == "staging" && certType == "production" {
271- // We're in staging mode, but we already have a production certificate
272- // That's fine - keep using the production certificate
273- logger.Info("Found production certificate while in staging mode",
274- "domain", domain,
275- "action", "using existing production certificate")
276- }
277-
278- return true, certType
279-}
280-
281-// getCertificateFileName returns the appropriate filename based on certificate type
282-func getCertificateFileName(domain string, certType string) string {
283- switch certType {
284- case "self-signed":
285- return fmt.Sprintf(SelfSignedCertPattern, domain)
286- case "staging":
287- return fmt.Sprintf(LetsEncryptStagingPattern, domain)
288- case "production":
289- return fmt.Sprintf(LetsEncryptProdPattern, domain)
290- default:
291- // Fallback to just the domain name for backward compatibility
292- return domain
293- }
294-}
295-
296-// getCertificateType determines the type of certificate by examining its contents
297-func getCertificateType(cert *x509.Certificate) string {
298- // Check issuer to determine the certificate type
299- issuer := cert.Issuer.CommonName
300-
301- // Check for self-signed certificate
302- if cert.Issuer.CommonName == cert.Subject.CommonName {
303- return "self-signed"
304- }
305-
306- // Check for Let's Encrypt staging certificate
307- if strings.Contains(issuer, "STAGING") ||
308- strings.Contains(issuer, "Fake LE") ||
309- strings.Contains(issuer, "Counterfeit") ||
310- strings.Contains(issuer, "False Fennel") {
311- return "staging"
312- }
313-
314- // Check for Let's Encrypt production certificate
315- if strings.Contains(issuer, "Let's Encrypt") ||
316- strings.Contains(issuer, "R3") ||
317- strings.Contains(issuer, "E1") {
318- return "production"
319- }
320-
321- // Unknown issuer
322- return "unknown"
323-}
324-
325-// isCertificateValid checks if a certificate PEM data is valid and not expired
326-// Returns whether it's valid and how many days until expiration, and the certificate type
327-func isCertificateValid(certData []byte, domain string) (bool, float64, string, *x509.Certificate) {
328- // Parse the certificate to check its validity
329- block, rest := pem.Decode(certData)
330- if block == nil || block.Type != "CERTIFICATE" {
331- logger.Warn("Invalid certificate data - not a valid PEM CERTIFICATE block",
332- "domain", domain,
333- "dataLength", len(certData))
334-
335- // Debug: output first few bytes to help diagnose format issues
336- if len(certData) > 0 {
337- previewLen := 50
338- if len(certData) < previewLen {
339- previewLen = len(certData)
340- }
341- preview := string(certData[:previewLen])
342- logger.Debug("Certificate data preview",
343- "domain", domain,
344- "preview", preview+"...")
345- }
346- return false, 0, "unknown", nil
347- }
348-
349- // Check if there are additional certificates in the chain
350- var certChain []*x509.Certificate
351- currentRest := rest
352-
353- for len(currentRest) > 0 {
354- var nextBlock *pem.Block
355- nextBlock, currentRest = pem.Decode(currentRest)
356- if nextBlock == nil || nextBlock.Type != "CERTIFICATE" {
357- break
358- }
359- cert, err := x509.ParseCertificate(nextBlock.Bytes)
360- if err != nil {
361- logger.Debug("Failed to parse certificate in chain",
362- "domain", domain,
363- "error", err)
364- continue
365- }
366- certChain = append(certChain, cert)
367- }
368-
369- if len(certChain) > 0 {
370- logger.Debug("Found certificate chain with additional certificates",
371- "domain", domain,
372- "chain_length", len(certChain))
373- }
374-
375- cert, err := x509.ParseCertificate(block.Bytes)
376- if err != nil {
377- logger.Warn("Failed to parse certificate",
378- "domain", domain,
379- "error", err)
380- return false, 0, "unknown", nil
381- }
382-
383- // Verify the certificate includes the domain we're checking for
384- var domainFound bool
385- if len(cert.DNSNames) > 0 {
386- logger.Debug("Certificate DNS names",
387- "domain", domain,
388- "certificate_domains", cert.DNSNames)
389-
390- for _, dnsName := range cert.DNSNames {
391- if dnsName == domain {
392- domainFound = true
393- break
394- }
395- }
396- }
397-
398- if !domainFound && cert.Subject.CommonName != domain {
399- logger.Warn("Certificate doesn't match the required domain",
400- "domain", domain,
401- "cert_common_name", cert.Subject.CommonName,
402- "cert_dns_names", cert.DNSNames)
403- return false, 0, "unknown", cert
404- }
405-
406- // Determine the certificate type
407- certType := getCertificateType(cert)
408- logger.Debug("Determined certificate type",
409- "domain", domain,
410- "type", certType,
411- "issuer", cert.Issuer.CommonName)
412-
413- // Check if the certificate is still valid
414- now := time.Now()
415- if now.After(cert.NotAfter) || now.Before(cert.NotBefore) {
416- logger.Info("Certificate has expired or is not yet valid",
417- "domain", domain,
418- "not_before", cert.NotBefore,
419- "not_after", cert.NotAfter,
420- "type", certType)
421- return false, 0, certType, cert
422- }
423-
424- // Calculate days until expiration
425- expiresInDays := cert.NotAfter.Sub(now).Hours() / 24
426-
427- // Check if the certificate is about to expire (within 30 days)
428- if expiresInDays < 30 {
429- logger.Info("Certificate is valid but will expire soon",
430- "domain", domain,
431- "expires_in", expiresInDays,
432- "days",
433- "type", certType)
434- // Return false to trigger renewal if it's about to expire
435- return false, expiresInDays, certType, cert
436- }
437-
438- return true, expiresInDays, certType, cert
439-}
440-
441-// checkCertificateInFilesystem checks if valid certificate files exist directly in the certs directory
442-// This is useful when the database record might be missing but valid certificate files exist
443-func (p *Proxy) checkCertificateInFilesystem(domain string) (bool, string) {
444- if p.certManager == nil {
445- return false, ""
446- }
447-
448- // Get the certificate directory path
449- certDir := p.config.CertDir
450- if certDir == "" {
451- certDir = p.app.GetConfig().General.StorageDir + "/certs"
452- }
453-
454- // First check for known certificate patterns
455- certFiles := []string{
456- // Standard autocert naming
457- domain,
458- // Our naming patterns
459- getCertificateFileName(domain, "self-signed"),
460- getCertificateFileName(domain, "staging"),
461- getCertificateFileName(domain, "production"),
462- }
463-
464- // Check each possible certificate file using known patterns
465- for _, baseName := range certFiles {
466- certFile := filepath.Join(certDir, baseName)
467- keyFile := filepath.Join(certDir, baseName+"+rsa")
468-
469- // Skip if either file doesn't exist
470- if _, err := os.Stat(certFile); os.IsNotExist(err) {
471- continue
472- }
473-
474- if _, err := os.Stat(keyFile); os.IsNotExist(err) {
475- continue
476- }
477-
478- // If we found both files, validate the certificate
479- certData, err := os.ReadFile(certFile)
480- if err != nil {
481- logger.Warn("Failed to read certificate file",
482- "domain", domain,
483- "file", certFile,
484- "error", err)
485- continue
486- }
487-
488- // Validate the certificate data
489- validCert, expiresInDays, certType, _ := isCertificateValid(certData, domain)
490- if !validCert {
491- continue
492- }
493-
494- logger.Info("Valid certificate found in filesystem with standard naming pattern",
495- "domain", domain,
496- "expires_in", expiresInDays,
497- "days",
498- "type", certType,
499- "cert_file", certFile)
500-
501- // Check if certificate matches current mode
502- if p.config.LetsEncryptMode == "production" && certType == "staging" {
503- logger.Info("Found staging certificate but production mode is enabled",
504- "domain", domain,
505- "action", "will request new production certificate")
506- // Delete the staging certificate
507- deleteCertificateFiles(certDir, baseName)
508- return false, certType
509- } else if p.config.LetsEncryptMode == "staging" && certType == "production" {
510- // We're in staging mode, but we already have a production certificate
511- // That's fine - keep using the production certificate
512- logger.Info("Found production certificate while in staging mode",
513- "domain", domain,
514- "action", "using existing production certificate")
515- }
516-
517- // Log some debug information about the certificate files
518- certFileInfo, _ := os.Stat(certFile)
519- keyFileInfo, _ := os.Stat(keyFile)
520- logger.Debug("Certificate file details",
521- "domain", domain,
522- "cert_file", certFile,
523- "cert_size", certFileInfo.Size(),
524- "cert_modified", certFileInfo.ModTime(),
525- "key_file", keyFile,
526- "key_size", keyFileInfo.Size(),
527- "key_modified", keyFileInfo.ModTime())
528-
529- // Attempt to add to autocert cache if it's not already there
530- if certType == "production" || (certType == "staging" && p.config.LetsEncryptMode == "staging") {
531- // Add to autocert cache for future use
532- p.addCertificateToCache(domain, certFile, keyFile)
533- }
534-
535- return true, certType
536- }
537-
538- // If no matches with standard patterns, try a glob search for any files starting with the domain name
539- // This will catch files with any suffix like domain_letsencrypt-production
540- // and domain_letsencrypt-staging
541- logger.Debug("No standard certificate files found, searching for any matching certificates",
542- "domain", domain)
543-
544- files, err := filepath.Glob(filepath.Join(certDir, domain+"*"))
545- if err != nil {
546- logger.Warn("Error searching for certificate files",
547- "domain", domain,
548- "error", err)
549- return false, ""
550- }
551-
552- // Extract base names without the path
553- var candidates []string
554- for _, file := range files {
555- // Skip key files ending with +rsa since we'll process them together with certs
556- if strings.HasSuffix(file, "+rsa") {
557- continue
558- }
559-
560- baseFile := filepath.Base(file)
561- // Check if the corresponding key file exists
562- keyFile := file + "+rsa"
563- if _, err := os.Stat(keyFile); os.IsNotExist(err) {
564- continue
565- }
566-
567- // Add this candidate
568- candidates = append(candidates, baseFile)
569- }
570-
571- // No certificate files found
572- if len(candidates) == 0 {
573- logger.Debug("No certificate files found in filesystem glob search",
574- "domain", domain)
575- return false, ""
576- }
577-
578- logger.Debug("Found certificate candidates in glob search",
579- "domain", domain,
580- "candidates", candidates)
581-
582- // Process each candidate (if there are multiple matches)
583- for _, baseName := range candidates {
584- certFile := filepath.Join(certDir, baseName)
585- keyFile := filepath.Join(certDir, baseName+"+rsa")
586-
587- // Read and validate the certificate
588- certData, err := os.ReadFile(certFile)
589- if err != nil {
590- logger.Warn("Failed to read certificate file from glob search",
591- "domain", domain,
592- "file", certFile,
593- "error", err)
594- continue
595- }
596-
597- // Validate the certificate data
598- validCert, expiresInDays, certType, _ := isCertificateValid(certData, domain)
599- if !validCert {
600- continue
601- }
602-
603- logger.Info("Valid certificate found in filesystem with non-standard naming",
604- "domain", domain,
605- "expires_in", expiresInDays,
606- "days",
607- "type", certType,
608- "cert_file", certFile)
609-
610- // If we're in production mode and this is a production certificate, use it
611- if p.config.LetsEncryptMode == "production" && certType == "production" {
612- // Attempt to add to autocert cache
613- p.addCertificateToCache(domain, certFile, keyFile)
614- return true, certType
615- }
616-
617- // If we're in staging mode, prefer production certificates if available
618- if p.config.LetsEncryptMode == "staging" {
619- if certType == "production" {
620- // Attempt to add to autocert cache
621- p.addCertificateToCache(domain, certFile, keyFile)
622- return true, certType
623- } else if certType == "staging" {
624- // If this is a staging certificate and we're in staging mode, use it
625- // Attempt to add to autocert cache
626- p.addCertificateToCache(domain, certFile, keyFile)
627- return true, certType
628- }
629- }
630-
631- // If this is a staging certificate but we're in production mode, ignore it
632- // and continue checking other candidates
633- if p.config.LetsEncryptMode == "production" && certType == "staging" {
634- logger.Info("Ignoring staging certificate in production mode",
635- "domain", domain,
636- "cert_file", certFile)
637- continue
638- }
639-
640- // Fallback: use the certificate if it's valid
641- // Attempt to add to autocert cache
642- p.addCertificateToCache(domain, certFile, keyFile)
643- return true, certType
644- }
645-
646- logger.Debug("No valid certificate files found in filesystem",
647- "domain", domain)
648- return false, ""
649-}
650-
651-// deleteCertificateFiles removes certificate files from the filesystem
652-func deleteCertificateFiles(certDir string, baseName string) {
653- certFile := filepath.Join(certDir, baseName)
654- keyFile := filepath.Join(certDir, baseName+"+rsa")
655-
656- // Delete certificate file
657- if err := os.Remove(certFile); err != nil && !os.IsNotExist(err) {
658- logger.Warn("Failed to delete certificate file",
659- "file", certFile,
660- "error", err)
661- } else if !os.IsNotExist(err) {
662- logger.Info("Deleted certificate file", "file", certFile)
663- }
664-
665- // Delete key file
666- if err := os.Remove(keyFile); err != nil && !os.IsNotExist(err) {
667- logger.Warn("Failed to delete certificate key file",
668- "file", keyFile,
669- "error", err)
670- } else if !os.IsNotExist(err) {
671- logger.Info("Deleted certificate key file", "file", keyFile)
672- }
673-}
674-
675-// addCertificateToCache adds a certificate found in the filesystem to the autocert cache
676-// This ensures certificates found on disk are also available for the autocert manager
677-func (p *Proxy) addCertificateToCache(domain string, certFile, keyFile string) {
678- // Skip if we don't have a certificate manager
679- if p.certManager == nil || p.certManager.Cache == nil {
680- logger.Warn("Cannot add certificate to cache: certificate manager not initialized",
681- "domain", domain)
682- return
683- }
684-
685- // Create context for cache operations
686- ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
687- defer cancel()
688-
689- // Read the certificate and key files
690- certData, err := os.ReadFile(certFile)
691- if err != nil {
692- logger.Warn("Failed to read certificate file for cache addition",
693- "domain", domain,
694- "file", certFile,
695- "error", err)
696- return
697- }
698-
699- keyData, err := os.ReadFile(keyFile)
700- if err != nil {
701- logger.Warn("Failed to read certificate key file for cache addition",
702- "domain", domain,
703- "file", keyFile,
704- "error", err)
705- return
706- }
707-
708- // Format of cache key used by autocert is "cert-" + domain
709- cacheKey := "cert-" + domain
710-
711- // Check if certificate is already in the cache
712- _, getCacheErr := p.certManager.Cache.Get(ctx, cacheKey)
713- if getCacheErr == nil {
714- // Certificate already exists in cache, no need to add
715- logger.Debug("Certificate already exists in autocert cache, skipping addition",
716- "domain", domain)
717- return
718- }
719-
720- // Add certificate to cache
721- err = p.certManager.Cache.Put(ctx, cacheKey, certData)
722- if err != nil {
723- logger.Warn("Failed to add certificate to autocert cache",
724- "domain", domain,
725- "error", err)
726- return
727- }
728-
729- // Add key to cache with proper key format
730- keyKey := "key-" + domain
731- err = p.certManager.Cache.Put(ctx, keyKey, keyData)
732- if err != nil {
733- logger.Warn("Failed to add certificate key to autocert cache",
734- "domain", domain,
735- "error", err)
736- return
737- }
738-
739- logger.Info("Successfully added certificate to autocert cache",
740- "domain", domain,
741- "from_cert_file", certFile,
742- "from_key_file", keyFile)
743-}
744-
745-// testCertificateWithTLSConnection attempts to establish a TLS connection to verify the certificate
746-// This is an optional verification step for certificates we find on the filesystem
747-func testCertificateWithTLSConnection(domain string, port string) bool {
748- if port == "" {
749- port = "443" // Default HTTPS port
750- }
751-
752- // Create a connection with a short timeout
753- dialer := &net.Dialer{
754- Timeout: 5 * time.Second,
755- }
756-
757- // Connect to the server
758- logger.Debug("Testing certificate by establishing TLS connection",
759- "domain", domain,
760- "address", domain+":"+port)
761-
762- conn, err := tls.DialWithDialer(dialer, "tcp", domain+":"+port, &tls.Config{
763- // Skip verification since we're testing if the connection works
764- // not validating the cert chain (which might use custom CA)
765- InsecureSkipVerify: true,
766- ServerName: domain,
767- })
768-
769- if err != nil {
770- logger.Debug("TLS connection test failed",
771- "domain", domain,
772- "error", err)
773- return false
774- }
775-
776- // Close the connection when done
777- defer conn.Close()
778-
779- // Verify the connection state
780- state := conn.ConnectionState()
781-
782- // Check if the certificate is valid
783- for _, cert := range state.PeerCertificates {
784- // Verify the certificate is valid for the domain
785- if err := cert.VerifyHostname(domain); err != nil {
786- logger.Debug("Certificate hostname verification failed",
787- "domain", domain,
788- "cert_domains", cert.DNSNames,
789- "error", err)
790- continue
791- }
792-
793- // Check certificate expiration
794- now := time.Now()
795- if now.After(cert.NotAfter) || now.Before(cert.NotBefore) {
796- logger.Debug("Certificate from TLS connection is not valid",
797- "domain", domain,
798- "not_before", cert.NotBefore,
799- "not_after", cert.NotAfter)
800- continue
801- }
802-
803- // We found a valid certificate
804- expiresInDays := cert.NotAfter.Sub(now).Hours() / 24
805- logger.Info("Successfully verified certificate via TLS connection",
806- "domain", domain,
807- "expires_in", expiresInDays,
808- "days",
809- "issuer", cert.Issuer.CommonName)
810- return true
811- }
812-
813- logger.Debug("No valid certificate found from TLS connection",
814- "domain", domain)
815- return false
816-}
817-
818-// checkCertificateExists is a helper function that checks if a valid certificate exists
819-// either in the cache or the filesystem. Returns true if a valid certificate is found,
820-// along with the certificate type.
821-func (p *Proxy) checkCertificateExists(domain string) (bool, string) {
822- // First check in the cache
823- validInCache, cacheType := p.checkCertificateInCache(domain)
824- if validInCache {
825- logger.Debug("Certificate exists in cache",
826- "domain", domain,
827- "type", cacheType)
828- return true, cacheType
829- }
830-
831- // If not in cache, check the filesystem
832- validInFS, fsType := p.checkCertificateInFilesystem(domain)
833- if validInFS {
834- logger.Debug("Certificate exists in filesystem but not in cache",
835- "domain", domain,
836- "type", fsType)
837- return true, fsType
838- }
839-
840- // No valid certificate found
841- logger.Debug("No valid certificate found for domain", "domain", domain)
842- return false, ""
843-}
844-
845-// requestAdminCertificate preemptively requests a Let's Encrypt certificate
846-// for the Gordon admin interface
847-func (p *Proxy) requestAdminCertificate() {
848- adminDomain := p.app.GetConfig().Http.FullDomain()
849- if adminDomain == "" {
850- return
851- }
852-
853- // Only proceed if HTTPS is enabled
854- if !p.app.GetConfig().Http.Https {
855- logger.Debug("HTTPS is disabled, skipping admin certificate request")
856- return
857- }
858-
859- // Check for upstream proxy with certificate skipping enabled
860- isProxied := p.detectUpstreamProxy()
861- if isProxied && p.config.SkipCertificates {
862- logger.Info("Upstream TLS-terminating proxy detected, skipping admin certificate request",
863- "domain", adminDomain,
864- "skip_reason", "behind TLS-terminating proxy")
865- return
866- }
867-
868- // First, check if certificate is already in the autocert cache
869- // This is the critical step to avoid unnecessary Let's Encrypt requests
870- cacheKey := "cert-" + adminDomain
871- ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
872- defer cancel()
873-
874- if p.certManager != nil && p.certManager.Cache != nil {
875- certData, err := p.certManager.Cache.Get(ctx, cacheKey)
876- if err == nil && len(certData) > 0 {
877- // Check if the cached certificate is valid
878- isValid, expiresInDays, certType, _ := isCertificateValid(certData, adminDomain)
879- if isValid {
880- logger.Info("Using valid certificate from cache",
881- "domain", adminDomain,
882- "type", certType,
883- "expires_in_days", expiresInDays)
884- return
885- }
886-
887- // If certificate is invalid or expiring soon, log but continue
888- logger.Info("Cached certificate is invalid or expiring soon",
889- "domain", adminDomain,
890- "expires_in_days", expiresInDays,
891- "valid", isValid)
892- }
893- }
894-
895- // Additional rate limit safety check
896- // Check if we've recently hit a rate limit for this domain
897- // This is a temporary memory-only safeguard
898- rlCacheKey := adminDomain + "_ratelimit_history"
899- rateLimitData, _ := p.certManager.Cache.Get(ctx, rlCacheKey)
900- if len(rateLimitData) > 0 {
901- logger.Warn("Rate limit history found for admin domain, skipping certificate request",
902- "domain", adminDomain,
903- "data", string(rateLimitData),
904- "solution", "Wait for rate limit to expire")
905- return
906- }
907-
908- // Extract hostname from admin domain, resolving it to see if it's publicly accessible
909- host := adminDomain
910- ips, err := net.LookupIP(host)
911- if err != nil {
912- logger.Error("Could not resolve admin domain, Let's Encrypt will likely fail",
913- "domain", adminDomain,
914- "error", err.Error(),
915- "solution", "Check DNS settings and ensure domain points to this server")
916- return
917- }
918-
919- var ipStrings []string
920- for _, ip := range ips {
921- ipStrings = append(ipStrings, ip.String())
922- }
923-
924- logger.Info("Successfully resolved admin domain",
925- "domain", adminDomain,
926- "ips", ipStrings)
927-
928- // Check if we already have a valid certificate (cache or filesystem)
929- hasValidCert, certType := p.checkCertificateExists(adminDomain)
930-
931- // If we have a valid cert that matches our mode, use it
932- if hasValidCert {
933- // Check for mode mismatch
934- if p.config.LetsEncryptMode == "production" && certType == "staging" {
935- logger.Info("Found staging certificate but production mode is enabled",
936- "domain", adminDomain,
937- "action", "requesting new production certificate")
938- // Continue to request a new certificate
939- } else {
940- logger.Info("Using existing certificate",
941- "domain", adminDomain,
942- "type", certType,
943- "action", "skipping Let's Encrypt request to avoid rate limits")
944-
945- // If a valid certificate was found in the filesystem but not in the cache,
946- // try to verify it with a TLS connection (non-blocking)
947- go func() {
948- // Only attempt TLS verification if certificate was found in filesystem
949- // and not in cache (to avoid unnecessary check for cached certificates)
950- validInCache, _ := p.checkCertificateInCache(adminDomain)
951- validInFS, _ := p.checkCertificateInFilesystem(adminDomain)
952- if !validInCache && validInFS {
953- if testCertificateWithTLSConnection(adminDomain, p.config.Port) {
954- logger.Debug("Certificate verification succeeded via TLS connection",
955- "domain", adminDomain,
956- "note", "Certificate is trusted and working properly")
957- } else {
958- logger.Warn("Certificate verification via TLS connection failed",
959- "domain", adminDomain,
960- "note", "Certificate exists but might not be trusted or properly configured")
961- }
962- }
963- }()
964-
965- return
966- }
967- }
968-
969- // Get the certificate directory path for naming the output files
970- certDir := p.config.CertDir
971- if certDir == "" {
972- certDir = p.app.GetConfig().General.StorageDir + "/certs"
973- }
974-
975- // Ensure certificate directory exists
976- if err := os.MkdirAll(certDir, 0755); err != nil {
977- logger.Warn("Failed to create certificate directory",
978- "dir", certDir,
979- "error", err)
980- }
981-
982- // Check if environment is production
983- mode := p.config.LetsEncryptMode
984- email := p.config.Email
985-
986- // Log the certificate request intent
987- logger.Info("Initiating Let's Encrypt certificate request for admin domain",
988- "domain", adminDomain,
989- "email", email,
990- "mode", mode)
991-
992- logger.Info("⏳ Waiting for Let's Encrypt to validate domain ownership",
993- "domain", adminDomain,
994- "validation_method", "HTTP-01 challenge",
995- "requirements", "Domain must be publicly accessible on port 80",
996- "timeout", "1 minute")
997-
998- // Create a context with timeout for the initial certificate request
999- ctx, cancel = context.WithTimeout(context.Background(), 1*time.Minute)
1000- defer cancel()
1001-
1002- // Try to obtain the certificate with timeout
1003- var certResult *tls.Certificate
1004- var certErr error
1005-
1006- // Use a channel to handle the timeout
1007- certChan := make(chan struct {
1008- cert *tls.Certificate
1009- err error
1010- })
1011-
1012- go func() {
1013- c, e := p.certManager.GetCertificate(&tls.ClientHelloInfo{
1014- ServerName: adminDomain,
1015- })
1016- certChan <- struct {
1017- cert *tls.Certificate
1018- err error
1019- }{c, e}
1020- }()
1021-
1022- // Wait for either the certificate request to complete or the timeout
1023- select {
1024- case result := <-certChan:
1025- certResult = result.cert
1026- certErr = result.err
1027- case <-ctx.Done():
1028- certErr = fmt.Errorf("certificate request timed out after 2 minutes: %w", ctx.Err())
1029- logger.Error("Let's Encrypt certificate request timed out",
1030- "domain", adminDomain,
1031- "timeout", "2 minutes",
1032- "error", certErr)
1033- }
1034-
1035- if certErr != nil {
1036- // Check for rate limit errors
1037- if strings.Contains(strings.ToLower(certErr.Error()), "ratelimited") {
1038- // Extract the retry-after time if available in the error message
1039- retryAfterStr := ""
1040- re := regexp.MustCompile(`retry after ([^:]+)`)
1041- matches := re.FindStringSubmatch(certErr.Error())
1042- if len(matches) > 1 {
1043- retryAfterStr = strings.TrimSpace(matches[1])
1044- if idx := strings.Index(retryAfterStr, ":"); idx > 0 {
1045- retryAfterStr = retryAfterStr[:idx]
1046- }
1047- }
1048-
1049- logger.Error("Let's Encrypt rate limit reached",
1050- "domain", adminDomain,
1051- "error", certErr,
1052- "retry_after", retryAfterStr,
1053- "solution", "Wait until the rate limit expires, then restart the server")
1054-
1055- // Store rate limit history in cache to prevent further requests
1056- rlCacheKey := adminDomain + "_ratelimit_history"
1057- rlCtx, rlCancel := context.WithTimeout(context.Background(), 5*time.Second)
1058- defer rlCancel()
1059-
1060- // Store the error message and timestamp in the cache
1061- rlData := []byte(fmt.Sprintf("Rate limited at %s: %s",
1062- time.Now().Format(time.RFC3339), certErr.Error()))
1063-
1064- if p.certManager != nil && p.certManager.Cache != nil {
1065- // Store with a long expiration - Let's Encrypt rate limits are typically for a week
1066- err := p.certManager.Cache.Put(rlCtx, rlCacheKey, rlData)
1067- if err != nil {
1068- logger.Debug("Failed to store rate limit history in cache",
1069- "domain", adminDomain, "error", err)
1070- } else {
1071- logger.Info("Stored rate limit history in cache to prevent further requests",
1072- "domain", adminDomain, "retry_after", retryAfterStr)
1073- }
1074- }
1075-
1076- // Don't retry on rate limits
1077- logger.Info("Skipping certificate request retries due to rate limiting",
1078- "domain", adminDomain)
1079- } else {
1080- logger.Error("Failed to obtain Let's Encrypt certificate",
1081- "domain", adminDomain,
1082- "error", certErr)
1083-
1084- logger.Info("⏳ Retrying certificate request with exponential backoff",
1085- "domain", adminDomain,
1086- "timeout", "2 minutes")
1087- // Implement retry logic with backoff
1088- go p.retryCertificateRequest(adminDomain, 3, 10*time.Second)
1089- }
1090- } else if certResult != nil {
1091- // Rename the certificate files based on the mode
1092- certType := "production"
1093- if p.config.LetsEncryptMode == "staging" {
1094- certType = "staging"
1095- }
1096-
1097- // Try to rename the certificate files
1098- oldCertPath := filepath.Join(certDir, adminDomain)
1099- oldKeyPath := filepath.Join(certDir, adminDomain+"+rsa")
1100-
1101- newName := getCertificateFileName(adminDomain, certType)
1102- newCertPath := filepath.Join(certDir, newName)
1103- newKeyPath := filepath.Join(certDir, newName+"+rsa")
1104-
1105- // Debug log certificate file paths
1106- logger.Debug("Certificate file paths",
1107- "domain", adminDomain,
1108- "old_cert", oldCertPath,
1109- "new_cert", newCertPath,
1110- "old_key", oldKeyPath,
1111- "new_key", newKeyPath)
1112-
1113- // Check if files exist first
1114- oldCertExists := false
1115- if _, err := os.Stat(oldCertPath); err == nil {
1116- oldCertExists = true
1117- logger.Debug("Old certificate file exists", "path", oldCertPath)
1118- } else {
1119- logger.Debug("Old certificate file not found", "path", oldCertPath, "error", err)
1120- }
1121-
1122- // Read the certificate files directly to verify they contain valid data
1123- if oldCertExists {
1124- certBytes, err := os.ReadFile(oldCertPath)
1125- if err != nil {
1126- logger.Error("Failed to read certificate file",
1127- "path", oldCertPath,
1128- "error", err)
1129- } else {
1130- logger.Debug("Successfully read certificate file",
1131- "path", oldCertPath,
1132- "size", len(certBytes))
1133-
1134- // Verify it's a valid PEM format
1135- block, _ := pem.Decode(certBytes)
1136- if block == nil {
1137- logger.Error("Certificate file does not contain valid PEM data",
1138- "path", oldCertPath)
1139- } else {
1140- logger.Debug("Certificate file contains valid PEM data",
1141- "path", oldCertPath,
1142- "type", block.Type)
1143- }
1144- }
1145-
1146- // Check permissions on cert and key files
1147- certFileInfo, err := os.Stat(oldCertPath)
1148- if err == nil {
1149- mode := certFileInfo.Mode()
1150- logger.Debug("Certificate file permissions",
1151- "path", oldCertPath,
1152- "mode", mode.String())
1153-
1154- // Ensure cert file is readable by all
1155- if mode.Perm()&0444 != 0444 {
1156- logger.Warn("Certificate file has restrictive permissions, fixing",
1157- "path", oldCertPath,
1158- "current_mode", mode.String())
1159- if err := os.Chmod(oldCertPath, 0644); err != nil {
1160- logger.Error("Failed to fix certificate file permissions",
1161- "path", oldCertPath,
1162- "error", err)
1163- }
1164- }
1165- }
1166-
1167- // Check key file permissions
1168- if _, err := os.Stat(oldKeyPath); err == nil {
1169- keyFileInfo, err := os.Stat(oldKeyPath)
1170- if err == nil {
1171- mode := keyFileInfo.Mode()
1172- logger.Debug("Key file permissions",
1173- "path", oldKeyPath,
1174- "mode", mode.String())
1175-
1176- // Ensure key file is only readable by owner
1177- if mode.Perm()&0400 != 0400 || mode.Perm()&0077 != 0 {
1178- logger.Warn("Key file has incorrect permissions, fixing",
1179- "path", oldKeyPath,
1180- "current_mode", mode.String())
1181- if err := os.Chmod(oldKeyPath, 0600); err != nil {
1182- logger.Error("Failed to fix key file permissions",
1183- "path", oldKeyPath,
1184- "error", err)
1185- }
1186- }
1187- }
1188- }
1189- }
1190-
1191- // Only rename if the files exist and the new names are different
1192- if oldCertPath != newCertPath {
1193- // Check if old cert exists
1194- if oldCertExists {
1195- // Rename cert
1196- if err := os.Rename(oldCertPath, newCertPath); err != nil {
1197- logger.Warn("Failed to rename certificate file",
1198- "domain", adminDomain,
1199- "from", oldCertPath,
1200- "to", newCertPath,
1201- "error", err)
1202- } else {
1203- logger.Info("Renamed certificate file with appropriate type",
1204- "domain", adminDomain,
1205- "type", certType,
1206- "from", oldCertPath,
1207- "to", newCertPath)
1208- }
1209-
1210- // Rename key
1211- if _, err := os.Stat(oldKeyPath); err == nil {
1212- if err := os.Rename(oldKeyPath, newKeyPath); err != nil {
1213- logger.Warn("Failed to rename certificate key file",
1214- "domain", adminDomain,
1215- "from", oldKeyPath,
1216- "to", newKeyPath,
1217- "error", err)
1218- } else {
1219- logger.Info("Renamed certificate key file with appropriate type",
1220- "domain", adminDomain,
1221- "type", certType,
1222- "from", oldKeyPath,
1223- "to", newKeyPath)
1224- }
1225- } else {
1226- logger.Warn("Old key file not found", "path", oldKeyPath, "error", err)
1227- }
1228- }
1229- }
1230-
1231- logger.Info("Successfully obtained Let's Encrypt certificate",
1232- "domain", adminDomain,
1233- "type", certType)
1234-
1235- // Verify the certificate files are accessible in the cache
1236- cacheKey := "cert-" + adminDomain
1237- cacheCtx, cacheCancel := context.WithTimeout(context.Background(), 5*time.Second)
1238- defer cacheCancel()
1239-
1240- certData, cacheErr := p.certManager.Cache.Get(cacheCtx, cacheKey)
1241- if cacheErr != nil {
1242- logger.Warn("Certificate not found in cache after successful obtainment",
1243- "domain", adminDomain,
1244- "error", cacheErr,
1245- "solution", "Check certificate permissions and file paths")
1246-
1247- // If certificate isn't in autocert cache but exists in filesystem, try to manually add it
1248- newCertExists := false
1249- if _, err := os.Stat(newCertPath); err == nil {
1250- newCertExists = true
1251- }
1252-
1253- if oldCertExists || newCertExists {
1254- logger.Info("Attempting to manually add certificate to cache from filesystem",
1255- "domain", adminDomain)
1256-
1257- sourceCertPath := newCertPath
1258- sourceKeyPath := newKeyPath
1259-
1260- // If the file wasn't renamed, use the old paths
1261- if _, err := os.Stat(sourceCertPath); err != nil {
1262- sourceCertPath = oldCertPath
1263- sourceKeyPath = oldKeyPath
1264- }
1265-
1266- // Read the certificate and key files
1267- certBytes, err := os.ReadFile(sourceCertPath)
1268- if err != nil {
1269- logger.Error("Failed to read certificate file for manual cache addition",
1270- "path", sourceCertPath,
1271- "error", err)
1272- } else {
1273- keyBytes, err := os.ReadFile(sourceKeyPath)
1274- if err != nil {
1275- logger.Error("Failed to read key file for manual cache addition",
1276- "path", sourceKeyPath,
1277- "error", err)
1278- } else {
1279- // Add to autocert cache
1280- if err := p.certManager.Cache.Put(cacheCtx, cacheKey, certBytes); err != nil {
1281- logger.Error("Failed to manually add certificate to cache",
1282- "domain", adminDomain,
1283- "error", err)
1284- } else {
1285- keyKey := "key-" + adminDomain
1286- if err := p.certManager.Cache.Put(cacheCtx, keyKey, keyBytes); err != nil {
1287- logger.Error("Failed to manually add key to cache",
1288- "domain", adminDomain,
1289- "error", err)
1290- } else {
1291- logger.Info("Successfully manually added certificate to cache",
1292- "domain", adminDomain)
1293- }
1294- }
1295- }
1296- }
1297- }
1298- } else {
1299- logger.Debug("Certificate successfully stored in cache",
1300- "domain", adminDomain,
1301- "cert_size", len(certData))
1302- }
1303- } else {
1304- logger.Error("Unexpected error: received nil certificate but no error",
1305- "domain", adminDomain,
1306- "timeout", "2 minutes")
1307- // Implement retry logic with backoff
1308- go p.retryCertificateRequest(adminDomain, 3, 10*time.Second)
1309- }
1310-}
1311-
1312-// requestDomainCertificate proactively requests a Let's Encrypt certificate
1313-// for a container domain
1314-func (p *Proxy) requestDomainCertificate(domain string) {
1315- if domain == "" {
1316- logger.Warn("Empty domain provided to requestDomainCertificate")
1317- return
1318- }
1319-
1320- // Check for upstream proxy with certificate skipping enabled
1321- isProxied := p.detectUpstreamProxy()
1322- if isProxied && p.config.SkipCertificates {
1323- logger.Info("Upstream TLS-terminating proxy detected, skipping domain certificate request",
1324- "domain", domain,
1325- "skip_reason", "behind TLS-terminating proxy")
1326- return
1327- }
1328-
1329- // First, check if certificate is already in the autocert cache
1330- // This is the critical step to avoid unnecessary Let's Encrypt requests
1331- cacheKey := "cert-" + domain
1332- ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
1333- defer cancel()
1334-
1335- if p.certManager != nil && p.certManager.Cache != nil {
1336- certData, err := p.certManager.Cache.Get(ctx, cacheKey)
1337- if err == nil && len(certData) > 0 {
1338- // Check if the cached certificate is valid
1339- isValid, expiresInDays, certType, _ := isCertificateValid(certData, domain)
1340- if isValid {
1341- logger.Info("Using valid certificate from cache",
1342- "domain", domain,
1343- "type", certType,
1344- "expires_in_days", expiresInDays)
1345- return
1346- }
1347-
1348- // If certificate is invalid or expiring soon, log but continue
1349- logger.Info("Cached certificate is invalid or expiring soon",
1350- "domain", domain,
1351- "expires_in_days", expiresInDays,
1352- "valid", isValid)
1353- }
1354- }
1355-
1356- // Extract hostname from domain, resolving it to see if it's publicly accessible
1357- ips, err := net.LookupIP(domain)
1358- if err != nil {
1359- logger.Error("Could not resolve domain, Let's Encrypt will likely fail",
1360- "domain", domain,
1361- "error", err.Error(),
1362- "solution", "Check DNS settings and ensure domain points to this server")
1363- return
1364- }
1365-
1366- var ipStrings []string
1367- for _, ip := range ips {
1368- ipStrings = append(ipStrings, ip.String())
1369- }
1370-
1371- logger.Info("Successfully resolved domain for certificate request",
1372- "domain", domain,
1373- "ips", ipStrings)
1374-
1375- // Check if we already have a valid certificate (cache or filesystem)
1376- hasValidCert, certType := p.checkCertificateExists(domain)
1377-
1378- // If we have a valid cert that matches our mode, use it
1379- if hasValidCert {
1380- // Check for mode mismatch
1381- if p.config.LetsEncryptMode == "production" && certType == "staging" {
1382- logger.Info("Found staging certificate but production mode is enabled",
1383- "domain", domain,
1384- "action", "requesting new production certificate")
1385- // Continue to request a new certificate
1386- } else {
1387- logger.Info("Using existing certificate",
1388- "domain", domain,
1389- "type", certType,
1390- "action", "skipping Let's Encrypt request to avoid rate limits")
1391-
1392- // If a valid certificate was found in the filesystem but not in the cache,
1393- // try to verify it with a TLS connection (non-blocking)
1394- go func() {
1395- // Only attempt TLS verification if certificate was found in filesystem
1396- // and not in cache (to avoid unnecessary check for cached certificates)
1397- validInCache, _ := p.checkCertificateInCache(domain)
1398- validInFS, _ := p.checkCertificateInFilesystem(domain)
1399- if !validInCache && validInFS {
1400- // Try to load the certificate into the cache if it's not there already
1401- // This helps prevent future Let's Encrypt requests
1402- hasOnDisk, certType := p.checkCertificateInFilesystem(domain)
1403- if hasOnDisk {
1404- logger.Info("Loading filesystem certificate into cache",
1405- "domain", domain,
1406- "type", certType)
1407-
1408- // Get the certificate directory path for finding files
1409- certDir := p.config.CertDir
1410- if certDir == "" {
1411- certDir = p.app.GetConfig().General.StorageDir + "/certs"
1412- }
1413-
1414- // Look for production certificate first, then staging
1415- certFile := getCertificateFileName(domain, "production")
1416- certPath := filepath.Join(certDir, certFile)
1417- keyPath := filepath.Join(certDir, certFile+"+rsa")
1418-
1419- // If production cert doesn't exist, try staging
1420- if _, err := os.Stat(certPath); os.IsNotExist(err) {
1421- certFile = getCertificateFileName(domain, "staging")
1422- certPath = filepath.Join(certDir, certFile)
1423- keyPath = filepath.Join(certDir, certFile+"+rsa")
1424- }
1425-
1426- // If no specific cert found, try the domain name directly
1427- if _, err := os.Stat(certPath); os.IsNotExist(err) {
1428- certPath = filepath.Join(certDir, domain)
1429- keyPath = filepath.Join(certDir, domain+"+rsa")
1430- }
1431-
1432- // Add to cache if files exist
1433- if _, err := os.Stat(certPath); err == nil {
1434- if _, err := os.Stat(keyPath); err == nil {
1435- p.addCertificateToCache(domain, certPath, keyPath)
1436- }
1437- }
1438- }
1439-
1440- if testCertificateWithTLSConnection(domain, p.config.Port) {
1441- logger.Debug("Certificate verification succeeded via TLS connection",
1442- "domain", domain,
1443- "note", "Certificate is trusted and working properly")
1444- } else {
1445- logger.Warn("Certificate verification via TLS connection failed",
1446- "domain", domain,
1447- "note", "Certificate exists but might not be trusted or properly configured")
1448- }
1449- }
1450- }()
1451-
1452- return
1453- }
1454- }
1455-
1456- // Additional rate limit safety check
1457- // Check if we've recently hit a rate limit for this domain
1458- // This is a temporary memory-only safeguard
1459- cacheKey = domain + "_ratelimit_history"
1460- rateLimitData, _ := p.certManager.Cache.Get(ctx, cacheKey)
1461- if len(rateLimitData) > 0 {
1462- logger.Warn("Rate limit history found for this domain, skipping certificate request",
1463- "domain", domain,
1464- "solution", "Wait for rate limit to expire")
1465- return
1466- }
1467-
1468- // Get the certificate directory path for naming the output files
1469- certDir := p.config.CertDir
1470- if certDir == "" {
1471- certDir = p.app.GetConfig().General.StorageDir + "/certs"
1472- }
1473-
1474- // Ensure certificate directory exists
1475- if err := os.MkdirAll(certDir, 0755); err != nil {
1476- logger.Warn("Failed to create certificate directory",
1477- "dir", certDir,
1478- "error", err)
1479- }
1480-
1481- // Use the configured mode
1482- mode := p.config.LetsEncryptMode
1483- email := p.config.Email
1484-
1485- // Log the certificate request intent
1486- logger.Info("Initiating Let's Encrypt certificate request for container domain",
1487- "domain", domain,
1488- "email", email,
1489- "mode", mode)
1490-
1491- logger.Info("⏳ Waiting for Let's Encrypt to validate domain ownership",
1492- "domain", domain,
1493- "validation_method", "HTTP-01 challenge",
1494- "requirements", "Domain must be publicly accessible on port 80",
1495- "timeout", "1 minute")
1496-
1497- // Create a context with timeout for the initial certificate request
1498- ctx, cancel = context.WithTimeout(context.Background(), 1*time.Minute)
1499- defer cancel()
1500-
1501- // Try to obtain the certificate with timeout
1502- var certResult *tls.Certificate
1503- var certErr error
1504-
1505- // Use a channel to handle the timeout
1506- certChan := make(chan struct {
1507- cert *tls.Certificate
1508- err error
1509- })
1510-
1511- go func() {
1512- c, e := p.certManager.GetCertificate(&tls.ClientHelloInfo{
1513- ServerName: domain,
1514- })
1515- certChan <- struct {
1516- cert *tls.Certificate
1517- err error
1518- }{c, e}
1519- }()
1520-
1521- // Wait for either the certificate request to complete or the timeout
1522- select {
1523- case result := <-certChan:
1524- certResult = result.cert
1525- certErr = result.err
1526- case <-ctx.Done():
1527- certErr = fmt.Errorf("certificate request timed out after 1 minute: %w", ctx.Err())
1528- logger.Error("Let's Encrypt certificate request timed out",
1529- "domain", domain,
1530- "timeout", "1 minute",
1531- "error", certErr)
1532- }
1533-
1534- if certErr != nil {
1535- // Check for rate limit errors
1536- if strings.Contains(strings.ToLower(certErr.Error()), "ratelimited") {
1537- // Extract the retry-after time if available in the error message
1538- retryAfterStr := ""
1539- re := regexp.MustCompile(`retry after ([^:]+)`)
1540- matches := re.FindStringSubmatch(certErr.Error())
1541- if len(matches) > 1 {
1542- retryAfterStr = strings.TrimSpace(matches[1])
1543- if idx := strings.Index(retryAfterStr, ":"); idx > 0 {
1544- retryAfterStr = retryAfterStr[:idx]
1545- }
1546- }
1547-
1548- logger.Error("Let's Encrypt rate limit reached",
1549- "domain", domain,
1550- "error", certErr,
1551- "retry_after", retryAfterStr,
1552- "solution", "Wait until the rate limit expires, then restart the server")
1553-
1554- // Store rate limit history in cache to prevent further requests
1555- rlCacheKey := domain + "_ratelimit_history"
1556- rlCtx, rlCancel := context.WithTimeout(context.Background(), 5*time.Second)
1557- defer rlCancel()
1558-
1559- // Store the error message and timestamp in the cache
1560- rlData := []byte(fmt.Sprintf("Rate limited at %s: %s",
1561- time.Now().Format(time.RFC3339), certErr.Error()))
1562-
1563- if p.certManager != nil && p.certManager.Cache != nil {
1564- // Store with a long expiration - Let's Encrypt rate limits are typically for a week
1565- err := p.certManager.Cache.Put(rlCtx, rlCacheKey, rlData)
1566- if err != nil {
1567- logger.Debug("Failed to store rate limit history in cache",
1568- "domain", domain, "error", err)
1569- } else {
1570- logger.Info("Stored rate limit history in cache to prevent further requests",
1571- "domain", domain, "retry_after", retryAfterStr)
1572- }
1573- }
1574-
1575- // Don't retry on rate limits
1576- logger.Info("Skipping certificate request retries due to rate limiting",
1577- "domain", domain)
1578- } else {
1579- logger.Error("Failed to obtain Let's Encrypt certificate",
1580- "domain", domain,
1581- "error", certErr)
1582-
1583- logger.Info("⏳ Retrying certificate request with exponential backoff",
1584- "domain", domain,
1585- "timeout", "2 minutes")
1586- // Implement retry logic with backoff
1587- go p.retryCertificateRequest(domain, 3, 10*time.Second)
1588- }
1589- } else if certResult != nil {
1590- // Rename the certificate files based on the mode
1591- certType := "production"
1592- if p.config.LetsEncryptMode == "staging" {
1593- certType = "staging"
1594- }
1595-
1596- // Try to rename the certificate files
1597- oldCertPath := filepath.Join(certDir, domain)
1598- oldKeyPath := filepath.Join(certDir, domain+"+rsa")
1599-
1600- newName := getCertificateFileName(domain, certType)
1601- newCertPath := filepath.Join(certDir, newName)
1602- newKeyPath := filepath.Join(certDir, newName+"+rsa")
1603-
1604- // Only rename if the files exist and the new names are different
1605- if oldCertPath != newCertPath {
1606- // Check if old cert exists
1607- if _, err := os.Stat(oldCertPath); err == nil {
1608- // Rename cert
1609- if err := os.Rename(oldCertPath, newCertPath); err != nil {
1610- logger.Warn("Failed to rename certificate file",
1611- "domain", domain,
1612- "from", oldCertPath,
1613- "to", newCertPath,
1614- "error", err)
1615- } else {
1616- logger.Info("Renamed certificate file with appropriate type",
1617- "domain", domain,
1618- "type", certType,
1619- "from", oldCertPath,
1620- "to", newCertPath)
1621- }
1622-
1623- // Rename key
1624- if _, err := os.Stat(oldKeyPath); err == nil {
1625- if err := os.Rename(oldKeyPath, newKeyPath); err != nil {
1626- logger.Warn("Failed to rename certificate key file",
1627- "domain", domain,
1628- "from", oldKeyPath,
1629- "to", newKeyPath,
1630- "error", err)
1631- } else {
1632- logger.Info("Renamed certificate key file with appropriate type",
1633- "domain", domain,
1634- "type", certType,
1635- "from", oldKeyPath,
1636- "to", newKeyPath)
1637- }
1638- } else {
1639- logger.Warn("Old key file not found", "path", oldKeyPath, "error", err)
1640- }
1641- }
1642- }
1643-
1644- logger.Info("Successfully obtained Let's Encrypt certificate",
1645- "domain", domain,
1646- "type", certType)
1647- } else {
1648- logger.Error("Unexpected error: received nil certificate but no error",
1649- "domain", domain,
1650- "timeout", "1 minute")
1651- // Implement retry logic with backoff
1652- go p.retryCertificateRequest(domain, 3, 10*time.Second)
1653- }
1654-}
1655-
1656-// retryCertificateRequest attempts to request a certificate with exponential backoff
1657-func (p *Proxy) retryCertificateRequest(domain string, maxRetries int, initialBackoff time.Duration) {
1658- backoff := initialBackoff
1659-
1660- // Get the certificate directory for renaming
1661- certDir := p.config.CertDir
1662- if certDir == "" {
1663- certDir = p.app.GetConfig().General.StorageDir + "/certs"
1664- }
1665-
1666- for attempt := 1; attempt <= maxRetries; attempt++ {
1667- // Wait for backoff period
1668- logger.Info("Retrying Let's Encrypt certificate request",
1669- "domain", domain,
1670- "attempt", attempt,
1671- "max_retries", maxRetries,
1672- "wait_time", backoff)
1673- time.Sleep(backoff)
1674-
1675- // Before each retry, check if the HTTP challenge endpoint is accessible
1676- if attempt > 1 {
1677- // Try connecting to our own HTTP server on port 80
1678- // to verify its availability for Let's Encrypt validation
1679- client := &http.Client{
1680- Timeout: 5 * time.Second,
1681- CheckRedirect: func(req *http.Request, via []*http.Request) error {
1682- // Don't follow redirects for this test
1683- return http.ErrUseLastResponse
1684- },
1685- }
1686-
1687- // Test the ACME challenge path with a fake token
1688- testURL := fmt.Sprintf("http://%s/.well-known/acme-challenge/test-token", domain)
1689- resp, err := client.Get(testURL)
1690-
1691- if err != nil {
1692- logger.Warn("HTTP challenge endpoint might not be accessible",
1693- "domain", domain,
1694- "url", testURL,
1695- "error", err,
1696- "solution", "Ensure port 80 is accessible and not blocked by firewall")
1697- } else {
1698- resp.Body.Close()
1699- logger.Info("HTTP challenge endpoint is accessible",
1700- "domain", domain,
1701- "url", testURL,
1702- "status", resp.StatusCode)
1703- }
1704- }
1705-
1706- // Create context with timeout for this attempt
1707- ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
1708-
1709- // Request certificate
1710- err := func(ctx context.Context) error {
1711- defer cancel()
1712- cert, certErr := p.certManager.GetCertificate(&tls.ClientHelloInfo{
1713- ServerName: domain,
1714- })
1715- if cert == nil && certErr == nil {
1716- return fmt.Errorf("certificate is nil but no error returned")
1717- }
1718- return certErr
1719- }(ctx)
1720-
1721- // Check result
1722- if err == nil {
1723- // Determine certificate type based on mode
1724- certType := "production"
1725- if p.config.LetsEncryptMode == "staging" {
1726- certType = "staging"
1727- }
1728-
1729- // Rename the certificate files to include the type
1730- oldCertPath := filepath.Join(certDir, domain)
1731- oldKeyPath := filepath.Join(certDir, domain+"+rsa")
1732-
1733- newName := getCertificateFileName(domain, certType)
1734- newCertPath := filepath.Join(certDir, newName)
1735- newKeyPath := filepath.Join(certDir, newName+"+rsa")
1736-
1737- // Only rename if the files exist and the new names are different
1738- if oldCertPath != newCertPath {
1739- // Check if old cert exists
1740- if _, err := os.Stat(oldCertPath); err == nil {
1741- // Rename cert
1742- if err := os.Rename(oldCertPath, newCertPath); err != nil {
1743- logger.Warn("Failed to rename certificate file",
1744- "domain", domain,
1745- "from", oldCertPath,
1746- "to", newCertPath,
1747- "error", err)
1748- } else {
1749- logger.Info("Renamed certificate file with appropriate type",
1750- "domain", domain,
1751- "type", certType,
1752- "from", oldCertPath,
1753- "to", newCertPath)
1754- }
1755-
1756- // Rename key
1757- if _, err := os.Stat(oldKeyPath); err == nil {
1758- if err := os.Rename(oldKeyPath, newKeyPath); err != nil {
1759- logger.Warn("Failed to rename certificate key file",
1760- "domain", domain,
1761- "from", oldKeyPath,
1762- "to", newKeyPath,
1763- "error", err)
1764- } else {
1765- logger.Info("Renamed certificate key file with appropriate type",
1766- "domain", domain,
1767- "type", certType,
1768- "from", oldKeyPath,
1769- "to", newKeyPath)
1770- }
1771- } else {
1772- logger.Warn("Old key file not found", "path", oldKeyPath, "error", err)
1773- }
1774- }
1775- }
1776-
1777- logger.Info("Successfully obtained Let's Encrypt certificate on retry",
1778- "domain", domain,
1779- "attempt", attempt,
1780- "type", certType)
1781- return
1782- }
1783-
1784- logger.Error("Let's Encrypt certificate request retry failed",
1785- "domain", domain,
1786- "attempt", attempt,
1787- "error", err)
1788-
1789- // Provide more detailed diagnostics based on the error
1790- if strings.Contains(strings.ToLower(err.Error()), "connection refused") ||
1791- strings.Contains(strings.ToLower(err.Error()), "timeout") {
1792- logger.Error("Let's Encrypt connection failed - this typically indicates:",
1793- "issue_1", "Port 80 is not accessible from the internet",
1794- "issue_2", "Firewall is blocking inbound connections",
1795- "issue_3", "DNS records not properly propagated",
1796- "solution", "Check firewall settings and DNS configuration")
1797- } else if strings.Contains(strings.ToLower(err.Error()), "unauthorized") {
1798- logger.Error("Let's Encrypt authorization failed - this typically indicates:",
1799- "issue", "Domain ownership validation failed",
1800- "solution", "Ensure the server is publicly accessible on port 80")
1801- } else if strings.Contains(strings.ToLower(err.Error()), "ratelimited") {
1802- // Extract the retry-after date if present
1803- retryAfterStr := "see error message for details"
1804- if strings.Contains(err.Error(), "retry after") {
1805- parts := strings.Split(err.Error(), "retry after")
1806- if len(parts) > 1 {
1807- retryAfterStr = strings.TrimSpace(parts[1])
1808- if idx := strings.Index(retryAfterStr, ":"); idx > 0 {
1809- retryAfterStr = retryAfterStr[:idx]
1810- }
1811- }
1812- }
1813-
1814- logger.Error("Let's Encrypt rate limit reached - cannot issue more certificates for this domain yet",
1815- "domain", domain,
1816- "retry_after", retryAfterStr,
1817- "solution", "Wait until the rate limit expires, then restart the server")
1818-
1819- // No point in retrying on rate limit errors
1820- return
1821- }
1822-
1823- // Increase backoff for next attempt (exponential backoff)
1824- backoff *= 2
1825- }
1826-
1827- // Generate a self-signed certificate as a fallback
1828- logger.Error("All Let's Encrypt certificate request retries failed",
1829- "domain", domain,
1830- "max_retries", maxRetries,
1831- "fallback", "Generating self-signed certificate")
1832-
1833- // Generate a self-signed certificate for this domain
1834- fallbackCert, err := generateFallbackCertificates([]string{domain})
1835- if err != nil {
1836- logger.Error("Failed to generate self-signed fallback certificate",
1837- "domain", domain,
1838- "error", err)
1839- } else {
1840- // Save the self-signed certificate to disk with appropriate naming
1841- selfSignedName := getCertificateFileName(domain, "self-signed")
1842- certPath := filepath.Join(certDir, selfSignedName)
1843- keyPath := filepath.Join(certDir, selfSignedName+"+rsa")
1844-
1845- // Save the certificate and key
1846- certPEM := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: fallbackCert.Certificate[0]})
1847- keyBytes, err := x509.MarshalPKCS8PrivateKey(fallbackCert.PrivateKey)
1848- if err == nil {
1849- keyPEM := pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: keyBytes})
1850-
1851- // Write files
1852- if err := os.WriteFile(certPath, certPEM, 0644); err != nil {
1853- logger.Error("Failed to write self-signed certificate to disk",
1854- "domain", domain,
1855- "path", certPath,
1856- "error", err)
1857- }
1858-
1859- if err := os.WriteFile(keyPath, keyPEM, 0600); err != nil {
1860- logger.Error("Failed to write self-signed key to disk",
1861- "domain", domain,
1862- "path", keyPath,
1863- "error", err)
1864- }
1865-
1866- logger.Info("Generated and saved self-signed fallback certificate",
1867- "domain", domain,
1868- "cert_path", certPath,
1869- "key_path", keyPath,
1870- "valid_until", time.Now().Add(24*time.Hour).Format("2006-01-02 15:04:05"))
1871- }
1872- }
1873-
1874- // At this point, all retries have failed, so we'll rely on the fallback self-signed certificate
1875- // But let's log this prominently for debugging
1876- logger.Error("⚠️ HTTPS is using a self-signed certificate which browsers will warn about",
1877- "domain", domain,
1878- "reason", "Let's Encrypt certificate issuance failed",
1879- "solution", "Check network settings and Let's Encrypt status")
1880-
1881- // Suggest checking the common issues
1882- logger.Error("Common Let's Encrypt issues to check:",
1883- "check_1", "Ensure ports 80 and 443 are open on your firewall",
1884- "check_2", "Verify DNS records point to your server IP",
1885- "check_3", "Make sure no other services are running on ports 80/443",
1886- "check_4", "Check if Let's Encrypt service is having issues (https://letsencrypt.status.io/)")
1887-}
1888-
1889-// generateFallbackCertificates generates a self-signed fallback certificate
1890-func generateFallbackCertificates(domains []string) (*tls.Certificate, error) {
1891- if len(domains) == 0 {
1892- return nil, fmt.Errorf("no domains provided for fallback certificate")
1893- }
1894-
1895- // Create a new private key
1896- privateKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
1897- if err != nil {
1898- return nil, fmt.Errorf("failed to generate private key: %w", err)
1899- }
1900-
1901- // Create a template for the certificate
1902- notBefore := time.Now()
1903- notAfter := notBefore.Add(24 * time.Hour) // Valid for 24 hours
1904-
1905- serialNumberLimit := new(big.Int).Lsh(big.NewInt(1), 128)
1906- serialNumber, err := rand.Int(rand.Reader, serialNumberLimit)
1907- if err != nil {
1908- return nil, fmt.Errorf("failed to generate serial number: %w", err)
1909- }
1910-
1911- template := x509.Certificate{
1912- SerialNumber: serialNumber,
1913- Subject: pkix.Name{
1914- Organization: []string{"Gordon Self-Signed Certificate"},
1915- CommonName: domains[0], // Use the first domain as the common name
1916- },
1917- NotBefore: notBefore,
1918- NotAfter: notAfter,
1919- KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature,
1920- ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
1921- BasicConstraintsValid: true,
1922- // Add all domains as SANs
1923- DNSNames: domains,
1924- }
1925-
1926- // Add IP addresses to the certificate if domains contain IP addresses
1927- for _, domain := range domains {
1928- ip := net.ParseIP(domain)
1929- if ip != nil {
1930- template.IPAddresses = append(template.IPAddresses, ip)
1931- }
1932- }
1933-
1934- // Self-sign the certificate
1935- derBytes, err := x509.CreateCertificate(rand.Reader, &template, &template, &privateKey.PublicKey, privateKey)
1936- if err != nil {
1937- return nil, fmt.Errorf("failed to create certificate: %w", err)
1938- }
1939-
1940- // Encode certificate to PEM
1941- certPEM := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: derBytes})
1942- if certPEM == nil {
1943- return nil, fmt.Errorf("failed to encode certificate to PEM")
1944- }
1945-
1946- // Encode private key to PEM
1947- privateKeyBytes, err := x509.MarshalPKCS8PrivateKey(privateKey)
1948- if err != nil {
1949- return nil, fmt.Errorf("failed to marshal private key: %w", err)
1950- }
1951-
1952- privateKeyPEM := pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: privateKeyBytes})
1953- if privateKeyPEM == nil {
1954- return nil, fmt.Errorf("failed to encode private key to PEM")
1955- }
1956-
1957- // Load the certificate
1958- cert, err := tls.X509KeyPair(certPEM, privateKeyPEM)
1959- if err != nil {
1960- return nil, fmt.Errorf("failed to load certificate: %w", err)
1961- }
1962-
1963- return &cert, nil
1964-}
1965-
1966-// preloadCertificates scans the certificate directory and loads all valid certificates
1967-// into the autocert cache to prevent unnecessary requests to Let's Encrypt
1968-func (p *Proxy) preloadCertificates() {
1969- if p.certManager == nil || p.certManager.Cache == nil {
1970- logger.Warn("Cannot preload certificates: certificate manager or cache not initialized")
1971- return
1972- }
1973-
1974- // Get the certificate directory path
1975- certDir := p.config.CertDir
1976- if certDir == "" {
1977- certDir = p.app.GetConfig().General.StorageDir + "/certs"
1978- }
1979-
1980- logger.Info("Preloading certificates from directory", "dir", certDir)
1981-
1982- // Ensure directory exists
1983- if _, err := os.Stat(certDir); os.IsNotExist(err) {
1984- logger.Warn("Certificate directory does not exist, creating it", "dir", certDir)
1985- if err := os.MkdirAll(certDir, 0755); err != nil {
1986- logger.Error("Failed to create certificate directory", "dir", certDir, "error", err)
1987- return
1988- }
1989- }
1990-
1991- // List all files in the certificate directory
1992- files, err := os.ReadDir(certDir)
1993- if err != nil {
1994- logger.Error("Failed to read certificate directory", "dir", certDir, "error", err)
1995- return
1996- }
1997-
1998- // Log all files in the directory to help troubleshoot
1999- var fileNames []string
2000- for _, file := range files {
2001- fileNames = append(fileNames, file.Name())
2002-
2003- // Check file permissions and fix if too restrictive
2004- filePath := filepath.Join(certDir, file.Name())
2005- info, err := os.Stat(filePath)
2006- if err != nil {
2007- logger.Warn("Could not stat file", "path", filePath, "error", err)
2008- continue
2009- }
2010-
2011- mode := info.Mode()
2012- if strings.HasSuffix(file.Name(), "+rsa") {
2013- // Key files should be 0600 (owner read/write only)
2014- if mode.Perm() != 0600 {
2015- logger.Info("Fixing key file permissions", "file", file.Name(), "current_mode", mode.String(), "target_mode", "0600")
2016- if err := os.Chmod(filePath, 0600); err != nil {
2017- logger.Warn("Failed to fix key file permissions", "file", file.Name(), "error", err)
2018- }
2019- }
2020- } else {
2021- // Certificate files should be 0644 (readable by all)
2022- if mode.Perm() != 0644 {
2023- logger.Info("Fixing certificate file permissions", "file", file.Name(), "current_mode", mode.String(), "target_mode", "0644")
2024- if err := os.Chmod(filePath, 0644); err != nil {
2025- logger.Warn("Failed to fix certificate file permissions", "file", file.Name(), "error", err)
2026- }
2027- }
2028- }
2029- }
2030-
2031- logger.Debug("Files in certificate directory", "dir", certDir, "files", fileNames)
2032-
2033- // Track the number of certificates loaded
2034- loadedCount := 0
2035-
2036- // Create a context for cache operations
2037- ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
2038- defer cancel()
2039-
2040- // Find and process certificate files (excluding key files with +rsa suffix)
2041- // Group them by domain for easier processing
2042- domainCerts := make(map[string][]string)
2043-
2044- for _, file := range files {
2045- name := file.Name()
2046-
2047- // Skip account keys and +rsa key files as we'll handle these separately
2048- if strings.HasSuffix(name, "+rsa") || strings.Contains(name, "acme_account") {
2049- continue
2050- }
2051-
2052- // Get associated key file name (file+rsa)
2053- keyName := name + "+rsa"
2054- keyPath := filepath.Join(certDir, keyName)
2055-
2056- // Skip certificates without key files
2057- if _, err := os.Stat(keyPath); os.IsNotExist(err) {
2058- logger.Debug("Skipping certificate without key file", "cert", name, "key", keyName)
2059- continue
2060- }
2061-
2062- // Try to determine the domain from the filename
2063- domain := name
2064-
2065- // Handle special patterns like domain_letsencrypt-production
2066- for _, pattern := range []string{"_letsencrypt-production", "_letsencrypt-staging", "_self-signed"} {
2067- if strings.HasSuffix(domain, pattern) {
2068- domain = strings.TrimSuffix(domain, pattern)
2069- break
2070- }
2071- }
2072-
2073- // Add to domain certificates map
2074- domainCerts[domain] = append(domainCerts[domain], name)
2075- }
2076-
2077- // Process certificates for each domain
2078- for domain, certFiles := range domainCerts {
2079- logger.Debug("Found certificate files for domain", "domain", domain, "files", certFiles)
2080-
2081- // Prioritize production certificates if multiple exist
2082- var bestCert string
2083- bestCertType := ""
2084-
2085- for _, certFile := range certFiles {
2086- certPath := filepath.Join(certDir, certFile)
2087-
2088- // Read certificate to check its validity
2089- certData, err := os.ReadFile(certPath)
2090- if err != nil {
2091- logger.Warn("Failed to read certificate file", "path", certPath, "error", err)
2092- continue
2093- }
2094-
2095- // Debug certificate content
2096- if len(certData) < 100 {
2097- logger.Warn("Certificate file too small, may be corrupted",
2098- "path", certPath, "size", len(certData))
2099- continue
2100- }
2101-
2102- // Validate the certificate
2103- valid, expiresInDays, certType, _ := isCertificateValid(certData, domain)
2104- if !valid {
2105- logger.Debug("Invalid certificate found", "domain", domain, "file", certFile)
2106- continue
2107- }
2108-
2109- logger.Info("Found valid certificate", "domain", domain, "type", certType,
2110- "expires_in_days", expiresInDays, "file", certFile)
2111-
2112- // Prioritize production certificates over staging ones
2113- if bestCertType == "" ||
2114- (bestCertType != "production" && certType == "production") {
2115- bestCert = certFile
2116- bestCertType = certType
2117- }
2118- }
2119-
2120- // If we found a valid certificate for this domain, add it to the cache
2121- if bestCert != "" {
2122- certPath := filepath.Join(certDir, bestCert)
2123- keyPath := filepath.Join(certDir, bestCert+"+rsa")
2124-
2125- // Read certificate and key data
2126- certData, err := os.ReadFile(certPath)
2127- if err != nil {
2128- logger.Warn("Failed to read certificate file", "path", certPath, "error", err)
2129- continue
2130- }
2131-
2132- keyData, err := os.ReadFile(keyPath)
2133- if err != nil {
2134- logger.Warn("Failed to read key file", "path", keyPath, "error", err)
2135- continue
2136- }
2137-
2138- // Add certificate to autocert cache
2139- cacheKey := "cert-" + domain
2140- err = p.certManager.Cache.Put(ctx, cacheKey, certData)
2141- if err != nil {
2142- logger.Error("Failed to add certificate to cache", "domain", domain, "error", err)
2143- continue
2144- }
2145-
2146- // Add key to cache
2147- keyKey := "key-" + domain
2148- err = p.certManager.Cache.Put(ctx, keyKey, keyData)
2149- if err != nil {
2150- logger.Error("Failed to add key to cache", "domain", domain, "error", err)
2151- continue
2152- }
2153-
2154- logger.Info("Preloaded certificate into cache", "domain", domain, "type", bestCertType)
2155- loadedCount++
2156- } else {
2157- logger.Warn("No valid certificate found for domain after checking all files",
2158- "domain", domain,
2159- "files_checked", certFiles)
2160- }
2161- }
2162-
2163- logger.Info("Certificate preloading complete", "loaded_certificates", loadedCount)
2164-}
2165-
2166-// manuallyLoadCertificate is a fallback method to load certificates directly
2167-// when autocert cache is failing. This directly builds the tls.Certificate from files.
2168-func (p *Proxy) manuallyLoadCertificate(domain string) (*tls.Certificate, error) {
2169- // Get certificate directory
2170- certDir := p.config.CertDir
2171- if certDir == "" {
2172- certDir = p.app.GetConfig().General.StorageDir + "/certs"
2173- }
2174-
2175- // Check preferred file naming patterns
2176- filePatterns := []string{
2177- // Try production certificate first
2178- fmt.Sprintf(LetsEncryptProdPattern, domain),
2179- // Then staging certificate
2180- fmt.Sprintf(LetsEncryptStagingPattern, domain),
2181- // Then self-signed
2182- fmt.Sprintf(SelfSignedCertPattern, domain),
2183- // Finally plain domain name
2184- domain,
2185- }
2186-
2187- for _, pattern := range filePatterns {
2188- certPath := filepath.Join(certDir, pattern)
2189- keyPath := filepath.Join(certDir, pattern+"+rsa")
2190-
2191- // Check if both files exist
2192- if _, err := os.Stat(certPath); os.IsNotExist(err) {
2193- continue
2194- }
2195- if _, err := os.Stat(keyPath); os.IsNotExist(err) {
2196- continue
2197- }
2198-
2199- // Try to load the certificate pair
2200- cert, err := tls.LoadX509KeyPair(certPath, keyPath)
2201- if err != nil {
2202- logger.Warn("Failed to load certificate key pair",
2203- "domain", domain,
2204- "certPath", certPath,
2205- "keyPath", keyPath,
2206- "error", err)
2207-
2208- // Additional diagnostics
2209- if data, readErr := os.ReadFile(certPath); readErr == nil {
2210- logger.Debug("Certificate file content preview",
2211- "path", certPath,
2212- "size", len(data),
2213- "preview", string(data[:min(100, len(data))]))
2214-
2215- // Try to parse the certificate - this may help diagnose format issues
2216- block, _ := pem.Decode(data)
2217- if block == nil {
2218- logger.Error("Certificate file does not contain valid PEM data",
2219- "path", certPath)
2220- } else {
2221- logger.Debug("Certificate PEM block info",
2222- "type", block.Type,
2223- "headers", block.Headers)
2224- }
2225- }
2226-
2227- continue
2228- }
2229-
2230- // Success!
2231- logger.Info("Manually loaded certificate from filesystem",
2232- "domain", domain,
2233- "certPath", certPath)
2234- return &cert, nil
2235- }
2236-
2237- // Also try with glob pattern
2238- filePaths, err := filepath.Glob(filepath.Join(certDir, domain+"*"))
2239- if err != nil {
2240- return nil, fmt.Errorf("glob error: %w", err)
2241- }
2242-
2243- for _, certPath := range filePaths {
2244- // Skip if it's an RSA key
2245- if strings.HasSuffix(certPath, "+rsa") {
2246- continue
2247- }
2248-
2249- // Check if key exists
2250- keyPath := certPath + "+rsa"
2251- if _, err := os.Stat(keyPath); os.IsNotExist(err) {
2252- continue
2253- }
2254-
2255- // Try to load
2256- cert, err := tls.LoadX509KeyPair(certPath, keyPath)
2257- if err != nil {
2258- logger.Debug("Failed to load certificate from glob match",
2259- "domain", domain,
2260- "certPath", certPath,
2261- "error", err)
2262- continue
2263- }
2264-
2265- logger.Info("Manually loaded certificate from glob search",
2266- "domain", domain,
2267- "certPath", certPath)
2268- return &cert, nil
2269- }
2270-
2271- return nil, fmt.Errorf("no valid certificate found for domain %s", domain)
2272-}
2273-
2274-// min returns the smaller of x or y.
2275-func min(x, y int) int {
2276- if x < y {
2277- return x
2278- }
2279- return y
2280-}
M internal/proxy/containers.go +20 -49
@@ -111,63 +111,34 @@ func (p *Proxy) scanContainersAndCheckCertificates() error {
111111
112112 // Check certificates for each domain
113113 for _, domain := range domains {
114- // Check certificate for the full domain
115- hasCert, certType := p.checkCertificateInCache(domain)
114+ // With the refactored CertificateManager, we don't need to manually check
115+ // the cache here. We can just attempt to ensure a certificate exists
116+ // by calling requestDomainCertificate, which delegates to GetCertificate.
117+ // GetCertificate will use autocert's cache and request a new cert only if needed.
116118
117- if certType == "staging" && p.config.LetsEncryptMode == "production" {
118- logger.Info("Found staging certificate but production mode is enabled",
119- "domain", domain,
120- "action", "will request new production certificate")
121- }
119+ logger.Debug("Ensuring certificate exists for domain",
120+ "domain", domain,
121+ "container", containerName)
122122
123- if hasCert {
124- logger.Debug("Certificate exists for domain",
125- "domain", domain,
126- "container", containerName)
127- } else {
128- logger.Info("Certificate not found in cache, requesting",
129- "domain", domain,
130- "container", containerName)
131-
132- // Request certificate (non-blocking)
133- go func(domain string) {
134- // Set flag to indicate we're processing a specific domain
135- p.processingSpecificDomain = true
136- // Try to request certificate for domain
137- p.requestDomainCertificate(domain)
138- // Reset flag when done
139- p.processingSpecificDomain = false
140- }(domain)
141- }
123+ // Request certificate (non-blocking).
124+ // GetCertificate handles checking cache/validity and requesting if necessary.
125+ go p.requestDomainCertificate(domain) // Simplified call
142126
143- // If this is a subdomain, check the parent domain too
127+ // If this is a subdomain, ensure the parent domain cert exists too
144128 parts := strings.Split(domain, ".")
145129 if len(parts) >= 3 {
146- // This is a subdomain (e.g., sub.domain.com)
147130 parentDomain := strings.Join(parts[1:], ".")
148131
149132 // Only check parent domain if it's not an ordinary TLD
150- if len(parts) > 2 && !isCommonTLD(parts[len(parts)-1]) {
151- hasCert, certType := p.checkCertificateInCache(parentDomain)
152- if certType == "staging" && p.config.LetsEncryptMode == "production" {
153- logger.Info("Found staging certificate but production mode is enabled",
154- "domain", parentDomain,
155- "action", "will request new production certificate")
156- }
157- if !hasCert {
158- logger.Info("Certificate not found for parent domain",
159- "domain", parentDomain,
160- "container", containerName)
161-
162- // Request certificate for parent domain also
163- go func(domain string) {
164- // Set flag to indicate we're processing a specific domain
165- p.processingSpecificDomain = true
166- p.requestDomainCertificate(domain)
167- // Reset flag when done
168- p.processingSpecificDomain = false
169- }(parentDomain)
170- }
133+ // and not the same as the root domain (to avoid redundant checks)
134+ if len(parts) > 2 && !isCommonTLD(parts[len(parts)-1]) && parentDomain != p.app.GetConfig().Http.Domain {
135+ logger.Debug("Ensuring certificate exists for parent domain",
136+ "parent_domain", parentDomain,
137+ "original_domain", domain,
138+ "container", containerName)
139+
140+ // Request certificate for parent domain also
141+ go p.requestDomainCertificate(parentDomain) // Simplified call
171142 }
172143 }
173144 }
M internal/proxy/core.go +228 -40
@@ -1,11 +1,16 @@
11 package proxy
22
33 import (
4+ "context"
5+ "fmt"
46 "net"
7+ "os"
8+ "strings"
59 "sync"
610 "time"
711
812 "crypto/tls"
13+ "database/sql"
914 "net/http"
1015
1116 "github.com/bnema/gordon/internal/common"
@@ -14,7 +19,6 @@ import (
1419 "github.com/bnema/gordon/pkg/docker"
1520 "github.com/bnema/gordon/pkg/logger"
1621 "github.com/labstack/echo/v4"
17- "golang.org/x/crypto/acme/autocert"
1822 )
1923
2024 // - core.go - Core proxy structure and constructor
@@ -29,39 +33,32 @@ type Proxy struct {
2933 config common.ReverseProxyConfig
3034 app interfaces.AppInterface
3135 httpsServer *echo.Echo
32- httpServer *echo.Echo
3336 routes map[string]*ProxyRouteInfo
3437 mu sync.RWMutex
35- certManager *autocert.Manager
36- fallbackCert *tls.Certificate // Fallback self-signed certificate
3738 serverStarted bool
39+ ctx context.Context // Context for storing shared data
3840
39- // Flag to track specific domain certificate operations
41+ // Flag to track specific domain certificate operations (May need review if CertManager handles this)
4042 processingSpecificDomain bool
4143
4244 // Store Gordon's own container ID at startup for reliable identification
4345 gordonContainerID string
4446
45- reverseProxyClient *http.Client
46- processingGlobalCert bool
47- dbMaxRetries int
48- dbRetryBaseDelay time.Duration
49- acmeManager *autocert.Manager
50- acmePerDomainRefreshMutex sync.Mutex
51- acmeDirCache autocert.DirCache
52- shutdown chan struct{} // Channel for signaling shutdown to background goroutines
47+ reverseProxyClient *http.Client
48+ dbMaxRetries int
49+ dbRetryBaseDelay time.Duration
50+ shutdown chan struct{} // Channel for signaling shutdown to background goroutines
5351
5452 // Track recently created containers to prevent recreation logic
5553 recentContainers map[string]time.Time
5654 recentContainersMu sync.RWMutex
5755 containerCooldownPeriod time.Duration
5856
59- // Upstream proxy detection
60- upstreamProxyDetected bool
61- upstreamProxyMu sync.RWMutex
57+ // Certificate Management
58+ certificateManager *CertificateManager // New refactored manager
6259
63- // SQL queries
64- queries *queries.ProxyQueries
60+ // SQL Queries
61+ Queries *queries.ProxyQueries
6562 }
6663
6764 // NewProxy creates a new instance of the reverse proxy
@@ -80,6 +77,7 @@ func NewProxy(app interfaces.AppInterface) (*Proxy, error) {
8077 app: app,
8178 serverStarted: false,
8279 shutdown: make(chan struct{}),
80+ ctx: context.Background(),
8381 }, nil
8482 }
8583
@@ -96,6 +94,7 @@ func NewProxy(app interfaces.AppInterface) (*Proxy, error) {
9694 app: app,
9795 serverStarted: false,
9896 shutdown: make(chan struct{}),
97+ ctx: context.Background(),
9998 }, nil
10099 }
101100
@@ -103,10 +102,6 @@ func NewProxy(app interfaces.AppInterface) (*Proxy, error) {
103102 httpsServer := echo.New()
104103 httpsServer.HideBanner = true
105104
106- // Set up the echo server for HTTP traffic (redirects to HTTPS)
107- httpServer := echo.New()
108- httpServer.HideBanner = true
109-
110105 // Initialize container storage and cooldown settings
111106 recentContainers := make(map[string]time.Time)
112107 containerCooldownPeriod := time.Duration(10) * time.Second
@@ -145,10 +140,10 @@ func NewProxy(app interfaces.AppInterface) (*Proxy, error) {
145140
146141 // Create the proxy instance
147142 p := &Proxy{
148- config: config,
149- app: app,
150- httpsServer: httpsServer,
151- httpServer: httpServer,
143+ config: config,
144+ app: app,
145+ httpsServer: httpsServer,
146+ // httpServer: httpServer, // Removed assignment
152147 routes: make(map[string]*ProxyRouteInfo),
153148 reverseProxyClient: reverseProxyClient,
154149 dbMaxRetries: dbMaxRetries,
@@ -156,20 +151,51 @@ func NewProxy(app interfaces.AppInterface) (*Proxy, error) {
156151 recentContainers: recentContainers,
157152 containerCooldownPeriod: containerCooldownPeriod,
158153 shutdown: make(chan struct{}),
159- queries: proxyQueries,
154+ Queries: proxyQueries,
155+ ctx: context.Background(),
156+ }
157+
158+ // --- Certificate Manager Setup ---
159+ // Determine cert directory
160+ certDir := config.CertDir
161+ if certDir == "" {
162+ certDir = app.GetConfig().General.StorageDir + "/certs"
160163 }
161164
162- // Setup the certificate manager for HTTPS
163- p.setupCertManager()
165+ // Initial upstream proxy detection (before creating CertManager)
166+ initialBehindTLSProxy := detectInitialUpstreamProxy(&config)
164167
165- // Verify the certificate manager was properly initialized
166- if p.certManager == nil {
167- logger.Error("Certificate manager initialization failed",
168- "solution", "Check certificate directory permissions and configuration")
168+ // Create Certificate Manager Config
169+ certManagerConfig := CertManagerConfig{
170+ CertDir: certDir,
171+ Email: config.Email,
172+ Mode: config.LetsEncryptMode, // staging or production
173+ SkipCertificates: config.SkipCertificates,
174+ BehindTLSProxy: initialBehindTLSProxy, // Pass initial detection result
175+ AdminDomain: app.GetConfig().Http.FullDomain(),
176+ RootDomain: app.GetConfig().Http.Domain,
177+ // RouteValidator: p.isHostInRoutes, // TODO: Pass a function if needed by hostPolicy
178+ }
179+
180+ // Create the Certificate Manager
181+ certMgr, err := NewCertificateManager(certManagerConfig, app, p.Queries)
182+ if err != nil {
183+ logger.Error("Failed to initialize Certificate Manager", "error", err)
184+ // Return an error to prevent starting a broken proxy
185+ return nil, fmt.Errorf("failed to initialize certificate manager: %w", err)
169186 } else {
170187 logger.Debug("Certificate manager successfully initialized")
188+ p.certificateManager = certMgr // Store the new manager
189+ }
190+
191+ // Ensure admin domain has proper ACME configuration
192+ if err := p.ensureAdminDomainConfig(); err != nil {
193+ logger.Error("Failed to ensure admin domain configuration", "error", err)
194+ return nil, fmt.Errorf("failed to ensure admin domain configuration: %w", err)
171195 }
172196
197+ // --- End Certificate Manager Setup ---
198+
173199 // Detect our container ID for later use
174200 p.detectGordonContainer()
175201
@@ -181,13 +207,6 @@ func (p *Proxy) Close() {
181207 // Signal all background goroutines to stop
182208 close(p.shutdown)
183209
184- // Close the HTTP and HTTPS servers if they exist
185- if p.httpServer != nil {
186- if err := p.httpServer.Close(); err != nil {
187- logger.Error("Error closing HTTP server", "error", err)
188- }
189- }
190-
191210 if p.httpsServer != nil {
192211 if err := p.httpsServer.Close(); err != nil {
193212 logger.Error("Error closing HTTPS server", "error", err)
@@ -196,3 +215,172 @@ func (p *Proxy) Close() {
196215
197216 logger.Info("Proxy resources cleaned up")
198217 }
218+
219+// detectInitialUpstreamProxy checks environment variables for signs of an upstream proxy
220+// This is used during initialization before the CertificateManager is created.
221+func detectInitialUpstreamProxy(config *common.ReverseProxyConfig) bool {
222+ // Check if detection is explicitly disabled
223+ if !config.DetectUpstreamProxy {
224+ logger.Debug("Upstream proxy detection disabled by config")
225+ return false
226+ }
227+
228+ // Check for common environment variables set by upstream proxies
229+ if os.Getenv("HTTPS") == "on" ||
230+ os.Getenv("HTTP_X_FORWARDED_PROTO") == "https" ||
231+ os.Getenv("HTTP_X_FORWARDED_SSL") == "on" {
232+ logger.Info("Initial upstream TLS-terminating proxy detected via environment variables",
233+ "detection_method", "environment_variables",
234+ "https", os.Getenv("HTTPS"),
235+ "x_forwarded_proto", os.Getenv("HTTP_X_FORWARDED_PROTO"),
236+ "x_forwarded_ssl", os.Getenv("HTTP_X_FORWARDED_SSL"))
237+ return true
238+ }
239+
240+ // Check Cloudflare headers
241+ if cfVisitor := os.Getenv("HTTP_CF_VISITOR"); cfVisitor != "" {
242+ if strings.Contains(cfVisitor, "\"scheme\":\"https\"") {
243+ logger.Info("Initial Cloudflare proxy detected via environment variables",
244+ "detection_method", "cloudflare_headers",
245+ "cf_visitor", cfVisitor)
246+ return true
247+ }
248+ }
249+
250+ logger.Debug("No initial upstream proxy detected via environment variables")
251+ return false
252+}
253+
254+// requestDomainCertificate requests a certificate for a domain
255+func (p *Proxy) requestDomainCertificate(domainName string) error {
256+ logger.Debug("Requesting certificate for domain", "domain", domainName)
257+
258+ // Check if we're already processing this domain
259+ if p.processingSpecificDomain {
260+ logger.Debug("Already processing a specific domain, skipping", "domain", domainName)
261+ return nil
262+ }
263+
264+ // Get domain configuration
265+ var acmeEnabled bool
266+ // Use sql.NullString for potentially nullable string columns
267+ var acmeChallengeType, acmeDnsProvider, acmeDnsCredentialsRef sql.NullString
268+ err := p.app.GetDB().QueryRow(p.Queries.GetDomainAcmeConfig, domainName).Scan(
269+ &acmeEnabled, &acmeChallengeType, &acmeDnsProvider, &acmeDnsCredentialsRef,
270+ )
271+ if err != nil {
272+ // Handle case where the domain might not exist in the acme_configs table yet
273+ if err == sql.ErrNoRows {
274+ logger.Debug("No ACME configuration found for domain, assuming ACME is disabled", "domain", domainName)
275+ return nil // Treat as ACME disabled if no config exists
276+ }
277+ logger.Error("Failed to get domain ACME configuration", "domain", domainName, "error", err)
278+ return err
279+ }
280+
281+ if !acmeEnabled {
282+ logger.Debug("ACME not enabled for domain", "domain", domainName)
283+ return nil
284+ }
285+
286+ // Check if certificate exists and is valid
287+ var certFile, keyFile string
288+ var issuedAtStr, expiresAtStr string // Read timestamps as strings first
289+ var issuer, status string
290+ var issuedAt, expiresAt time.Time // Keep these for use after parsing
291+
292+ // Scan from DB, reading timestamps as strings
293+ err = p.app.GetDB().QueryRow(p.Queries.GetCertificateByDomain, domainName).Scan(
294+ &certFile, &keyFile, &issuedAtStr, &expiresAtStr, &issuer, &status,
295+ )
296+
297+ // Handle potential sql.ErrNoRows if the certificate doesn't exist yet
298+ if err != nil && err != sql.ErrNoRows {
299+ logger.Error("Failed to check existing certificate", "domain", domainName, "error", err)
300+ return err
301+ }
302+
303+ // If a certificate was found (no error or ErrNoRows), parse the timestamps
304+ if err == nil {
305+ issuedAt, err = time.Parse(time.RFC3339, issuedAtStr)
306+ if err != nil {
307+ logger.Error("Failed to parse issued_at timestamp from DB", "domain", domainName, "value", issuedAtStr, "error", err)
308+ // Return the parse error as it indicates data corruption or an unexpected format
309+ return fmt.Errorf("failed to parse issued_at for domain %s: %w", domainName, err)
310+ }
311+ expiresAt, err = time.Parse(time.RFC3339, expiresAtStr)
312+ if err != nil {
313+ logger.Error("Failed to parse expires_at timestamp from DB", "domain", domainName, "value", expiresAtStr, "error", err)
314+ // Return the parse error
315+ return fmt.Errorf("failed to parse expires_at for domain %s: %w", domainName, err)
316+ }
317+ logger.Debug("Successfully scanned and parsed certificate timestamps", "domain", domainName, "issued", issuedAt, "expires", expiresAt)
318+ }
319+ // Note: If err was sql.ErrNoRows, issuedAt/expiresAt will remain zero time.Time, which is handled correctly later.
320+
321+ // If certificate exists and is valid
322+ if err == nil && status == "valid" {
323+ // Check if certificate needs renewal
324+ renewBefore := time.Duration(p.config.RenewBefore) * 24 * time.Hour
325+ if time.Now().Add(renewBefore).Before(expiresAt) {
326+ logger.Debug("Certificate is still valid and does not need renewal",
327+ "domain", domainName,
328+ "expires_at", expiresAt,
329+ "renew_before_days", p.config.RenewBefore)
330+ return nil // Certificate is valid, no action needed
331+ }
332+ logger.Debug("Certificate requires renewal", "domain", domainName, "expires_at", expiresAt)
333+ } else if err == sql.ErrNoRows {
334+ logger.Debug("No existing certificate found for domain", "domain", domainName)
335+ } else {
336+ logger.Debug("Existing certificate is not valid", "domain", domainName, "status", status, "scan_error", err)
337+ }
338+
339+ // --- Proceed with obtaining/renewing certificate ---
340+
341+ // Check if required ACME config fields are present now that we know ACME is enabled
342+ if !acmeChallengeType.Valid || acmeChallengeType.String == "" {
343+ err := fmt.Errorf("ACME challenge type is missing for domain %s", domainName)
344+ logger.Error(err.Error())
345+ return err
346+ }
347+ // If DNS challenge, provider and credentials ref are required
348+ if acmeChallengeType.String == "dns-01" {
349+ if !acmeDnsProvider.Valid || acmeDnsProvider.String == "" {
350+ err := fmt.Errorf("ACME DNS provider is missing for domain %s using dns-01 challenge", domainName)
351+ logger.Error(err.Error())
352+ return err
353+ }
354+ if !acmeDnsCredentialsRef.Valid || acmeDnsCredentialsRef.String == "" {
355+ err := fmt.Errorf("ACME DNS credentials reference is missing for domain %s using dns-01 challenge", domainName)
356+ logger.Error(err.Error())
357+ return err
358+ }
359+ // Log the DNS provider being used (optional, but helpful for debugging)
360+ logger.Debug("Using DNS provider for ACME challenge", "domain", domainName, "provider", acmeDnsProvider.String)
361+ }
362+
363+ // Set processing flag
364+ p.processingSpecificDomain = true
365+ defer func() {
366+ p.processingSpecificDomain = false
367+ }()
368+
369+ // Request new certificate using the CertificateManager
370+ cert, err := p.certificateManager.ObtainCertificate(domainName, p) // Pass 'p' (Proxy instance)
371+ if err != nil {
372+ logger.Error("Failed to obtain/renew certificate", "domain", domainName, "error", err)
373+ // Update certificate status to 'failed' in DB?
374+ // _, dbErr := p.app.GetDB().Exec(p.Queries.UpdateCertificateStatus, "failed", domainName)
375+ // if dbErr != nil {
376+ // logger.Error("Failed to update certificate status after obtain failure", "domain", domainName, "db_error", dbErr)
377+ // }
378+ return err // Return the obtain error
379+ }
380+
381+ logger.Info("Successfully obtained certificate",
382+ "domain", domainName,
383+ "cert_url", cert.CertURL)
384+
385+ return nil
386+}
M internal/proxy/db_helpers.go +60 -0
@@ -2,8 +2,11 @@ package proxy
22
33 import (
44 "database/sql"
5+ "fmt"
6+ "time"
57
68 "github.com/bnema/gordon/internal/db"
9+ "github.com/bnema/gordon/pkg/logger"
710 )
811
912 // dbExecWithRetry performs a database execution with retry logic for handling locked database
@@ -25,3 +28,60 @@ func (p *Proxy) dbBeginWithRetry() (*sql.Tx, error) {
2528 func txExecWithRetry(tx *sql.Tx, query string, args ...interface{}) (sql.Result, error) {
2629 return db.TxExecWithRetry(tx, query, args...)
2730 }
31+
32+// ensureAdminDomainConfig ensures the admin domain has proper ACME configuration in the database
33+func (p *Proxy) ensureAdminDomainConfig() error {
34+ adminDomain := p.app.GetConfig().Http.FullDomain()
35+ adminAccountID := "" // Variable to store the admin account ID
36+
37+ // Check if domain exists
38+ var exists bool
39+ err := p.app.GetDB().QueryRow(p.Queries.CheckDomainExists, adminDomain).Scan(&exists)
40+ if err != nil {
41+ return fmt.Errorf("failed to check admin domain existence: %w", err)
42+ }
43+
44+ // Get the first account ID (assumed to be the admin)
45+ err = p.app.GetDB().QueryRow(p.Queries.GetFirstAccount).Scan(&adminAccountID)
46+ if err != nil {
47+ if err == sql.ErrNoRows {
48+ // This case should ideally not happen if account setup is done first
49+ logger.Error("No accounts found in the database. Cannot associate admin domain.")
50+ return fmt.Errorf("no accounts found in database")
51+ }
52+ return fmt.Errorf("failed to get admin account ID: %w", err)
53+ }
54+ if adminAccountID == "" {
55+ // Sanity check
56+ logger.Error("Admin account ID is empty. Cannot associate admin domain.")
57+ return fmt.Errorf("admin account ID is empty")
58+ }
59+
60+ if !exists {
61+ // Create the admin domain with proper ACME config and account ID
62+ now := time.Now().Format(time.RFC3339)
63+ _, err = p.app.GetDB().Exec(p.Queries.CreateAdminDomain,
64+ adminDomain, // ID (assuming domain name is unique ID here?)
65+ adminDomain, // Name
66+ adminAccountID, // Account ID (NEW)
67+ now, // Created At
68+ now, // Updated At
69+ true, // ACME Enabled
70+ "http-01", // ACME Challenge Type
71+ )
72+ if err != nil {
73+ return fmt.Errorf("failed to create admin domain: %w", err)
74+ }
75+ logger.Info("Created admin domain with ACME configuration", "domain", adminDomain, "account_id", adminAccountID)
76+ } else {
77+ // Update existing domain to ensure ACME is enabled (no need to update account_id here, assume it's set correctly)
78+ now := time.Now().Format(time.RFC3339)
79+ _, err = p.app.GetDB().Exec(p.Queries.UpdateAdminDomainAcme, now, adminDomain)
80+ if err != nil {
81+ return fmt.Errorf("failed to update admin domain ACME config: %w", err)
82+ }
83+ logger.Info("Updated admin domain ACME configuration", "domain", adminDomain)
84+ }
85+
86+ return nil
87+}
M internal/proxy/middleware.go +18 -14
@@ -21,22 +21,20 @@ const RequestIDKey = "request_id"
2121
2222 // setupMiddleware configures middleware for both HTTP and HTTPS servers
2323 func (p *Proxy) setupMiddleware() {
24- // Configure Echo to trust X-Forwarded-* headers
25- // Since we are the reverse proxy, we need to make sure we identify client IPs correctly
26- p.httpsServer.IPExtractor = echo.ExtractIPFromXFFHeader()
27- p.httpServer.IPExtractor = echo.ExtractIPFromXFFHeader()
24+ // Configure Echo IP extraction. Since the proxy is edge-facing (directly exposed),
25+ // ignore X-Forwarded-For and use the direct remote address.
26+ p.httpsServer.IPExtractor = echo.ExtractIPDirect()
2827
2928 // Add UUID generator middleware for both HTTP and HTTPS servers
3029 p.httpsServer.Use(p.createRequestIDMiddleware())
31- p.httpServer.Use(p.createRequestIDMiddleware())
3230
3331 // Check if rate limiting is disabled via configuration
3432 if p.config.EnableRateLimit {
3533 // Create a new Starskey rate limiter store for HTTPS server
3634 httpsRateLimiter, err := kv.NewStarskeyRateLimiterStore(
3735 "./data/ratelimiter/https", // Path to store rate limit data
38- 10, // Rate: 10 requests per second (lowered from 10)
39- 30, // Burst: 30 requests (lowered from 30)
36+ 10, // Rate: 10 requests per second
37+ 30, // Burst: 30 requests
4038 3*time.Minute, // Expires in 3 minutes
4139 )
4240 if err != nil {
@@ -46,12 +44,14 @@ func (p *Proxy) setupMiddleware() {
4644 p.httpsServer.Use(middleware.RateLimiterWithConfig(middleware.RateLimiterConfig{
4745 Store: middleware.NewRateLimiterMemoryStoreWithConfig(
4846 middleware.RateLimiterMemoryStoreConfig{
49- Rate: rate.Limit(10), // 10 requests per second (lowered from 10)
50- Burst: 30, // Burst of 30 requests (lowered from 30)
47+ Rate: rate.Limit(10), // 10 requests per second
48+ Burst: 30, // Burst of 30 requests
5149 ExpiresIn: 3 * time.Minute, // Store expiration
5250 },
5351 ),
5452 DenyHandler: func(context echo.Context, identifier string, err error) error {
53+ logger.Info("[RateLimit Debug] DenyHandler invoked for Memory Fallback", "identifier", identifier, "error", err.Error())
54+
5555 logger.Warn("Rate limit exceeded",
5656 "ip", identifier,
5757 "path", context.Request().URL.Path,
@@ -68,6 +68,8 @@ func (p *Proxy) setupMiddleware() {
6868 Store: httpsRateLimiter,
6969 IdentifierExtractor: func(ctx echo.Context) (string, error) {
7070 id := ctx.RealIP()
71+ logger.Info("[RateLimit Debug] IdentifierExtractor called", "ip_identified", id, "remote_addr", ctx.Request().RemoteAddr)
72+
7173 logger.Debug("Rate limiting HTTPS request",
7274 "ip", id,
7375 "remote_addr", ctx.Request().RemoteAddr)
@@ -78,6 +80,8 @@ func (p *Proxy) setupMiddleware() {
7880 return id, nil
7981 },
8082 DenyHandler: func(context echo.Context, identifier string, err error) error {
83+ logger.Info("[RateLimit Debug] DenyHandler invoked for Starskey", "identifier", identifier, "error", err.Error())
84+
8185 logger.Warn("HTTPS rate limit exceeded",
8286 "ip", identifier,
8387 "path", context.Request().URL.Path,
@@ -124,7 +128,7 @@ func (p *Proxy) setupMiddleware() {
124128 logger.Error("Failed to create HTTP rate limiter store, falling back to memory store", "error", err)
125129
126130 // Fallback to memory store if Starskey store creation fails
127- p.httpServer.Use(middleware.RateLimiterWithConfig(middleware.RateLimiterConfig{
131+ p.httpsServer.Use(middleware.RateLimiterWithConfig(middleware.RateLimiterConfig{
128132 Store: middleware.NewRateLimiterMemoryStoreWithConfig(
129133 middleware.RateLimiterMemoryStoreConfig{
130134 Rate: rate.Limit(2), // 2 requests per second (lowered from 10)
@@ -145,7 +149,7 @@ func (p *Proxy) setupMiddleware() {
145149 logger.Debug("Using Starskey rate limiter for HTTP server")
146150
147151 // Add custom Starskey rate limiter middleware
148- p.httpServer.Use(middleware.RateLimiterWithConfig(middleware.RateLimiterConfig{
152+ p.httpsServer.Use(middleware.RateLimiterWithConfig(middleware.RateLimiterConfig{
149153 Store: httpRateLimiter,
150154 IdentifierExtractor: func(ctx echo.Context) (string, error) {
151155 id := ctx.RealIP()
@@ -180,16 +184,16 @@ func (p *Proxy) setupMiddleware() {
180184
181185 // Add custom logger middleware with request ID for HTTP server if enabled in config
182186 if p.config.EnableHttpLogs {
183- p.httpServer.Use(middleware.LoggerWithConfig(middleware.LoggerConfig{
187+ p.httpsServer.Use(middleware.LoggerWithConfig(middleware.LoggerConfig{
184188 Format: `{"time":"${time_rfc3339_nano}","id":"${id}","remote_ip":"${remote_ip}","host":"${host}","method":"${method}","uri":"${uri}","user_agent":"${user_agent}","status":${status},"error":"${error}","latency":${latency},"latency_human":"${latency_human}","bytes_in":${bytes_in},"bytes_out":${bytes_out}}` + "\n",
185189 CustomTimeFormat: "2006-01-02T15:04:05.00000Z07:00",
186190 Skipper: func(c echo.Context) bool {
187191 return !p.config.EnableHttpLogs
188192 },
189193 }))
190- logger.Debug("HTTP request logging enabled for HTTP server")
194+ logger.Debug("HTTP request logging enabled for HTTPS server")
191195 } else {
192- logger.Debug("HTTP request logging disabled for HTTP server")
196+ logger.Debug("HTTP request logging disabled for HTTPS server")
193197 }
194198 }
195199
M internal/proxy/routes.go +53 -43
@@ -52,7 +52,7 @@ func (p *Proxy) loadRoutes() error {
5252 }
5353
5454 // Query the database for active proxy routes using retry mechanism
55- rows, err := p.dbQueryWithRetry(p.queries.GetActiveRoutes)
55+ rows, err := p.dbQueryWithRetry(p.Queries.GetActiveRoutes)
5656 if err != nil {
5757 if err == sql.ErrNoRows {
5858 logger.Info("No active proxy routes found")
@@ -163,11 +163,6 @@ func (p *Proxy) configureRoutes() {
163163 if p.httpsServer != nil {
164164 p.httpsServer.Any("/*", p.proxyRequest)
165165 }
166-
167- // Also add the handler to the HTTP server for HTTP-01 challenges
168- if p.httpServer != nil {
169- p.httpServer.Any("/*", p.proxyRequest)
170- }
171166 }
172167
173168 // AddRoute adds a new route to the database and reloads the proxy
@@ -230,7 +225,7 @@ func (p *Proxy) AddRoute(domainName, containerID, containerIP, containerPort, pr
230225
231226 // Get the account ID - default to the first account if available
232227 var accountID string
233- err = tx.QueryRow(p.queries.GetFirstAccount).Scan(&accountID)
228+ err = tx.QueryRow(p.Queries.GetFirstAccount).Scan(&accountID)
234229 if err != nil {
235230 if err == sql.ErrNoRows {
236231 logger.Warn("No account found when adding route, domains will have NULL account_id")
@@ -242,14 +237,14 @@ func (p *Proxy) AddRoute(domainName, containerID, containerIP, containerPort, pr
242237
243238 // Check if the domain exists
244239 var domainID string
245- err = tx.QueryRow(p.queries.GetDomainByName, hostname).Scan(&domainID)
240+ err = tx.QueryRow(p.Queries.GetDomainByName, hostname).Scan(&domainID)
246241 if err != nil {
247242 if err == sql.ErrNoRows {
248243 // Domain doesn't exist, create it
249244 domainID = uuid.New().String()
250245 now := time.Now().Format(time.RFC3339)
251246 _, err = txExecWithRetry(tx,
252- p.queries.InsertDomain,
247+ p.Queries.InsertDomain,
253248 domainID, hostname, accountID, now, now,
254249 )
255250 if err != nil {
@@ -278,7 +273,7 @@ func (p *Proxy) AddRoute(domainName, containerID, containerIP, containerPort, pr
278273 // Domain part doesn't exist, create it
279274 domainPartID = uuid.New().String()
280275 _, err = txExecWithRetry(tx,
281- p.queries.InsertDomain,
276+ p.Queries.InsertDomain,
282277 domainPartID, domainPart, accountID, now, now,
283278 )
284279 if err != nil {
@@ -305,7 +300,7 @@ func (p *Proxy) AddRoute(domainName, containerID, containerIP, containerPort, pr
305300 var existingRouteID string
306301 var existingContainerID string
307302 var existingContainerPort string
308- err = tx.QueryRow(p.queries.GetRouteByDomain, domainID).Scan(&existingRouteID, &existingContainerID, &existingContainerPort)
303+ err = tx.QueryRow(p.Queries.GetRouteByDomain, domainID).Scan(&existingRouteID, &existingContainerID, &existingContainerPort)
309304
310305 if err == nil {
311306 // Route exists, update it
@@ -326,16 +321,9 @@ func (p *Proxy) AddRoute(domainName, containerID, containerIP, containerPort, pr
326321 "new_port", containerPort)
327322 }
328323
324+ // Use the UpdateRoute query from the queries package
329325 _, err = txExecWithRetry(tx,
330- `UPDATE proxy_route SET
331- container_id = ?,
332- container_ip = ?,
333- container_port = ?,
334- protocol = ?,
335- path = ?,
336- active = ?,
337- updated_at = ?
338- WHERE id = ?`,
326+ p.Queries.UpdateRoute,
339327 containerID, containerIP, containerPort, protocol, path, true, now, existingRouteID,
340328 )
341329 if err != nil {
@@ -349,7 +337,7 @@ func (p *Proxy) AddRoute(domainName, containerID, containerIP, containerPort, pr
349337 routeID := uuid.New().String()
350338 now := time.Now().Format(time.RFC3339)
351339 _, err = txExecWithRetry(tx,
352- p.queries.InsertRoute,
340+ p.Queries.InsertRoute,
353341 routeID, domainID, containerID, containerIP, containerPort,
354342 protocol, path, true, now, now,
355343 )
@@ -414,7 +402,7 @@ func (p *Proxy) AddRoute(domainName, containerID, containerIP, containerPort, pr
414402 // Set the flag to indicate we're processing a specific domain
415403 p.processingSpecificDomain = true
416404 // Try to request certificate for domain
417- p.requestDomainCertificate(domain)
405+ p.requestCertificateForDomain(domain)
418406 // Reset the flag after we're done
419407 p.processingSpecificDomain = false
420408 }(hostname)
@@ -442,7 +430,7 @@ func (p *Proxy) RemoveRoute(domainName string) error {
442430
443431 // Get the domain ID
444432 var domainID string
445- err = tx.QueryRow(p.queries.GetDomainByName, domainName).Scan(&domainID)
433+ err = tx.QueryRow(p.Queries.GetDomainByName, domainName).Scan(&domainID)
446434 if err != nil {
447435 if err == sql.ErrNoRows {
448436 return fmt.Errorf("domain not found: %s", domainName)
@@ -451,13 +439,13 @@ func (p *Proxy) RemoveRoute(domainName string) error {
451439 }
452440
453441 // Delete the route with retry
454- _, err = txExecWithRetry(tx, p.queries.DeleteRouteByDomainID, domainID)
442+ _, err = txExecWithRetry(tx, p.Queries.DeleteRouteByDomainID, domainID)
455443 if err != nil {
456444 return fmt.Errorf("failed to delete route: %w", err)
457445 }
458446
459447 // Delete the domain with retry
460- _, err = txExecWithRetry(tx, p.queries.DeleteDomainByID, domainID)
448+ _, err = txExecWithRetry(tx, p.Queries.DeleteDomainByID, domainID)
461449 if err != nil {
462450 return fmt.Errorf("failed to delete domain: %w", err)
463451 }
@@ -521,7 +509,7 @@ func (p *Proxy) Reload() error {
521509 newRoutes := make(map[string]*ProxyRouteInfo)
522510
523511 // Load routes from the database into the new map
524- rows, err := p.app.GetDB().Query(p.queries.GetActiveRoutes)
512+ rows, err := p.app.GetDB().Query(p.Queries.GetActiveRoutes)
525513 if err != nil {
526514 if err == sql.ErrNoRows {
527515 logger.Info("No active proxy routes found during reload")
@@ -644,12 +632,6 @@ func (p *Proxy) Reload() error {
644632 // Rebuild route configuration
645633 p.configureRoutes()
646634
647- // Force recreation of the route handlers if servers are initialized
648- if p.httpServer != nil {
649- logger.Debug("Refreshing HTTP routes")
650- p.httpServer.Routes()
651- }
652-
653635 if p.httpsServer != nil {
654636 logger.Debug("Refreshing HTTPS routes")
655637 p.httpsServer.Routes()
@@ -726,7 +708,7 @@ func (p *Proxy) ForceUpdateRouteIP(domain string, newIP string) error {
726708 // Next, update the database
727709 // Get the domain ID with retry
728710 var domainID string
729- rows, err := p.app.GetDB().Query(p.queries.GetDomainByName, domain)
711+ rows, err := p.app.GetDB().Query(p.Queries.GetDomainByName, domain)
730712 if err != nil {
731713 logger.Error("Failed to query domain for IP update", "domain", domain, "error", err)
732714 return fmt.Errorf("failed to query domain: %w", err)
@@ -747,7 +729,7 @@ func (p *Proxy) ForceUpdateRouteIP(domain string, newIP string) error {
747729
748730 // Update the proxy_route record with retry and ensure it's marked as active
749731 now := time.Now().Format(time.RFC3339)
750- result, err := p.app.GetDB().Exec(p.queries.UpdateRouteIP, newIP, now, domainID, containerID)
732+ result, err := p.app.GetDB().Exec(p.Queries.UpdateRouteIP, newIP, now, domainID, containerID)
751733
752734 if err != nil {
753735 logger.Error("Failed to update IP in database", "domain", domain, "error", err)
@@ -817,7 +799,7 @@ func (p *Proxy) FindRoutesByOldName(containerName string) map[string]*ProxyRoute
817799 result := make(map[string]*ProxyRouteInfo)
818800
819801 // Query the database with retry to find if any route might be associated with this container name
820- rows, err := p.app.GetDB().Query(p.queries.GetAllRoutes)
802+ rows, err := p.app.GetDB().Query(p.Queries.GetAllRoutes)
821803 if err != nil {
822804 logger.Error("Failed to query database for routes by old name", "error", err)
823805 return result
@@ -1182,7 +1164,7 @@ func (p *Proxy) StartContainerEventListener() error {
11821164
11831165 // Update the IP in the database directly
11841166 now := time.Now().Format(time.RFC3339)
1185- _, err = p.dbExecWithRetry(p.queries.UpdateRouteIP,
1167+ _, err = p.dbExecWithRetry(p.Queries.UpdateRouteIP,
11861168 containerIP, now, domainID, containerID)
11871169
11881170 if err != nil {
@@ -1374,7 +1356,7 @@ func (p *Proxy) markRoutesInactive(domains []string) {
13741356 for _, domain := range domains {
13751357 // First, get the domain ID
13761358 var domainID string
1377- err := tx.QueryRow(p.queries.GetDomainByName, domain).Scan(&domainID)
1359+ err := tx.QueryRow(p.Queries.GetDomainByName, domain).Scan(&domainID)
13781360 if err != nil {
13791361 logger.Error("Failed to get domain ID for inactive route", "domain", domain, "error", err)
13801362 continue
@@ -1382,7 +1364,7 @@ func (p *Proxy) markRoutesInactive(domains []string) {
13821364
13831365 // Update the route to mark it as inactive
13841366 now := time.Now().Format(time.RFC3339)
1385- _, err = tx.Exec(p.queries.MarkRouteInactive, now, domainID)
1367+ _, err = tx.Exec(p.Queries.MarkRouteInactive, now, domainID)
13861368 if err != nil {
13871369 logger.Error("Failed to mark route as inactive in database", "domain", domain, "error", err)
13881370 continue
@@ -1785,11 +1767,11 @@ func (p *Proxy) proxyRequest(c echo.Context) error {
17851767 host = strings.TrimSuffix(strings.ToLower(host), ".")
17861768
17871769 // Debug log the incoming request
1788- logger.Debug("Proxy request received",
1789- "host", host,
1790- "method", c.Request().Method,
1791- "path", c.Request().URL.Path,
1792- "client_ip", c.RealIP())
1770+ // logger.Debug("Proxy request received",
1771+ // "host", host,
1772+ // "method", c.Request().Method,
1773+ // "path", c.Request().URL.Path,
1774+ // "client_ip", c.RealIP())
17931775
17941776 // Get the route for this host
17951777 p.mu.RLock()
@@ -2086,3 +2068,31 @@ func (p *Proxy) IsContainerInCooldownPeriod(containerID string) bool {
20862068 // Check if the container is still in its cooldown period
20872069 return time.Since(timestamp) < p.containerCooldownPeriod
20882070 }
2071+
2072+// Add specific domain handling to use our new certificate manager
2073+
2074+// requestCertificateForDomain requests a certificate for a specific domain using our new refactored system
2075+func (p *Proxy) requestCertificateForDomain(domain string) {
2076+ // Use the certificate manager stored on the Proxy struct
2077+ if p.certificateManager != nil {
2078+ // Request the certificate using the new manager
2079+ _, err := p.certificateManager.GetCertificate(domain)
2080+ if err != nil {
2081+ logger.Error("Failed to get certificate with new manager",
2082+ "domain", domain,
2083+ "error", err)
2084+ } else {
2085+ logger.Info("Successfully secured certificate for domain",
2086+ "domain", domain)
2087+ }
2088+ return
2089+ }
2090+
2091+ // Fall back to old method if new manager not available
2092+ // logger.Debug("Using legacy certificate manager for domain",
2093+ // "domain", domain)
2094+ // p.requestDomainCertificateRefactored(domain) // Removed fallback to non-existent method
2095+
2096+ // If manager is nil, log an error (should have been caught earlier)
2097+ logger.Error("requestCertificateForDomain called but certificateManager is nil", "domain", domain)
2098+}
M internal/proxy/server.go +123 -207
@@ -11,8 +11,6 @@ import (
1111
1212 "github.com/bnema/gordon/pkg/docker"
1313 "github.com/bnema/gordon/pkg/logger"
14- "github.com/labstack/echo/v4"
15- "golang.org/x/crypto/acme"
1614 )
1715
1816 // This file contains server operations functions for the proxy.
@@ -153,205 +151,12 @@ func (p *Proxy) Start() error {
153151 // Configure routes
154152 p.configureRoutes()
155153
156- // Configure HTTP server to handle Let's Encrypt HTTP-01 challenges
157- // and redirect everything else to HTTPS
158- p.httpServer.Any("/.well-known/acme-challenge/*", echo.WrapHandler(p.certManager.HTTPHandler(nil)))
159-
160- // Create a custom HTTPS server with proper TLS config
161- httpsServer := &http.Server{
162- Addr: fmt.Sprintf(":%s", p.config.Port),
163- Handler: p.httpsServer,
164- TLSConfig: &tls.Config{
165- GetCertificate: func(hello *tls.ClientHelloInfo) (*tls.Certificate, error) {
166- // Log SNI information in debug mode
167- if hello.ServerName != "" {
168- logger.Debug("TLS handshake with SNI",
169- "server_name", hello.ServerName)
170- }
171-
172- // Verify certificate manager is properly initialized
173- if p.certManager == nil {
174- logger.Error("TLS certificate manager is nil during handshake - certificate initialization failed",
175- "server_name", hello.ServerName,
176- "solution", "Check certificate configuration and logs for setup errors")
177-
178- // Use fallback certificate if we have one
179- if p.fallbackCert != nil {
180- logger.Warn("Using fallback self-signed certificate due to missing certificate manager",
181- "server_name", hello.ServerName)
182- return p.fallbackCert, nil
183- }
184-
185- return nil, fmt.Errorf("certificate manager not initialized")
186- }
187-
188- // Try to get the certificate from the autocert manager
189- logger.Debug("Attempting to get certificate from manager",
190- "server_name", hello.ServerName,
191- "has_fallback", p.fallbackCert != nil)
192-
193- // First try checking if we have a valid certificate in cache
194- cacheKey := "cert-" + hello.ServerName
195- certCache := p.certManager.Cache
196- ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
197- defer cancel()
198-
199- if certCache != nil {
200- certData, err := certCache.Get(ctx, cacheKey)
201- if err == nil && len(certData) > 0 {
202- logger.Debug("Found certificate in cache", "domain", hello.ServerName)
203- // Use directly if found in cache - skip the potentially slower autocert.Manager GetCertificate
204- keyKey := "key-" + hello.ServerName
205- keyData, keyErr := certCache.Get(ctx, keyKey)
206- if keyErr == nil && len(keyData) > 0 {
207- cert, err := tls.X509KeyPair(certData, keyData)
208- if err == nil {
209- logger.Debug("Successfully loaded certificate from cache",
210- "domain", hello.ServerName)
211- return &cert, nil
212- }
213- logger.Error("Failed to create X509KeyPair from cache",
214- "domain", hello.ServerName,
215- "error", err)
216- } else {
217- logger.Error("Certificate found in cache but key is missing or invalid",
218- "domain", hello.ServerName,
219- "key_error", keyErr)
220- }
221- } else if err != nil {
222- logger.Debug("Certificate not found in cache during TLS handshake",
223- "domain", hello.ServerName,
224- "error", err)
225-
226- // Check if we have it on disk but not in cache (unlikely after preloading)
227- hasOnDisk, certType := p.checkCertificateInFilesystem(hello.ServerName)
228- if hasOnDisk {
229- logger.Warn("Certificate found on disk but not in cache - this should not happen after preloading",
230- "domain", hello.ServerName,
231- "type", certType,
232- "solution", "Check preloadCertificates implementation")
233- }
234- }
235- }
236-
237- // If not in cache, use the standard GetCertificate method
238- // Let's Encrypt may be contacted if no certificate exists
239- cert, err := p.certManager.GetCertificate(hello)
240-
241- if err != nil {
242- logger.Error("Error getting certificate from manager",
243- "server_name", hello.ServerName,
244- "error", err)
245-
246- // Check for rate limit errors
247- if strings.Contains(err.Error(), "too many certificates") ||
248- strings.Contains(err.Error(), "rateLimited") {
249- logger.Error("Let's Encrypt rate limit hit during TLS handshake",
250- "domain", hello.ServerName,
251- "solution", "Wait for rate limit to expire or manually add certificate")
252- }
253-
254- // Try manually loading certificate files directly as a fallback before using fallback cert
255- if manualCert, manualErr := p.manuallyLoadCertificate(hello.ServerName); manualErr == nil {
256- logger.Info("Successfully loaded certificate manually as fallback",
257- "domain", hello.ServerName)
258- return manualCert, nil
259- } else {
260- logger.Debug("Manual certificate loading also failed",
261- "domain", hello.ServerName,
262- "error", manualErr)
263- }
264-
265- // If we can't get a certificate and we have a fallback, use it for the admin domain
266- if hello.ServerName == p.app.GetConfig().Http.FullDomain() && p.fallbackCert != nil {
267- logger.Debug("Using fallback certificate for admin domain",
268- "domain", hello.ServerName,
269- "error", err)
270- return p.fallbackCert, nil
271- }
272- } else if cert == nil {
273- logger.Error("Certificate manager returned nil certificate without error",
274- "server_name", hello.ServerName)
275-
276- // Try manually loading certificate files directly as a fallback
277- if manualCert, manualErr := p.manuallyLoadCertificate(hello.ServerName); manualErr == nil {
278- logger.Info("Successfully loaded certificate manually when autocert returned nil",
279- "domain", hello.ServerName)
280- return manualCert, nil
281- }
282-
283- // If we can't get a certificate and we have a fallback, use it for the admin domain
284- if hello.ServerName == p.app.GetConfig().Http.FullDomain() && p.fallbackCert != nil {
285- logger.Debug("Using fallback certificate for admin domain",
286- "domain", hello.ServerName)
287- return p.fallbackCert, nil
288- }
289- } else {
290- logger.Debug("Successfully obtained certificate from manager",
291- "server_name", hello.ServerName)
292- }
293-
294- return cert, err
295- },
296- MinVersion: tls.VersionTLS12,
297- // Add server name to use when client doesn't send SNI
298- ServerName: p.app.GetConfig().Http.FullDomain(),
299- // Add support for TLS-ALPN-01 challenges
300- NextProtos: []string{acme.ALPNProto},
301- },
302- // Add increased timeouts to prevent hanging connections
303- ReadTimeout: 60 * time.Second,
304- WriteTimeout: 60 * time.Second,
305- IdleTimeout: 120 * time.Second,
306- ReadHeaderTimeout: 30 * time.Second, // Add header timeout
154+ // Configure HTTPS and HTTP servers using our refactored method
155+ if err := p.StartHTTPSServer(); err != nil {
156+ logger.Error("Failed to start HTTPS server", "error", err)
157+ return err
307158 }
308159
309- // Store the HTTPS server for later use in Stop()
310- p.httpsServer.Server = httpsServer
311-
312- // Create a custom HTTP server
313- httpServer := &http.Server{
314- Addr: fmt.Sprintf(":%s", p.config.HttpPort),
315- Handler: p.httpServer,
316- ReadTimeout: 30 * time.Second,
317- WriteTimeout: 30 * time.Second,
318- IdleTimeout: 120 * time.Second,
319- }
320-
321- p.serverStarted = true
322-
323- // Start HTTPS server with more error details
324- go func() {
325- logger.Info("Starting HTTPS server", "address", httpsServer.Addr)
326- // Using empty strings for cert and key files since we're using GetCertificate
327- if err := httpsServer.ListenAndServeTLS("", ""); err != nil && err != http.ErrServerClosed {
328- logger.Error("HTTPS server failed", "error", err, "address", httpsServer.Addr)
329-
330- // Print more diagnostic information
331- if strings.Contains(err.Error(), "bind: address already in use") {
332- logger.Error("Port 443 is already in use by another process. Check if another web server is running.",
333- "solution", "Stop other services using port 443 or configure Gordon to use a different HTTPS port")
334- } else if strings.Contains(err.Error(), "permission denied") {
335- logger.Error("Permission denied when binding to port 443. This often happens when running without root privileges.",
336- "solution", "Use setcap to allow binding to privileged ports or run in a container with proper port mapping")
337- } else if strings.Contains(err.Error(), "certificate") {
338- logger.Error("Certificate-related error when starting HTTPS server",
339- "solution", "Check certificate files and permissions in the certificate directory")
340- }
341- }
342- }()
343-
344- // Start HTTP server
345- go func() {
346- logger.Info("Starting HTTP server", "address", httpServer.Addr)
347- if err := httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
348- logger.Error("HTTP server failed", "error", err)
349- }
350- }()
351-
352- // Test admin connection after server starts
353- go p.TestAdminConnectionLater()
354-
355160 return nil
356161 }
357162
@@ -381,12 +186,12 @@ func (p *Proxy) Stop() error {
381186 ctx, cancel := context.WithTimeout(context.Background(), time.Duration(timeout)*time.Second)
382187 defer cancel()
383188
384- // Shutdown HTTP server if it's running
385- if p.httpServer != nil {
386- if err := p.httpServer.Shutdown(ctx); err != nil {
387- logger.Error("Failed to gracefully shutdown HTTP server", "error", err)
388- }
389- }
189+ // Removed HTTP server shutdown logic
190+ // if p.httpServer != nil {
191+ // if err := p.httpServer.Shutdown(ctx); err != nil {
192+ // logger.Error("Failed to gracefully shutdown HTTP server", "error", err)
193+ // }
194+ // }
390195
391196 // Shutdown HTTPS server if it's running
392197 if p.httpsServer != nil {
@@ -536,9 +341,9 @@ func (p *Proxy) checkExternalPortAccess() {
536341 }
537342
538343 // Verify TLS settings are consistent
539- if p.certManager == nil {
344+ if p.certificateManager == nil { // Updated check
540345 logger.Error("Certificate manager is nil - TLS will not work correctly",
541- "solution", "Check setupCertManager implementation")
346+ "solution", "Check NewProxy implementation and logs")
542347 }
543348
544349 // First try a basic TCP connection to port 443 to test if it's open
@@ -654,3 +459,114 @@ func (p *Proxy) checkExternalPortAccess() {
654459 // logger.Debug("Could not auto-detect Gordon container, using default name", "container_name", "gordon")
655460 // return "gordon"
656461 // }
462+
463+// StartHTTPSServer starts the HTTPS server
464+func (p *Proxy) StartHTTPSServer() error {
465+ logger.Info("Starting HTTPS server",
466+ "port", p.config.Port,
467+ "mode", p.config.LetsEncryptMode)
468+
469+ // Verify the certificate manager is initialized (it's done in NewProxy now)
470+ if p.certificateManager == nil {
471+ // This should ideally not happen if NewProxy succeeded
472+ logger.Error("Certificate manager is nil in StartHTTPSServer, cannot start HTTPS")
473+ return fmt.Errorf("certificate manager not initialized")
474+ }
475+ logger.Info("Using new certificate management system")
476+
477+ // Setup TLS configuration
478+ tlsConfig := &tls.Config{
479+ // Use the helper function that gets the GetCertificate func from the manager
480+ GetCertificate: p.getTLSCertificateFunc(),
481+ // Ensure we use TLS 1.2 or higher
482+ MinVersion: tls.VersionTLS12,
483+ // Optimize cipher suites for security and performance
484+ CipherSuites: []uint16{
485+ tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
486+ tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
487+ tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
488+ tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
489+ tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,
490+ tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,
491+ },
492+ // Use server's preferred ciphersuite
493+ PreferServerCipherSuites: true,
494+ // Enable HTTP/2
495+ NextProtos: []string{"h2", "http/1.1"},
496+ // Add server name to use when client doesn't send SNI
497+ ServerName: p.app.GetConfig().Http.FullDomain(),
498+ }
499+
500+ // Removed setup for ACME challenge handler on the persistent HTTP server,
501+ // as it's no longer running. Lego's temporary server will handle challenges.
502+ // p.httpServer.Any("/.well-known/acme-challenge/*", echo.WrapHandler(p.getACMEHTTPHandler()))
503+
504+ // Create a custom HTTPS server with proper TLS config
505+ httpsServer := &http.Server{
506+ Addr: fmt.Sprintf(":%s", p.config.Port),
507+ Handler: p.httpsServer,
508+ TLSConfig: tlsConfig,
509+ // Add increased timeouts to prevent hanging connections
510+ ReadTimeout: 60 * time.Second,
511+ WriteTimeout: 60 * time.Second,
512+ IdleTimeout: 120 * time.Second,
513+ ReadHeaderTimeout: 30 * time.Second, // Add header timeout
514+ }
515+
516+ // Store the HTTPS server for later use in Stop()
517+ p.httpsServer.Server = httpsServer
518+
519+ // // Create a custom HTTP server
520+ // httpServer := &http.Server{
521+ // Addr: fmt.Sprintf(":%s", p.config.HttpPort),
522+ // Handler: p.httpServer,
523+ // ReadTimeout: 30 * time.Second,
524+ // WriteTimeout: 30 * time.Second,
525+ // IdleTimeout: 120 * time.Second,
526+ // }
527+
528+ p.serverStarted = true
529+
530+ // httpServer is no longer needed as ACME HTTP-01 uses its own temporary server
531+ // httpServer := &http.Server{
532+ // Addr: fmt.Sprintf(":%s", p.config.HttpPort),
533+ // Handler: p.httpServer,
534+ // ReadTimeout: 30 * time.Second,
535+ // WriteTimeout: 30 * time.Second,
536+ // IdleTimeout: 120 * time.Second,
537+ // }
538+
539+ // Start HTTPS server with more error details
540+ go func() {
541+ logger.Info("Starting HTTPS server", "address", httpsServer.Addr)
542+ // Using empty strings for cert and key files since we're using GetCertificate
543+ if err := httpsServer.ListenAndServeTLS("", ""); err != nil && err != http.ErrServerClosed {
544+ logger.Error("HTTPS server failed", "error", err, "address", httpsServer.Addr)
545+
546+ // Print more diagnostic information
547+ if strings.Contains(err.Error(), "bind: address already in use") {
548+ logger.Error("Port 443 is already in use by another process. Check if another web server is running.",
549+ "solution", "Stop other services using port 443 or configure Gordon to use a different HTTPS port")
550+ } else if strings.Contains(err.Error(), "permission denied") {
551+ logger.Error("Permission denied when binding to port 443. This often happens when running without root privileges.",
552+ "solution", "Use setcap to allow binding to privileged ports or run in a container with proper port mapping")
553+ } else if strings.Contains(err.Error(), "certificate") {
554+ logger.Error("Certificate-related error when starting HTTPS server",
555+ "solution", "Check certificate files and permissions in the certificate directory")
556+ }
557+ }
558+ }()
559+
560+ // Test admin connection after server starts
561+ go p.TestAdminConnectionLater()
562+
563+ return nil
564+}
565+
566+// getTLSCertificateFunc returns a function that gets certificates for TLS handshakes
567+func (p *Proxy) getTLSCertificateFunc() func(*tls.ClientHelloInfo) (*tls.Certificate, error) {
568+ return func(hello *tls.ClientHelloInfo) (*tls.Certificate, error) {
569+ logger.Debug("Getting certificate for TLS handshake", "server_name", hello.ServerName)
570+ return p.certificateManager.GetCertificateForTLS(hello)
571+ }
572+}
M internal/server/app_server.go +1 -0
@@ -38,6 +38,7 @@ type DBTables struct {
3838 Domain db.Domain `sql:"domain"`
3939 Certificate db.Certificate `sql:"certificate"`
4040 ProxyRoute db.ProxyRoute `sql:"proxy_route"`
41+ AcmeAccount db.AcmeAccount `sql:"acme_account"`
4142 }
4243
4344 // GetConfig returns the configuration
M internal/server/sqlite_init.go +193 -163
@@ -6,10 +6,14 @@ import (
66 "fmt"
77 "os"
88 "path/filepath"
9+ "reflect"
10+ "strings"
911 "time"
1012
1113 "github.com/bnema/gordon/internal/db"
14+ pkgsqlite "github.com/bnema/gordon/pkg/sqlite"
1215 "github.com/charmbracelet/log"
16+ "github.com/google/uuid"
1317 _ "modernc.org/sqlite"
1418 )
1519
@@ -37,113 +41,228 @@ func (a *App) ensureDBDir() error {
3741 return nil
3842 }
3943
40-// checkDBFile checks if the database file exists.
41-func (a *App) checkDBFile(dbPath string) (bool, error) {
42- if _, err := os.Stat(dbPath); os.IsNotExist(err) {
43- log.Debug("Database file does not exist", "path", dbPath)
44- return false, nil
45- } else if err != nil {
46- return false, err
44+// InitializeDB initializes the database in memory and loads data from disk if available
45+func InitializeDB(a *App) (*sql.DB, error) {
46+ log.Debug("Initializing database")
47+ if err := a.ensureDBDir(); err != nil {
48+ return nil, fmt.Errorf("failed to ensure DB directory: %w", err)
4749 }
4850
49- // File exists, but let's also check if the tables exist
50- db, err := sql.Open("sqlite", dbPath)
51- if err != nil {
52- return false, fmt.Errorf("failed to open DB to check tables: %w", err)
51+ diskDBPath := a.getDiskDBFilePath()
52+ a.DBPath = diskDBPath
53+
54+ // --- Start Refactored Schema Handling ---
55+ log.Debug("Ensuring database schema is up-to-date on disk", "path", diskDBPath)
56+ // UpdateDatabase will create the file and tables if they don't exist,
57+ // or update the schema if they do. It also handles backups.
58+ if err := pkgsqlite.UpdateDatabase(diskDBPath, a.DBTables); err != nil {
59+ return nil, fmt.Errorf("failed to create/update database schema on disk: %w", err)
5360 }
54- defer db.Close()
61+ log.Debug("Database schema on disk is up-to-date.")
62+ // --- End Refactored Schema Handling ---
5563
56- // Check if domain table exists
57- var tableCount int
58- err = db.QueryRow("SELECT count(*) FROM sqlite_master WHERE type='table' AND name='domain'").Scan(&tableCount)
64+ // --- BEGIN Seed Admin Account if Necessary ---
65+ log.Debug("Checking if admin account seeding is necessary", "path", diskDBPath)
66+ // Need to temporarily open the disk DB to check and potentially write
67+ seedDB, err := sql.Open("sqlite", diskDBPath)
5968 if err != nil {
60- return false, fmt.Errorf("failed to check for domain table: %w", err)
69+ return nil, fmt.Errorf("failed to open disk DB for seeding check: %w", err)
6170 }
6271
63- if tableCount == 0 {
64- log.Debug("Database file exists but domain table is missing, will reinitialize")
65- // The file exists but doesn't have the required tables, treat as non-existent
66- if err := os.Remove(dbPath); err != nil {
67- return false, fmt.Errorf("failed to remove corrupted DB file: %w", err)
72+ var accountCount int
73+ err = seedDB.QueryRow("SELECT COUNT(*) FROM account").Scan(&accountCount)
74+ if err != nil {
75+ seedDB.Close()
76+ // If the table doesn't exist yet (shouldn't happen after UpdateDatabase), log it
77+ if pkgsqlite.IsNoSuchTableError(err) {
78+ log.Warn("Account table not found during seeding check, skipping seed.", "error", err)
79+ // Proceed without seeding, assuming UpdateDatabase handled creation correctly
80+ } else {
81+ return nil, fmt.Errorf("failed to query account count from disk DB: %w", err)
82+ }
83+ } else if accountCount == 0 {
84+ log.Info("No accounts found, seeding default admin user and account.")
85+ // Begin transaction for seeding
86+ seedTx, txErr := seedDB.Begin()
87+ if txErr != nil {
88+ seedDB.Close()
89+ return nil, fmt.Errorf("failed to begin transaction for seeding: %w", txErr)
6890 }
69- return false, nil
70- }
7191
72- log.Debug("Database file exists and contains required tables", "path", dbPath)
73- return true, nil
74-}
92+ adminUserID := uuid.NewString()
93+ adminAccountID := uuid.NewString()
94+ adminEmail := "admin@local.host" // Or some other placeholder
95+
96+ // Insert User
97+ _, txErr = seedTx.Exec("INSERT INTO user (id, name, email) VALUES (?, ?, ?)",
98+ adminUserID, "Admin User", adminEmail)
99+ if txErr != nil {
100+ seedTx.Rollback()
101+ seedDB.Close()
102+ return nil, fmt.Errorf("failed to insert default admin user: %w", txErr)
103+ }
75104
76-// InitializeDB initializes the database in memory and loads data from disk if available
77-func InitializeDB(a *App) (*sql.DB, error) {
78- log.Debug("Initializing in-memory database")
79- if err := a.ensureDBDir(); err != nil {
80- return nil, fmt.Errorf("failed to ensure DB directory: %w", err)
105+ // Insert Account
106+ _, txErr = seedTx.Exec("INSERT INTO account (id, user_id) VALUES (?, ?)",
107+ adminAccountID, adminUserID)
108+ if txErr != nil {
109+ seedTx.Rollback()
110+ seedDB.Close()
111+ return nil, fmt.Errorf("failed to insert default admin account: %w", txErr)
112+ }
113+
114+ // Commit transaction
115+ txErr = seedTx.Commit()
116+ if txErr != nil {
117+ seedDB.Close()
118+ return nil, fmt.Errorf("failed to commit seeding transaction: %w", txErr)
119+ }
120+ log.Info("Default admin user and account seeded successfully", "user_id", adminUserID, "account_id", adminAccountID)
121+ } else {
122+ log.Debug("Accounts already exist, skipping admin seeding.", "count", accountCount)
81123 }
82124
83- diskDBPath := a.getDiskDBFilePath()
84- a.DBPath = diskDBPath
125+ // Close the temporary connection used for seeding
126+ if err := seedDB.Close(); err != nil {
127+ // Log error but don't necessarily fail initialization at this point
128+ log.Error("Error closing temporary seed DB connection", "error", err)
129+ }
130+ // --- END Seed Admin Account if Necessary ---
85131
86- // Create in-memory database connection
132+ // Create in-memory database connection (remains the same)
87133 log.Debug("Opening in-memory SQLite database")
88134 memDB, err := sql.Open("sqlite", "file::memory:?cache=shared")
89135 if err != nil {
90136 return nil, fmt.Errorf("failed to open in-memory DB: %w", err)
91137 }
92138
93- // Test that the in-memory database is working
139+ // Test that the in-memory database is working (remains the same)
94140 log.Debug("Testing in-memory database connection")
95141 if err := memDB.Ping(); err != nil {
142+ memDB.Close() // Close memDB if ping fails
96143 return nil, fmt.Errorf("failed to ping in-memory database: %w", err)
97144 }
98145
99- // Check if disk database exists
100- dbExists, err := a.checkDBFile(diskDBPath)
146+ // --- Load from Disk to Memory ---
147+ // Now that the disk schema is guaranteed to be correct, load it into memory.
148+ // We need to check if the file actually contains data before loading.
149+ // A simple way is to check if the file size is > 0 after UpdateDatabase.
150+ fileInfo, err := os.Stat(diskDBPath)
151+ loadNeeded := true
101152 if err != nil {
102- return nil, fmt.Errorf("failed to check DB file: %w", err)
153+ if os.IsNotExist(err) {
154+ log.Debug("Database file still doesn't exist after schema update (likely first run with no tables?), skipping load from disk.")
155+ loadNeeded = false // Nothing to load
156+ } else {
157+ memDB.Close()
158+ return nil, fmt.Errorf("failed to stat disk DB file after schema update: %w", err)
159+ }
160+ } else if fileInfo.Size() == 0 {
161+ log.Debug("Database file is empty after schema update, skipping load from disk.")
162+ loadNeeded = false // Nothing to load
103163 }
104164
105- if !dbExists {
106- log.Debug("Database needs bootstrapping", "path", diskDBPath)
107- // Create tables in memory
108- log.Debug("Creating tables in memory")
109- if err := bootstrapDB(memDB); err != nil {
110- return nil, fmt.Errorf("failed to bootstrap in-memory DB: %w", err)
165+ if loadNeeded {
166+ log.Debug("Loading database from disk into memory", "path", diskDBPath)
167+ if err := loadDiskDBToMemory(diskDBPath, memDB); err != nil {
168+ memDB.Close()
169+ return nil, fmt.Errorf("failed to load disk DB into memory: %w", err)
170+ }
171+ log.Debug("Successfully loaded disk database into memory")
172+ } else {
173+ // If we didn't load, the memory DB is empty but has the correct schema
174+ // implicitly because UpdateDatabase already ran on the (possibly new) disk file.
175+ // However, loadDiskDBToMemory *also* creates the schema in memory based on the disk file.
176+ // If the disk file was *just* created by UpdateDatabase and is empty,
177+ // loadDiskDBToMemory won't run, and memDB won't have the tables yet.
178+ // We need to ensure the schema exists in memory even if the disk file was initially empty.
179+ // Easiest way: re-apply the schema creation logic to memDB.
180+ log.Debug("Applying schema to empty in-memory database")
181+ memTx, err := memDB.Begin()
182+ if err != nil {
183+ memDB.Close()
184+ return nil, fmt.Errorf("failed to begin transaction on memory DB for schema creation: %w", err)
185+ }
186+ schemaCreated := false
187+ // Iterate through DBTables fields using reflection to get correct table names
188+ dbTablesType := reflect.TypeOf(a.DBTables)
189+ dbTablesValue := reflect.ValueOf(a.DBTables)
190+ if dbTablesType.Kind() == reflect.Ptr {
191+ dbTablesType = dbTablesType.Elem()
192+ dbTablesValue = dbTablesValue.Elem()
111193 }
112194
113- // Save the initialized in-memory database to disk
114- log.Debug("Saving initialized in-memory database to disk")
115- if err := saveMemDBToDisk(memDB, diskDBPath); err != nil {
116- return nil, fmt.Errorf("failed to save initial in-memory DB to disk: %w", err)
195+ if dbTablesType.Kind() == reflect.Struct {
196+ log.Debug("Reflecting DBTables struct for in-memory schema creation", "type", dbTablesType.Name())
197+ for i := 0; i < dbTablesType.NumField(); i++ {
198+ field := dbTablesType.Field(i)
199+ fieldValue := dbTablesValue.Field(i).Interface() // e.g., db.ProxyRoute instance
200+ sqlTag := field.Tag.Get("sql")
201+
202+ if sqlTag == "" || sqlTag == "-" {
203+ log.Debug("Skipping field without sql tag in memory schema creation", "field", field.Name)
204+ continue // Skip fields without a sql tag or marked to be ignored
205+ }
206+
207+ tableName := sqlTag // Use the tag directly as the table name
208+ log.Debug("Attempting to create table in memory", "tableName", tableName, "structType", reflect.TypeOf(fieldValue).Name())
209+
210+ if err := pkgsqlite.CreateTableInTx(memTx, fieldValue, tableName); err != nil {
211+ // Ignore "table already exists" errors if somehow they occur
212+ if !strings.Contains(strings.ToLower(err.Error()), "already exists") {
213+ log.Error("Failed to create table in memory", "table", tableName, "error", err)
214+ memTx.Rollback()
215+ memDB.Close()
216+ return nil, fmt.Errorf("failed to create table %s in memory: %w", tableName, err)
217+ } else {
218+ log.Debug("Table already exists in memory, skipping creation", "table", tableName)
219+ }
220+ } else {
221+ log.Debug("Successfully created table in memory", "table", tableName)
222+ schemaCreated = true
223+ }
224+ }
225+ } else {
226+ log.Error("a.DBTables is not a struct, cannot create schema in memory")
227+ memDB.Close()
228+ return nil, fmt.Errorf("internal error: DBTables is not a struct type (%T)", a.DBTables)
117229 }
118230
119- log.Debug("Initialized new in-memory database and saved to disk")
120- } else {
121- // Load existing database from disk into memory
122- log.Debug("Loading existing database from disk into memory", "path", diskDBPath)
123- if err := loadDiskDBToMemory(diskDBPath, memDB); err != nil {
124- return nil, fmt.Errorf("failed to load disk DB into memory: %w", err)
231+ if err := memTx.Commit(); err != nil {
232+ memDB.Close()
233+ return nil, fmt.Errorf("failed to commit schema creation transaction to memory DB: %w", err)
234+ }
235+ if schemaCreated {
236+ log.Debug("Successfully applied schema to in-memory database")
237+ } else {
238+ log.Debug("In-memory schema already existed or no tables defined.")
125239 }
126240
127- log.Debug("Successfully loaded disk database into memory")
128241 }
242+ // --- End Load from Disk to Memory ---
129243
130244 a.DB = memDB
131245
132- if dbExists {
133- // Populate the struct tables from the database
134- log.Debug("Populating struct tables from existing database")
246+ // Populate struct tables if data was loaded
247+ if loadNeeded {
248+ log.Debug("Populating struct tables from loaded database")
135249 if err := PopulateStructWithTables(a); err != nil {
136- return nil, err
250+ // Don't fail initialization for this, maybe just log an error
251+ log.Error("Failed to populate struct tables after loading DB", "error", err)
252+ // Depending on severity, you might want to return nil, err here
137253 }
254+ } else {
255+ log.Debug("Skipping struct table population as no data was loaded from disk.")
138256 }
139257
140- // Create a context for the auto-save routine that will be canceled when the app is shutting down
258+ // Create context for auto-save (remains the same)
141259 a.DBSaveCtx, a.DBSaveCancel = context.WithCancel(context.Background())
142260
143- // Start auto-save routine
144- log.Debug("Starting auto-save routine with interval of", "minutes", AutoSaveInterval)
261+ // Start auto-save routine (remains the same)
262+ log.Debug("Starting auto-save routine with interval", "minutes", AutoSaveInterval)
145263 go autoSaveDB(a.DBSaveCtx, a, AutoSaveInterval)
146264
265+ log.Info("Database initialization completed successfully")
147266 return memDB, nil
148267 }
149268
@@ -446,105 +565,6 @@ func joinStrings(strings []string, separator string) string {
446565 return result
447566 }
448567
449-// bootstrapDB creates the initial database schema
450-func bootstrapDB(db *sql.DB) error {
451- log.Debug("Bootstrapping database with initial schema")
452-
453- // Begin a transaction
454- tx, err := db.Begin()
455- if err != nil {
456- return fmt.Errorf("error beginning transaction: %w", err)
457- }
458-
459- // Create tables
460- log.Debug("Creating database tables")
461- _, err = tx.Exec(`
462- CREATE TABLE IF NOT EXISTS user (
463- id TEXT PRIMARY KEY,
464- name TEXT,
465- email TEXT
466- );
467- CREATE TABLE IF NOT EXISTS account (
468- id TEXT PRIMARY KEY,
469- user_id TEXT,
470- FOREIGN KEY (user_id) REFERENCES user(id)
471- );
472- CREATE TABLE IF NOT EXISTS sessions (
473- id TEXT PRIMARY KEY,
474- account_id TEXT,
475- browser_info TEXT,
476- access_token TEXT,
477- expires TEXT,
478- is_online INTEGER,
479- FOREIGN KEY (account_id) REFERENCES account(id)
480- );
481- CREATE TABLE IF NOT EXISTS provider (
482- id TEXT PRIMARY KEY,
483- account_id TEXT,
484- name TEXT,
485- login TEXT,
486- avatar_url TEXT,
487- profile_url TEXT,
488- email TEXT,
489- FOREIGN KEY (account_id) REFERENCES account(id)
490- );
491- CREATE TABLE IF NOT EXISTS clients (
492- id TEXT PRIMARY KEY,
493- account_id TEXT,
494- os TEXT,
495- ip TEXT,
496- hostname TEXT,
497- expires TEXT,
498- FOREIGN KEY (account_id) REFERENCES account(id)
499- );
500- CREATE TABLE IF NOT EXISTS domain (
501- id TEXT PRIMARY KEY,
502- name TEXT,
503- account_id TEXT,
504- created_at TEXT,
505- updated_at TEXT,
506- FOREIGN KEY (account_id) REFERENCES account(id)
507- );
508- CREATE TABLE IF NOT EXISTS certificate (
509- id TEXT PRIMARY KEY,
510- domain_id TEXT,
511- cert_file TEXT,
512- key_file TEXT,
513- issued_at TEXT,
514- expires_at TEXT,
515- issuer TEXT,
516- status TEXT,
517- FOREIGN KEY (domain_id) REFERENCES domain(id)
518- );
519- CREATE TABLE IF NOT EXISTS proxy_route (
520- id TEXT PRIMARY KEY,
521- domain_id TEXT,
522- container_id TEXT,
523- container_ip TEXT,
524- container_port TEXT,
525- protocol TEXT,
526- path TEXT,
527- active INTEGER,
528- created_at TEXT,
529- updated_at TEXT,
530- FOREIGN KEY (domain_id) REFERENCES domain(id)
531- );
532- `)
533- if err != nil {
534- tx.Rollback()
535- return fmt.Errorf("error creating tables: %w", err)
536- }
537-
538- // Commit the transaction
539- log.Debug("Committing transaction with initial schema")
540- if err := tx.Commit(); err != nil {
541- return fmt.Errorf("error committing transaction: %w", err)
542- }
543-
544- log.Debug("Database bootstrap completed successfully")
545- return nil
546-}
547-
548568 type DBTable interface {
549569 ScanFromRows(rows *sql.Rows) error
550570 }
@@ -618,7 +638,17 @@ func (c *ClientsWrapper) ScanFromRows(rows *sql.Rows) error {
618638 }
619639
620640 func (d *DomainWrapper) ScanFromRows(rows *sql.Rows) error {
621- return rows.Scan(&d.ID, &d.Name, &d.AccountID, &d.CreatedAt, &d.UpdatedAt)
641+ return rows.Scan(
642+ &d.ID,
643+ &d.Name,
644+ &d.AccountID,
645+ &d.CreatedAt,
646+ &d.UpdatedAt,
647+ &d.AcmeEnabled,
648+ &d.AcmeChallengeType,
649+ &d.AcmeDnsProvider,
650+ &d.AcmeDnsCredentialsRef,
651+ )
622652 }
623653
624654 func (c *CertificateWrapper) ScanFromRows(rows *sql.Rows) error {
@@ -663,7 +693,7 @@ func PopulateStructWithTables(a *App) error {
663693
664694 // Populate the Domain table
665695 domainWrapper := &DomainWrapper{Domain: &a.DBTables.Domain}
666- if err := populateTableFromDB(a, "SELECT id, name, account_id, created_at, updated_at FROM domain", domainWrapper); err != nil {
696+ if err := populateTableFromDB(a, "SELECT id, name, account_id, created_at, updated_at, acme_enabled, acme_challenge_type, acme_dns_provider, acme_dns_credentials_ref FROM domain", domainWrapper); err != nil {
667697 return err
668698 }
669699
M internal/server/token_init.go +1 -5
@@ -25,11 +25,7 @@ func HandleNewTokenInitialization(a *App) (string, error) {
2525 token, err := generateRandomToken(16) // 16 bytes
2626 if err != nil {
2727 return "", fmt.Errorf("failed to generate token: %v", err)
28- }
29-
30- // url = config.Http.SubDomain + config.Http.TopDomain + config.Http.Port + config.Admin.Path
31-
32- fmt.Printf("Login with the new token: %s\n", token)
28+
3329
3430 // Store the token in the config file
3531 a.Config.General.Token = token
M pkg/sqlite/db_create.go +3 -3
@@ -7,8 +7,8 @@ import (
77 "strings"
88 )
99
10-// CreateTable creates a table in the database based on the given struct.
11-func CreateTable(db *sql.DB, table interface{}, tableName string) error {
10+// CreateTableInTx creates a table within a transaction based on the given struct.
11+func CreateTableInTx(tx *sql.Tx, table interface{}, tableName string) error {
1212 t := reflect.TypeOf(table)
1313
1414 var fields []string
@@ -53,7 +53,7 @@ func CreateTable(db *sql.DB, table interface{}, tableName string) error {
5353 }
5454 createTableSQL += ");"
5555
56- _, err := db.Exec(createTableSQL)
56+ _, err := tx.Exec(createTableSQL)
5757 if err != nil {
5858 return fmt.Errorf("failed to create table %s: %v", tableName, err)
5959 }
A pkg/sqlite/db_update.go +328 -0
@@ -0,0 +1,328 @@
1+package pkgsqlite
2+
3+import (
4+ "database/sql"
5+ "fmt"
6+ "os"
7+ "path/filepath"
8+ "reflect"
9+ "strings"
10+ "time"
11+
12+ "github.com/charmbracelet/log"
13+)
14+
15+// UpdateDatabase compares struct definitions with the actual database schema on disk and updates it accordingly
16+// Accepts the DBTables struct (or similar) containing the table definitions as fields with `sql` tags.
17+func UpdateDatabase(dbPath string, dbTablesStruct interface{}) error {
18+ log.Debug("Starting database schema update check on disk", "path", dbPath)
19+
20+ // Check if the database file exists BEFORE attempting backup
21+ _, statErr := os.Stat(dbPath)
22+ if statErr == nil {
23+ // File exists, proceed with backup
24+ log.Debug("Database file exists, attempting backup.", "path", dbPath)
25+ if err := backupDatabase(dbPath); err != nil {
26+ // If backup fails on an existing file, that's a real problem
27+ return fmt.Errorf("failed to backup existing database: %w", err)
28+ }
29+ } else if os.IsNotExist(statErr) {
30+ // File does not exist, skip backup
31+ log.Debug("Database file doesn't exist, skipping backup.", "path", dbPath)
32+ } else {
33+ // Other error during stat (e.g., permissions), return it
34+ return fmt.Errorf("failed to check database file status before backup: %w", statErr)
35+ }
36+
37+ // Open connection to the disk database
38+ // sql.Open will create the file if it doesn't exist when a connection is made/queried
39+ diskDB, err := sql.Open("sqlite", dbPath)
40+ if err != nil {
41+ return fmt.Errorf("failed to open disk DB for schema update: %w", err)
42+ }
43+ defer diskDB.Close()
44+
45+ // Optional: Ping to ensure the connection is valid and file is created if needed
46+ if err := diskDB.Ping(); err != nil {
47+ // It's possible ping fails if the directory doesn't exist yet, ensure dir first.
48+ // Let's rely on the transaction to create the file implicitly if needed.
49+ // Commenting out ping as it might add complexity here.
50+ // return fmt.Errorf("failed to ping disk DB after opening: %w", err)
51+ }
52+
53+ // Begin transaction on disk DB
54+ tx, err := diskDB.Begin()
55+ if err != nil {
56+ return fmt.Errorf("failed to begin transaction on disk DB: %w", err)
57+ }
58+ // Using defer with a named return error allows checking commit error
59+ var txErr error
60+ defer func() {
61+ if r := recover(); r != nil {
62+ _ = tx.Rollback() // Attempt rollback on panic
63+ // Re-panic or handle as needed
64+ panic(r)
65+ } else if txErr != nil {
66+ log.Debug("Rolling back transaction due to error", "error", txErr)
67+ if rbErr := tx.Rollback(); rbErr != nil {
68+ log.Error("Failed to rollback transaction", "rollback_error", rbErr)
69+ }
70+ }
71+ }()
72+
73+ // Update each table schema using reflection on the provided struct
74+ updated := false
75+ dbTablesType := reflect.TypeOf(dbTablesStruct)
76+ dbTablesValue := reflect.ValueOf(dbTablesStruct)
77+ if dbTablesType.Kind() == reflect.Ptr {
78+ dbTablesType = dbTablesType.Elem()
79+ dbTablesValue = dbTablesValue.Elem()
80+ }
81+
82+ if dbTablesType.Kind() != reflect.Struct {
83+ _ = tx.Rollback() // Attempt rollback before returning error
84+ return fmt.Errorf("UpdateDatabase expects a struct, got %T", dbTablesStruct)
85+ }
86+ log.Debug("Reflecting DBTables struct for disk schema update", "type", dbTablesType.Name(), "numFields", dbTablesType.NumField())
87+
88+ for i := 0; i < dbTablesType.NumField(); i++ {
89+ field := dbTablesType.Field(i)
90+ fieldValue := dbTablesValue.Field(i).Interface() // e.g., db.ProxyRoute instance
91+ sqlTag := field.Tag.Get("sql")
92+
93+ if sqlTag == "" || sqlTag == "-" {
94+ log.Debug("Skipping field without sql tag in schema update", "field", field.Name)
95+ continue // Skip fields without a sql tag or marked to be ignored
96+ }
97+
98+ tableName := sqlTag // Use the tag directly as the table name
99+ log.Debug("Processing field for schema update", "field", field.Name, "tableName", tableName, "structType", reflect.TypeOf(fieldValue).Name())
100+
101+ var tableUpdated bool
102+ // Pass the correct tableName from the tag and the actual struct instance (fieldValue)
103+ tableUpdated, txErr = updateTableSchema(tx, tableName, fieldValue)
104+ if txErr != nil {
105+ // updateTableSchema handles creation if table is missing
106+ // Return the error wrapped, defer will handle rollback
107+ return fmt.Errorf("failed to update/create table '%s' (from field '%s'): %w", tableName, field.Name, txErr)
108+ }
109+ if tableUpdated {
110+ updated = true
111+ }
112+ }
113+
114+ // Commit transaction - Assign error to txErr for defer check
115+ txErr = tx.Commit()
116+ if txErr != nil {
117+ return fmt.Errorf("failed to commit transaction on disk DB: %w", txErr)
118+ }
119+
120+ if updated {
121+ log.Info("Successfully updated/created database schema on disk")
122+ } else {
123+ log.Debug("Database schema on disk is up-to-date")
124+ }
125+ return nil
126+}
127+
128+// updateTableSchema updates a single table's schema based on struct definition
129+// Returns true if the schema was updated or created, false otherwise.
130+func updateTableSchema(tx *sql.Tx, tableName string, structType interface{}) (bool, error) {
131+ log.Debug("Starting schema update check for table", "table", tableName)
132+ updated := false
133+ // Get existing columns
134+ existingColumns, err := getTableColumns(tx, tableName)
135+ if err != nil {
136+ // If the error is "no such table", try creating it
137+ if IsNoSuchTableError(err) {
138+ log.Info("Table does not exist (getTableColumns returned error), attempting to create", "table", tableName)
139+ if createErr := CreateTableInTx(tx, structType, tableName); createErr != nil {
140+ return false, fmt.Errorf("failed to create missing table %s: %w", tableName, createErr)
141+ }
142+ log.Info("Successfully created table", "table", tableName)
143+ return true, nil // Table was created (updated=true)
144+ }
145+ // Otherwise, it's a different error when getting columns
146+ return false, fmt.Errorf("failed to get existing columns for table %s: %w", tableName, err)
147+ }
148+
149+ // Check if getTableColumns succeeded but found no columns.
150+ // Treat this as if the table needs creation.
151+ if len(existingColumns) == 0 {
152+ log.Info("Table exists according to PRAGMA but has no columns, treating as missing/invalid, attempting to create", "table", tableName)
153+ // Attempt to drop the potentially problematic empty/invalid table first (optional but safer)
154+ // We can ignore errors here, as it might not actually exist for DROP.
155+ _, _ = tx.Exec(fmt.Sprintf("DROP TABLE IF EXISTS %s", tableName))
156+
157+ // Now create it properly
158+ if createErr := CreateTableInTx(tx, structType, tableName); createErr != nil {
159+ return false, fmt.Errorf("failed to create missing/invalid table %s: %w", tableName, createErr)
160+ }
161+ log.Info("Successfully recreated table", "table", tableName)
162+ return true, nil // Table was created/recreated (updated=true)
163+ }
164+
165+ log.Debug("Existing columns found", "table", tableName, "columns", existingColumns)
166+
167+ // Get struct fields
168+ structFields := getStructFields(structType)
169+ log.Debug("Struct fields found", "table", tableName, "fields", structFields)
170+
171+ // Compare and add missing columns
172+ for fieldName, fieldType := range structFields {
173+ if _, exists := existingColumns[fieldName]; !exists {
174+ sqlType := convertGoTypeToSQLType(fieldType) // Use the helper function
175+ query := fmt.Sprintf("ALTER TABLE %s ADD COLUMN %s %s", tableName, fieldName, sqlType)
176+
177+ log.Debug("Attempting to add missing column", "table", tableName, "column", fieldName, "type", sqlType, "query", query)
178+ if _, err := tx.Exec(query); err != nil {
179+ if isDuplicateColumnError(err) {
180+ log.Debug("Column already exists (detected via duplicate error), skipping", "table", tableName, "column", fieldName)
181+ } else {
182+ log.Error("Failed to execute ALTER TABLE", "table", tableName, "column", fieldName, "query", query, "error", err)
183+ return false, fmt.Errorf("failed to add column %s to table %s: %w", fieldName, tableName, err)
184+ }
185+ } else {
186+ log.Info("Successfully added column", "table", tableName, "column", fieldName)
187+ updated = true // Column was successfully added
188+ }
189+ }
190+ }
191+
192+ log.Debug("Finished schema update check for table", "table", tableName, "schema_was_modified", updated)
193+ return updated, nil
194+}
195+
196+// getTableColumns returns a map of existing column names in the table
197+func getTableColumns(tx *sql.Tx, tableName string) (map[string]bool, error) {
198+ columns := make(map[string]bool)
199+
200+ query := fmt.Sprintf("PRAGMA table_info(%s)", tableName)
201+ rows, err := tx.Query(query)
202+ if err != nil {
203+ // Specifically check if the error indicates "no such table" from the PRAGMA itself
204+ if IsNoSuchTableError(err) {
205+ // Return the specific error type so the caller can handle it
206+ return nil, err
207+ }
208+ // Otherwise, wrap the generic error
209+ return nil, fmt.Errorf("failed to query table info for %s: %w", tableName, err)
210+ }
211+ defer rows.Close()
212+
213+ scanFailed := false // Flag to track scan errors
214+ for rows.Next() {
215+ var (
216+ cid int
217+ name string
218+ colType string
219+ notnull int
220+ dflt sql.NullString
221+ pk int
222+ )
223+ if err := rows.Scan(&cid, &name, &colType, &notnull, &dflt, &pk); err != nil {
224+ log.Error("Failed to scan column info row", "table", tableName, "error", err)
225+ scanFailed = true // Set flag on scan error
226+ break // Stop processing rows on error
227+ }
228+ columns[name] = true
229+ }
230+
231+ // Check for errors after the loop (including scan errors)
232+ if err := rows.Err(); err != nil {
233+ // Prioritize returning the specific scan error if it happened
234+ if scanFailed {
235+ // We already logged the scan error, return a generic iteration error
236+ return nil, fmt.Errorf("error scanning table info for %s: %w", tableName, err)
237+ }
238+ // Otherwise, return the iteration error from rows.Err()
239+ return nil, fmt.Errorf("error iterating table info for %s: %w", tableName, err)
240+ }
241+
242+ // If scan failed, even if rows.Err() is nil, return an error
243+ if scanFailed {
244+ return nil, fmt.Errorf("failed to scan one or more column info rows for table %s", tableName)
245+ }
246+
247+ // If we get here, PRAGMA succeeded and iteration/scanning completed without error.
248+ // It's safe to return the map (which might be empty) and nil error.
249+ return columns, nil
250+}
251+
252+// getStructFields returns a map of field names to their types from a struct
253+func getStructFields(structType interface{}) map[string]string {
254+ fields := make(map[string]string)
255+ t := reflect.TypeOf(structType)
256+
257+ // If it's a pointer, get the element type
258+ if t.Kind() == reflect.Ptr {
259+ t = t.Elem()
260+ }
261+
262+ // Ensure it's actually a struct before proceeding
263+ if t.Kind() != reflect.Struct {
264+ log.Warn("getStructFields called with non-struct type", "type", t.Name())
265+ return fields // Return empty map
266+ }
267+
268+ for i := 0; i < t.NumField(); i++ {
269+ field := t.Field(i)
270+ sqlTag := field.Tag.Get("sql")
271+ if sqlTag == "" {
272+ continue
273+ }
274+
275+ // Get the SQL column name from the tag
276+ parts := strings.Split(sqlTag, ",")
277+ sqlName := parts[0]
278+
279+ // Store the field type
280+ fields[sqlName] = field.Type.Name()
281+ }
282+
283+ return fields
284+}
285+
286+// backupDatabase creates a backup of the database file
287+// Assumes the dbPath already exists when this is called.
288+func backupDatabase(dbPath string) error {
289+ // Create backup directory if it doesn't exist
290+ backupDir := filepath.Join(filepath.Dir(dbPath), "backups")
291+ if err := os.MkdirAll(backupDir, 0755); err != nil {
292+ return fmt.Errorf("failed to create backup directory: %w", err)
293+ }
294+
295+ // Generate backup filename with timestamp
296+ timestamp := time.Now().Format("20060102_150405")
297+ backupFile := filepath.Join(backupDir, fmt.Sprintf("%s_%s.db", filepath.Base(dbPath), timestamp))
298+
299+ // Read the database file
300+ data, err := os.ReadFile(dbPath)
301+ if err != nil {
302+ // This shouldn't happen now based on the check in UpdateDatabase, but check again.
303+ if os.IsNotExist(err) {
304+ log.Warn("backupDatabase called but file vanished", "path", dbPath)
305+ return nil // File disappeared between check and read? Treat as success.
306+ }
307+ return fmt.Errorf("failed to read database file for backup: %w", err)
308+ }
309+
310+ // Write to backup file
311+ if err := os.WriteFile(backupFile, data, 0644); err != nil {
312+ return fmt.Errorf("failed to write backup file: %w", err)
313+ }
314+
315+ log.Info("Created database backup", "backup_file", backupFile)
316+ return nil
317+}
318+
319+// isDuplicateColumnError checks if the error is due to a column already existing (SQLite specific)
320+func isDuplicateColumnError(err error) bool {
321+ return err != nil && strings.Contains(strings.ToLower(err.Error()), "duplicate column name")
322+}
323+
324+// IsNoSuchTableError checks if the error is due to a table not existing (Exported: changed i to I)
325+func IsNoSuchTableError(err error) bool {
326+ // Check for the specific SQLite error message
327+ return err != nil && strings.Contains(strings.ToLower(err.Error()), "no such table")
328+}

Keyboard shortcuts

Single-key shortcuts are on. ctrl k and esc work whatever this says.

ctrl k :
go to a section, branch, tag, file or repository
?
this list
j k
move the row cursor down / up
enter
open the row under the cursor; on Files, a directory opens or closes in place
↑ ↓ ← → home end
in the file tree (focus on a row): previous / next row, close / open a directory or go to its parent / first entry, first / last row. They never open a file
l h
on Files: open a directory or a file / close a directory or go up to its parent, then to the parent page. Elsewhere: into the side panel (on a merge request, its files) / back to the content; there j k move between its controls
g g G
first / last row
/
filter the list; on Files, find a file anywhere in the repository (esc clears, enter opens the best match)
b
switch branch or tag
g s g f g c g b g m g i g r g a
tabs: summary, files, commits, branches & tags, merge requests, issues, releases, actions (when available)
g h
home
w
wide mode: use the whole width of the screen, or not (remembered)
y
copy a permalink pinned to the commit
d u x
diff: split / unified / collapse or expand all
e E
diff: the whole file in view / every file, with the lines the hunks hide
n
merge request: next note in the diff
esc
close a dialog, leave a field

Branches 7

Tags 106

↑↓ moveenter openesc close