1# Blob storage
2
3Release assets and CI artifact contents share one blob store. Metadata stays in
4`$ERMITE_DATA_DIR/kv`; Git repositories and CI logs stay on local disk.
5
6With no `ERMITE_S3_BUCKET`, objects live under
7`$ERMITE_DATA_DIR/blobs/objects/`. Directories are private (0700), files 0600.
8Back up the entire data directory while the server is stopped. With S3, back up
9both the data directory and the bucket prefix consistently.
10
11## S3-compatible storage
12
13Create a bucket yourself, then configure ermite. All examples use explicit keys:
14
15```sh
16export ERMITE_S3_BUCKET=forge-assets
17export ERMITE_S3_PREFIX=ermite/blobs
18export ERMITE_S3_ACCESS_KEY_ID_FILE=/run/secrets/s3-id
19export ERMITE_S3_SECRET_ACCESS_KEY_FILE=/run/secrets/s3-secret
20```
21
22Select the endpoint and signing region for your provider:
23
24| Service | `ERMITE_S3_ENDPOINT` | `ERMITE_S3_REGION` | `ERMITE_S3_FORCE_PATH_STYLE` |
25|---|---|---|---|
26| AWS S3 | unset (SDK resolves from region) | bucket region, e.g. `eu-west-1` | `0` |
27| Scaleway | `https://s3.fr-par.scw.cloud` | `fr-par` | `0` |
28| Cloudflare R2 | `https://<account-id>.r2.cloudflarestorage.com` | `auto` | `0` |
29| MinIO (compose network) | `http://minio:9000` | `us-east-1` | `1` |
30
31Any S3-compatible endpoint implementing the subset below can be used. There is
32no ambient AWS credential chain: `AWS_*`, shared credential files and instance
33roles are not consulted. Region and both keys are required with a bucket.
34Each key supports `_FILE`; direct value and file are mutually exclusive.
35
36On startup, ermite probes the store (disk write probe; S3 `HeadBucket`, falling
37back to `ListObjectsV2` with one key on 403). Failure stops startup and names
38`ERMITE_S3_BUCKET`. Success logs `blob store ready`, never credential values.
39The S3 probe checks reachability/read permission, not every write permission.
40
41Grant access to objects under the configured prefix: put, get/head, delete,
42list, and create/upload/complete/abort/list multipart uploads. On AWS these are
43`s3:PutObject`, `s3:GetObject`, `s3:DeleteObject`,
44`s3:AbortMultipartUpload`, `s3:ListMultipartUploadParts` on the object resource,
45and `s3:ListBucket`, `s3:ListBucketMultipartUploads` on the bucket resource.
46Constrain listing to the prefix where supported. ermite never creates buckets or
47calls bucket administration APIs (policy, ACL, tagging, lifecycle, versioning).
48The client uses checksums only when required, not automatic CRC32 on every PUT.
49
50Downloads are relayed through ermite with repository authorization and Range
51support: no redirects or presigned URLs. Multipart uploads use 8 MiB parts.
52The daily sweep removes unreferenced blobs and unfinished uploads older than
5324 hours. Release metadata with a missing blob is removed only after two sweeps.
54MinIO lists unfinished uploads only for an exact object key, so the sweep does
55not see them there; MinIO removes them itself (`stale_uploads_expiry`, 24 hours
56by default).
57
58## Backend changes
59
60Legacy artifacts under `$ERMITE_DATA_DIR/ci/artifacts` remain readable until
61expiry; there is no automatic copy. New artifacts and all release assets use the
62currently selected backend. **Do not switch backends with live blob-backed data**:
63records identify blobs, not their backend. Switching disk to S3, changing bucket
64or prefix, or switching back makes earlier blob-backed contents unavailable.
65Keep the selected backend stable, or arrange a complete offline copy yourself.
66
67## Development compose
68
69Disk (no MinIO services):
70
71```sh
72ERMITE_S3_BUCKET= docker compose up
73```
74
75MinIO provisioning must finish before the forge starts its probe:
76
77```sh
78docker compose --profile s3 up -d minio
79docker compose --profile s3 run --rm minio-init
80ERMITE_S3_BUCKET=ermite ERMITE_S3_ENDPOINT=http://minio:9000 \
81ERMITE_S3_REGION=us-east-1 ERMITE_S3_ACCESS_KEY_ID=minioadmin \
82ERMITE_S3_SECRET_ACCESS_KEY=minioadmin ERMITE_S3_FORCE_PATH_STYLE=1 \
83ERMITE_S3_PREFIX=dev/ docker compose --profile s3 up ermite
84```
85
86Console: <http://localhost:9001>, development credentials `minioadmin` /
87`minioadmin`. The init service creates `ermite` and `ermite-test`; only that
88provisioning service has bucket creation duties. For Scaleway or another external
89service, use the same environment variables with its existing bucket, endpoint,
90region and keys, and `docker compose up ermite` without the profile. These are
91development settings; never use the placeholder admin token or MinIO keys in
92production.
93
94## Issue history (schema 8)
95
96Issue events are protobuf `IssueEvent` records under
97`isev/<repo>/<issue:010d>/<seq:010d>`. Sequences are dense, per-issue and
981-based. Events naming a merge request are copied verbatim to
99`mrev/<repo>/<mr:010d>/<issue:010d>/<seq:010d>` in the same atomic batch as
100the issue update or link diff. Issue history is read in sequence order; MR
101history is ordered by timestamp, then issue and sequence (at most 500 rows).
102Each issue retains at most 500 events; further events are silently dropped
103without preventing state or link updates. Removing a link preserves history;
104deleting its repository removes both prefixes.
105
106Schema 8 adds the resolved issue state and migrates closed issues with a
107nonzero `closed_by_merge_request` to resolved, without changing timestamps,
108open indexes, or creating retroactive events. The migration and schema bump
109are atomic and idempotent. Like token upgrades, this uses a single batch
110(16 MiB limit, typically over 10k issue records). Older builds refuse schema 8.