Configure CP S3 Storage Mount¶

The huddo-boards-cp chart deploys a SeaweedFS S3 service (huddo-boards-cp-s3) for file storage. This page prepares the NFS mount referenced by s3.persistence.nfs in your values file.

Warning

s3.persistence is required. Without it SeaweedFS has no durable storage and data will be lost on pod restart. Use either the NFS mount below or s3.persistence.storageClassName for dynamic provisioning.

Note

Upgrading an existing installation that uses MinIO? Follow the SeaweedFS migration guide instead — it copies your data across before the cutover. The legacy MinIO mount instructions remain available.

Deploy instructions¶

Create the folder on the nfs.server
```
sudo mkdir /pv-connections/huddo-boards-s3
sudo chown 1000:1000 /pv-connections/huddo-boards-s3
sudo chmod 755 /pv-connections/huddo-boards-s3
```
Note

As of chart 2.3.5 the SeaweedFS pod runs as user 1000 (the same as the MinIO pods), so the directory must be writable by uid 1000. The chart also chowns it for you on start, so the chown above is belt-and-braces — but it is required if you set s3.persistence.fixOwnership: false (see the chart history).

On charts 2.1.0–2.3.4 SeaweedFS ran as root and required a no_root_squash export.
Ensure each Node in your Kubernetes cluster can mount this location.

Please modify the file /etc/exports on your NFS Server to include this line
```
/pv-connections/huddo-boards-s3 <IP_RANGE_OF_YOUR_SERVERS>/<SUBNET_MASK>(rw)
```
For example:
```
/pv-connections/huddo-boards-s3 192.168.0.0/255.255.0.0(rw)
```
Apply new NFS storage to exports
```
exportfs -ra
```

Reference the mount in your values file

s3:
    persistence:
        nfs:
            server: <nfs-server-ip>
            path: /pv-connections/huddo-boards-s3
            # Optional (chart >= 2.3.5) — extra NFS mount options, e.g.:
            # mountOptions:
            #     - sec=sys

Capacity planning¶

SeaweedFS stores objects in volume files of up to 30000 MB (about 31 GB) each, and one server holds at most s3.volumeMax of them, so the ceiling is:

s3.volumeMax  x  s3.volumeSizeLimitMB

Volume slots are not shared evenly, though. On the first write to a collection SeaweedFS creates 7 volumes at once, and a SeaweedFS install has two collections:

the filer's internal metadata log, written about a minute after startup
the huddo-boards bucket

Charts before 2.4.1 default to 8 slots, which can leave the bucket about 31 GB

Charts before 2.4.1 ship the upstream default of 8 slots; 2.4.1 defaults to 64. The first collection to write takes 7 of them and the other gets 1. The metadata log usually writes first, so the huddo-boards bucket is often left a single ~31 GB volume, not the 240 GB that 8 x 30000 MB suggests. Once that volume is full, uploads fail with an error containing no free volumes left.

Upgrade to 2.4.1, or set s3.volumeMax: "64" yourself on 2.3.6 or 2.4.0. On a running install the SeaweedFS pod restarts with the new limit on the next helm upgrade, and the bucket grows as its volume fills. Volumes already created stay where they are.

Set s3.volumeMax before you migrate:

s3:
    # At least 14 + (dataset + headroom) / 30 GB: 7 slots for the metadata
    # log, the rest for the bucket. Quoted: "0" means auto configure from
    # free disk space (on a small export that can be a single slot), and a
    # per-disk list ("64,64") is also accepted.
    volumeMax: "64"
    # Size of a single volume in MB (default 30000, which is also the
    # maximum — SeaweedFS refuses to start above it). Grow capacity with
    # volumeMax, not this.
    # volumeSizeLimitMB: 30000

64 gives a ceiling of about 2 TB in total (64 x 30000 MiB), of which the bucket can use about 1.8 TB once the metadata log has its 7 volumes. The rule above keeps one batch of 7 spare, so raise it for more than about 1.5 TB of data.

A few points worth knowing:

Volumes are created on demand and are not preallocated, so a generous volumeMax costs nothing until the data actually arrives. Prefer setting it comfortably high over resizing later.
The volume ceiling does not reserve disk. Size the NFS export (or s3.persistence.size when using storageClassName) for the real dataset as well — whichever of the two is smaller is what you actually get.
Deleted objects are not reclaimed immediately. SeaweedFS marks them deleted and recovers the space during compaction, so allow headroom above the raw dataset size.

To check what a running instance is using, exec into the SeaweedFS pod and ask the master:

kubectl exec -n <namespace> deploy/huddo-boards-cp-s3 -- sh -c 'echo "volume.list" | weed shell'

The Topology line reports the volume count as volume:<used>/<max> along with volumeSizeLimit:

> Topology volumeSizeLimit:30000 MB hdd(volume:14/64 active:14 free:50 remote:0)

Each volume line also names its Collection: huddo-boards for the bucket, empty for the metadata log. A free:0 on a single-node install means no further volumes can be created, though the existing ones keep accepting writes until each reaches the size limit.

Migrating a large dataset¶

The migration Job copies everything from MinIO in one pass, so SeaweedFS must be able to hold the whole dataset before it starts. Check the size of the source first:

kubectl exec -n <namespace> deploy/huddo-boards-cp-minio -- du -sh /data

Then set volumeMax to at least 14 + (that size + headroom) / 30 GB in the same values file you use for the migration. For example a 263 GB dataset needs 9 volumes for the bucket alone, which the default 8 cannot give it, so volumeMax: "64" leaves comfortable room to grow.