Skip to content

Configure CP S3 Storage Mount

The huddo-boards-cp chart deploys a SeaweedFS S3 service (huddo-boards-cp-s3) for file storage. This page prepares the NFS mount referenced by s3.persistence.nfs in your values file.

Warning

s3.persistence is required. Without it SeaweedFS has no durable storage and data will be lost on pod restart. Use either the NFS mount below or s3.persistence.storageClassName for dynamic provisioning.

Note

Upgrading an existing installation that uses MinIO? Follow the SeaweedFS migration guide instead — it copies your data across before the cutover. The legacy MinIO mount instructions remain available.

Deploy instructions

  1. Create the folder on the nfs.server

    sudo mkdir /pv-connections/huddo-boards-s3
    sudo chown 1000:1000 /pv-connections/huddo-boards-s3
    sudo chmod 755 /pv-connections/huddo-boards-s3
    

    Note

    As of chart 2.3.5 the SeaweedFS pod runs as user 1000 (the same as the MinIO pods), so the directory must be writable by uid 1000. The chart also chowns it for you on start, so the chown above is belt-and-braces — but it is required if you set s3.persistence.fixOwnership: false (see the chart history).

    On charts 2.1.02.3.4 SeaweedFS ran as root and required a no_root_squash export.

  2. Ensure each Node in your Kubernetes cluster can mount this location.

    Please modify the file /etc/exports on your NFS Server to include this line

    /pv-connections/huddo-boards-s3 <IP_RANGE_OF_YOUR_SERVERS>/<SUBNET_MASK>(rw)
    

    For example:

    /pv-connections/huddo-boards-s3 192.168.0.0/255.255.0.0(rw)
    
  3. Apply new NFS storage to exports

    exportfs -ra
    
  4. Reference the mount in your values file

    s3:
        persistence:
            nfs:
                server: <nfs-server-ip>
                path: /pv-connections/huddo-boards-s3
                # Optional (chart >= 2.3.5) — extra NFS mount options, e.g.:
                # mountOptions:
                #     - sec=sys
    

Capacity planning

SeaweedFS stores objects inside a fixed number of volume files. The total capacity of the instance is:

s3.volumeMax  x  s3.volumeSizeLimitMB

The chart ships the SeaweedFS defaults, 8 volumes of 30000 MB, so roughly 240 GB. (SeaweedFS counts that limit in MiB, so the true figure is nearer 252 GB — the round numbers below stay on the conservative side.) These are upstream defaults rather than a sizing recommendation. Once every volume slot is allocated and full, writes fail with an error containing no free volumes left.

Set s3.volumeMax for your dataset before you migrate:

s3:
    # >= (dataset + headroom) / 30 GB. Quoted: "0" means auto configure
    # from free disk space, and a per-disk list ("8,8") is also accepted.
    volumeMax: "32"
    # Size of a single volume in MB (default 30000, which is also the
    # maximum — SeaweedFS refuses to start above it). Grow capacity with
    # volumeMax, not this.
    # volumeSizeLimitMB: 30000

A few points worth knowing:

  • Volumes are created on demand and are not preallocated, so a generous volumeMax costs nothing until the data actually arrives. Prefer setting it comfortably high over resizing later.
  • The volume ceiling does not reserve disk. Size the NFS export (or s3.persistence.size when using storageClassName) for the real dataset as well — whichever of the two is smaller is what you actually get.
  • Deleted objects are not reclaimed immediately. SeaweedFS marks them deleted and recovers the space during compaction, so allow headroom above the raw dataset size.

To check what a running instance is using, exec into the SeaweedFS pod and ask the master:

kubectl exec -n <namespace> deploy/huddo-boards-cp-s3 -- sh -c 'echo "volume.list" | weed shell'

The Topology line reports the volume count as volume:<used>/<max> along with volumeSizeLimit:

> Topology volumeSizeLimit:30000 MB hdd(volume:8/32 active:8 free:24 remote:0)

A free:0 on a single-node install means no further volumes can be created, though the existing ones keep accepting writes until each reaches the size limit.

Migrating a dataset larger than the default

The migration Job copies everything from MinIO in one pass, so SeaweedFS must be able to hold the whole dataset before it starts. Check the size of the source first:

kubectl exec -n <namespace> deploy/huddo-boards-cp-minio -- du -sh /data

Then set volumeMax to at least (that size + headroom) / 30 GB in the same values file you use for the migration. For example a 263 GB dataset needs a minimum of 9 volumes to fit and would not fit in the default 8, so volumeMax: "32" (about 960 GB) leaves comfortable room to grow.