S3-compatible Object Storage for Kubernetes

S3-compatible Object Storage for Kubernetes: Store uploads in buckets, use familiar SDKs, and manage your application files within your Loopback project.

6 min read
S3-compatible Object Storage for Kubernetes

Applications work with files: users upload images, teams share documents, and background jobs generate exports. These files need a storage location that remains independent of individual containers and deployments.

With S3-compatible Object Storage on Loopback, you store files in buckets and access them from your application using familiar APIs and SDKs. You manage your buckets in the same project as your Kubernetes clusters. Your chosen S3 provider determines the provider and data location.

Store files independently of pods

Imagine an application that processes uploaded documents. Initially, it runs as a single instance. Each upload goes into the container's local filesystem and is served from there.

Running multiple instances introduces another requirement: every instance must be able to access the same documents. A file stored only in one container's filesystem is not automatically available to other instances. Writable container filesystems are also ephemeral. When a container is recreated, changes stored there are lost. Kubernetes provides different volume types and persistent storage mechanisms for this purpose.

Object Storage suits uploads and other files that your application reads and writes as complete objects. The application sends a file to a bucket and retrieves it through the S3 API when needed.

File storage then becomes independent of the lifecycle of an individual application instance. A new pod can access existing files through the same bucket when it has the appropriate configuration and access permissions.

S3-compatible Object Storage in detail

Object Storage organizes data in buckets. Within a bucket, each object has a key, called an Object Key. This key identifies the object. An object consists of its content and associated metadata.

A document store might use the following structure:

ComponentExampleMeaning
Bucketacme-documentsShared storage area
Object Keyuploads/2026/11/report.pdfIdentifier of the object within the bucket
ContentPDF fileThe stored data

Your application works with the bucket and Object Key. Downloading a file does not require a path to a particular container or node.

Loopback provides this storage through S3-compatible interfaces. You can use familiar S3 APIs, SDKs, and tools, configuring the connection with your selected S3 endpoint.

Object Storage and persistent volumes

Object Storage and persistent volumes serve different purposes. The deciding factor is how your application accesses data.

A persistent volume provides storage that you mount into a pod. Your application then uses filesystem operations. Object Storage is accessed through API calls.

RequirementSuitable approach
An application expects a mounted data directoryPersistent volume
A service uploads and downloads documentsObject Storage
Multiple application instances need API access to the same filesObject Storage
An application needs traditional filesystem operationsVolume with suitable properties

For your architecture, this means checking how your application accesses data before choosing a storage type. An existing program that expects a local directory needs an appropriate integration to access S3 directly.

On Loopback, Object Storage is a separate service within the project. Persistent volumes belong to the Kubernetes Workspace. Both forms of storage can serve different purposes within the same application.

An example application architecture

A document application could use the following architecture:

  1. A user uploads a document through the application.
  2. The backend stores the document under a defined Object Key in the bucket.
  3. When the document is requested later, the backend reads the object from the bucket.
  4. The backend delivers the file to the authorized user.

The bucket serves as the shared file store. You can decide independently which application instance handles an upload or download.

A private bucket suits this model. The application checks which user may retrieve which document. The backend's technical access to storage and a user's permission to access a document are two separate decisions.

This separation is particularly useful for applications where different users or teams manage their own documents.

Manage buckets and access in Loopback

In Loopback, you create buckets within your project. You specify the name, provider, location, and visibility. You can create and revoke access keys for storage access. A file browser also lets you upload, browse, and delete files through the interface.

Choose visibility to match the intended use:

  • Private buckets suit documents that your application delivers only after checking permissions.
  • Public buckets suit content intended to be publicly accessible.

For a project with internal documents and public assets, organizing these files in separate buckets makes sense. This also keeps the intended visibility clear when managing storage.

Access storage with an S3 SDK

You can implement access with Python and Boto3, for example. The SDK provides methods for file uploads and downloads. When creating the client, you can specify a custom endpoint and the required region.

First, install the SDK:

python -m pip install boto3

The following example assumes an existing bucket and a local file named report.pdf. The endpoint, region, bucket, and credentials are supplied through environment variables:

  • S3_ENDPOINT_URL
  • S3_REGION
  • S3_BUCKET
  • S3_ACCESS_KEY_ID
  • S3_SECRET_ACCESS_KEY

Use the connection details of your selected S3 service.

import os

import boto3


def main():
    client = boto3.client(
        "s3",
        endpoint_url=os.environ["S3_ENDPOINT_URL"],
        region_name=os.environ["S3_REGION"],
        aws_access_key_id=os.environ["S3_ACCESS_KEY_ID"],
        aws_secret_access_key=os.environ["S3_SECRET_ACCESS_KEY"],
    )

    bucket = os.environ["S3_BUCKET"]
    object_key = "uploads/report.pdf"

    client.upload_file(
        "report.pdf",
        bucket,
        object_key,
    )

    client.download_file(
        bucket,
        object_key,
        "report-downloaded.pdf",
    )

    print("File uploaded and downloaded again.")


if __name__ == "__main__":
    main()

The example shows the basic data flow: a local file is stored under an Object Key and then downloaded again. These methods are part of Boto3's standard S3 client.

Choose a provider and data location

Loopback offers several S3 providers. Your chosen provider determines where your data is stored.

Include your application's location in this decision. If a backend regularly transfers large files, the connection between the application and storage, along with expected transfer volume, belongs in your architecture planning.

A concrete data flow helps you estimate usage: how large is a typical upload? How often is the file read afterward? Does every download pass through the backend? These questions provide a more useful picture than stored data volume alone.

File storage as part of your stack

Object Storage gives your application a file store accessed through an established interface. Uploads, documents, and generated files remain accessible through their bucket and Object Key, regardless of which application instance is running.

Loopback connects this file store with the rest of your project: you manage your clusters and buckets on the same platform. For data access, you use familiar S3 tools and integrate storage directly into your application.

Explore Object Storage on Loopback

Your stack. One platform.

Start with managed Kubernetes, object storage, registry, or Bundles. One account, one permission model.