Managing your Object Store with s3cmd

S3cmd is a command-line tool that you use to manage your objects and containers in your S3-compatible OpenStack Swift Object Store. It allows you to perform tasks such as uploading, downloading, synchronizing and deleting files, and configuring access permissions, all from the command line. In this guide you install and configure S3cmd.

Installation on Ubuntu

Stap 1

Connect to your Linux server via SSH.

Stap 2

Update the package manager, install s3cmd, and test that it works:

sudo apt -y update
sudo apt -y install s3cmd
s3cmd --version

Then continue immediately with the configuration of s3cmd.

Installation on Windows

Stap 1

Start the command prompt CMD (click the Windows Start button, search for 'CMD' and click the result).

Stap 2

Update the Python package installer PIP and install s3cmd:

py -m pip install --upgrade pip
py -m pip install --user s3cmd

Stap 3

In Windows you can only call S3cmd via Python; directly adding the directory to Path unfortunately does not make it executable in CMD, but there is a workaround:

Use the following command in CMD:

where s3cmd

You will now see a location in the output, for example C:\Users\Administrator\AppData\Local\Programs\Python\Python312\Scripts\s3cmd. Copy this location.

Stap 4

Create a .cmd file in a directory such as c:\tools\s3cmd.cmd and give the file the following contents:

@echo off
py "C:\Users\Administrator\AppData\Local\Programs\Python\Python312\Scripts\s3cmd" %*

Replace C:\Users\Administrator\AppData\Local\Programs\Python\Python312\Scripts\s3cmd with the output of the command from step 3 (\s3cmd must be at the end).

Stap 5

Press the Windows Start icon + R on your keyboard at the same time. The run window appears. Type the command 'sysdm.cpl' and press 'OK' or the enter key.

Stap 6

Click the 'Advanced' tab > 'Environment Variables'.

Stap 7

You will see a 'User variables for ...' section and a 'System variables' section. Under 'User variables' or 'System variables' (for all users on your computer/server), scroll to 'Path' and click 'Edit'.

Stap 8

Click 'New' and enter the directory you created in step 1 (case-sensitive). Finally, click 'OK' three times to close all open windows.

That's it! Restart CMD and test that it works with the command:

s3cmd --version

Then continue directly with the configuration of s3cmd.

Configure S3cmd for Object Store

Stap 1

Start the configuration of your Object Store with the command:

s3cmd --configure

Stap 2

You will see a number of prompts. Answer the prompts as in the example below and enter the details of your S3 token. The S3 endpoint looks for example like 540436a14e67c7d45a6.objectstore.eu/

The encryption password and GPG password are optional but recommended: they encrypt files while they are being transferred to/from your Object Store. On Windows you can use, for example, Gpg4win for this (make sure to specify the full path to the .exe). 

Access Key: YOUR_ACCESS_KEY
Secret Key: YOUR_SECRET_KEY
Default Region: EU

Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.
S3 Endpoint: YOUR_PROJECT_ID.objectstore.eu

Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used
if the target S3 system supports dns based buckets.
DNS-style bucket+hostname:port template: YOUR_PROJECT_ID.objectstore.eu/%(location)s

Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password: OPTIONAL_PASSWORD
Path to GPG program: OPTIONAL_GPG_PATH

When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol: True

On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can't connect to S3 directly
HTTP Proxy server name:

Test access with supplied credentials? [Y/n] Y
Save settings? [y/N] Y

After a successful configuration you will see the location where your configuration has been saved, for example:

Configuration saved to 'C:\Users\Administrator\AppData\Roaming\s3cmd.ini'

Basic commands

Below you find an overview of the most common management commands. In the commands below, replace container-name with the name of your container.

Create containers (buckets):

s3cmd mb s3://<container-name>

List containers (buckets):

s3cmd ls

Show objects in a container:

s3cmd ls s3://container-name/

Upload a file:

s3cmd put local-file.txt s3://container-name/

Download a file:

s3cmd get s3://container-name/remote-file.txt local-file.txt

Delete a file:

s3cmd del s3://container-name/file-to-delete.txt

Need help?

View all options of s3cmd with s3cmd --help. You can also take a look at the FAQ from the makers of s3cmd. If you want to know more about our OpenStack services or Object Store in general, start at the product pages of Public Cloud and OpenStack.