Claus-Theodor Riegg
2 years
Claus-Theodor Riegg
2 years
Andreas Vöst
2 years
Claus-Theodor Riegg
7 years
Claus-Theodor Riegg
6 years
Marius Schuller
3 years
Marius Schuller
3 years
Henning Koch
8 years
Andreas Herz
4 years
Claus-Theodor Riegg
4 years
Claus-Theodor Riegg
5 years
Claus-Theodor Riegg
5 years

Listing sizes in AWS S3 Buckets

Posted about 2 years ago. Visible to the public.

Getting the whole bucket size

aws s3 ls s3://$BUCKETNAME/ --recursive --human-readable --summarize | tail -n2

Tail is used because otherwise all files will be printed on screen (but you may want that for some reason).

Getting the size of a specific directory/file

You just need to add the path to the bucket name:

aws s3 ls s3://$BUCKETNAME/some/dir --recursive --human-readable --summarize | tail -n2

Getting the size of all subdirs

If you imagine there is something like ncdu or du -ha -d 1 /foo/bar you're wrong. If you want to get the size of all subdirs you have to do it yourself:

Your Bucket contains in $BUCKETNAME/some/dir multiple subdirs (for e.g. 2020, 2019 and 2018) and you want to see the size of these 3 directories.

S3PATH='s3://$BUCKETNAME/some/dir/'
for dir in $(aws s3 ls "${S3PATH}" | grep PRE | awk '{print $2}'); do 
    echo "${S3PATH}${dir}:"
    aws s3 ls "${S3PATH}${dir}" --recursive \
                                --human-readable \
                                --summarize\
    | tail -n2
done

A | column after done could improve the readability.

Avatar
Claus-Theodor Riegg
Last edit
About 2 years ago
Stefan Langenmaier
Keywords
summarize
Tags
About this deck
We are makandra and do test-driven, agile Ruby on Rails software development.