Nav Malik
Nav's Blog

Follow

Nav's Blog

Follow
Copy All Files In S3 Bucket To Local With Aws Cli

Copy All Files In S3 Bucket To Local With Aws Cli

Nav Malik's photo
Nav Malik
·Sep 17, 2021·

1 min read

The AWS CLI makes working with files in S3 very easy. However, the file globbing available on most Unix/Linux systems is not quite as easy to use with the AWS CLI. S3 doesn’t have folders, but it does use the concept of folders by using the “/” character in S3 object keys as a folder delimiter (Also known as prefixes).

To copy all objects in an S3 bucket to your local machine simply use the below command with --recursive option.

aws s3 cp --recursive

For example

aws s3 cp s3://temp-bucket/ ./ --recursive

will copy all files from the “temp-bucket" bucket to the current working directory on your local machine. If there are folders represented in the object keys (keys containing “/” characters), they will be downloaded as separate directories in the target location.

The command aws s3 cp s3://temp-bucket/folder1 ./ --recursive is almost the same as the one above, but this command will only copy files from “folder1” folder (objects with keys starting with “folder1/”).

For using wildcards and patterns to copy only certain files, refer to Using Wildcards with AWS CLI on how to correctly use the --include and --exclude options.

 
Share this