The AWS CLI makes working with files in S3 very easy. However, the file globbing available on most Unix/Linux systems is not quite as easy to use with the AWS CLI. S3 doesn’t have folders, but it does use the concept of folders by using the “/” character in S3 object keys as a folder delimiter (Also known as prefixes).

To copy all objects in an S3 bucket to your local machine simply use the below command with --recursive option.

aws s3 cp --recursive

For example

aws s3 cp s3://temp-bucket/ ./ --recursive

will copy all files from the “temp-bucket" bucket to the current working directory on your local machine. If there are folders represented in the object keys (keys containing “/” characters), they will be downloaded as separate directories in the target location.

The command aws s3 cp s3://temp-bucket/folder1 ./ --recursive is almost the same as the one above, but this command will only copy files from “folder1” folder (objects with keys starting with “folder1/”).

For using wildcards and patterns to copy only certain files, refer to Using Wildcards with AWS CLI on how to correctly use the --include and --exclude options.