Thursday, 25 November 2010

Setting permissions on Amazon AWS S3 buckets using S3cmd

Calvium stores virtually all of its toolkit data on S3 servers. We often find ourselves using Cyberduck to manually upload and download files when required. One thing that is a pain though is setting permissions. Cyberduck doesn't appear to allow you to set the permissions on all files within a folder recursively. When you have a lot of files this can be a real pain. Happily s3cmd comes to the rescue! s3cmd is a third-party (e.g. not created by amazon) command line client for S3 which runs on linux, unix, and also MacOSX. It may compile on Windows, I haven't tried.


1) Make everything in a bucket public (e.g. everyone has read access)


s3cmd setacl s3://myexamplebucket.calvium.com/ --acl-public --recursive


2) Make everything in a bucket private (e.g. accessible only to the creator)


s3cmd setacl s3://myexamplebucket.calvium.com/ --acl-private --recursive


The above commands also work on folders within buckets, or on individual files (the --recursive option is not required in this last case). As you can imagine there are many more options (type s3cmd --help to see more help).

3) Disable Directory Listing in a bucket


s3cmd setacl s3://myexamplebucket.calvium.com/ --acl-private


By default, new buckets will list all of the files within them. This isn't usually a good idea though, as it makes it very easy indeed for someone to scape all the content of your site. The command above just omits the --recursive option. For new buckets it's probably best to run command (1) before (3) so that your files are actually readable.

8 comments:

  1. Any option to specify object-lifecycle-mgmt http://docs.amazonwebservices.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html using s3cmd ?

    ReplyDelete
  2. Found this useful, thanks!

    -G

    ReplyDelete
  3. Very helpful. I'm loving s3cmd! Thank you.

    ReplyDelete
  4. Hi,I want that all files in the bucket are readed from "Owner" and "All Uthenticated Users" only, can you help me please?

    ReplyDelete
  5. One of the biggest challenges that organizations face today is having inaccurate data and being unresponsive to the needs of the Amazon AWS Users Email List organization.

    ReplyDelete
  6. Thanks for this great post! - This provides good insight. You might also be interested to know more about generating more leads and getting the right intelligence to engage prospects.
    Techno Data Group implements new lead gen ideas and strategies for generating more leads and targeting the right leads and accounts.
    Amazon AWS Users Email & Mailing List

    ReplyDelete