Thursday 25 November 2010

Setting permissions on Amazon AWS S3 buckets using S3cmd

Calvium stores virtually all of its toolkit data on S3 servers. We often find ourselves using Cyberduck to manually upload and download files when required. One thing that is a pain though is setting permissions. Cyberduck doesn't appear to allow you to set the permissions on all files within a folder recursively. When you have a lot of files this can be a real pain. Happily s3cmd comes to the rescue! s3cmd is a third-party (e.g. not created by amazon) command line client for S3 which runs on linux, unix, and also MacOSX. It may compile on Windows, I haven't tried.


1) Make everything in a bucket public (e.g. everyone has read access)


s3cmd setacl s3://myexamplebucket.calvium.com/ --acl-public --recursive


2) Make everything in a bucket private (e.g. accessible only to the creator)


s3cmd setacl s3://myexamplebucket.calvium.com/ --acl-private --recursive


The above commands also work on folders within buckets, or on individual files (the --recursive option is not required in this last case). As you can imagine there are many more options (type s3cmd --help to see more help).

3) Disable Directory Listing in a bucket


s3cmd setacl s3://myexamplebucket.calvium.com/ --acl-private


By default, new buckets will list all of the files within them. This isn't usually a good idea though, as it makes it very easy indeed for someone to scape all the content of your site. The command above just omits the --recursive option. For new buckets it's probably best to run command (1) before (3) so that your files are actually readable.

15 comments:

  1. Any option to specify object-lifecycle-mgmt http://docs.amazonwebservices.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html using s3cmd ?

    ReplyDelete
  2. Found this useful, thanks!

    -G

    ReplyDelete
  3. Very helpful. I'm loving s3cmd! Thank you.

    ReplyDelete
  4. Hi,I want that all files in the bucket are readed from "Owner" and "All Uthenticated Users" only, can you help me please?

    ReplyDelete
  5. One of the biggest challenges that organizations face today is having inaccurate data and being unresponsive to the needs of the Amazon AWS Users Email List organization.

    ReplyDelete
  6. Thanks for this great post! - This provides good insight. You might also be interested to know more about generating more leads and getting the right intelligence to engage prospects.
    Techno Data Group implements new lead gen ideas and strategies for generating more leads and targeting the right leads and accounts.
    Amazon AWS Users Email & Mailing List

    ReplyDelete

  7. Wonderful blog!!! I liked the complete article…. great written,Thanks for all the information you have provided…
    Amazon-AWS Training in Hyderabad

    Hadoop Training in Hyderabad

    ReplyDelete
  8. After reading this blog I am very strong in this topics and this blog is really helpful to all.. Explanation are very clear so it is easy to understand.. Thanks for sharing this blog…
    AWS Training in Chennai|Best AWS Training in Chennai

    ReplyDelete
  9. AWS Training In Bangalore
    Your sending the nice information regarding AWS. And also i am learning lot of things from your site. Please post this type of information in future also

    ReplyDelete
  10. Nice article, users are attracted when they see your post thanks for posting keep updating AWS Online Training Hyderabad

    ReplyDelete

  11. Thanks for sharing,excellent information.It is very useful for me to learn and understand easily.Tableau is a powerful and fastest growing data visualization tool used in the Business Intelligence Industry. Business Intelligence Industry suggest to take tableau course to enhance their skills
    tableau training institute in bangalore

    ReplyDelete