How to search for Open Amazon s3 Buckets and their contents — https://buckets.grayhatwarfare.com

GrayHatWarfare created https://buckets.grayhatwarfare.com/ a free tool that lists open s3 buckets and helps you search for interesting files.

For an intro on what Amazon open buckets are, please read the following: https://blog.rapid7.com/2013/03/27/open-s3-buckets/

In essence, many files are publicly accessible, some by design. These files sometimes include very sensitive data. https://github.com/nagwww/s3-leaks has a list of the biggest leaks recorded.

Another reason why application owners who provide backups solutions and cloud storage should be encrypting their data before placing it on S3.  Anything less is negligence on their part.

Since this was exposed, many projects have been created that can enumerate s3 buckets:

All these tools/projects have some common problems:

  • The real problem is where to find the list to brute-force for buckets,  and not actually doing the brute-force.
  • All tools/projects only scan the first page for results.
  • thebuckhacker.com includes uninteresting files and useful results tend to be lost in the noise. Also, the first 1000 results of each bucket are fairly limited.
  • The process is slow and not productive. It’s not very useful for pen-testers to run a tool to run for days, save the exports somewhere and then grep them whenever they want to search for something. What is better is a useful tool in front of a large database.

 

Now there is  http://buckets.grayhatwarfare.com/.  Which took the ideas of the many projects and tools previously mentioned above.

The project’s features are:

  • It is a searchable database of open buckets.
  • Includes millions of results within buckets (In the future might be more).
  • Cleaned up and removed uninteresting files like images. Most images names are auto-generated.
  • Currently has ~180.000.000 files. Included are all images that number would go up to a few billion, which is a completely different system.
  • As of today, 70 000 and growing buckets are listed (not all of them have “interesting” files)
  • Full-text search with binary logic (can search for keywords and also stopwords)
  • List of the buckets.
  • The user can browse the contents of the bucket.
  • Excluded a lot of other things that are not interesting like cloud-watch logs.
  • Found a solution to the problem on how to generate possible names for buckets. The process reviles some hundreds of new buckets per day.
  • Automated the process.

The project is currently free and running on servers paid by me. There are some limitations in place to protect resources, but otherwise pentesters can use this on their daily tasks.

Whats to come in grayhatwarfare.com

Lots of cool things:

  • Subdomains pointing to expired buckets which can lead to something like this: https://www.reddit.com/r/netsec/comments/8t0pb2/how_i_hacked_applecom/
  • Huge lists of exposed version control (.git) which can expose the website’s repository (Source, password files, log files etc).
  • Exposed cameras/IOT devices.
  • Huge resources like extremely large (actual) cracked password lists.

 

Ref Source