How to search for Open Amazon s3 Buckets and their contents — https://buckets.grayhatwarfare.com
GrayHatWarfare created https://buckets.grayhatwarfare.com/ a free tool that lists open s3 buckets and helps you search for interesting files.
For an intro on what Amazon open buckets are, please read the following: https://blog.rapid7.com/2013/03/27/open-s3-buckets/
In essence, many files are publicly accessible, some by design. These files sometimes include very sensitive data. https://github.com/nagwww/s3-leaks has a list of the biggest leaks recorded.
Another reason why application owners who provide backups solutions and cloud storage should be encrypting their data before placing it on S3. Anything less is negligence on their part.
Since this was exposed, many projects have been created that can enumerate s3 buckets:
- https://github.com/jordanpotti/AWSBucketDump — takes an argument a wordlist and can check if buckets with that name exist. Also, it can download the files from the first page of results.
- https://www.thebuckhacker.com/ — Search engine for some buckets. It includes (according to their posts) ~100 000 buckets and growing, and they index the first page of each.
- https://github.com/sa7mon/S3Scanner —
- https://github.com/kromtech/s3-inspector — Uses amazon keys to check if some of your buckets are open.
All these tools/projects have some common problems:
- The real problem is where to find the list to brute-force for buckets, and not actually doing the brute-force.
- All tools/projects only scan the first page for results.
- thebuckhacker.com includes uninteresting files and useful results tend to be lost in the noise. Also, the first 1000 results of each bucket are fairly limited.
- The process is slow and not productive. It’s not very useful for pen-testers to run a tool to run for days, save the exports somewhere and then grep them whenever they want to search for something. What is better is a useful tool in front of a large database.
Now there is http://buckets.grayhatwarfare.com/. Which took the ideas of the many projects and tools previously mentioned above.
The project’s features are:
- It is a searchable database of open buckets.
- Includes millions of results within buckets (In the future might be more).
- Cleaned up and removed uninteresting files like images. Most images names are auto-generated.
- Currently has ~180.000.000 files. Included are all images that number would go up to a few billion, which is a completely different system.
- As of today, 70 000 and growing buckets are listed (not all of them have “interesting” files)
- Full-text search with binary logic (can search for keywords and also stopwords)
- List of the buckets.
- The user can browse the contents of the bucket.
- Excluded a lot of other things that are not interesting like cloud-watch logs.
- Found a solution to the problem on how to generate possible names for buckets. The process reviles some hundreds of new buckets per day.
- Automated the process.
The project is currently free and running on servers paid by me. There are some limitations in place to protect resources, but otherwise pentesters can use this on their daily tasks.
Whats to come in grayhatwarfare.com
Lots of cool things:
- Subdomains pointing to expired buckets which can lead to something like this: https://www.reddit.com/r/netsec/comments/8t0pb2/how_i_hacked_applecom/
- Huge lists of exposed version control (.git) which can expose the website’s repository (Source, password files, log files etc).
- Exposed cameras/IOT devices.
- Huge resources like extremely large (actual) cracked password lists.
Ref Source