BBOT
Recursive OSINT Machine
Black Lantern Security is proud to announce the release of BBOT (BEE·bot): an open-source OSINT automation framework for hackers.
BBOT is the OSINT tool you’ve been waiting for. In a single command, it can execute subdomain enumeration, port scanning, web screenshots, vulnerability scans, and more.
Install and Run
At Black Lantern Security, we do a lot of OSINT. In a way, OSINT is the foundation of everything we do. It's the common denominator between every type of engagement, whether for a red-team, attack surface evaluation, or the usual pentest. If you want to hack something, you first have to find it.
Historically, when it comes to OSINT, each of us have our own tools and our own processes that we like best. For a long time, we stuck to our individual methods because none of us could agree on which tools were best, and because old habits die hard.
But eventually, we began to see that our OSINT was suffering because of this fragmented approach. Some tools were good at some things and some were good at others, but no tool or tools were good at everything, and none of them fit together in a cohesive way.
So we began to envision a way of improving our capabilities by pooling our efforts into a tool that we could all use and trust -- a tool that would replace everything else. This is what we set out to create with BBOT.
But there are already plenty of OSINT frameworks available. Why choose BBOT?
BBOT is the culmination of years of manual OSINT-gathering experience. In short, it solves quite a lot of problems. But here I'm going to try and stay as brief as possible, hitting the main points that differentiate BBOT from other tools.
BBOT Features
Recursive
Graphing
Modular
Multi-Target
Automatic Dependencies
Smart Dictionary Attacks
Scope Distance
Easily Configurable via YAML
Great. Let's get into it.
Recursive
Most tools take a Phased Approach to OSINT. Combine the output from a bunch of miscellaneous command-line tools, sort, clean, and uniq it, then feed it into the next set of tools, sort, clean, and uniq it, and repeat.
What's the problem with doing it that way?
The problem is that a Phased Approach has a tendency to miss things.
Imagine you're enumerating subdomains for evilcorp.com. Your phases may look like this:
Query a bunch of subdomain APIs and do a big DNS brute-force
Combine the subdomains from the previous two steps, resolve them to IP addresses, and resolve the IP addresses back to domains again
Perform a port scan on every host
Visit the open ports on HTTP/HTTPS and parse results for more subdomains
Visit the open ports for SSL certs and parse results for more subdomains
Finally, clean and dedupe into a final list, making sure to filter out troublesome garbage like wildcard DNS entries.
If you've done any OSINT yourself, you may already see the problem. What if Step 5 yields a unique subdomain that you didn't have in Step 3? It's possible that subdomain has additional goodies such as PTR records, open ports, and web pages. Should you go back to Step 3 and run your port scan again? Should you go back to Step 2 and resolve it again? Or should you go all the way back to Step 1 and run the brute-force again?
The problem is that each of these steps rely on the output from the step before, which means that if your goal is to find as much as possible, it's not enough to only do this once. You have to repeat these steps again and again, each time feeding their output back into the beginning.
The point is that OSINT is simply not well-suited to a linear workflow. BBOT solves this with Recursion, which is its characteristic feature.
Instead of working in phases, BBOT's modules continually feed results to each other in real-time as they are discovered. Phases are not used because each newly-found piece of information is acted upon immediately, being fed back into the machine and used to continue the discovery process. This allows for the discovery of distant and well-hidden nuggets that a phased approach would not uncover.
You may recognize this recursive model from Spiderfoot. It is worth mentioning that before BBOT, Spiderfoot was my tool of choice, and it's where I learned the value of a recursive approach to OSINT. Spiderfoot is a brilliant tool, and I wanted it to be the one-stop OSINT tool at Black Lantern Security. But, ultimately, it couldn't stand up to our demands, and my efforts led, instead, to the development of something entirely new.
Graph Output
BBOT natively supports output to Neo4j. Other output formats include JSON, CSV, TXT, HTTP, Websockets, and more. You can even output to multiple destinations at once.
Modules
BBOT has over 50 modules and counting. Modules are written in Python, and each one is its own .py file. Modules are easy to write, and they have an arsenal of helper functions at their disposal to help make them the best they can be:
Built-in threading and parallelization
Zero-effort data sanitization and deduplication
Automatic dependency installation (more on this later)
Interact.sh integration for finding SSRFs and other web goodies
Smart target-specific wordlist for DNS enumeration, bucket brute-forcing, dirbusting, etc. (more on this later)
Configurable options via the CLI or BBOT's YAML config file
A sample module (mymodule.py):
Multi-Target
BBOT accepts unlimited targets, both from files and directly via the command line. Targets can be of differing types, e.g. IP addresses, DNS names, Subnets, etc.
Whitelists and blacklists are also supported, enabling granular scope control for pentest and bug bounty targets.
Dependency Handling
BBOT automates module dependencies with Ansible. When you install BBOT, you pipx install, and that's it. There's no fiddling with third-party tools and dependencies. Even if a module requires a special binary or a specific OS package, BBOT will install it for you.
Word Cloud
During each scan, BBOT gathers target-specific keywords from DNS names, web pages, etc., and compiles them into a shared wordlist that is accessible to all modules. This wordlist is saved at the end of a each scan, and can be used for password cracking, etc., or reused in subsequent scans.
BBOT's massdns module makes extensive use of the word cloud for subdomain brute-forcing. But this type of data is useful for all sorts of other tasks, such as dirbuster and storage bucket enumeration.
Scope Distance
Scope Distance is a unique feature of BBOT that allows you to control how far out BBOT will explore. Since BBOT is recursive, it needs to have an "exit condition" (i.e., a mechanism that tells it when to stop searching). This is where Scope Distance comes in. Each newly-discovered piece of data—an "Event" in BBOT terms—gets assigned a scope distance that represents how many hops away it is from the main scope.
By default, BBOT searches up to one hop away from your defined target, and although its DNS resolution spiders out to three hops, it will only output the data if it finds something that's in scope. Of course all of these settings are configurable via bbot.yaml.
YAML Config
BBOT's scanner and modules are configurable via a single YAML file. This is where you put things like API keys.
Conclusion
We hope you're as excited about BBOT as we are. But, why are you still reading about it? Go try it out!
https://github.com/blacklanternsecurity/bbot
Be sure to stay tuned for more BBOT content, including live demos and head-to-head comparisons with other OSINT tools.