BlastDNS
The highlight of the 3.1 release of BBOT is significant DNS performance improvements, following a fuller adoption of our own BlastDNS resolver. This means faster scans and potentially more results. We introduced BlastDNS in 3.0, but DNS brute-forcing still ran through massdns. After redesigning BlastDNS around a new brute-forcing mode and integrating it into BBOT, 3.1 now clearly outperforms 3.0.
The best way to understand the performance gains is to look at two different scenarios: a healthy link, where the bottleneck is the DNS servers themselves, and a congested link, where a bottleneck happens before traffic even reaches them. BBOT 3.1 is better in both, but previous versions especially struggled on a congested link: they would keep piling requests into the unhealthy link and cause a death spiral.
For the “congested” link, we used a link behind a NAT, which can only track a limited number of connections. Every DNS server BBOT talks to uses one of those slots, and once they run out, new connections fail for everything on the network.
Uncongested
The lighter color represents failed DNS queries. Even on a healthy link, massdns pushed too hard: at about the same query rate (roughly 1,000 per second), 3.0 got an answer to only about 65% of its queries, while 3.1 gets 93-96%. With BlastDNS handling all DNS traffic, including brute-forcing, its adaptive rate control keeps failures low, and the whole scan is faster as a result.
A typical subdomain enumeration scan finishes in about 40% less time, and finds the same number of subdomains. Part of that comes from brute-forcing, and part from ordinary resolution: 3.1 keeps 5x more normal DNS queries in flight by default, which is safe now that BlastDNS backs off on its own when a resolver starts dropping queries.
Congested
On a genuinely congested link, the difference is even more pronounced.
Running the exact same scan on a highly limited link, 3.0 knocked it offline. About 60% of the other web traffic on the link failed during brute-forcing (24% over the whole run), and we stopped the scan at 52 minutes, after the link went down completely. It’s a death spiral: almost every query fails, the retries pile on top of each other, and the link gets even worse.
Part of the reason is how many resolvers were in play. 3.0 sprayed queries across the entire public resolver list, almost 5,000 servers, and behind a NAT every one of those is another mapping the router has to track. 3.1 draws 600 of them at random per scan, drops the dead ones at startup, and keeps one socket open per resolver.
BBOT 3.1 adjusts to the conditions as it goes. On the same link, it finished in about 25 minutes, found about as many subdomains as it did on the uncongested link, and less than 1% of the other web traffic failed while it ran.
Most importantly, when the link can handle it, it pushes. BlastDNS doesn’t slow down until queries start to drop. When they do, it cuts its rate by 15%, then creeps back up a few percent at a time, so it hovers just under the most the link can carry instead of backing off for good.
With this change, you’ll also get more direct control of BBOT’s resolvers. The new --custom-nameservers flag lets you choose which resolvers BBOT uses for normal DNS resolution. Brute-forcing still uses its own list of public resolvers.
New Modules
js_unpacker - Transparently unpacks source maps, webpack, obfuscator.io, Next.js and Dean Edwards’ packer JavaScript, and feeds the results back into BBOT. This means more endpoints discovered, and much better leaked secrets detection when paired with TruffleHog. Added to the js-audit preset.
nowafpls - Our implementation of the “nowafpls” style WAF-bypass technique (pushing payload below large junk data padding). The module alone will detect whenever this technique is confirmed usable, but it’s also integrated directly into lightfuzz and other modules so it’s possible to test for vulnerabilities transparently right through the WAF in those cases.
favicon - A boost to our technology detection capabilities, this utilizes known mmh3 favicon hashes to emit TECHNOLOGY events.
otilabs - A new passive subdomain enumeration module supporting OTI Labs (API key required).
scanmalware - A new passive subdomain source, from ScanMalware’s public scan archive.
Significant Module Updates
telerik - Near-total rewrite of the telerik module to reduce external dependencies and support native detection. Automatically detects exploitable systems using known keys (even patched versions) with our badsecrets known-key spray technique.
nuclei - Nine new pass-through options for template selection, including template conditions (EPSS and CVSS score, CVE ID, vendor, product), tags, template IDs and protocol types, and new-templates-only. Two new presets: nuclei-recommended, built on ProjectDiscovery’s curated “recommended” profile, and nuclei-new. Budget mode is fixed: budget: 10 now means about 15 requests per host, down from 134. Template indexing is cached, so startup drops from about 35 seconds to near-instant.
excavate - New excavate submodule to detect “long-redirects.”
lightfuzz - Fewer false positives on SQLi detections and a few bug fixes and performance improvements.
wayback - Added a check to ensure hosts are alive before emitting URLs, potentially saving thousands of failed HTTP requests.
waf_bypass - Now only tests scan targets by default, dramatically decreasing the number of requests and time to finish the scan.
dnsdumpster - Now authenticated only (requires API key).
Python Support
BBOT 3.1 requires Python 3.11 or newer, and is fully compatible with Python 3.15.
BadDNS
We’ve also been working on significant upgrades to BadDNS, our DNS misconfiguration tool, and pointed BBOT 3.1 at the new version (BadDNS 2.5.0).
Here’s a quick highlight of the new BadDNS features:
New DELEGATION module: finds dangling CNAMEs on _acme-challenge, _dmarc and DKIM records.
On-demand TLS platforms: finds abandoned custom domains by looking at how the TLS handshake fails.
S3/GCS buckets in HTML: finds bucket references in page content that anyone could claim.
Total Signature Refresh: Removed 37 stale signatures, updated 78 existing signatures, and added 5 new ones.
Performance Improvements: word matching now runs every signature in a single YARA pass per response, instead of scanning once per signature.
Core Improvements
CDN/WAF port filtering is on by default. CDN edges answers on lots of ports that rarely lead anywhere. BBOT now drops open ports other than 80 and 443 on hosts tagged cdn or waf, cutting a lot of noise and wasted requests.
--allow-cdn-portsreverts to the 3.0 behavior.Smarter API rate limiting. API modules now honor Retry-After, rotate to the next API key on a 429, and back off between real failures. A rate limit no longer counts as a failure, so modules stop disabling themselves partway through a scan on a busy API.
Performance: the deepdiff dependency is gone, replaced with faster purpose-built comparisons, and simhash computation is faster.
Breaking Changes
Python 3.11 or newer is required. Python 3.10 is no longer supported.
portfilter is gone. CDN/WAF port filtering is built in and on by default. Remove portfilter from your presets;
--allow-cdn-portsturns the filter off.Custom module folders: module_dirs only works as a top-level preset key, and BBOT asks before loading each folder (-y to accept).
Preset conditions are read-only. They can warn or abort, but can’t modify the preset.
dnsdumpster requires an API key, and waf_bypass only tests scan targets by default.
Output: CPE-based technologies are named by product, and in-scope events are no longer tagged affiliate.
API requests now send a BBOT/<version> User-Agent.
helpers.response_to_json()is removed; useresponse_to_event_dict().lightfuzz can now fuzz WAF-protected hosts with default settings. In 3.0 it skipped hosts tagged as behind a WAF entirely. In 3.1 it runs a nowafpls probe on them and fuzzes them IF the probe successfully gets past the WAF.
Please visit our 3.1 Breaking Changes guide for a more comprehensive list.
Security Hardening
We’ve tightened security around loading presets and custom modules, and BBOT now handles archives and other data pulled from targets more defensively.