Subdomain Enumeration Tool Face-off 2022
Comparing the industry's top subdomain enumeration tools
Update: 2023 version here.
What is the best subdomain enumeration tool in 2022?
The goal of this face-off is to answer this question, ranking the top subdomain enumeration tools based on:
1) The number of subdomains they're able to find
2) The time it takes to find them.
Tools being tested:
If you're a veteran OSINTer, you're probably familiar with most of the tools on this list — with the exception maybe of BBOT. BBOT is a new OSINT tool inspired by Spiderfoot, written here at Black Lantern Security, and released a little over a month ago. Like Spiderfoot, BBOT isn't exclusively a subdomain enumeration tool, but we designed it with subdomain enumeration in mind, which is a reason why we chose subdomain enumeration for its first real test. A subdomain enumeration benchmark seems like a good “christening” that will either mock or (hopefully) validate our efforts on the tool.
After sharing the results, we'll go over the exact command we ran for each tool, and details on any unique subdomains it uncovered.
Rules
Target: tesla.com
Active enumeration is allowed (HTTP/SSL)
Free APIs only. No paid APIs will be used in this test.
The purpose of this rule is to gauge the effectiveness of a tool in gathering FREE and OPEN SOURCE information without any tedious/expensive preconfiguration.
Wildcards and unresolved subdomains don't count. Subdomains must have at least one valid DNS record of type A, AAAA, MX, TXT, NS, SOA, SRV, or CNAME. The output from every tool will be cleaned using the same method.
DNS brute-forcing is allowed. However all tools will be configured to use the same base subdomain wordlist. This is intended to show off each tool's unique capabilities instead of only its wordlist quality. Also, since runtime is a factor in the results, this helps keep the scan durations more comparable.
Default settings only —no weird config changes.
With the exception of 1) the common brute wordlist and 2) bumping the thread count (because we have blazing fast internet), we will be using tools' default settings. Tool defaults should always be reasonable and we will avoid messing with them as giving more attention to the configuration of a single tool may give it an unfair advantage over the others.
Setup
The benchmarks will be performed on a modest Linode VM with the following specs:
2 dedicated CPU Cores
4GB RAM
~3,000Mbps download / ~2600Mbps upload according to speedtest.net
Each tool will be run separately, with ample pause between runs to account for rate limiting, etc.
Results
Analysis
The true winner of this benchmark may differ depending on whether speed or thoroughness is more important for your OSINT process. For example, BBOT found the highest number of subdomains, but Subfinder found 90% of what BBOT did, and in about a tenth of the time.
BBOT uses a recursive model similar to Spiderfoot which is one of the reasons for its higher subdomain yield and subsequently higher runtime. But even considering the higher runtime, it severely outperformed Spiderfoot for the purposes of subdomain enumeration, as seen in both of the above charts. In our testing, we let Spiderfoot run for 48 hours, after which we were forced to cancel it.
One interesting data point which we've included below is the number of unique subdomains found by each tool. This number is calculated by comparing each tool's output to the combined output of every other tool, and asking which subdomains it found that none of the other ones did.
BBOT’s massdns module, which pairs its recursive methodology with smart mutations, ensured that BBOT claimed the highest number of unique subdomains. Amass found several thanks to its unique Maltiverse module, and Lepus' Markov Chain / Permutation functionality was able to uncover a few as well.
Details
# Note: Each tool's default resolver thread count was multiplied by 10
bbot -t tesla.com -f subdomain-enum -m httpx -c modules.massdns.max_resolvers=5000https://github.com/blacklanternsecurity/bbot
Version: v1.0.3.736 (04 October 2022)
Subdomains: 409
Runtime: 12 minutes 19 seconds
Unique subdomains: 13
# Note: theharvester provides no way to specify custom wordlist, so its wordlists were manually modified
# theharvester did not allow for increased thread count
theHarvester.py --domain tesla.com --dns-lookup --dns-brute --source anubis,baidu,bevigil,binaryedge,bing,bingapi,bufferoverun,censys,certspotter,crtsh,dnsdumpster,duckduckgo,fullhunt,github-code,hackertarget,hunter,intelx,omnisint,otx,pentesttools,projectdiscovery,qwant,rapiddns,rocketreach,securityTrails,sublist3r,threatcrowd,threatminer,urlscan,virustotal,yahoo,zoomeyehttps://github.com/laramies/theHarvester
Version: v4.2.0 (13 August 2022)
Subdomains: 376
Runtime: 7 minutes 10 seconds
Unique subdomains: None
# Note: subfinder does not support DNS brute-forcing
subfinder -domain tesla.com -all -t 100https://github.com/projectdiscovery/subfinder
Version: v2.5.3 (03 August 2022)
Subdomains: 373
Runtime: 1 minute 17 seconds
Unique subdomains: None
# Note: Amass did not allow for increased thread count
amass enum -d tesla.com -active -brute -w subdomains-top1million-5000.txthttps://github.com/OWASP/Amass
Version: v3.20.0 (22 September 2022)
Subdomains: 342
Runtime: 8 minutes 42 seconds
Unique subdomains: 5
# Note: oneforall.py provides no way to specify custom wordlist, so its wordlists were manually modified
# oneforall.py did not allow for increased thread count
oneforall.py --target tesla.com runhttps://github.com/shmilylty/OneForAll
Version: v0.4.5 (10 July 2022)
Subdomains: 312
Runtime: 2 minutes 4 seconds
Unique subdomains: 1
# Note: Each tool's default resolver thread count was multiplied by 10
lepus.py tesla.com --permutate --reverse --ripe --portscan --markovify -w subdomains-top1million-5000.txt --threads 1000Version: git clone (05 October 2022)
Subdomains: 171
Runtime: 15 minutes 59 seconds
Unique subdomains:
# Note: Spiderfoot's dnsbrute module doesn't support custom wordlists, so its wordlists were manually modified
# Spiderfoot's command line interface does not support increased thread count
sf.py -t INTERNET_NAME -n -s tesla.comhttps://github.com/smicallef/spiderfoot
Version: v4.0 (07 April 2022)
Subdomains: 129
Runtime: 48+ hours (cancelled)
Unique subdomains: None
# Note: Note: Each tool's default resolver thread count was multiplied by 10
# Sublist3r doesn't support custom wordlists, so its wordlists were manually modified
sublist3r.py --domain tesla.com --bruteforce --threads 300https://github.com/aboul3la/Sublist3r
Version: git clone (04 October 2022)
Subdomains: 46
Runtime: 3 minutes 39 seconds
Unique subdomains: None
Conclusion
Comparing these OSINT tools was a fun and educational exercise, and we hope you find the results helpful.
The best subdomain enumeration tools, by the standards of this benchmark, are:
BBOT — MOST SUBDOMAINS
SUBFINDER — FASTEST
THEHARVESTER — Runner-up for both categories
Happy hacking!