SpiderFoot vs SearchWebCode
SpiderFoot
SpiderFoot is an open-source (MIT) tool that automates OSINT collection for threat intelligence and for mapping an organisation's attack surface. Given a target such as a domain, IP address, email address or name, it runs over 200 modules against data sources and correlates the results, using a YAML-configurable correlation engine with 37 pre-defined rules. It runs as a self-hosted web application or from the command line, supports Docker deployment, can search the dark web through Tor, and can call other tools such as Nmap and DNSTwist. The project has been on GitHub since 2012.
Pros
- Free and open source under the MIT licence
- Broad automated collection from one scan
- Self-hosted, so scan data stays on your own infrastructure
Things to check
- Self-hosted: you install, run and maintain it
Pricing: Free and open source (MIT).
SearchWebCode
SearchWebCode is a search engine for the source code of public websites. It indexes the HTML, JavaScript and CSS of about 131 million homepage domains and matches exact strings or regular expressions, returning every site that contains the match together with its full source. Security uses include finding sites that load a known-bad script or an outdated library, and linking sites that share a tracking or analytics ID; it is also used to map technology stacks. A signature library identifies common technologies by their fingerprints, and results can be filtered by domain and exported to CSV or read through an API.
Pros
- Regex search across web source code at a low per-search price
- Useful for tracing shared analytics IDs and for finding sites loading a vulnerable script
- CSV export and API on every plan
Things to check
- Indexes homepages by default; every page is an Enterprise option
- Results reflect the last crawl; the index is refreshed monthly, per its Capterra listing
- Unlimited plan carries a fair-use limit of 500 requests a day
Pricing: Pay as you go at $0.05 a search (empty results are free), or Unlimited at $10/month billed $120 a year with a fair-use limit of 500 requests a day. Enterprise on request.