We're excited to announce the release of SpiderSuite 2.0.0, a major update that transforms how you crawl, analyze, and interact with web applications. This release introduces powerful new capabilities for security researchers, penetration testers, and developers who need comprehensive web crawling and analysis tools.
Major Features
Redesigned User Interface
SpiderSuite 2.0.0 comes with a completely revamped GUI that enhances usability and workflow efficiency. The new interface provides a more intuitive experience for managing complex crawling operations and analyzing results.
Advanced Crawler Engines
We've introduced four specialized crawler engines to handle different crawling scenarios:
Headless Crawler
Navigate and crawl JavaScript-heavy single-page applications (SPAs) with ease. The headless crawler executes JavaScript, handles dynamic content loading, and captures the full rendered state of modern web applications.
Bruteforce Crawler
Discover hidden pages and endpoints through intelligent directory and file enumeration. Perfect for comprehensive site mapping and discovering undocumented resources.
Links Crawler
Import and crawl from multiple data sources including:
- Sitemap files (sitemap.xml)
- CSV files
- ZAProxy exports
- Atom feeds (atom.xml)
- RSS feeds (rss.xml)
- And more...
Onion Crawler
Safely explore and map darkweb sites through the Tor network. Built with security and anonymity in mind for researchers investigating onion services.
HTTP Traffic Tools
SpiderSuite 2.0.0 introduces a complete suite of HTTP traffic manipulation tools:
Intercept
A powerful proxy tool for intercepting HTTP(S) traffic in real-time. Modify requests before they're sent and responses before they reach the client. Essential for security testing and debugging.
Request
Craft, send, and analyze custom HTTP(S) requests with full control over headers, methods, body content, and more. Perfect for API testing and manual security assessments.
Fuzzer
Automate security testing with intelligent HTTP(S) request fuzzing. Test for injection vulnerabilities, authentication bypasses, and other security issues by systematically varying request parameters.
HTTPQL Query Language
Introducing HTTPQL, a powerful domain-specific query language for searching and filtering crawled pages developed by the team at caido.
Write expressive queries to find specific requests, responses, or patterns across your entire crawl dataset.
req.ext.ne:".js" AND req.ext.ne:".css" AND req.ext.ne:".png" AND req.ext.ne:".jpg"
Minor Features
Universal Import Support
SpiderSuite 2.0.0 can now import data from a wide variety of sources:
File Formats:
- HTTP Archive (HAR)
- WARC (.warc, .warc.gz)
Third-Party Tools:
- Burp Suite (.xml)
- Fiddler (.saz)
- Katana
- Caido
Flexible Export Options
Export your crawl data in the format that best suits your workflow:
Standard Formats:
- HAR (HTTP Archive)
- WARC (.warc, .warc.gz)
- Sitemap XML
- Filesystem export
Data Formats:
- CSV
- HTML
- JSON
- JSONL
- LLM-ready data (optimized JSON & JSONL for AI training and analysis)
Bug Fixes & Improvements
- Fixed SSL/TLS errors: Resolved certificate validation issues that could cause crawls to fail on certain sites
- Optimized graph engine: New, more efficient graph implementation for better performance when analyzing site structures and relationships
- General stability improvements and performance optimizations
Getting Started
Upgrade to SpiderSuite 2.0.0 today and experience the next generation of web crawling and analysis. Whether you're mapping a new target, hunting for vulnerabilities, or analyzing web application architecture, SpiderSuite 2.0.0 has the tools you need.
We'd love to hear your feedback on this release. Report issues, request features, or share your use cases with the SpiderSuite community.