ARACHNE – Autonomous Research Agent Cluster with Hierarchical Navigation Engine

THE IDEA

ARACHNE – Autonomous Research Agent Cluster with Hierarchical Navigation Engine

Type: Software Status: POC Tech Stack: Python, FastAPI, Ollama, Playwright, ChromaDB (RAG), SQLite, React, WebSocket, Docker Problem Statement Manual research on complex, rapidly changing topics (crisis situations, markets, technology trends) is time-consuming, error-prone, and not scalable. Individual search engines only provide snapshots, not continuously updated, quality-assessed knowledge bases. Existing monitoring tools are limited to specific data sources and offer no intelligent filtering or deduplication. Description A distributed, AI-driven multi-agent research system that autonomously collects, evaluates, and builds structured knowledge from the web. A fast orchestrator model distributes research tasks to multiple autonomous browser agents that independently navigate websites, collect data, and evaluate it. A curator process filters, deduplicates, and builds a RAG knowledge base. Supports multi-topic management with endless mode for continuous monitoring (crises, stocks, Polymarket), anti-junk system with trust scores, blacklists, and fingerprinting, plus a real-time dashboard with WebSocket events. Use Case A team of AI assistants that search the internet around the clock and summarize the most important information on a topic. Link: https://github.com/rawk7000/Arachne (private repo)