TinyFish Launches BigSet: An Open-Source Multi-Agent System That Builds Structured Live Datasets from Plain-English Descriptions

TinyFish has unveiled BigSet, an innovative open-source multi-agent system designed to revolutionize how organizations create structured datasets. By accepting simple plain-English descriptions, BigSet automatically generates comprehensive, live datasets in structured table format through coordinated AI agents that research the web in parallel. This advancement addresses a critical bottleneck in data engineering and machine learning workflows, where dataset creation traditionally requires significant manual effort and technical expertise.

BigSet operates through a sophisticated orchestration system paired with specialized sub-agents working simultaneously across the internet. Users need only provide a single-sentence description of their desired dataset—such as "Fortune 500 companies and their current CEOs"—and the system handles the rest. The orchestrator coordinates multiple agents that conduct live web research, aggregate information, and structure findings into clean, organized tables ready for immediate use.

The platform operates entirely in open-source format, enabling developers and organizations to customize, deploy, and integrate the system into existing workflows without proprietary constraints. This approach democratizes access to advanced data collection and structuring capabilities previously available only through expensive commercial solutions or extensive in-house development.

• Accelerated Data Pipeline Development: Organizations can dramatically reduce time spent on manual data collection and cleaning
• Reduced Operational Costs: Eliminates need for dedicated data teams for routine dataset creation tasks
• Real-Time Data Access: Agents research live web sources, ensuring datasets remain current and relevant
• Scalability: Parallel agent architecture enables simultaneous processing of complex data requests
• Democratized AI Development: Open-source availability enables smaller teams and startups to leverage enterprise-grade data capabilities
• Customization Potential: Organizations can adapt agents for industry-specific data collection needs

BigSet represents a significant shift in how organizations approach data infrastructure. By automating the historically labor-intensive process of dataset creation, the platform enables teams to redirect resources toward higher-value analytics and machine learning work. As artificial intelligence applications increasingly depend on high-quality, structured data, tools like BigSet that reduce friction in data preparation become essential infrastructure for competitive advantage in the AI economy.