Quivr

This technical deep dive explores Quivr's innovative RAG (Retrieval-Augmented Generation) technology and its practical applications. From parsing complex documents to managing enterprise-scale knowledge bases, Quivr combines advanced natural language processing with robust system architecture. We'll examine how this open-source platform balances community-driven development with enterprise-level functionality, delivering fast and accurate search results through reliable system integration.

Quivr's Technical Foundation

Megaparse forms the foundation for extracting structured data from various file formats, with the current focus on optimizing parsing capabilities for PDF, Docx, Pptx, Text, and EPub files. The medium-term roadmap includes advancing table parsing accuracy and delivering more structured output formats to achieve state-of-the-art performance levels.

Quivr Core specializes in knowledge retrieval efficiency, implementing a single retrieval algorithm initially before expanding to multiple tailored algorithms. This system ensures the extraction of contextually relevant data from extensive datasets, forming a key component of their RAG technology framework.

Le Juge serves as the evaluation framework for Quivr's RAG system, starting with internal evaluation before scaling to assess all RAG systems across the platform. This structured approach enables continuous improvement in both retrieval and generation capabilities, maintaining high standards for result quality and relevance.

The technical foundation supports a clear separation between open-source development and enterprise innovation. While the deep tech components remain open-source through three dedicated repositories, Quivr Enterprise develops business-specific features built upon these technological foundations. This strategic approach balances community-driven development with enterprise-level functionality, as demonstrated by the platform's proven capabilities in delivering fast, accurate search results through reliable system integration.

RAG System Components

Parsing is the foundational capability that enables Quivr to extract structured data from various file formats, with the current focus on optimizing PDF, Docx, Pptx, Text, and EPub files. The parsing engine identifies key elements, important relationships, and concepts, preparing data for retrieval and evaluation. The technical roadmap emphasizes improving table parsing accuracy and delivering more structured output formats to achieve state-of-the-art performance levels.

The retrieval system, implemented through Quivr Core, focuses on extracting contextually relevant data from vast datasets. The platform initially deploys a single retrieval algorithm, with plans to develop multiple tailored algorithms in the medium term. This approach ensures that the system can efficiently locate relevant information from large knowledge bases or corpora, maintaining high standards for result quality and relevance.

The evaluation framework, Le Juge, is responsible for filtering the retrieved data to ensure quality and relevance. The evaluation process measures results based on predefined criteria, with initial focus on Quivr's internal systems before expanding to assess all RAG systems across the platform. This structured approach enables continuous improvement in both retrieval and generation capabilities, maintaining the platform's commitment to delivering fast, accurate search results through reliable system integration.

Platform Architecture and Implementation

The platform architecture maintains a clear separation between open-source development and enterprise innovation, as demonstrated by the separation of Megaparse, Quivr Core, and Le Juge into dedicated repositories. This strategic division enables simultaneous improvements in the underlying technology while developing enterprise-specific features through Quivr Enterprise.

Quivr offers a choice between self-hosted and cloud deployment options, with the platform designed for rapid deployment in minutes. The open-source version requires a commercial license for some production features, while the Python package remains free for open-source projects with unlimited brain capacity.

Three paid plans cater to various business needs: the Professional Cloud plan at $15/month annually provides 20 AI Assistants and professional cloud support. The Enterprise plan, starting at $384/month, offers unlimited users, personalization, and premium support. Basic and Business plans range from $96 to $192 per month, supporting small to growing teams with up to 20 users and 40GB of data per user.

The platform architecture includes robust security features, with data stored in Europe and accessed through dedicated servers. Message credits auto-renew monthly, and the system automatically handles failover instances. Compliance requirements are met through GDPR, SSO (Google), SAML (SSO), SOC2, HIPAA, and HDS standards, with support via Discord community, priority email/chat, and dedicated Slack channels.

Knowledge management functionality combines cloud integration with local data processing. The platform supports multiple remote file sources including Google Drive, Dropbox, and SharePoint, with all sync operations conducted read-only from Quivr's side. Local files are stored in Quivr while maintaining a single view of local and remote files, with immediate knowledge hints displayed after syncing.

The system processes knowledge asynchronously using nested folder structure and parallel Workers for efficient file handling. Recursive processing ensures linked folders maintain correct brain associations, while error recovery allows instant relinking of processed knowledge. The unified knowledge management system updates local files every 8 hours using Redis caching with 1-minute TTL to maintain current knowledge status.

Brain System Integration

The Quivr platform's Brain system integrates with multiple data sources, including Google Drive, Dropbox, and SharePoint. The system supports remote file integration and URL connections, allowing users to customize their knowledge environment with specific prompts and settings, including maximum token usage.

Brain functions include collaborative features such as shared access between users and brain customization options. The system utilizes advanced AI technologies like GPT and Mistral for conversational interactions, enabling users to engage directly with AI models including GPT-4 and Mistral for data-driven responses.

The platform's Brain component processes knowledge through asynchronous operations using a nested folder structure and parallel Worker implementation. It maintains a unified knowledge management system that updates local files every 8 hours via Redis caching with a 1-minute Time-To-Live (TTL) setting. The system supports advanced features including drag-and-drop file management, recursive folder processing, and real-time error recovery mechanisms.

The Brain system also facilitates comprehensive data integration across multiple sources. It enables users to manage both local and remote files through a unified interface that shows immediate knowledge hints upon syncing. The platform implements strict security measures, with all data stored in European servers and accessed through dedicated infrastructure to ensure compliance with GDPR, SSO (Google), and SAML standards.

Knowledge Management Features

The unified Knowledge Management System (KMS) allows users to maintain a single view of both local and remote files across multiple sources, including Google Drive, Dropbox, and Sharepoint. The system processes uploads asynchronously, using a nested folder structure and parallel processing Workers for efficient handling.

Local files are stored within the Quivr platform, with sync operations conducted read-only from Quivr's side. The system fetches the latest state of synced files every 8 hours via Redis caching with a 1-minute Time-To-Live (TTL) setting. Users can manage files through drag-and-drop functionality, retain folder brains when moving knowledge between locations, and instantly relink processed knowledge during error recovery.

The Brain system supports collaborative features with shared access between users and customized brain settings, including maximum token usage limits. Each Brain can be connected through the Knowledge Management System's connect button or bulk connect feature, with processing that occurs asynchronously when adding knowledge to a brain. Success or error status is shown for each knowledge addition, allowing users to retry failed processes through unlink and relink functionality.

Coming soon features include keyword search capabilities across knowledge sources and filtered search options within the system. Future enhancements will also incorporate tagging functionality for improved folder/file organization and enhanced discoverability.