Defog

Defog is an AI-powered platform that enables users to query relational databases using natural language, bridging the gap between technical databases and non-technical users who need data-driven insights. This comprehensive guide walks you through setting up and deploying Defog, integrating it with your databases, and leveraging its advanced features for enterprise-level data access and analysis.

Quick Start and API Setup

To begin using Defog, users need to sign up for an API key at https://defog.ai/signup. The free tier allows querying up to 5 tables with 25 total columns and 1,000 queries per month. For larger database requirements, users can upgrade to a paid plan.

The environment variable configuration file (.env) requires two variables: DEFOG_API_KEY and DEFOG_API_KEY_NAMES. The first variable accepts a comma-separated list of API keys, while the second variable contains human-readable names for these keys, separated by commas. This setup enables multi-database querying through a single platform interface.

Defog deployment requires Docker and Docker Compose, which users can install from https://docs.docker.com/get-docker/. The official Docker repository can be cloned with git clone https://github.com/defog-ai/defog-docker, followed by a directory change to cd defog-docker.

The primary command to start the application is docker compose up. For security, the platform comes with default admin credentials of "admin" for both username and password.

The enterprise plan offers both cloud-hosted and self-hosted options. The cloud-hosted plan, starting at $5,000/month, provides 20,000+ queries per month, support for 200 queries per minute, and access to features like SSO authentication, custom AI tool development, and priority support. Self-hosted deployment requires a multi-year commitment and allows unlimted queries with no rate limits.

Database integration involves setting up credentials through the Manage Database section and extracting metadata using the Extract Metadata tab. After adding database details, users select tables to query, extract their metadata, and review column descriptions. Proper schema definition significantly reduces query errors, with the system guaranteeing over 90% accuracy when descriptions are well-defined. Additional guidance is available through the cookbook documentation at https://defog.notion.site/Cookbook-for-schema-definitions-1650a6855ea447fdb0be75d39975571b.

Basic Operations and Workflow

To begin using Defog, users first need to clone the official Docker repository from GitHub:

git clone https://github.com/defog-ai/defog-docker

cd defog-docker

Next, ensure Docker and Docker Compose are installed. Installation instructions can be found here: https://docs.docker.com/get-docker/. The platform installation process is initiated by running:

docker compose up

Users access Defog via their web browser, where they'll find a default admin login of "admin" with "admin" as both the username and password.

The basic workflow involves formulating queries in plain language. The system's AI engine processes these natural language requests and retrieves the corresponding data. The platform visualizes results in a format that matches the users' query requirements.

For database setup, users begin by adding their credentials through the Manage Database section of the top navbar. This requires specifying detailed connection parameters.

Once credentials are set, users navigate to the Extract Metadata tab. Here, they select tables for querying from a dropdown menu before clicking Extract Table Metadata to begin the metadata extraction process.

After extraction, users review the metadata. Proper column description, obtained either through automatic extraction or manual input from the provided cookbook documentation, significantly improves query accuracy. The system guarantees over 90% accuracy when schemas are well-defined.

To integrate SSO authentication, users can utilize the platform's enterprise features, though setup requires a separate configuration process not covered in the basic installation guide.

Pricing and Service Plans

The enterprise plan offers two deployment options: cloud-hosted and self-hosted. The cloud-hosted version starts at $5,000 per month and provides 20,000+ queries per month, supporting 200 queries per minute. This tier includes features like SSO authentication integration, custom AI tool development, personalized onboarding and training, priority support, and various service level agreements including MSA (Master Service Agreement) and DPA (Data Processing Agreement).

For organizations preferring full control over their infrastructure, the self-hosted option requires a multi-year commitment. This deployment model allows hosting Defog models on their own infrastructure, supporting unlimited queries with no rate limits. The available model sizes include 8b, 14b, and 32b configurations.

Both deployment models offer comprehensive support for production environments, with detailed docker deployment instructions provided in official documentation. Users needing to scale beyond basic capabilities can upgrade from the free tier, which limits queries to 5 tables with 25 total columns and grants 1,000 queries per month.

Database Integration and Metadata Management

The database integration process begins with adding credentials via the Manage Database section in the platform's top navbar. Users must provide specific connection parameters for their database, though the exact requirements depend on their system configuration.

After credentials are established, users navigate to the Extract Metadata tab and select the tables they wish to include in their queries from a dropdown menu. By clicking Extract Table Metadata, Defog initiates the metadata extraction process, which populates the system with essential information about each selected table's structure.

For optimal query accuracy, users should review and properly define column descriptions. While the system attempts to automatically extract this information, manual input based on the provided cookbook documentation (https://defog.notion.site/Cookbook-for-schema-definitions-1650a6855ea447fdb0be75d39975571b) ensures the highest level of query precision. The platform guarantees 90% or greater reduction in query errors when schemas are well-defined.

The metadata review process enables users to verify and refine the extracted information, making it particularly crucial for complex or legacy database structures. Properly defined column descriptions significantly enhance query accuracy, with the system achieving over 90% precision when schemas are thoroughly documented.

Advanced Features and Customization

Defog offers advanced features for enterprises needing to integrate AI data analysis into their workflows. One key capability is single-sign-on (SSO) authentication integration, allowing organizations to manage user access through their existing security frameworks. While the basic installation guide does not cover SSO setup, the platform's documentation outlines the additional configuration steps required for this advanced feature.

For teams building custom AI applications, Defog provides development tools to integrate directly with their workflows. This includes access to the underlying AI models and APIs, though developers will need to work within the platform's architecture to implement custom solutions.

The platform's infrastructure offerings give organizations flexibility in how they consume AI services. The self-hosted option, available through a multi-year commitment, allows customers to run Defog models on their own infrastructure with no rate limits on query volume. This deployment model is particularly useful for businesses with strict data sovereignty requirements or those needing to scale beyond the capacity of the cloud-hosted option.

Pricing for both deployment models includes a range of service level agreements (SLAs) to match different business needs. The cloud-hosted plan offers basic SLA terms with standard support response times, while premium options include master service agreements (MSAs) and data processing agreements (DPAs) for businesses requiring additional legal and operational protections. These advanced SLA options come with increased service costs but provide enhanced support and legal frameworks for enterprise deployments.