Whatthediff

What-the-Diff represents a significant advancement in software development workflows by leveraging AI-powered code review capabilities. This comprehensive exploration examines the tool's integration mechanisms, feature set, pricing model, and unique offering for open-source projects, providing insights into how teams can optimize their development processes through automated code analysis and review.

Integration and Access

Account Setup and Authorization

What-the-Diff requires separate accounts for integrating with GitHub and GitLab. After logging in, users must authorize access to specific organization repositories. These repositories must be manually enabled within the dashboard, allowing users to add more later if needed.

Event Triggers and Configuration

The service operates on two main modes: pull request description updates or AI-generated comments triggered by specific events. For pull request description updates, users can add shortcodes to their pull request descriptions such as wtd:summary, wtd:joke, or wtd:poem, which are automatically replaced with AI-generated content. Users retain control over processing status and have the option to make adjustments before submission.

Repository Settings and File Exclusion

Additional repository settings include a maximum token limit of 50,000 per PR, with options for customization through the "latest pull requests" menu. Users can implement granular control options for legacy triggers and exclude specific files based on patterns, with examples including build folders with minified JavaScript and Markdown documentation. The configuration supports wildcard exclusion for CI tools that create branches automatically and allows ignoring PR titles for WIP or draft requests requiring later summaries.

Notification Customization

Notifications can be sent to project Slack channels, Webhook endpoints, or product managers/designers. Summaries can be translated into non-technical language and different languages. The tool provides summary notifications using the pull request's summarized description and generates weekly reports combining all PR summaries into a single report, available in multiple languages including English and additional supported languages.

Feature Overview

What-the-Diff generates pull request descriptions through AI shortcodes and triggers. To utilize these features, users incorporate predefined shortcodes into their pull request descriptions, such as wtd:summary, wtd:joke, or wtd:poem. These shortcodes trigger AI-generated content that replaces the placeholders upon pull request submission. The AI uses the GitHub or GitLab API to analyze the pull request diff and generate descriptive comments in plain English or multiple supported languages.

The tool supports nearly all programming languages and processes diffs through token-based pricing. Average pull requests require approximately 2,300 tokens, with the service automatically detecting language syntax and providing explanations in either technical or non-technical terms. When users run out of tokens, they can upgrade to a larger plan or wait for their next billing cycle, as tokens do not roll over monthly. The service employs the GitHub and GitLab APIs exclusively for diff processing and does not store any user code or diffs, ensuring secure handling of sensitive information.

For automated code refactoring, users can employ the tool's inline comment capabilities. By prefixing desired changes with /wtd, users initiate an AI-driven refactoring process that suggests and accepts modifications with a single click. This feature streamlines code improvement while maintaining developer productivity. The system also generates changelogs from selected pull request descriptions, creating summarized change logs that can be shared internally or published externally as JSON feeds. These generated documents summarize changes in non-technical language, making them accessible to stakeholders without technical expertise.

Token-Based Usage

What-the-Diff processes pull request diffs using a token-based pricing system, with the average pull request requiring approximately 2,300 tokens. The service employs the GitHub and GitLab APIs to access pull request diffs and an AI model to generate change descriptions in plain English or supported languages.

The pricing structure offers two plans: a free tier limited to 25,000 tokens per month, and a Pro-200k tier providing 200,000 tokens per month with priority support. Both plans support nearly all programming languages and generate changelogs from pull request descriptions. When token limits are exceeded, users can upgrade their plan or wait for their next billing cycle, though unused tokens do not carry over monthly.

The company's token consumption model includes a default limit of 50,000 tokens per pull request, with the option to customize settings through the "latest pull requests" menu. Users can exclude specific files based on patterns, including build folders with minified JavaScript and Markdown documentation, using wildcard notation for directory inclusion. The system also supports automatic exclusion of CI-generated branches and can be configured to ignore WIP or draft PR titles.

All interactions with code are conducted through the GitHub and GitLab APIs, with no storage of user code or diffs retained by the service. The AI model receives full program access to read pull request diffs, as current API permissions do not allow selective diff reading.

Open Source Integration

Beyond Code collaborates with What-the-Diff to provide free plans specifically for open source projects. Projects must meet the following criteria:

• Be in active development with regular commits in the last three months

• Not provide commercial versions or services

• Not be founded by a company

• Not pay core developers

• Have a track record of at least 10 contributors or 100 GitHub stars

The free account corresponds to the requested open source repository, offering full access to core features without requiring a credit card. All paid features, including advanced reporting and priority support, remain available through this sponsorship model.

The open source integration enables automatic processing of pull requests using the same token-based pricing structure as commercial accounts. While the free tier allows 25,000 tokens per month - approximately ten full pull requests - projects can scale up to the Pro-200k plan ($19/month) for intensive development environments.

Usage and Configuration

The default maximum token limit per pull request is set to 50,000, though users can customize this through the "latest pull requests" menu. This basic configuration allows for significant flexibility in managing token usage across different projects and repository sizes.

File exclusion provides a powerful layer of customization, enabling precise control over which changes are processed. Users can exclude specific files based on pattern matching, including build folders with minified JavaScript and Markdown documentation, using wildcard (*) notation for directory inclusion. This feature proves particularly valuable for managing large codebases with distinct build structures or documentation repositories.

Branch management receives special attention through flexible exclusion rules. The system supports wildcard patterns for automatically generated branches by CI tools, helping projects maintain clean token consumption even as development workflows evolve. Additionally, PR titles can be configured to trigger summary generation selectively, allowing developers to distinguish between work-in-progress (WIP) or draft requests that may not require immediate summary processing.

Notification customizations extend beyond basic alerts, offering nuanced control over how feedback reaches team members. Users can direct notifications to project Slack channels, custom Webhook endpoints, or specific team members designated as product managers or designers. The system supports translating summaries into both technical and non-technical language, with options for multiple supported languages. Developers appreciate the ability to generate weekly reports combining all PR summaries into comprehensive documents, available in their preferred languages.