Buildt

Buildt's AI-driven platform revolutionizes software development through sophisticated cost optimization and scalable engineering solutions. By fine-tuning smaller language models to match the performance of larger ones, the company achieves remarkable cost reductions while maintaining industry-leading capabilities. Through proprietary techniques in code indexing, memory management, and model training, Buildt continues to push the boundaries of AI-powered engineering. The platform's success stands out in its ability to handle enterprise-scale codebases while maintaining privacy and delivering human-like problem-solving through advanced logical reasoning.

Cost Optimization through Model Scaling

Buildt employs a sophisticated approach to reduce costs while maintaining model performance. The company uses OpenAI's foundational models for codebase search, understanding, and augmentation, but faces challenges with latency and cost. Larger models like davinci are both slow and expensive, while smaller models like babbage offer superior performance.

To address these issues, Buildt developed a solution that generates a moderate corpus of completions from davinci, then fine-tunes a model like babbage to perform the same task. This approach achieves nearly identical results or 90% similarity at 40x lower cost and 4-5x better latency. The company has successfully implemented this technique in two primary ways:

1. Fine-tuning babbage to identify code characteristics, using ChatGPT to create a basic web UI for human review and improvement of initial completions.

2. Achieving a 1% drop in completions per 10ms of latency reduction, as noted by Alex Graveley.

The effectiveness of this approach depends significantly on the number of examples provided, ranging from a few hundred for proseic output to multiple thousands for DSL tasks. Buildt's internal fine-tuning pipeline automates the process of adapting to changing output structures, enabling efficient model scaling.

Implementation and Scaling

The company's initial engineering focus was to create a vector index for codebases, but scaling demands introduced by enterprise customers required additional complexity. To manage enterprise-scale codebases while maintaining performance and privacy, Buildt developed a scalable vector indexing solution.

The company's initial approach used hnswlib-node, but architectural limitations led them to explore alternative solutions. After evaluating multiple options, they selected WebAssembly (WASM) as the most viable approach, providing both architecture agnosticism and compatibility with their existing project infrastructure.

Performance Optimization

To optimize performance across various codebases, the team implemented sophisticated chunking techniques and optimized backend handling of multiple concurrent HTTPS streams. These efforts have enabled the system to process extended duration streams without compromising performance.

Memory Management

A critical challenge emerged during implementation, as the indexing service experienced unexpected memory growth. Analysis revealed a memory leak in the tree-sitter input function, which was only detected after several months of operation. The leak was successfully addressed by adding a destructor to the relevant class, subsequently eliminating retained objects in subsequent runs.

Buildt continues to refine its platform, with upcoming developments focusing on improved AST traversal capabilities and language-agnostic implementation. The company also plans to expand private cloud functionality, allowing enterprises to maintain shared embedding repositories while maintaining data privacy.

AI-Driven Code Analysis and Engineering

Genie represents a significant advancement in AI software engineering, achieving a 30% success rate on industry-standard benchmarks while surpassing previous state-of-the-art models by 56% (Amazon's Q) and 118% (Cognition's Devin). The model's capabilities extend beyond code understanding to fully autonomous bug fixing, feature building, and code refactoring, demonstrating human-like problem-solving through logical reasoning derived from real developer behavior.

The underlying technology combines sophisticated data collection techniques with cutting-edge AI training methods, using OpenAI's foundational models as its base. Buildt has developed a proprietary dataset reconstruction process that forensically derives the decision-making patterns of human engineers from commit histories, pull requests, and issue tracking. This approach emphasizes the importance of high-quality data from experienced developers, leveraging real-world coding scenarios to train their models.

Key technical aspects of Genie's development include its ability to handle extended context lengths through advanced LLM processing and its generalization capabilities enabled by training on massive datasets. The company's long-term strategy involves creating a scalable infrastructure that allows for the creation of multiple model variants—smaller models for simpler tasks and larger models for complex problem-solving. This modular approach enables efficient performance tuning while maintaining operational costs below those of major competitors.

Buildt maintains a strategic approach to model transparency, choosing to keep internal workings confidential while publishing final outputs on GitHub for independent verification. This methodology has allowed them to maintain a significant performance edge in the AI engineering field, with plans to expand their model portfolio to support multiple programming languages and frameworks. Current capabilities focus on the latest and most widely used technologies, with ongoing improvements in step-by-step verification and self-improvement through data feedback loops.

Scalability Challenges and Solutions

The company's initial approach to indexing codebases proved insufficient when faced with the demands of enterprise-scale projects. The decision to store vector indexes on user devices for privacy and reduced latency created significant technical challenges, particularly with the performance limitations of consumer-grade hardware. Early attempts to scale the system using the hnswlib-node library demonstrated architectural shortcomings, particularly with inconsistent behavior across different machine architectures.

To address these issues, the team explored WebAssembly (WASM) as a viable alternative, motivated by lessons learned from the popular TV show "Mythbusters." The new approach leveraged WASM's architecture-agnostic nature and browser compatibility to create a more robust indexing solution. This change required significant engineering effort to adapt the existing vector indexing algorithms for web-based execution, but ultimately proved successful in supporting larger codebases while maintaining performance.

The indexing process itself faced multiple challenges related to memory management. Early implementations experienced persistent memory leaks in the tree-sitter input function, which required months of analysis to fully understand and resolve. The leak was eventually traced to the CallbackInput class in src/parser.cc, where adding a destructor successfully eliminated retained objects in subsequent runs.

As the system scaled to process larger codebases, the team implemented sophisticated chunking techniques to manage memory usage during code parsing. These improvements, combined with optimized backend handling of multiple concurrent HTTPS streams, enabled the system to process extended duration codebases without performance degradation. The project continues to evolve, with future development focused on creating a fully language-agnostic indexing solution and expanding support for private cloud deployment scenarios.

Memory Management and Optimization

The indexing service experiences a progressive memory growth issue that only becomes apparent under sustained operation. The root cause traces back to the input function within index.js from tree-sitter, where a steady accumulation of objects occurs. Analysis using the Cosine tool reveals significant object retention, particularly in the CallbackInput class located in src/parser.cc.

The memory leak mechanism operates through the parser's recursive descent implementation. When processing code input, the parser creates temporary structures to hold partial string data. These structures remain in memory even after their intended use, leading to a gradual accumulation of retained objects. The specific leak occurs in the tree-sitter/parser.cc file, where the CallbackInput class maintains references to both callback and partial_string without releasing them.

Building a comprehensive solution requires careful attention to service performance monitoring and proactive issue resolution. The Buildt team implemented targeted changes to reset these retained objects, adding a destructor to the CallbackInput class that explicitly clears both callback and partial_string references. This modification successfully eliminated the retained objects in subsequent runs, indicating a complete resolution of the memory leak issue.

The effectiveness of this approach highlights the importance of thorough testing and continuous monitoring in complex software systems. By implementing this fix, Buildt demonstrates their commitment to maintaining high standards of memory management and service reliability. The team's proactive response to a subtle memory leak underscores their dedication to optimizing both performance and developer experience within their indexing platform.