MediumGPT

In the evolving landscape of content consumption, platforms like Medium continue to transform how we read, learn, and engage with information. As these platforms expand their functionality, developers face increasing challenges in integrating diverse services while maintaining performance and reliability. This article explores the development of a MediumGPT plugin that bridges Medium's internal systems with the external APIs of platforms like ChatGPT. Through detailed examination of API design, performance optimization, and database management, we uncover the technical complexities of building scalable web applications and the innovative solutions developed to overcome them.

Plugin Development and Integration

The MediumGPT plugin architecture centers on middleware that translates internal GraphQL API operations into standard REST API calls compatible with ChatGPT. The core API structure is defined using OpenAPI 3.0.3, exposing four primary endpoints for interacting with Medium content:

1. Trending Stories - Retrieves a list of current trending stories on Medium, returning an array of objects containing story ID and title information.

2. Post Content Queries - Fetches full content of specific posts, with functionality to handle paywall-protected content.

3. Topic Queries - Fetches trending or latest posts within specified topics, along with related topic information.

4. Content Search - Enables searching for posts based on specific queries, utilizing the defined API structure.

During development, the plugin team encountered significant performance challenges during scaling tests. Testing revealed that while SQLAlchemy's ORM handles batch insert operations efficiently (achieving 3.0 seconds for 100,000 rows), auto-incrementing primary keys led to disproportionately high latency when scaling beyond 100 sequences with multiple annotations.

The team developed a specialized solution called sqlalchemy_batch_inserts, which optimizes performance through two primary mechanisms:

1. Primary Key Pre-Generation - Utilizes Postgres sequences (like sequence_id_seq) to pre-generate integer IDs before database insert operations, reducing the number of required round trips from N to 2 per table.

2. Batch Insert Optimization - Restructures model insertion operations to batch insert records of the same type together, further reducing database round trips from 4 to 2 per table type.

Implementation of these optimizations requires minimal code changes, adding a single line to the SQLAlchemy setup configuration. Deployment of these improvements yielded significant performance gains, with real-world testing demonstrating a reduction from 4.9 seconds per 1,000 rows at 3 ms latency to just 0.14 seconds, nearly matching the performance of native PostgreSQL insert operations.

OpenAPI Specification and API Design

The MediumGPT plugin implementation relies on a detailed OpenAPI specification that defines the API structure for integrating with ChatGPT. The plugin employs a middleware approach, converting internal GraphQL API operations into standard REST API calls that ChatGPT can understand.

The plugin architecture centers on exposing four primary API endpoints for interaction with Medium content:

1. Trending Stories - This endpoint fetches a list of current trending stories on Medium, returning an array of objects containing story ID and title information. The implementation uses GraphQL internally but presents a REST API interface to ChatGPT.

2. Post Content Queries - This endpoint retrieves the full content of specific posts, with functionality to handle paywall-protected content. The implementation ensures that stories behind paywalls are correctly identified and displayed to the user.

3. Topic Queries - This endpoint fetches trending or latest posts within specified topics, along with related topic information. The implementation structure is defined in OpenAPI 3.0.3 format, providing a clear framework for API interaction.

4. Content Search - This endpoint enables searching for posts based on specific queries. The implementation demonstrates the plugin's ability to chain multiple API calls to produce comprehensive results, even when handling paywall-protected content.

The plugin serves as a bridge between Medium's internal systems and ChatGPT's API requirements. During development, the team faced challenges with performance during scaling tests, particularly with database operations involving large numbers of rows.

To address these performance issues, the team developed specialized solutions for database optimization. The primary challenges stemmed from SQLAlchemy's ORM behavior when handling auto-incrementing primary keys, which led to disproportionately high latency during scaling.

The optimized solution, implemented through the sqlalchemy_batch_inserts module, addresses these issues through two primary mechanisms:

1. Primary Key Pre-Generation - This approach utilizes Postgres sequences to pre-generate integer IDs before database insertion operations, reducing the number of required round trips from N to 2 per table.

2. Batch Insert Optimization - The solution restructures model insertion operations to batch insert records of the same type together, further reducing database round trips from 4 to 2 per table type.

Deployment of these improvements yielded significant performance gains. Real-world testing demonstrated a reduction from 4.9 seconds per 1,000 rows at 3 ms latency to just 0.14 seconds, nearly matching the performance of native PostgreSQL insert operations.

Performance Optimization

The team developed a specialized solution to address these performance challenges through the creation of a specialized sqlalchemy_batch_inserts module. This module optimizes database insert operations by addressing two primary areas: primary key generation and batch insert optimization.

Primary Key Generation

SQLAlchemy's ORM batches inserts when primary keys are defined. The implementation utilizes Postgres sequences to pre-generate integer IDs for models before database insertion operations. This approach reduces the number of required round trips from N to 2 per table. The process involves 4 round trips: 1 to generate IDs for each table and 1 to batch insert for each table. While the theoretical speedup is 50x (from 200 to 4 round trips), actual performance achieved a 10x improvement.

Batch Insert Optimization

The module restructures model insertion operations to batch records of the same type together, reducing database round trips from 4 to 2 per table type. This optimization works by pre-filling IDs for new models of the same type and reordering models to insert those of the same type together. The implementation abstracts these changes into a SQLAlchemy hook that runs before the commit process. The solution handles joined table inheritance and ensures models are inserted in the order they were created in the session.

Performance Testing

Real-world testing demonstrated significant performance improvements. For inserting 100,000 rows, the baseline performance using SQLAlchemy ORM with auto-incrementing primary keys was 7.3 seconds. Enabling the batch mode with the sqlalchemy_batch_inserts module reduced this to 0.14 seconds for 1,000 rows, which is nearly matching the performance of native PostgreSQL insert operations. The tests were conducted with 3 ms latency and transatlantic AWS latency of 90 ms, showing consistent performance improvements across different batch sizes and latency conditions.