Live Captions

Live Captions revolutionizes meeting accessibility through AI-powered live transcription. This comprehensive platform supports 140+ languages and integrates seamlessly with popular streaming tools. Our technical breakdown reveals how this simple three-step setup delivers accurate, real-time captions for both live events and recordings.

Product Overview

Real-time Captioning

Live Captions enables real-time accessibility during meetings and conferences through its automated speech-to-text functionality. The system processes audio and video streams in real-time, generating captions that display alongside the primary content. This feature supports both live events and recordings, making it versatile for various use cases. The service handles multiple languages simultaneously, ensuring that content remains accessible to all participants regardless of their linguistic background.

Multi-lingual Support

The platform offers comprehensive language support across almost 140 languages and dialects, including lesser-known options like Afrikaans (South Africa) and Amharic (Ethiopia). This extensive language coverage makes the service particularly valuable for international events or organizations with global audiences. Users can select from a wide range of language options when setting up their events, ensuring that captions are available in the preferred language of the participants.

Integration Process

Integrating the Live Captions service into existing workflows requires minimal technical expertise. The basic setup involves three straightforward steps:

1. Scheduling the event through the account management interface

2. Customizing and copying the necessary widgets to the website

3. Displaying captions and interactive transcripts in real-time

These simple procedures allow users to quickly implement the service without extensive configuration. For more advanced use cases, the platform provides a programmatic API that enables automated event management and more complex integrations.

Technical Setup

The service operates using standard RTMP publishing protocols, making it compatible with popular streaming software like OBS Studio and Zoom. Technical configuration focuses on setting up the publishing URL and specifying the event's time zone and language preferences. The platform includes several customization options to improve caption accuracy, including the ability to define specific words or names that may appear in the content. Users can also adjust the synchronization of captions using a "target delay" setting to ensure accurate time alignment with the audio stream.

Basic Usage and Integration

The basic integration process for Live Captions requires minimal technical expertise and consists of three straightforward steps:

1. Scheduling the Event: Users create an account and configure their time zone and language settings. During event creation, they specify details such as the event name, start date, time zone, language, publishing URL (using RTMP protocol or HTTPS for HLS), and optional transcript embedding on the website. The publishing URL is a crucial configuration step, as it determines how the system receives the audio/video stream data.

2. Customizing and Copying Widgets: Once the event details are set up, users customize and copy the necessary widgets to their website. These widgets enable real-time display of captions and interactive transcripts during the event.

3. Displaying Captions/Interactive Transcript: With the widgets in place, captions and interactive transcripts appear alongside the primary content during live events. For recorded media, the system generates WebVTT captions files that can be directly embedded into players.

The platform supports programmatic API methods for automated event management, including creating, updating, and deleting captions for both live and recorded events. The SaveEvent method handles new event creation and updates, requiring parameters such as event name, start date, publishing point URL, target delay, event time zone, event wait time, event maximum time, event language, user phrases, and silence timeout in milliseconds. The response format indicates whether the operation was successful and provides an event ID if applicable.

When working with recorded media, the SaveRecordedEvent method creates captions using similar parameters but focuses on media URL upload. This method returns the event ID and success message upon completion. The API also supports advanced operations like deleting events, retrieving live captions, and downloading transcripts in various formats including SRT, CSV, and WebVTT.

Scheduling and Configuration

The scheduling process for Live Captions operates through an account-based system that requires basic configuration for time zone and language settings. When creating an event, users have several key parameters to configure:

Event Details

Users define the basic event information during creation, including:

• Event name

• Start date

• Time zone

• Language

• Publishing URL, which supports both RTMP and HTTPS for HLS protocols

• Optional settings for embedding the transcript widget on the website

Configuration Options

The system offers tools for configuring captions and transcript display, with particular attention to synchronization. Key settings include:

• Target delay adjustment for caption timing

• Optional user phrase definitions to enhance recognition accuracy

• Custom event data wait and maximum times

Website Integration

The integration process focuses on straightforward web implementation:

• Users create and customize widgets for their website

• The widget code is placed beneath the video player to display captions in real-time

Programmatic Access

For more complex setups, the platform provides API methods to automate event management:

• SaveEvent method handles new and existing events, requiring detailed configuration parameters

• SaveRecordedEvent method processes media files with similar parameters

• Additional functions include event deletion, live caption retrieval, and transcript downloads in various formats

The comprehensive language support enables configuration of almost 140 languages, from widely spoken options like English and Spanish to more specialized dialects such as Afrikaans and Amharic. This setup ensures that the service can accommodate diverse linguistic needs while maintaining technical accessibility for users through its straightforward configuration options.

API Features

The platform provides two primary API methods for managing captions and transcript data:

Event Management

• SaveEvent Method: Creates or updates scheduled events for live captions

• Allows configuration of event name, start date, publishing URL, target delay, time zone, language, user phrases, and silence timeout

• Returns a boolean success status, event ID for new objects, and an optional success message

• SaveRecordedEvent Method: Creates captions for already recorded media files

• Requires media URL upload, event time zone, and language selection

• Returns the event ID and success message upon completion

Real-time Data Access

• GetLiveCaptions Method: Retrieves real-time captions data

• Accepts event ID and timestamp parameters

• Returns caption text, timestamp, and duration information

• GetTranscript Method: Downloads complete event transcript

• Provides session-specific transcript data for uninterrupted events

• Supports SRT (1), CSV (2), and WebVTT (3) download formats

Account and Status Information

• GetUserAccountInfo Method: Retrieves account data

• Includes plan type, available credits, and last top-up date

• GetEventStatus Method: Checks event processing status

• Returns status codes for event progress (IN_PROGRESS, COMPLETED, etc.)

• DownloadTranscript Method: Downloads complete transcript data

• Supports SRT (1), CSV (2), and WebVTT (3) format options

• Handles both live and recorded event transcripts

These API features enable seamless event management and enhanced accessibility options through programmatic integration. The comprehensive suite of methods supports all aspects of caption creation, retrieval, and management, making the platform flexible for both simple and complex use cases.

Technical Details and Setup

The technical foundation of Live Captions' operation centers on robust support for Real-Time Messaging Protocol (RTMP), enabling seamless integration with popular streaming applications including OBS Studio and the Zoom app. This protocol choice facilitates efficient audio/video stream processing and ensures compatibility across a broad spectrum of streaming solutions.

The system's language capabilities demonstrate remarkable versatility, supporting nearly 140 languages and dialects—ranging from widely spoken options like English, German, and French to more specialized choices such as Afrikaans (South Africa) and Amharic (Ethiopia). This comprehensive linguistic support is complemented by advanced automation features that enhance caption accuracy through multiple mechanisms.

One critical component of the automation process is the "target delay" setting, which allows users to fine-tune caption synchronization. This feature works in conjunction with the service's recognition technology to reduce latency between audio input and caption display, improving the overall user experience. The system's ability to adjust synchronization settings demonstrates its commitment to optimizing both performance and accessibility.

Programmatically, Live Captions exposes a powerful API that enables automated event management through several key methods. The SaveEvent method allows users to create or update scheduled events, requiring parameters that include the event name, start date, publishing URL (supporting RTMP or HTTPS for HLS protocols), target delay, event time zone, language selection, user-defined phrases, and silence timeout settings. This method returns a boolean success status, an event ID for new objects, and an optional success message, providing clear feedback on API interactions.

For managing recorded content, the SaveRecordedEvent method handles caption generation by processing media URL uploads while maintaining similar configuration parameters. Both methods support advanced operations including event deletion, real-time data retrieval, and comprehensive transcript downloads in various formats (SRT, CSV, WebVTT). This comprehensive suite of API features enables seamless event management and enhanced accessibility options through programmatic integration, supporting both simple and complex use cases.