Simultaneous Interpretation

Start a long-running spoken-language interpretation session and receive incremental source text, translated text, optional TTS audio, events, and a final report.

See AI Session Events for the SimultaneousInterpretationEventType lifecycle used by onEvent.

Animated workflow

Interpretation session lifecycle

Coordinate provider startup, the active audio source, ordered incremental results, interruption handling, and final shutdown.

01Host app

H

Configure Languages

Set compatible source and target languages plus TTS and playback options.

02AI service

S

Start Session

Start the provider-backed simultaneous interpretation service.

03Host app

H

Retain Session

Store the returned session and inspect whether the AIBuds AI SDK records internally.

04Host app + device

D

Provide Audio

When SDK internal recording is disabled, feed external PCM to the session; device recording is one possible source.

05AI service → app

%

Stream Results

Order definite source and target segments and handle optional TTS audio.

incremental results

06Host app

H

Handle Runtime Events

Process events, recoverable exceptions, and interruption-driven stops.

07Device

D

Stop External Audio

If device recording is active, stop it before stopping interpretation.

08AI service

S

Stop Interpretation

Request shutdown of the current simultaneous interpretation session.

09Final callback

✓

Finish Session

Consume the optional report and clear the retained active session.

↳A recoverable onException callback does not automatically mean the session has stopped.

Prerequisites

• AIBudsAISDK is initialized and a registered provider is selected and authenticated.
• The provider supports SimultaneousInterpretationServiceAPI.
• Source and target language identifiers use a hyphenated format and are not the same.
• When AIBuds AI SDK internal recording is disabled, the host app provides external PCM. If that audio comes from the connected device, it conforms to DeviceAudioRecordingAPI.

Implement with AI Assistance

Build with AI

Implement this workflow with AI

Use the official Implement AIBuds Simultaneous Interpretation skill to adapt this workflow to your app.

Read and follow https://docs-aibuds.github.io/skills/implement-aibuds-simultaneous-interpretation. Use it to implement Implement AIBuds Simultaneous Interpretation in this iOS project and verify the result.

View official skill↗

API Reference

Framework

AIBudsAI.xcframework

Import

Swift

Swift

import AIBuds
import AIBudsAI
import AIBudsAIFoundation

Objective-C

Objective-C

#import <AIBuds/AIBuds-Swift.h>
#import <AIBudsAI/AIBudsAI-Swift.h>

Declaration

Swift

Swift

/// Starts a simultaneous interpretation session.
/// - Parameters:
///   - config: The session configuration.
///   - onStartSuccess: Called with the started session.
///   - onStartFailure: Called when the session cannot start.
///   - onStopByInterruption: Called when an interruption stops the session.
///   - onException: Called for a recoverable session exception. The app decides
///     whether the session should stop.
///   - streamResultHandler: Called with incremental interpretation results.
///   - onEvent: Called for session-level events.
///   - onFinish: Called with the optional final report.
public static func startSimultaneousInterpretation(_ config: SimultaneousInterpretationConfig = .default,
                                             onStartSuccess: ((_ session: SimultaneousInterpretationSessionConvertible) -> Void)? = nil,
                                             onStartFailure: ((_ error: NSError) -> Void)? = nil,
                                       onStopByInterruption: ((_ error: NSError?) -> Void)? = nil,
                                                onException: ((_ error: NSError) -> Void)? = nil,
                                        streamResultHandler: ((
                                            _ isFinal: Bool,
                                            _ response: SimultaneousInterpretationDataModel?,
                                            _ error: Error?
                                        ) -> Void)? = nil,
                                                    onEvent: ((_ event: SimultaneousInterpretationEventModel) -> Void)? = nil,
                                                   onFinish: ((_ report: SimultaneousInterpretationReportModel?) -> Void)? = nil)

/// Stops the current simultaneous interpretation session.
public static func stopSimultaneousInterpretation()

Objective-C

Objective-C

/// Starts a simultaneous interpretation session.
/// - Parameters:
///   - config: The session configuration.
///   - onStartSuccess: Called with the started session.
///   - onStartFailure: Called when the session cannot start.
///   - onStopByInterruption: Called when an interruption stops the session.
///   - onException: Called for a recoverable session exception.
///   - streamResultHandler: Called with incremental interpretation results.
///   - onEvent: Called for session-level events.
///   - onFinish: Called with the optional final report.
+ (void)startSimultaneousInterpretationWithConfig:(AIBudsSimultaneousInterpretationConfig * _Nonnull)config
                                   onStartSuccess:(void (^ _Nullable)(id <AIBudsSimultaneousInterpretationSessionConvertible> _Nonnull))onStartSuccess
                                   onStartFailure:(void (^ _Nullable)(NSError * _Nonnull))onStartFailure
                             onStopByInterruption:(void (^ _Nullable)(NSError * _Nullable))onStopByInterruption
                                      onException:(void (^ _Nullable)(NSError * _Nonnull))onException
                              streamResultHandler:(void (^ _Nullable)(BOOL, AIBudsSimultaneousInterpretationDataModel * _Nullable, NSError * _Nullable))streamResultHandler
                                          onEvent:(void (^ _Nullable)(AIBudsSimultaneousInterpretationEventModel * _Nonnull))onEvent
                                         onFinish:(void (^ _Nullable)(AIBudsSimultaneousInterpretationReportModel * _Nullable))onFinish;

/// Stops the current simultaneous interpretation session.
+ (void)stopSimultaneousInterpretation;

See startSimultaneousInterpretation and stopSimultaneousInterpretation.

Configuration

SimultaneousInterpretationConfig exposes:

Property	Default	Description
sourceLanguage	App language	Optional source language; an empty string enables auto-detection.
targetLanguage	en-US	Required target language.
enableTTS	true	Whether translated speech is synthesized.
enableVoicePlayback	true	Whether synthesized voice playback is enabled.
usesInternalAudioRecording	true	Whether the AIBuds AI SDK records audio for the session.
preferSpeakerOutput	false	Whether speaker output is preferred.

When usesInternalAudioRecording is false, the AIBuds AI SDK does not capture audio for the session. The host app must retain the returned session and supply external PCM through appendInt16PCM(_:isFinal:) or appendAudioPCMBuffer(_:isFinal:).

Usage Examples

Swift

Swift

let config = SimultaneousInterpretationConfig.default
config.sourceLanguage = "zh-CN"
config.targetLanguage = "en-US"
config.usesInternalAudioRecording = true
config.preferSpeakerOutput = false

AIBudsAISDK.startSimultaneousInterpretation(
    config,
    onStartSuccess: { session in
        currentSession = session
    },
    onStartFailure: { error in
        print("Unable to start: \(error.localizedDescription)")
    },
    onStopByInterruption: { error in
        print(error?.localizedDescription ?? "Session interrupted")
        currentSession = nil
    },
    onException: { error in
        print("Session exception: \(error.localizedDescription)")
    },
    streamResultHandler: { isFinal, response, error in
        if let error {
            print(error.localizedDescription)
            return
        }
        guard let response else { return }

        if response.isSourceTextDefinite {
            print("Source: \(response.sourceText ?? "")")
        }
        if response.isTargetTextDefinite {
            print("Target: \(response.targetText ?? "")")
        }
        if isFinal { print("Final result") }
    },
    onEvent: { event in
        print(event)
    },
    onFinish: { report in
        currentSession = nil
        print(report ?? "No report")
    }
)

Objective-C

Objective-C

AIBudsSimultaneousInterpretationConfig *config =
    [AIBudsSimultaneousInterpretationConfig defaultConfig];
config.sourceLanguage = @"zh-CN";
config.targetLanguage = @"en-US";
config.usesInternalAudioRecording = YES;
config.preferSpeakerOutput = NO;

[AIBudsAISDK startSimultaneousInterpretationWithConfig:config
    onStartSuccess:^(id<AIBudsSimultaneousInterpretationSessionConvertible> session) {
        self.currentSession = session;
    }
    onStartFailure:^(NSError *error) {
        NSLog(@"Unable to start: %@", error.localizedDescription);
    }
    onStopByInterruption:^(NSError *error) {
        NSLog(@"%@", error.localizedDescription ?: @"Session interrupted");
        self.currentSession = nil;
    }
    onException:^(NSError *error) {
        NSLog(@"Session exception: %@", error.localizedDescription);
    }
    streamResultHandler:^(
        BOOL isFinal, AIBudsSimultaneousInterpretationDataModel *response, NSError *error) {
        if (error != nil) {
            NSLog(@"%@", error.localizedDescription);
            return;
        }
        if (response.isSourceTextDefinite) {
            NSLog(@"Source: %@", response.sourceText ?: @"");
        }
        if (response.isTargetTextDefinite) {
            NSLog(@"Target: %@", response.targetText ?: @"");
        }
    }
    onEvent:^(AIBudsSimultaneousInterpretationEventModel *event) {
        NSLog(@"%@", event);
    }
    onFinish:^(AIBudsSimultaneousInterpretationReportModel *report) {
        self.currentSession = nil;
        NSLog(@"%@", report);
    }];

When session.isRecordingInternally is false, the host app owns the external audio path. The Demo starts device-side AI recording after onStartSuccess, forwards each decoded PCM batch to session.appendInt16PCM(_:isFinal:), stops device recording first, and then calls AIBudsAISDK.stopSimultaneousInterpretation().

Notes

• Retain the SimultaneousInterpretationSessionConvertible returned at startup to track the active session and its recording mode.
• onException does not necessarily stop the session. Decide whether to continue or call stopSimultaneousInterpretation().
• Use sourceTextSequence and targetTextSequence to order definite segments instead of blindly appending every incremental callback.
• TTS audio can be exposed as a relative file, full file path, or Base64 PCM data on SimultaneousInterpretationDataModel.