AI Chat

Run an AI voice chat session initiated by a supported device, receive conversational data and intents, and route decoded PCM audio to the active provider session when required.

Use AI Session Events to interpret AIChatEventType and AI Intents to validate recognized actions before executing them.

Animated workflow

Device-driven AI chat lifecycle

The device requests an AI chat and declares whether speech will arrive through Opus or an app-side SCO recording. Keep the device and AI session states coordinated until the final report arrives.

01Device → app

D

Receive Device Request

The device requests a new AI chat and specifies SCO or Opus as the speech-input channel.

02Host app

H

Prepare Requested Audio Input

Use the channel declared by the device: receive Opus packets from the device, or prepare app-side SCO recording.

03AIBudsAISDK

S

Start AI Chat

Start the provider-backed session with the prepared configuration.

04App ↔ device

✓

Confirm Startup

Retain the session and report start success or failure to an Opus device when required.

05Device → app → session

%

Capture or Forward Speech

SCO captures the device microphone through the app; Opus is sent by the device, decoded, and appended to the retained session.

live audio

06AI service → app

%

Consume AI Results

Handle chat data, intents, voice data, VAD, and session events.

streaming callbacks

07App + audio output

S

Deliver Response

Render text and allow configured AI voice playback over the selected output path.

08App ↔ device

D

Coordinate Stop

Handle device terminate, state conflict, auto-end, explicit stop, or runtime failure.

09Final callback

✓

Finish and Release

Consume the final report, clear the retained session, and restore idle UI state.

↳On startup failure, report failure to an Opus device when required. On terminate, state conflict, auto-end, or an unrecoverable error, stop the active session and release it after the terminal callback.

Prerequisites

• AIBudsAISDK is initialized, device information is configured, and a registered provider is selected and authenticated.
• The provider supports AIChatServiceAPI.
• The device delivers AI chat session events and, for Opus input, conforms to DeviceAIChatAPI.
• Read the requested channel from the device event instead of choosing it independently in the app.

Implement with AI Assistance

Build with AI

Implement this workflow with AI

Use the official Implement AIBuds AI Chat skill to adapt this workflow to your app.

Read and follow https://docs-aibuds.github.io/skills/implement-aibuds-ai-chat. Use it to implement Implement AIBuds AI Chat in this iOS project and verify the result.

View official skill↗

API Reference

Framework

AIBudsAI.xcframework

Import

Swift

Swift

import AIBuds
import AIBudsAI
import AIBudsAIFoundation

Objective-C

Objective-C

#import <AIBuds/AIBuds-Swift.h>
#import <AIBudsAI/AIBudsAI-Swift.h>

Declaration

Swift

Swift

/// Starts an AI chat session.
/// - Parameters:
///   - config: The chat session configuration.
///   - onStartSuccess: Called with the active session.
///   - onStartFailure: Called when the session cannot start.
///   - onChatData: Called when new conversational data arrives.
///   - onIntent: Called when the provider detects an intent.
///   - onVoiceData: Called when voice data is produced.
///   - onEvent: Called for session-level events.
///   - onError: Called for runtime session errors.
///   - onFinish: Called with the final session report.
public static func startAIChat(_ config: AIChatSessionConfig = .default,
                         onStartSuccess: ((_ session: AIChatSessionConvertible) -> Void)? = nil,
                         onStartFailure: ((_ error: Error) -> Void)? = nil,
                             onChatData: ((_ chatData: AIChatDataModel) -> Void)? = nil,
                               onIntent: ((_ intent: AIChatIntentModel) -> Void)? = nil,
                            onVoiceData: ((_ voiceData: AIChatVoiceDataModel) -> Void)? = nil,
                                onEvent: ((_ event: AIChatEventModel) -> Void)? = nil,
                                onError: ((_ error: NSError) -> Void)? = nil,
                               onFinish: ((_ report: AIChatSessionReportModel) -> Void)? = nil) -> Void

/// Stops the active AI chat session. This is safe when no session is active.
public static func stopAIChat()

Objective-C

Objective-C

/// Starts an AI chat session.
/// - Parameters:
///   - config: The chat session configuration.
///   - onStartSuccess: Called with the active session.
///   - onStartFailure: Called when the session cannot start.
///   - onChatData: Called when new conversational data arrives.
///   - onIntent: Called when the provider detects an intent.
///   - onVoiceData: Called when voice data is produced.
///   - onEvent: Called for session-level events.
///   - onError: Called for runtime session errors.
///   - onFinish: Called with the final session report.
+ (void)startAIChatWithConfig:(AIBudsAIChatSessionConfig * _Nonnull)config
               onStartSuccess:(void (^ _Nullable)(id <AIBudsAIChatSessionConvertible> _Nonnull))onStartSuccess
               onStartFailure:(void (^ _Nullable)(NSError * _Nonnull))onStartFailure
                   onChatData:(void (^ _Nullable)(AIBudsAIChatDataModel * _Nonnull))onChatData
                     onIntent:(void (^ _Nullable)(AIBudsAIChatIntentModel * _Nonnull))onIntent
                  onVoiceData:(void (^ _Nullable)(AIBudsAIChatVoiceDataModel * _Nonnull))onVoiceData
                      onEvent:(void (^ _Nullable)(AIBudsAIChatEventModel * _Nonnull))onEvent
                      onError:(void (^ _Nullable)(NSError * _Nonnull))onError
                     onFinish:(void (^ _Nullable)(AIBudsAIChatSessionReportModel * _Nonnull))onFinish;

/// Stops the active AI chat session. This is safe when no session is active.
+ (void)stopAIChat;

See startAIChat and stopAIChat.

Configuration

AIChatSessionConfig exposes the settings used by AIChatSettingsController:

Property	Default	Purpose
languageForSpeechInput	App language	Hyphenated speech-input language supported by the selected provider, for example zh-CN.
audioChannel	.opusInA2dpOut	Audio transport used by this chat session. It must match the device event that starts the session.
allowUserToInterruptAIResponse	true	Whether user input may interrupt an AI response, normally voice playback.
maxPauseDurationBeforeAIResponds	0.8 seconds	Maximum permitted speech pause before the AI responds.
autoEndSessionAfterNoInputDuration	15.0 seconds	Idle duration before the session ends automatically.
enableVoicePlayback	true	Whether generated voice playback is enabled.
shouldSaveVoiceForDebugging	false	Whether diagnostic voice data is retained. Keep disabled in production unless policy explicitly permits it.
additionalOptions	[:]	Provider-specific agent, speaker, plan, prompt, intent, or denoising options.
autoSelectAgentIfNotSpecified	true	Whether the SDK selects an agent when no provider-specific agent is supplied.

Use the public AdditionalOptionKey... constants rather than hard-coded provider option keys.

Configure a Chat Session

The Demo keeps provider selection separate from the session configuration. When the user changes provider, select it first, query its supported languages, and complete app-initiated authentication before starting chat. Agent IDs, speaker IDs, intent codes, usage plans, and initial prompts are issued by the provider and must not be copied from the Demo as universal values.

Swift

Swift

let vendor: AIServiceVendor = .starBurst
AIBudsAISDK.setAIServiceVendor(vendor)

let supportedLanguages = AIBudsAISDK.allSupportedLanguages(for: vendor)
let language = supportedLanguages.first?.languageCode

var options: [String: Any] = [:]
if let agentID = providerAgentID {
    options[AIChatSessionConfig.AdditionalOptionKeyStarburstAgentId] = agentID
}
if let speakerID = providerSpeakerID {
    options[AIChatSessionConfig.AdditionalOptionKeyStarburstSpeakerId] = speakerID
}

let config = AIChatSessionConfig(
    languageForSpeechInput: language,
    audioChannel: .opusInA2dpOut,
    allowUserToInterruptAIResponse: true,
    maxPauseDurationBeforeAIResponds: 0.8,
    autoEndSessionAfterNoInputDuration: 15,
    enableVoicePlayback: true,
    shouldSaveVoiceForDebugging: false,
    additionalOptions: options
)
config.autoSelectAgentIfNotSpecified = providerAgentID == nil

Objective-C

Objective-C

AIBudsAIServiceVendor vendor = AIBudsAIServiceVendorStarBurst;
[AIBudsAISDK setAIServiceVendor:vendor];

NSArray<AIBudsAIServiceLanguage *> *supportedLanguages =
    [AIBudsAISDK allSupportedLanguagesForVendor:vendor];
NSString *language = supportedLanguages.firstObject.languageCode;

NSMutableDictionary<NSString *, id> *options = [NSMutableDictionary dictionary];
if (self.providerAgentID.length > 0) {
    options[AIBudsAIChatSessionConfig.AdditionalOptionKeyStarburstAgentId] = self.providerAgentID;
}
if (self.providerSpeakerID.length > 0) {
    options[AIBudsAIChatSessionConfig.AdditionalOptionKeyStarburstSpeakerId] =
        self.providerSpeakerID;
}

AIBudsAIChatSessionConfig *config = [[AIBudsAIChatSessionConfig alloc]
        initWithLanguageForSpeechInput:language
                          audioChannel:AIBudsAIChatAudioChannelOpusInA2dpOut
        allowUserToInterruptAIResponse:YES
      maxPauseDurationBeforeAIResponds:0.8
    autoEndSessionAfterNoInputDuration:15.0
                   enableVoicePlayback:YES
           shouldSaveVoiceForDebugging:NO
                     additionalOptions:options];
config.autoSelectAgentIfNotSpecified = self.providerAgentID.length == 0;

For .mltcloud, use the corresponding AdditionalOptionKeyMltCloud... constants. Only set a StarBurst usage plan or provider-specific identifier when your provider configuration supplies a valid value.

Usage Examples

The device normally initiates a chat with .initiateWithSCO or .initiateWithOpus. Copy that requested channel into the session configuration before starting AI chat. For Opus, forward the decoded PCM received from the device. For SCO, starting the .sco session establishes app-side SCO recording for the device microphone.

Swift

Swift

let config = AIChatSessionConfig.default
config.audioChannel = .opusInA2dpOut
config.languageForSpeechInput = "en-US"

AIBudsAISDK.startAIChat(
    config,
    onStartSuccess: { session in
        currentSession = session
    },
    onStartFailure: { error in
        print("Chat start failed: \(error)")
    },
    onChatData: { chatData in
        conversation.append(chatData)
    },
    onIntent: { intent in
        handle(intent)
    },
    onVoiceData: { voiceData in
        handle(voiceData)
    },
    onEvent: { event in
        handle(event)
    },
    onError: { error in
        print(error.localizedDescription)
    },
    onFinish: { report in
        currentSession = nil
        save(report)
    }
)

When the device supplies decoded 16-bit PCM for an Opus chat session, forward it to the retained session:

Swift

currentSession?.appendInt16PCM?(decodedPCMData)

Objective-C

Objective-C

AIBudsAIChatSessionConfig *config = [AIBudsAIChatSessionConfig defaultConfig];
config.audioChannel = AIBudsAIChatAudioChannelOpusInA2dpOut;
config.languageForSpeechInput = @"en-US";

[AIBudsAISDK startAIChatWithConfig:config
    onStartSuccess:^(id<AIBudsAIChatSessionConvertible> session) {
        self.currentSession = session;
    }
    onStartFailure:^(NSError *error) {
        NSLog(@"Chat start failed: %@", error);
    }
    onChatData:^(AIBudsAIChatDataModel *chatData) {
        [self recordChatData:chatData];
    }
    onIntent:^(AIBudsAIChatIntentModel *intent) {
        [self handleIntent:intent];
    }
    onVoiceData:^(AIBudsAIChatVoiceDataModel *voiceData) {
        [self handleVoiceData:voiceData];
    }
    onEvent:^(AIBudsAIChatEventModel *event) {
        [self handleEvent:event];
    }
    onError:^(NSError *error) {
        NSLog(@"%@", error.localizedDescription);
    }
    onFinish:^(AIBudsAIChatSessionReportModel *report) {
        self.currentSession = nil;
        [self saveReport:report];
    }];

Forward decoded PCM when required:

Objective-C

[self.currentSession appendInt16PCM:decodedPCMData];

Stop the Session

When the device requests termination, report the stopped state through DeviceAIChatAPI when required by the device flow, then call:

Swift

Swift

AIBudsAISDK.stopAIChat()
currentSession = nil

Objective-C

Objective-C

[AIBudsAISDK stopAIChat];
self.currentSession = nil;

Notes

• This is an audio-session API, not a text sendMessage / message-history API.
• Retain AIChatSessionConvertible until onFinish, an unrecoverable error, or explicit stop.
• For SCO sessions, the AIBuds AI SDK records the device microphone through the app's SCO link. For Opus sessions, the device sends Opus to the app; forward the decoded PCM through appendInt16PCM(_:).
• Avoid enabling shouldSaveVoiceForDebugging in production unless your privacy and retention policies explicitly allow it.