AI Services Overview
The AIBuds AI framework provides a common entry point for AI chat, audio recording, streaming speech recognition, text-to-speech, summaries, text translation, simultaneous interpretation, conversation translation, image generation, and AI asking services.
Each AI service is supplied by a registered provider SDK that conforms to AIConnectSDK. Your app initializes AIBudsAISDK with the provider SDKs it includes, selects an AIServiceVendor, and then calls the service-specific APIs exposed by AIBudsAISDK.
Available Features
Select AI Provider
Register AI provider SDKs and select the provider used by AIBuds AI services
AI Audio Recording
Run an AI audio recording session with live transcription and a final report
Streaming ASR
Transcribe audio data or an audio file through the selected AI provider
Text to Speech
Synthesize text into a local audio file through the selected AI provider
AI Summary
Generate and cancel an incremental text summary through the selected AI provider
AI Text Translation
Translate text incrementally through the selected AI service provider
Simultaneous Interpretation
Run a spoken-language interpretation session with incremental source and target results
Conversation Translation
Build alternating push-to-talk translation turns with simultaneous interpretation
AI Chat
Run a device-driven AI voice chat session and handle chat, intent, voice, and event callbacks
Image Generation
Generate images from a prompt with optional provider limits and styles
AI Asking
Send a text question and consume streamed answer updates
Integration Flow
- Install the AI framework and the provider SDKs required by your application.
- Register the provider SDK implementations when initializing
AIBudsAISDK. - After a device is connected, provide its AI device information to the AI framework.
- Select
.starBurstor.mltcloudwithsetAIServiceVendor. - Complete provider authentication when its authentication mode is app initiated.
- Start the required AI service and handle its callbacks or session events.
Service Providers
The public SDK currently defines two selectable service providers:
| Enum Case | Service Provider |
|---|---|
.starBurst | StarBurst AI (ByteDance) |
.mltcloud | MltCloud AI (Meilc) |
.none means that no provider has been selected. It is the default value and is not a usable AI service provider.
Provider registration and provider selection are separate operations. Registration makes a provider implementation available to the framework; selection determines which registered provider receives subsequent AI service requests.
Service Availability
An installed provider may not implement every service or support the same languages as another provider. Use the provider metadata exposed by AIBudsAISDK, including allSupportedLanguages, and handle unsupported-service errors returned by individual operations.
Prerequisites
- Initialize the base AIBuds SDK and
AIBudsAISDKbefore using AI services. - Connect a supported device and configure its AI device information.
- Select a registered AI service provider.
- Complete authentication when required by that provider.
- Maintain network connectivity for cloud-backed services.
See Select AI Provider for provider initialization, selection, and authentication checks.