Usage

The AI Assistant module provides intelligent conversation capabilities, supporting real-time Q&A, chat history, and audio processing features.

Main Methods

Usage Example

// 初始化 AI 助手（独立模式）
await aiCore.aiAssistant.setupAIAssistant(
  {
    aiAssistantCode: 'your-code',
    aiAssistantToken: 'your-token',
  },
  { independent: true },
);

// 发送问题
await aiCore.aiAssistant.sendAIQuestionMsg({
  content: '请介绍一下视频内容',
});

// 获取聊天历史
const history = await aiCore.aiAssistant.getAIAssistantChatHistory();

// 音频合成（TTS）
const { taskId } = await aiCore.aiAssistant.createAiAssistantAudioTask({
  text: '你好，我是 AI 助手',
  rate: 1.0,
});

// 音频识别
const text = await aiCore.aiAssistant.recognizeAudioBlobToText({
  audioBlob: audioFile,
  audioType: 'wav',
});

Whether AI Assistant Chat is Supported

API Method: supportAIAssistantChat(): SupportResult

Return Value Description: SupportResult type

Get AI Assistant Related Configuration

API Method: getAIAssistantConfig(): PolyvAIAssistantConfig

Return Value Description: AI Assistant configuration, PolyvAIAssistantConfig type. Detailed type description is as follows:

Get AI Assistant Information

Description: Get the configuration information of the current AI Assistant, including assistant ID, name, digital human information, etc.

API Method: getAIAssistantInfo(): PolyvAIAssistantInfo

Return Value Description: AI Assistant information object, PolyvAIAssistantInfo type. Detailed type description is as follows:

Example:

const assistantModule = new AIAssistantModule();
await assistantModule.setupAIAssistant({ aiAssistantCode: 'xxx' });
const info = assistantModule.getAIAssistantInfo();
console.log('助手ID:', info.aiAssistantId);
console.log('助手名称:', info.aiAssistantName);
console.log('欢迎语:', info.welcomeSpeech);
console.log('数字人ID:', info.digitalHumanId);
console.log('数字人名称:', info.digitalHumanName);
console.log('说话视频地址:', info.videoSource?.speakUrl);
console.log('不说话视频地址:', info.videoSource?.noSpeakUrl);

Initialize AI Assistant

API Method: setupAIAssistant(aiAssistantInfo: Partial<PolyvAIAssistantInfo>, opts?: Object): Promise<void>

Parameter Description:

• aiAssistantInfo: Partial<PolyvAIAssistantInfo> type, required

• opts: Object type, optional. Detailed type description is as follows:

Send AI Question Message

Used by the audience to send a question message to the AI Assistant. During the call, events such as AIAssistantEvents.AIAssistantChatStatusChange, AIAssistantEvents.AIAssistantChatMessage, and AIAssistantEvents.ReplaceAIAssistantChatMessage will be triggered.

API Method: sendAIQuestionMsg(__namedParameters: SendAIQuestionMsgOptions): Promise<void>

Parameter Description:

• __namedParameters: SendAIQuestionMsgOptions type, required. Detailed type description is as follows:

Get AI Assistant Chat History

Description: Get the chat history between the current user and the AI Assistant, including questions and answers.

API Method: getAIAssistantChatHistory(): Promise<PolyvAIAssistantChatMsgType[]>

Return Value Description: Array of chat messages, sorted chronologically, Promise<PolyvAIAssistantChatMsgType[]> type

Example:

const assistantModule = new AIAssistantModule();
try {
  const history = await assistantModule.getAIAssistantChatHistory();
  console.log('历史消息数量:', history.length);

  history.forEach(msg => {
    if (msg.msgSource === PolyvAIAssistantChatMsgSource.Question) {
      console.log('【用户提问】:', msg.content);
    } else {
      console.log('【AI 回答】:', msg.content);
    }
  });
} catch (error) {
  console.error('获取聊天历史失败:', error);
}

Create AI Assistant Audio Synthesis Task

Description: Convert text to speech and generate an audio file, supporting custom speech rate and voice type.

API Method: createAiAssistantAudioTask(opts: Object): Promise<Object>

Parameter Description:

• opts: Audio synthesis parameters, Object type, required. Detailed type description is as follows:

Return Value Description: Object containing the task ID, used to retrieve the audio result later, Promise<Object> type. Detailed type description is as follows:

Example:

const assistantModule = new AIAssistantModule();

// 创建音频合成任务
const { taskId } = await assistantModule.createAiAssistantAudioTask({
  text: '欢迎观看直播，这是 AI 助手为您生成的语音内容',
  rate: 1,
  ttsVoiceId: 'female_zh_1'
});
console.log('任务ID:', taskId);

// 等待一段时间后获取音频结果
const audioResult = await assistantModule.getAiAssistantAudioTaskResult(taskId);
console.log('音频URL:', audioResult.audioUrl);

Get AI Assistant Audio Synthesis Task Result

Description: Get the result of a previously created audio synthesis task based on the task ID, including the audio URL and status.

API Method: getAiAssistantAudioTaskResult(taskId: string): Promise<PostAiAssistantGetAudioTaskResultResponseData>

Parameter Description:

• taskId: Audio synthesis task ID (obtained via createAiAssistantAudioTask), string type, required

Return Value Description: Audio task result object, containing audio URL and status information, Promise<PostAiAssistantGetAudioTaskResultResponseData> type

Recognize Audio and Convert to Text

Description: Recognize and convert user-uploaded audio files into text content.

API Method: recognizeAudioBlobToText(params: Object): Promise<string>

Parameter Description:

• params: Audio recognition parameters, Object type, required. Detailed type description is as follows:

Return Value Description: Recognized text content, Promise<string> type

Example:

const assistantModule = new AIAssistantModule();

// 假设已有音频 blob
const audioBlob = new Blob([audioData], { type: 'audio/wav' });

try {
  const text = await assistantModule.recognizeAudioBlobToText({
    audioBlob,
    audioType: 'wav'
  });
  console.log('识别结果:', text);

  // 可以将识别结果发送给 AI 助手
  await assistantModule.sendAIQuestionMsg({ content: text });
} catch (error) {
  console.error('音频识别失败:', error);
}