Polyv Help Center

Help Center

Live Web SDK

AI-SDK Logic Layer

Updated: 2026-01-19 09:16:42

Description

@polyv/live-watch-ai-sdk is the Polyv live streaming viewing AI SDK, providing core JavaScript SDK for AI features such as AI Assistant and Smart Outline. It includes the following features:

  • AI Assistant Chat: Supports real-time conversations with the AI Assistant, including streaming responses
  • AI Smart Outline: Retrieves smart outlines, subtitle parsing, and interactive quizzes for playback videos
  • Digital Human Support: Integrates AI digital human functionality, supporting speech synthesis and video playback
  • Multi-language Support: Supports multiple languages including Chinese, English, Japanese, Korean, and Russian
  • Modular Design: Adopts a modular architecture for easy extension and maintenance
  • Event-Driven: Event-driven API design for convenient state management

Installation

npm install @polyv/live-watch-ai-sdk
# 或
yarn add @polyv/live-watch-ai-sdk
# 或
pnpm add @polyv/live-watch-ai-sdk

Quick Start

Basic Usage

import { PolyvWatchAICore } from '@polyv/live-watch-ai-sdk';

// 初始化 SDK
const aiCore = new PolyvWatchAICore({
  lang: 'zh_CN',
  domainInfo: {
    polyvWatchApiDomain: 'https://watch-api.polyv.cn',
  },
  getViewerToken: async () => ({
    viewerToken: 'your-viewer-token',
  }),
  getChatToken: async () => ({
    chatToken: 'your-chat-token',
  }),
  getChannelInfo: async () => ({
    channelId: 'your-channel-id',
  }),
  getUserInfo: async () => ({
    userId: 'user-123',
    nick: '用户昵称',
    pic: 'https://example.com/avatar.jpg',
  }),
});

// 使用 AI 助手模块
const aiAssistant = aiCore.aiAssistant;

// 监听 AI 助手事件
aiAssistant.eventEmitter.on('AIAssistantChatMessage', ({ AIChatMsg }) => {
  console.log('收到 AI 消息:', AIChatMsg);
});

// 发送问题给 AI 助手
try {
  await aiAssistant.sendAIQuestionMsg({
    content: '你好，请介绍一下这个视频的主要内容',
  });
} catch (error) {
  console.error('发送问题失败:', error);
}

Core Modules

PolyvWatchAICore

The core class of the SDK, managing all modules and configurations.

Constructor Configuration

interface AICoreConfig {
  /** 语言类型 */
  lang?: LangType;
  /** 域名信息 */
  domainInfo?: Partial<DomainInfo>;
  /** 获取用户令牌信息 */
  getViewerToken?: () => Promise<ViewerTokenData> | ViewerTokenData;
  /** 获取聊天室服务令牌信息 */
  getChatToken?: () => Promise<ChatTokenData> | ChatTokenData;
  /** 获取频道信息 */
  getChannelInfo?: () => Promise<ChannelInfo> | ChannelInfo;
  /** 获取频道配置 */
  getChannelConfig?: () => Promise<ChannelConfig> | ChannelConfig;
  /** 获取用户信息 */
  getUserInfo?: () => Promise<Partial<UserInfo>> | Partial<UserInfo>;
}

Core Properties

  • aiAssistant: AI Assistant module
  • aiSummary: AI Smart Outline module
  • domain: Domain module

Core Methods

  • getAppConfig(): Get the current application configuration
  • updateAppConfig(): Update the application configuration
  • getViewerToken(): Get the viewer token
  • getChatToken(): Get the chat token
  • getChannelId(): Get the channel ID
  • getUserInfo(): Get user information
  • destroy(): Destroy the SDK instance

AI Assistant Module

Provides AI Assistant chat functionality, supporting digital human interaction.

Main Features

  1. AI Assistant Chat

    • Supports streaming conversations
    • Message group management
    • Chat state management

  2. Digital Human Features

    • Digital human video playback
    • Text-to-Speech (TTS)
    • Audio task management

  3. Audio Processing

    • Automatic Speech Recognition (ASR)
    • Audio-to-text conversion
    • Audio file processing

Usage Example

// 初始化 AI 助手
await aiCore.aiAssistant.setupAIAssistant(
  {
    aiAssistantId: 123,
    aiAssistantName: 'AI助手',
    aiAssistantCode: 'assistant-code',
  },
  { independent: true },
);

// 发送问题
await aiCore.aiAssistant.sendAIQuestionMsg({
  content: '请介绍一下这个视频',
});

// 获取聊天历史
const history = await aiCore.aiAssistant.getAIAssistantChatHistory();

// 创建音频任务（TTS）
const { taskId } = await aiCore.aiAssistant.createAiAssistantAudioTask({
  text: '你好，我是AI助手',
  rate: 1.0,
  ttsVoiceId: 'voice-001',
});

// 语音识别
const text = await aiCore.aiAssistant.recognizeAudioBlobToText({
  audioBlob: audioFile,
  audioType: 'wav',
});

AI Smart Outline Module

Provides intelligent analysis features for playback videos.

Main Features

  1. Playback Outline

    • Get video summaries
    • Segment content summaries
    • Keyword extraction

  2. Subtitle Processing

    • Get AI-generated subtitles
    • SRT format parsing
    • Timeline alignment

  3. Interactive Quizzes

    • Get interactive questions from the video
    • Question type identification
    • Answer verification

Usage Example

// 获取回放大纲
const outline = await aiCore.aiSummary.getPlaybackOutline({
  id: 'video-123',
  type: 'playback', // 'record' 或 'playback'
});

if (outline) {
  console.log('视频摘要:', outline.introduction);
  console.log('分段数量:', outline.outlineContent.length);
}

// 获取并解析字幕
const subtitleContent = await aiCore.aiSummary.getAISubtitleContent(
  'https://example.com/subtitle.srt',
);
const parsedSubtitles = aiCore.aiSummary.parseAISubtitleContent(subtitleContent);

parsedSubtitles.forEach(item => {
  console.log(`时间: ${item.start}-${item.end}ms, 内容: ${item.text}`);
});

// 获取互动答题
const questionData = await aiCore.aiSummary.getAIPlaybackQuestionData(
  'https://example.com/questions.json',
);

Event System

AI Assistant Events

enum AIAssistantEvents {
  // AI 助手设置完成
  AIAssistantChatSetupedIndependent = 'AIAssistantChatSetupedIndependent',
  AIAssistantChatSetupedComplete = 'AIAssistantChatSetupedComplete',

  // 聊天状态变化
  AIAssistantChatStatusChange = 'AIAssistantChatStatusChange',

  // 聊天消息
  AIAssistantChatMessage = 'AIAssistantChatMessage',
  ReplaceAIAssistantChatMessage = 'ReplaceAIAssistantChatMessage',
}

Chat States

enum PolyvAIAssistantChatStatus {
  Wait = 0, // 等待用户输入
  Replaying = 1, // AI 正在回答
  Busy = 2, // AI 繁忙
}

Event Listening Example

// 监听 AI 助手状态变化
aiCore.aiAssistant.eventEmitter.on('AIAssistantChatStatusChange', ({ status }) => {
  console.log('AI 助手状态变化:', status);
});

// 监听聊天消息
aiCore.aiAssistant.eventEmitter.on('AIAssistantChatMessage', ({ AIChatMsg }) => {
  console.log('收到消息:', AIChatMsg);
});

// 监听消息更新（用于流式响应）
aiCore.aiAssistant.eventEmitter.on('ReplaceAIAssistantChatMessage', ({ id, AIChatMsg }) => {
  console.log('消息更新:', AIChatMsg.content);
});

Utility Functions

Debugging Tools

import { setDebugMode } from '@polyv/live-watch-ai-sdk';

// 开启调试模式
setDebugMode(true);

Practical Utilities

import { plvInterval, plvWait, plvParseJson } from '@polyv/live-watch-ai-sdk';

// 定时器
const timer = plvInterval(
  () => {
    console.log('定时执行');
  },
  1000,
  { maxCount: 10 },
);

// 延迟执行
await plvWait(3000);

// JSON 解析
const result = plvParseJson<{ data: string }>('{"data": "test"}');
if (result.success) {
  console.log(result.data);
}

Best Practices

Performance Optimization

  1. Lazy Load Modules: Initialize modules on demand to reduce initial load time

    const aiCore = new PolyvWatchAICore(config);

if (needAIAssistant) {
  await aiCore.aiAssistant.setupAIAssistant(assistantConfig);
}

  2. Event Listener Management: Clean up event listeners promptly to avoid memory leaks

    const messageHandler = ({ AIChatMsg }) => {
  console.log('收到消息:', AIChatMsg);
};

aiAssistant.eventEmitter.on('AIAssistantChatMessage', messageHandler);

// 组件销毁时清理
function cleanup() {
  aiAssistant.eventEmitter.off('AIAssistantChatMessage', messageHandler);
}

State Management

// 在 Vue/React 中，将 SDK 状态与组件状态绑定
const messages = ref([]);
const chatStatus = ref(PolyvAIAssistantChatStatus.Wait);

aiAssistant.eventEmitter.on('AIAssistantChatMessage', ({ AIChatMsg }) => {
  messages.value.push(AIChatMsg);
});

aiAssistant.eventEmitter.on('AIAssistantChatStatusChange', ({ status }) => {
  chatStatus.value = status;
});

Frequently Asked Questions

1. How to obtain the necessary tokens?

The SDK requires the following tokens to function properly:

  • viewerToken: Viewer token, used for API authentication
  • chatToken: Chat token, used for AI Assistant chat

These tokens need to be obtained through the business backend API and provided to the SDK via callback functions during initialization.

2. What if the AI Assistant does not respond?

Check the following configurations:

  1. Ensure aiAssistantCode is set correctly
  2. Verify that getChatToken returns a valid chat token
  3. Check the network connection and domain configuration
  4. Look for error messages in the browser console

3. How to customize the domain?

const aiCore = new PolyvWatchAICore({
  domainInfo: {
    polyvWatchApiDomain: 'https://your-custom-domain.com',
  },
});
联系客服，在线咨询