Video Player Principles
Video and audio technology mainly includes the following aspects: container technology, video compression coding technology, and audio compression coding technology. If network transmission is considered, it also includes streaming media protocol technology.
When a video player plays a video file from the internet, it needs to go through the following steps: protocol demultiplexing, container demultiplexing, video/audio decoding, and audio-video synchronization. If playing a local file, protocol demultiplexing is not required, and the steps are: container demultiplexing, video/audio decoding, and audio-video synchronization. The process is illustrated in the figure below.
The role of protocol demultiplexing is to parse streaming media protocol data into standard corresponding container format data. When video and audio are transmitted over the network, various streaming media protocols are often used, such as HTTP, RTMP, or MMS. These protocols transmit signaling data along with the video and audio data. The signaling data includes playback control (play, pause, stop) or descriptions of network status. During protocol demultiplexing, the signaling data is removed, leaving only the video and audio data. For example, data transmitted using the RTMP protocol, after protocol demultiplexing, outputs data in FLV format.
The role of container demultiplexing is to separate the input container format data into audio stream compressed coding data and video stream compressed coding data. There are many container formats, such as MP4, MKV, RMVB, TS, FLV, AVI, etc. Their purpose is to combine compressed video data and audio data into a single file according to a specific format. For example, after container demultiplexing of FLV format data, it outputs H.264 encoded video stream and AAC encoded audio stream.
The role of decoding is to decompress video/audio compressed coding data into uncompressed raw video/audio data. Audio compression coding standards include AAC, MP3, AC-3, etc., while video compression coding standards include H.264, MPEG2, VC-1, etc. Decoding is the most important and complex part of the entire system. Through decoding, compressed video data is output as uncompressed color data, such as YUV420P, RGB, etc.; compressed audio data is output as uncompressed audio sample data, such as PCM data.
The role of audio-video synchronization is to synchronize the decoded video and audio data based on the parameter information obtained during the container demultiplexing process, and then send the video and audio data to the system's graphics card and sound card for playback.
