为Chromium实现MediaConfigAPI-过程分享_UTF_8

落选的村支书

背景在实现实现任何功能前，一般都需要有数据统计支撑，作为收益验证的手段以及测试 Case 收集的手段。因此，在尝试实现 HEVC 硬解前，首先需要统计有多少视频是 HEVC 视频，并拿到视频的解码参数。很不幸，浏览器层面并未给我们暴露出视频的解码信息，比如分辨率，编码，Profile，色彩空间。这些一系列的视频参数，对于前端都是无感知且无法感知的。因此，在往常，如果需要获取视频信息做数据统计，首先需要拿到视频的二进制数据，在前端解析视频的元数据（比如 HEVC VPS、SPS、PPS），来提取视频参数。这样做，缺点是显而易见的，这种方式并不支持原生的 Video 标签播放的内容，且需要额外二次发起请求，且需要针对每种编码均实现一遍。因此，如果能从 Renderer 层，以浏览器视角，监听视频的解码信息，那或许我们则不需要做任何 Trick 的操作？我在这个过程尝试探索了一下，并成功实现了这个诉求。Media Internals 实现原理Media Internals 简介Chrome（包括 Chromium 内核的 Edge）实现了很多用于排查问题的 Debug 页面，比如我们常用的chrome://flags等等，那么对于音视频，是否也有这样的页面？答案是，有的，它叫作“Media Internal”，在你的浏览器输入chrome://media-internal,并随意打开一个有视频播放的页面，例如 youtube。则你会在这个页面看到视频的一些内部解码信息，以及MediaLog, 经过一番观察可以看到，红色画圈部分是我们需要的，如果我们能在视频播放时直接从在渲染进程获取到这部分信息，则我们即可轻松实现数据收集功能。media-internal 支持所有平台，且默认开启:src/chrome/browser/about_flags.cc在 url 输入 chrome://media-internals，背后发生了什么？路由判断，并加载对应模板的代码Chrome 内部，会根据 url.host_piece()是否为 kChromeUIMediaInternalsHost 且开启了 media Internal 功能，创建 MediaInternalsUI 实例:src/content/browser/webui/content_web_ui_controller_factory.ccWeb-UI 的主进程模块加载在主进程创建MediaInternalsMessageHandler实例监听后，执行CreateMediaInternalsHTMLSource(),即通过由 c++代码并渲染前端展示的 htmlsrc/content/browser/media/media-internals-ui.cc//入口WebUIDataSource*CreateMediaInternalsHTMLSource(){//创建一个叫`media-internal`的WebUIDataSourceWebUIDataSource*source=WebUIDataSource::Create(kChromeUIMediaInternalsHost);//将资源的路径添加到sourcesource->UseStringsJs();source->AddResourcePaths(base::make_span(kMediaInternalsResources,kMediaInternalsResourcesSize));//设置默认入口为media_internals.htmlsource->SetDefaultResource(IDR_MEDIA_INTERNALS_MEDIA_INTERNALS_HTML);returnsource;}继续观察代码实现，可以看到，我们在 media-internals 页面拿到的内容，其实就是由这部分代码生成的：media-internals 本质上会收集所有 renderer 进程产生的媒体信息事件（专业叫法叫做：MediaLog，Chrome media 模块单独定义的一种 log 类型）其中我们要的视频编码信息，在这里被定义为了一个叫做VideoDecoderConfig的类，因此我们从 media-internals 实现得到了一个重要信息，那就是：只要能获取到VideoDecoderConfig，就可以实现我们要的功能。//拿到了这个，我们就拿到了想要的一切structVideoDecoderConfig{VideoCodeccodec;VideoCodecProfileprofile;uint32level;boolhas_alpha;VideoTransformationtransformation;gfx.mojom.Sizecoded_size;gfx.mojom.Rectvisible_rect;gfx.mojom.Sizenatural_size;arrayextra_data;EncryptionSchemeencryption_scheme;VideoColorSpacecolor_space_info;gfx.mojom.HDRMetadatahdr_metadata;};类似的，其实音频的信息，在 Chromium 内同样有一个类似的类，叫做AudioDecoderConfig。因此实现目标变为：拿到**VideoDecoderConfig**以及**AudioDecoderConfig**。媒体编码信息获取 API 实现通过搜索 Chromium 的代码，我们发现，对于普通视频，媒体信息的获取是在主进程通过 FFMpegDemuxer 解析得到，MSE 视频则是利用 ChunkDemuxer 解析得到，并最终利用 Mojo 进行 IPC 通信传输到 Renderer 进程，OK，这说明理论上该操作是可行的。设计 MediaConfig API在实现前，首先需要思考，应该以何种形式拿到视频的信息，并优雅暴露到前端，这里我能想到的最好办法是直接在 HTMLMediaElement 类实现我们要的 API，这样最终不管是 video 标签还是 audio 标签，都可以拿到音视频编码信息，这里先放上最终设计好后的 IDL：interfaceMediaDecoderConfig{//总比特率（暂时不支持ObjectURL）readonlytotalBitrate:number;//视频流总比特率（暂时不支持ObjectURL）readonlyvideoBitrate:number;//视频编码readonlyvideoCodec:string;//视频配置readonlyvideoProfile:string;//视频级别readonlyvideoLevel:string;//视频色彩空间原色readonlyvideoColorSpacePrimaries:string;//视频色彩空间传递函数readonlyvideoColorSpaceTransfer:string;//视频色彩空间矩阵readonlyvideoColorSpaceMatrix:string;//视频色彩空间范围readonlyvideoColorSpaceRange:string;//视频编码宽度readonlyvideoCodedSizeWidth:number;//视频编码高度readonlyvideoCodedSizeHeight:number;//视频原始宽度readonlyvideoNaturalSizeWidth:number;//视频原始高度readonlyvideoNaturalSizeHeight:number;//视频可使区X点readonlyvideoVisibleRectX:number;//视频可使区X点readonlyvideoVisibleRectY:number;//视频可使区宽度readonlyvideoVisibleRectWidth:number;//视频可使区高度readonlyvideoVisibleRectHeight:number;//视频是否加密readonlyvideoEncryption:string;//视频HDR元数据-亮度范围readonlyvideoHdrMetadataLuminanceRange:string;//视频HDR元数据-主阵列readonlyvideoHdrMetadataPrimaries:string;//视频HDR元数据-MaxContentLightLevelreadonlyvideoHdrMetadataMaxContentLightLevel:number;//视频HDR元数据-MaxFrameAverageLightLevelreadonlyvideoHdrMetadataMaxFrameAverageLightLevel:number;//视频旋转角度readonlyvideoTransformationRotation:string;//视频是否翻转readonlyvideoTransformationFlipped:boolean;//视频是否有Alpha通道readonlyvideoHasAlpha:boolean;//视频是否有（内部）额外数据，可忽略readonlyvideoHasExtraData:boolean;//音频比特率readonlyaudioBitrate:number;//音频编码readonlyaudioCodec:string;//音频配置readonlyaudioProfile:string;//音频每通道比特数readonlyaudioBytesPerChannel:number;//音频通道格式readonlyaudioChannelLayout:string;//音频通道数readonlyaudioChannels:number;//音频采样速率readonlyaudioSamplesPerSecond:number;//音频采样格式readonlyaudioSampleFormat:string;//音频每帧字节数readonlyaudioBytesPerFrame:number;//音频预滚readonlyaudioSeekPreroll:string;//音频延迟readonlyaudioCodecDelay:number;//音频是否加密readonlyaudioEncryption:string;//音频输出通道格式readonlyaudioTargetOutputChannelLayout:string;//音频是否应该忽略解码器延迟readonlyaudioShouldDiscardDecoderDelay:boolean;//音频是否有（内部）额外数据，可忽略readonlyaudioHasExtraData:boolean;}interfaceHTMLMediaElementEventMapextendsHTMLElementEventMap{mediaconfigchange:Event;}interfaceHTMLMediaElementextendsHTMLElement{//视频配置属性，可在DOM查看readonlyvideoConfig:string;//音频配置属性，可在DOM读取readonlyaudioConfig:string;//当该事件触发时，可获取到视频配置onmediaconfigchange(this:HTMLMediaElement,ev:Event)=>any)|null;//获取音视频配置getMediaConfig():MediaDecoderConfig;}监听音视频信息变化通过观测发现，Chromium 在 Blink 的 WebMediaPlayerImpl 模块实现了 onMetadata 方法，当音视频信息变化时，会触发该方法，因此能实现我们要的 API 的本质实际上就是利用了 onMetadata 方法的触发diff--gita/third_party/blink/renderer/platform/media/web_media_player_impl.ccb/third_party/blink/renderer/platform/media/web_media_player_impl.ccindex18b3ef5fe312d..003b26e1d123f100644---a/third_party/blink/renderer/platform/media/web_media_player_impl.cc+++b/third_party/blink/renderer/platform/media/web_media_player_impl.cc@@-1919,6+1919,9@@voidWebMediaPlayerImpl::OnMetadata(constmedia:ipelineMetadata&metadata){media_metrics_provider_->SetHasAudio(metadata.audio_decoder_config.codec());RecordEncryptionScheme("Audio",metadata.audio_decoder_config.encryption_scheme());//实现一个onAudioConfigChange方法，获取音频变化++if(client_)+client_->OnAudioConfigChange(metadata.audio_decoder_config);}if(HasVideo()){@@-1926,6+1929,9@@voidWebMediaPlayerImpl::OnMetadata(constmedia:ipelineMetadata&metadata){RecordEncryptionScheme("Video",metadata.video_decoder_config.encryption_scheme());//实现一个OnVideoConfigChange方法，获取视频变化+if(client_)+client_->OnVideoConfigChange(metadata.video_decoder_config);+if(overlay_enabled_){//SurfaceViewdoesn'tsupportrotatedvideo,sotransitionbackif//thevideoisnowrotated.If`always_enable_overlays_`,wekeepthe@@-1948,6+1954,9@@voidWebMediaPlayerImpl::OnMetadata(constmedia:ipelineMetadata&metadata){}}//实现最终的OnMediaConfigChange方法，用于记录是否含有音频或视频+if(client_)+client_->OnMediaConfigChange(HasVideo(),HasAudio());+if(observer_)observer_->OnMetadataChanged(pipeline_metadata_);diff--gita/third_party/blink/public/platform/web_media_player_client.hb/third_party/blink/public/platform/web_media_player_client.hindex55e80c1e54281..c6ef56e36c614100644---a/third_party/blink/public/platform/web_media_player_client.h+++b/third_party/blink/public/platform/web_media_player_client.h@@-218,6+218,13@@classBLINK_PLATFORM_EXPORTWebMediaPlayerClient{//Seehttps://wicg.github.io/video-rvfc/.virtualvoidOnRequestVideoFrameCallback(){}+//Onmetadatareadyandvideoconfigaquired+virtualvoidOnVideoConfigChange(constmedia::VideoDecoderConfig&video_config)=0;+//Onmetadatareadyandaudioconfigaquired+virtualvoidOnAudioConfigChange(constmedia::AudioDecoderConfig&audio_config)=0;+//Onmeteadatereadyandaudio/configaquiredthenfireevent+virtualvoidOnMediaConfigChange(boolhas_video,boolhas_audio)=0;+structFeatures{WebStringid;WebStringwidth;在 HTMLMediaElement 实现 API拿到音视频信息变化后，我们需要在 HTMLMediaElement 实现我们要的方法。diff--gita/third_party/blink/renderer/core/html/media/html_media_element.ccb/third_party/blink/renderer/core/html/media/html_media_element.ccindexcfcb19002a16a..cde5b8b8cc4ac100644---a/third_party/blink/renderer/core/html/media/html_media_element.cc+++b/third_party/blink/renderer/core/html/media/html_media_element.cc@@-29,6+29,7@@#include#include+#include#include"base/auto_reset.h"#include"base/debug/crash_logging.h"#include"base/feature_list.h"@@-56,6+57,7@@#include"third_party/blink/public/platform/web_media_player_source.h"#include"third_party/blink/renderer/bindings/core/v8/script_controller.h"#include"third_party/blink/renderer/bindings/core/v8/script_promise_resolver.h"+#include"third_party/blink/renderer/bindings/core/v8/v8_binding_for_core.h"#include"third_party/blink/renderer/core/core_initializer.h"#include"third_party/blink/renderer/core/core_probes_inl.h"#include"third_party/blink/renderer/core/css/media_list.h"@@-83,6+85,7@@#include"third_party/blink/renderer/core/html/media/autoplay_policy.h"#include"third_party/blink/renderer/core/html/media/html_media_element_controls_list.h"#include"third_party/blink/renderer/core/html/media/media_controls.h"+#include"third_party/blink/renderer/core/html/media/media_decoder_config.h"#include"third_party/blink/renderer/core/html/media/media_error.h"#include"third_party/blink/renderer/core/html/media/media_fragment_uri_parser.h"#include"third_party/blink/renderer/core/html/media/media_source_attachment.h"@@-1539,6+1542,24@@voidHTMLMediaElement:idAudioOutputSinkChanged(observer->OnAudioOutputSinkChanged(hashed_device_id);}+voidHTMLMediaElement::OnMediaConfigChange(boolhas_video,boolhas_audio){//记录是否有音、视频配置+has_video_=has_video;+has_audio_=has_audio;//在HTMLMediaElement上Dispatch一个事件，通知前端+ScheduleEvent(event_type_names::kMediaconfigchange);+}++voidHTMLMediaElement::OnVideoConfigChange(constmedia::VideoDecoderConfig&config){//存储VideoDecoderConfig+video_config_=config;+conststd::stringvideo_config=config.AsHumanReadableString();//在HTMLMediaElement上设置属性+setAttribute(html_names::kVideoconfigAttr,AtomicString(video_config.c_str()));+}++voidHTMLMediaElement::OnAudioConfigChange(constmedia::AudioDecoderConfig&config){//存储AudioDecoderConfig+audio_config_=config;+conststd::stringaudio_config=config.AsHumanReadableString();//在HTMLMediaElement上设置属性+setAttribute(html_names::kAudioconfigAttr,AtomicString(audio_config.c_str()));+}+voidHTMLMediaElement::SetMediaPlayerHostForTesting(mojo:endingAssociatedRemotehost){media_player_host_remote_->Value().Bind(@@-2528,6+2549,115@@boolHTMLMediaElement::Autoplay()const{returnFastHasAttribute(html_names::kAutoplayAttr);}//为HTMLMediaElement实现的videoconfig属性，只读+StringHTMLMediaElement::videoConfig()const{+LOG(INFO)<<__FUNCTION__<<"getVideoConfig";+returnFastGetAttribute(html_names::kVideoconfigAttr).GetString();+}++//为HTMLMediaElement实现的videoconfig属性，只读+StringHTMLMediaElement::audioConfig()const{+LOG(INFO)<<__FUNCTION__<<"getAudioConfig";+returnFastGetAttribute(html_names::kAudioconfigAttr).GetString();+}+//这里是我们最终要实现的getMediaConfigAPI的实现函数，大体逻辑即+//将音视频配置转换为BlinkIDL支持的类型+MediaDecoderConfig*HTMLMediaElement::getMediaConfig(){+MediaDecoderConfig*config=MakeGarbageCollected();++if(has_video_){+constinttotal_bitrate=video_config_.GetHumanReadableTotalBitrate();+constintvideo_bitrate=video_config_.GetHumanReadableVideoBitrate();+constAtomicStringvideo_codec=AtomicString(video_config_.GetHumanReadableCodecName().c_str());+constAtomicStringvideo_profile=AtomicString(video_config_.GetHumanReadableProfile().c_str());+constAtomicStringvideo_level=AtomicString(video_config_.GetHumanReadableLevel().c_str());+constAtomicStringvideo_color_space_primaries=AtomicString(video_config_.GetHumanReadableColorSpacePrimaries().c_str());+constAtomicStringvideo_color_space_transfer=AtomicString(video_config_.GetHumanReadableColorSpaceTransfer().c_str());+constAtomicStringvideo_color_space_matrix=AtomicString(video_config_.GetHumanReadableColorSpaceMatrix().c_str());+constAtomicStringvideo_color_space_range=AtomicString(video_config_.GetHumanReadableColorSpaceRange().c_str());+constfloatvideo_coded_size_width=video_config_.GetHumanReadableCodedSizeWidth();+constfloatvideo_coded_size_height=video_config_.GetHumanReadableCodedSizeHeight();+constfloatvideo_natural_size_width=video_config_.GetHumanReadableNaturalSizeWidth();+constfloatvideo_natural_size_height=video_config_.GetHumanReadableNaturalSizeHeight();+constfloatvideo_visible_rect_x=video_config_.GetHumanReadableVisibleRectX();+constfloatvideo_visible_rect_y=video_config_.GetHumanReadableVisibleRectY();+constfloatvideo_visible_rect_width=video_config_.GetHumanReadableVisibleRectWidth();+constfloatvideo_visible_rect_height=video_config_.GetHumanReadableVisibleRectHeight();+constAtomicStringvideo_encryption=AtomicString(video_config_.GetHumanReadableEncryption().c_str());+constAtomicStringvideo_hdr_metadata_luminance_range=AtomicString(video_config_.GetHumanReadableHDRMetadataLuminanceRange().c_str());+constAtomicStringvideo_hdr_metadata_primaries=AtomicString(video_config_.GetHumanReadableHDRMetadataPrimaries().c_str());+constunsignedvideo_hdr_metadata_max_content_light_level=video_config_.GetHumanReadableHDRMetadataMaxContentLightLevel();+constunsignedvideo_hdr_metadata_max_frame_average_light_level=video_config_.GetHumanReadableHDRMetadataMaxFrameAverageLightLevel();+constAtomicStringvideo_transformation_rotation=AtomicString(video_config_.GetHumanReadableTransformationRotation().c_str());+constboolvideo_transformation_flipped=video_config_.GetHumanReadableTransformationFlipped();+constboolvideo_has_alpha=video_config_.GetHumanReadableHasAlpha();+constboolvideo_has_extra_data=video_config_.GetHumanReadableHasExtraData();++config->SetVideoConfig(total_bitrate,+video_bitrate,+video_codec,+video_profile,+video_level,+video_color_space_primaries,+video_color_space_transfer,+video_color_space_matrix,+video_color_space_range,+video_coded_size_width,+video_coded_size_height,+video_natural_size_width,+video_natural_size_height,+video_visible_rect_x,+video_visible_rect_y,+video_visible_rect_width,+video_visible_rect_height,+video_encryption,+video_hdr_metadata_luminance_range,+video_hdr_metadata_primaries,+video_hdr_metadata_max_content_light_level,+video_hdr_metadata_max_frame_average_light_level,+video_transformation_rotation,+video_transformation_flipped,+video_has_alpha,+video_has_extra_data);+}++if(has_audio_){+constinttotal_bitrate=audio_config_.GetHumanReadableTotalBitrate();+constintaudio_bitrate=audio_config_.GetHumanReadableAudioBitrate();、//知识点，std:string需要转换为AtomicString，才可暴露给前端+constAtomicStringaudio_codec=AtomicString(audio_config_.GetHumanReadableCodecName().c_str());+constAtomicStringaudio_profile=AtomicString(audio_config_.GetHumanReadableProfile().c_str());+constintaudio_bytes_per_channel=audio_config_.GetHumanReadableBytesPerChannel();+constAtomicStringaudio_channel_layout=AtomicString(audio_config_.GetHumanReadableChannelLayout().c_str());+constintaudio_channels=audio_config_.GetHumanReadableChannels();+constintaudio_samples_per_second=audio_config_.GetHumanReadableSamplesPerSecond();+constAtomicStringaudio_sample_format=AtomicString(audio_config_.GetHumanReadableSampleFormat().c_str());+constintaudio_bytes_per_frame=audio_config_.GetHumanReadableBytesPerFrame();+constAtomicStringaudio_seek_preroll=AtomicString(audio_config_.GetHumanReadableSeekPreroll().c_str());+constintaudio_codec_delay=audio_config_.GetHumanReadableCodecDelay();+constAtomicStringaudio_encryption=AtomicString(audio_config_.GetHumanReadableEncryption().c_str());+constAtomicStringaudio_target_output_channel_layout=AtomicString(audio_config_.GetHumanReadableTargetOutputChannelLayout().c_str());+constboolaudio_should_discard_decoder_delay=audio_config_.GetHumanReadableShouldDiscardDecoderDelay();+constboolaudio_has_extra_data=audio_config_.GetHumanReadableHasExtraData();++config->SetAudioConfig(total_bitrate,+audio_bitrate,+audio_codec,+audio_profile,+audio_bytes_per_channel,+audio_channel_layout,+audio_channels,+audio_samples_per_second,+audio_sample_format,+audio_bytes_per_frame,+audio_seek_preroll,+audio_codec_delay,+audio_encryption,+audio_target_output_channel_layout,+audio_should_discard_decoder_delay,+audio_has_extra_data);+}++returnconfig;+}+StringHTMLMediaElement::preload()const{if(GetLoadType()==WebMediaPlayer::kLoadTypeMediaStream)returnPreloadTypeToString(WebMediaPlayer::kPreloadNone);diff--gita/third_party/blink/renderer/core/html/media/html_media_element.hb/third_party/blink/renderer/core/html/media/html_media_element.hindex4af2fe7c1c04b..eb5d1956196c2100644---a/third_party/blink/renderer/core/html/media/html_media_element.h+++b/third_party/blink/renderer/core/html/media/html_media_element.h@@-31,6+31,7@@#include"base/time/time.h"#include"base/timer/elapsed_timer.h"+#include"media/base/audio_decoder.h"#include"media/mojo/mojom/media_player.mojom-blink.h"#include"third_party/abseil-cpp/absl/types/optional.h"#include"third_party/blink/public/common/media/display_type.h"@@-56,6+57,7@@#include"third_party/blink/renderer/platform/timer.h"#include"third_party/blink/renderer/platform/weborigin/kurl.h"#include"third_party/blink/renderer/platform/wtf/threading_primitives.h"+#include"third_party/blink/renderer/core/html/media/media_decoder_config.h"namespacecc{classLayer;@@-93,6+95,7@@classVideoTrack;classVideoTrackList;classWebInbandTextTrack;classWebRemotePlaybackClient;+classMediaDecoderConfig;classCORE_EXPORTHTMLMediaElement:publicHTMLElement,@@-190,6+193,10@@classCORE_EXPORTHTMLMediaElementStringEffectivePreload()const;WebMediaPlayer:reloadEffectivePreloadType()const;+StringvideoConfig()const;+StringaudioConfig()const;+MediaDecoderConfig*getMediaConfig();+WebTimeRangesBufferedInternal()const;TimeRanges*buffered()const;voidload();@@-254,6+261,11@@classCORE_EXPORTHTMLMediaElementvoidSetUserWantsControlsVisible(boolvisible);boolUserWantsControlsVisible()const;+voidOnMediaConfigChange(boolhas_video,boolhas_audio)override;+voidOnVideoConfigChange(constmedia::VideoDecoderConfig&video_config)override;+voidOnAudioConfigChange(constmedia::AudioDecoderConfig&audio_config)override;+voidTogglePlayState();AudioTrackList&audioTracks();@@-663,6+675,11@@classCORE_EXPORTHTMLMediaElementKURLcurrent_src_after_redirects_;Membersrc_object_;+boolhas_video_=false;+boolhas_audio_=false;+media::VideoDecoderConfigvideo_config_;+media::AudioDecoderConfigaudio_config_;+//TopreventpotentialregressionwhenextendedbytheMSEAPI,donotset//|error_|outsideofconstructorandSetError().Membererror_;接着需要为我们新增的 API 定义 IDL：diff--gita/third_party/blink/renderer/core/html/media/html_media_element.idlb/third_party/blink/renderer/core/html/media/html_media_element.idlindexa041c328f1dec..ed2d93a8f403f100644---a/third_party/blink/renderer/core/html/media/html_media_element.idl+++b/third_party/blink/renderer/core/html/media/html_media_element.idl@@-46,6+46,9@@enumCanPlayTypeResult{""/*emptystring*/,"maybe","probably"};constunsignedshortNETWORK_NO_SOURCE=3;[ImplementedAs=getNetworkState]readonlyattributeunsignedshortnetworkState;[CEReactions]attributeDOMStringpreload;+readonlyattributeDOMStringvideoConfig;+readonlyattributeDOMStringaudioConfig;+MediaDecoderConfiggetMediaConfig();readonlyattributeTimeRangesbuffered;voidload();[CallWith=ExecutionContext,HighEntropy,Measure]CanPlayTypeResultcanPlayType(DOMStringtype);计算音视频的比特率由于VideoDecoderConfig缺失了比特率这个重要的参数，为了收集到更详细的视频信息，我们还需要在 Demux 阶段计算一下音视频轨道的比特率。diff--gita/third_party/blink/public/platform/web_media_player.hb/third_party/blink/public/platform/web_media_player.hindex306f13ae8af00..da703e6e7bf1d100644---a/third_party/blink/public/platform/web_media_player.h+++b/third_party/blink/public/platform/web_media_player.h@@-186,6+186,8@@classWebMediaPlayer{virtualvoidFlingingStarted(){}virtualvoidFlingingStopped(){}virtualvoidSetPreload(Preload){}+virtualvoidSetVideoConfig(constAtomicString&){}+virtualvoidSetAudioConfig(constAtomicString&){}virtualWebTimeRangesBuffered()const=0;virtualWebTimeRangesSeekable()const=0;diff--gita/media/filters/ffmpeg_demuxer.hb/media/filters/ffmpeg_demuxer.hindex8d4c66e7611ac..ec8893131bb66100644---a/media/filters/ffmpeg_demuxer.h+++b/media/filters/ffmpeg_demuxer.h@@-123,6+123,9@@classMEDIA_EXPORTFFmpegDemuxerStream:publicDemuxerStream{AudioDecoderConfigaudio_decoder_config()override;VideoDecoderConfigvideo_decoder_config()override;+voidSetAudioBitrate(intaudio_bitrate,intbitrate);+voidSetVideoBitrate(intvideo_bitrate,intbitrate);+boolIsEnabled()const;voidSetEnabled(boolenabled,base::TimeDeltatimestamp);由于音视频可能有多个流，因此需要将流的比特率加和。diff--gita/media/filters/ffmpeg_demuxer.ccb/media/filters/ffmpeg_demuxer.ccindexa76be9a01c5be..47ded334541aa100644---a/media/filters/ffmpeg_demuxer.cc+++b/media/filters/ffmpeg_demuxer.cc@@-830,6+830,20@@VideoDecoderConfigFFmpegDemuxerStream::video_decoder_config(){return*video_config_;}//记录音频总比特率和音视频总比特率+voidFFmpegDemuxerStream::SetAudioBitrate(intaudio_bitrate,intbitrate){+if(audio_config_->IsValidConfig()){+audio_config_->SetAudioStreamBitrate(audio_bitrate);+audio_config_->SetTotalStreamBitrate(bitrate);+}+}+//记录视频总比特率和音视频总比特率+voidFFmpegDemuxerStream::SetVideoBitrate(intvideo_bitrate,intbitrate){+if(video_config_->IsValidConfig()){+video_config_->SetVideoStreamBitrate(video_bitrate);+video_config_->SetTotalStreamBitrate(bitrate);+}+}+boolFFmpegDemuxerStream::IsEnabled()const{DCHECK(task_runner_->RunsTasksInCurrentSequence());returnis_enabled_;@@-1236,6+1250,40@@staticintCalculateBitrate(AVFormatContext*format_context,returnbase::ClampRound(filesize_in_bytes*duration.ToHz()*8);}//计算音频总比特率的逻辑+staticintCalculateAudioBitrate(AVFormatContext*format_context){+intaudio_stream_bitrate=0;++for(size_ti=0;inb_streams;++i){+AVCodecParameters*codec_parameters=format_context->streams[i]->codecpar;+AVMediaTypecodec_type=codec_parameters->codec_type;+if(codec_type==AVMEDIA_TYPE_AUDIO){+audio_stream_bitrate+=codec_parameters->bit_rate;+}+}++if(audio_stream_bitrate>0)+returnaudio_stream_bitrate;++return0;+}+//计算视频总比特率的逻辑+staticintCalculateVideoBitrate(AVFormatContext*format_context){+intvideo_stream_bitrate=0;++for(size_ti=0;inb_streams;++i){+AVCodecParameters*codec_parameters=format_context->streams[i]->codecpar;+AVMediaTypecodec_type=codec_parameters->codec_type;+if(codec_type==AVMEDIA_TYPE_VIDEO){+video_stream_bitrate+=codec_parameters->bit_rate;+}+}++if(video_stream_bitrate>0)+returnvideo_stream_bitrate;++return0;+}+voidFFmpegDemuxer::OnOpenContextDone(boolresult){DCHECK(task_runner_->RunsTasksInCurrentSequence());if(stopped_){@@-1563,6+1611,20@@voidFFmpegDemuxer::OnFindStreamInfoDone(intresult){if(bitrate_>0)data_source_->SetBitrate(bitrate_);+intvideo_stream_bitrate=CalculateVideoBitrate(format_context);+intaudio_stream_bitrate=CalculateAudioBitrate(format_context);+inttotal_stream_bitrate=video_stream_bitrate+audio_stream_bitrate;++for(constauto&stream:streams_){+if(stream){+if(stream->type()==DemuxerStream::AUDIO){+stream->SetAudioBitrate(audio_stream_bitrate,total_stream_bitrate);+}elseif(stream->type()==DemuxerStream::VIDEO){+stream->SetVideoBitrate(video_stream_bitrate,total_stream_bitrate);+}+}+}+LogMetadata(format_context,max_duration);media_tracks_updated_cb_.Run(std::move(media_tracks));扩展 VideoDecoderConfig我们需要将VideoDecoderConfig的属性提取成我们需要的类型，且能支持我们保存比特率，因此这里需要的额外扩展一下 DecoderConfig：diff--gita/media/base/video_decoder_config.ccb/media/base/video_decoder_config.ccindex40c9707fae4d8..d1b311003b975100644---a/media/base/video_decoder_config.cc+++b/media/base/video_decoder_config.cc//将我们要的参数转换为我们要的格式@@-106,6+106,11@@std::stringVideoDecoderConfig::AsHumanReadableString()const{<<",rotation:"<