写点什么

iOS 平台如何实现毫秒级延迟的 RTMP|RTSP 播放器

作者:音视频牛哥
  • 2024-02-06
    上海
  • 本文字数:7312 字

    阅读完需:约 24 分钟

技术背景

在我的 blog 里面,最近很少有提到 iOS 平台 RTMP 推送|轻量级 RTSP 服务和 RTMP|RTSP 直播播放模块,实际上,我们在 2016 年就发布了 iOS 平台直播推拉流、转发模块,只是因为传统行业,对 iOS 的需求比较少,所以一直没单独说明,本文主要介绍下,如何在 iOS 平台播放 RTMP 或 RTSP 流。

技术实现

先说播放实现,iOS 端,RTMP|RTSP 直播播放,我们实现的功能如下:

  •  [支持播放协议]高稳定、超低延迟(毫秒级)

  •  [多实例播放]支持多实例播放;

  •  [事件回调]支持网络状态、buffer 状态等回调;

  •  [视频格式]支持 RTMP 扩展 H.265,H.264;

  •  [音频格式]支持 AAC/PCMA/PCMU/Speex;

  •  [H.264/H.265 软解码]支持 H.264/H.265 软解;

  •  [H.264 硬解码]Windows/Android/iOS 支持特定机型 H.264 硬解;

  •  [H.265 硬解]Windows/Android/iOS 支持特定机型 H.265 硬解;

  •  [H.264/H.265 硬解码]Android 支持设置 Surface 模式硬解和普通模式硬解码;

  •  [缓冲时间设置]支持 buffer time 设置;

  •  [首屏秒开]支持首屏秒开模式;

  •  [低延迟模式]支持低延迟模式设置(公网 200~400ms);

  •  [复杂网络处理]支持断网重连等各种网络环境自动适配;

  •  [快速切换 URL]支持播放过程中,快速切换其他 URL,内容切换更快;

  •  [实时静音]支持播放过程中,实时静音/取消静音;

  •  [实时音量调节]支持播放过程中实时调节音量;

  •  [实时快照]支持播放过程中截取当前播放画面;

  •  [渲染角度]支持 0°,90°,180°和 270°四个视频画面渲染角度设置;

  •  [渲染镜像]支持水平反转、垂直反转模式设置;

  •  [等比例缩放]支持图像等比例缩放绘制(Android 设置 surface 模式硬解模式不支持);

  •  [实时下载速度更新]支持当前下载速度实时回调(支持设置回调时间间隔);

  •  [解码前视频数据回调]支持 H.264/H.265 数据回调;

  •  [解码后视频数据回调]支持解码后 YUV 数据回调;

  •  [解码前音频数据回调]支持 AAC/PCMA/PCMU/SPEEX 数据回调;

  •  [音视频自适应]支持播放过程中,音视频信息改变后自适应;

  •  [扩展录像功能]完美支持和录像 SDK 组合使用。

下面,我们看看技术实现细节,先说开始播放逻辑:

////  ViewController.m//  SmartiOSPlayerV2////  Author: daniusdk.com//  Created by daniulive on 2016/01/03.//- (void)playBtn:(UIButton *)button {        NSLog(@"playBtn only++");        button.selected = !button.selected;        if (button.selected)    {        if(is_playing_)            return;                [self InitPlayer];                //如需处理回调的用户数据+++++++++        __weak __typeof(self) weakSelf = self;                _smart_player_sdk.spUserDataCallBack = ^(int data_type, unsigned char *data, unsigned int size, unsigned long long timestamp, unsigned long long reserve1, long long reserve2, unsigned char *reserve3)        {            [weakSelf OnUserDataCallBack:data_type data:data size:size timestamp:timestamp reserve1:reserve1 reserve2:reserve2 reserve3:reserve3];        };                Boolean enableUserDataCallback = YES;        [_smart_player_sdk SmartPlayerSetUserDataCallback:enableUserDataCallback];         //如需处理回调的用户数据---------                if(![self StartPlayer])        {            NSLog(@"Call StartPlayer failed..");        }                [playbackButton setTitle:@"停止播放" forState:UIControlStateNormal];                is_playing_ = YES;    }    else    {        if ( !is_playing_ )            return;                [self StopPlayer];                if(!is_recording_)        {            [self UnInitPlayer];        }                [playbackButton setTitle:@"开始播放" forState:UIControlStateNormal];                is_mute_ = NO;        [muteButton setTitle:@"实时静音" forState:UIControlStateNormal];                is_playing_ = NO;    }}
复制代码

其中,InitPlayer 实现如下:

-(bool)InitPlayer{    NSLog(@"InitPlayer++");        if(is_inited_player_)    {        NSLog(@"InitPlayer: has inited before..");        return true;    }        //NSString* in_cid = @"";    //NSString* in_key = @"";        //[SmartPlayerSDK SmartPlayerSetSDKClientKey:in_cid in_key:in_key reserve1:0 reserve2:nil];        _smart_player_sdk = [[SmartPlayerSDK alloc] init];        if (_smart_player_sdk ==nil ) {        NSLog(@"SmartPlayerSDK init failed..");        return false;    }        if (playback_url_.length == 0) {        NSLog(@"playback url is nil..");        return false;    }        if (_smart_player_sdk.delegate == nil)    {        _smart_player_sdk.delegate = self;        NSLog(@"SmartPlayerSDK _player.delegate:%@", _smart_player_sdk);    }        NSInteger initRet = [_smart_player_sdk SmartPlayerInitPlayer];    if ( initRet != DANIULIVE_RETURN_OK )    {        NSLog(@"SmartPlayerSDK call SmartPlayerInitPlayer failed, ret=%ld", (long)initRet);        return false;    }        [_smart_player_sdk SmartPlayerSetPlayURL:playback_url_];    //[self try_set_rtsp_url:playback_url_];        //超低延迟模式设置    [_smart_player_sdk SmartPlayerSetLowLatencyMode:(NSInteger)is_low_latency_mode_];        //buffer time设置    if(buffer_time_ >= 0)    {        [_smart_player_sdk SmartPlayerSetBuffer:buffer_time_];    }        //快速启动模式设置    [_smart_player_sdk SmartPlayerSetFastStartup:(NSInteger)is_fast_startup_];        NSLog(@"[SmartPlayerV2]is_fast_startup_:%d, buffer_time_:%ld", is_fast_startup_, (long)buffer_time_);        //RTSP TCP还是UDP模式    [_smart_player_sdk SmartPlayerSetRTSPTcpMode:is_rtsp_tcp_mode_];     //设置RTSP超时时间    NSInteger rtsp_timeout = 10;    [_smart_player_sdk SmartPlayerSetRTSPTimeout:rtsp_timeout];        //设置RTSP TCP/UDP自动切换    NSInteger is_tcp_udp_auto_switch = 1;    [_smart_player_sdk SmartPlayerSetRTSPAutoSwitchTcpUdp:is_tcp_udp_auto_switch];        //快照设置 如需快照 参数传1    [_smart_player_sdk SmartPlayerSaveImageFlag:save_image_flag_];        //如需查看实时流量信息,可打开以下接口    NSInteger is_report = 1;    NSInteger report_interval = 3;    [_smart_player_sdk SmartPlayerSetReportDownloadSpeed:is_report report_interval:report_interval];        //录像端音频,是否转AAC后保存    NSInteger is_transcode = 1;    [_smart_player_sdk SmartPlayerSetRecorderAudioTranscodeAAC:is_transcode];        //录制MP4文件 是否录制视频    NSInteger is_record_video = 1;    [_smart_player_sdk SmartPlayerSetRecorderVideo:is_record_video];        //录制MP4文件 是否录制音频    NSInteger is_record_audio = 1;    [_smart_player_sdk SmartPlayerSetRecorderAudio:is_record_audio];            is_inited_player_ = YES;        NSLog(@"InitPlayer--");    return true;}
复制代码


​停止播放 StopPlayer 实现如下:

if (_smart_player_sdk != nil){    [_smart_player_sdk SmartPlayerStop];}
if (!is_audio_only_) { if (_glView != nil) { [_glView removeFromSuperview]; [SmartPlayerSDK SmartPlayeReleasePlayView:(__bridge void *)(_glView)]; _glView = nil; }}
NSLog(@"StopPlayer--");return true;
复制代码


UnInitPlayer 实现如下:

if (_smart_player_sdk != nil){    [_smart_player_sdk SmartPlayerUnInitPlayer];        if (_smart_player_sdk.delegate != nil)    {        _smart_player_sdk.delegate = nil;    }        _smart_player_sdk = nil;}
is_inited_player_ = NO;
NSLog(@"UnInitPlayer--");return true;
复制代码


实时录像:

- (void)RecorderBtn:(UIButton *)button {        NSLog(@"record Stream only++");        button.selected = !button.selected;        if (button.selected)    {        if(is_recording_)            return;                [self InitPlayer];                //设置录像目录        NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);        NSString *recorderDir = [paths objectAtIndex:0];                if([_smart_player_sdk SmartPlayerSetRecorderDirectory:recorderDir] != DANIULIVE_RETURN_OK)        {            NSLog(@"Call SmartPlayerSetRecorderDirectory failed..");        }                //每个录像文件大小        NSInteger size = 200;        if([_smart_player_sdk SmartPlayerSetRecorderFileMaxSize:size] != DANIULIVE_RETURN_OK)        {            NSLog(@"Call SmartPlayerSetRecorderFileMaxSize failed..");        }                [_smart_player_sdk SmartPlayerStartRecorder];        [recButton setTitle:@"停止录像" forState:UIControlStateNormal];                is_recording_ = YES;    }    else    {        [_smart_player_sdk SmartPlayerStopRecorder];        [recButton setTitle:@"开始录像" forState:UIControlStateNormal];                if(!is_playing_)        {            [self UnInitPlayer];        }                is_recording_ = NO;    }}
复制代码


实时快照:

- (void)SaveImageBtn:(UIButton *)button {    if ( _smart_player_sdk != nil )    {        //设置快照目录        NSLog(@"[SaveImageBtn] path++");                NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);        NSString *saveImageDir = [paths objectAtIndex:0];                NSLog(@"[SaveImageBtn] path: %@", saveImageDir);                NSString* symbol = @"/";                NSString* png = @".png";                // 1.创建时间        NSDate *datenow = [NSDate date];        // 2.创建时间格式化        NSDateFormatter *formatter = [[NSDateFormatter alloc] init];        // 3.指定格式        formatter.dateFormat = @"yyyyMMdd_HHmmss";        // 4.格式化时间        NSString *timeSp = [formatter stringFromDate:datenow];                NSString* image_name =  [saveImageDir stringByAppendingString:symbol];                image_name = [image_name stringByAppendingString:timeSp];                image_name = [image_name stringByAppendingString:png];                NSLog(@"[SaveImageBtn] image_name: %@", image_name);                [_smart_player_sdk SmartPlayerSaveCurImage:image_name];    }}
复制代码


​Event 回调处理如下:

- (NSInteger) handleSmartPlayerEvent:(NSInteger)nID param1:(unsigned long long)param1 param2:(unsigned long long)param2 param3:(NSString*)param3 param4:(NSString*)param4 pObj:(void *)pObj;{    NSString* player_event = @"";    NSString* lable = @"";        if (nID == EVENT_DANIULIVE_ERC_PLAYER_STARTED) {        player_event = @"[event]开始播放..";    }    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CONNECTING)    {        player_event = @"[event]连接中..";    }    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CONNECTION_FAILED)    {        player_event = @"[event]连接失败..";    }    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CONNECTED)    {        player_event = @"[event]已连接..";    }    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_DISCONNECTED)    {        player_event = @"[event]断开连接..";    }    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_STOP)    {        player_event = @"[event]停止播放..";    }    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_RESOLUTION_INFO)    {        NSString *str_w = [NSString stringWithFormat:@"%ld", (long)param1];        NSString *str_h = [NSString stringWithFormat:@"%ld", (long)param2];                lable = @"[event]视频解码分辨率信息: ";        player_event = [lable stringByAppendingFormat:@"%@*%@", str_w, str_h];    }    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_NO_MEDIADATA_RECEIVED)    {        player_event = @"[event]收不到RTMP数据..";    }    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_SWITCH_URL)    {        player_event = @"[event]快速切换url..";    }    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CAPTURE_IMAGE)    {        if ((int)param1 == 0)        {            NSLog(@"[event]快照成功: %@", param3);            lable = @"[event]快照成功:";            player_event = [lable stringByAppendingFormat:@"%@", param3];                        tmp_path_ = param3;                        image_path_ = [ UIImage imageNamed:param3];                        UIImageWriteToSavedPhotosAlbum(image_path_, self, @selector(image:didFinishSavingWithError:contextInfo:), NULL);        }        else        {            lable = @"[event]快照失败";            player_event = [lable stringByAppendingFormat:@"%@", param3];        }    }    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_RECORDER_START_NEW_FILE)    {        lable = @"[event]录像写入新文件..文件名:";        player_event = [lable stringByAppendingFormat:@"%@", param3];    }    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_ONE_RECORDER_FILE_FINISHED)    {        lable = @"一个录像文件完成..文件名:";        player_event = [lable stringByAppendingFormat:@"%@", param3];    }    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_START_BUFFERING)    {        //NSLog(@"[event]开始buffer..");    }    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_BUFFERING)    {        NSLog(@"[event]buffer百分比: %lld", param1);    }    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_STOP_BUFFERING)    {        //NSLog(@"[event]停止buffer..");    }    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_DOWNLOAD_SPEED)    {        NSInteger speed_kbps = (NSInteger)param1*8/1000;        NSInteger speed_KBs = (NSInteger)param1/1024;                lable = @"[event]download speed :";        player_event = [lable stringByAppendingFormat:@"%ld kbps - %ld KB/s", (long)speed_kbps, (long)speed_KBs];    }    else if(nID == EVENT_DANIULIVE_ERC_PLAYER_RTSP_STATUS_CODE)    {        lable = @"[event]RTSP status code received:";        player_event = [lable stringByAppendingFormat:@"%ld", (long)param1];                dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{            dispatch_async(dispatch_get_main_queue(), ^{                UIAlertController *aleView=[UIAlertController alertControllerWithTitle:@"RTSP错误状态" message:player_event preferredStyle:UIAlertControllerStyleAlert];                UIAlertAction *action_ok=[UIAlertAction actionWithTitle:@"确定" style:UIAlertActionStyleCancel handler:nil];                [aleView addAction:action_ok];                                [self presentViewController:aleView animated:YES completion:nil];            });        });    }    else if(nID == EVENT_DANIULIVE_ERC_PLAYER_NEED_KEY)    {        player_event = @"[event]RTMP加密流,请设置播放需要的Key..";    }    else if(nID == EVENT_DANIULIVE_ERC_PLAYER_KEY_ERROR)    {        player_event = @"[event]RTMP加密流,Key错误,请重新设置..";    }    else        NSLog(@"[event]nID:%lx", (long)nID);        NSString* player_event_tag = @"当前状态:";    NSString* event = [player_event_tag stringByAppendingFormat:@"%@", player_event];        if ( player_event.length != 0)    {        NSLog(@"%@", event);    }        dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{            dispatch_async(dispatch_get_main_queue(), ^{                self.textPlayerEventLabel.text = event;            });    });        return 0;}
复制代码

总结

iOS 平台播放,由于设备和系统比较单一,所以优先考虑硬解码,除了基础播放外,我们还实现了实时快照、实时录像、实时回调 YUV 数据、实时音量调节等,实际体验下来,iOS 平台 RTMP 和 RTSP,可以轻松毫秒级,感兴趣的开发者,可以和我单独交流。


用户头像

致力于跨平台RTMP|RTSP推流、播放 2021-11-23 加入

还未添加个人简介

评论

发布
暂无评论
iOS平台如何实现毫秒级延迟的RTMP|RTSP播放器_IOS RTSP播放器_音视频牛哥_InfoQ写作社区