视频一方是recvonly时Freeswitch不转包问题记录
问题：非代理模式下，视频媒体主叫如果是recvonly，主叫收不到被叫发过来的视频包；
分析：主叫是recvonly时，被叫是sendonly，最开始以为是跟主被叫的rmode有关系，最后发现还真是有关系；
tcpdump抓包分析sendonly方明确有发包，但并没有转出去的包，所以分析可以明确是freeswitch没有转包！
freeswitch为什么没有转包呢，连同发现有下面几个问题：
1、发送方的payload type和协商的不一致，freeswitch过滤了payload type不一致的包；
#if 1//lyz for payload error.
if (bytes && !rtp_session->flags[SWITCH_RTP_FLAG_PROXY_MEDIA] && !rtp_session->flags[SWITCH_RTP_FLAG_UDPTL] &&
rtp_session->last_rtp_hdr.pt != 13 &&
rtp_session->last_rtp_hdr.pt != rtp_session->recv_te &&
rtp_session->last_rtp_hdr.pt != rtp_session->cng_pt) {
int accept_packet = 1;
int required_pt = 0;
if (!(rtp_session->rtp_bugs & RTP_BUG_ACCEPT_ANY_PAYLOAD) &&
!(rtp_session->rtp_bugs & RTP_BUG_ACCEPT_ANY_PACKETS) && rtp_session->pmaps && *rtp_session->pmaps) {
payload_map_t *pmap;
accept_packet = 0;
switch_mutex_lock(rtp_session->flag_mutex);
for (pmap = *rtp_session->pmaps; pmap && pmap->allocated; pmap = pmap->next) {
if (ntohl(*(int *)(b+4)) == ZRTP_MAGIC_COOKIE) {
accept_packet = 1;
break;
}
if (!pmap->negotiated) {
continue;
}
if (rtp_session->last_rtp_hdr.pt == pmap->pt) {
accept_packet = 1;
if (pmapP) {
*pmapP = pmap;
}
break;
}else{
required_pt = pmap->pt;
}
}
switch_mutex_unlock(rtp_session->flag_mutex);
}
if (!accept_packet) {
if (rtp_session->recv_msg.header.pt != required_pt && required_pt != 0){
rtp_session->recv_msg.header.pt = required_pt;//lyz force modify
}else{
switch_log_printf(SWITCH_CHANNEL_SESSION_LOG(rtp_session->session), SWITCH_LOG_INFO,
"Invalid Packet SEQ: %d TS: %d PT:%d ignored\n",
ntohs(rtp_session->recv_msg.header.seq), ntohl(rtp_session->last_rtp_hdr.ts), rtp_session->last_rtp_hdr.pt);
*bytes = 0;
}
}
}
#endif//lyz for payload type 不一致2、freeswitch接收到一方的包，解码完后，并没有转发给对方；
SWITCH_DECLARE(switch_status_t) switch_core_session_write_video_frame(switch_core_session_t *session, switch_frame_t *frame, switch_io_flag_t flags,
int stream_id)
#if 0//del for need write frame to recvonly
if (switch_core_session_media_flow(session, SWITCH_MEDIA_TYPE_VIDEO) == SWITCH_MEDIA_FLOW_RECVONLY || switch_core_session_media_flow(session, SWITCH_MEDIA_TYPE_VIDEO) == SWITCH_MEDIA_FLOW_INACTIVE) {
switch_log_printf(SWITCH_CHANNEL_SESSION_LOG(session), SWITCH_LOG_INFO, "Writing video to RECVONLY/INACTIVE session\n");
return SWITCH_STATUS_SUCCESS;
}
#else
if (switch_core_session_media_flow(session, SWITCH_MEDIA_TYPE_VIDEO) == SWITCH_MEDIA_FLOW_SENDONLY || switch_core_session_media_flow(session, SWITCH_MEDIA_TYPE_VIDEO) == SWITCH_MEDIA_FLOW_INACTIVE) {
switch_log_printf(SWITCH_CHANNEL_SESSION_LOG(session), SWITCH_LOG_INFO, "Writing video to RECVONLY/INACTIVE session\n");
return SWITCH_STATUS_SUCCESS;
}
#endif//为啥代理模式下freeswitch能转发给recvonly的主叫方呢，原因是代理模式下并没有设置media_flow，所以并没有走这个分支；
3、freeswitch作为背靠背代理，如何进行调试？
所谓背靠背代理，就是每通呼叫会创建有aleg和bleg两个会话，每个会话都有一个channel处理rtp媒体数据；
aleg接收a发送的数据，然后转发给b；
bleg接收b发过来的数据，然后转发给a；
所以相同的程序a_leg和b_leg会同时运行，那么如何调试a_leg而不是b_leg的问题呢？
方法1：gdb attach绑定到freeswitch的进程，然后设置条件断点进行调试分析；
方法2：打印switch_channel_get_name可以区分不同的session和channel；
switch_log_printf(SWITCH_CHANNEL_SESSION_LOG(vh->session_a), SWITCH_LOG_DEBUG, "%s video thread ended.\n", switch_channel_get_name(channel));
2022-05-10 14:21:31.455431 [DEBUG] switch_core_media.c:7594 sofia/internal/4002-0xb202c1f8@192.168.16.102:50450 Video thread ended
freeswitch代码逻辑还是相当复杂的，一个函数通常都是上千行的代码，所以跟踪分析问题并不是那么容易，最好是gdb结合log打印跟踪分析问题；
4、视频的转发堆栈；
#0 switch_core_session_write_video_frame (session=0x15039c8, frame=0x7f343007f920, flags=flags@entry=0, stream_id=stream_id@entry=0) at src/switch_core_media.c:14554 #1 0x00007f345cdb416d in video_bridge_thread (session=0x7f3430068db8, obj=0x7f343409e790) at src/switch_ivr_bridge.c:242 #2 0x00007f345cd707ce in video_helper_thread (thread=<optimized out>, obj=0x7f3430084470) at src/switch_core_media.c:7548 #3 0x00007f345d03c74c in dummy_worker (opaque=0x7f343c1ca4e0) at threadproc/unix/thread.c:151 #4 0x00007f345a74f17a in start_thread () from /lib64/libpthread.so.0 #5 0x00007f3459d01dc3 in clone () from /lib64/libc.so.6 (gdb) c
视频收包，解码逻辑：
Thread 33 "freeswitch" hit Breakpoint 1, switch_h264_decode (codec=0x2425598, frame=0x2425500) at avcodec.c:1806 1806 h264_codec_context_t *context = (h264_codec_context_t *)codec->private_info; (gdb) bt #0 switch_h264_decode (codec=0x2425598, frame=0x2425500) at avcodec.c:1806 #1 0x00007efc2c450422 in switch_core_codec_decode_video (codec=0x2425598, frame=0x2425500) at src/switch_core_codec.c:864 #2 0x00007efc2c4a2d01 in switch_core_session_read_video_frame (session=0x26a6c98, frame=frame@entry=0x7efc04072be8, flags=flags@entry=0, stream_id=stream_id@entry=0) at src/switch_core_media.c:14974 #3 0x00007efc2c4e7055 in video_bridge_thread (session=0x26a6c98, obj=0x7efbfb142790) at src/switch_ivr_bridge.c:227 #4 0x00007efc2c4a372e in video_helper_thread (thread=<optimized out>, obj=0x242a050) at src/switch_core_media.c:7548 #5 0x00007efc2c76f6ac in dummy_worker (opaque=0x7efc000789b0) at threadproc/unix/thread.c:151 #6 0x00007efc29e8217a in start_thread () from /lib64/libpthread.so.0 #7 0x00007efc29434dc3 in clone () from /lib64/libc.so.6
音频媒体流的转发堆栈：
Thread 31 "freeswitch" hit Breakpoint 1, switch_core_media_write_frame (session=session@entry=0x7f8d6c0dda38, frame=frame@entry=0x2014148,
flags=flags@entry=0, stream_id=stream_id@entry=0, type=type@entry=SWITCH_MEDIA_TYPE_AUDIO) at src/switch_core_media.c:3419
3419 switch_assert(session);
(gdb) bt
#0 switch_core_media_write_frame (session=session@entry=0x7f8d6c0dda38, frame=frame@entry=0x2014148, flags=flags@entry=0,
stream_id=stream_id@entry=0, type=type@entry=SWITCH_MEDIA_TYPE_AUDIO) at src/switch_core_media.c:3419
#1 0x00007f8d6aaf36d3 in sofia_write_frame (session=0x7f8d6c0dda38, frame=0x2014148, flags=0, stream_id=0) at mod_sofia.c:1235
#2 0x00007f8d75d0fb9b in perform_write (session=session@entry=0x7f8d6c0dda38, frame=0x2014148, flags=0, stream_id=0)
at src/switch_core_media.c:7157
#3 0x00007f8d75d3e47e in switch_core_session_write_frame (session=session@entry=0x7f8d6c0dda38, frame=<optimized out>, flags=flags@entry=0,
stream_id=stream_id@entry=0) at src/switch_core_media.c:16309
#4 0x00007f8d75d822fe in audio_bridge_thread (obj=obj@entry=0x7f8d6c1dce78, thread=0x0) at src/switch_ivr_bridge.c:823
#5 0x00007f8d75d84756 in switch_ivr_multi_threaded_bridge (session=session@entry=0x21a39c8, peer_session=0x7f8d6c0dda38,
input_callback=<optimized out>, session_data=session_data@entry=0x0, peer_session_data=peer_session_data@entry=0x0)
at src/switch_ivr_bridge.c:1797
#6 0x00007f8d4fbeee68 in audio_bridge_function (session=<optimized out>, data=<optimized out>) at mod_dptools.c:3672
#7 0x00007f8d75cfdcaf in switch_core_session_exec (session=session@entry=0x21a39c8,
application_interface=application_interface@entry=0x1e11f90, arg=arg@entry=0x7f8d6c12bcc0 "user/4002@${domain_name}")
at src/switch_core_session.c:2891
#8 0x00007f8d75cfe33f in switch_core_session_execute_application_get_flags (session=session@entry=0x21a39c8, app=0x7f8d6c12bcb8 "bridge",
arg=0x7f8d6c12bcc0 "user/4002@${domain_name}", flags=flags@entry=0x0) at src/switch_core_session.c:2756
#9 0x00007f8d75d02682 in switch_core_standard_on_execute (session=0x21a39c8) at src/switch_core_state_machine.c:354
#10 switch_core_session_run (session=0x21a39c8) at src/switch_core_state_machine.c:651
#11 0x00007f8d75cfb6ae in switch_core_session_thread (thread=<optimized out>, obj=0x21a39c8) at src/switch_core_session.c:1709
#12 0x00007f8d75cf70c3 in switch_core_session_thread_pool_worker (thread=0x1fc7f20, obj=<optimized out>) at src/switch_core_session.c:1772
#13 0x00007f8d7600878c in dummy_worker (opaque=0x1fc7f20) at threadproc/unix/thread.c:151
#14 0x00007f8d7371b17a in start_thread () from /lib64/libpthread.so.0
#15 0x00007f8d72ccddc3 in clone () from /lib64/libc.so.65、只有一路流的情况下，freeswitch转码，并且转发给rtmp服务器一路流，发现cpu占比还是很高的：
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 696684 root -2 -10 848068 144700 24264 S 15.0 3.6 23:25.80 freeswitch
视频编码都是h264，虽然开启了record，是不是也可以不做解码和编码的处理呢，下一步就是修改freeswitch在默认模式下，对视频的解码、编码处理逻辑做出修改，降低freeswitch视频转码产生的cpu持续高负载的情况；
-------------------广告线---------------
项目、合作，欢迎勾搭，邮箱：promall@qq.com
本文为呱牛笔记原创文章,转载无需和我联系,但请注明来自呱牛笔记 ,it3q.com
- 上一篇： Hi3516ev300调通rtp取视频流
- 下一篇： 鸿蒙liteos-a初体验
请先登录后发表评论
- 最新评论
- 总共0条评论
