Playing a busy sound on call disconnect (iOS)

M
Marc
Fri, Aug 12, 2016 11:39 PM

Is there any built-in way to play a busy sound effect for a few seconds after call disconnect?

In my case trying to play a sound at call state PJSIP_INV_STATE_DISCONNECTED (checking for call_info->last_status to be 486 (PJSIP_SC_BUSY_HERE)).

The problem is that if I play a sound using AVAudioPlayer, PJSIP will disable the shared audio session of the app in coreaudio_dev.m ca_stream_stop(), [stream->sess setActive:NO error:nil]

This causes our AVAudioPlayer to fail to playback the sound effect we want, with a visible warning printed to the console.

Any ideas before we have to modify PJSIP to not modify a shared resource like AVAudioSession without warning?

Is there a way to properly do this, perhaps a notification when ca_stream_stop() ends so we can restart the AVAudioSession for our purposes?

Thanks

  • Marc

Hi Marc,

The green frames in the beginning of a video call is likely caused by
missing keyframe. Keyframe requests are sent regularly every
PJSUA_VID_REQ_KEYFRAME_INTERVAL, and application can also explicitly
send keyframe using the API pjsua_call_set_vid_strm() with op
PJSUA_CALL_VID_STRM_SEND_KEYFRAME. We also have a pending ticket for
this #1910 (https://trac.pjsip.org/repos/ticket/1910) which aims to do
this automatically within the library.

To change the color to black, instead of green, you can modify
pjmedia/src/pjmedia/vid_port.c, in function pjmedia_vid_port_create(),
where the initial buffer of the video frame is allocated. Below is an
example of how to set the frame to black in I420 format. For other YUV
formats, which have different ways to represent black, you may need to
modify the code accordingly.

if (need_frame_buf) {
.....
vp->frm_buf->type = PJMEDIA_FRAME_TYPE_NONE;
/* Insert the below code after you find the above line ---- /
/
Note: this will only work for I420 format /
if (vparam.fmt.id == PJMEDIA_FORMAT_I420) {
pj_memset(vp->frm_buf->buf, 0x80, vp->frm_buf_size);
pj_bzero(vp->frm_buf->buf,
vparam.fmt.det.vid.size.w
vparam.fmt.det.vid.size.h);
}
/* ---- */

Unfortunately, currently there's no notification for the first decoded
frame/keyframe received.

Best regards,
Ming

On Thu, Jul 21, 2016 at 7:04 AM, Marc marc@taplightsoftware.com wrote:

When making a call using OpenH264, we get a few seconds of green frames before the first video frames show up successfully.

It seems that this isn?t an OpenH264 (1.0.0 as that is the only version that compiles with PJSIP) issue, but a PJSIP one.

As a side note, it seems that later versions of OpenH264 have switched to gray: https://github.com/cisco/openh264/issues/1163

I have commented out oh264_codec_decode and oh264_got_decoded_frame to just return Pj_SUCCESS and do nothing else, and I still get the green screen and obviously no video.

I have also commented out decode_frame in vid_stream.c

This is on iOS using CAEAGLayer as the renderer.

0 in YUV colorspace turns out to be green. Where exactly should I look to fix this issue so we can get either one of 2 things.

  1. The green should become black (0x80 in YUV)
  2. A proper notification that we actually have the first decoded frame (or the first key frame) from the RTP H264 stream and we should display the video window.

Thanks


Visit our blog: http://blog.pjsip.org

pjsip mailing list
pjsip@lists.pjsip.org
http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org

Is there any built-in way to play a busy sound effect for a few seconds after call disconnect? In my case trying to play a sound at call state PJSIP_INV_STATE_DISCONNECTED (checking for call_info->last_status to be 486 (PJSIP_SC_BUSY_HERE)). The problem is that if I play a sound using AVAudioPlayer, PJSIP will disable the shared audio session of the app in coreaudio_dev.m ca_stream_stop(), [stream->sess setActive:NO error:nil] This causes our AVAudioPlayer to fail to playback the sound effect we want, with a visible warning printed to the console. Any ideas before we have to modify PJSIP to not modify a shared resource like AVAudioSession without warning? Is there a way to properly do this, perhaps a notification when ca_stream_stop() ends so we can restart the AVAudioSession for our purposes? Thanks - Marc > Hi Marc, > > The green frames in the beginning of a video call is likely caused by > missing keyframe. Keyframe requests are sent regularly every > PJSUA_VID_REQ_KEYFRAME_INTERVAL, and application can also explicitly > send keyframe using the API pjsua_call_set_vid_strm() with op > PJSUA_CALL_VID_STRM_SEND_KEYFRAME. We also have a pending ticket for > this #1910 (https://trac.pjsip.org/repos/ticket/1910) which aims to do > this automatically within the library. > > To change the color to black, instead of green, you can modify > pjmedia/src/pjmedia/vid_port.c, in function pjmedia_vid_port_create(), > where the initial buffer of the video frame is allocated. Below is an > example of how to set the frame to black in I420 format. For other YUV > formats, which have different ways to represent black, you may need to > modify the code accordingly. > > if (need_frame_buf) { > ..... > vp->frm_buf->type = PJMEDIA_FRAME_TYPE_NONE; > /* Insert the below code after you find the above line ---- */ > /* Note: this will only work for I420 format */ > if (vparam.fmt.id == PJMEDIA_FORMAT_I420) { > pj_memset(vp->frm_buf->buf, 0x80, vp->frm_buf_size); > pj_bzero(vp->frm_buf->buf, > vparam.fmt.det.vid.size.w*vparam.fmt.det.vid.size.h); > } > /* ---- */ > > Unfortunately, currently there's no notification for the first decoded > frame/keyframe received. > > Best regards, > Ming > > On Thu, Jul 21, 2016 at 7:04 AM, Marc <marc@taplightsoftware.com> wrote: >> When making a call using OpenH264, we get a few seconds of green frames before the first video frames show up successfully. >> >> It seems that this isn?t an OpenH264 (1.0.0 as that is the only version that compiles with PJSIP) issue, but a PJSIP one. >> >> As a side note, it seems that later versions of OpenH264 have switched to gray: https://github.com/cisco/openh264/issues/1163 >> >> I have commented out oh264_codec_decode and oh264_got_decoded_frame to just return Pj_SUCCESS and do nothing else, and I still get the green screen and obviously no video. >> >> I have also commented out decode_frame in vid_stream.c >> >> This is on iOS using CAEAGLayer as the renderer. >> >> 0 in YUV colorspace turns out to be green. Where exactly should I look to fix this issue so we can get either one of 2 things. >> >> 1. The green should become black (0x80 in YUV) >> 2. A proper notification that we actually have the first decoded frame (or the first key frame) from the RTP H264 stream and we should display the video window. >> >> Thanks >> >> >> _______________________________________________ >> Visit our blog: http://blog.pjsip.org >> >> pjsip mailing list >> pjsip@lists.pjsip.org >> http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org