Bug in sending heart-beat / silence frame / jitterbuffer

TS
Tobias Schneider
Mon, Jul 23, 2018 2:31 PM

If stream input is not connected to any transmitter, it will send
silence frames through RTP to keep NAT active. Unfortunately this leads
to a jitterbuffer filled to capacity if early media is used but audio
stream is not connected to any sink (e.g. if only video stream is being
connected in early media) because audio frames are not fetched from
jitterbuffer. If users than connects audio stream with audio device
after call is accepted, they will encounter several discarded frames and
therefore frame losts, even though the network stream is fine and does
not have any lost frames itself, just because jitterbuffer algorithm now
tries to reduce jitterbuffer size.

I deeply analyzed this using G722 codec with PJProject 2.4.5 but the
problem seems to be still existent in current version of PJProject. I
made two patches to resolve this issue but I am not sure, which one
should be preferred, so I want you to check whether one of those patches
is correct and can be used safely. I will send those patches as several
posting.

If stream input is not connected to any transmitter, it will send silence frames through RTP to keep NAT active. Unfortunately this leads to a jitterbuffer filled to capacity if early media is used but audio stream is not connected to any sink (e.g. if only video stream is being connected in early media) because audio frames are not fetched from jitterbuffer. If users than connects audio stream with audio device after call is accepted, they will encounter several discarded frames and therefore frame losts, even though the network stream is fine and does not have any lost frames itself, just because jitterbuffer algorithm now tries to reduce jitterbuffer size. I deeply analyzed this using G722 codec with PJProject 2.4.5 but the problem seems to be still existent in current version of PJProject. I made two patches to resolve this issue but I am not sure, which one should be preferred, so I want you to check whether one of those patches is correct and can be used safely. I will send those patches as several posting.