ALSA audio buffer underflow/overflow with AEC and far-end to near-end signal delay

MR
Mario Raffin
Thu, Feb 11, 2016 5:01 PM

Dear All,
I am developing an application using pjsip on a custom board based
on an ARM CORTEX A8 CPU.

At the beginning I didn't defined PJMEDIA_AUDIO_DEV_HAS_ALSA and my
software used the portaudio interface.
The latency was very low and the Speex AEC worked pretty good.
The problem was that the portaudio interface was not very stable and
after a while the audio crashed keeping the device busy (too much irq
requests, I suppose).

Then I switched to the ALSA interface. Here I can not decrease the
latency below the 100ms (from pjsystest), but it is enough.

When the AEC is disabled the audio is good, apart from the echo.
When I enable the Speex AEC, I can not achieve a good echo cancellation
but the worst thing is that after few seconds, I get a lot of buffer
errors as shown below.

I have these few questions:

  1. how can I avoid these buffer issues? (maybe increasing the priority
    of the audio thread?)
    In any case I don't undertand the capture overrun: each captured frame
    corresponds to a played frame. If the input samples are not enough
    (there could be many reasons for that), it could be better to play an
    ampty frame but the capure thread should not overrun, isn't it?

  2. I know quite well the Speex aec. The far-end (speaker) and the
    near-end (mic) signals must be aligned properly.
    After I set the ALSA buffer sizes with the rec latency, and play latency
    parameters, I get a certain far-end to near-end delay, 106ms in my case.
    Now I have to tell to the AEC to delay the far end signal of more of 106
    ms. How can I do that?

  3. The far-end to near-end latency depends on the buffer management,
    when I open the audio channel the latency is only 10ms but every time a
    new buffer is allocated, this latency increases of 20ms. Usually the
    number of buffers allocated at the beginning is constant but it should
    be nice to allocate them once, when audio is opened.

Here what I get with pjsystest, thesting the AEC:

AEC/AES Test
AEC/AES test is running. Press OK to stop this test.
1:OK
05:23:20.316  Master/sound !Underflow, buf_cnt=0, will generate 1 frame
05:23:20.318  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:20.325  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:20.345  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:26.105    alsa_dev.c !ca_thread_func: overrun!
05:23:26.124    alsa_dev.c !pb_thread_func: underrun!
05:23:26.354    alsa_dev.c !ca_thread_func: overrun!
05:23:26.400  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:26.653    alsa_dev.c !ca_thread_func: overrun!
05:23:26.654    alsa_dev.c !pb_thread_func: underrun!
05:23:26.655  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:26.656  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:26.658  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:26.671  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:26.711  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:30.332    alsa_dev.c  pb_thread_func: underrun!
05:23:30.335  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:30.350  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:38.771    alsa_dev.c  pb_thread_func: underrun!
05:23:38.774  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:38.790  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:39.810    alsa_dev.c !pb_thread_func: underrun!
05:23:39.828  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:39.889    alsa_dev.c !pb_thread_func: underrun!
05:23:39.908  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:39.969    alsa_dev.c !pb_thread_func: underrun!
05:23:39.987  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:40.049    alsa_dev.c !pb_thread_func: underrun!
05:23:40.068  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:40.449    alsa_dev.c !pb_thread_func: underrun!
05:23:40.467  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:40.529    alsa_dev.c !pb_thread_func: underrun!
05:23:40.547  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:40.609    alsa_dev.c !pb_thread_func: underrun!
05:23:40.627  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:40.689    alsa_dev.c !pb_thread_func: underrun!
05:23:40.708  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:41.089    alsa_dev.c !pb_thread_func: underrun!
05:23:41.107  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:41.169    alsa_dev.c !pb_thread_func: underrun!
05:23:41.187  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:41.248    alsa_dev.c !pb_thread_func: underrun!
05:23:41.267  Master/sound  Underflow, buf_cnt=0, will generate 1 frame
05:23:41.329    alsa_dev.c !pb_thread_func: underrun!
05:23:41.348  Master/sound  Underflow, buf_cnt=0, will generate 1 frame

Thank you in advance for the support.

Best Regards,

Mario Raffin

Dear All, I am developing an application using pjsip on a custom board based on an ARM CORTEX A8 CPU. At the beginning I didn't defined PJMEDIA_AUDIO_DEV_HAS_ALSA and my software used the portaudio interface. The latency was very low and the Speex AEC worked pretty good. The problem was that the portaudio interface was not very stable and after a while the audio crashed keeping the device busy (too much irq requests, I suppose). Then I switched to the ALSA interface. Here I can not decrease the latency below the 100ms (from pjsystest), but it is enough. When the AEC is disabled the audio is good, apart from the echo. When I enable the Speex AEC, I can not achieve a good echo cancellation but the worst thing is that after few seconds, I get a lot of buffer errors as shown below. I have these few questions: 1) how can I avoid these buffer issues? (maybe increasing the priority of the audio thread?) In any case I don't undertand the capture overrun: each captured frame corresponds to a played frame. If the input samples are not enough (there could be many reasons for that), it could be better to play an ampty frame but the capure thread should not overrun, isn't it? 2) I know quite well the Speex aec. The far-end (speaker) and the near-end (mic) signals must be aligned properly. After I set the ALSA buffer sizes with the rec latency, and play latency parameters, I get a certain far-end to near-end delay, 106ms in my case. Now I have to tell to the AEC to delay the far end signal of more of 106 ms. How can I do that? 3) The far-end to near-end latency depends on the buffer management, when I open the audio channel the latency is only 10ms but every time a new buffer is allocated, this latency increases of 20ms. Usually the number of buffers allocated at the beginning is constant but it should be nice to allocate them once, when audio is opened. Here what I get with pjsystest, thesting the AEC: AEC/AES Test AEC/AES test is running. Press OK to stop this test. 1:OK 05:23:20.316 Master/sound !Underflow, buf_cnt=0, will generate 1 frame 05:23:20.318 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:20.325 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:20.345 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:26.105 alsa_dev.c !ca_thread_func: overrun! 05:23:26.124 alsa_dev.c !pb_thread_func: underrun! 05:23:26.354 alsa_dev.c !ca_thread_func: overrun! 05:23:26.400 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:26.653 alsa_dev.c !ca_thread_func: overrun! 05:23:26.654 alsa_dev.c !pb_thread_func: underrun! 05:23:26.655 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:26.656 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:26.658 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:26.671 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:26.711 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:30.332 alsa_dev.c pb_thread_func: underrun! 05:23:30.335 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:30.350 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:38.771 alsa_dev.c pb_thread_func: underrun! 05:23:38.774 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:38.790 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:39.810 alsa_dev.c !pb_thread_func: underrun! 05:23:39.828 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:39.889 alsa_dev.c !pb_thread_func: underrun! 05:23:39.908 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:39.969 alsa_dev.c !pb_thread_func: underrun! 05:23:39.987 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:40.049 alsa_dev.c !pb_thread_func: underrun! 05:23:40.068 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:40.449 alsa_dev.c !pb_thread_func: underrun! 05:23:40.467 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:40.529 alsa_dev.c !pb_thread_func: underrun! 05:23:40.547 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:40.609 alsa_dev.c !pb_thread_func: underrun! 05:23:40.627 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:40.689 alsa_dev.c !pb_thread_func: underrun! 05:23:40.708 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:41.089 alsa_dev.c !pb_thread_func: underrun! 05:23:41.107 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:41.169 alsa_dev.c !pb_thread_func: underrun! 05:23:41.187 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:41.248 alsa_dev.c !pb_thread_func: underrun! 05:23:41.267 Master/sound Underflow, buf_cnt=0, will generate 1 frame 05:23:41.329 alsa_dev.c !pb_thread_func: underrun! 05:23:41.348 Master/sound Underflow, buf_cnt=0, will generate 1 frame Thank you in advance for the support. Best Regards, Mario Raffin