Android Issue on Ticket#1861(Video capture orientation)

XD
Xiaoming Deng
Tue, May 9, 2017 7:18 AM

Hi,

The change set for #1861 https://trac.pjsip.org/repos/ticket/1861 on
Android does not work well. Testing on org.pjsip.pjsua2.app get could not
get the "proper" orientation result,  i.e: head upside whatever the device
orientation is.

The culprit should be the I420Rotate() invoked
by pjmedia_vid_dev_conv_resize_and_rotate() in
pjmedia/src/pjmedia-videodev/util.c
https://trac.pjsip.org/repos/browser/pjproject/trunk/pjmedia/src/pjmedia-videodev/util.c?rev=5118.
The libyuv document
https://chromium.googlesource.com/libyuv/libyuv/+/master/docs/rotation.md#I420Rotatesays
"Rotate by 180 can be done in place, but 90 and 270 can not."

In addition, change set # https://trac.pjsip.org/repos/changeset/5118
5138  https://trac.pjsip.org/repos/changeset/5138#file1introduces a
convert for the stream. Another convert, however, was created for
pjmedia_vid_port and is dependent on the source video size too.

An alternative solution is to publish a PJMEDIA_EVENT_FMT_CHANGED event on
PJMEDIA_DIR_ENCODING and finally invoke client_port_event_cb() in
vid_port.c, and the client_port_event_cb() should be amended to NOT
overwrite the destination size on PJMEDIA_DIR_ENCODING. The stream convert
is not required.

Is there any problem or potential risk on this solution?

--
Deng Xiaoming
dengxiaoming@gmail.com

Hi, The change set for #1861 <https://trac.pjsip.org/repos/ticket/1861> on Android does not work well. Testing on org.pjsip.pjsua2.app get could not get the "proper" orientation result, i.e: head upside whatever the device orientation is. The culprit should be the I420Rotate() invoked by pjmedia_vid_dev_conv_resize_and_rotate() in pjmedia/src/pjmedia-videodev/util.c <https://trac.pjsip.org/repos/browser/pjproject/trunk/pjmedia/src/pjmedia-videodev/util.c?rev=5118>. The libyuv document <https://chromium.googlesource.com/libyuv/libyuv/+/master/docs/rotation.md#I420Rotate>says "Rotate by 180 can be done in place, but 90 and 270 can not." In addition, change set # <https://trac.pjsip.org/repos/changeset/5118> 5138 <https://trac.pjsip.org/repos/changeset/5138#file1>introduces a convert for the stream. Another convert, however, was created for pjmedia_vid_port and is dependent on the source video size too. An alternative solution is to publish a PJMEDIA_EVENT_FMT_CHANGED event on PJMEDIA_DIR_ENCODING and finally invoke client_port_event_cb() in vid_port.c, and the client_port_event_cb() should be amended to NOT overwrite the destination size on PJMEDIA_DIR_ENCODING. The stream convert is not required. Is there any problem or potential risk on this solution? -- Deng Xiaoming dengxiaoming@gmail.com