[FFmpeg-trac] #5223(avutil:new): avcodec_find_best_pix_fmt_of_list depends on the order?

[FFmpeg]

#5223: avcodec_find_best_pix_fmt_of_list depends on the order?
--------------------------------+----------------------------------
             Reporter:  MattE   |                     Type:  defect
               Status:  new     |                 Priority:  normal
            Component:  avutil  |                  Version:
             Keywords:          |               Blocked By:
             Blocking:          |  Reproduced by developer:  0
Analyzed by developer:  0       |
--------------------------------+----------------------------------
 avcodec_find_best_pix_fmt_of_list seems to be returning different values
 depending on the order of the input.

 Following is a example using python/cython, but it should be easy to
 understand:
 {{{
 AVPixelFormat *fmts2 = [AV_PIX_FMT_RGB24, AV_PIX_FMT_GRAY8,
 AV_PIX_FMT_NONE]
 AVPixelFormat *fmts3 = [AV_PIX_FMT_GRAY8, AV_PIX_FMT_RGB24,
 AV_PIX_FMT_NONE]
 print <int>avcodec_find_best_pix_fmt_of_list(fmts2, AV_PIX_FMT_YUV420P, 0,
 &loss)
 print <int>avcodec_find_best_pix_fmt_of_list(fmts3, AV_PIX_FMT_YUV420P, 0,
 &loss)
 }}}


 Which prints:
 {{{
 2
 8
 }}}


 I.e. if rgb24 is first that is returned, otherwise gray is preferred. If
 this is somewhat intentional, then it should be documented as such as as
 well as mentioned that the highest quality pixel formats should be listed
 first.



 {{{
 $ ffmpeg
 ffmpeg version N-76278-gd897d4c Copyright (c) 2000-2015 the FFmpeg
 developers
   built with gcc 5.2.0 (GCC)
   configuration: --disable-static --enable-shared --enable-gpl --enable-
 version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-
 fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass
 --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec
 --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc
 --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb
 --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-
 librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex
 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-
 aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx
 --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265
 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink
 --enable-zlib
   libavutil      55.  4.100 / 55.  4.100
   libavcodec     57. 10.100 / 57. 10.100
   libavformat    57. 11.100 / 57. 11.100
   libavdevice    57.  0.100 / 57.  0.100
   libavfilter     6. 14.100 /  6. 14.100
   libswscale      4.  0.100 /  4.  0.100
   libswresample   2.  0.100 /  2.  0.100
   libpostproc    54.  0.100 / 54.  0.100
 }}}

--
Ticket URL: <https://trac.ffmpeg.org/ticket/5223>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker