[FFmpeg-trac] #2801(avformat:new): Concat protocol with too many files

[FFmpeg]

#2801: Concat protocol with too many files
----------------------------------+--------------------------------------
             Reporter:  jbvsmo    |                     Type:  defect
               Status:  new       |                 Priority:  normal
            Component:  avformat  |                  Version:  git-master
             Keywords:  concat    |               Blocked By:
             Blocking:            |  Reproduced by developer:  0
Analyzed by developer:  0         |
----------------------------------+--------------------------------------
 Summary of the bug:
 When using the concat protocol with a certain number of files, it just
 crashes at some point when parsing the input. The open files limit in my
 server was 1024, so it crashes when trying to concatenate that amount of
 files.


 I see two problems here:
  1 - FFmpeg doesn't tell me whats wrong. It just ends the process with
 error code 1 and no useful information whatsoever. Even gdb can't show any
 information.
 I had to dig in the "libavformat/concat.c" file with some prints to see
 the system error.

  2 - Why are all the files being opened at once? There should be a way to
 call "ffurl_open" only when the file is going to be used.



 The command input line is quite huge to put here so I just wrote the first
 two files.
 FFmpeg will truncate the text when printing the names.


 How to reproduce:
 {{{
 $ ffmpeg -v 99 -y -i
 "concat:/video/videox/CCD_orig_0.h264|/video/videox/CCD_orig_1.h264|..."
 -c:v copy foo.mp4

 ffmpeg version N-38232-g1f7acf3 Copyright (c) 2000-2013 the FFmpeg
 developers
   built on Jul 21 2013 01:55:20 with gcc 4.6 (Ubuntu/Linaro
 4.6.3-1ubuntu5)
   configuration: --enable-gpl --enable-libass --enable-libfaac --enable-
 libfdk-aac --enable-libmp3lame --enable-libopencore-amrnb --enable-
 libopencore-amrwb --enable-libspeex --enable-librtmp --enable-libtheora
 --enable-libvorbis --enable-libvpx --enable-x11grab --enable-libx264
 --enable-nonfree --enable-version3
   libavutil      52. 40.100 / 52. 40.100
   libavcodec     55. 18.102 / 55. 18.102
   libavformat    55. 12.102 / 55. 12.102
   libavdevice    55.  3.100 / 55.  3.100
   libavfilter     3. 81.102 /  3. 81.102
   libswscale      2.  4.100 /  2.  4.100
   libswresample   0. 17.102 /  0. 17.102
   libpostproc    52.  3.100 / 52.  3.100
 Splitting the commandline.
 Reading option '-v' ... matched as option 'v' (set logging level) with
 argument '99'.
 Reading option '-y' ... matched as option 'y' (overwrite output files)
 with argument '1'.
 Reading option '-i' ... matched as input file with argument
 'concat:/video/videox/CCD_orig_0.h264|/video/videox/CCD_orig_1.h264|/video/videox/CCD_orig_2.h264|/video/videox/CCD_orig_3.h264|/video/videox/CCD_orig_4.h264|/video/videox/CCD_orig_5.h264|/video/videox/CCD_orig_6.h264|/video/videox/CCD_orig_7.h264|/video/videox/CCD_orig_8.h264|/video/videox/CCD_orig_9.h264|/video/videox/CCD_orig_10.h264|/video/videox/CCD_orig_11.h264|/video/videox/CCD_orig_12.h264|/video/videox/CCD_orig_13.h264|/video/videox/CCD_orig_14.h264|/video/videox/CCD_orig_15.h264|/video/videox/CCD_orig_16.h264|/video/videox/CCD_orig_17.h264|/video/videox/CCD_orig_18.h264|/video/videox/CCD_orig_19.h264|/video/videox/CCD_orig_20.h264|/video/videox/CCD_orig_21.h264|/video/videox/CCD_orig_22.h264|/video/videox/CCD_orig_23.h264|/video/videox/CCD_orig_24.h264|/video/videox/CCD_orig_25.h264|/video/videox/CCD_orig_26.h264|/video/videox/CCD_orig_27.h264|/video/videox/CCD_orig_28.h264|/video/videox/CCD_orig_29.h264|/video/videox/CCD_orig_30.h264|/video/videox/CCD_orig_31.hReading
 option '-c:v' ... matched as option 'c' (codec name) with argument 'copy'.
 Reading option 'foo.mp4' ... matched as output file.
 Finished splitting the commandline.
 Parsing a group of options: global .
 Applying option v (set logging level) with argument 99.
 Applying option y (overwrite output files) with argument 1.
 Successfully parsed a group of options.
 Parsing a group of options: input file
 concat:/video/videox/CCD_orig_0.h264|/video/videox/CCD_orig_1.h264|/video/videox/CCD_orig_2.h264|/video/videox/CCD_orig_3.h264|/video/videox/CCD_orig_4.h264|/video/videox/CCD_orig_5.h264|/video/videox/CCD_orig_6.h264|/video/videox/CCD_orig_7.h264|/video/videox/CCD_orig_8.h264|/video/videox/CCD_orig_9.h264|/video/videox/CCD_orig_10.h264|/video/videox/CCD_orig_11.h264|/video/videox/CCD_orig_12.h264|/video/videox/CCD_orig_13.h264|/video/videox/CCD_orig_14.h264|/video/videox/CCD_orig_15.h264|/video/videox/CCD_orig_16.h264|/video/videox/CCD_orig_17.h264|/video/videox/CCD_orig_18.h264|/video/videox/CCD_orig_19.h264|/video/videox/CCD_orig_20.h264|/video/videox/CCD_orig_21.h264|/video/videox/CCD_orig_22.h264|/video/videox/CCD_orig_23.h264|/video/videox/CCD_orig_24.h264|/video/videox/CCD_orig_25.h264|/video/videox/CCD_orig_26.h264|/video/videox/CCD_orig_27.h264|/video/videox/CCD_orig_28.h264|/video/videox/CCD_orig_29.h264|/video/videox/CCD_orig_30.h264|/video/videox/CCD_orig_31.Successfully
 parsed a group of options.
 Opening an input file:
 concat:/video/videox/CCD_orig_0.h264|/video/videox/CCD_orig_1.h264|/video/videox/CCD_orig_2.h264|/video/videox/CCD_orig_3.h264|/video/videox/CCD_orig_4.h264|/video/videox/CCD_orig_5.h264|/video/videox/CCD_orig_6.h264|/video/videox/CCD_orig_7.h264|/video/videox/CCD_orig_8.h264|/video/videox/CCD_orig_9.h264|/video/videox/CCD_orig_10.h264|/video/videox/CCD_orig_11.h264|/video/videox/CCD_orig_12.h264|/video/videox/CCD_orig_13.h264|/video/videox/CCD_orig_14.h264|/video/videox/CCD_orig_15.h264|/video/videox/CCD_orig_16.h264|/video/videox/CCD_orig_17.h264|/video/videox/CCD_orig_18.h264|/video/videox/CCD_orig_19.h264|/video/videox/CCD_orig_20.h264|/video/videox/CCD_orig_21.h264|/video/videox/CCD_orig_22.h264|/video/videox/CCD_orig_23.h264|/video/videox/CCD_orig_24.h264|/video/videox/CCD_orig_25.h264|/video/videox/CCD_orig_26.h264|/video/videox/CCD_orig_27.h264|/video/videox/CCD_orig_28.h264|/video/videox/CCD_orig_29.h264|/video/videox/CCD_orig_30.h264|/video/videox/CCD_orig_31.h264|/video/videconcat:/video/videox/CCD_orig_0.h264|/video/videox/CCD_orig_1.h264|/video/videox/CCD_orig_2.h264|/video/videox/CCD_orig_3.h264|/video/videox/CCD_orig_4.h264|/video/videox/CCD_orig_5.h264|/video/videox/CCD_orig_6.h264|/video/videox/CCD_orig_7.h264|/video/videox/CCD_orig_8.h264|/video/videox/CCD_orig_9.h264|/video/videox/CCD_orig_10.h264|/video/videox/CCD_orig_11.h264|/video/videox/CCD_orig_12.h264|/video/videox/CCD_orig_13.h264|/video/videox/CCD_orig_14.h264|/video/videox/CCD_orig_15.h264|/video/videox/CCD_orig_16.h264|/video/videox/CCD_orig_17.h264|/video/videox/CCD_orig_18.h264|/video/videox/CCD_orig_19.h264|/video/videox/CCD_orig_20.h264|/video/videox/CCD_orig_21.h264|/video/videox/CCD_orig_22.h264|/video/videox/CCD_orig_23.h264|/video/videox/CCD_orig_24.h264|/video/videox/CCD_orig_25.h264|/video/videox/CCD_orig_26.h264|/video/videox/CCD_orig_27.h264|/video/videox/CCD_orig_28.h264|/video/videox/CCD_orig_29.h264|/video/videox/CCD_orig_30.h264|/video/videox/CCD_orig_31.h264|/video/videox/CCD_orig_32.h264|/vi

 }}}

 The problem cannot be on x264 (or any other external library) because it
 happens while still parsing the text.

-- 
Ticket URL: <https://ffmpeg.org/trac/ffmpeg/ticket/2801>
FFmpeg <http://ffmpeg.org>
FFmpeg issue tracker