[FFmpeg-trac] #6566(undetermined:new): Overlay drops frames when audio is also processed
FFmpeg
trac at avcodec.org
Thu Aug 3 21:05:02 EEST 2017
#6566: Overlay drops frames when audio is also processed
-------------------------------------+-------------------------------------
Reporter: Gyan | Type: defect
Status: new | Priority: important
Component: | Version: git-
undetermined | master
Keywords: | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
When performing an overlay of one video over the other, ffmpeg reports
'''Buffer queue overflow''' and the overlay freezes at multiple points.
Adding fifo filters to the two video feeds doesn't help. However, omitting
the audio, resolves the issue.
''' # 1 ''' Generate source files
{{{
ffmpeg -f lavfi -i testsrc2=d=55 -f lavfi -i testsrc=d=10 \
-f lavfi -i sine=d=55 -f lavfi -i sine=640:d=10 \
-map 0 -map 2 in.mp4 \
-map 1 -map 3 ol.mp4
}}}
(The behaviour doesn't occur with direct lavfi source feeds)
'''# 2''' Overlay with audio - '''overlay video freezes'''
{{{
ffmpeg -i in.mp4 -i ol.mp4 -strict -2 -filter_complex \
"[0:v]setpts=PTS-STARTPTS[v0];[1:v]setpts=PTS-STARTPTS+5/TB[v1]; \
[v0][v1]overlay=(main_w-overlay_w)/2:(main_h-
overlay_h)/2:eof_action=pass[outv]; \
[1:a]adelay=5000[a1];[0:a][a1]amix[outa]" \
-map [outv] -map [outa] out.mp4
}}}
{{{
ffmpeg version N-86920-g1193301 Copyright (c) 2000-2017 the FFmpeg
developers
built with gcc 7.1.0 (GCC)
configuration: --enable-gpl --enable-version3 --enable-cuda --enable-
cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc
--enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r
--enable-gnutls --enable-iconv --enable-libass --enable-libbluray
--enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme
--enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame
--enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264
--enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy
--enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame
--enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-
libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-
libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma
--enable-zlib
libavutil 55. 69.100 / 55. 69.100
libavcodec 57.102.100 / 57.102.100
libavformat 57. 76.100 / 57. 76.100
libavdevice 57. 7.100 / 57. 7.100
libavfilter 6. 95.100 / 6. 95.100
libswscale 4. 7.101 / 4. 7.101
libswresample 2. 8.100 / 2. 8.100
libpostproc 54. 6.100 / 54. 6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'in.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.76.100
Duration: 00:00:55.02, start: 0.000000, bitrate: 354 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p,
320x240 [SAR 1:1 DAR 4:3], 278 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc
(default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono,
fltp, 69 kb/s (default)
Metadata:
handler_name : SoundHandler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'ol.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.76.100
Duration: 00:00:10.02, start: 0.000000, bitrate: 111 kb/s
Stream #1:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 /
0x31637661), yuv444p, 320x240 [SAR 1:1 DAR 4:3], 34 kb/s, 25 fps, 25 tbr,
12800 tbn, 50 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #1:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono,
fltp, 69 kb/s (default)
Metadata:
handler_name : SoundHandler
Stream mapping:
Stream #0:0 (h264) -> setpts
Stream #0:1 (aac) -> amix:input0
Stream #1:0 (h264) -> setpts
Stream #1:1 (aac) -> adelay
overlay -> Stream #0:0 (libx264)
amix -> Stream #0:1 (aac)
Press [q] to stop, [?] for help
[libx264 @ 0000000002a41da0] using SAR=1/1
[libx264 @ 0000000002a41da0] using cpu capabilities: MMX2 SSE2Fast SSSE3
SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000000002a41da0] profile High, level 1.3
[libx264 @ 0000000002a41da0] 264 - core 152 r2851 ba24899 - H.264/MPEG-4
AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html -
options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7
psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1
8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6
lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0
bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1
b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25
scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0
qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'out.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.76.100
Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661),
yuv420p(progressive), 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 25 fps, 12800
tbn, 25 tbc (default)
Metadata:
encoder : Lavc57.102.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono,
fltp, 69 kb/s (default)
Metadata:
encoder : Lavc57.102.100 aac
[Parsed_overlay_2 @ 00000000003cd5a0] [framesync @ 00000000047c0328]
Buffer queue overflow, dropping.
Last message repeated 115 times
frame= 328 fps=0.0 q=28.0 size= 256kB time=00:00:13.07 bitrate=
160.4kbits/s speed=26.1x
frame= 709 fps=709 q=28.0 size= 768kB time=00:00:28.32 bitrate=
222.1kbits/s speed=28.3x
frame= 1054 fps=703 q=28.0 size= 1280kB time=00:00:42.16 bitrate=
248.7kbits/s speed=28.1x
frame= 1375 fps=695 q=-1.0 Lsize= 2007kB time=00:00:55.00 bitrate=
298.8kbits/s speed=27.8x
video:1496kB audio:464kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: 2.348816%
[libx264 @ 0000000002a41da0] frame I:7 Avg QP:17.46 size: 5572
[libx264 @ 0000000002a41da0] frame P:391 Avg QP:23.73 size: 1686
[libx264 @ 0000000002a41da0] frame B:977 Avg QP:28.18 size: 853
[libx264 @ 0000000002a41da0] consecutive B-frames: 2.0% 7.1% 8.3% 82.6%
[libx264 @ 0000000002a41da0] mb I I16..4: 44.1% 27.7% 28.2%
[libx264 @ 0000000002a41da0] mb P I16..4: 2.1% 3.9% 1.0% P16..4:
14.1% 8.5% 5.9% 0.0% 0.0% skip:64.4%
[libx264 @ 0000000002a41da0] mb B I16..4: 0.2% 0.1% 0.1% B16..8:
17.2% 5.4% 1.5% direct: 2.2% skip:73.4% L0:50.9% L1:44.2% BI: 4.9%
[libx264 @ 0000000002a41da0] 8x8 transform intra:48.4% inter:34.6%
[libx264 @ 0000000002a41da0] coded y,uvDC,uvAC intra: 12.0% 35.6% 29.4%
inter: 4.3% 13.3% 10.9%
[libx264 @ 0000000002a41da0] i16 v,h,dc,p: 72% 20% 6% 2%
[libx264 @ 0000000002a41da0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 10% 78%
1% 0% 0% 0% 0% 1%
[libx264 @ 0000000002a41da0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 38% 22% 31%
1% 1% 1% 2% 1% 2%
[libx264 @ 0000000002a41da0] i8c dc,h,v,p: 47% 18% 33% 2%
[libx264 @ 0000000002a41da0] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000000002a41da0] ref P L0: 59.3% 8.1% 20.0% 12.6%
[libx264 @ 0000000002a41da0] ref B L0: 80.6% 15.8% 3.6%
[libx264 @ 0000000002a41da0] ref B L1: 94.3% 5.7%
[libx264 @ 0000000002a41da0] kb/s:222.77
[aac @ 000000000031d160] Qavg: 1015.382
}}}
'''# 3''' Overlay with no audio - '''output normal'''
{{{
ffmpeg -i in.mp4 -i ol.mp4 -strict -2 -filter_complex \
"[0:v]setpts=PTS-STARTPTS[v0];[1:v]setpts=PTS-STARTPTS+5/TB[v1]; \
[v0][v1]overlay=(main_w-overlay_w)/2:(main_h-
overlay_h)/2:eof_action=pass[outv]" \
-map [outv] out-noaud.mp4
}}}
{{{
ffmpeg version N-86920-g1193301 Copyright (c) 2000-2017 the FFmpeg
developers
built with gcc 7.1.0 (GCC)
configuration: --enable-gpl --enable-version3 --enable-cuda --enable-
cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc
--enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r
--enable-gnutls --enable-iconv --enable-libass --enable-libbluray
--enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme
--enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame
--enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264
--enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy
--enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame
--enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-
libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-
libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma
--enable-zlib
libavutil 55. 69.100 / 55. 69.100
libavcodec 57.102.100 / 57.102.100
libavformat 57. 76.100 / 57. 76.100
libavdevice 57. 7.100 / 57. 7.100
libavfilter 6. 95.100 / 6. 95.100
libswscale 4. 7.101 / 4. 7.101
libswresample 2. 8.100 / 2. 8.100
libpostproc 54. 6.100 / 54. 6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'in.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.76.100
Duration: 00:00:55.02, start: 0.000000, bitrate: 354 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p,
320x240 [SAR 1:1 DAR 4:3], 278 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc
(default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono,
fltp, 69 kb/s (default)
Metadata:
handler_name : SoundHandler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'ol.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.76.100
Duration: 00:00:10.02, start: 0.000000, bitrate: 111 kb/s
Stream #1:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 /
0x31637661), yuv444p, 320x240 [SAR 1:1 DAR 4:3], 34 kb/s, 25 fps, 25 tbr,
12800 tbn, 50 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #1:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono,
fltp, 69 kb/s (default)
Metadata:
handler_name : SoundHandler
Stream mapping:
Stream #0:0 (h264) -> setpts
Stream #1:0 (h264) -> setpts
overlay -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
[libx264 @ 0000000000547f60] using SAR=1/1
[libx264 @ 0000000000547f60] using cpu capabilities: MMX2 SSE2Fast SSSE3
SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000000000547f60] profile High, level 1.3
[libx264 @ 0000000000547f60] 264 - core 152 r2851 ba24899 - H.264/MPEG-4
AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html -
options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7
psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1
8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6
lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0
bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1
b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25
scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0
qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'out-noaud.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.76.100
Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p,
320x240 [SAR 1:1 DAR 4:3], q=-1--1, 25 fps, 12800 tbn, 25 tbc (default)
Metadata:
encoder : Lavc57.102.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
frame= 614 fps=0.0 q=28.0 size= 256kB time=00:00:22.44 bitrate=
93.5kbits/s speed=44.9x
frame= 1143 fps=1143 q=28.0 size= 1024kB time=00:00:43.60 bitrate=
192.4kbits/s speed=43.6x
frame= 1375 fps=1092 q=-1.0 Lsize= 1515kB time=00:00:54.88 bitrate=
226.2kbits/s speed=43.6x
video:1499kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: 1.111186%
[libx264 @ 0000000000547f60] frame I:6 Avg QP:16.11 size: 5481
[libx264 @ 0000000000547f60] frame P:394 Avg QP:23.82 size: 1700
[libx264 @ 0000000000547f60] frame B:975 Avg QP:28.20 size: 853
[libx264 @ 0000000000547f60] consecutive B-frames: 2.1% 7.3% 8.3% 82.3%
[libx264 @ 0000000000547f60] mb I I16..4: 51.7% 21.3% 27.1%
[libx264 @ 0000000000547f60] mb P I16..4: 2.2% 3.8% 1.0% P16..4:
14.7% 8.9% 6.1% 0.0% 0.0% skip:63.4%
[libx264 @ 0000000000547f60] mb B I16..4: 0.2% 0.1% 0.1% B16..8:
17.4% 5.3% 1.5% direct: 2.2% skip:73.2% L0:50.9% L1:44.1% BI: 4.9%
[libx264 @ 0000000000547f60] 8x8 transform intra:47.0% inter:34.2%
[libx264 @ 0000000000547f60] coded y,uvDC,uvAC intra: 11.4% 35.0% 28.3%
inter: 4.3% 13.5% 11.1%
[libx264 @ 0000000000547f60] i16 v,h,dc,p: 73% 20% 5% 2%
[libx264 @ 0000000000547f60] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 8% 10% 79%
1% 0% 0% 0% 0% 1%
[libx264 @ 0000000000547f60] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 38% 23% 32%
1% 1% 1% 2% 1% 2%
[libx264 @ 0000000000547f60] i8c dc,h,v,p: 46% 18% 34% 2%
[libx264 @ 0000000000547f60] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000000000547f60] ref P L0: 59.5% 8.3% 20.2% 12.0%
[libx264 @ 0000000000547f60] ref B L0: 80.6% 15.8% 3.5%
[libx264 @ 0000000000547f60] ref B L1: 94.2% 5.8%
[libx264 @ 0000000000547f60] kb/s:223.11
}}}
--
Ticket URL: <https://trac.ffmpeg.org/ticket/6566>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list