[FFmpeg-trac] #6241(avformat:new): hls_flags delete_segments – file desctriptors not freeing, ffmpeg segfaults when system limit is reached
FFmpeg
trac at avcodec.org
Thu Mar 16 14:02:58 EET 2017
#6241: hls_flags delete_segments – file desctriptors not freeing, ffmpeg segfaults
when system limit is reached
-------------------------------------+-------------------------------------
Reporter: rafamiga | Type: defect
Status: new | Priority: important
Component: avformat | Version: git-
Keywords: descriptors | master
hls hls_flags delete_segments | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
Summary of the bug:
When delete_segments HLS flag is enabled, the FFmpeg binary segfaults when
max open files limit is reached.
How to reproduce:
{{{
/usr/local/bin/ffmpeg -loglevel info -thread_queue_size 1024 -f decklink
-i DeckLink 4K Extreme at 2 -audio_input embedded -video_input sdi -threads 0
-fflags +genpts -flags +global_header -c:a libfdk_aac
-map 0:v -s:v 1280x720 -b:v 3600k -minrate:v 3000k -maxrate:v 3700k
-bufsize:v 3600k -pix_fmt yuv420p -vf yadif
-map 0:a -af aresample=48000 -b:a 128k
-c:v libx264 -preset slow -profile:v main -x264opts keyint=100:min-
keyint=100:scenecut=-1 -hls_time 4 -hls_list_size 5
-hls_start_number_source epoch -hls_flags delete_segments
/var/www/html/live/test1/live3600k.m3u8
-map 0:v -s:v 1024x576 -b:v 2400k -minrate:v 2000k -maxrate:v 2500k
-bufsize:v 2400k -pix_fmt yuv420p -vf yadif
-map 0:a -af aresample=44100 -b:a 128k
-c:v libx264 -preset slow -profile:v main -x264opts keyint=100:min-
keyint=100:scenecut=-1 -hls_time 4 -hls_list_size 5
-hls_start_number_source epoch -hls_flags delete_segments
/var/www/html/live/test1/live2400k.m3u8
-map 0:v -s:v 720x404 -b:v 1700k -minrate:v 1000k -maxrate:v 1800k
-bufsize:v 1700k -pix_fmt yuv420p -vf yadif
-map 0:a -af aresample=44100 -b:a 96k -c:v libx264 -preset slow -profile:v
main -x264opts keyint=100:min-keyint=100:scenecut=-1 -hls_time 4
-hls_list_size 5 -hls_start_number_source epoch -hls_flags delete_segments
/var/www/html/live/test1/live1700k.m3u8
-map 0:v -s:v 512x288 -b:v 900k -minrate:v 800k -maxrate:v 1000k
-bufsize:v 900k -pix_fmt yuv420p -vf yadif
-map 0:a -af aresample=44100 -b:a 64k
-c:v libx264 -preset slow -profile:v main -x264opts keyint=100:min-
keyint=100:scenecut=-1 -hls_time 4 -hls_list_size 5
-hls_start_number_source epoch -hls_flags delete_segments
/var/www/html/live/test1/live900k.m3u8
-map 0:v -s:v 512x288 -b:v 450k -minrate:v 400k -maxrate:v 500k -bufsize:v
450k -pix_fmt yuv420p -vf yadif
-map 0:a -af aresample=44100 -b:a 64k
-c:v libx264 -preset slow -profile:v main -x264opts keyint=100:min-
keyint=100:scenecut=-1 -hls_time 4 -hls_list_size 5
-hls_start_number_source epoch -hls_flags delete_segments
/var/www/html/live/test1/live450k.m3u8
-map 0:v -s:v 512x288 -b:v 120k -minrate:v 100k -maxrate:v 150k -bufsize:v
120k -pix_fmt yuv420p -vf yadif
-map 0:a -af aresample=32000 -b:a 32k
-c:v libx264 -preset slow -profile:v main -x264opts keyint=100:min-
keyint=100:scenecut=-1 -hls_time 4 -hls_list_size 5
-hls_start_number_source epoch -hls_flags delete_segments
/var/www/html/live/test1/live120k.m3u8
Build:
ffmpeg version N-83663-g7e9ba78 Copyright (c) 2000-2017 the FFmpeg
developers
built with gcc 4.9.2 (Debian 4.9.2-10)
configuration: --prefix=/home/vagrant/ffmpeg-src/ffmpeg/
--bindir=/usr/local --pkg-config-flags=--static --extra-
cflags='-I/home/vagrant/decklink-include -static' --extra-
ldflags=-L/home/vagrant/decklink-include --enable-gpl --disable-shared
--disable-doc --enable-static --enable-libfdk-aac --enable-libfreetype
--enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis
--enable-libvpx --enable-libx264 --enable-nonfree --enable-decklink
libavutil 55. 47.100 / 55. 47.100
libavcodec 57. 81.100 / 57. 81.100
libavformat 57. 66.102 / 57. 66.102
libavdevice 57. 2.100 / 57. 2.100
libavfilter 6. 74.100 / 6. 74.100
libswscale 4. 3.101 / 4. 3.101
libswresample 2. 4.100 / 2. 4.100
libpostproc 54. 2.100 / 54. 2.100
Hyper fast Audio and Video encoder
}}}
How to check:
{{{
... after a few minutes of encoding ...
lsof -p `pidof ffmpeg`|grep '(deleted)'|wc -l
1812
}}}
Temporary fix:
{{{
# grep NOFILE /etc/systemd/system/ffhls_decklink.service
LimitNOFILE=65500
}}}
Comment:
I've looked into the source and it seems that this is a problem in
libavformat/hlsenc.c hls_delete_old_segments function:
{{{
proto = avio_find_protocol_name(s->filename);
if (hls->method || (proto && !av_strcasecmp(proto, "http"))) {
av_dict_set(&options, "method", "DELETE", 0);
if ((ret = hls->avf->io_open(hls->avf, &out, path,
AVIO_FLAG_WRITE, &options)) < 0)
goto fail;
ff_format_io_close(hls->avf, &out);
} else if (unlink(path) < 0) {
av_log(hls, AV_LOG_ERROR, "failed to delete old segment %s:
%s\n",
path, strerror(errno));
}
}}}
I'm no expert, in fact I'm a rookie when it comes to debuging ffmpeg
source, but unlink() without some sort of close() [is it
ff_format_io_close()?] may be the cause of this problem. [And the same
applies to unlink(sub_path) found later in that function.]
--
Ticket URL: <https://trac.ffmpeg.org/ticket/6241>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list