[FFmpeg-trac] #5224(avcodec:new): Excessive memory use in H.264 decoder with threading enabled

[FFmpeg]

#5224: Excessive memory use in H.264 decoder with threading enabled
---------------------------------+---------------------------------------
             Reporter:  jkqxz    |                     Type:  defect
               Status:  new      |                 Priority:  normal
            Component:  avcodec  |                  Version:  unspecified
             Keywords:           |               Blocked By:
             Blocking:           |  Reproduced by developer:  0
Analyzed by developer:  0        |
---------------------------------+---------------------------------------
 Given a stream with gaps in frame_num, the threaded decoder may allocate
 many more frames than it should.  (Up to thread count * num_ref_frames
 whole frame buffers.)

 See attached stream.

 This has parameters:
 * Baseline profile
 * num_ref_frames = 16
 * log2_max_frame_num_minus4 = 12
 * gaps_in_frame_num_value_allowed_flag = 1

 The stream is then a single IDR frame of black, followed by all-skip P
 frames with frame_num decreasing by one each time (65535, 65534, ...).

 Decode this stream with:
 {{{
 % ffmpeg -v 55 -vsync 0 -threads 8 -thread_type frame+slice -i
 large_frame_num_gaps.264 -f null -
 }}}
 Virtual memory use is much higher than expected, though this is rather
 hard to see.  (Since the frames are never actually touched, the real
 memory use is not excessive.)

 To see the problem more effectively, apply the following patch to
 instrument malloc/free:
 {{{
 diff --git a/libavutil/mem.c b/libavutil/mem.c
 index 8dfaad8..bddb0d1 100644
 --- a/libavutil/mem.c
 +++ b/libavutil/mem.c
 @@ -69,6 +69,7 @@ void  free(void *ptr);
   * Note that this will cost performance. */

  static size_t max_alloc_size= INT_MAX;
 +static void *big_mem_list[100];

  void av_max_alloc(size_t max){
      max_alloc_size = max;
 @@ -139,6 +140,18 @@ void *av_malloc(size_t size)
      if (ptr)
          memset(ptr, FF_MEMORY_POISON, size);
  #endif
 +
 +#if 1
 +    if(size > 1000000) {
 +        int i;
 +        av_log(0, AV_LOG_DEBUG, "malloc(%zu) = %p\n", size, ptr);
 +        for(i = 0; i < FF_ARRAY_ELEMS(big_mem_list) && big_mem_list[i];
 i++);
 +        if(i >= FF_ARRAY_ELEMS(big_mem_list))
 +            av_assert0(0 && "Too many big allocations.");
 +        big_mem_list[i] = ptr;
 +    }
 +#endif
 +
      return ptr;
  }

 @@ -227,6 +240,19 @@ int av_reallocp_array(void *ptr, size_t nmemb, size_t
 size)

  void av_free(void *ptr)
  {
 +#if 1
 +    if(ptr) {
 +        int i;
 +        for(i = 0; i < FF_ARRAY_ELEMS(big_mem_list); i++) {
 +            if(big_mem_list[i] == ptr) {
 +                av_log(0, AV_LOG_DEBUG, "free(%p)\n", ptr);
 +                big_mem_list[i] = 0;
 +                break;
 +            }
 +        }
 +    }
 +#endif
 +
  #if CONFIG_MEMALIGN_HACK
      if (ptr) {
          int v= ((char *)ptr)[-1];
 }}}

 Now the first command will abort with threads = 8 (and not with threads =
 1), because it tries to allocate more than 100 frame buffers.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/5224>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker