Tjyylli

TL;DR

I would recommend the following:

• 
  libx264: -g X -keyint_min X (and optionally add -force_key_frames "expr:gte(t,n_forced*N)")


• 
  libx265: -x265-params "keyint=X:min-keyint=X"


• 
  libvpx-vp9: -g X
  

where X is the interval in frames and N is the interval in seconds. For example, for a 2-second interval with a 30fps video, X = 60 and N = 2.

A note about different frame types

In order to properly explain this topic, we first have to define the two types of I-frames / keyframes:

• Instantaneous Decoder Refresh (IDR) frames: These allow independent decoding of the following frames, without access to frames previous to the IDR frame.


• Non-IDR-frames: These require a previous IDR frame for the decoding to work. Non-IDR frames can be used for scene cuts in the middle of a GOP (group of pictures).

What is recommended for streaming?

For the streaming case, you want to:

• Ensure that all IDR frames are at regular positions (e.g. at 2, 4, 6, … seconds) so that the video can be split up into segments of equal length.


• Enable scene cut detection, so as to improve coding efficiency / quality. This means allowing I-frames to be placed in between IDR frames. You can still work with scene cut detection disabled (and this is part of many guides, still), but it's not necessary.

What do the parameters do?

In order to configure the encoder, we have to understand what the keyframe parameters do. I did some tests and discovered the following, for the three encoders libx264, libx265 and libvpx-vp9 in FFmpeg:

• 
  

  libx264:

  
  
  
  
  
  • 
    -g sets the keyframe interval.
  
  
  • 
    -keyint_min sets the minimum keyframe interval.
  
  
  • 
    -x264-params "keyint=x:min-keyint=y" is the same as -g x -keyint_min y.
  
  
  • 
    

    Note: When setting both to the same value, the minimum is internally set to half the maximum interval plus one, as seen in the x264 code:

    
    
    

    h->param.i_keyint_min = x264_clip3( h->param.i_keyint_min, 1, h->param.i_keyint_max/2+1 );
    
    

    
    
  
  
  
  


• 
  

  libx265:

  
  
  
  
  
  • 
    -g is not implemented.
  
  
  • 
    -x265-params "keyint=x:min-keyint=y" works.
  
  
  
  


• 
  

  libvpx-vp9:

  
  
  
  
  
  • 
    -g sets the keyframe interval.
  
  
  • 
    -keyint_min sets the minimum keyframe interval
  
  
  • 
    

    Note: Due to how FFmpeg works, -keyint_min is only forwarded to the encoder when it is the same as -g. In the code from libvpxenc.c in FFmpeg we can find:

    
    
    

    if (avctx->keyint_min >= 0 && avctx->keyint_min == avctx->gop_size)
    
        enccfg.kf_min_dist = avctx->keyint_min;
    
    if (avctx->gop_size >= 0)
    
        enccfg.kf_max_dist = avctx->gop_size;
    
    

    
    
    

    This might be a bug (or lack of feature?), since libvpx definitely supports setting a different value for kf_min_dist.

    
    
  
  
  
  

Should you use -force_key_frames?

The -force_key_frames option forcibly inserts keyframes at the given interval (expression). This works for all encoders, but it might mess with the rate control mechanism. Especially for VP9, I've noticed severe quality fluctuations, so I cannot recommend using it in this case.

Here is my fifty cents for the case.

Method 1:

messing with libx264's arguments

-c:v libx264 -x264opts keyint=GOPSIZE:min-keyint=GOPSIZE:scenecut=-1

Generate iframes only at the desired intervals.

Example 1:

ffmpeg -i test.mp4 -codec:v libx264 

-r 23.976 

-x264opts "keyint=48:min-keyint=48:no-scenecut" 

-c:a copy 

-y test_keyint_48.mp4

Generate iframes as expected like this:

Iframes     Seconds

1           0

49          2

97          4

145         6

193         8

241         10

289         12

337         14

385         16

433         18

481         20

529         22

577         24

625         26

673         28

721         30

769         32

817         34

865         36

913         38

961         40

1009        42

1057        44

1105        46

1153        48

1201        50

1249        52

1297        54

1345        56

1393        58

Method 2 is depreciated. Ommitted.

Method 3:

insert a keyframe every N seconds (MAYBE):

-force_key_frames expr:gte(t,n_forced*GOP_LEN_IN_SECONDS)

Example 2

ffmpeg -i test.mp4 -codec:v libx264 

-r 23.976 

-force_key_frames "expr:gte(t,n_forced*2)"

-c:a copy 

-y test_fkf_2.mp4

Generate an iframes in a slightly different way:

Iframes     Seconds

1           0

49          2

97          4

145         6

193         8

241         10

289         12

337         14

385         16

433         18

481         20

519         21.58333333

529         22

577         24

625         26

673         28

721         30

769         32

817         34

865         36

913         38

931         38.75

941         39.16666667

961         40

1008        42

1056        44

1104        46

1152        48

1200        50

1248        52

1296        54

1305        54.375

1344        56

1367        56.95833333

1392        58

1430        59.58333333

1440        60

1475        61.45833333

1488        62

1536        64

1544        64.33333333

1584        66

1591        66.29166667

1632        68

1680        70

1728        72

1765        73.54166667

1776        74

1811        75.45833333

1824        75.95833333

1853        77.16666667

1872        77.95833333

1896        78.95833333

1920        79.95833333

1939        80.75

1968        81.95833333

As you can see it places iframes every 2 seconds AND on scenecut (seconds with floating part) which is important for video stream complexity in my opinion.

Genearated file sizes are pretty the same. Very strange that even with more keyframes in Method 3 it generates sometimes less file than standard x264 library algorithm.

For generating multiple bitrate files for HLS stream we choose method three. It perfectly aligned with 2 seconds between chunks, they have iframe at the beginning of every chunk and they have additional iframes on complex scenes which provides better experience for users who has an outdated devices and can not playback x264 high profiles.

Hope it helps someone.

I wanted to add some info here since my googling pulled up this discussion quite a bit in my quest to find info on trying to find a way to segment my DASH encoding the way I wanted, and none of the info I found was totally correct.

First several misconceptions to get rid of:

1. Not all I-frames are the same. There's big "I" frames and little "i" frames. Or to use correct terminology, IDR I-Frames and non-IDR I-Frames. IDR I-frames (sometimes called "keyframes") will create a new GOP. The non-IDR frames will not. They are handy to have inside of a GOP where there is a scene change.




2. -x264opts keyint=GOPSIZE:min-keyint=GOPSIZE ← This does not do what you think it does. This took me a little while to figure out. It turns out the min-keyint is limited in the code. It is not allowed to be greater than (keyint / 2) + 1. So assigning the same value to these two variables results in the value for min-keyint getting knocked down by half when encoding.

Here's the thing: scene-cut is really great, especially in video that has fast hard cuts. It keeps it nice and crisp, so I don't want to disable it, but at the same time I couldn't get a fixed GOP size as long as it was enabled. I wanted to enable scene-cut, but to only have it use non-IDR I-frames. But it wasn't working. Until I figured out (from lots of reading) about misconception #2.

It turns out I needed to set keyint to double my desired GOP size. This means that min-keyint can be set to my desired GOP size (without the internal code cutting it in half), which prevents scene-cut detection from using IDR I-frames inside the GOP size because the frame count since the last IDR I-Frame is always less than min-keyinit.

And finally setting the force_key_frame option overrides the double size keyint. So here's what works:

I prefer segments in 2 second chunks, so my GOPSIZE = Framerate * 2

ffmpeg <other_options> -force_key_frames "expr:eq(mod(n,<GOPSIZE>),0)" -x264opts rc-lookahead=<GOPSIZE>:keyint=<GOPSIZE * 2>:min-keyint=<GOPSIZE> <other_options>

You can verify using ffprobe:

ffprobe <SRC_FLE> -select_streams v -show_frames -of csv -show_entries frame=coded_picture_number,key_frame,pict_type > frames.csv

In the generated CSV file each line will tell you: frame, [is_an_IDR_?], [frame_type], [frame_number]:

frame,1,I,60  <-- frame 60, is I frame, 1 means is an IDR I-frame (aka KeyFrame)

frame,0,I,71  <-- frame 71, is I frame, 0 means not an IDR I_frame

The result is that you should only see IDR I-Frames at fixed GOPSIZE intervals, while all other I frames are non-IDR I-frames inserted as needed by scene-cut detection.

It seems that this syntax doesnt allways work.. Ive tested quite alot on our VOD content as well as live content (file dumps) and sometimes scenecut doesnt work and trigger a inbetween iframe:

Syntax for a i50->p50 upconvertion, 2 sec gop/segment, IDR at the start, iframes inbetween if needed

ffmpeg.exe -loglevel verbose -i avc_50i.ts -pix_fmt yuv420p -filter_complex yadif=1,scale=1920:1080 -vcodec libx264 -preset fast -x264-params "rc-lookahead=100:keyint=200:min-keyint=100:hrd=1:vbv_maxrate=12000:vbv_bufsize=12000:no-open-gop=1" -r 50 -crf 22 -force_key_frames "expr:eq(mod(n,100),0)" -codec:a aac -b:a 128k -y target.ts

Twitch has a post about this. They explain that they decided to use their own program for several reasons; one of them was that ffmpeg doesn't let you run different x264 instances in different threads, but instead devotes all specified threads to one frame in one output before moving on to the next output.

If you aren't doing real-time streaming, you have more luxury. The 'correct' way is probably to encode at one resolution with just the GOP size specified with -g, and then encode the other resolutions forcing keyframes at the same places.

If you wanted to do that, you might use ffprobe to get the keyframe times and then use a shell script or an actual programming language to convert that into an ffmpeg command.

But for most content, there's very little difference between having one keyframe every 5 seconds and two keyframes every 5 seconds (one forced and one from scenecut). This is about the average I-frame size vs the size of P-frames and B-frames. If you use x264 with typical settings (the only reason I think you should do anything to affect these is if you set -qmin, as a poor way of preventing x264 from using bitrate on easy content; this limits all frame types to the same value, I think) and get a result like I-frame average size of 46 kB, P-frame 24 kB, B-frame 17 kB (half as frequent as P-frames), then an extra I-frame every second at 30 fps is only a 3% increase in file size. The difference between h264 and h263 might be made up of a bunch of 3% decreases, but a single one isn't very important.

On other types of content, frame sizes will be different. To be fair, this is about temporal complexity and not spatial complexity, so it isn't just easy content vs hard content. But generally, streaming video sites have a bitrate limit, and content with relatively large I-frames is easy content that will be encoded at high quality no matter how many extra keyframes are added. It's wasteful, but this waste will usually not be noticed. The most wasteful case is probably a video that's just a static image accompanying a song, where each keyframe is exactly the same.

One thing I'm not sure of is how forced keyframes interact with the rate limiter set with -maxrate and -bufsize. I think even YouTube has had recent problems correctly configuring buffer settings to give consistent quality. If you're just using average bitrate settings as can be seen by some sites (since you can inspect x264's options in the header/mov atom? with a hex editor) then the buffer model isn't a problem, but if you're serving user-generated content, average bitrate encourages users to add a black screen at the end of their video.

Ffmpeg's -g option, or any other encoder option that you use, is mapped to the encoder-specific option. So '-x264-params keyint=GOPSIZE' is equivalent to '-g GOPSIZE'.

One problem with using scene detection is if you prefer keyframes near specific numbers for whatever reason. If you specify keyframes every 5 seconds and use scene detection, and there's a scene change at 4.5, then it should be detected, but then the next keyframe will be at 9.5. If the time keeps getting stepped up like this, you could end up with keyframes at 42.5, 47.5, 52.5, etc., instead of 40, 45, 50, 55. Conversely, if there's a scene change at 5.5, then there will be a keyframe at 5 and 5.5 will be too early for another one. Ffmpeg doesn't let you specify "make a keyframe here if there's no scene change within the next 30 frames". Someone who understands C could add that option, though.

For variable-frame-rate video, when you're not live-streaming like Twitch, you should be able to use scene changes without converting permanently to constant frame-rate. If you use the 'select' filter in ffmpeg and use the 'scene' constant in the expression, then the debug output (-v debug or press '+' several times while encoding) shows the scene change number. This is probably different from, and not as useful as, the number used by x264, but it could still be useful.

The procedure, then, would probably be to do a test video that's only for keyframe changes, but maybe could be used for rate control data if using 2-pass. (Not sure if the generated data is at all useful for different resolutions and settings; the macroblock-tree data won't be.) Convert it to constant-framerate video, but see this bug about stuttering output when halving framerate if you ever decide to use the fps filter for other purposes. Run it through x264 with your desired keyframe and GOP settings.

Then just use these keyframe times with the original variable frame-rate video.

If you allow completely crazy user-generated content with a 20-second gap between frames, then for the variable frame-rate encode, you could split the output, use fps filter, somehow use select filter (maybe build a really long expression that has every keyframe time)... or maybe you could use the test video as input and either decode only keyframes, if that ffmpeg option works, or use the select filter to select keyframes. Then scale it to the correct size (there's even a scale2ref filter for this) and overlay the original video on it. Then use the interleave filter to combine these destined-to-be forced keyframes with the original video. If this results in two frames that are 0.001 sec apart that the interleave filter doesn't prevent, then address this problem yourself with another select filter. Dealing with frame buffer limits for the interleave filter could be the main problem here. These could all work: use some kind of filter to buffer the denser stream (fifo filter?); refer to the input file multiple times so it's decoded more than once and frames don't have to be stored; use the 'streamselect' filter, which I have never done, at exactly the times of the keyframes; improve the interleave filter by changing its default behaviour or adding an option to output the oldest frame in a buffer instead of dropping a frame.