[Request]: Support automatic `level` calculation for encoders

[lexano-ivs]

When using the AVC, HEVC, or AV1 encoders, the codec level needs to be set by the application. It would be an improvement to offer an Auto option where AMF calculates the best-fit level for the encode operation.

Using HEVC as an example, AMF_VIDEO_ENCODER_HEVC_PROFILE_LEVEL should have AMF_LEVEL_AUTO. This option should apply to all encoders.

Refer to the OBS upstream PR at obsproject/obs-studio#11548 for an example of what would be needed.

[rhutsAMD]

The encoder profile level is automatically increased based on the frame size if the given profile level is found to not be sufficient. For HEVC, the default profile level set is the highest one, which is why you are not seeing it get corrected upwards.

For example, if you set a low profile level for the H264 encoder such as AMF_H264_LEVEL__1, you should see it get automatically increased to AMF_H264_LEVEL__4 for 1920x1080 input. Similarly if you leave the default profile level for H264, you should see it increase if you use, for example, a 4k input.

If you find this correction doesn't work, could you please provide an example?

[cs9kc]

Is there a reason why AMF's HEVC uses the highest "level" by default? People have noticed this before when using OBS Studio in the past, but I don't think anyone has directly asked "why"?

[lexano-ivs]

@rhutsAMD For HEVC, the AMF default is the highest level (6.2). It didn't occur to me to try using the lowest level and testing it that way, as it's a bit counterintuitive.

The documentation for AMF_VIDEO_ENCODER_HEVC_PROFILE_LEVEL indicates "Description: Selects the HEVC Profile Level." This implies the level has to be selected, and because the enumeration doesn't list an "Auto" option, it could easily be interpreted as "the level is set here and we won't change it".

Is there any documentation that describes the automatic level adjustment?

Also, as @cs9kc points out, defaulting to the highest level is questionable. Given the description you provided, wouldn't it make sense to default it to the lowest level and let AMF adjust it?

[rhutsAMD]

I have lowered the default profile level for the HEVC encoder to AMF_LEVEL_5_2 to closer match the default of the other encoders considering the corresponding frame sizes. This new default profile level will take effect in a future driver release.

[cgutman]

Is there a reason not to have all AMF encoders default to the lowest level that can meet the requested resolution, frame rate, bitrate, and reference frame count? If you don't want to do it in AMF itself, we can always do it in FFmpeg (which right now defaults to leaving the level unset and using AMF's default) if it makes sense.

Most encoders and tools that have an Auto option for level selection use this "pick lowest possible level" behavior. I suspect it's what users are expecting for the broadest possible decoder compatibility.

[rhutsAMD]

I have also updated the documentation and it will include notes on automatic level adjustment upwards based on the frame size in a subsequent AMF SDK release.

[nyanmisaka]

FYI, just implement a table like this in your AMF runtime so that the level will automatically be set to the lowest value that meets the spec. Both the Intel and NVIDIA encoders behave this way.

typedef struct H264eLevelInfo_t {
    H264Level       level;
    RK_S32          max_MBPS;       /* Max macroblock rate per second */
    RK_S32          max_MBs;        /* Max frame size */
    RK_S32          max_DpbMBs;     /* Max decoded picture buffer size */
    RK_S32          max_BR;         /* Max video bitrate */
    const char      *name;
} H264eLevelInfo;

H264eLevelInfo level_infos[] = {
    /*  level             max_MBs  max_MBPS  max_DpbMBs  max_BR  name  */
    {   H264_LEVEL_1_0,     1485,       99,        396,      64, "1"   },
    {   H264_LEVEL_1_b,     1485,       99,        396,     128, "1.b" },
    {   H264_LEVEL_1_1,     3000,      396,        900,     192, "1.1" },
    {   H264_LEVEL_1_2,     6000,      396,       2376,     384, "1.2" },
    {   H264_LEVEL_1_3,    11880,      396,       2376,     768, "1.3" },
    {   H264_LEVEL_2_0,    11880,      396,       2376,    2000, "2"   },
    {   H264_LEVEL_2_1,    19800,      792,       4752,    4000, "2.1" },
    {   H264_LEVEL_2_2,    20250,     1620,       8100,    4000, "2.2" },
    {   H264_LEVEL_3_0,    40500,     1620,       8100,   10000, "3"   },
    {   H264_LEVEL_3_1,   108000,     3600,      18000,   14000, "3.1" },
    {   H264_LEVEL_3_2,   216000,     5120,      20480,   20000, "3.2" },
    {   H264_LEVEL_4_0,   245760,     8192,      32768,   20000, "4"   },
    {   H264_LEVEL_4_1,   245760,     8192,      32768,   50000, "4.1" },
    {   H264_LEVEL_4_2,   522240,     8704,      34816,   50000, "4.2" },
    {   H264_LEVEL_5_0,   589824,    22080,     110400,  135000, "5"   },
    {   H264_LEVEL_5_1,   983040,    36864,     184320,  240000, "5.1" },
    {   H264_LEVEL_5_2,  2073600,    36864,     184320,  240000, "5.2" },
    {   H264_LEVEL_6_0,  4177920,   139264,     696320,  240000, "6"   },
    {   H264_LEVEL_6_1,  8355840,   139264,     696320,  480000, "6.1" },
    {   H264_LEVEL_6_2, 16711680,   139264,     696320,  800000, "6.2" },
};

typedef struct H265levelspec_t {
    RK_U32 maxLumaSamples;
    RK_U32 maxLumaSamplesPerSecond;
    RK_U32 maxBitrateMain;
    RK_U32 maxBitrateHigh;
    RK_U32 maxCpbSizeMain;
    RK_U32 maxCpbSizeHigh;
    RK_U32 minCompressionRatio;
    RK_S32 levelEnum;
    const char* name;
    RK_S32 levelIdc;
} H265levelspec;

H265levelspec levels[] = {
    { 36864,    552960,     128,      MAX_UINT, 350,    MAX_UINT, 2, H265_LEVEL1,   "1",   10 },
    { 122880,   3686400,    1500,     MAX_UINT, 1500,   MAX_UINT, 2, H265_LEVEL2,   "2",   20 },
    { 245760,   7372800,    3000,     MAX_UINT, 3000,   MAX_UINT, 2, H265_LEVEL2_1, "2.1", 21 },
    { 552960,   16588800,   6000,     MAX_UINT, 6000,   MAX_UINT, 2, H265_LEVEL3,   "3",   30 },
    { 983040,   33177600,   10000,    MAX_UINT, 10000,  MAX_UINT, 2, H265_LEVEL3_1, "3.1", 31 },
    { 2228224,  66846720,   12000,    30000,    12000,  30000,    4, H265_LEVEL4,   "4",   40 },
    { 2228224,  133693440,  20000,    50000,    20000,  50000,    4, H265_LEVEL4_1, "4.1", 41 },
    { 8912896,  267386880,  25000,    100000,   25000,  100000,   6, H265_LEVEL5,   "5",   50 },
    { 8912896,  534773760,  40000,    160000,   40000,  160000,   8, H265_LEVEL5_1, "5.1", 51 },
    { 8912896,  1069547520, 60000,    240000,   60000,  240000,   8, H265_LEVEL5_2, "5.2", 52 },
    { 35651584, 1069547520, 60000,    240000,   60000,  240000,   8, H265_LEVEL6,   "6",   60 },
    { 35651584, 2139095040, 120000,   480000,   120000, 480000,   8, H265_LEVEL6_1, "6.1", 61 },
    { 35651584, 4278190080U, 240000,  800000,   240000, 800000,   6, H265_LEVEL6_2, "6.2", 62 },
    { MAX_UINT, MAX_UINT, MAX_UINT, MAX_UINT, MAX_UINT, MAX_UINT, 1, H265_LEVEL8_5, "8.5", 85 },
};

[lexano-ivs]

I have lowered the default profile level for the HEVC encoder to AMF_LEVEL_5_2 to closer match the default of the other encoders considering the corresponding frame sizes. This new default profile level will take effect in a future driver release.

This helps, however I think a better overall approach would be to have an explicit "AUTO" mode that will calculate the best-fit level in AMF. It's what I've done in OBS with this commit, and there's another example posted by @nyanmisaka.

[lexano-ivs]

I have also updated the documentation and it will include notes on automatic level adjustment upwards based on the frame size in a subsequent AMF SDK release.

Thanks, this sounds good. Is the documentation on Github already? I'd like to understand the behavior, and I couldn't find it.

[rhutsAMD]

FYI, just implement a table like this in your AMF runtime so that the level will automatically be set to the lowest value that meets the spec. Both the Intel and NVIDIA encoders behave this way.

typedef struct H264eLevelInfo_t {
H264Level level;
RK_S32 max_MBPS; /* Max macroblock rate per second /
RK_S32 max_MBs; / Max frame size /
RK_S32 max_DpbMBs; / Max decoded picture buffer size /
RK_S32 max_BR; / Max video bitrate */
const char *name;
} H264eLevelInfo;

H264eLevelInfo level_infos[] = {
/* level max_MBs max_MBPS max_DpbMBs max_BR name /
{ H264_LEVEL_1_0, 1485, 99, 396, 64, "1" },
{ H264_LEVEL_1_b, 1485, 99, 396, 128, "1.b" },
{ H264_LEVEL_1_1, 3000, 396, 900, 192, "1.1" },
{ H264_LEVEL_1_2, 6000, 396, 2376, 384, "1.2" },
{ H264_LEVEL_1_3, 11880, 396, 2376, 768, "1.3" },
{ H264_LEVEL_2_0, 11880, 396, 2376, 2000, "2" },
{ H264_LEVEL_2_1, 19800, 792, 4752, 4000, "2.1" },
{ H264_LEVEL_2_2, 20250, 1620, 8100, 4000, "2.2" },
{ H264_LEVEL_3_0, 40500, 1620, 8100, 10000, "3" },
{ H264_LEVEL_3_1, 108000, 3600, 18000, 14000, "3.1" },
{ H264_LEVEL_3_2, 216000, 5120, 20480, 20000, "3.2" },
{ H264_LEVEL_4_0, 245760, 8192, 32768, 20000, "4" },
{ H264_LEVEL_4_1, 245760, 8192, 32768, 50000, "4.1" },
{ H264_LEVEL_4_2, 522240, 8704, 34816, 50000, "4.2" },
{ H264_LEVEL_5_0, 589824, 22080, 110400, 135000, "5" },
{ H264_LEVEL_5_1, 983040, 36864, 184320, 240000, "5.1" },
{ H264_LEVEL_5_2, 2073600, 36864, 184320, 240000, "5.2" },
{ H264_LEVEL_6_0, 4177920, 139264, 696320, 240000, "6" },
{ H264_LEVEL_6_1, 8355840, 139264, 696320, 480000, "6.1" },
{ H264_LEVEL_6_2, 16711680, 139264, 696320, 800000, "6.2" },
};
typedef struct H265levelspec_t {
RK_U32 maxLumaSamples;
RK_U32 maxLumaSamplesPerSecond;
RK_U32 maxBitrateMain;
RK_U32 maxBitrateHigh;
RK_U32 maxCpbSizeMain;
RK_U32 maxCpbSizeHigh;
RK_U32 minCompressionRatio;
RK_S32 levelEnum;
const char name;
RK_S32 levelIdc;
} H265levelspec;

H265levelspec levels[] = {
{ 36864, 552960, 128, MAX_UINT, 350, MAX_UINT, 2, H265_LEVEL1, "1", 10 },
{ 122880, 3686400, 1500, MAX_UINT, 1500, MAX_UINT, 2, H265_LEVEL2, "2", 20 },
{ 245760, 7372800, 3000, MAX_UINT, 3000, MAX_UINT, 2, H265_LEVEL2_1, "2.1", 21 },
{ 552960, 16588800, 6000, MAX_UINT, 6000, MAX_UINT, 2, H265_LEVEL3, "3", 30 },
{ 983040, 33177600, 10000, MAX_UINT, 10000, MAX_UINT, 2, H265_LEVEL3_1, "3.1", 31 },
{ 2228224, 66846720, 12000, 30000, 12000, 30000, 4, H265_LEVEL4, "4", 40 },
{ 2228224, 133693440, 20000, 50000, 20000, 50000, 4, H265_LEVEL4_1, "4.1", 41 },
{ 8912896, 267386880, 25000, 100000, 25000, 100000, 6, H265_LEVEL5, "5", 50 },
{ 8912896, 534773760, 40000, 160000, 40000, 160000, 8, H265_LEVEL5_1, "5.1", 51 },
{ 8912896, 1069547520, 60000, 240000, 60000, 240000, 8, H265_LEVEL5_2, "5.2", 52 },
{ 35651584, 1069547520, 60000, 240000, 60000, 240000, 8, H265_LEVEL6, "6", 60 },
{ 35651584, 2139095040, 120000, 480000, 120000, 480000, 8, H265_LEVEL6_1, "6.1", 61 },
{ 35651584, 4278190080U, 240000, 800000, 240000, 800000, 6, H265_LEVEL6_2, "6.2", 62 },
{ MAX_UINT, MAX_UINT, MAX_UINT, MAX_UINT, MAX_UINT, MAX_UINT, 1, H265_LEVEL8_5, "8.5", 85 },
};

Thanks :) we will consider doing this in future releases.