Dynamic conversion of HDR10+ SEI to DV P8 NAL

Unsupported/EOL HW/SW Community Projects
21

Major reason is CE team in general - from prior discussions - want to limit changes to the kernel side (keeps the upgrade path simpler for them) and prefer change in kodi where possible.

It also puts all conversions in one place alongside the other rpu conversions etc. and interactions with quietvoid’s library can be in one place.

Lastly it can be implemented in other Kodi like Android so have a wider community usage and feedback ultimately.

Saying that if you have a better way to pull into kodi and ready to do the other implementation then can switch - was not much work to create them.

22

Didn’t mean to put it in the kernel. I just meant why not use the header from that tool in the kodi code instead of making structs yourself. Realise it wouldn’t have been much work to do yourself (probably easier than getting the library itself into the code) - it was more a question why you took that approach.

23

Oh yeah - out and about - so not reading carefully and replying on the fly!

Was going to have a go at doing the actual conversion in a very basic way to P8.1 all in Kodi side to start with based on what I am understanding of the Rust implementation and refactor later to use library when the generator is exposed, so just getting something in place to start to read the data and understand it some more - first frame no curves so not the best example.

And yeah simpler to just have own struts rather than pull in another library currently - I have most of the conversion down to the next level of actual required data and skeleton of creating the RPU but still some more to learn.

1 Like
24

FYI you could probably just have used the FFmpeg API?
av_dynamic_hdr_plus_from_t35 should be usable in libavutil/hdr_dynamic_metadata.h

1 Like
25

This would be an awesome feature for LG users like me. I just ordered my first HDR10+ blu-ray (Sakra) which I’ll convert offline but it’d be great to have CoreELEC do this automatically

1 Like
26

Does anyone have an HDR10+ mkv with offset metadata like the DV rpu offset test file - i.e. where can obviously see the metadata changing the video scene. thx.

27

I don’t have an HDR10+ TV. Maybe here

1 Like
28

Building on the work of @quitevoid and @doppingkoala, I think I now have a working solution, needs quite a bit of refinement and has some gaps want to close so not sharing anytime soon, but getting DV out now!

Funny thing is though the sample from @bluesea does not work, something else different with that sample, on tv-led conversion I just get a black screen (not dug into it yet - as working on other areas), interestingly the same black screen happens when trying to do (HDR10+) HDR10 → DV using VS10 for that sample as well. A very quick search online indicates some kind of bug which this sample highlighted and was fixed in other players/TV but not much on the specifics - but maybe related.

So at this point parking that sample, but that means putting the call out again for an alternative mkv HDR10+ where can easily see a change in dtm!

@doppingkoala - does that sample work in your solution?

Would also be interested if that sample converted via dovi_tool works ok - has anyone has done that?

2 Likes
29

yes, I can see the changes in the metadata.

That sample is slightly different though in that it is 60fps. So on my device I do see flickers on the screen, but that is unrelated to the HDR10+ / DV work - it happens for all content on my device with a 4k resolution and 60 fps video.

30

Maybe you could change the frame rate of the sample to 24?

31

I’ve done that. Works fine when played using the internal player of my TV.

32

If you don’t mind sharing would be helpful (save me time!).

34
1 Like
35

More fun times…

I looks to me the HDR10 Metadata is not being picked up on the Kodi side!
In xbmc/cores/VideoPlayer/DVDDemuxers/DVDDemuxFFmpeg.cpp

Added some quick logging which are not being called e.g. av_stream_get_side_data is not returning data.

So I cannot correctly convert the HDR10+ to RPU.

Anyone know about this issue?

  side_data = av_stream_get_side_data(pStream, AV_PKT_DATA_MASTERING_DISPLAY_METADATA, &size);
  if (side_data && size)
  {
    CLog::Log(LOGINFO, "DVDDemuxFFmpeg::AddStream - mastering display metadata");
    st->masteringMetaData = std::make_shared<AVMasteringDisplayMetadata>(
        *reinterpret_cast<AVMasteringDisplayMetadata*>(side_data));
  }

  side_data = av_stream_get_side_data(pStream, AV_PKT_DATA_CONTENT_LIGHT_LEVEL, &size);
  if (side_data && size)
  {
    CLog::Log(LOGINFO, "DVDDemuxFFmpeg::AddStream - content light metadata");
    st->contentLightMetaData = std::make_shared<AVContentLightMetadata>(
        *reinterpret_cast<AVContentLightMetadata*>(side_data));
  }
1 Like
36

Great work as always
Maybe @DMDreview or @R3S3T_9999 could help providing MKVs or testing as they happen to be the most experienced I can think of

37

Could not find a way to get the MASTERING_DISPLAY or CONTENT_LIGHT_LEVEL via the ffmpeg lib in the flow, so have extracted them later in the process directly from the Prefix SEI NAL in the bitstream, so that bit out of the way for now.

Also noted the ffmpeg lib included in ng 21.1 does not have the parser for the hdr10+ SEI anywhere I can see, so continuing to use my own parser and thus own metadata structure for now.

Testing files

A. Amazon Prime HDR10+ (Reacher)
B. FF Pictures HDR10+ system test

So far observe the following:

A. Concentrating on the the Amazon Prime intro splash - it looks to have changes, but translating into large RPU changes (obviously wrong) not sure were the issue is currently - but thinking something to do with multiple Windows in HDR10+ (more on that below).

B. I see no easily noticeable change in this file at all, but then again from doppingkoala’s DV converted mkv I don’t see any either - so maybe something more fundamental broken.

HDR10+ Processing Windows

In the Metadata it looks to have up to 3 “processing windows”, these look to apply different metadata to different areas of the frame?

I can see the number of windows is changing between scenes and frames 1, 2 or 3, also see some with 0 windows.

If I am not mistaken (good possibility I am!) the conversion to DV by dovi_tool is picking the last window (if any available) to convert to DV. doppingkoala’s port I think is picking the first window. If no window then defaulting values?

I am guessing the windows is an attempt to apply different metadata to different elements in the frame (though not go my head around why that is really necessary - possibly can preserve details where a strong highlight in a small section needs to be mapped down but the rest of the frame can be left alone and that is better?)

If number of windows is 0 should it just continue with the last RPU or have none maybe?

Not found any good info yet on the windows - and how it should be used, but my presumption is we should look across them all? find the max etc. It maybe a bit of a show stopper to translate down to only one value for DV tone mapping at the complete frame level.

EDIT: This element from below docs I would think is referring to the windows -and it’s approach better than “other” presumably talking about DV.

HDR10+ dynamic metadata benefits

Unlike other dynamic metadata systems, HDR10+ also identifies the most important
areas of each scene to improve the reproduction of those areas. There are three steps.

  1. During HDR10+ mastering, the system gathers comprehensive “luminance
    distribution” statistics that describe all the grayscale levels of each scene. The
    metadata include the exact number of pixels in nine different grayscale ranges.
  2. The mastering system then uses this data to construct a highly sophisticated,
    customized tone mapping curve for each scene. This “guided curve” is also
    included in the metadata.
  3. As with HDR10, each consumer device adjusts the tone mapping curve according
    to the specific brightness capability of the individual display.
    As a result, HDR10+ does much more than deliver highlights optimized on a scene-byscene basis. You also get better rendering of subtle shades in the most important areas of
    each scene, where image elements like skin tones reside.

Whitepaper and pdf

Have looked at the following but nothing on windows so far.

Blu Ray with HDR10+

List for the interested, not checked but guess most will have DV also already, or could HYBRID in an RPU from another source in most cases.

https://www.blu-ray.com/movies/search.php?action=search&ultrahd=1&hdr10plus=1&sortby=relevance

2 Likes
38

I’ve check that file again on the internal player of my TV. Clear changes. File was generated using a recent copy of DoVi_Scripts.

See https://pub.smpte.org/doc/st2094-40/20200409-pub/st2094-40-2020.pdf

Of note:

Processing Window 0 shall be always present and shall cover all pixels in an image and shall not be extended with the elliptical pixel selector.

Also:

A metadata set shall contain exactly one of each of the following:
…
ApplicationVersion (= 0 or 1)

and then later,

Metadata Set ApplicationVersion = 0 Constraints
…
Any ProcessingWindow.WindowNumber > 0 should not be present

and

Metadata Set ApplicationVersion = 1 Constraints
…
Any ProcessingWindow.WindowNumber > 0 ~should~ shall not be present

Not sure I am understanding this properly, but does this essentially say that only one windows is ever allowed? Doesn’t really seem to make sense

Didn’t test all that thoroughly, but the data I looked at only had one window. From the above it seems like a sensible window to use as it covers all the pixels and should always be present (and only one is allowed ?).

Didn’t actually realise dovi_tool was picking the last one, I only worked out that it was using only one. I did compare some of the produced data and ended up with the same result. Must have only been one window.

2 Likes
39

Link to a sample showing this?

40

Thanks for the smpte link - lots of details to absorb.

I am thinking my parser is wrong so would not put to much into the observations at this point.

Good to know - guess I broke something elsewhere.

So from reading the doc my understanding is there is a base (window 0) which can be augmented by two more elliptical windows (window 1 and 2)

The amlogic parser implies for the windows it parses it contains bit space for the ellipse data, which according to the spec is only applicable to windows 1 and 2.

But the spec also implies there is always corner data and window number for each window and just elliptical data for elliptical windows.

For the luminance (maxscl et. al.) it is processing 0 to number of windows, so should always have values for the none-elliptical window “0”.

Section 9.3 is certainly odd - one small diff - I think you had a typo:

9.3 Metadata Set ApplicationVersion = 0 Constraints
Any ProcessingWindow.WindowNumber > 0 should not be present.

D.2 Revision 1
ApplicationVersion = 0 aims to maintain compatibility with the 2016 edition and includes:
Some optional metadata items and WindowNumber > 0 are not recommended. (Section 9.3)

9.4 Metadata Set ApplicationVersion = 1 Constraints
Any ProcessingWindow.WindowNumber > 0 shall not be present.

D.2 Revision 1
ApplicationVersion = 1 addresses new constraints or requirements:
Some optional metadata items and WindowNumber > 0 are not permitted. (Section 9.4)

Still odd though implies for version 0 it should not be there but it could be?

Overall it seems to imply currently no version actually uses the elliptical windows (though may be there so parse for it!).

I will ping over a link later to the Amazon Prime where I was seeing multiple windows in my parser (could well be a mirage from my parser not being correct) or maybe they should just be ignored. If you want to grab before then - as may not be able to put up for a while - it is Reacher Season 1.

41

File removed.