Learning about Dolby Vision and CoreELEC development

General
84

While a limited range output isn’t what we were after (and reconstructing signals after full/limited conversion may not be exact due to rounding), I don’t think technically a limited range output is incorrect per se.

However, returning to the results from your captures. The values you have captured cannot be explained by a conversion from full to limited range RGB alone - but can be with incorrect color space conversions.

Taking the outer background as a concrete example. In the original png (full range 8-bit RGB) the value is (128, 16, 0) and the captured values are (119, 19, 17), even when included a conversion to align the ranges a significant differences (over 10) still exists.

However, the mp4 you made of the test pattern proved useful. This video is encoded in YCbCr (limited range), the values for the background are (49, 110, 178). These values correspond to a conversion from full range 8-bit RGB to YCbCr (limited range) using BT.709. Again, the values from the capture do not match a conversion back from BT.709 limited range YCbCr to limited range RGB.

What does, however, match the captured values is when the YCbCr values (obtained using BT.709 conversion) are incorrectly converted to limited RGB using the BT.601 conversion.

Oddly enough, this also appears to happen in the capture of the png. It appear that coreelec/kodi is taking the full range RGB image, converting to limited range YCbCr using a BT.709 conversion, and then incorrectly converting those values to limited range RGB using a BT.601 conversion. As to why a RGB image is being transformed to YCbCr and back in any form, I have no idea …

TL;DR
CoreELEC/kodi uses inconsistent/incorrect color space conversions that has resulted in significant errors when outputting RGB.

Questions from this result open for anyone to answer:

  • Are any/the right conversions used when playing back BT.709 video with a YUV output? Probably need a capture to confirm
  • Does anyone know how to tell via ssh what color space conversions are currently being used?
  • Is this a known bug?
  • Any pointers to where in the code the logic for selected which color space conversion to use exists?
85

it’s not only because of the limited range effect, it doesn’t lead to such big errors, compared with Fire tv it can’t full range either, but it has less conversion errors, and the red color is a different shade, so it could well be because of bt601 instead of bt709

86

have you tried:

echo “rgb,8bit,now” > /sys/devices/virtual/amhdmitx/amhdmitx0/attr

do you have a ssh to enable DV-STD?

force_csc_type in drivers/amlogic/media/enhancement/amvecm/amcsc.c maybe something worth looking at.

87

No. I did try adding a call to hdmitx_set_vsif_pkt() into the function that runs when I print the display capabilities via ssh. That approach alone hasn’t seemed to work though, as it didn’t even trigger the LLDV mode in the TV.

I closest I have is the approach I used in this post.

Thanks for the links on the rgb and color space conversions

88

That is definitely the case, I’ve found the offending part in the code and checked that it is enabled when using an RGB output.

From what I gather, it appears that color space conversions (csc) can happen in more than one place. At (what I assume) is the final possible location, the hdmi_tx_hw.c code can additionally perform a csc. This occurs whenever the input format to the hdmi_tx_hw code and the output format do not match, i.e. YUV444 input and RGB output.

The problem is, unlike the color space conversion code linked to by @cpm , the hdmi_tx_hw.c code considers only a very limited number of color spaces, ranges, and conversions. Only two color space conversions defined - convert to limited RGB using bt.601 or convert from RGB using bt.601.

It also appears that the hdmi_tx_hw.c has been hardcoded to only have a YUV444 signal as it’s input. This leads to CoreELEC always outputting RGB with values that are obtained by performing a YUV to limited range RGB conversion using a bt.601 transform.

TL;DR.
It appears CoreELEC cannot output in RGB even close to correctly due to the hdmitx_csc_config() function being hardcoded to output RGB with values calculated from YUV444 using a bt.601 transform for a limited range RGB output. This occurs regardless of what color space / range / anything the YUV values are based on.

1 Like
89

Affirmative.
Here is the capture with rgb8 bit output and with YUV 8bit.
With rgb the green channel is lost. With yuv everything is correct.
I take it there is no way to fix this?
I checked on my second pro capture card, I can do capture in rgb 8 bit.
Here is an example.
adding file test 8bit rgb2.avi

color 8bit rgb
color 8bit rgb1920×1080 45.5 KB

color-yuv444-8
color-yuv444-81920×1080 52.6 KB

90

I think there is - that part of the code actually appears to have options for the hdmi_tx module to accept: Pixel format: 0=RGB444; 1=YCbCr444; 2=Rsrv; 3=YCbCr422, /* Pixel range: 0=16-235/240; 1=16-240; 2=1-254; 3=0-255., and /* Pixel bit width: 4=24-bit; 5=30-bit; 6=36-bit; 7=48-bit.

Looking at the history of the code, a YUV444 format to the internal input to the hdmi_tx module was hardcoded by someone at amlogic 7 years ago when the driver was initially added - and hasn’t been touched since.

This would also explain you observations in your chroma-subsampling video where you suggested that all these boxes may be internally converting to a 444 format as in intermediately step. It appears that this is the case - to me this seems to make the options the have a 422 output rather pointless.

I have made a test build with this variable changed - and this does disable the unwanted color space conversion in the hdmi module. At the moment though all the output colors are messed up - I’m guessing the rest of the code was setup to transform everything to YUV444 before it is passed along - and this is then sent to the TV to be interpreted as RGB.

I haven’t yet worked out how to change that, but looking at the code it does appears that we should be able to enable / disable / set pretty much and color space conversion (including custom) / range / chroma sampling you want.

It would probably take a fair bit of time to get to that point - and input from one of the actual CoreELEC devs. But for the meantime, I think I can stumble through and butcher the code enough to achieve an unaltered full range RGB output.

I’ll give you a test build when I get to that point

1 Like
91

Iirc there was a change recently made that the code had to be set tobYUV444 instead of RGB 444 (or at least its equivalent) for TV-led or else it’d result in a black screen. Then the cube 2nd gen also had this behavior I thought but in reverse? I can’t but wonder if this is related somehow?

92

Very well could be, I haven’t yet looked into how tv-led DV works with this code much.

Looking at datasheets for these chips though, the dolby vision related stuff is in the VPU section. As the VPU is before the hdmi module in the video pipeline it seems to be a good guess the metadata is already embedded into the LSB of the chroma channels before the hdmi module gets involved. If the output mode is already set as RGB when hdmitx_CSC_config() is called, this would result in enabling the csc which would ruin the signal and corrupt the embedded metadata. Might be the cause of the black screen

93

@Astrotrain I saw you mention this bug in your FEL support post. Just to clarify, this bug doesn’t occur in DV tunnelled mode - if it did tv-led dv wouldn’t work at all. It seems to just impact any other use of RGB output

I suspect this bug is avoided by the order that functions are called in - the function in question must be called when the output mode is still in YUV mode, so it wouldn’t enabled the bt.601 conversion. Either that, or the DV code simply overrides the hdmi properties elsewhere and doesn’t use this part of the code at all

94

@DMDreview What is this a capture of? Seems very close, but not identical, to your earlier capture of the mp4 you made of that test pattern

95

Weren’t you already doing that with your previous captures, just with 2 bits of padding?

96

Woot, that’s great news. I updated my post.

Out of curiosity, does this bug impact HDR10 or HDR10+?

97

Haven’t looked into it. If the output mode is set to RGB I guess so.

It looks like if the hmdi output is set to any form of YUV the csc isn’t enabled, so the unwanted bt.601 conversion is avoided. Seems like it only occurs with an actual RGB output (not tv-led DV)

edit: the hardcoded YUV444 setting may still be causing chroma upsampling and then down sampling if a YUV 422 output is used though. Would need to look more to have any confidence to say this either does or doesn’t occur though …

98

For the DoVI Tunnelling (tv-led). I think there are two different things going on - the processing and image is YUV for DoVi but the AVI Info packet is set to RGB - that would make sense for tunnelling, so the hdmi tx work/methods are being called in YUV and not actually RGB - just the VSI/AVI Info Packet is set RGB.

I.e. it is not actual RGB data in the signal.

The logging and on screen saying RGB looks like coming from the VSI/AVI Info Packet information.

99

Thanks. Hopefully, we can get a definite answer for YUV 422 output. @DMDreview Do you think you can run a test for YUV 422 content?

100

did the same thing by capturing rgb from oppo, it can output RGB 8bit full. It turned out to be a conversion, the colors are almost equal to the original, but there is a difference on the blue small on colors. I.e. Oppo probably does the RGB ycbcr-rgb conversion internally too.
And only when I made an image capture from a laptop with Intel video card, set it to rgb 8bit full range, I was able to get a 100% identical picture with the original png file.
It also turned out that BMD card also knows how to capture properly in rgb 8 bit, only you have to use a different utility to capture. Then there’s the honest 8 bit rgb codec.

101

what do you mean?

102

Think your right here.

The thought to do the same trick occurred to me when I saw the completely messed up colors when I disabled the bt.601 conversion and set the output mode to rgb.

103

@DMDreview If you are talking about the file test 8bit rgb2.avi, that capture is limited range rgb. The white values are (235, 234, 235) and the black is (16, 15, 16). Don’t know why these aren’t all 235 or 16, respectively though.

edit: the file dv-test-orig video.mp4 you made doesn’t explicitly flag the range as limited, guess that leaves it up to the player to make a guess… So the oppo may be outputting full-range RGB but interpreting the video as full range when it is limited.