Kodi Audio Out of Sync playing file or after seeking

If i remeber correctly Openelec and Libreelec had a default of -175ms delay in advancedsetting made on installation back in the day.

1 Like

You can do a more objective test by making a video recording of the playback. Then load the recording into video editing software that shows the audio waveform. Then step through that frame-by-frame and look for the clear spike in the audio waveform.

To account for any possible audio sync errors of your camera, you can also do the same thing with something that physically makes noise, i.e., recording a clap.

Repeat of

with new log and dmesg:
kodi log: dtshd_20241123090328_kodi.log (2.5 MB)
dmesg: https://paste.coreelec.org/BinderFilms

Reading latency by EDID will not work, itā€™s not safe and not supported by most systems. Also itā€™s fact latency is different for different resolutions:

vLatency:  Invalid/Unknown
aLatency:  Invalid/Unknown
i_vLatency:  Invalid/Unknown
i_aLatency:  Invalid/Unknown

So I added a new latency tweak to advanced settings:

<advancedsettings>
	<video>
		<latency>
			<resolution>
				<strId>2160p24hz</strId>
				<delay>-166.833344</delay>
			</resolution>
		</latency>
	</video>
</advancedsettings>

So for each resolution a different latency can be set.
Itā€™s applied on end. First is delay, may override by sub node refresh delay, may override by sub node resolution delay.

So latency is now calculated by:

  • tweak latency [advanced settings latency]
  • + display latency [(framebuffercount + 1) / framerate]
  • - user set audio delay

So for the sample above with resolution latency of -166.833344ms the final value will be 0ms latency because [(3 + 1) / 23,976] = 166.833344ms.

Here a test image:

Itā€™s now the question is this enough? It can not be auto corrected as the latency is different for every system. The default do cover mostly of hardware setups but some need a tweak.
Or is there another tweak need like somehow by resolution + audio format?
Or is resolution alone enough?

Itā€™s also the question if same like presentation buffer number can/should be used instead a ms value. Default is 3 for triple buffering and can be decreased to double buffering. But maybe 2160p24hz do just need another number like 5 or 6.

7 Likes

Wow, thatā€™s great! Adding those advanced settings should help with a lot of unique latency cases for some setups. I created a plugin to help manage the audio offsets based on HDR type, audio format, and FPS type, so maybe in combination with your advanced settings it will cover most different setups and videos.

But, I suppose adding everything to advanced settings instead of using a plugin would be more granular, maybe just not as user friendly.

1 Like

Just found this audio offset manager addon anybody tested? Not releated but maybe usefull?

https://forum.kodi.tv/showthread.php?tid=379179#

Thatā€™s the one I just shared above your message :laughing:

1 Like

This addon looks quite good.
Itā€™s hard to code all combinations into advanced settings.
So maybe a live tweak latency value what just can be changed on the fly is a better idea. Then the addon just set a float value depending on user settings.

3 Likes

@matthane I did now add a new Integer Setting: videoscreen.latency.
Kodi do check if your addon script.audiooffsetmanager is installed and enabled. If yes it read this value and add it to the latency. The limits are in frames -20ā€¦20, default is 0.

The latency in milliseconds is then calculated by frames * frame time.
I guess itā€™s easier for users to adjust a latency in frames than in milliseconds.

So when your addon is installed and you change this integer setting while video playback you should see a live change as it directly change the render pts for current frame. I guess itā€™s then easy to adjust the latency while playing a calibration video by moving like a slider for- or backwards until audio and video is in sync.

The actual integer setting value is always reset to 0 on playback start. So you need to set the value format dependent again on every playback start if need.

Itā€™s not tested but maybe it just worksā€¦
When you have something ready to change the latency you can check it by kodi debug log with video component enabled if the value is used at all. Itā€™s called m_latencyVideoscreen in the log file.

3 Likes

Ok cool, I will try this latest build and see how it works. However, the download link is giving an error now saying itā€™s been removed.

Edit: Download worked now, thanks

Yes, I did now test and fixed it finally. Link is update above.

I installed your addon and did change the value live by jsonrpc:
Read:

curl -v -X POST -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","id": 1, "method":"Settings.GetSettingValue", "params": {"setting": "videoscreen.latency"} }' http://127.0.0.1:8080/jsonrpc

Write:

curl -v -X POST -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","id": 1, "method":"Settings.SetSettingValue", "params": {"setting": "videoscreen.latency", "value": 5} }' http://127.0.0.1:8080/jsonrpc

And I did see it clear the ā€œbeepā€ is moving +5 frames away while playing.

The check if addon is installed will be replaced by like a new setting.
Something like ā€œvideoscreen.enablelatencyā€.
But for first tests it should be enough.

Example in kodi.log:

2024-11-26 14:02:40.417 T:4027    debug <general>: PrepareNextRender: frameOnScreen: 8.354 renderPts: 8.571 nextFramePts: 8.634 -> diff: -0.063  render: 0 forceNext: 0, m_queued: 15, m_queued empty: true, begin idx: 0, front: 0, resInfo.strId: 2160p24hz, m_latencyTweak: 0.000000, m_latencyVideoscreen: 0.000, GetDisplayLatency: 166.833344, m_videoDelay: 0, GetFrameLatencyAdjustment: 0.000000
2024-11-26 14:02:40.421 T:4221    debug <CWebserver[8080]>: request received for /jsonrpc
2024-11-26 14:02:40.421 T:4221    debug <general>: JSONRPC: Incoming request: {"jsonrpc":"2.0","id": 1, "method":"Settings.SetSettingValue", "params": {"setting": "videoscreen.latency", "value": 5} }
2024-11-26 14:02:40.459 T:4027    debug <general>: PrepareNextRender: frameOnScreen: 8.354 renderPts: 8.780 nextFramePts: 8.634 -> diff: 0.146  render: 1 forceNext: 0, m_queued: 15, m_queued empty: true, begin idx: 0, front: 0, resInfo.strId: 2160p24hz, m_latencyTweak: 0.000000, m_latencyVideoscreen: 208.542, GetDisplayLatency: 166.833344, m_videoDelay: 0, GetFrameLatencyAdjustment: 0.000000

So m_latencyVideoscreen: 0.000 did change to m_latencyVideoscreen: 208.542 by 5 frames with 23.967Hz.

1 Like

Ah, thanks for the detail.

My addon mainly works by:

  1. Reading stream info (source HDR, audio format, fps)
  2. Checking stream info against addon settings configured by user
  3. Applying audio offset accordingly using JSON-RPC method ā€œPlayer.SetAudioDelayā€.

Iā€™m just curious on what the value of adding another latency value setting is? Is it adding complexity for little gain? I appreciate your work but I want to make sure that we arenā€™t adding complexity or double work since the addon can live fully on its own without needing direct integration.

Iā€™m not a programmer by trade, so itā€™s entirely possible Iā€™m missing the point of what youā€™re trying to do :smile: I think that what you did initially with adding resolution specific audio settings makes sense, because then that can be used as baseline to account for differences in output time per resolution, while still using the addon to control for the rest of the delay due to audio formats.

I did not know there is some value already and I am not sure if it should be changed as itā€™s user gui setting.
You can check m_videoDelay in kodi.log if the value do change while playing when you use Player.SetAudioDelay. When it does change live while playing you can directly use this and the videoscreen.latency is not need.

Ok, yes Iā€™m using SetAudioDelay and also monitoring the GUI slider for changes during playback (if active monitoring mode in addon is enabled) to write back into settings to keep everything consistent in the GUI. I submitted the addon to Kodi repo a few weeks ago, and itā€™s still in their PR queue. Iā€™m waiting to see if they have any problems with it :laughing:

Currently, I had the best results with your previous builds with hardcoded latency totally removed. In that build, I was able to set offsets that worked very well and did not fluctuate much at all when seeking around a video. I think I observed some slight modulations, maybe + or - one frame in rare cases, but overall it was very stable across different FPS types. Very happy with that one!!

Ok, then the extra settings value is not need at all.
The latency removed will not be happen as it does match like for 99% of hardware setups out there. But itā€™s a constant as it is just a offset of 4 or 3 frames. This is why I think a handling by frames is more comfortable as it does not so much confuse like milliseconds.

Implement this as advanced settings is hard as there are so many combinations like resolution, Hz, audio codec and passthrough.

So a solution by a addon is the way to go. When the addon will not be merged by TK we can include it on CE addon repo.

6 Likes

One month without any comment, we might add it directly to our repo?

So you might open a PR to the CE add-ons:

Iā€™ve tested this release (CoreELEC-Amlogic-ng.arm-21.1.1-Omega_devel_20241125100838.tar) and it worked but not the way I thought initially :slight_smile:

I thought the contents of the advancedsettings.xml file were a 1:1 transposition of the audio delay slider on per video settings but they werenā€™t. At first, I used a delay of -250 like in the slider but didnā€™t worked and I had to manually set the video delay to ā€œDelayed by 500 msā€ on the video to get in sync. Obviously, the double result showed that I needed to set a positive delay in the XML file.

Long story short : For me, setting strId=2160p24hz and delay=250 solved everything! All my 13 short clips of lip sync tests worked perfectly, even by jumping forward or backward. These clips use TrueHD/Atmos, DTS-MA/X and EAC-3 and all are extracted from movies. All the 1080p files I tested also worked fine (ie: not affected by any delay).

The PhotoJoseph 23.976 video beeped right at 0 this time :smile: and the other tests like 25, 29.970, 30, 50 and 60 are of course not affected and are still delayed. Iā€™m waiting to have real movie content to further configure the XML file instead of relying on these beeps.

This is perfect! Thank you very much for your hard work.

Through out my career, I dealt with lots of companies charging pretty penny for ā€œtechnical supportā€ which often boiled down to ā€œdid you rebootā€-level recommendations, and there are the wizards of CoreELEC! Several orders of magnitude higher in quality, thank you!

6 Likes

@Portisch Ok, how does CE maintain itā€™s addon repo? Should I submit to lowest compatible branch like Kodi does? For example, Iā€™m using some Kodi python code that is only compatible with v20 and above. Should I PR in coreelec-20 branch? Or PR in latest coreelec-22?

CE-22, we will backport to CE-21.
CE-20 is EOL and therefore no new add-ons are included/build.

You will need such new package.mk, just look at some other add-ons how they should look like.

2 Likes

Thanks for all your help, Portisch. I will investigate when I have some more time and see if I can easily convert my Kodi addon structure into CE addon structure. Btw, is there any documentation about this?

In the mean time, someone from TK commented on my forum post there and liked the addon, so hopefully it can get approval on their Github soon :smile: