I have a large video library on a home server. For several years I use CoreElec 9.2.8 (on a S912 box with 3GB RAM; this is the last CoreElec version which supports this CPU). I used to a long delay when entering the samba share folder with very long list of TV shows (it takes up to 35 seconds for the first entry to populate all the library content from Database). I can enter some sub-folder, and then return back to the main Large folder, it takes less then 2 seconds to repopulate the large list of shows.
I recently acquired several TV boxes with more recent Amlogic CPUs (S905Y4, and two S905W2, all are with 4GB RAM).
I had to install COreElec-ne 20.2 (9.2.8 does not support any of these new CPUs).
I copied the library (took a while to upgrade to a new DB format), it is working properly. But re-read of the Large samba folder is extremely slow on these boxes.
First entry takes the same long time (35 sec or so). But re-entry to the same folder (return from subfolder) is much slower - about 16 seconds. It is very annoying.
Looks like CoreElec 20.2 does not keep the content of the large folder in the cache, it tries to re-read it every time. And I can not find the settings for cache size.
I do use fast SD-card (based on the results of Crystall DiskMark, it is 2-fold faster then my old Samsung EVO SD card which I use in old S912 box). Thus, slow population of large samba folder content is NOT because of SD card.
Try this and put it in advancedsetings.xml
Maybe it helps…
<advancedsettings version="1.0">
<samba>
<statfiles>false</statfiles> <!-- Set to false to disable smb stat() on files to speed up listings of large directories (over slow links) -->
</samba>
</advancedsettings>
Thanks.
I will try those settings.
But I am not optimistic about that.
Yesterday, I tried fresh installation of CoreElec 20.2 (with empty library). Re-entry into large Samba folder is less then 2 seconds (similar to what I used to on CoreElec 9.2.8 on old Amlogic box).
It’s the presence of library content which slows the things down. Since library is locally stored, I doubt Samba stat() settings will help. I will try anyway, cause I am getting desperate.
May be there are options to run 9.2.8 on newer Amlogic CPUs (s4)?
indeed, it did not help.
I was analyzing the kodi.log file, and I stumbled across a LOT of debug messages like:
error : Unable to lookup host: ‘elephant’
elephant is my media server.
Kodi has no problem with browsing my
smb://elephant/storage/
folder, and playing files from it, nevertheless it has a LOT of these debug error messages in the log file. They are popping up every 10 seconds or so. If I leave kodi alone, kodi.log is simply flooded with these non-stopping messages:
2023-11-21 09:49:02.176 T:28299 info : Skipped 398 duplicate messages…
2023-11-21 09:49:02.176 T:28299 error : Unable to lookup host: ‘elephant’
2023-11-21 09:49:12.183 T:28299 info : Skipped 480 duplicate messages…
2023-11-21 09:49:12.183 T:28299 error : Unable to lookup host: ‘elephant’
2023-11-21 09:49:22.195 T:28299 info : Skipped 502 duplicate messages…
2023-11-21 09:49:22.195 T:28299 error : Unable to lookup host: ‘elephant’
2023-11-21 09:49:32.219 T:28299 info : Skipped 491 duplicate messages…
2023-11-21 09:49:32.219 T:28299 error : Unable to lookup host: ‘elephant’
2023-11-21 09:49:42.256 T:28299 info : Skipped 480 duplicate messages…
2023-11-21 09:49:42.256 T:28299 error : Unable to lookup host: ‘elephant’
never mind, edited ~/.config/hosts.conf to include
10.0.0.102 elephant
Now, I don’t have this "Unable to lookup host: ‘elephant’ " error in the log file, but long pause upon return to parent folder on a samba server is still there
here is an excerpt from kodi.log (I deleted only preceding lines, I did NOT delete anything between these lines in excerpt).
It shows results of my SMB folder browsing.
Originally, I was inside folder smb://ELEPHANT/storage_enc/TV series/3 Percent/Season 1/
and then I clicked “back” button on remote twice to return to main folder (smb://ELEPHANT/storage_enc/TV series/ is a LARGE folder, more then 1000 subfolders).
As you can see, when I am inside a folder with very few files or folders, thread “BackgroundLoader” starts pretty much immediately after thread “waiting” is terminated.
But when I go back inside LARGE folder, there is 10+ seconds delay between end of “waiting” and start of “BackgroundLoader” threads. And there is nothing between these two lines.
Why thread BackgroundLoader is starting so late?
Is this 10+ second gap the time when kodi is re-reading the information from DataBase (though, there are no any traces in the log file)?
2023-11-21 10:18:27.115 T:3741 debug <general>: LIRC: - NEW 160 0 KEY_OK devinput (KEY_OK)
2023-11-21 10:18:27.147 T:3737 debug <general>: HandleKey: 11 (0xb, obc244) pressed, window 10025, action is Select
2023-11-21 10:18:27.148 T:3737 debug <general>: CGUIMediaWindow::GetDirectory (smb://ELEPHANT/storage_enc/TV series/3 Percent/Season 1/)
2023-11-21 10:18:27.148 T:3737 debug <general>: ParentPath = [smb://ELEPHANT/storage_enc/TV series/3 Percent/]
2023-11-21 10:18:27.149 T:3931 debug <general>: Thread waiting start, auto delete: false
2023-11-21 10:18:27.160 T:3931 debug <general>: Thread waiting 3829396160 terminating
2023-11-21 10:18:27.367 T:3932 debug <general>: Thread BackgroundLoader start, auto delete: false
2023-11-21 10:18:27.473 T:3932 debug <general>: Thread BackgroundLoader 3518509760 terminating
2023-11-21 10:18:29.044 T:3741 debug <general>: LIRC: - NEW 9e 0 KEY_BACK devinput (KEY_BACK)
2023-11-21 10:18:29.080 T:3737 debug <general>: HandleKey: menu (0xd8) pressed, window 10025, action is Back
2023-11-21 10:18:29.081 T:3737 debug <general>: CGUIMediaWindow::GetDirectory (smb://ELEPHANT/storage_enc/TV series/3 Percent/)
2023-11-21 10:18:29.081 T:3737 debug <general>: ParentPath = [smb://ELEPHANT/storage_enc/TV series/]
2023-11-21 10:18:29.082 T:3933 debug <general>: Thread waiting start, auto delete: false
2023-11-21 10:18:29.089 T:3933 debug <general>: Thread waiting 3829396160 terminating
2023-11-21 10:18:29.293 T:3934 debug <general>: Thread BackgroundLoader start, auto delete: false
2023-11-21 10:18:29.458 T:3934 debug <general>: Thread BackgroundLoader 3518509760 terminating
2023-11-21 10:18:30.432 T:3741 debug <general>: LIRC: - NEW 9e 0 KEY_BACK devinput (KEY_BACK)
2023-11-21 10:18:30.467 T:3737 debug <general>: HandleKey: menu (0xd8) pressed, window 10025, action is Back
2023-11-21 10:18:30.467 T:3737 debug <general>: CGUIMediaWindow::GetDirectory (smb://ELEPHANT/storage_enc/TV series/)
2023-11-21 10:18:30.467 T:3737 debug <general>: ParentPath = [smb://ELEPHANT/storage_enc/]
2023-11-21 10:18:30.468 T:3935 debug <general>: Thread waiting start, auto delete: false
2023-11-21 10:18:30.568 T:3737 debug <general>: ------ Window Init (DialogBusy.xml) ------
2023-11-21 10:18:31.148 T:3935 debug <general>: Thread waiting 3829396160 terminating
2023-11-21 10:18:31.150 T:3737 debug <general>: ------ Window Deinit (DialogBusy.xml) ------
2023-11-21 10:18:41.942 T:3936 debug <general>: Thread BackgroundLoader start, auto delete: false
2023-11-21 10:18:42.571 T:3927 debug <general>: ffmpeg[0x2184b58]: [image2] Custom AVIOContext makes no sense and will be ignored with AVFMT_NOFILE format.
2023-11-21 10:18:42.577 T:3928 debug <general>: ffmpeg[0x27dd230]: [image2] Custom AVIOContext makes no sense and will be ignored with AVFMT_NOFILE format.
I bet there is a bug in Samba lib, minimum since Kodi Leia. When you resume from suspend and access the NAS where the HDD have spinned down Kodi freeze and must be killed and restarted.
But it’s very hard to debug such behavior and I don’t think TK or LE have an idea about it at all.
So far, the only solution I was able to find for my s905w2-based TV-box:
use preinstalled Android 11, install Kodi 18.9 (Leia) from apk file and copy the library from my CoreElec 9.2.8 box (Database and Thumbnails folders).
Works good, less then 2 seconds delay upon returning to LARGE samba folder.
After Leia, Kodi has troubles to populate the info from the database for LARGE folders.
Kodi 18.9 crashes once in a while on Android 11, but at least it is usable. Starting from version 19, Kodi is heavily broken (to the point that it s not-usable in my environment).
External share is MergerFS (several ext4 grouped together).
As I said, share is large. Takes few seconds for Windows to list the folder for the first time (on 1000Mbps ethernet), then it works with cached file system and makes instant switching between folders.
On or TV-box (amlogic CPUs have built in 100Mbps ethernet controller) it is significantly longer to open the large folder for the first time (30+ seconds), and then again it works with cached file system - much faster.
While Kodi 18.x needs up to 2 seconds to retrieve the info from library database for that large folder,
Kodi 19+ needs significantly longer time to do that (12 seconds or so). You can even see the progress of these two separate processes looking at the rotating “clock” symbol. First 30 seconds (when it reads the info from samba folder) “clock” is “ticking” every second, and then additional 12 seconds it is simply frozen (I presume, this is the period when Kodi reads the matching info from the Database; though this is just mine assumption, since there are absolutely no debug traces in the log file during these 12 seconds). Upon returning to the same large folder (when Kodi works with cached FS), it does not show any “waiting clock”, it simply get frozen for the same 12 seconds (every single time).
It is very easy to get frustrated and click the “return” button one more time (or even couple of times). When Kodi 19+ unfreezes, it starts to process these extra “return” clicks and eventually you find yourself in the home screen.
What I am trying to say, it does not matter what FS is used by samba share (after first read, Kodi works with cached info). Though, library/database read is significantly slower starting from Kodi 19. Kodi 18.x is noticeably faster. I am not sure Kodi is using cache to retrieve info from Database, at least I did not find any settings for that in advancedsettings.xml
my logic behind “slow database read” theory:
on fresh Kodi 20.2 install with empty video library, when I brows to large samba folder it takes 30+ seconds to read the content for the first time. But after getting into this folder few seconds later (after browsing into subfolders), content list populates really fast (less then 2 seconds).
Long delays start after copying (or creating from scratch) the video library.