Graphical front panel display

noggin · 20 October 2020 10:13

OK - have bought an identical TFT 320x240 display from Amazon UK to that used by @mblovell and also a 128x128 colour OLED 1.5" which looks like it should be supported by luma.oled which might be fun to try (if a bit small at 1.5" and a bit low-res. Wish there were 256x256 or 320x320 OLEDs…). Might also see if I can get graphics to a conventional DPI/HDMI display as I have loads of them (including a spare 720x720 Hyperpixel )

My first attempt will be running a Pi 3A+ with WiFi and the TFT connected to see if I can get that to talk to another box (probably an N2 or Shield TV) running Kodi.

Will report back!

(My aim is to get PVR integration for current channel name and number, current channel logo, current show name, duration and elapsed time etc. as well as movie / TV show art for playing Movies and TV Shows)

noggin · 20 October 2020 14:27

OK - with a very few tweaks I’ve got this running on with a Raspberry Pi 3A+ and a local Kodi install and the spi-connected ssd1351 128x128 1.5" OLED (helpfully it uses the same pinout as the TFT - and tested it with luma.oled examples to check the Pi was talking to it OK).

In kodi_panel I peplaced luma.lcd with luma.oled, commented out references to backlight and changed the display dimensions and the text heights to something more 128x128 friendly!). My ssd1351 likes bgr rather than rgb so needed to enable that in the display class with bgr=True

Currently just 128 row height artwork thumbnails displayed as the text positions and duration etc. all seem to be hardcoded for a 320x240 display.

device = ssd1351(serial,  width=128, height=128,
                 bgr=True
                 )

I wonder if it’s worth making the text positions relative rather than absolute values, or having defined field positions as variables rather than specific numbers so you only have to edit them in a header rather than in-line?

mblovell · 20 October 2020 15:31

That’s great, @noggin!!

Yes, I note in a comment block ahead of update_display() that locations and font sizes are all hard-coded. The hard-coding has really since moved into the new audio_screens() function, but it’s all still there.

I have some configuration as global variables at the start of the script. Making use of a separate configuration file could be done, but dealing with the varying amount of information that different screen sizes can accommodate might be challenging. Scaling from 320x240 down to 128x128 or even 128x64, one might not want to try squeezing everything in!

Some of the other display projects listed in the summary post (far up on this thread) go so far as to support different layouts via XML files. I don’t know (at this point), if going that far is merited. If people have ideas on how to support some configurability without getting too complex, we could pursue that.

Lacking that, you at least now have examples of the JSON-RPC calls and what information you get back.

Thoughts?

mblovell · 20 October 2020 15:39

In earlier posts, using pygame for rendering was mentioned. Programs like WeatherPi_TFT use pygame for image rendering and some basic animation, using a framebuffer as the mechanism for getting that content to the attached LCD. For Raspberry Pi, that set up involves Device Tree Overlays (dtoverlay). Not knowing much about that, I assembled the collection of info below.

Raspberry Pi

This page seems to cover all the fundamentals, at least for Raspberry Pi:

https://www.raspberrypi.org/documentation/configuration/device-tree.md

Now, being familiar with chipsets for Itanium and x86 computers, I’m more familiar with ACPI for hardware discovery. Device Trees for Linux on ARM can be contrasted against ACPI tables on x86. This page has info on the fairly recent introduction of ACPI for ARM-based servers, for the curious:

https://www.kernel.org/doc/html/latest/arm64/arm-acpi.html

fbtft

Frame buffer support for small LCDs under Linux was originally developed via FBTFT – a project since merged into the Linux kernel. Here’s the now-old github page for that project and where it currently resides:

https://github.com/notro/fbtft/
https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/fbtft?h=staging-testing

Wiki:
https://github.com/notro/fbtft/wiki

The wiki appears to state that fbtft is currently unmaintained? UPDATE: See Post #80 below

Usage examples:

Two nice blog posts that describe the steps, as a user, for getting a framebuffer-based display up and running:

https://avikdas.com/2018/12/31/setting-up-lcd-screen-on-raspberry-pi.html
https://avikdas.com/2019/01/23/writing-gui-applications-on-raspberry-pi-without-x.html

The Adafruit 480x320 HX8357-based display is supported via this framebuffer approach:

https://www.adafruit.com/product/2441
https://learn.adafruit.com/adafruit-pitft-3-dot-5-touch-screen-for-raspberry-pi

That particular display has a DPI (8-bit parallel) option for use, similar to the HyperPixel displays. Seems like one could get quite a fast display going!

Odroid

Odroid wiki entry on Device Tree Overlay:

https://wiki.odroid.com/common/application_note/software/device_tree_overlay

and on using a framebuffer with the N2 for a 3.2" LCD shield:

https://wiki.odroid.com/accessory/display/3.2inch_tft_touchscreen_shield/n2/start

The main challenge, I think, for running in CoreELEC would be getting the necessary kernel module for framebuffer support compiled.

noggin · 20 October 2020 15:47

I looked at pygame solutions - and on some platforms (like OSMC, Core/LibreElec etc.) they seemed a bit involved to install.

As for hardcoding positions vs configuration files etc. I wonder if a middle ground might be defining the various positions, heights etc. as global variables in a single location in the code - alongside the image size and text height and fonts?

That way you’d be able to edit them all simply - rather than ploughing through code?

I saw the XML approach used by others - and whilst it’s neat - it’s a lot of work.

mblovell · 20 October 2020 15:51

I can look into that change. I’ll likely end up stuffing the different screen layouts into a dictionary. That way, at least, it just counts as one additional global.

mblovell · 20 October 2020 15:58

Larger, decent resolution and full-color OLEDs would indeed be really nice to have. I wonder if they’re just too expensive (or if the manufacturers just prefer to focus on making big panels for televisions).

mblovell · 20 October 2020 23:05

Here’s an initial stab at using a dictionary for layout details:

# Screen layout details
LAYOUT = \
{ ADisplay.DEFAULT : 
  {
    # Artwork position and size
    "thumb"   : { "pos": (5, 5), "size": thumb_height },

    # Progress bar.  Two versions are possible, short and long,
    # depending upon the MusicPlayer.Time string.
    "prog"  : { "pos": (150, 7),
                "short_len": 104,  "long_len": 164,
                "height": 8 },

    # All other text fields, including any labels
    "fields" : 
    [
        { "name": "MusicPlayer.Time",          "pos": (148, 20), "font": font7S, "fill":color7S },
        
        { "name":  "MusicPlayer.TrackNumber",  "pos": (148, 73),  "font": font7S,     "fill": color7S,
          "label": "Track",                   "lpos": (148, 60), "lfont": font_tiny, "lfill": "white" },
        
        { "name": "MusicPlayer.Duration", "pos": (230, 60), "font": font_tiny, "fill": "white" },
        { "name": "codec",                "pos": (230, 74), "font": font_tiny, "fill": "white" },
        { "name": "MusicPlayer.Genre",    "pos": (230, 88), "font": font_tiny, "fill": "white", "trunc":1 },
        { "name": "MusicPlayer.Year",     "pos": (230,102), "font": font_tiny, "fill": "white" },
        
        { "name": "MusicPlayer.Title",    "pos": (5, 152),  "font": font_main, "fill": "white", "trunc":1},
        { "name": "MusicPlayer.Album",    "pos": (5, 180),  "font": font_sm,   "fill": "white", "trunc":1 },
        { "name": "MusicPlayer.Artist",   "pos": (5, 205),  "font": font_sm,   "fill": color_artist, "trunc":1 },          
    ]
  }
}

The corresponding code in audio_screens() isn’t entirely generic, since it has special treatment for the codec field (another dictionary lookup) and Artist field (substitution with Composer possible). Nevertheless, it is now mostly a for-loop over the “fields” array above.

I need to deploy this on the C4 and see how it does for speed.

mblovell · 21 October 2020 02:49

@noggin, take a look at v0.75, now up on github.

For pure layout changes, you should be able to just update the AUDIO_LAYOUT data structure at the start of the script. Per the comments, text fields can also be removed by commenting out the relevant array entry within that data structure. (Adding a completely new field, or adding special treatment for a field, still requires coding Python.)

Regards,
Matt

mblovell · 21 October 2020 04:14

Interestingly, there is a noticeable difference in kodi_panel’s update_display() time seemingly dependent upon what is being played.

mp3 playback:

2020-10-20 22:07:49.763438 Elapsed time is  0:00:00.014268
2020-10-20 22:07:50.678853 Elapsed time is  0:00:00.014201
2020-10-20 22:07:51.593449 Elapsed time is  0:00:00.013398
2020-10-20 22:07:52.509022 Elapsed time is  0:00:00.014374
2020-10-20 22:07:53.423423 Elapsed time is  0:00:00.013982

flac playback (just 2-channel, 16 bit, 44.1 kHz)

2020-10-20 22:07:12.490034 Elapsed time is  0:00:00.377940
2020-10-20 22:07:13.758048 Elapsed time is  0:00:00.366812
2020-10-20 22:07:15.041331 Elapsed time is  0:00:00.382096
2020-10-20 22:07:16.305848 Elapsed time is  0:00:00.363306
2020-10-20 22:07:17.573553 Elapsed time is  0:00:00.366486
2020-10-20 22:07:18.827768 Elapsed time is  0:00:00.353025

I’ve verified that get_image() is not retrieving and resizing artwork multiple times. That occurs once whenever the image path gets changed.

I’ve also verified that it’s not the Player.GetProperties JSON-RPC request in which kodi_panel is requesting a percentage-done for the progress bar. The time doesn’t really change with that code entirely commented out.

That must mean that the XMBC.GetInfoLabels call is taking longer when a flac file is playing?

mblovell · 21 October 2020 04:33

That must mean that the XMBC.GetInfoLabels call is taking longer when a flac file is playing?

Nope, it wasn’t that!

With a ridiculously long Artist name, the process of calling truncate_text() is responsible for that time difference. I’ll have to come up with something a bit more efficient than what’s currently there!

noggin · 21 October 2020 11:05

I think the tech used for small OLED displays, phone OLED screens and TVs is all a bit different. (I think in particular the small OLED screens use something different). It may be P OLED vs AM OLED?

noggin · 21 October 2020 11:06

FLAC is a much higher bitrate than MP3 - and may require more processing as a result - I wonder if that - or an inherently different block size/duration plays a part?

mblovell · 21 October 2020 13:54

No, it wasn’t that. It was my own silly (and evidently fairly slow) truncate_text() function. That, combined with a FLAC album purchased online where someone thought a 108-character “Artist” (listing the soloist twice, the orchestra twice, and the composer) would be useful.

I need to edit that album’s tags, but I can also improve truncate_text().

The reason I noticed the issue is the elapsed time display. I try to have the loop sleep time such that the script only wakes up and polls once a second. With text truncation causing the update to take so much longer (over 25x worse!), the time update was occasionally missing a beat, so to speak. After observing that hiccup I instrumented various portions of the code and started trying to track down what was consuming the time.

mblovell · 21 October 2020 15:05

I need to edit that album’s tags, but I can also improve truncate_text().

Fixed (hopefully well enough) in v0.78.

mblovell · 21 October 2020 15:15

For anyone using a framebuffer-driven display, this page seems promising for adapting PIL / Pillow!

Python Framebuffer Imageviewer
https://raspi.muth.org/framebuffer.html

The example from that page, in the “Displaying Content with PIL and Pytorinox” section:

from PIL import Image, ImageDraw
from framebuffer import Framebuffer  # pytorinox

fb = Framebuffer("/dev/fb1")
buffer = Image.new(mode="RGB", size=fb.size)
draw = ImageDraw.Draw(buffer)
cx = fb.size[0] // 2
cy = fb.size[1] // 2
draw.rectangle((cx - 10, cy -10, cx + 10,  cy + 10), "white") 
fb.show(buffer)

This perhaps provides an alternative to using pygame.

Pytorinox on github.

mblovell · 22 October 2020 15:33

v0.8 of kodi_panel (committed on github but not yet released) adds two new booleans, USE_BACKLIGHT and USE_PWM.

The USE_BACKLIGHT variable is really for convenience, so that OLED users don’t have to look through the code for all the backlight() calls.

The USE_PWM and accompanying PWM_* variables are an experiment. Unfortunately. it seems that RPi.GPIO-Odroid continues to use software-controlled PWM. As a consequence, some flickering is possible. I have not yet figured out if there’s a nice way to make use of the Odroid C4’s hardware-based PWM (while still leaving luma.lcd able to turn the backlight on and off).

noggin · 26 October 2020 13:12

Just installed your GitHub repo on a WiFi Pi3A+ and a 320x240 TFT (identical to the one in your picture I think). Works really nicely! I decided to go with your TFT solution rather than an OLED as the OLED 128x128 res is probably just a bit too restrictive.

Time to try and learn some Python and JSON-RPC to implement other displays…

Thanks for the hard work you have clearly put into this.

mblovell · 26 October 2020 14:29

@noggin, thanks. Glad it’s working for you!

Don’t forget to checkout out the AUDIO_LAYOUT dictionary:

https://github.com/mattblovell/kodi_panel/blob/8d7d80fa626331c637ac7701a706bd9d3ffd9999/kodi_panel.py#L131-L169

Per one of your earlier suggestions, it should permit for layout, font, and color changes (as well as omitting text fields) with not much programming. (Modifying data structures with some understanding does, in my book, count as programming. )

Augmenting the existing ADisplay enumerated type (well, class since it’s Python), one can also create entirely new layouts (e.g., a screen showing elapsed time only in some large font). This basic structure – AUDIO_LAYOUT defining what to display and where for audio_screens() – is what I plan on replicating for making movie/video now-playing screens.

mblovell · 31 October 2020 00:27

Now that I have something working, naturally I want to see if I can get it working via some other means! Rather than using luma.lcd to drive the SPI display, I thought I would see if I could use it as a framebuffer device on the RPi. That would then open the door to using DPI-based displays.

Unfortunately, all of the earlier info I found regarding fbtft is somewhat stale, since fbtft_device (a supporting element) was removed from the Linux kernel starting with 5.4. A lot of the information one can find googling around is now outdated. The Linux maintainers file shows fbtft as currently orphaned.

The wiki for fbtft now notes the following:

fbtft will now only work with Device Tree due to the above mentioned disruptive gpio rework.

Not a lot of further information is provided (at least not there).

One part of how things seem like they might work now is tinyDRM:

tinydrm was created because fbtft couldn’t move out of staging due to fbdev being closed to new drivers. This means that the fbtft drivers have to be converted to DRM.

Does anyone know what DRM stands for in this context? I keep thinking Digital Rights Management, but that doesn’t seem right!

One can find a few Device Tree Overlay files nominally for the RPi in tinyDRM’s github:

https://github.com/notro/tinydrm/tree/master/rpi-overlays

Prior to today, I didn’t know how overlays got associated with specific device drivers. That association turns out to be entirely guided by the compatible (string) property that gets specified in the tree. (A nice presentation from Thomas Petazzoni explains matters fairly well.)

The "mi,mi0283qt" that gets referenced in tinyDRM’s rpi-display-overlay.dts looks like it ultimately references this device driver:

drivers/gpu/drm/tiny/mi0283qt.c

The latest Raspberry Pi OS has its own dtoverlay files as well, and its rpi-displays.dt[os] file refers to a different driver:

drivers/gpu/drm/tiny/ili9341.c

In spite of the different names, both drivers are meant to communicate with an ILI9341 controller.
The two files seem pretty similar, with just some very minor differences in the initialization code and some reset differences.

Unfortunately, I was not able to get the native RPi rpi-displays.dto to do much of anything last night. The overlay would load, and a /dev/fb1 file would be created. The display itself was not actually responsive though. (The backlight stayed off, and reconnecting the LED signal to 3.3V just showed an all-white screen, so it had never been properly initialized.)

So, I’ll play some more this weekend and see if I can make any headway. The framebuffer approach definitely seems a bit more complicated.

Cheers,
Matt