Tutorial: How to integrate several epg guides in tvheadend

How to integrate several epg guides in tvheadend

The default configuration of tvheadend with respect to the epg guides in CoreELEC is somewhat limited. You can use the Over-the_air EPG grabbers and it will work fine if we have several tuners, no antenna motor, and our satellite provider provides data for one or more days. When this does not happen it is better to periodically download the epg guide every day from a trusted website and integrate it into tvheadend.

CoreELEC includes a single EPG Internal Grabber to download a single EPG guide. If we need to download more guides this solution does not work.

The simplest method to implement several epg guides is to create as many EPG Internal Grabbers as epg guides we need.

How to create an EPG Internal Grabber in tvheadend within CoreELEC?

1.- The file will be named as /storage/.kodi/addons/service.tvheadend42/bin/tv_grab_xxxxxx
where xxxxxx will be the name that identifies the epg guide (example: rytecDE_SportMovies).

2.- The file, as will be a script file, will have 755 execution permissions.

3.- The file will have the following content

1.     #!/bin/sh
2.     XMLTV_TYPE="WEB"
3.     XMLTV_LOCATION_FILE=""
4.     XMLTV_LOCATION_WEB="http://web_link_download_for_epg_guide"
5.     dflag=
6.     vflag=
7.     cflag=
8.     if [ $# -lt 1 ]; then
9.       if [ "$XMLTV_TYPE" = "FILE" ]; then
10.         cat "$XMLTV_LOCATION_FILE"
11.         exit 0
12.       elif [ "$XMLTV_TYPE" = "WEB" ]; then
13.         wget -qO - "$XMLTV_LOCATION_WEB" | xzcat
14.         exit 0
15.       fi
16.     fi
17.     for arg; do
18.         delim=""
19.         case "$arg" in
20.         #translate --gnu-long-options to -g (short options)
21.            --description) args="${args}-d " ;;
22.            --version) args="${args}-v " ;;
23.            --capabilities) args="${args}-c " ;;
24.            #pass through anything else
25.            *) [[ "${arg:0:1}" == "-" ]] || delim = "\""
26.                args="${args}${delim}${arg}${delim} " ;;
27.         esac
28.     done
29.     #Reset the positional parameters to the short options
30.     eval set -- $args
31.     while getopts "dvc" option; do
32.         case $option in
33.             d)  dflag=1;;
34.             v)  vflag=1;;
35.             c)  cflag=1;;
36.             \?) echo "unknown option: -%s\n" $OPTARG
37.                 echo "Usage: %s: [--description] [--version] [--capabilities] \n" $(basename $0)
38.                 exit 2
39.                 ;;
40.         esac >&2
41.     done
42.     if [ "$dflag" ]; then
43.        echo "tv_grab_xxxxxx"
44.     fi
45.     if [ "$vflag" ]; then
46.        echo "0.1\n"
47.     fi
48.     if [ "$cflag" ]; then
49.        echo "baseline\n"
50.     fi

where we will modify the following:

line 4: we will indicate the link of the epg download file
line 13: we will use ‘xzcat’ command for epg files in ‘xz’ format, and ‘zcat’ command for files in ‘gz’ format, and ‘cat’ for uncompressed files
line 43: we will replace ‘tv_grab_xxxxxx’ by the name of our file

4.- Restart CoreELEC.

5.- Activate the new EPG Internal Grabber on tvheadend

1 Like

Just tried this and getting the following error in the logs:

2019-07-06 12:47:51.889 [   INFO] spawn: Executing "/storage/.kodi/addons/service.tvheadend42/bin/tv_grab_sydney"
2019-07-06 12:47:51.891 [  ERROR] spawn: /storage/.kodi/addons/service.tvheadend42/bin/tv_grab_sydney: line 28: syntax error: unexpected newline (expecting ")")

Here is my file:

#!/bin/sh
     XMLTV_TYPE="WEB"
     XMLTV_LOCATION_FILE=""
     XMLTV_LOCATION_WEB="http://au.freepg.tv/xmltv/oztivo/NSW.Sydney.gz?UID=07A6567C&K=AFD8208EC38FA71C327227560D6E442B"
     dflag=
     vflag=
     cflag=
     if [ $# -lt 1 ]; then
       if [ "$XMLTV_TYPE" = "FILE"]; then
         cat "$XMLTV_LOCATION_FILE"
         exit 0
       elif [ "$XMLTV_TYPE" = "WEB"]; then
         wget -qO - "$XMLTV_LOCATION_WEB" | zcat
         exit 0
       fi
     fi
     for arg; do
         delim=""
         case "$arg" in
         #translate --gnu-long-options to -g (short options)
            --description) args="${args} -d " ;;
            --version) args="${args} -v " ;;
            --capabilities) args="${args} -c " ;;
            #pass through anything else
            *) [[ "${arg:0:1}" == "-" ]] || delim = "\""
                args="${args}${delim}${arg}${delim}" ;;
         exac
     done
     #Reset the positional parameters to the short options
     eval set -- $args
     while getopts "dvc" option; do
         case $option in
             d)  dflag=1;;
             v)  vflag=1;;
             c)  cflag=1;;
             \?) echo "unknown option: -%s\n" $OPTARG
                 echo "Usage: %s: [--description] [--version] [--capabilities] \n" $(basename $0)
                 exit 2
                 ;;
         esac >&2
     done
     if [ "$ dflag" ]; then
        echo "tv_grab_sydney"
     fi
     if [ "$ vflag" ]; then
        echo "0.1\n"
     fi
     if [ "$ cflag" ]; then
        echo "baseline\n"
     fi

Replace the exac with esac

Thanks for trying the script. I have corrected my transcription errors because I use the Google translator.

This tv_grab_sidney should work (if I have not made more mistakes)

#!/bin/sh
XMLTV_TYPE="WEB"
XMLTV_LOCATION_FILE=""
XMLTV_LOCATION_WEB="http://au.freepg.tv/xmltv/oztivo/NSW.Sydney.gz?UID=07A6567C&K=AFD8208EC38FA71C327227560D6E442B"
dflag=
vflag=
cflag=
if [ $# -lt 1 ]; then
  if [ "$XMLTV_TYPE" = "FILE" ]; then
    cat "$XMLTV_LOCATION_FILE"
    exit 0
  elif [ "$XMLTV_TYPE" = "WEB" ]; then
    wget -qO - "$XMLTV_LOCATION_WEB" | zcat
    exit 0
  fi
fi
for arg; do
    delim=""
    case "$arg" in
    #translate --gnu-long-options to -g (short options)
       --description) args="${args}-d ";;
       --version) args="${args}-v ";;
       --capabilities) args="${args}-c ";;
       #pass through anything else
       *) [[ "${arg:0:1}" == "-" ]] || delim="\""
           args="${args}${delim}${arg}${delim} ";;
    esac
done
#Reset the positional parameters to the short options
eval set -- $args
while getopts "dvc" option; do
    case $option in
        d)  dflag=1;;
        v)  vflag=1;;
        c)  cflag=1;;
        \?) echo "unknown option: -%s\n" $OPTARG
            echo "Usage: %s: [--description] [--version] [--capabilities] \n" $(basename $0)
            exit 2
            ;;
    esac >&2
done
if [ "$dflag" ]; then
   echo "tv_grab_sidney"
fi
if [ "$vflag" ]; then
   echo "0.1\n"
fi
if [ "$cflag" ]; then
   echo "baseline\n"
fi

Yep, this runs without errors so will see if it works within TvH now.

For 10 minutes I installed it on my tvheadend and saw Sidney’s EPG :grinning:

@cubimol - I live in an area that can get reception from 3 different areas, each with different epg’s, so is there a way to merge all these into one epg?

This is what this topic solves. Create several tv_grab EPG guides !, one for each area (sidney_area1, sidney_area2, sidney_area3, …). The alternative of creating a single tv_grab guide that adds other guides is much more complicated.

I use 6 tv_grab guides to see the EPG of two countries. Obviously, some TV channels are duplicated because some guides overlap. From tvheadend channels you just have to select the origin of the guide that interests us.

Just attempted this and am seeing the following errors

When set to zcat:
zcat: no gzip/bzip2/xz magic
When set to xzcat:
xzcat: corrupted data

Looks like gzip is missing from CE
Any clues ?

Shoog

Find out first what kind of compression the EPG file you downloaded has. If you see the notice of ‘corrupted data’ is that the decompressor has not recognized the compression format. ‘xzcat’ works perfectly with ‘xz’ files, and ‘zcat’ works perfectly with ‘gz’ files.

To find out where xzcat is, use the command: which xzcat, to know where zcat is, use the command: which zcat.

When I added the link to my webbrowser it simply downloaded and opened the file. There is no indication that it is compressed at all.

The webaddress is
http://www.xmltv.co.uk/feed/6743

Shoog

Shoog, it is already uncompressed XML file. I didn’t look what scripts does but just use cat instead of xzcat or whatever.

Ok thats what I thought.
Thats working now.

Shoog

Blockquote
#!/bin/sh
XMLTV_TYPE=“WEB”
XMLTV_LOCATION_FILE=“”
XMLTV_LOCATION_WEB=“Domain Auction | Buy & Sell Distinctive Domains - GoDaddy”
dflag=
vflag=
cflag=
if [ $# -lt 1 ]; then
if [ “$XMLTV_TYPE” = “FILE” ]; then
cat “$XMLTV_LOCATION_FILE”
exit 0
elif [ “$XMLTV_TYPE” = “WEB” ]; then
wget -qO - “$XMLTV_LOCATION_WEB” | cat
exit 0
fi
fi
for arg; do
delim=“”
case “$arg” in
#translate --gnu-long-options to -g (short options)
–description) args=“${args}-d “;;
–version) args=”${args}-v “;;
–capabilities) args=”${args}-c “;;
#pass through anything else
*) [[ “${arg:0:1}” == “-” ]] || delim=”"”
args="${args}${delim}${arg}${delim} ";;
esac
done
#Reset the positional parameters to the short options
eval set – $args
while getopts “dvc” option; do
case $option in
d) dflag=1;;
v) vflag=1;;
c) cflag=1;;
?) echo “unknown option: -%s\n” $OPTARG
echo “Usage: %s: [–description] [–version] [–capabilities] \n” $(basename $0)
exit 2
;;
esac >&2
done
if [ “$dflag” ]; then
echo “tv_grabber_uk”
fi
if [ “$vflag” ]; then
echo “0.1\n”
fi
if [ “$cflag” ]; then
echo “baseline\n”
fi

So this is my exact script. It runs and outputs text to the terminal, but it doesn’t seem to be writing out the text to the file
/storage/.kodi/addons/service.tvheadend42/tv_grabber_uk

Clues ?

UPDATE: I know understand, it doesn’t create an xml file - it pipes the output to TVheadend directly.

Shoog

Tip: As every time that tvheadend42 is updated, all our personal internal grabbers created according to this guide will be deleted, save them in the folder /storage/.config/scripts and add the following line to the file /storage/.config/autostart.sh to solve this:
…
cp /storage/.config/scripts/tv_grab* /storage/.kodi/addons/service.tvheadend42/bin
…

hi cubimol, thank you very much for this tutorial, it works well, many thanks.

thank you @cubimol for this tutorial.
I would like to use a second EPG-Grabber for some IPTV channels I recently added.
I use DVB-S and some IPTV channels in Germany. (EPG for DVB-Channels work fine due OTA-EPG)
Now i need a provider of XML-EPG data for my German IPTV Channels.
Does somebody has an idea where to get this information from?

There are many places to get EPG guides. You can Google “rytec epg germany”, you can also search on github, forums for enigma2 devices, IPTV provider websites, …

1 Like

I don’t understand what are you trying to do exactly. Why don’t you enable external xmltv grabber in TVH settings and load xmltv via socket with one single command?

If you know any better alternative tell us. The solution that I propose allows to easily add multiple internal epg grabbers to the tvheadend server, I, for example, use seven new internal epg grabbers for different countries and TV networks, this cannot be done with the external tvheadend grabber. Also when I wrote this topic the external grabber was very limited and did not always work with all kinds of epg download links, I think now this has been solved but I am not sure.

PS.- When I started thinking about this, my first alternative was to use a single external grabber and load the different epg guides at different times. Finally I found it more efficient and elegant to create as many internal grabbers as epg guides and let the tvheadend server take care of everything.