IceTV Forum

IceTV IceBox => General => Topic started by: DeltaMikeCharlie on June 24, 2023, 11:53:15 AM

Title: Potential EPG Data Loss due to Power Failure
Post by: DeltaMikeCharlie on June 24, 2023, 11:53:15 AM
I have been investigating how tvheadend (TVH) stores its EPG data and from what I have found, TVH holds the EPG data in memory and only writes it to disk during a clean exit.  I need to do some more testing, however, I suspect that this means that any new EPG data received between system startup and a power failure will be lost.

For FTA EIT, this would not represent a huge problem because the EPG grabber task will run on startup and the EPG will be replenished.

For ICE, however, I believe that a power failure will result in a hole in the EPG data.  This hole will span all of the EPG data received between the last clean shutdown and the restart after the power failure.

As the ICE addon receives new EPG data, that data is processed, sent to TVH and then ICE saves its 'last update' time (I have confirmed that this is actually written to disk every minute).  For the next EPG cycle, ICE requests EPG changes since the 'last update', applies only those changes and the process repeats.

However, if a power failure has occurred, TVH will not have written any newly received EPG data to its database file.  ICE will not be aware of this and continue to request EPG data since its 'last update' received, not the last update retained by TVH.

I'd like to propose 2 possible resolutions:

1) ICE sets a flag in its local settings file on startup and then resets that flag when shutdown cleanly.  If the ICE addon starts and finds the flag set, there is a good chance that there was a power failure and it reloads the entire EPG into TVH.

2) ICE keeps track of the signature (eventid/channel/start/stop) of the last EPG entry successfully sent to TVH.  ICE also keeps track of the 'last update' value that the previous session started with.  On startup, ICE looks for the EPG entry signature and if it is not found, reloads the EPG from the 'last update' of the previous clean session.

Option 1 is probably easier to implement, however, it has a larger network impact for frequent power failures.  Option 2 has slightly more moving parts, but only re-requests the potentially missing data, not the whole EPG.

The one last thing that I have to investigate is whether the Kodi DVR function caches EPG information to disk, or if it obtains a fresh EPG data set via HTSP on startup.

If I have misinterpreted anything, I'm happy to be corrected.
Title: Re: Potential EPG Data Loss due to Power Failure
Post by: IanL-S on June 24, 2023, 12:17:02 PM
The IceTV add-on configuration includes option to redownload full EPG, so your option one could hook onto that code to repopulate the EPG.

Uncertain how much the data load would be in that case; depends on the number of active IceBox and how widespread the power outage happens to be. With the current update cycle being only 1 minute, it may be possible to cache requests to balance the load.
Title: Re: Potential EPG Data Loss due to Power Failure
Post by: DeltaMikeCharlie on July 03, 2023, 06:07:39 AM
It looks like TVH already provides a solution:

TVH EPG Save Setting.png

I have not changed these settings yet.  It appears that by default, the EPG is saved hourly.  As ICETV uses XMLTV, I'm considering enabling the option to save after an import as well.

I suspect that there will still be a short period of risk, however, it would probably be on the order of seconds to minutes rather than hours.