IceTV Forum

IceTV IceBox => General => Topic started by: TimC on August 31, 2023, 08:20:14 PM

Title: Soft and Hard Resets
Post by: TimC on August 31, 2023, 08:20:14 PM
I had to do a fair bit of chasing around to discover these steps. It would be good if they could be added to the User manual (Withe Pics) or a Useful Tips Topic added to the page added to the forum IceTV IceBox list, where basic setup suggestions can be added.
 
For the nubes a Soft Reset involves:
1. From the main menu, navigating to and selecting the Cog (Settings) icon at the top of the main menu.
2. Select the Librelec settings button (Top Right Button)
3. From the Librelec settings page select System (usually selected by default?)
4. On the system settings pane scroll down to and select "Reset System Settings to defaults"

The box then goes through a series of steps and promps before starting the initial setup actions described in the manual (Login, Connection, (WIFI/Ethernet) and a channel scan. It then checks for any existing recordings and adds them to the Recordings list.

A hard reset involves steps 1 to 3 above and,
4. On the system settings pane scroll down to and select "Reset IceBox to defaults"

The box then goes through the same initial setup steps, as above, but does not re-establish any recordings.

Any preferences you had before either of the resets will have to be re-done.

My question is, what happens to any plugins for Hard and soft resets"

I note the Icebox plugin is retained, which suggests the plugins will remain, but it would be nice to know what would be affected. I fear the next need for a reset, hard or Soft.

Especially given that the SSD changeover will be happening soon.

Are there actions that I can take to retain my setup?
I know I will need to take a copy of the recordings first so they can be transferred to the new SSD.
Is there anything else I can take a copy of to minimise the impact of the SSD changeover?

I see a Backup option in the Libreelec settings. Does anyone know what it backs up and where?
Title: Re: Soft and Hard Resets
Post by: Daniel Hall at IceTV on August 31, 2023, 08:42:33 PM
A soft reset will clear out any downloaded plugins, any user data/settings/plugin data but leaves any of the media directories of the storage partition intact (this is why there is a step in the IceTV setup wizard to 'restore recordings' so they show up under recordings)

A hard reset just formats the storage partition.

With both scenarios the non writable boot partition that includes the read only firmware is untouched, and both will give the same experience on 'first boot' after the reset including the normal setup wizards.

As for the hard drive swap, we will be emailing out all the options that are available closer to the hard drives arriving for us. I can say there will be options for us to just send you a drive preloaded with the software and another that has the IceBox sent back to us for the changeover (with the option of also transferring any current recordings across for you), but these will all be confirmed in that email.
Title: Re: Soft and Hard Resets
Post by: TimC on September 01, 2023, 12:37:54 AM
Quote from: Daniel Hall at IceTV on August 31, 2023, 08:42:33 PMWith both scenarios the non writable boot partition that includes the read only firmware is untouched, and both will give the same experience on 'first boot' after the reset including the normal setup wizards.

OK. That sounds like the Plugin data will be lost with either reset. 

As for the SSD swap, is make sense that everything user defined including recordings will be lost as the SSD is new, i.e., it will effectively be a brand-new IceBox. However, I believe it would be possible to temporarily make a copy of the recordings folder to a USB drive. In my case I can see the IceBox on my Local Area Network.

IceBoxFolders.png

I seem to be able to transfer the files to/from it, so I expect I will be able to copy the recordings data to my PC and transfer it back after the new SSD is installed.
I expect I will need to copy the Picon folder if I want to retain my  My questions remain. 

Is there a way to take a copy of my plugin setup and copy it back after the new SSD is installed?


Is there anything else I can take a copy of to minimise the impact of the SSD changeover?
For example, I think I see the plugin data in the Userdata folder. Is it feasible to take a copy that data and put is back on the new SSD?
Are there other linkages within the software/settings that are created as part of importing the plugins that mean that the plugins have to be reinstalled?
 
Can my plugin data be backed up using the LIBREelec backup function?

Replacing the SSD is a rare enough occurrence that setting everything back up again after replacing it is something I can live with. However resets are likely to be much more common, especially given that there will be a number of updates in the future as some of the kinks get ironed out.

I am looking to avoid having to spend a day fixing up all the settings and plugins every time reset is needed. If that is needed it sort of turns the IceBox into a hobby rather than a serious PVR.

I guess there is an argument that IceTV is not responsible for plugins other than their own and I should be researching this on the LIBREelec and Kodi forums, however, to do that I need to know a lot more about what the IceBox runs, hardware and software. I have actually looked at the forums and I find much of the terminology goes over my head. Mainly because I am not across the concepts of NUCs, LIBREelec or Kodi.

If I had purchased a NUC and loaded LIBREelec and Kodi in it, I would have had a better idea. However, I bought an IceBox as an IceTV capable PVR. My two Skippas had failed, and I was looking to replace them. I had no idea what a NUC was. While I expected that there would be a period of ironing out kinks, I had made an assumption that the IceBox would be a Smart TV capable device. Most PVRs are these days.

With the plugins loaded it is a bit rough around the edges, but close enough with a few pros and cons, so I am not complaining. It does the job and is IceTV capable, which is the most important factor. It also has lots of potential that run of the mill PVRs just don't have.

So far my wife is tolerating it fairly well, but I am not so sure how she will react to losing access to it for a day every time I need to do an update or the IceBox needs a reset.
Title: Re: Soft and Hard Resets
Post by: IanL-S on September 01, 2023, 08:02:01 AM
I have not bothered to load any third-party add-ons, in part due to the need to have a standard configuration for undertaking beta testing. I do have a "spare" BYOB on which I could try but far too many other things to do.

I too am unsure what happens with the Kodi options is to backup settings; not sure which settings are backed-up. To be useful you would need to copy the backup directory to an external storage, then restore it once the IceBox has had its soft/hard reset. You would need to be careful that you only restore the settings for the third-party add-ons.

Not sure if there is an easy way to install third party add-ons (that is those other than IceBox and TVHeadend), which will need to take place before restoring settings. Delta-Mike-Charlie may be able to point you in the right direction.

I suspect that the IceBox installer installs a build of both the IceTV and TVheadend addon, which is updated by the both the hard and soft reset routine to the latest version.

Ian
Title: Re: Soft and Hard Resets
Post by: MD on September 01, 2023, 10:59:59 AM
I've talked this through with Daniel as, in a previous post, we have offered a cloning option if the IceBox is sent back to us which means nothing will be lost in the clone from the old faulty drive to the new one.  However that does mean a probable one week turn around time with the frustrating loss of ongoing IceBox recording in that time.

We may have a cloning solution that users can do in their own home using the IceBox as the PC and all it will need is the purchase of a USB to SATA cable so the new drive can be connected to an IceBox USB port and a USB stick loaded with the Ubuntu OS used to reboot the IceBox.  Then Ubuntu's own cloning software can be used to clone from the IceBox internal SSD to the new one, connected via the cable.  Daniel just needs to do a little research and a test and, if OK, we'll post the procedure in detail. The cloning would take some hours so needs to be done outside "recording time".
Title: Re: Soft and Hard Resets
Post by: TimC on September 01, 2023, 02:25:50 PM
Quote from: MD on September 01, 2023, 10:59:59 AMWe may have a cloning solution that users can do in their own home using the IceBox as the PC and all it will need is the purchase of a USB to SATA cable so the new drive can be connected to an IceBox USB port and a USB stick loaded with the Ubuntu OS used to reboot the IceBox.

That looks interesting, however, I imagine cloning would overwrite any Updates that come with the new SSDs.

As I have said I can live with manually rebuilding my plugins with the new SSD as that is a one-off project.

I am much more interested in methodologies to restore an existing plugin setup for the times when a hard or soft reset is needed.

This is more of a what to capture issue rather than a how at the moment. The how depends on what needs to be transferred.
 
I had a quick play with the backup function that is available in the LIBREelec System settings.
 
When you select the backup option, you are presented with a folder list very similar to the one I showed above, except that:
1. The "Userdata" folder (where a lot of the plugin data is held) is not shown and,
2. An extra folder "LostandFound" is shown.

You then can select a folder and back it up. Where the backup is held is a little interesting. If you back a folder at the, I'll say the "Root" level (I suspect "Root" is not strictly correct though as clicking ".." takes you to a lower level), the backup is stored in the Backup folder. However, if you navigate to, for example the "Picons/tvh" folder, where the channel Icon files are held, and back it up, the backup is stored in that folder.

As well you can only backup at the folder level, i.e., you can't see the files the selected folder contains.

It is also worth noting that the backup file name contains a system generated filename that doesn't help you work out where data from it belongs.

Currently I am unwilling to risk my setup by attempting any sort of restore though.

I have also been trying a little to locate LIBREelec and Kodi manuals that may explain a bit about how all this stuff fits together, but the info I have seen so far in the Wikis and the forums is generally too specific to solving one individual's problem. Sometimes you can glean a gem, but it takes a lot of hunting through a lot of unintelligible posts (I lack the basic info to understand much of what is said).

There is also the issue of the IceBox being what it is. i.e., it is not a standard NUC/LIBREelec/Kodi setup, not that a standard NUC/LIBREelec/Kodi setup actually exists. However, it is a specific IceTV based NUC/LIBREelec/Kodi setup. What sort of things can break this specific setup permanently.  For example, it I install a new LIBREelec update, will that Break IceTV?

That being said though, I suspect questions like that are probably a bit premature. I am trying to start some discussion though.

If someone has some ideas, or places to look at, that progress the discussion, please contribute.
Title: Re: Soft and Hard Resets
Post by: DeltaMikeCharlie on September 01, 2023, 04:01:14 PM
Quote from: IanL-S on September 01, 2023, 08:02:01 AMDelta-Mike-Charlie may be able to point you in the right direction.

I'd go with a backup/restore process for the whole of '/storage'.

The IceBox is just a Linux device and luckily seems to have the 'dd' utility available via busybox.

It seems that '/storage' is mounted from the '/dev/sda2' partition using 'ext4' formatting.

Assuming that the Kodi and tvheadend tasks are not running.

It should be possible to connect a large enough (1TB?) external drive via USB and use 'dd' to create a disk image of the existing partition.  Once the new SSD has been installed, the process can be reversed and the disk image restored to the 'sda2' partition.

If the user is comfortable enough with the command line, ICE could just publish the appropriate commands.  ICE could also consider writing a very small Kodi Python addon that automates the same commands.

The 'tar' command could also be used to achieve similar results without having to create a whole disk image and thus potentially copy empty blocks.  Backup '/storage' to a tar file on an external USB drive and then restore the files.

systemctl stop service.tvheadend42.service
systemctl stop kodi
tar cvf /path/to/usb/mountpoint/icebox.tar /storage/*
or
tar cvfz /path/to/usb/mountpoint/icebox.tar.gz /storage/*

Depending upon the volume of data in question, 'tar' would probably be quicker, however, 'dd' would be more likely to produce and exact copy.  Backing up 33GB with uncompressed 'tar' took me 8 minutes to an external USB3 drive.  The compressed version took over an hour with almost zero compression.  This makes sense as the TS recordings are already heavily compressed.

Both 'dd' and 'tar' would be two-step processes but would require external USB/SATA devices, etc.  This would come at the cost of speed.  Copying from the old SSD to the new SSD would be quicker and not required temporary external storage but would require an external adapter.

The LibreELEC forum may also have some recommendations.

If you really really really really want to backup the Kodi part, look in '/storage/.kodi'.  The actual addon code is stored in the 'addons' subdirectory and the settings are stored in the 'userdata' subdirectory.  Luckily, TVH is treated like a Kodi addon, so all of that data is also there.

You would also need to backup your recordings and channel icons, etc, etc, so it's probably easier to just backup and restore the lot.
Title: Re: Soft and Hard Resets
Post by: TimC on September 02, 2023, 01:33:58 AM
DMC,

Full credit to your superior knowledge. It looks like a solution, unfortunately I have no inkling of what you have said.

Quote from: DeltaMikeCharlie on September 01, 2023, 04:01:14 PMThe IceBox is just a Linux device and luckily seems to have the 'dd' utility available via busybox.

It seems that '/storage' is mounted from the '/dev/sda2' partition using 'ext4' formatting.

What is a 'dd' utility and how do you access it? I assume it is a Linux thing. I have no idea what BusyBox is either.

I understand what a partition is, but what '/storage', the '/dev/sda2' partition and 'ext4' are, is similarly meaning less to me.

The same goes for the 'tar' command.

Please don't take any of this as criticism. It is just me establishing my level of ignorance. I don't know how Guru was put against my profile. It is a long way from an accurate assessment. I know some things, but...

However, is copying data to the level that you have discussed really necessary?

I can't help thinking that there would be a lot of complaints over in the LIBREelec and Kodi worlds, if every time a Soft Reset was done, all the plugins they have installed were lost and had to be reinstalled.

I imagine there is a simpler solution that they generally use.  Unfortunately, most of the forum posts use a similar level of terminology to what you used in your post and as such I simply don't understand what they are saying.

One thing I wonder about is whether the LIBREelec Backup functions could be used at all.  My understanding is that the new SSD will have everything LIBREelec Kodi and IceBox installed and there may also be a more recent update included as well.

I would like to avoid overwriting any updates.

For example, it would be easy enough to create backups of the Backup, Downloads, Recordings and Picons folders. The Userdata folder is not available to the LIBREelec Backup utility, So it would have to be a straight copy. I had planned to do a LAN transfer to my PC which has 1.5 TB of Free Space, but would that transfer the data in a way that retained any updates but was able to restore the Plugins?

I am hoping to get to hashing out a procedure that anyone with the courage to use it, could retain their plugins. Whatever it is it would have to involve first copying the correct data to an external storage and then transferring the data back to the original locations after the drive replacement or a reset.

If I had a bare bones IceBox, I am sure I could backup the Recording folder, transfer the backup to external storage, do the reset/drive replacement and transfer the data back to the recording folder and all would probably be good. A soft reset may be needed though. 

However, as it sounds like a soft reset would lose the plugins, a soft reset would probably be counterproductive.  There must be a simple way of doing this that doesn't involve getting down to the Linux level.

As I said earlier, I can help thinking that is is more about what needs to be copied rather than how.
Title: Re: Soft and Hard Resets
Post by: MD on September 02, 2023, 08:21:20 AM
No updates or add ons will be lost.  A clone is an exact copy.
Title: Re: Soft and Hard Resets
Post by: TimC on September 02, 2023, 09:44:31 AM
Quote from: MD on September 02, 2023, 08:21:20 AMNo updates or add ons will be lost.  A clone is an exact copy.

MD a slight misunderstanding. The updates I was talking about were any updates installed on the new SSD. If there are any. If not, a clone would of course be the way to go.
Title: Re: Soft and Hard Resets
Post by: Daniel Hall at IceTV on September 02, 2023, 10:02:10 AM
There will no software changes required for the new SSD's, the problem was in the hardware, so we just need to get the new hardware in the machine (with as little disruption as possible).
Title: Re: Soft and Hard Resets
Post by: DeltaMikeCharlie on September 02, 2023, 10:34:56 AM
Quote from: TimC on September 02, 2023, 01:33:58 AMI am hoping to get to hashing out a procedure that anyone with the courage to use it, could retain their plugins. Whatever it is it would have to involve first copying the correct data to an external storage and then transferring the data back to the original locations after the drive replacement or a reset.
This has already been provided.  The fact that the LibreELEC forum also shows complicated Linux commands (like mine) suggests that this may not be a common task or that the average user has slightly more technical knowledge.

I'd like to try to paraphrase your concerns:

If IceBox software versions A and B are 'in the wild' and ICE provides everyone with a new SSD that contains software version C, what happens to all of the settings from systems with versions A or B?

I think that you need to know the exact versions of Kodi and all associated addons that are already installed on your IceBox.  You then need to know from ICE exactly which versions of Kodi and associated addons will come pre-loaded on the new SSD.  You can then compare these two lists to identify potential compatibility issues.
Title: Re: Soft and Hard Resets
Post by: MD on September 02, 2023, 12:39:18 PM
DMC, we are going around in circles a bit here.  That is why we will either clone the SSD for you or, as I posted earlier, give precise instructions how to clone it yourselves (albeit you'll need to but a USB - SATA cable). When cloned whatever is on the old SSD, whether it be the OS, plug ins (add ons) or personalised settings, will be replicated on the new SSD.

If you run backup software for a PC or Mac exactly as I described happens if the HDD/SSD fails so one can restore to the new drive as if "nothing" had happened.
Title: Re: Soft and Hard Resets
Post by: TimC on September 02, 2023, 01:15:02 PM
Quote from: MD on September 02, 2023, 12:39:18 PMgive precise instructions how to clone it yourselves (albeit you'll need to buy a USB - SATA cable).

In terms of the SSD change Over, this sounds great to me.

Will this do the job? UMART - Generic USB Type C to SATA III for 2.5in SSD HDD Adapter Cable (https://www.umart.com.au/product/generic-usb-type-c-to-sata-iii-for-2-5in-ssd-hdd-adapter-cable-60462)

However, I am still looking for a solution for a Soft or Hard reset, which was the original intent of this post.

At the moment I an unwilling to risk my configuration, however with the knowledge of having a cloned SSD, I will be able to do some experimentation on the old SSD to see what works. Perhaps that is the most suitable way forward.
Title: Re: Soft and Hard Resets
Post by: MD on September 02, 2023, 01:40:13 PM
Ah no, that's a type "C", you'll need one to match the IceBox which are Type-A, albeit it should be a USB 3 for maximum data output using one of the blue USB sockets on the IceBox.
Title: Re: Soft and Hard Resets
Post by: DeltaMikeCharlie on September 02, 2023, 02:03:44 PM
Quote from: MD on September 02, 2023, 12:39:18 PMDMC, we are going around in circles a bit here.
OK, let's try this:

For simplicity, let's say that the SSD on the IceBox that I received came with 2 pre-configured partition.  The first partition is read-only and contains the 'firmware' (Linux + Kodi + Tvheadend).  The second partition is read-write and contains the '/storage' directory that users will eventually fill with settings, recordings, etc.

Will the replacement SSD be:

1) Totally blank, unpartitioned and unformatted;
2) Pre-installed with the default 2 partitions as described above?

If 1) above, then the whole SSD (both partitions) will need to be cloned by either ICE or the user.
If 2) above, then only the read-write partition will need to be cloned OR backed-up and restored by ICE or the user.  Also, which version of the IceBox firmware will be pre-installed?
Title: Re: Soft and Hard Resets
Post by: IanL-S on September 02, 2023, 03:24:33 PM
Well spotted MD. I have two USB 3 (A) USB SATA adapters. One (a UGREEN) supports external power connector (useful for 3.5" SATA drives that can be problematic when relying on USB port for power) - works well with 2.5" SSDs and HDDs as they need less power.
Title: Re: Soft and Hard Resets
Post by: TimC on September 02, 2023, 06:34:04 PM
Of Course. Too Many USB types these days.Rotanium USB3.0 to SATA Converter Adapter (https://www.umart.com.au/product/rotanium-usb3-0-to-sata-converter-adapter-66520)

This better?
Title: Re: Soft and Hard Resets
Post by: TimC on September 02, 2023, 06:38:22 PM
This is an Amazon version as well. SZJUNXIAO SATA to USB 3.0 Adapter Cable Support SATA III UASP, USB to SATA 22 Pin Converter (https://www.amazon.com.au/SZJUNXIAO-SATA-USB-3-0-Adapter/dp/B0BVKM9QPT/ref=sr_1_2_sspa?crid=4IFEHV1AK88F&keywords=USB+A+to+SATA&qid=1693643682&sprefix=usb+a+to+sata%2Caps%2C302&sr=8-2-spons&sp_csd=d2lkZ2V0TmFtZT1zcF9hdGY&psc=1)
Title: Re: Soft and Hard Resets
Post by: TimC on September 03, 2023, 02:17:15 PM
All,

I have added some key parts of the discussion above to the Replacement SSDs thread (https://forum.icetv.com.au/index.php?topic=7683.0) as the discussion about the SSD s is more relevant there.

I would like to return this thread to its original intent, which was to identify an approach to restore a Plugin setup following a Hard or Soft reset without having to rebuild it all from scratch.