IceTV Forum

IceTV Guide for IceTV enabled PVRs => Topfield => Topic started by: IanL-S on February 22, 2012, 08:00:34 AM

Title: IceTV server is down?
Post by: IanL-S on February 22, 2012, 08:00:34 AM
Is it just me, or is the server which the PC Gadget and the Toppy gets its EPG (and timer) data down.  (7.45 am 22-02-12). I only notice the issue when I found the Toppy frozen, and had to do a hard reboot and no EPG data displayed. The I booted up the computer and the Gadget failed to load any data.

Ian
Title: Re: IceTV server is down?
Post by: cheechaway on February 22, 2012, 11:25:21 AM
Having the same issue. Last checked on Sunday and all was well. Have had "Error - Cannot connect to server" since yesterday, cannot use any EPG functions except to Exit that section, and definitely have a solid internet connection. The app on the ipad seems to have no trouble seeing the guide.
Title: Re: IceTV server is down?
Post by: grampus on February 22, 2012, 12:22:03 PM
Able to get to the Ice server via PC, IPAD, BWs, Windows MCE.
At this time
Title: Re: IceTV server is down?
Post by: wally3218 on February 22, 2012, 01:50:07 PM
Server is still down my 2460 cant connect to the server.
Web access and FTP to the 2460 works ok am using a wired lan.
Title: Re: IceTV server is down?
Post by: lukem on February 22, 2012, 01:53:18 PM
Hi Guys,

We migrated our servers to Amazon EC2 last night. While most of the cut-over went well there were some feeds which were not being updated. This should be resolved now, please let us know if there are any further issues.

Kind Regards,
Luke
Title: Re: IceTV server is down?
Post by: wally3218 on February 22, 2012, 02:02:49 PM
Just tried Reset EPG data on the 2460 and still getting the "Cannot connect to server " message
Title: Re: IceTV server is down?
Post by: IanL-S on February 22, 2012, 02:04:11 PM
IceTV PC widget still cannot contact the server.

Ian
Title: Re: IceTV server is down?
Post by: j s on February 22, 2012, 03:13:42 PM
ditto for my TF-2460 - last epg retrieval was at 8:25 this morning and cannot connect now
Title: Re: IceTV server is down?
Post by: wally3218 on February 22, 2012, 03:18:18 PM
SERVERS ARE STILL DOWN

It would be good of IceTV to advise there customers that the servers are down. A note on the TV Guide page saying there are problems.
Letting customers manually set the timers so as not to miss any recordings that they have setup today.
Title: Re: IceTV server is down?
Post by: Daniel Hall at IceTV on February 22, 2012, 03:22:37 PM
I can confirm that the servers are up and running now, and have been since around 2am this morning.

We are still seeing some issues with some Topfield customers and we are investigating these issues, I can confirm that its not all Topfield customers as we are definitely seeing requests coming through and our test machines are updating without issues from a normal ADSL connection that we use to test on.
Title: Re: IceTV server is down?
Post by: Fester on February 22, 2012, 03:31:05 PM
My account has: Last EPG fetch: 22 February, 2012 at 04:27am with Topfield TRF-2400

Rebooted a few times but no update!
Title: Re: IceTV server is down?
Post by: wally3218 on February 22, 2012, 03:42:39 PM
I have reset my network settings on my 2460 and can play Youtube videos on my 2460
Have checked user name and password for IceTV again.
But i cannot connect to the IceTV server so my conclusion is that Cannot connect to Server  Server is not working.
Theres not much else that I can do or check.

Last EPG fetches:
22 February, 2012 at 12:12pm with Topfield TRF-2400 ( note that my device type is a 2460 dont know why it shows up as a 2400)
Title: Re: IceTV server is down?
Post by: IanL-S on February 22, 2012, 03:53:03 PM
The IceTV Interactive site says that the last EPG fetch was at 03:29pm; one 2400 is refusing to talk to the IceTV server. I get cannot connect to server message. Not sure which, if any, of the other 3 Toppys is talking to the Server. It is also claimed that the widget updated at 2.02pm, but nothing is showing on the widget. I have stopped and started the widget several times.

I can access the Toppy by ftp and can access YouTube from the Toppy, so it would appear that the problem is not at my end.

Could it be ISP related; I use BigPond Cable and I'm located in Melbourne.

I will try setting timers for the other 3 Toppys and see if they get downloaded; will also set on for the Toppy that cannot connect to the server.

Ian
Title: Re: IceTV server is down?
Post by: Daniel Hall at IceTV on February 22, 2012, 04:00:01 PM
After more testing now, it looks like the Topfield TRF-2400, TRF-2460 and TF7100HDPlus are having issues downloading the device list from the server, we are still investigating this issue to try and find out why this is the case.

It also looks like on these specific models are affected, the older TF7100HDPVRt is not affected and works fine, along with all other PVR's.

We will update as soon as we have additional info.
Title: Re: IceTV server is down?
Post by: wally3218 on February 22, 2012, 04:06:22 PM
Thanks for the confirmation it wasnt us Daniel , Hopefully we can get a warning when IceTV is upgraded next time
Title: Re: IceTV server is down?
Post by: IanL-S on February 22, 2012, 04:16:09 PM
Both my 7100+ have not IceTV EPG and do not seem to be downloading EPG data. They were both put into standby briefly this morning to avoid EPG and timer stalling. One 2400 froze overnight and was restarted this morning; that was when I discovered the problem with no EPG data. The other 2400 running January 24 beta was not put into standby and still has IceTV EPG data, but it is unclear if it is successfully communicating with the IceTV Server.

None of this explains why the IceTV Widget is not working on my Win7 64 system. It has not worked at all today ... one indicator that something was wrong with IceTV.

Ian
Title: Re: IceTV server is down?
Post by: Markh on February 22, 2012, 05:33:55 PM
Why is this taking so long to fix?M??K?K?K?! My 2400 has NO guide data at present. Like a paper weight!!! Grrrr pissed off with icetv
Title: Re: IceTV server is down?
Post by: IanL-S on February 22, 2012, 05:49:44 PM
Definitely not good. Not sure how long IceTV have been aware of the issue and how serious it is. I started this thread at 7.00 am according to the first post (I suspect it was actually 8.00 am as the forum seems to be a daylight saving denier. These things can be difficult to pin down ... and to get a fix. I also logged a service issue at about the same time ... I have not had a response as yet (not even an automated one). So, IceTV for Toppys may not be the only victim of the server migration.

I suppose we can always revert to free to air EPG data .. I have done this with one of my Toppys.

Ian
Title: Re: IceTV server is down?
Post by: wally3218 on February 22, 2012, 06:28:22 PM
YES Definately not happy with IceTV at the moment.
We had to convince IceTv that they had a problem after having them tell us everything was ok with the server.
Now they seem to go missing just like the EPG

Title: Re: IceTV server is down?
Post by: Daniel Hall at IceTV on February 22, 2012, 06:42:22 PM
The unfortunate part of all this is that there is something wrong in the way the Topfield handles getting the device list from the IceTV server, and whatever is causing the issue reading the list is not affecting any other PVR that we support. Also when I say Topfield here I specifically mean the newer TMS models (these being the TRF-2400, TRF-2460 and TF7100HDPlus), the older models are working fine.

We put in place a workaround earlier to try and get the affected customers back up and running ASAP but sadly it required a change to DNS records and these can take a while to replicate out. If you are still not seeing the guide we recommend power cycling the Topfield to ensure nothing is being cached and let it try to download the guide again. If however the guide is showing up you should not turn off the Topfield or put it in standby as it will attempt to update its device list at startup and stop updating, while it stays on though it will be fine (as we have confirmed that only the initial device list update is affected).

And I don't know about going us going missing, I would rather work at fixing the problem than have to keep putting out updates.
Title: Re: IceTV server is down?
Post by: AndrewT on February 22, 2012, 06:47:03 PM
Re-starting my TF7100HD plus has not done anything  at all - any idea of an eta when this will be sorted?
Title: Re: IceTV server is down?
Post by: Daniel Hall at IceTV on February 22, 2012, 06:50:20 PM
Quote from: AndrewT on February 22, 2012, 06:47:03 PM
Re-starting my TF7100HD plus has not done anything  at all - any idea of an eta when this will be sorted?

Unfortunately there is no way to predict how long it will take DNS to propogate out, however we are still working a better workaround to this (unfortunately a proper fix would require a firmware update from Topfield, and we don't want any delays to occur with the release of the next firwmare).
Title: Re: IceTV server is down?
Post by: wally3218 on February 22, 2012, 06:55:57 PM
IceTV have known that the topfield 2460 models have had problems with the firmware for ages
Now we find that there is another topfield firmware/ IceTV server problems.
Why are these topfield models recommended by IcerTV when you know theres problems with these units.
For the novice first time users of IceTv and PVR's it must be hell for them to understand.
Title: Re: IceTV server is down?
Post by: IanL-S on February 22, 2012, 07:12:18 PM
Quote from: Daniel Hall at IceTV on February 22, 2012, 06:50:20 PM
Quote from: AndrewT on February 22, 2012, 06:47:03 PM
Re-starting my TF7100HD plus has not done anything  at all - any idea of an eta when this will be sorted?

Unfortunately there is no way to predict how long it will take DNS to propogate out, however we are still working a better workaround to this (unfortunately a proper fix would require a firmware update from Topfield, and we don't want any delays to occur with the release of the next firwmare).

Managed to get 2 x 7100+ back on line. However, a 2400 running January beta firmware is still unable to get device. Is it possible that there is a difference in the way the beta firmware inteacts that means the work around does not fix the issue? If I get a chance I may try reinstalling the firmware and see if it is still problematic.

Ian
Title: Re: IceTV server is down?
Post by: MB4913 on February 22, 2012, 07:15:00 PM
Still "cannot connect to server" with TRF 2400. Hope it is fixed soon with an announcement on website.

By the way IanL-S, you need to go to Profile -> Look & Layout, and adjust the Time offset. Obscure, I know, but there it is.  :D
Title: Re: IceTV server is down?
Post by: csutak40 on February 22, 2012, 07:35:41 PM
I only just rebooted my PC  and the Yahoo  Widgets keeps crashing.  I am presuming it has something to do with this?
Title: Re: IceTV server is down?
Post by: MB4913 on February 22, 2012, 07:38:23 PM
Ok, it's just come good.  :) EPG fetch and full guide a few minutes ago.

Thankyou, Daniel.
Title: Re: IceTV server is down?
Post by: wally3218 on February 22, 2012, 07:53:07 PM
All back to normal here as well hope thats the last of the problems.
Title: Re: IceTV server is down?
Post by: Daniel Hall at IceTV on February 22, 2012, 07:54:04 PM
Excellent news, we will of course be monitoring the situation and responding if anything else arises.
Title: Re: IceTV server is down?
Post by: AndrewT on February 22, 2012, 09:04:35 PM
All working for me and my TF7100 plus - had an issue with my timers when the epg and timers were restored but simply deleted them and re- sent - all is good again.
Title: Re: IceTV server is down?
Post by: IanL-S on February 23, 2012, 07:34:37 AM
My recalcitrant 2400 eventually was able to log in an I could then select a device. Not 100% clear why it was more reluctant. Possibly due to switching to FTA EPG data with this unit - did not do it for the other Toppys.

Well, under 12 hours to a solution is pretty good given the complexities of configuring servers.

I notice the Ice TV Widget is still missing in action. Any idea when it will be fixed?

Ian
Title: Re: IceTV server is down?
Post by: mikeathome on February 23, 2012, 11:45:24 AM
Now that most of the dust appears to have settled I would like to make a comment.

In a post a page or so back Daniel from ICE explains that the failure discussed in this thread is due to an error in the way theTopfield TMS units operate. Thus suggesting that the cause of the problem is with Topfield (at least that is how I have interpreted his post).

Now I cannot be counted as a Topfield apologist - on the contrary I think that a business model that incorporates the release of untested firmware and a reliance on users to identify problems is both unethical and unprofessional. However in this case it would appear that the ICE people changed the way that their system operates, did not test before implementation, and as a result caused a situation where all TMS Topfields failed to operate with ICE in the way that they had been doing for 2 + years previously. The underlying root cause may well be a difference in the way that the TMS models operate compared with other units supported by ICE. But this should have been picked up by ICE when the new server configuration was tested, and remedied before implementation. It is not Topfield's fault that someone at ICE didn't do their job professionally.

Hopefully those at ICE can learn from this experience and ensure that any change - particularly those regarded as minor - be thoroughly and professionally tested before being implemented.
Title: Re: IceTV server is down?
Post by: IanL-S on February 23, 2012, 03:38:03 PM
It is possible to subject a Toppy firmware to rigorous testing before it is released. Some firmware issues take time to emerge even under rigorous testing. There is a trade-off between the need to put in place fixes to know issues and the risk of new issues going undetected. In some case it may be decided it release a firmware with minor issues uncorrected, given the time needed to address those issues.

It is considerably more difficult to test a server configuration when moving from one server to another. If the new and old server have identical configuration, then there should be no problems. If for some reason it is necessary to tweak the configuration (for example when a different provider is being used) the problems in testing before going live can be significant, particularly when testing interaction with remote connections.

The sever outage was somewhat annoying, but it was fixed swiftly. Toppy firmware issues are far more difficult to solve and the time frame is months rather than hours.

Ian

Title: Re: IceTV server is down?
Post by: mikeathome on February 23, 2012, 05:57:11 PM
What's so difficult about testing whether the PVRs that you support can communicate with your server? We're not talking about some obscure bug here that might reasonably have slipped through the net but a major failure that totally prevented any communication. Last time I spoke to the ICE support people (late last year) they had a TMS model available and in use for testing purposes so this should have made it even easier. The fact that testing may (or may not) be difficult is hardly an excuse for not doing it, simple as that. And IT professionals are well aware of that.
Title: Re: IceTV server is down?
Post by: Markh on February 29, 2012, 05:59:19 PM
Down again!?!?!?!?!?!?!!

Cannot connect to server again  >:(
Title: Re: IceTV server is down?
Post by: prl on February 29, 2012, 06:08:42 PM
I'm having IceTV server access problems (http://forum.icetv.com.au/iceforum/index.php?topic=3317.msg16408#msg16408) on Beyonwiz, too. Was working earlier today.
Title: Re: IceTV server is down?
Post by: AndrewT on February 29, 2012, 06:13:48 PM
And here as well on my Topfields and EyeTV - exactly a week from the last server issue.

Title: Re: IceTV server is down?
Post by: Markh on February 29, 2012, 07:16:51 PM
And again it seems no reponse from icetv. Pathetic really for a service we pay for.
Title: Re: IceTV server is down?
Post by: prl on February 29, 2012, 07:24:28 PM
Quote from: Markh on February 29, 2012, 07:16:51 PM
And again it seems no reponse from icetv. Pathetic really for a service we pay for.
This isn't an official support channel, even though IceTV staff do respond here sometimes. Contact IceTV Support if you want support.
Title: Re: IceTV server is down?
Post by: grampus on February 29, 2012, 09:40:47 PM
Quote from: Markh on February 29, 2012, 07:16:51 PM
And again it seems no reponse from icetv. Pathetic really for a service we pay for.
I reported a problem tonight via official channels, they responded with their explanation of my problems pretty much within a few hours, then the following exchange.
Very happy with the service.

My query,
May I broadcast this response in the forum?
I believe there was a comment re no advice from the support team.
Probably because the forum is not the proper place to ask for assistance?


Ice support response
We've been flat out today with more than normal phone calls. And yes, the forum is not the place to ask for official support
Title: Re: IceTV server is down?
Post by: markb on February 29, 2012, 10:31:51 PM
Quote from: grampus on February 29, 2012, 09:40:47 PM
We've been flat out today with more than normal phone calls. And yes, the forum is not the place to ask for official support

How silly and inefficient. There is not a lot of volume on the ICE forums so it would take a few minutes each day to scan new posts and respond where hundreds of people can find answers. They really prefer that each one of us phone or email them for a one on one response? Some of the things ICETV do don't make sense to me.
Title: Re: IceTV server is down?
Post by: prl on February 29, 2012, 11:18:28 PM
Quote from: markb on February 29, 2012, 10:31:51 PM
Quote from: grampus on February 29, 2012, 09:40:47 PM
We've been flat out today with more than normal phone calls. And yes, the forum is not the place to ask for official support

How silly and inefficient. There is not a lot of volume on the ICE forums so it would take a few minutes each day to scan new posts and respond where hundreds of people can find answers. They really prefer that each one of us phone or email them for a one on one response? Some of the things ICETV do don't make sense to me.
But as I posted before, they do scan the forum and often respond. But as I also said, and as grampus quotes from IceTV support, the forum is not an official support channel. They also say so in the main Technical Support (http://www.icetv.com.au/cgi-bin/websupport.cgi?op=list_faq_support) page:
QuotePlease note that we can't guarantee responses for all questions on the forum. If you require a direct answer from an IceTV staff member, then please contact us directly.
Title: Re: IceTV server is down?
Post by: markb on February 29, 2012, 11:31:26 PM
Quote from: prl on February 29, 2012, 11:18:28 PM
But as I also said, and as grampus quotes from IceTV support, the forum is not an official support channel.

Yeah, we know that. I'm just saying it is silly and inefficient. If I was running ICETV support I would post here profusely. It would cost me little time and save heaps of one-on-one phone calls and emails. Communication tempers peoples frustrations.
Title: Re: IceTV server is down?
Post by: grampus on March 01, 2012, 07:37:55 AM
Quote from: markb on February 29, 2012, 11:31:26 PM
Yeah, we know that. I'm just saying it is silly and inefficient. If I was running ICETV support I would post here profusely. It would cost me little time and save heaps of one-on-one phone calls and emails. Communication tempers peoples frustrations.
Not too sure I agree with that.
A majority of support organizations will not use fora as a medium to advise.
They find that it is unproductive.
It's much better to have a formal support path that can be used for a one on one conversation regarding specific problems.

Having been in a a number of support organizations, and having seen what the forum method of support brings, I can only agree with the Ice attitude.