Web Acceleration failing

[GW]

Recently my HT2000W with current Software Version: 3.4.6.26 (M) has been going to failure on Web Acceleration showing yellow on System Status page link and showing the Down status and red X for Web Acceleration under System Status. This is with RSS of 80-90.

Under Diagnostics on the Advanced Configuration page it's been showing the red X virtually all the way across for DNS Acceleration (N) on both Hourly History and Current Hour History graphs.

I've been able to fix it momentarily by disabling/re enabling under Web Acceleration/Control. This would bring my System Status light back into the green for a little while but would go back to yellow and the red X very quickly. Resetting MBX seemed to have no effect.

I could only keep the Web Acceleration enabled for seconds early this morning so I bit the bullet and called in to Tech Support for a remote fix. After 40 minutes of going around in circles with phone rep, he finally sent some codes to the modem and helped the problem. I haven't seen yellow status and the red X for Web Acceleraration under System Status ever since but the Hourly and Current Hour graphs are still showing lots of red for the DNS Acceleration (N) item.

It works okay but the question is... does this type of problem indicate modem failure as a coming event? This modem has been in service for a few years.

[Remy]

GW,

After running some diagnostics, I saw a few persistent errors recurring on the modem.. I definitely feel a new modem will resolve your issue, so I'll be sending one to you, with all fees waived. First let's see if a modem swap will do the job. You can use its box to return the old modem. For your convenience, the return label is included in the outer pouch. 

Let me know how the new modem works out for you!

Thanks,

Remy

[GW]

Thank you. I'll be watching for it and will let you know.

[GW]

Hi Remy,

The modem arrived today. Thanks. It looks well-used and came with user installed passwords but it activated ok and it works.

It came with a weird set of diagnostic codes (0051-2000-0202-0480 and 0051-2005-1200-5126) and it's own array of red X's in the history graphs. Can you please have a look?

Edit: It lost the 0051-2000-0202-0480 code and traded it in for 0000-2020-4802-0487. I have no idea how to research these now. It seems like the previous generations of modems had an explainer function built in.

@Remy 

[Remy]

GW,

Upon looking into some diagnostics, it doesn't seem like the diagnostic codes you're receiving should have an affect on the performance of your modem. Is the new modem working properly? I'm glad the new package got out quickly to you.

Thanks,

Remy

[GW]

Hey Remy,

It's not doing really great but I hope it will get better. I'm getting some page load errors and it's giving my Thunderbird some problems with getting the mail.

Mid to low 80's signal is good considering we have a thick dust cloud from Africa clotting our atmosphere down here in Lower Alabammy. Ping is about average. Web response test to some server in North Carolina wasn't impressive at 4.5 seconds. Chicago was better at 2.6-3.0 seconds.

It's hanging in there with a 0.0.0 state code but throwing different diagnostic codes every hour. Odd looking codes. I hope it can settle down to the regular 0000-0000-0000-0005 like my modem. I've seen two instances of 0208-0000-0000-0005. I don't know what that one means but maybe it's getting closer to the no errors code.

It seems to be improving a bit in the RedX department. Now only showing the X for TCP and DNS (N) acceleration in the graphs. Down to a total of 12 Red X's at the moment.

It was doing some miserable DL tests of 13-15 Mbps at testmy but it's picking up a little bit now.

Dumping the reboot log into a spreadsheet for counting shows this thing has been power cycled 100 times since 1/29/19 with two boots for software upgrades. It's definitely got some serious mileage on it to go along with all the scratches and rub marks.

[GW]

It threw lots of state codes overnight. When I woke up my computer at 5:30 AM, status is in the yellow with Web Accelleration Down. RSS is 85. Diagnostic code is 0000-0000-0000-0005. State code is 30.1.1. At 5:40 AM it's back in the green.

11.2.1	1	195	06/26/2020 04:02:18	06/26/2020 04:05:33	0.3779%
11.2.2	1	90	06/26/2020 04:00:28	06/26/2020 04:02:18	0.1744%
11.3.1	1	10	06/26/2020 04:05:33	06/26/2020 04:05:43	0.0194%
20.2.1	1	39	06/26/2020 04:05:43	06/26/2020 04:06:22	0.0756%
30.1.1	2	623	06/26/2020 10:29:37	06/26/2020 10:39:37	1.2074%

Edit: The diagnostic codes are flipping all the time and it's loving State Code 30.1.1.

30.1.1

4

1823

06/26/2020 11:09:12

06/26/2020 11:19:12

3.3778%

Just a note, it probably isn't the radio. That was changed two weeks ago by a technician. He replaced the old Gen4 ODU with the new version.

[C0RR0SIVE]

20.2.1 and 30.1.1 makes me think congested/overloaded gateway and/or WAS...  I remember seeing 20.2.1 frequently at one time back when I was on ES17 with a heavily congested beam.

[GW]

Thanks for the info.

20.2.1 is listed as "A management problem has occurred"

30.1.1 is rapidly becoming the default state of operation. It's shut down again but I can hit the control and fix it.

[GW]

@Remy 

My Red X count is now at 27 in the history graphs.

The terminal downtime has exceeded 11% due to State Code 30.1.1.

[Remy]

GW,

Thank you for the updates. I will continue to look into this for you and will be getting back with you when I have a report with some more information on a possible fix. We appreciate your understanding and cooperation! 

Thanks,

Remy 

[GW]

I truly appreciate your help Remy. I'll keep you updated. I was hoping it would get better but it's not. Terminal downtime is still increasing. Now it's 17%. 5 hours out of the last 29 hours. 28 on the Red X count.

[GW]

@Remy 

Ok, it looks like network shenanigans to me. Check the zero count Red X's from 11 PM  Friday through 5 AM Sarturday local time.

And then note details of every 30.1.1 error I've captured and posted to this thread shows an even 10 minutes in duration.

[GW]

Here's a little more info for you Remy.

In past day this thing went to 30.1.1 a total of 51 times for a duration of 8½ hours.

I never realized how crippled Hughes is without fully functional web acceleration. There are many tasks that simply can't be done. I sure hope this isn't a solar coronavirus breaking my beam

[GW]

Recent diagnostic codes. The 0000-0000-0000-0005 coincides with a no Red X period in the hourly history

0204-8020-4802-0485	(10:00:00 - 13:00:00)
0000-0000-0000-0005	(07:00:00 - 10:00:00)
0204-8020-4800-0006	(04:00:00 - 07:00:00)
0204-8020-4802-0485	(01:00:00 - 04:00:00)
0204-8020-4802-0485	(22:00:00 - 01:00:00)
0204-8020-4802-0485	(19:00:00 - 22:00:00)
0204-8020-4802-0485	(16:00:00 - 19:00:00)
0204-8020-4802-0485	(13:00:00 - 16:00:00)

[Remy]

GW,

I appreciate you tracking these for me. Upon further investigation, it seems that these diagnostic codes won't have an effect on your services, as quite a few of them are just standard processes in the modem. In regards to the 30.1.1 that seems to be popping up, that state code refers to backing off on adding the modem to a specific web accelerator server, to avoid additional loading times. Does the service seem to be working well?

Thanks,

Remy

[GW]

---

@Remy wrote:

I appreciate you tracking these for me. Upon further investigation, it seems that these diagnostic codes won't have an effect on your services, as quite a few of them are just standard processes in the modem. In regards to the 30.1.1 that seems to be popping up, that state code refers to backing off on adding the modem to a specific web accelerator server, to avoid additional loading times. Does the service seem to be working well?

---

No Remy, it isn't working well at all. I guess you missed the message I posted for you last night. The one where I say:

"I never realized how crippled Hughes is without fully functional web acceleration. There are many tasks that simply can't be done."

I think you also may have missed the part where I advised you something is nuking the web acceleration function in precise 10 minute chunks of time. Per the logs, it's happening all day and all night with the notable exception of proper operation during the wee hours. From 2 AM until 5 AM local last night and from 11 PM until 5 AM Saturday night.

Otherwise, Web Acceleration has been in a state of failure 145 times for a total of almost 24 hours since I put this replacement modem into service 4 days ago. It's getting worse and happening several times an hour now.

After being with Hughes since 2011, I'm thinking this isn't a normal situation.

[Remy]

GW,

I apologize for misspeaking, I was asking more of if your service is working well when the state code is not showing. The 30.1.1 state code isn't one we can manually fix, as it is connected to a Web Accelerator server not being available at the time. As soon as one is available to the particular user, you should be placed there. For a possible fix, I suggest trying to turn off the Web Accelerator and turning it back on, as this can sometimes hasten the process.

Thanks,

Remy

[GW]

That didn't work. The only thing many resets of Web Acceleration may have accomplished is a slight slowing of the failure rate. Otherwise, no changes. No Red X's in the hourly history between 2 AM and 5 AM. Then right back to failures.

Is it the network or is it the modem?

[GW]

@Remy 

Still failing on 30.1.1 but now it's completely shutting off with a lot of downlink and gateway errors. I hope this is the result of someone trying to fix my problem.

[Remy]

GW,

30.1.1 itself is normally a network issue, however, seeing the rest of the state codes may be a power issue. Are the modem and cables plugged in securely? Are you connected to a surge protector, or directly to an outlet?

Thanks,

Remy