Using TestMy.net to compare your speed to others on your beam

[MrBuster]

I was poking around the 'TestMy.net Database' option under 'DB' on the TestMy.net page, and I noticed something that might be useful for knowing how your tests compare with other users of HughesNet that share your beam -- and thus experience the same "congestion" circumstances that you do.

After you select 'TestMy.net Database', there is a list of details that they use to ID you -- this includes a city.  As a satellite beam covers a wide area, the city I saw was some distance from me, but it appeared to be close to the center of my beam!  

Click the City, then click the ISPs tab, and check to if you see a speed average for HughesNet for this city which may  represent the average speed for your beam.

Would someone else try this and report what city you are seeing associated with your location along with your satellite beam number?  This will let me know if this idea actually holds water so to speak.  I have searched for others cities that I thought might be near a beam center and found some that I thought would be for Jupiter 1, VS1, and VS2 that appear to give speeds that are in the general range of what I guessed for each.

Does anyone familiar with TestMy.net know how to search a provider for cities that they have records for?  That would make it a lot easier to check if this is an idea with potential or just nonsense as the cities I found with records have been found with just wild guessing based on looking at a map.

Now, of course, many folks that run the test use the wrong test size (too small test size and the test result is dominated by the start up time due to latency), or they are using Wi-Fi, so the average result numbers are likely to be on the low side, but if this works, then this would give a way to know if you are experiencing performance on-par with others that share your beam.

[maratsade]

My wife and I tested this, each using our own Testmy.net account,  and we got two different cities in two different states.

[MarkJFine]

Different issues here, assuming the trick is to isolate the latency factors relating (or un-relating in this case) to a particular beam:

1. User location: The use of geoip to determine user's location from the IP, doesn't work at all on satellite internet. Conversely, if the device being tested has fairly good gps and signal you might be ok for that. I know the gps on my iPhone is a lot more accurate than the one in my MacBook Pro - could be a problem if you're on the edge of a beam footprint. There may be some outliers in their database in that regard, so this is kind of a minimal risk.

2. Gateway provider: Have noticed that a certain provider (CenturyLink) randomly inserts 1-2 minute gaps in service on a national scale. Such an inconsistency would significantly vary the results for a particular beam that always routes to a gateway.

3. TestMy server location: Different locations are going to create different paths from the gateway. Assuming the CL gaps aren't there, you'd have to create a standard where certain beam/gateway combinations would use a specific TestMy server. For example, beam 68/SDO gateway (San Diego) would need to use the closest server in LA, not the default one in Dallas, because it would generate fewer hops. Depending upon the backbone (again, thinking about CL, which services most of the country now) fewer hops would reduce the risk of anything corrupting the test.

4. Time: Congestion varies over time and you'd have to compare with others doing the test within the same timeframe. It's not a safe bet that you'll get enough people doing tests from the same beam in the same timeframe, unless it's pre-coordinated.

If you can safely isolate these four things, the only thing left is latency due to beam congestion. Can be done, but it's just not real probable.

Edit: Almost forgot another real important one: Latency tests over TestMy aren't that accurate to begin with, so don't know if you could trust it for a viable comparison.

[MrBuster]

Yes, they do not have a correct location -- the city they determined for me was also in the wrong state.  I am on the edge of the beam, and the city they selected as my location seemed to be toward the center of my beam.  I came in with a different connection and got the same city an so I wondered how consistently they were choosing that city and more generally if others on the beam would get that same city.  After digging through some tests I ran over a year ago, I see they had a different city chosen for my location -- also in a state different from me, but also toward the middle of the my beam, so it is not consistent.

So obviously since it does not consistently choose a city based upon the beam, it is impossible to compare a score against the average and know how your bandwidth compares to the beam in general -- as maratsade demonstrates by getting two different cities with different accounts.  The gateways handle multiple beams in different time zones, but I did not see a city chosen based upon the location for one of those other beams, so it seems like they can narrow it down to a general geographic location based upon the beam, but can still be 100 miles off from the true location.

I was doing the bandwidth test -- I agree the TestMy.net latency test is fairly pointless especially as the HT2000W records average latency for every 5 minute and 1 hour period anyway.

The average score even for the bandwidth test would also not be an accurate measure not only for the reason you mention, but also because there are a lot of tests done on small test sizes which will not be accurate with satellite.

If you don't mind, what city did you get?  Mark, I think you were beam 68 -- so did your city land toward to middle of beam 68 in northern VA?

Maratsade, would you mind sharing the cities you saw along with your beam?  Or at least give a rough impression of how far the cities were from the center of your beam?

I dug through a bunch of cities this way, and the average seems to be around 35Mbps for what I thought were selected from J2 beams, and 25Mbps for J1 beams (which might be a combination of gen 4 and gen 5 services if I understand correctly), which is considerably better than what I imagined based upon reading posts on this site!  It seems the great performance I get is not out of line with the areas I combed through.  I did see a city on the edge between beam 68 and 69--far from the center of a J2 beam--that had an average of around 50Mbps, and they had run a lot of tests.

Thanks!

[maratsade]

"Maratsade, would you mind sharing the cities you saw along with your beam? Or at least give a rough impression of how far the cities were from the center of your beam?"

I'm on beam 68. Where's the center of this beam (as in, what city/town is closest)?

EDIT: Sometimes the "city" you get is "US."

[maratsade]

OK, these are the approximate distances (in miles) from what looks like the town closest to the center of beam 68 (I'm guessing here, using SatBeams) and some of the towns/cities listed by TMN:

81, 75, 68, 55, 127, 35

[MrBuster]

---

@maratsade wrote:

OK, these are the approximate distances (in miles) from what looks like the town closest to the center of beam 68 (I'm guessing here, using SatBeams) and some of the towns/cities listed by TMN:

81, 75, 68, 55, 127, 35

---

Ok, I see -- I think you are showing the distance for different cities identified as your location to the approximate center of your beam, so with 127 miles it must be that any city in the beam can be shown and not just a city near (within 60 miles or so) of the center.  127 miles might almost be to the edge of the beam, so maybe the city I saw with the super fast Hughes net Mbps average could have been from someone in Maryland in (or there about) in either beam 68 or beam 69 since it was right at the edge.  Maybe somebody with really good test scores decided to do a few hundred tests in the last week to get that high average for the city!

[MrBuster]

---

@maratsade wrote:

"Maratsade, would you mind sharing the cities you saw along with your beam? Or at least give a rough impression of how far the cities were from the center of your beam?"

I'm on beam 68. Where's the center of this beam?

EDIT: Sometimes the "city" you get is "US."

---

Do you not get a city doing this?

From TestMy.net, select from the DB menu 'TestMy.net Database'

2. This leads to this display -- is there not a city shown?

3. Clicking the City leads to a display like this (the graph below would include all ISPs):

4. Clicking on the ISPs tab shows this:

This city is pretty far from me, but it does apear to be roughly in the middle of beam 82.  I see in the past I had a different city showing, but it was in the same rough location.

The averages shown here are lowered by tests with the wrong test size, but it does appear to include my tests, since when I run a test, the count here increases.  I do not know what they mean by '64 tests recently' perhaps 64 tests in the last week?

The average looks good -- about what I get a lot, and the impression is that others in this beam that happen to get this city are getting similar performance and therefore beam 82 is *not* over sold 🙂

Beam 68 appears to be centered over northern VA, right near the border of WV -- I am just going by that picture someone posted a while back.  I saw a link to some satellite page that shows spot beam positions more accurately, but I don't know much about it other than looking at it once or twice noticing the handful of spot beams J2 has down around Ecuador and in Mexico.

[maratsade]

"2. This leads to this display -- is there not a city shown?"

Yes, different cities with different tests.  I thought you wanted to know how far they were from the center of beam 68.   The distances to what I think is the center vary, as do the distances from each of these cities and my location. 

[MrBuster]

---

@maratsade wrote:

"2. This leads to this display -- is there not a city shown?"

Yes, different cities with different tests.  I thought you wanted to know how far they were from the center of beam 68.   The distances to what I think is the center vary, as do the distances from each of these cities and my location. 

---

Yes, that was what I wanted -- it appears the city can be anywhere in the beam.  I posted the above before I saw your reply with the distances.

If the cities are always within your beam, then it seems likely that the test results for HughesNet for the city are for users sharing the beam with you -- if the average is good, then the beam can not possibly be over sold, as obviously other users recently had scores high enough to raise the average given that many users are posting tests that artificially lower the average by using too small a test size, or by using Wi-Fi for the test.  What we can not see is the overall average for the entire beam, since the numbers are divided up to the cities in the beam.  Also, if the city can be anywhere in the beam, then some locations may be showing test results from J1/gen4 mixed in the results, depending on where you are.

Now, if we find a city showing in the above way that is nowhere in the beam foot print, then we can no longer think scores attributed to that city for HughesNet share your beam.  My thinking is, if someone says they have an over sold crowded beam, then given the beam number, it is possible to find cities in that beam and review an average of other users that are likely in the same beam.  If the average of recent tests good, it seems likely the issue is not that the beam is crowded since there has to be somebody with scores good enough to bring the average up.

[maratsade]

"if the average is good, then the beam can not possibly be over sold"

But there may be other reasons for speed issues -- in 2017 there were problems with Bean 68 and the speed was pretty bad for a while and didn't get better until the engineers fixed whatever the issue was. 

[MrBuster]

---

@maratsade wrote:

"if the average is good, then the beam can not possibly be over sold"

But there may be other reasons for speed issues -- in 2017 there were problems with Bean 68 and the speed was pretty bad for a while and didn't get better until the engineers fixed whatever the issue was. 

---

Yes, absolutely.  When there was that problem, presumably all recent tests run when the problem existed would have been bad and the recent average would be low. 

Just because the recent average is bad does not imply the beam is oversold.  But if the average of recent tests is good, then how can anyone say the beam is experiencing poor performance because it is oversold?  

If I have repeated bad test results recently, but the average of recent tests is good as shown in the DB, then I think the following are implied:

• The entire beam is not having problems
• The beam is not over sold

I think this can be a useful way to say beam X is *not* oversold, and appears to be running good recently.  It is not a demonstration that beam X is oversold.  It also lets us know approximately how we are doing compared to others in our area.  This kind of test does not offer proof the beam is oversold -- it simply provides evidence it is not.

[maratsade]

[Sorry, deleted something by mistake -- I had asked if there is evidence that an oversold beam functions poorly. Since we don't know what beams are oversold, if any, we can't really test, so all of this seems to be just speculation). 

EDIT: other reasons for poor performance could be issues with specific servers (such as web acceleration), and gateway issues.  Performance issues were present in several beams towards the beginning of Gen 5, and the beams were not even close to oversold.  Congestion happens very easily too, especially at prime time, and it doesn't take that many people to clog the beam. If memory serves, without optimisation, clogging happens with just 64 subscribers using the system at the same time.  I imagine optimisation raises this number, but since the amount of data to go around is not all that much, the quality can deteriorate pretty quickly, especially when those 64 or so people are all trying to stream. 

EDIT 2: This is where I got the 64 figure. 

[MrBuster]

---

@maratsade wrote:

Is there evidence that an oversold beam always functions poorly? 

---

Of course not.  But it is unlikely that an oversold beam will have an *average of many* recent tests above 25Mbps and therefore probable that the beam is not oversold, and so it looks to me to be evidence the beam is not oversold.

Of course, I laugh pretty hard at the idea of trying to figure out when I can run 50 tests when everyone else is asleep so as to make an oversold beam look good 🙂

Maybe we can coin a term for it -- "salting a beam" haha

[maratsade]

"I  can run 50 tests when everyone else is asleep so as to make an oversold beam look good :-)"

LOL. 

[MarkJFine]

Beam 68 is centered somewhere on the line between Culpeper and Rapahannock County, Virginia. TestMy keeps thinking I'm in Boyce, which is way up in Clark County, east of Winchester. So, I don't think it's that. More likely it's using some combination of geoip and whatever location your browser thinks it's at.

Here's a link to the footprint for beam 68: https://satbeams.com/footprints?beam=10175

[maratsade]

thanks, Mark. I thought maybe the center was somewhere around Culpeper.

[MarkJFine]

Incidentally, latency is usually the real tell that congestion exists because it will show delayed packets in the buffer. A beam could becoming congested and you'd not know it from looking at speed until the congestion becomes extreme. Then it's not just speed, but dropped packets as well, just because the system can't keep up with the buffered requests/responses.

[MrBuster]

---

@MarkJFine wrote:

Incidentally, latency is usually the real tell that congestion exists because it will show delayed packets in the buffer. A beam could becoming congested and you'd not know it from looking at speed until the congestion becomes extreme. Then it's not just speed, but dropped packets as well, just because the system can't keep up with the buffered requests/responses.

---

I agree -- if I download the log and graph the column of numbers behind the RTT flag, during normal week days for my beam it rises briefly at 2am, 3am, then rises 6:30am-7:30am, and does the big rise from 6:30pm to 11pm.  The highest numbers usually are around 8:30pm.  Other times it is at the minimum.  This number appears to only include the time for the queueing up and ride over the satellite, as they have some other columns that look at the time past that.  There is also a column for lost packets and %.  If this number goes above 2000ms average for a 5 minute period, you get a red X on the 5 minute chart.  If the 5 minute chart records 2 Xs (or maybe 3?) it gives a red X on the hourly chart -- simple and useful, I thought.

But while this seems to give some insight about my own beam and how busy it is at different times a day, it does not tell me anything about your beam.

If I was moving or considering Hughesnet, I think this gives me some idea what it is like around beam 68 recently.  If this is way off from your expectation and experience, please do let me know.  This score is not too shabby!   This score looks decent since a bunch of these tests would  have been run from phones over 2.4Ghz WiFi with small test blocks that lower the average!

Now on the other hand, if we find that 40 of these 109 tests were run from a beam in Idaho, and that the people shown here are not in beam 68, then this tells us very little.

I am running some more tests on my beam to see how much "salting" of the beam I can do since I am pulling around 50Mbps at the moment....

[MarkJFine]

Your data sounds about right:
More and more people are making use of the Bonus period to do heavy lifting (sofware updates, etc.). The ones you see at 0230 are likely automated while people are sleeping, and the ones at 0630 are likely manual as people are getting up. I personally do all mine at 0500 to beat the 0630-0700 rush hour.

I'd be willing to bet the evening rise actually starts around 1630 and continues to ramp to a frenzy between 20-2100, then becomes normal again between 22-2300. That's always been the trend, and almost always when people are trying to stream something.

Streaming always puts the biggest load on the system because of the number and consistency of the packets, as well as the potential need for unecessary resends due to the delays in a streaming server getting an ack that a segment was received correctly, and in the proper sequence. It puts a tremendous load on system resources, which is not good when resources are limited.

In a situation where latency increases with congestion, all these people that say they keep getting buffering yet continue to let it keep loading, playing, and buffering just make it worse. It's like holding a mic to a 1,000w amp speaker and wondering why the feedback just keeps getting louder and more intense until it can't take anymore and it just clips. Repeated buffering is the system's way of telling them "this isn't working, you're killing me, please stop" but they don't get the hint because god forbid they miss their Netfilix show, so everyone on the beam and ultimately the gateway suffers. Then they blame the network for something they did (Modern entitlement problems). </soapbox>

[MrBuster]

As anyone that has followed the thread above may have noticed, there is a button to display the data behind the median and average shown. So you can review the individual test results and copy them to exclude the suspicious results created by miscreants, nefarious individuals, dirty birds, or friendly flounder that may have been engaged in salting the beam with wild scores for fun or profit.

Ah! Netflix time! Personally, I like buffering because birds are incontinent by nature and I can use the breaks, so prime time here I come!  HD