My wife and I tested this, each using our own Testmy.net account, and we got two different cities in two different states.
Different issues here, assuming the trick is to isolate the latency factors relating (or un-relating in this case) to a particular beam:
1. User location: The use of geoip to determine user's location from the IP, doesn't work at all on satellite internet. Conversely, if the device being tested has fairly good gps and signal you might be ok for that. I know the gps on my iPhone is a lot more accurate than the one in my MacBook Pro - could be a problem if you're on the edge of a beam footprint. There may be some outliers in their database in that regard, so this is kind of a minimal risk.
2. Gateway provider: Have noticed that a certain provider (CenturyLink) randomly inserts 1-2 minute gaps in service on a national scale. Such an inconsistency would significantly vary the results for a particular beam that always routes to a gateway.
3. TestMy server location: Different locations are going to create different paths from the gateway. Assuming the CL gaps aren't there, you'd have to create a standard where certain beam/gateway combinations would use a specific TestMy server. For example, beam 68/SDO gateway (San Diego) would need to use the closest server in LA, not the default one in Dallas, because it would generate fewer hops. Depending upon the backbone (again, thinking about CL, which services most of the country now) fewer hops would reduce the risk of anything corrupting the test.
4. Time: Congestion varies over time and you'd have to compare with others doing the test within the same timeframe. It's not a safe bet that you'll get enough people doing tests from the same beam in the same timeframe, unless it's pre-coordinated.
If you can safely isolate these four things, the only thing left is latency due to beam congestion. Can be done, but it's just not real probable.
Edit: Almost forgot another real important one: Latency tests over TestMy aren't that accurate to begin with, so don't know if you could trust it for a viable comparison.
Yes, they do not have a correct location -- the city they determined for me was also in the wrong state. I am on the edge of the beam, and the city they selected as my location seemed to be toward the center of my beam. I came in with a different connection and got the same city an so I wondered how consistently they were choosing that city and more generally if others on the beam would get that same city. After digging through some tests I ran over a year ago, I see they had a different city chosen for my location -- also in a state different from me, but also toward the middle of the my beam, so it is not consistent.
So obviously since it does not consistently choose a city based upon the beam, it is impossible to compare a score against the average and know how your bandwidth compares to the beam in general -- as maratsade demonstrates by getting two different cities with different accounts. The gateways handle multiple beams in different time zones, but I did not see a city chosen based upon the location for one of those other beams, so it seems like they can narrow it down to a general geographic location based upon the beam, but can still be 100 miles off from the true location.
I was doing the bandwidth test -- I agree the TestMy.net latency test is fairly pointless especially as the HT2000W records average latency for every 5 minute and 1 hour period anyway.
The average score even for the bandwidth test would also not be an accurate measure not only for the reason you mention, but also because there are a lot of tests done on small test sizes which will not be accurate with satellite.
If you don't mind, what city did you get? Mark, I think you were beam 68 -- so did your city land toward to middle of beam 68 in northern VA?
Maratsade, would you mind sharing the cities you saw along with your beam? Or at least give a rough impression of how far the cities were from the center of your beam?
I dug through a bunch of cities this way, and the average seems to be around 35Mbps for what I thought were selected from J2 beams, and 25Mbps for J1 beams (which might be a combination of gen 4 and gen 5 services if I understand correctly), which is considerably better than what I imagined based upon reading posts on this site! It seems the great performance I get is not out of line with the areas I combed through. I did see a city on the edge between beam 68 and 69--far from the center of a J2 beam--that had an average of around 50Mbps, and they had run a lot of tests.
Thanks!
Beam 68 is centered somewhere on the line between Culpeper and Rapahannock County, Virginia. TestMy keeps thinking I'm in Boyce, which is way up in Clark County, east of Winchester. So, I don't think it's that. More likely it's using some combination of geoip and whatever location your browser thinks it's at.
Here's a link to the footprint for beam 68: https://satbeams.com/footprints?beam=10175
Incidentally, latency is usually the real tell that congestion exists because it will show delayed packets in the buffer. A beam could becoming congested and you'd not know it from looking at speed until the congestion becomes extreme. Then it's not just speed, but dropped packets as well, just because the system can't keep up with the buffered requests/responses.
"Maratsade, would you mind sharing the cities you saw along with your beam? Or at least give a rough impression of how far the cities were from the center of your beam?"
I'm on beam 68. Where's the center of this beam (as in, what city/town is closest)?
EDIT: Sometimes the "city" you get is "US."
OK, these are the approximate distances (in miles) from what looks like the town closest to the center of beam 68 (I'm guessing here, using SatBeams) and some of the towns/cities listed by TMN:
81, 75, 68, 55, 127, 35
