Request for slight modification to SPEEDTEST_BLOCKED_SERVERS behavior

[nwh3365]

With regard to PR 1896, I would like to suggest a slight modification to the process map provided.

I have been running SPEEDTEST_BLOCKED_SERVERS for two months, and the same server has been used 100% of the time; it is essentially behaving as if SPEEDTEST_SERVERS had been specified with a single server.

Separately, I have been running "speedtest -L" on a different server every two hours for nearly two months (2624 runs), and the results have been surprising (at least to me):

• Only 16 unique servers have appeared
• Only 6 unique orders of servers have appeared; one of these was due to a change in my ISPs network architecture
• 7 servers have appeared in nearly every list (5 servers have appeared in 100% of the lists; 1 in 99.9%; 1 in 99.8%)
• The same server has appeared first in the list 100% of the time

For me, this confirms that speedtest-tracker's use of the same server 100% of the time since I implemented SPEEDTEST_BLOCKED_SERVERS is in fact correct. I didn't really doubt that, but I did find it surprising that it is now using a single server 100% of the time. This led to my curiosity about how the list changed over time, and for me it doesn't change much.

I would like to suggest the "Blocked List Present" side of the process map choose a random server from the SPEEDTEST_BLOCKED_SERVERS filtered list (or maybe the top half of the list), rather than always choosing the first one. It seems like this might more closely mimick the behavior of letting speedtest choose, but with the benefit of the blocked servers list. When speedtest was allowed to choose, it didn't use the same server every time, but it did use a small collection of servers. While it did favor one server 54% of the time in my testing which isn't exactly random, it still had some randomness to it that speedtest-tracker with the blocked servers list is currently missing.

I don't know if the list of servers returned by speedtest -L for me is just incredibly stable (i.e., the same server has appeared first in the list for nearly two months straight, and of 2624 runs there have been only six unique orderings of the list), or if speedtest's automatic selection is somehow introducing a random, but biased behavior into it's server selection process, but I do miss being able to see how consistent (or not) performance was among the top servers rather than just seeing a single server all the time.

I'd be curious to hear if others using SPEEDTEST_BLOCKED_SERVERS have seen similar results, and how they feel about this suggested modification.

[svenvg93]

See the explanation how it works here #1979 (comment). Hope this makes it more clear 😄.

[nwh3365]

Background:

My original desire was to have speedtest auto-select from the ten "best" servers so that I could see a more rounded view from a handful of servers. I had three servers that consistently performed significantly worse than all the rest, which made the graph look far worse and more inconsistent than reality actually was. That's how my request for a blocked server list originated; I simply wanted to filter out the servers that consistently performed poorly, most likely due to limited bandwidth or overuse.

I know how I wanted it to work, and how I thought it would work. What I and others are saying is that SPEEDTEST_BLOCKED_SERVERS is not behaving as we wanted/expected. It's not a bug in the sense that the code is incorrect, it's a bug in the sense that the change isn't doing what we wanted/expected it to do; it has fixed one issue, but created another.

As @daviddavid indicated in #1978, if SPEEDTEST_BLOCKED_SERVERS is set, all future requests use the same server. His expected behavior was that it would continue to be randomized, but the servers listed by SPEEDTEST_BLOCKED_SERVERS would be filtered out from testing. This is exactly what my expectations were as well.

@daviddavid indicated he had ran 24 hours of tests to confirm the behavior. I have been running four tests an hour for 54 days with SPEEDTEST_BLOCKED_SERVERS and it has used the same server for every single test. This is not the behavior seen when speedtest is allowed to make it's own selection. This morning I removed SPEEDTEST_BLOCKED_SERVERS and it immediately began using random servers again. This is the issue we are trying to convey.

New Discovery, Probable Solution, New Roadblock

Alex's explanation in #1979 describes what the code does, which is how I understood it to be. The part we have incorrect, as I discovered this morning, is our belief that "speedtest" always selects the first entry from the "speedtest -L" list. It does not. This is why "speedtest" appears to have a sort of randomness to the servers selected, while SPEEDTEST_BLOCKED_SERVERS uses the same server every time.

Through additional testing I think I have identified the issue. I thought I had a solution until another roadblock popped up. Here's what I have learned.

"speedtest" performs a speedtest by auto-selecting a server

"speedtest -L" produces a list of the ten best servers for testing based on some unknown algorithm

"speedtest --selection-detail" appears to produce the same list (in the same order) as -L, but in a slightly different format, then proceeds to run a speedtest with the "best" server in the list, but not necessarily the first server in the list

I ran these three variations on the same machine simultaneously and discovered that the "speedtest -L / speedtest --selection-detail" lists are not in best-to-worst order. The best server, and the one "speedtest" appears to select, is the one that has the lowest latency. More times than not this is the first server in the list, but it could be any server in the list depending on network and server loads at the time speedtest is ran. This is what produces the "random with a preference" server selections.

I think the selection process needs to be: grab the list, filter it, select the server with the lowest latency (instead of the first one in the list). Unfortunately (i.e., roadblock), the latency values used for selection only appear in the --selection-detail list (not the -L list), AND --selection-detail automatically runs a test against the best server so there is no opportunity for filtering out the blocked servers. Thus far, I have not found a way to obtain the latency values from speedtest without also running a test.

As @daviddavid has suggested, one could use a random selection process. That would introduce randomness into the selection processs, however that is not what speedtest is actually doing; it's selecting the best server based on latencies. A truly random selection process could lead to more inconsistency. For example, a randomly chosen server that is overloaded or behind a congested network probably wouldn't have been chosen by speedtest based on latency.

A brute force method of replicating what speedtest is actually doing would be to grab the -L list, filter it, ping each server, and select the one with the lowest latency. Functional, but not necessarily elegant.

If only Ookla would include the latency values in the -L output.