WiFi Sucks In Peregrine... And Probably Elsewhere 10Jan19

Ok; we are pretty technically savvy, and Matt, Kala & I have begun experiencing some significant WiFi lags on our network in Peregrine. We embarked on analysis... a net code jock, an old net mgmt exec and a net tech prof united in pursuit of clean WiFi. It's especially bad (more frequent and more severe) on 2.4GHz but I've even experienced unacceptable delays/latency/ping times on 5GHz. We have ~40MB down & ~12MB up from CenturyLink - but here I am completely focussed on and isolating the local WiFi network... and I know that's where the problem is as my wired router connections are "full on always" and deliver only <.5msec delays and 40Mbps uploads. That is what I expect between my wired MBPro and my server on the same (bridged) net.

Wired Network Latency

At first I dove on the Nighthawk R7000 forum and found similar problem descriptions related to firmware releases - I down-revved the image and it fixed it for a bit but I was not thorough in re-experimenting. Throughput and latency were great for some time on both 2.4GHz and 5Ghz but it didn't sustain there for >10min. Debug led us all the way to taking down all our client devices and building various new networks in an attempt to assess via a one-client (on AP: either ours or a mac in ad-hoc mode) - one AP network... to isolate the cause of the intermittent "outages." Here I dedicated a 5GHz WiFi net between my AP and my MBP and saw a 90msec ping! That's acceptable maybe if I'm round trip to Himalaya... but not my own local dedicated net without other traffic on it.

Dedicated 5GHz Net Latency

On a dedicated 2.4GHz one-client net I did much worse... WTF!

Dedicated 2.4GHz Latency

Someone should make a tool or monitor for this stuff :)...  but the AirCheck, while informative, could not help us isolate. We even experienced the "bad WiFi" when re-locating our test networks to the park or the open space somewhat nearby. The 2.4GHz channels are packed and interferers must exist in great scale. The 5MHz band is better but the AFA must be beaming blockage our way. :0.

The S/N rations are ok actually... though signal drops have occurred. If my net could sustain an exclusive move to 5GHz and strictly .ac that'd maybe be OK (for a time) but so many devices are 2.4GHz only that it can't. My current working theory is, after perusing the neighborhood and diving for 2 days with many techniques including WiFi AirSniffers, many experiments and the like, is that the proliferation of devices over the last several years, especially mesh WiFi like Google's in our area, is that the air is very dirty/noisy and "clogged"... one house in our area had 8 APs and usurped the entire 2.4GHz channel 1 band with their 40MHz wide-channel settings alone. Plus, continuous monitoring of behavior has extended from cellular to WiFi by law enforcement - doubling the traffic on the air.

I've attached to several local WiFi nets of friendlies in Peregrine (names and nets withheld for obvious reasons) and am seeing the same remarkable shift from regular ~2msec latency on WiFi ~3yrs ago to sketchy +20msec or even +100msec measures. No one is playing intense games here on WiFi here as a result.

WiFi is absolutely great, but it is seemingly dying under its own success. I'm sure a more collaborative WiFi net is in the offing from Silicon Valley - one which collaborates with other nets in proximity to optimize for mutual benefit. Beware. The best would be more allocation of frequency to innovation IMO, not sales of frequency to Verizon, et. al. or access to monitored bandwidth by my government.

Please do some of your own dedicated net (best is one client to one AP but just some simple pings between phone or PC and AP as above are fine) experiments and send the anonymous results to me as Comments here or via private email - I can't tell crap[ from IP addresses!

UPDATE From Silverthorne
Even in Silverthorne, on the edge of the Arapahoe NF, the 2.4GHz band is littered with networks and Zigbee IoT, and yields highly varying latency times.

The 5Hz band is much cleaner and the response times show it in much greater consistency.

UPDATE From San Fransisco [bg]
Here in my flat, I see >100 networks available on 2.4GHz. At any location in my flat, there are always at least 3 strong signals on each of the 3 non-overlapping bands. See attached for a scan in a typical location.

As I understand it, with current protocols, the weak signals cause just as much problem as the strong ones. If your receiver can hear an interfering message, it will delay using the channel.

These folks who position their traffic at channels other than 1/6/11 are just managing to interfere with MORE traffic. And of course the 80MHz options are a disaster. They're supposed to shut themselves down when they see conflicts, but it's obvious they don't (see attached again).

To make things worse, here in SF (and many other places I think), Comcast has implemented a system where the routers they distribute to their customers include a second AP network which they use to create "public" hotspots available to their customers. This means that anywhere in SF, you will always see 5-10 Xfinity networks. Thanks Comcast for completely saturating the entire wifi spectrum. It's not quite as bad as it sounds since these networks probably aren't used much.

I'm pretty sure you know this, but there are a zillion other things that use the 2.4GHz band which could be interfering with you. Baby monitors, microwave ovens and garage door openers are notorious. And now that every freakin' piece of electronics in your house thinks it should be on the internet, there are probably lots of transmitters you don't know about or can't control.

A year ago, I decided that the only way to have decent wifi was to use a bunch of 5GHz APs. In that scenario, the low penetration is a benefit. Sadly, several key devices I use still don't have 5GHz radios.

UPDATE From Peregrine [02Feb19]
I rallied more gear and monitored for a longer time. I "tamed" or eliminated some of the hidden networks that were my fault but I can not get rid of Roku's hidden network - and it strangely follows my network as to channel. The biggest offenders in the area I see now are the Orbi77 (Logitech) with multiple APs and added hidden networks, 40MHz channels, and bad channel assignments and the Goblin (Google) with similar attributes... sheesh.