Revamped testing methodology
We've been testing routers and other wireless products using a fixed test procedure for a number of years now. Lately, it became more and more clear that our tests at distances of three and ten metres with line of sight often times no longer revealed substantial differences. As a result, we decided to retire these tests.
We did keep the test in which we connect three laptops to a router, because this still gives a good impression of a router's relative performance. In this test, we wirelessly connect three BTO laptops to the router that we're testing. We use laptops with Intel Centrino Ultimate-N 6300 chips for the 802.11n tests, and models with Intel Wireless-AC 7260 chips for the 802.11ac tests. We use Ixia's IxChariot software to perform these tests. We decided to go with the High Throughput script, as all routers are able to handle this script without any issues. The Ultra High Throughput script does cause many devices to buckle at the knees, resulting in rather low throughput rates.
Do note that the AC-7260 chip only has two antenna connectors, which means that it can't make use of all three or even four antenna connectors that contemporary flagship models have. However, as far as we can tell, it is the only 802.11ac chip that is currently being sold separately, so if you're looking to upgrade an existing notebook, you'll end up using this particular chip. Hence us using it anyway.
During the last few months, we've spent a substantial amount of time experimenting and developing a new test procedure that should give a more accurate estimation of a router's throughput. In the end, we decided to go with a few extra tests that we'll perform in our office. In these tests, we measure throughput rates in two different locations in the building. Bandwidth figures are obtained by wirelessly transferring a large file both from and to an Asustor AS-604T NAS on the network through the router that we're testing. The NAS has been outfitted with SSDs, ensuring that throughput rates of over 100 MB/s can be maintained when using a wired connection. We use TeraCopy to copy our test file, a video fragment with a file size of 307 MB.
We perform this test on two different locations, but during the set up of this test we actually tested on five locations. The remaining two locations give a good representation of real-world performance. Copying a file won't always go flawlessly, among other things because of several background processes on the laptop. This is why we performed the test multiple times, at different moments, after which we let out any possible outliers.
For the time being, we'll perform this test in two different locations. The first test is performed at a distance of ten metres with line of sight, in order to determine real-world throughput under nearly ideal circumstances. You'll find that this figure will always be a lot lower than the one measured with IxChariot, due to an increase in overhead. In the second test, we'll position the laptop in a room that's right above the router. The two devices are separated by a thick sheet of concrete that is similar to those found in modern homes. The distance between the router and the laptop is approximately four metres. While this might not seem like much, it turned out to be fairly challenging for the routers that we have tested thus far.