We spent weeks testing every aspect of these thermostats, going so far as to build a custom testing board to compare them side-by-side. This board used a series of components to convert the signal from the thermostat to the control of a standard wall outlet. We did this three times and hooked a space heater, a portable AC unit, and an auxiliary fan to each of the outlets. This meant that when a particular model called for the furnace to turn on, it was actually turning on the space heater, with the same pattern following for AC and the fan.
This allowed us to repeat the test on each thermostat immediately, leading (in our opinion!) to a better, direct comparison between models.
Ease of Use
The most important scoring metric that we looked at, and consequently the metric that made up the majority of our scores, was ease of use. We looked at the initial thermostat setup, setting the date and time, as well as setting a sample schedule — something that would be applicable to a standard office. Initially, we would attempt to accomplish each of these tasks without consulting the manual — as we know there are plenty of people out there that might not necessarily consult it — and only looking at it when we were truly stuck, ranking each product on its ease of use and intuitiveness. Some of the models guide you through a series of prompts, and were by far the easiest, while others definitely required the manual to have any chance of figuring out what to do.
After initializing each thermostat, we played with setting up the more advanced features, such as setting a lock on the thermostat, multiple heating stages, and notifications. Finally, the ease of adjusting the current temperature, and the variety of customization each thermostat allowed for holding the adjusted temperature. Some models would simply hold it indefinitely, while others would wait until the next scheduled change.
Installation & Support
Having already formed some opinion when we were wiring the base plate of each model, we also installed each model into the building where we work, to more accurately approximate what the typical person would go through. We also evaluated the supplied documentation, and whether or not we needed to consult online resources to obtain the knowledge to feel comfortable installing each thermostat. The final component of the install was to assess the difficulty of installing the companion app on a phone and pairing it with the thermostat
Now that we had completed the install, we evaluated the support each manufacturer provided for their customers. After removing the current thermostat in our office, we sent in a picture of the bare wires protruding from the wall to each manufacturer and asked if their thermostat would be compatible. Responses that were expedient and helpfully walked us through where and how to determine which wire was which scored highly, and companies that required follow-up emails or were less helpful scored poorly. A similar process was conducted for each company over the phone, though we took the time and number of transfers into account when determining scores.
We also rated the supplied documentation to aid with install for each product, either the hardcopy manual, guided prompts, or online resources. After figuring out how to connect each thermostat, we installed each model on our testing board, as well as in our office.
Finally, we looked at the ease of installing the companion app and connecting the thermostats to our WiFi network.
Remote Access
For our remote access metric, we essentially repeated the majority of our ease of use tests, now attempting to complete the various tasks remotely through the internet connection. We set schedules, modified basic and advanced settings, and adjusted the current temperature through the remote interface, and ranked their performance.
Accuracy
The final metric that we assessed — and one that we did not originally think would come into play — was the accuracy of the thermostats. We procured a lab-grade thermometer and compared it to the temperatures read by the thermostats, multiple times and across a large range.
For a more descriptive look at the side-by-side performance of these products, take a glance at our thermostat review, or take a look at our Buying Advice article for a detailed discussion of why you would even want one of these products in the first place.