In an era when electronic devices are becoming smaller and more geared to the individual, Bluetooth speakers aim to emancipate the music from our personal devices and share it with those around us. We based our testing on this common scenario: there is an impromptu gathering of friends somewhere outside, maybe a park, maybe a beach. After a bit of chatting you realize this soiree would be much better if it had a soundtrack. In this situation, which speaker would sound best? Which would you have most likely tossed into your bag just in case? Which would be loud enough? Which would survive if it suddenly started raining? Which would be most likely to run out of battery?
To answer these questions we divided our testing into four metrics: sound quality, portability, volume, and battery life.
Sound Quality
The first question you're most likely to ask of any speaker is, "How good does it sound?" to evaluate this we created a sound testing pedestal, on which we placed each of our speakers. This ensured consistent acoustics, as each speaker was in the exact same spot in the same room when we listened to them. We then had all of our testers listen to the same songs played at the same volume on each speaker, one after another. Each tester then rated each speaker based on its bass and treble quality, clarity, and dynamic range. Our lead testers were two life long musicians. One was a guitarist with a good sense of melodic instruments. The other was a percussionist more attuned to the bass end of the spectrum that could tell a lot about each speaker based on how they made snare drums sound.
Portability
We assessed portability via three different avenues: weight, packability (size/shape), and water resistance. Our reasoning was that the lighter, more easily packable, and better able to withstand the elements a speaker is, the more likely you'll be to take it with and actually use it. We weighed all of the speakers ourselves on the same scale to ensure an accurate and comparable measurement. To grade packability we noted the size and shape of each speaker and had our testers shove them into all sorts of different backpacks, duffels, and general bags of varying degrees of fullness. We also toted them along on normal trips to the beach, park, and our favorite sunset cocktail locations. We then rated each model based on how easily they fit into the bags and with the rest of the stuff we'd be carrying around anyway.
We also gave portability bonus points to those models that had some sort of water resistance rating. Of course we verified these water-resistant claims before awarding those bonus points. Somewhat unfortunately, all of the models lived up to their various water resistant ratings (we were kind of hoping to see one go up in smoke). For IPX4 and IPX5 rated models (essentially those officially rated as splash proof) we ran the speakers under a faucet for 5 minutes, tipping them around so that every surface was exposed to water. For IPX7 rated products (those rated to be completely submersible) we repeatedly dunked them into containers of water, and rivers, and lakes, really any body of water we happened to be near.
Volume
We began testing volume using a sound meter to ensure the maximum output of each speaker. However, every speaker recorded very similar maximum decibel levels. Not only that, some speakers sounded terrible when turned up loud, and even though all the decibels levels were similar the speakers didn't seem to fill rooms with sound to the same degree. So we switched to a more subjective test where we took all of the speakers to an average sized apartment, turned them up as high as they would go before sound quality started to degrade, and evaluated to what degree they were able to fill the apartment with sound, and whether ambient noise like talking could drown them out.
Battery life
To test battery life we first found every person we could that owned a Bluetooth speaker and asked them what kind of things they listened to on them, and in what ration. We also had all of our testers tally the type of listening they had been doing when taking the speakers home for real-world testing. We found that an average user listens to about 80% music and 20% podcasts or talk radio style shows. We then made an hour-long playlist that was 80% music and 20% podcasts to mimic this real-world use. After charging each speaker fully we played this playlist on a loop on each model via a Bluetooth connection. We standardized the volume at 75% of the maximum. We then let that loop play and play until each speaker died, noting the exact time that they kicked the bucket.
