Disclaimer: I am one of them—a consumer electronics reviewer.

Most of us are not engineers. I have a background in IT, which gives me at least a basic understanding of the principles of programming and networking, as well as a cursory knowledge of electronics, but mostly I’m just a guy who has played with a lot of gear. And I’m one of the nerdier mainstream gadget reviewers; many of the people working for general interest publications don’t have a technical background at all.

That’s not to say we shouldn’t be trusted on the technical details—a journalist’s job isn’t to be an expert, but to filter correct information from experts to the reader—but don’t put us on any sort of pedestal. Every hardware reviewer has made their fair share of boneheaded mistakes.

We’re just as susceptible to industry hype as the next guy—sometimes more so. Hardware reviewers have to soak in a lot of information, much of which comes direct from the public relations mouths of the companies providing the products. It’s far easier to let our guard down and regurgitate the bullet points of whiz-bangery than it is to maintain a healthy skepticism, because sometimes—okay, most of the time—we’re really fucking lazy. Plus, we’re all drunks, and the PR people buy us lots of drinks.

We have a skewed perspective. While I don’t work as a full-time reviewer anymore, there was a point in time when my office was filled from top to bottom with boxes of nearly identical gear. Trust me, after you’ve seen a dozen models of a new type of gadget, it becomes difficult to remember why the gadget might be important in the first place.

We work backwards from a mythical feature checklist. We might ding an HDTV because it lacks a certain feature that won’t make a whit of difference to the average consumer.

Take a VGA port on an HDTV, for example. I know why I want one—because I might want to hook up a PC or an Xbox 360—but most people won’t even know it’s missing. A good review will properly disclaim the level of importance of any criticism, but it’s far too easy to take that knowledge for granted. It’s far easier to write for either an expert or casual audience seperately than it is to write for both in the same article.

We spend very little time with each product. From the time I open the UPS box to the time I write the review, actual hands-on time with the product is probably just a few hours. Hidden interface hangnails may not have had time to make themselves known; Overall reliability has been impossible to determine. Plus, I’ve got a deadline to meet. If a company can make a product with a slick out-of-the-box experience, it’s likely to heavily skew the whole review. (To be fair, an easy first hour for a reviewer does mean an easy first hour for you, too.)

We don’t always review final hardware. For obvious reasons, this can make a big difference. Magazines have it even worse than online publications—the editorial copy often needs to be written 90 days or more before the product actually launches. Case in point: The time Sony sent me one of the very first review units of their eBook “Reader” product, sans software, making it impossible for me to test the one thing at which Sony has been historically bush league: the interface between their products and a PC. When I asked for a pre-release copy of the software, they offered to send me screenshots and give me a walkthrough. Confidence: Inspired! (For the record, I sent it back and told them to call me when they had a working package.)

I’ve just slagged the very type of writing from which I make a living! You’ll never trust me again! Watch me backpedal furiously with counter-argument after the jump.

You aren’t an engineer, either. (Probably.) As an average Joe, it’s daunting to try to parse an overly technical review by a power-geek filled with numbers and statistics. A good hardware reviewer has come full circle, past the obsession with raw numbers to return to the crux of any electronics purchase: Will this piece of hardware get the job done in as painless a fashion as possible?

We’ve become inured to hype. Every day I get email pitches from companies pushing new gadgets peppered with words like “revolutionary” and “a new way to.” I can barely read what they’re trying to tell me about before my fingers claw my mouse up to the delete button. We’re not as easily led to believe some new widget will be the panacea to all that ails the world of gadgetry, and by proxy we can filter that incredulousness through to you. (Note: This contradicts something I said earlier; the mysterious part is that they’re both true.)

We have a broad perspective. Because we see so many products throughout the year with nearly identical feature sets—often made by the same Chinese manufacturer and rebranded by half-a-dozen corporations—it takes something really special to make us sit up and take notice. If we’re flipping out over something, you can be reasonably sure it’s because it’s one of those rare devices that reenergizes our passion for keen gear. If someone as jaded as a hardware reviewer is excited, take notice.

We don’t need a lot of time with every gadget. Because we’ve played with so many of them, we know which features (or lack thereof) will make a difference in the long run. We can spot those failings often before we even put our hands on the product.

Over time, many of us become experts. I’m not there yet, but I’ve met many people over the last few years who rest soundly in the halls of proficiency. Loyd Case and Jason Cross at ExtremeTech, for instance—I’m constantly shocked by the science they drop off-hand. Many of the writers at Ars Technica. Walt “So Modest I Drive the Shitty Mercedes” Mossberg, depending on what he’s reviewing.

That’s really the crux of this precariously balanced warning: Hardware reviewers have more in common with movie critics and others who write about seemingly more subjective topics that you’d think, building a history of trust and shared perspective with their readers. Since a long-term relationship with a reviewer is unlikely when you just want the first site you Googled to tell you if an HDTV sucks or not—just let me know if it’s an eight-or-better out of ten, damn it!—approach any given review with all due incredulity.

I guess what I’m trying to say is that you’re pretty much screwed no matter what. So unclench—and happy shopping!


3 Responses to “Buying Electronics: Eleven Things to Remember When Reading a Review”

  1. 1 likewaterbaby

    Thanks for that Joel- the fact that you have some self-awareness of the pitfalls of reviewing inspires a bit of confidence in your recs…and that kind of article is a big part of what I like about Dethroner.

    You lazy drunk bastard you.

  2. 2 BananaRocket

    Can someone review this artical and tell me if it’s 8 out of 10 or higher? hehe.

    Thanks for the insight in to the deep dark depths of a tech reviewer.

  3. 3 Carlos

    Joel, do you know of a TV/electronics review site roughly equivalent to (or at least attempts to approach the quality of) dpreview.com in terms of review quality and thoroughness?

Leave a Reply







Close
E-mail It