Reviewing the critics
Join Our Community Subscribe to Paul's PostsA stereo reviewer is an audiophile with knowledge, experience, and the chops to write about it.
They are essential community assets.
Theirs is a tough job. Imagine the challenge of reviewing speakers. It’s hard enough for any of us to get a new pair of speakers and set them up properly. It must be a magnitude more difficult to do this for a review. Get the setup wrong and readers get an unfair evaluation of the speaker.
And then there’s the challenge of passion. A dispassionate clinical review—one that’s not clouded by personal bias—is what most of us think we’re after. To quote Sgt. Friday, “give us just the facts”.
But honestly, how many of us don’t thrill to a reviewer’s passion? It’s actually what I look for. Their level of excitement tells me more about a product’s virtues than any technical description or dispassionate analysis.
I care about how the equipment made them feel.
Because how the equipment makes us feel is what it’s all about anyway.
Having seen pictures and video of the listening rooms of some most famous audio reviewer I stopped trusting their findings. However there are some rare audio magazines which let their reviewers check a new component in a reference set-up placed in a reference listening room with heavy room treatment. Seems to be a more serious approach, imho.
Mike Fremer has a setup in his record library. I design media storage shelves as acoustic devices, and find that LPs make an excellent acoustic treatment; although the best is a mixture of LPs, books and CDs. My shelves also have angled fronts (3°-10°), which are more ergonomic in terms of finding and reaching for the album you want as well as breaking up standing waves.
Mr. Fremer’s brain, OTOH, is so totally grown and programmed for vinyl that it can’t recognize the superiority of DSD. I have to agree with him that a well tuned analog system is better for your health than Redbook, both direct to disc LPs and 30ips master tapes. I had a first year production ATR-102 half track and got my service training certificate from the Ampex factory so it was always in top form, and also got to listen to some multi-track masters on a Stevens 16, the best sounding large format deck in history.
Sometimes the shape of a room, the plants, paintings and walls of LP records can do as much for sound quality as a lot of expensive purpose-built room treatments placed in corners and at first reflection points. Persoally, I enjoy listening in a room that I would enjoy sitting in even if I were not listening to music. A room with a view of the outdoors is a coveted perk, in spite of having to deal with a glass wall…and yes, there are ways to deal with glass walls. 😎
Here’s a video by one of PS Audio’s distributors in Toronto Canada where the employees discuss their best and worst buys. Many of their worst buys were based on reviewer’s accolades that lead to bitter disappointment.
https://www.youtube.com/watch?v=kywBN2zdIfM
Here’s a story where Steve Guttenberg went all the way from his home in Brooklyn to Huntington Long Island by train, a very long journey to hear a highly reviews speaker by someone whose opinion he valued up to that point. What a disappointment it turned out to be.
https://www.youtube.com/watch?v=PCQUgsoTBUc
My experience with audiophiles is that they are very poor as critical listeners. I don’t trust any of their opinions. At least the test results have some limited value even if not much. Whether professional reviewers who get paid or amateurs I invariably disagree with them, the more critical listener I become the further from any credibility they all recede. Not to pick on Paul particularly but just to prove a point which I think can be extrapolated to pretty much all of them in my experience. Once the Hypex amplifier his team built was a super amplifier until he heard the Sonic Frontiers at Arnie Nudel’s house. They Hypex never even made it out of Paul’s car. The infinity IRS V was the greatest thing since sliced bread until Paul heard a live performance of the New York Metropolitan Opera. Suddenly IRS V was canned music, the best efforts of a team probably as good as any in the business. Every month you can get a magazine that will review the best amplifier, speaker, turntable, furniture caster to put under your equipment in the world…. until next month when a better one comes along. There are rave reviews of tiny tyke speakers that are priced in the thousands, far higher than they ought to be. There are some good ones for the money like the Elac Unifi UB5, considered a good value at $500 a pair but what’s the point when you need $5000 worth of electronics to make it produce what it can? No highs, no lows, it’s Harbeth P3ESR for about $2500 but they have great midrange. Sounds like the Quad ESL 57 that everyone raves about, must have, and then gets rid of.
We don’t know what experience these reviewers have, what their hearing acuity is (don’t tell me a guy who can’t hear about 10 khz can make any critical judgments that would convince me to go listen to what he liked) what their reference is which is rarely if ever live music. So I learned from experience with reviewers not to trust them at all. Their reviews are for me nothing more than entertainment. BTW, if something really great at a low price ever comes along the reviewer has to find some faults with it or he won’t have tons of equipment shipped to him for free to review. Here’s an example, a Crown icore 2 amplifier. Here’s a challenge, can you make one of those sound exactly like a very expensive model costing 10, 20, 50 times as much by adjusting the frequency response to match the more expensive one with an equalizer? For many amplifiers slight differences in frequency response is the only audible difference.
There was a pair of speakers that I auditioned a while back that had limited deep bass extension (started rolling off about 32hz), as information was missing during the audition. Right after listening to those speakers, I read a review of those same speakers where it was mentioned how great the speaker’s deep bass was, and how wonderful the sound was because of their deep bass. At first I chuckled, but soon realized that the perspective the reviewer came from was not the perspective that I came from. I was used to hearing one of Arnie Nudell’s creations that was flat down to 20hz, and the reviewer must have had speakers with limited bass response prior to listening to the speakers being reviewed.
And then there’s the challenge of passion. A dispassionate clinical review—one that’s not clouded by personal bias—is what most of us think we’re after. To quote Sgt. Friday, “give us just the facts”.
But honestly, how many of us don’t thrill to a reviewer’s passion?
As far as I’m concerned, reviewers should keep their passion for the bedroom. It turns me off completely.
Fortunately there are a lot of reviewers who are largely dispassionate, and have been at it a long time, such as Jason Kennedy, Alan Sircom and Andrew Everard. Jimmy Hughes is perhaps the best.
Darko really is a master of the art, especially in video format. No one comes close to explaining products with such clarity, and comparing different products and features, and how they work. He is engaging and clearly passionate about what he does, but never gets over-excited and keeps subjectivity to a minimum.
Mr Fremer is relatively new to me and I was very critical of a PS Audio review he did recently, which barely described the product, was internally inconsistent, full of meaningless language and – by far worst of all – placed a mid-price component in what seemed to be a $1million system, making the whole process futile. I was then pointed to his website, Analogue Planet, which is largely dispassionate, highly informative and an enjoyable read. Thank you, Mr Fremer!
Although I think it’s neither as black and white as Paul says nor as you say, I tend to your preference.
Speaking of car reviews, it’s also nice to read marginally how the reviewer felt when driving, but what I mainly want to read to be able to make more concrete conclusions, is rather more detailed and factual. But on the other hand there for sure are also folks who just want to get excited and don’t care much about the rest.
Those audio reviews with mainly flowery words mirroring enthusiasm for almost everything, rarely help the customer comparing, they just sound nice for the manufacturer reading about his own products.
I don’t understand how anyone who goes to live concerts can praise the current state of hi fi audio technology. They sound nothing alike. Many if not most have no knowledge of engineering and no knowledge of music except in the most superficial way. I don’t understand what the goals they have are, what they imagine the purpose of this equipment is, or why they get so enthusiastic about some of it. I also think many of them have a clear conflict of interest. I agree with acuvox. If most or all of what you listen to is recordings of music then you have no reference to go by that gives you something even subjective from memories to compare equipment to except other equipment. Paul is honest enough to admit that the state of the art is far from satisfactory if the purpose is to recreate the sound of real musical performances from recordings. I don’t know if reviewers are dishonest or just don’t know. It hardly matters. If Harry Pearson said one thing I agree with it’s “trust your own ears.” Even that fails me at times.
Some prefer home stereo to live. It doesn’t have to be either or in my opinion. Both can be great but different.
Compartmentalization. People who go to concerts occasionally have their hearing neuro-plasticity warped by audio and other noise pollution of daily life in the Industrial Age. Even professional musicians learn to hear live music and recordings in different neural circuits.
My criterion is listening to live acoustic music more hours per week than audio, from childhood. I was raised acoustically, but had to re-habilitate my ears for years before my acoustic acuity returned after 30 years of audio. This was mostly due to bad source material, as recording quality went to hell after the creative studio wizardry took over after bands like The Moody Blues, Pink Floyd, the Beatles and Jimi Hendrix Experience. I am not against the use of electronics, but when you go beyond what can be played live I draw the line.
I work almost exclusively with conservatory trained musicians, and some of them are impressed with recordings that I find un-listenable. They have totally different standards for playing music in the room and recordings. Most music students today are forced to learn music from YouTube and Spotify due to budget and time constraints, and this damages their hearing. Even Redbook and over-produced pop music is ruining the ears of most conservatory graduates because the entire Classical world is trying to be “relevant” – meaning hybridization with more “accessible” genres. They also have to endure mostly BAD PA systems trying to expand their reach to more venues, like the ill-advised “multi-use” makeover of Geffen Hall.
Besides learning to hear every knob in a recordings studio (and none I like) I came to the conclusion early that there was something I craved in music that was lost when there was no audience. Part of it is having to think both about the whole piece and the moment. When musicians are concerned about “getting a take with no mistakes” by splicing, it degrades the musical content. Further, music is communication and that means not only with each other (no overdubbing, isolation, headphones), but also with dedicated, educated listeners.
I deliberately moved to New York to maximize my concert going, and attended roughly 2500 concerts in the last 20 years. This is the greatest concentration of musicians both resident and itinerant, the greatest concentration of world class venues and the most sophisticated audience in the world. I further had to stop listening to studio recordings.
For me live music at its best has a wonderful sound that I find extremely pleasing. By contrast hi fi sound lacks many of the qualities I enjoy so much from live music. This is why it was a challenge to see what I could do to close the gap or at least narrow it. This was both an intellectual challenge and a way to develop and appreciate my perception of sound and music. Not only are those missing qualities so important for me but I’ve become acutely aware of even minor differences in them as well as the sound inherent in the recordings themselves. This meant that I had to design a system that would allow for a variety and range of adjustments that could get sound from each recording the way I want it to sound and like it most. Does that make me an objectivist or subjectivist? I think both.
I am very impressed with your dedication to music performance and the insights you have gained as a result. Thanks for sharing. I listen to live and studio recordings. I perfer recordings that are of a performance. Somehow I think they are more honest. Around 1979 I bought a Nakamichi 550 and Nakamichi microphones. I know some people will think this is almost primitive by today’s equipment but the first thing I noticed was the better fidelity I could achieve making my own recordings compared to what was available on vinyl records. The next thing I noticed is there is no substitute for the real thing, live performance. I record and listen mostly to jazz and folk music. I don’t record classical music but I would enjoy the opportunity.
Live music is the best. I used to listen to Jazz and Folk, but you can’t anymore without a club PA system with stage monitors, artificial reverb and a sound guy with blown out ears.
Cassettes are too convenient – I used to haul reel-to-reel recorders around, an Otari half track and Sony TC-854 4 track; and then in 1978 I got a truck with an MM1200-24 and ATR-102. The ATR at 30ips blows away Redbook.
Still, the sound of live is more important than audiophile niceties. I have sold off most of my “high end” studio gear, including DTAR, Grace, and Millennia preamps; Schoeps, Neumann, Earthworks mics and Beyerdynamic M130 ribbons; Meyer Sound CP-10 minimum phase parametric equalizer, etc.
I stuck to the most portable gear: a Korg MR-1000 DSF recorder with a pair of AKG C480B, a Zoom F4 for my Rode NT-SF1 Ambisonic mic and a pair of TASCAM DR-680 Mkii for battery powered 12 track. They synch through the S/PDIF ports and have individual track outputs for my OVOMOS audio systems (one voice, one mic, one speaker).
I have a variety of prosumer mics: AKG P170, Shure SM80/81/89, Beyerdynamic M58, CAD M179, CAD D82. I am short on “vocal” mics, with only Beyerdynamic M380 and TG-X50; and one token SM58 (the vocal mic for beatboxers and vocalists who can’t sing). The only expensive mic I kept was a Royer R122, because velocity mics are better for wind instruments (including TRAINED singers). This is also the best Cello/Contrabass mic. I also use Countryman high end flat omni lavalier mics for close miking string instruments. They beat the DPA 4099 in listening tests.
I also have some old Sony ECM-22P mics, which were the best acoustic steel string guitar mic ever, but the electrets all faded over time. No use getting NOS capsules, unfortunately. Have you noticed loss of gain and SNR in your Nak mics?
In my days of making purchases of new audio equipment there were several ways I went about it. If I read a review of a particular component that I was in the market for, I then hunted for the nearest Audio Salon that carried that brand and brought my own music with me. At least this process gave me some sense of the sound of the component because I asked the demonstrator to swap different ancillary components in an out while I was listening to be able to get some idea of the sound of the unit that I was auditioning.
The same scenario could go in reverse where I just strolled into an Audio Salon and was given a demo of a component to listen to and On rare occasion I bought it on the spot because it knocked me out or I went home and started to look for articles and reviews on that particular component.
There’s so many ways of making your own choices based on reviews but just basing your purchase on a review is not the best way to go in my opinion.
When Steve Gutenberg reviews headphones, which he does quite often, so much of the time he starts off with a Schitt Vali 2 Headphone amp and then switches to the PASS Headphone amplifier which is one hell of a jump. That’s why it’s critical for the listener to go out and try to get a demo and hopefully bring it home for a trial period. It’s better to spend some real time with a component without overdoing it to get a sense of the musicality that it presents.
There is always the approach of Sargeant Shultz, “I know nothing!”
Or the infamous Professor Irwin Corey (RIP)…“I forgot what the hell I was sayin https://images.findagrave.com/photos250/photos/2017/37/176094072_1486477283.jpg g“
https://images.findagrave.com/photos250/photos/2017/37/176094072_1486477283.jpg
stimpy2,
You’ll notice the PM likes to throw in “I forgot what the hell I was sayin” on some of his YT presentations 🙂
No I didn’t.
Great observation
Axpona is sponsoring a webinar tomorrow, Wednesday, Feb. 10th at 7:00 pm EST titled “Reviewers on Reviewing” featuring:
Panel Members:
Steve Guttenberg, The Audiophiliac Daily Show
Maurice Jeffries, Enjoy the Music, Positive Feedback
Nan Pincus, Part-Time Audiophile
Greg Weaver, the audio analyst (c), Enjoy the Music, Positive Feedback
BYOB
You can still go here to register: https://www.axpona.com/webinar.asp
The last sentence says it all. Get this point and one’s life is headed for all things good. Miss it and one’s life is doomed to be spent in the shallows as the Great Bard said. As for reviewers, no amount of praise is enough whether one agrees with them or not, whether they are concise and to the point or long winded, whether they are neutral or obviously biased, it really does not matter. If it gets boring as sometimes it does go to something else. The same boring article could be of great interest to someone else of course. The amount of information the reviewers bring to the readers would be impossible for someone to get on his own. In a way the reviewers are the unsung heroes and heroines of audio. I have bought a few things just based on a review and expensive at that but never regretted it. Of course the reviewers were known to be to the point in their reviews and they always seemed to run out of words when describing these products, the products being so good. Up went the antenna and with it the order and I am still using them With nary a thought of replacing them. Audio should have a reviewer’s day. One point to keep in mind. If a reviewer goes on and on then it is time to be careful. Regards.
“I care about how the equipment made them feel.
Because how the equipment makes us feel is what it’s all about anyway.”
I TOTALLY AGREE – except it’s the music that creates the feeling. Unless the audio enhances the music, or at least doesn’t reduce the affect in any way, it can’t claim accuracy. My motto is “All speakers suck. My speakers suck less.”
And that is why I kept going back further and further in time until before there was audio reproduction to find the answer. No reproduction ever gave me the feeling of a live concert. One of the steps in my trajectory was designing, building and operating a 24 track mobile recording studio, where I was listening to a live mix or solo mic off concert stages through the most accurate truck monitor system of the time, built out of pro components but using audiophile methods: bi-amping high efficiency speakers with huge high definition power and state of the art acoustic room design.
I found that the energy was there, I could hear the crowd response in every mic. The headroom produced realistic levels with no distortion There was an element of minimalism because we ran studio grade mics (Neumanns, AKGs, Sennheisers) to the multi-track with no processing. Our rack gear was minimal – a few compressors and a Furman RV-2 stereo reverb.
BUT, the clarity was severely limited by the stage sound, with floor wedges everywhere bleeding and the off-axis FoH bouncing onto the mics. So recording studios were the wrong place for musical capture, but so are amplified stages. Acoustic stages – well, that problem had been mainly solved in the 1950s with various Near Coincident Pair techniques: ORTF, NOP, RCA, Jecklin and variants. At the 1978 AES Convention I heard a demonstration of Ambisonics by Michael Gerzon et al, so I believed audio had a looming high tech 3D future.
So I worked on other problems from 1980 to 1997: water recycling, aeroponic greenhouses, Moore’s Law, breaking the 640K boundary, anti-viral software, pharmaceutical manufacturing, etc. In 2001 I got the amazing revelation from Dr. Manfred Schroeder that the human pinnae are directional PHASE encoders – which meant that 95% of what I knew about audio was wrong. Speakers have to be flat phase at all angles, mixing is distortion, and stereo is fake!
I started over from scratch, with the first postulate being NO MIXING. All audio systems have to be one microphone to one speaker, with separate tracks played back one speaker per track. Speaker accuracy had to include not only timbral accuracy (frequency response), but also temporal, transient and spatial accuracy. The latter has only been addressed partially by a handful of designers, including Linkwitz, Lyngdal and Putzeys.
My novel audio system solution proved out by passing the two most important criteria of all:
1. Conservatory trained musicians can’t tell when we were amplifying acoustic concerts – 95% of string players in the audience, and often even the performers couldn’t tell the difference between acoustic and speaker augmentation when the speakers were 10dB louder on stage. Not even people accustomed to critical listening going back and forth could tell, like a four time Classical Grammy winning engineer and a veteran NY Times concert reviewer.
2. In our concert series over 10% of events produce mass euphoria – goose bumps, giggling and remarkable feeling of belonging by all, and extreme avoidance behavior for leaving. It was like drinking Dom Perignon, or being in the movie “Babette’s Feast”. People left behind coats, purses, wallets, keys, cameras, laptops, even their musical instruments almost every night. They got ready to leave and then circled back like in Bunuel’s “Angel of Death”, said good byes repeatedly only to start new conversations. It was hard to close the venue an hour after the music stopped.
There has been no better feeling from audio, ever.
Did you serve Cailles en Sarcophage at these concerts?
Very interesting replies. How we each weigh the review factors will vary, but I don’t disagree or want to dispute with any of them. I have been involved in electronic performance equipment for a long time. In my mind Audiophile is just a ‘cult’ Label. Only a few, of all the listeners out in the buying world, care to label themselves this way.
I don’t make a living selling audio equipment any more. When it comes to a performance, it boils down to what the listener wants, can afford and likes to listen to. I have been with two people in the same room, at the same time, who state their dramatically different opinions of what they just heard. I have seen these polar opinions with live musical and recorded music.
We all own what we can afford, and like to listen to. That applies to the equipment and the recording. I use reviews as just an indicator of how the equipment operates. In the end I own the equipment I like to hear and will be dependable. If I can’t return it after I have tried in my home, I don’t audition it. When I sit to really listen to a recording, to fall into illusion, I prefer a vinyl record on my stereo. If I am using the music to fill in a void, when doing my work or various activities around my home, a CD, my iPod or even the radio, is just fine. What I use to watch and hear a movie is different from what I use to listen to music. Both occur in the same room. My wife doesn’t seem to notice any difference.
I have learned that in the end it’s just a personal opinion. A customer will buy what they like and will feel comfortable with. It may not be what they want but it is all they can afford. I have sold things to customers, what they really want, even though I would never buy it myself.
We live in a democracy, we have the right to express our opinions. But we should remember, that it may be the only real fact, is that the statement is just an opinion.
Wow, how do I feel about audio reviewers? When I first started reading audio reviews things like TAS and Stereophile did not exist yet. HiFi News and Record Review did exist, but I hadn’t heard of it yet. I agree that reviewers are an essential part of the audio community, but there are good reviewers and there are bad reviewers and you have to be careful that you know the difference between them. To this end I firmly believe that if you are about to buy a piece of audio gear based on reviews only and you have to ask yourself should I really spend this money on that gear, the answer is NO. For example if you buy a $50 turntable mat based on a review and it turns out to be a load of rubbish you will probably be able to chalk it up as a lesson learned about that reviewer, but if you buy a new phono preamp for $5000 based on reviews only and it turns out to be a load of rubbish you better be sure you have a 30 day return window for a full refund (less shipping).
If you have read reviews as I have for almost 50 years you might be able to have a short list of reviewers that really trust. My short list is three reviewers; one at TAS and two at Stereophile. I have never met the reviewer at TAS, but after reading his review of a turntable and later speakers that he decided to put into his own system I auditioned that same gear and also decided to put into my system. The fact that this reviewer’s audio “taste” are much the same as mine makes me trust him. In the case of the two at Stereophile I have interacted with both of them and they have each given me excellent audio advice that has led to purchases that a really enjoy. Thus, I trust them. As always YMMV.
Back in the late 80’s, I acquired my first system but it was in my car! I stopped in to the car stereo store in Davis, CA one Saturday and talked to a very knowledgeable dude who informed me that the way to do it was to start with the cleanest source possible, so I picked up a “reference series” Alpine CD player. Then, I moved on to separate Alpine tweeters (mounted at ear level) and mids. A ten inch bass tube went into the rear area of my little hatchback. Each set of components had their own amplifier and every piece got passive crossovers. When they were done with the installation, dude pulled out a Stevie Ray Vaughan CD and he selected a song and played it. Then he played it a second time and said, “Listen. You can hear the hum from his tube amp!” I was floored, and afterward there were certain songs on certain CDs that actually made me emotional. For the first time, music made me FEEL something! Of course, someone wanted my stereo more than me, and one day it was stolen and I was left vacuuming broken glass out of my car. I was heartbroken and never had anything that sounded like that again.
But recently, I’ve been picking up used home audio gear here and there in an effort to replicate that clean, powerful sound, and as I do, I use my memory of that setup as my goal. To me, those memories and those feelings are what it’s all about.
I think it’s the same with restaurant reviews. My wife would want to go to every place recommended by Giles Coren. I think she also wanted to have his babies. I doubt that’s an issue audio reviewers ever face.
Hilarious!
OFFTOPIC
Dear Fat Rat
ICYMI: WHO has issued statement that it is “extremely unlikely” Covid virus escaped from Chinese research laboratories according to BBC news online and also the virus is present in many bats found in Asia
I do enjoy your often humorous comments on this forum
Taiye
Taiye,
Thank you.
I’m sure that many of us here do try to get the ‘mix’ of humour, information & entertainment right with our comments & replies.
‘rwwear’ has pretty much nailed my response.
If the CCP (Chinese government) is not held responsible, at some point in time, for it’s (I just want to use one descriptive word here to encapsulate the half a dozen more accurate ones) BULLSH!T, thereby allowing this CoViD-19 virus to cause so much global destruction & human suffering, then I see no reason why we shouldn’t let all of those individuals in every country in the world that have committed manslaughter a free pass to leave prison & rejoin society immediately.
taiye315 that doesn’t absolve China for the cover-up and the WHO helping them with the obfuscation.
Unless you know an audio reviewer’s ‘hearing’, assessment(s) & opinions intimately, his/her reviews are only going to be, at best, to help you to consider a component or loudspeakers for inclusion in your shortlist for upcoming audio purchases, & at worst, just reading/listening for pure entertainment value…IMNSHO.
It’s always interesting how reviewers always proclaim the next version of a piece of gear vastly superior to the previous version, that only a couple of years ago they proclaimed the best they ever heard. In user forums one often finds a large number of users who prefer the sound of the previous version over the new. I wonder how much a reviewer’s opinion is influenced by the expectation that the new version will sound better and a felt obligation to the manufacturer to promote the new version. No doubt they hear sound differences between the old and new versions, but it would be refreshing if just once the reviewer would conclude that the old model that was cheaper sounded better!
I especially enjoy reading the reviews of Stereophile and the Absolute Sound but I read them all including just regular people on Audio blogs.