Baselines

October 11, 2016
 by Paul McGowan

Room after room at the audio show featured A/B demonstrations of one thing or another: footers, cables, MQA, clamps, tweaks and whatnot. In most cases, attendees were wowed with obvious improvements. That said, there is a problem making judgements based on these comparisons.

The problem is one of baseline. You can only judge whether A is better than B or C, which gives you no reference to the absolute.

If I demonstrated a pair of speakers with a tweeter I could switch on or off, you’d likely decide that switch improved the speaker’s performance. And, indeed, that’s exactly right. But, so what? Going from compromised to better isn’t of value to the listener.

While I can’t think of one manufacturer at the show that made A/B comparisons without the best of intentions, we must pity the poor listener. Without a proper baseline reference, like your home system, of what true value does such an A/B test offer?

The late HP did his best to simplify the baseline beyond the familiar home system (itself flawed because not that many home systems are accurate references). HP’s baseline was simple: acoustic music performed live, which he called “the absolute sound”.

And this describes the ultimate challenge for each of us: learning to develop our personal baselines.

Subscribe to Paul's Posts

34 comments on “Baselines”

  1. I don’t feel sorry for listeners at audio shows at all. If they think they can make an informed A/B test decision in a noisy reverberating hotel room they are seriously misguided, naive and/or misinformed. It took me one visit to an audio show in 2013 to work that out. I went back the following year (it was 45 minutes away) to say hello to a few people, but did not attempt to listen to any systems with any seriousness.

    The classic A/B tests were carried out by Briggs (Wharfedale) and Walker (Quad) in public demonstrations in 1955 and 1956 in the UK and the USA. Live performances and recordings were performed side by side in concert halls. In London the newly built Royal Festival Hall (built in 1951 for the Festival of Britain), with almost 3,000 seats, was used. It has a very pleasing acoustic, quite ‘live’ without being harsh, although the seating is uncomfortable. These tests were written up at great length and are available to Gramophone subscribers. They make interesting reading.

    I would suggest the only A/B test worth its salt at an audio show is a live voice against the same voice recorded. There are issues with this, including the microphone used, the recording environment and the familiarity and gender of the voice. However, it is easy to do, and requires little more than one speaker in the middle of the room and a chair. (Things like imaging are secondary to basic sonic performance and I often think stereo is over-rated. When I go to concerts I don’t hear stereo and the music usually comes from a very narrow arc of say 10 degrees, much less for a duet or soloist.) It is also primarily a loudspeaker test. I am reliably informed that most speakers fail this test, which may account for it being avoided.

    My final gripe is the choice of sound used for demonstrations. Anything amplified is pointless. For me the best test is a BBC radio recording of an actor or actress whom I am very familiar with, preferably from many live performances. Judi Dench and Ian McKellen. would be my top picks. After that, piano. However, at the Harbeth M40.2 launch demo (did anyone hear them at RM?) six different piano recordings were played consecutively, clearly demonstrating increasing recording distance from the piano and increasing room interaction. The recording mix is of course a matter of personal choice, sometimes by the recording artist, sometimes the producer, hopefully a bit of both. The point being, it just illustrates the problems in evaluating based on media choice, even when only one instrument is involved.

    1. Steven, I find your statement interesting, that when you go to concerts you don’t hear stereo, and the music comes through only a small arc. I suppose that if stereo is defined as the sound coming from two sources with the expectation that your hearing will reconstruct the original sound field, you make a good point. What I hear at a concert, and I tested this recently, is a panorama of music including the many reflections from all over. True the direct sound of music is through a rather narrow arc, but without the reflections the music loses a lot of life.
      I once had the opportunity to hear a small group perform in an anechoic chamber. Now, that was a rather unnerving experience. Temporarily I lost my sense of direction: with my eyes closed, I couldn’t tell where the performers were. Apparently my brain had to recalibrate, so dependent it was on reflections to help pinpoint direction.

  2. One problem still unsolved remains: in most cases we have no information about the recording and manipulations made by the sound engineer who finally has created a product according his individual taste – having listened to the mix via a system unknown to us. If you visit an ophthalmologist for checking you eye-sight it is most easy to judge better or sharper when the diopter is changed because the reference (characters and numbers) are known. Even the fonts used do not matter. At least a reference could be a known instrument played in the room at the spot of the speaker compared with a non-manipulated recording of the instrument! If a violin is played and the speaker seems to play a viola something must be wrong. But which component in the chain is primarily responsable for the shift in timbre? Having finally a real good system many recordings might sound unacceptable! Thus we stay fooling ourselves with “good sounding” systems based on a good sound produced by the artisan sound engineer.

    1. Mixing is distortion.

      Every microphone location has a different perspective on the acoustics of the room it is in. They all different distances from the walls, floor, ceiling and sound sources and capture an unique, identifiable pattern of reflections. Combining two or more microphones into one speaker (mixing) and splitting one microphone into two or more speakers (panning) produce an acoustic chimera, which is SPATIAL DISTORTION.

      Likewise, every other knob in a capture, mixdown or mastering studio distorts the spatial cues in the music. The best we can hope for is keeping all audio chains strictly one microphone to one speaker, with zero processing in between other than channel volume.

      Modern engineers attempt to get around this by recording in isolated, ultra-dead environments so there is no room ambience in the tracks and then applying pan pots and “stereo reverb” to create a bad illusion of acoustic space in the recording. This not only interferes with the sound quality in a way that I find unlistenable, it also interferes with the quality of performance by requiring the musicians to wear headphones and creating a disconnect between the ambience during recording and the ambience in playback.

      Bottom line: STUDIOS ARE THE WRONG WAY TO RECORD MUSIC.

      p.s. I can also hear the INTENTION to splice and overdub, so don’t imagine it can be done transparently.

  3. As an aside, I would even question if the live performance is the baseline. In piano concertos we are used to hearing the piano booming out over the orchestra, even with a 100 piece band for Rachmaninov. Not so 200 years ago. I go to quite a lot of performances that are live radio broadcast by the BBC and then available for the next month. I went to one a few weeks ago, the Freiburger Baroque Orchestra with Kristian Bezeidenhout, who directs from an 1804 pianoforte, the type of instrument that would have delighted Beethoven. He plays it frequently. I heard a complete Mozart violin sonata cycle with it (and Alina Ibragimova), which as superb. However, even with the 20-piece Freiberg band, it was drowned out at Row 10. It was close mic’ed for the radio recording, which I found musically more pleasing.

    1. The principles of balancing have been the same since we started making musical instruments 35,000 years ago. If this concert was unbalanced, it is likely because:

      (A) the FBO has grown out of all proportion to Baroque era ensembles. I saw them perform two operas in concert in August and they had around 45 musicians. To balance a fortepiano you need one or two players per part, for a total of five to fourteen or so, like The English Concert, Venice Baroque, Rebel Baroque, Quicksilver, L’Arpeggiata, Pomo D’Oro, etc.

      (B) Baroque chamber halls seated well under 200 people. The science and art of acoustics have created good chamber halls with 400 seats (Guilder-Lehrman) and even 600 at Zankel Hall. I have heard Bezuidenhout perform on Fortepiano in both, and they are radically better than the steel framed behemoths where the reverb is louder than the direct sound by the third row, like Carnegie’s Stern Auditorium, Lincoln Center’s Geffen Hall and the Alice Tully Starr Auditorium.

      We stopped following Bezuidenhout when he moved up to the larger venues of New York and he is one of my favorite Early Music keyboardists.

      1. The FBO were about 20 – 9 violin, 3 viola, 2 cello, bass, 2 horns, 2 or 3 woodwind. It was a tight squeeze, for whatever reason they played standing up throughout. The Wigmore Hall holds about 450 downstairs, 100 up. It would not comply with modern acoustic theory but is revered by audiences and the world’s leading artists, and is favoured for official first recitals. There is an interesting report on the acoustics, when they were thinking of some changes, including raising the floor at the rear (thankfully not done). See:

        http://www.kahle.be/articles/Wulfrank_Wigmore_Hall_2006.pdf

    2. For cases where the orchestration does not balance acoustically, I have developed a series of speakers for live amplification that are indistinguishable from traditional instruments by conservatory trained players. Unfortunately PA speakers have given amplification such a bad reputation (and deservedly so!) that the halls and audiences won’t let me demonstrate my work at traditional Classical venues.

      This would make it practical for a fortepiano, harpsichord or classical guitar concerti to be played with even a modern orchestra.

      Instead, I hear conductors and producers continue to use inferior audio gear and continue the audience hatred of speakers, most recently the original Ondes Martinot speakers used for the “Turangalila Symphony” at Carnegie Hall. They have a breakup mode in the upper register which sharply decreased the radiation angle, so the musician seated behind the speaker increased the volume as it beamed at higher intensity where I was seated in front of the speaker. It drowned out the monster Simon Bolivar Orchestra and was so annoying I have no doubt it caused complaints.

  4. Once upon a time when I was an audiophile live music was one thing and recordings were another. All I needed to do was enjoy what I heard from either of them. I have to admit I wasn’t a very discriminating listener. I drooled over equipment I couldn’t afford and believed the people who said their equipment would get better and better until one day it would sound just like live music. I accepted all of that. I was away out of the US for a couple of years, had no access to audio equipment, and when I came back I could hardly wait to experience quadraphonic sound, the great leap forward. I set up my own experiments. What a dud. And what others produced was no better than my efforts. I began to wonder if a recording could ever sound like live music. I didn’t think much about it but unknown to my conscious mind my subconscious mind went into overdrive working on the problem using all of my then sharp analytical tools. And then maybe a week later it got the answer and presented it to my conscious mind. It was a startling moment, one of the most memorable in my life. Then all of my experiences hearing live music came flooding back. I wanted to be able to hear space. The understanding this “aha moment” when the lightbulb went on in my brain showed me the correct answer and full blown, its complexity was mind numbing. Building the ultimate machine was out of the question. Could the idea be simplified, compromised, or as I thought of it “bastardized” and still get anything like what I wanted? I doubted it but I gave it a try anyway. It took two years before I found out. The surprising answer was yes, it is possible but I’d have to throw away all my old audiophile ideas. That didn’t phase me in the least. It was my best effort at the time.

    Around a dozen years later when I had no place suitable to install this experiment (the room itself was critical to this engineered approach which was different from its ultimate potential which would require a very large anechoic chamber) I began to think about the tonality of musical instruments. Now I had what I thought were some pretty good speakers and a graphic equalizer. But try as I might, those speakers never came close to the sound produced by real musical instruments. Not only was the tone always wrong, it was always music in a box, canned music. I looked at what other people were doing that I thought gave better results if not perfect and began to try out their ideas only to extremes. Things got better. And then one day I realized that what I was doing was applying the same theory, the same equations I’d used before in a different way. Suddenly the problem was not just to optimize the approach but to integrate it with the other idea. Now the game was afoot. Now for the first time I had a fighting chance. Powerful new digital processors became available but only one met my needs. I bought that one and paid nearly full price for it but I didn’t have a space to use it in until 2000. In fact I didn’t even try it until 2002. Results were fair but there was something wrong. Around 2008 I figured out what it was and how to fix it. It required more digital processing to fix inherent poor design choices locked into the design of the original DSP. And that is how I arrived at where I am now. I was satisfied I’d met my goal, achieved my baseline. And how did what I was able to do to satisfy my baseline compare to what other peoples’ efforts achieved. Well, I’m not always polite so I’ll just say by comparison their efforts no matter how extreme give results that to me are laughable. But then they had no lightbulb go off in their heads. How could they, their brains didn’t have the electrical power to turn one on. Sorry all the rest of you. 🙂

    1. Ancient seafarers knew very early that earth wasn’t flat at all. Maybe ancient tribes living on higher altitudes also had that knowledge. However the majority had a strong belief in flat earth theory for many centuries. Thus why should believers in stereo theory change their minds within some decades?!

      1. Eratosthenses knew the earth was round and calculated the circumference to within 40 miles of the current known distance, and that was around 200 B.C.

        There is no limit to man’s ability to forget simple truths or ignore them for theological or commercial gain. Man can also invent non-truths for the same reason; MQA comes to mind.

    2. I don’t know what your rig is, but it will not do what a good set of pro headphones will do ( forget the expensive audiophile headphones) Not even close. And I’m not picking on you. No hifi in a living room will do all the things a good set of headphones will do.

      Unfortunately, headphones only mirror the signal. They do not generate an illusion of people and instruments playing sort-a in front of you the way we normally listen to music. I find them to be ultimately a let down.

      And for my generation, listening to headphones is kind of anti social. Not so much for my kids.

      I guess binaural recordings will do it all but I’ve never heard any.

      The closest I’ve heard to live, was a stack of giant 9 foot tall soundlabs ( maybe 8 of them) in a giant ballroom. Playing one-off DSD recordings. Sadly I don’t have any of those or a ballroom.

      1. Ralph Glasgal’s “home system” is a circle of 18 SoundLabs in a room that is 40′ x 40′ x 50′. Each one has a DSP processor generating a simulated room ambience from each direction based on measurements of different concert halls. This worked really well for some symphonic recordings.

        I have heard similar quality of sound from headphones. It was an ambisonic recording demonstration by Michael Gerzon (4 track from tetrahedral microphone array) played back through a custom processor that converted to binaural. The intermediate stages of processing included knobs for effective microphone pattern, angle, distance and height above the stage. It was like flying!

        At the time I was listening a lot to my Koss PRO-4AA, so I was acclimated to headphones, and it sounded more real than anything else to date. Enough time and different systems elapsed between then (1978) and my first binaural recording experience (1982) that I can’t make a good comparison.

        1. I heard Ray Kimber’s demo using his odd ball mic arrangement that captures the rear ambience for 4 channel replay. He has pictures here http://www.isomike.com/ASMF/

          Sadly when I asked for the rear channels to be turned off, the magic simply disappeared. We will never solve this puzzle as long as we have 2 channel thinking dominate the industry.

  5. I have posted it before that my goal is to recreate what I hear live. To me, anything else is trying to create a cool “effect” that you will forever elusively try to grasp.

    I like audio shows. I wanted to make this years RMAF, but had conflicts. You have to take it for what it is. There may not be the best of setup options, but you get more than the general flavor of sound. It’s particularly helpful if you are shopping for speakers.

  6. Those of us who love the electric guitar know that it is but a paintbrush accompanied by electronic modulators that can create audible colors far beyond the limitations of the rainbow. Where is your “absolute sound” reference for that? And in what venue? An audio show is a marvelous place to learn about system set-up and room interactions as well as particular combinations (brands) of equipment. You can learn from the mistakes as well as the successes. It also gives you the opportunity to hear a lot of gear you can’t really afford or find in your local shops back home, particularly new introductions and tweaks that may or may not ever become available.

    1. All those modifications of the signal still need to be turned into sound, and the speaker is the most important part of the chain if you want to produce music instead of noise and novelty.

      I started making “Extended Guitar” speakers because of pitch shifting and electronic instruments with more pitch range than standard electric guitars. The largest has a frequency range from 40Hz to 20KHz, an extra octave of bass and two extra octaves of treble. Note that traditional bass guitar speakers have a high pass at 80Hz, as do the Fender and Rickenbacker basses.

      These speakers have a clarity of sound like a studio DI to monitor rig, but with spatial characteristics of Fender open backed cabinets and subtle voicing including calibrated dynamic compression that make electronic instruments like Theremin and synthesizer and electric instruments like Rhodes and Wurlitzer sound more like acoustic instruments with wooden sound boards. The 15″ model is a favorite of acoustic Jazz bassists, it has a lot of punch for pizzicato and a little bass boost to keep up with a kick drum.

      The extended treble means they “glitch” better than guitar amps with a natural bite, and bring out the bell tones of a Telecaster as never before heard on stage.

      1. Two years ago I was “blown away” by Stanley Clarke accompanied by a much younger drummer and much younger piano player. On both acoustic and electric bass. I couldn’t ask any audio system to try and replicate that live experience.

        I’ve often wondered how such great music can come out of such ordinary guitar speakers. I know drivers are made to different standards, but you seem to be taking the technology to a much higher level with your “Extended Guitar”. It’s great to hear about people who are stretching the envelop. I always enjoy reading your comments, Acuvox.

        On another point, I’ve often thought it ironic how one guitarist needs a herd of guitars for his concerts, while another gets through an entire career with just a single instrument.

        1. My speaker building odyssey began with bass. I used to play bass guitar as an amateur, but had an epiphany that there were no speakers I liked. I figured out why, but fixing the problem was impractical at the time (1997).

          I now have a “brute force” solution based on a hand built woofer with 28mm of travel, 98dB sensitivity and extension to 5KHz from an ultra-low .3mH voice coil inductance. I also built a “portable” version for acoustic/electric doublers with a 12″ woofer that is 93dB efficient – it needs over 500 Watts for stage use, but there are a plethora of lightweight digital bass amps in this power class.

          The way to reproduce stage sound at home is to setup like a stage, with separate speakers for bass, guitar, keyboards and three or more speakers for drums. I use two or three speakers for grand piano, including a Steinway Model D.

          I have a 12 channel ADC interface, maybe I will make some recordings configured like this.

          1. I think you should hire stronger roadies to carry your large bass woofer.
            I once visited a customer’s home and learned he was a musician. He had a large room with three of the four walls all glass from floor to ceiling (3-4m) filled with microphone stands, mixers and guitar amps. Once a week his “band” would visit with their instruments and they would play music all night. But they never performed in public. Strange–to be so committed to music, but not play for an audience.
            It seems like you would need 12-track recordings (eg. master tapes) to play back through the speaker arrangement you mention above. Those are difficult to find. Very interesting idea, though.

  7. Well said Longplayer. I feel exactly the same.
    Visiting an audioshow is always a pleasant trip for me.
    Listening to the sound of Magico, accompanied by Soulution amps (just one example).
    Sounds incredible. I can never afford it. Not now and not for the rest of my life, alas.
    That doesn’t mean I’m not satisfied with my own system.
    But you don’t get the chance to often to hear this kind of world class equipment.

    1. Knowledge and creativity can go a long way in helping David slay Goliath in this hobby.
      And you can pick up a lot of knowledge at the shows. If you take High End Audio as a quest for perfection, you will ultimately loose. If you treat it as a means to entertainment, you will be fulfilled along your entire journey.
      BTW, Peter Madnick, one of the principals in the Constellation brand (a competitor of Solution) has revived his Audio Alchemy brand that sounds very good at considerably more affordable prices.

      1. Well said.
        Audio Alchemy is typical of being a brand based on a good engineer making sensible domestic audio products at sensible prices. Don’t have to reinvent the wheel, just come up with a good design that works well.
        One has to put aside that companies mainly in Asia manufacture some products just as good and much cheaper due to economies of scale.
        My preference is for local designs made in China. My DAC and amplifiers (Audiolab, Quad) are such, designed in the UK, made in China, probably halves the price.

  8. Interesting point paul.I wonder if you’re thinking what I’m thinking? I’m thinking about the magical room and the gentleman that had a special power conditioner based on some kind of electromagnetic cells. He was giving a demonstration where he was switching this special kind of what he called ‘ The biasing circuit ” on and off. It sounded better with his bias Circuit turned on then it did off. However, with The bias off The sound was overall unacceptable in general.you follow me? You guys follow me?

Leave a Reply

© 2023 PS Audio, Inc.

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram