Mother, should I build a wall?

1989. The year the Berlin Wall came down. The end of thirty-eight years of the city’s physical division in which the ‘West’ had been ringfenced by two walls and a ‘deathstrip’. The Wall’s fall marked the beginning of the end for the extremist thinking that had defined much of 20th Century (eastern) European history. And good riddance!

That same year, on the other side of the Atlantic Ocean, John Atkinson had begun measuring audio equipment for Stereophile magazine. Work that he continues to this day and work that I take great pleasure in reading — and learning from. The key that unlocks Atkinson’s graphs and charts are his carefully articulated explanations and interpretations. These are what set Atkinson apart from his peers. Without them, I’d estimate that 99% of Stereophile readers would find themselves staring blankly at a page of graphs and charts or – worse – reaching for their own erroneous conclusions.

Surprised? Some readers might see my so-called ‘Subjectivism’ (listening tests) as the antithesis of Atkinson’s so-called ‘Objectivism’ (measurements). But only those who insist on a black and white world will see things this way. A black and white world that does not exist.

Whether we take an interest in measurements or not (and I do!), we can’t put together a hifi system without bumping up against objective reality. Learning if our amplifier has enough go-juice on tap to drive our loudspeakers forces us to consider the amplifier’s specified watts per channel and the loudspeaker’s sensitivity rating. We’ll not get high SPLs from a flea-watt SET driving a pair of 84dB standmounts, especially in a large room. Many standmount loudspeakers don’t do the bottom 20Hz. Specifications matter.

It doesn’t end there. The objective reality of one’s listening room will sooner or later skewer the anything-goes attitude of the wannabe subjective purist. My lounge/listening room measures 6m x 5m. Mathematical acoustic modelling tells me, incontrovertibly, that the room has a 35Hz mode. REW room measurement software has confirmed as much on at least three occasions. No amount of subjectivist willpower can magic this problem away.

Ergo: there is no such thing as an absolute subjectivist. (White does not exist). Every subjectivist audiophile will be, in part, an objectivist.

I extend my objectivist interests as much as possible by digging into John Atkinson’s measurement work over at Stereophile.

His most recent piece is a video presentation that explains why measurements matter — effectively a summary of his thirty years in the saddle — and how time and experience have ultimately led him to become (in many audiophiles’ eyes) the grandfather of hifi measurements. In his video lecture, Atkinson lays out the four reasons why he started measuring audio gear for Stereophile back in 1989:

1) Measurements would turn up compatibility problems between components e.g. a loudspeaker and an amplifier

2) Measurements would expose components that are, in effect, expensive ‘tone controls’

3) Measurements would separate the products designed by masters of their craft from their amateur-designed rivals

4) A database of measurements would enable one to predict how a product would sound from the way it measures…

…at least that was the plan.

A thirty-year database of audio measurements is something to behold but here comes the kicker: according to Atkinson, expensive tone controls notwithstanding, measurements’ ability to predict sound quality remains as elusive as ever.

This jives with a recent email from a very experienced digital and analogue engineer (whom I won’t name). John Atkinson had recently measured his latest pre-amplifier and found it to be a fine specimen from a measurement point of view. The engineer also told me that he could alter the pre-amplifier’s sound in numerous different ways without affecting its measurements. That’s interesting, no?

It also jives with the above video’s middle section, in which the presenter GoldenSound explains how measurements can point us at any mistakes within a product’s design but are wholly unreliable when it comes to predicting sound quality. From the video’s description box:

“The Magnius is a balanced, low-cost amplifier that I’m not too keen on, but also don’t think Schiit had much choice but to make this way. It’s an amp that represents the movements the budget audio market is making. In this video, I talk about why I dislike the Magnius, why I feel that it isn’t Schiit’s fault, and a little bit about why you shouldn’t blindly trust measurements as a guarantee of quality.”

That in turn jives with HEDD Audio found Klaus Heinz’s take on the matter: that measurements can tell him if he’s made a mistake but not how his loudspeakers will sound.

In his 1991 Stereophile column Subjective Fact or Objective Fantasy?, Atkinson suggests, “that it is as much fantasy to expect that measurements can predict sound quality as it would be to think that they could predict the quality of music. All that the reductionism inherent in scientific method can do is to look after the fact for possible explanations for what is heard.”

Paraphrasing a line from Atkinsons’s 2021 video presentation: measurements effectively boil a product’s complex multi-dimensional behaviour down to a handful of two- or three-dimensional graphs.

The takeaway? No matter how much we value measurement data and its interpretation by a skilled practitioner, that measurement data will tell us very little about a product’s sound. We still need to listen to a hifi component if we wish to get a better handle on its sound quality.

This is no doubt why Stereophile continues to publish listening test reviews alongside Atkinson’s measurement analysis. They are conducted by a writer who doesn’t get to see the measurement results until everything goes to print.

I’m in the business of talking about sound quality and how it relates to music. The limitations and biases of sighted listening tests are well known. Schiit Audio co-founder Jason Stoddard takes us on a journey through these biases in his excellent Head-Fi post and Schiit Happened instalment “How we fool ourselves”.

Stoddard lists our personal biases as follows:

1. Good mood or bad mood
2. Altered state of consciousness
3. The total (shopping) experience
4. Operational preferences (and functionality)
5. Packaging bias
6. Brand bias
7. County of origin bias
8. Review goggles
9. Measurement glasses
10. Scientism
11. Topology prejudice
12. Designerism
13. Boredom, or familiarity breeds discontent

“Differences between audio components are much smaller than most audiophiles make them out to be, at least to the majority of listeners,” he says.

But jiving with John Atkinson’s more pragmatic world view, Stoddard goes on to say: “There are, however, differences that aren’t readily explained in terms of one frequency, one measurement, one number, and these differences may matter to some listeners.”

And what does Atkinson do when his direct listening experiences and measurements don’t align? “Stereophile practices it [science] in its true form: when experiments give results which contraindicate reality, the experiment is rejected, not the reality”, he says.

That leads us to a bias that exists outside of listening tests: 10. Scientism — the belief that measurements are all we need to know in order to state a preference or make a purchasing decision. According to Stoddard, this is a personal bias like any other.

Of all the audiophile world’s personal biases, Scientism has seen a significant number of people rally behind it in recent years. And when Scientism is married to the conspiracy theory that measurements expose many a manufacturer’s deception and enough people attend the wedding reception, we net a populist ideology that serves as a breeding ground for 1) intolerance of anyone who questions it; 2) dogma to convert others to the cause and 3) a furthering of the conspiracy theory that the existing power base is out to take advantage of an unsuspecting populace. Sound familiar?

Let’s take a closer look at this conspiracy theorising:

A manufacturer introduces Product X, which is quickly found to offer no measurable difference to that which precedes it. The manufacturer is therefore accused of engaging in fraud; a very serious allegation that is often sugar-coated with the term ‘snake oil’. I don’t know about you but when I read ‘snake oil’, I read ‘fraud’.

But for this fraud allegation to hold water the following must also be true:

1. If the manufacturer has been in business for many years, not a single member of staff (past or present) will have stepped forward to blow the whistle on the company’s fraudulent product/s
2. Every single reviewer claiming an audible difference when inserting Product X into a hi-fi/head-fi system must have been a) paid off by the manufacturer or b) suffered from confirmation bias or similar placebo effect during their listening tests
3. Every single consumer buying Product X and claiming to hear an audible difference from its usage must have also suffered from confirmation bias or a similar placebo effect when auditioning the product
4. Every single audio dealer and distributor around the world selling Product X is a) in on the scam and b) not a single member of staff (past or present) will have stepped forward as whistleblower
5. The wrong things have not been measured and their interpretations made without error

I really don’t know how to engage with people whose thinking is locked to all five points being true — the conspiracy theory hinges on some seriously long odds. This is doubly true (with a cherry on top) when I am asked to address such thinkers via the Internet where body language and tone of voice are MIA and the likelihood of talking past one another is high. The conversation almost always ends the same way: I’m called an idiot or a heretic (or both) for not trusting unflinchingly in ‘the objective measurements’; never mind that they contain their own subjective components (which we’ll get to shortly); and never mind that if the manufacturer of Product X’s global product fraud is true then we really ought to be able to turn up someone within their HQ or dealer network, past or present, to serve as a second source.

Does this polarised and poisonous social media landscape offer a clue as to why Scientism has become so popular in the audiophile world? I think it might. Pose a question about a product on a forum or Facebook group and few respondents will have had sufficient first-hand experience with the product to offer up a reliable opinion, especially if that opinion revolves around sound quality. But present a series of measurements and everyone in that forum or Facebook group is a part of the conversation. Everyone can add their 2 cents. Measurements’ conversational inclusivity makes them an armchair audiophile’s wet dream. Why haul ass to go and listen to something new when you can stay at home with the gear you’ve owned for a decade and sneer at the idiots who are buying the new/expensive/poorly-measuring stuff? Remember: ideologies don’t allow for flexible thinking; and even if they don’t outwardly promote intolerance of ‘other’, they sure as hell let it slide when it does occur.

Audiophile Scientism is the root cause of measurement angst among some (so-called) subjectivist reviewers. I can see why: being told that your listening commentary is inferior to measurements when the latter’s ability to paint a picture of sound quality (beyond exposing expensive tone controls) contains just as many holes must be galling. Pointing out this double-standard in “Why measurements don’t matter” op-eds only serves to push the (so-called) objectivists into personal attack mode, especially when arguments are poorly articulated. And so the animosity goes around and around.

As Will Gore argues in his alternative take on the UK’s post-lockdown litter problem, “social media has a tendency to amplify divisions: righteousness cuts through in a way that humility does not. Anonymity breeds contempt.”

Time for a pause. And a deep breath. Suggesting that subjectivist objectors are attention-seeking dolts is to draw from the demagogue playbook. We only need to look beyond the headlines to see how these complainants are not objecting to measurements per se but to the way in which they are wielded; as Scientism swords in a powerplay. A more mature response from those heavily invested in measurements might be to put the ad hominem jibes to one side and calmly explain what the measurements do tell us — and also what they don’t, just as John Atkinson does.

Despite measurements being the first pillar of Scientism, according to John Atkinson, they don’t allow us to sidestep our subjective biases as easily as we might think. Is this well known? I’m not so sure it is. Are those who preach it aware that Scientism is itself a subjective bias? Again, I’m not so sure.

Does this, therefore, mean that measurements have no utility? Of course not. Measurements matter.

Soundstage!’s Doug Schneider describes it this way in his October 2020 More Measurements: Here’s Why op-ed: “As we’ve found out in the 25 years we’ve been publishing, although there’s broad consensus in the engineering and scientific communities about which measurements are valuable, how that measuring should be performed, and what good and bad measurement outcomes look like in terms of well-engineered, technically proficient products, there’s little consensus about how that translates to what we hear, especially when the differences in measured performance are minute.” (My emphasis).

Atkinson harmonises: “The measurements (individually) don’t tell you how things sound BUT what they can do is inform the customer if the mind behind that product a master of the craft (or not)”.

Atkinson and Schneider seem to be in agreement on what measurements can tell us as much as what they cannot. This pragmatic approach is right up my street because it refuses to deal in Scientism.

Holes? Swords? If Atkinson and Schneider are to be believed, idealists claiming that measurements can tell us all we need to know about the sound of a hifi component are just trying it on. Their ideological posturing is nothing but play-acting, their swords of truth imaginary.

Atkinson sums it up like this: “Quality can only be inferred via a holistic approach, which is of necessity subjective”. Atkinson is saying that he, like anyone measuring audio gear and interpreting their findings is, in part, a subjectivist.

Even Audio Precision staffers – the folk who design and manufacture some of the most widely used measurement hardware – think like Atkinson and Schneider: that a consensus on how ‘what is measured’ translates to ‘what we hear’ has yet to be agreed upon.

In 2015, Jonathan Novick, then an 11-year veteran of Audio Precision, gave a presentation at RMAF on “What The Specs Don’t Tell You… And Why” (see above). His video intro blurb runs as follows: “Better specifications mean better-sounding products. Right? Wrong! Audiophiles have known for decades this isn’t always the case. Meanwhile, the scientific community likes to believe if the specs are the same, the products sound the same. Which side is right?”

Six years later, Novick introduced his RMAF video to yours truly via email with the following background information: “I am an electrical engineer with 30 years of experience working for test & measurement companies. I also used to follow the hi-fi market (but stayed relatively modest in my purchases). I spent 13 years with Audio Precision selling test gear to all sorts of manufacturers. I was also a VP of the Audio Engineering Society and sat on several standards committees within the Consumer Technology Association (nee CEA). However, I was keenly aware that measurements don’t describe the sounds we hear. Just over 10 years ago, I built a test box to prove this with listening tests side by side with measurements and went on the road with a presentation. Most of my audiences were AES members but I also went to several Hi-Fi shows (RMAF, T.H.E. and others).” (My emphasis).

Flipping it around, if the numbers don’t lie (as we’re often told) then why have multiple people measure the same product? Because the measurement process is partly subjectivist. Interpretations matter. They are the engine that drives the data forward. Without ’em, we have only a hunk of metal sitting in the driveway. We go nowhere and we learn nothing.

This thread was picked up by a pair of ‘Certainty Uncertain’ editorials over at Michael Lavorgna’s Twittering Machines. Lavorgna illustrates with examples that all reviewers who measure gear are, in part, subjectivists when 1) they choose what to measure, 2) they choose how to measure it and 3) they interpret the results. “Interpretations are, by definition, subjective”, he chuckled at me down the phone last week. “And to call yourself an objectivist when parts of your process are subjective is like insisting that a compression algorithm that’s both lossless and lossy is still lossless.”

Ergo: there is no such thing as an absolute objectivist. (Black does not exist either).

The subjectivist/objectivist dichotomy sees no grey. And yet we – and our experiences – are all grey. Every audiophile’s perspective is a mixture of measurement and listening test interest. Proponents of the subjective/objective axis perform only one task: they build a wall between the two worlds, word by word, brick by brick. They bury nuance and complexity under concrete slabs.

My psychologist friend Herman, who has lived in the former ‘West’ Berlin since the 1960s, says that any kind of extremist ‘black and white’ thinking is as old as time itself: defining ‘other’ can open the door to implying ‘inferiority’. “As we all know, the Nazis took this kind of thinking to its extreme with devastating results,” he says. He pointed me at the following extract from Lena Dominelli’s Anti-Oppressive Social Work Theory and Practice:

Thankfully, Objectivist/Subjectivist labelling puts no-one in the hi-fi world in mortal danger. No families will be torn apart. But history tells us that we should be wary of black-and-white (read: overly-simplistic) thinking – where only one way is prescribed – and of those who shout loudly to promote it. History teaches us that the ideologists’ ‘legibility’ playbook runs like this:

• “Look at a complex and confusing reality
• Fail to understand all the subtleties of how the complex reality works
• Attribute that failure to the irrationality of what you are looking at, rather than your own limitations
• Come up with an idealized blank-slate vision of what that reality ought to look like
• Argue that the relative simplicity and platonic orderliness of the vision represents rationality
• Use authoritarian power to impose that vision, by demolishing the old reality if necessary
• Watch your rational Utopia fail horribly”

(Source: A Big Little Idea Called Legibility by Venkatesh Rao)

Evidence of a failed Utopia is scattered around my neighbourhood. I live just inside the former East Berlin. Checkpoint Charlie sits three streets away; a reminder of the city’s military-enforced division. The boom gates are thirty-one years gone but the hut and signage about entering/leaving the Russian/American sector remain. It all looks so quaint flanked by gift shops, McDonald’s and KFC. But we should never forget that the Stasi’s thought policing was anything but.

“Wanna go for a ride?”
“No thanks.”
“No thanks? What does that mean?”
“It means I don’t wanna go.”
“Go where?”
“For a ride.”
“A RIDE! NOW THAT’S A GOOD IDEA!”

Online, the pernicious pulling power of demagoguery lives on. “I am right and anyone who disagrees with me is either wrong or an idiot” is heard louder than ever. Audiophile community division, linguistic or otherwise, pits one subgroup against another. It gives a leg-up to conspiracy theories and it normalises ad hominem attacks. It pushes well-reasoned counter-arguments into the back seat to sit alongside good manners for a joyride around town. The resulting engine noise and tyre marks run in the opposite direction of community-mindedness. Ideological battles repel newcomers to stunt community growth.

This is one reason why I ache for the end of the pandemic and the return of audio shows. Real-life events underscore a sense of community and provide a welcome break from the online world’s inclination towards identity politics. In the real world, this Objectivist/Subjectivist nonsense simply evaporates to allow hifi enthusiasts to use their ears and measurements in concert (but to varying degrees) in pursuit of better sound.

Further reading: Jason Stoddard’s Schiit Happened – 2016, Chapter 5 – The Subjectivist/Objectivist Synthesis