How The Magazines Report (and Distort) Performance Numbers

Part 1: Why did C/D and R&T print different results from a single acceleration test?

seasoned 8 year old kart racer forced to drive $15k childrens toy, not thrilled

My brother Bark, who just celebrated forty-five years of following his bliss this past Saturday, once said something along the lines of “Corvette drivers only talk about two things: the best way to detail their cars, and how fast the magazines say their cars go.” I treat jerkoffs like that with all the contempt they so richly deserve — imagine using another man’s results with a car as a basis for personal pride — right up to the moment that it’s time to talk about motorcycles, at which point I’m like,

“Well, the ZX14R did a 9.48@144… for Cycle World, of course.” I couldn’t tell you what a ZX14R does in the quarter-mile when I’m riding one, because I have literally never dropped the clutch on it from a standing start. My motorcycle skills are modest and they do not extend to drag racing.

(Top speed is another matter; I can tell you that the bike will briefly report about 190 on the LCD screen before hitting a very soft limiter with a few useful revs obviously in hand.)

Let’s face it: men are creatures of statistics as much as we are creatures of anything else, and performance numbers matter. Very few of us have what the late Jeff Cooper called “the room, the range, and the time” to get our own numbers on everything, so we rely on third parties. We assume, correctly, that these numbers are exceptionally important to automakers and customers, and that therefore there is a competition to get the best numbers. What’s not quite so obvious: that there is a competition between the magazines themselves to get the best numbers.

And the best — the only — way to win that competition is to fake it.

C/D Is The Fastest, And Here’s Why

Two years ago, Dave Vanderwerp wrote this very honest-sounding article about C/D testing:

All of our straight-line acceleration results are the average of the best run in opposite directions, to account for wind. Ambient weather conditions—we record absolute barometric pressure and wet- and dry-bulb temperatures trackside—determine how much power an engine makes. Because of that, we also correct acceleration results to 60 degrees Fahrenheit at sea level. Cooler air is denser and contains more oxygen, allowing the engine to burn more fuel and make more power. Similarly, high barometric pressure produces more power than low pressure, and dry air has more oxygen than moist air. All of our standing-start acceleration times also include rollout, a short period of time (typically about 0.3 second) that we subtract from the acceleration figures. It's a phenomenon that stems from the physics of the timing lights at a drag strip, where a car can be rolling for 12 inches or more before the clock starts. We recently changed our procedure to now use the industry standard 1-foot rollout.

As many of you likely know, it was Car and Driver that set the standard for normalized industry testing during the Seventies and Eighties, using Chrysler’s proving grounds facilities for most tests and the famed “Mrs. Orcutt’s Driveway” for top-speed testing. During the Eighties and Nineties, C/D reliably published the fastest testing numbers, particularly with regards to 0-60 and quarter-mile times.

When David E. Davis Jr. left to start Automobile Magazine (and collect a reported million-dollar bonus for hitting his promised numbers) he didn’t bother with performance testing. This was seen as heretical — what was a magazine for, if not to record the numbers? — but DED wasn’t interested in the dirty, grubby hands-on aspects of car magazines. (Or many of the cars, as evidenced by his “shitty little cars for shitty little people” comment regarding affordable subcompacts.) He also likely figured it would be impossible to beat C/D’s numbers…

…because by then the magazine had made a practice of mathematically adjusting their results. I’ve been told more than once, by people who were on the ground in that era, that C/D was highly aggressive in their adjustment, to the point that the published numbers didn’t often bear much resemblance to what was observed at the test track.

It’s easy to argue that Car and Driver’s acknowledged status as the best car magazine in America was based on just two things:

0. Comparison tests where a BMW or Honda usually won, in an era where people were often eager to be the kind of cognoscenti who only bought Bimmers and Hondas;

1. The fastest performance numbers.

Remember that this was an era before instant publishing, so the magazines were all ignorant of what numbers their competition would be publishing on the same new cars they were testing. Nowadays, you could wait until C/D released their numbers then simply adjust yours a little harder, but back then the magazines were sent to the printer a full month in advance.

C/D always had the best numbers, followed by Motor Trend, with Road&Track lagging considerably behind. Example: In overseas testing away from their normal proving grounds, R&T got a Euro Countach 5000S to run a 14.8@100, but C/D got 13.5@107. That’s basically the difference between a 4-cylinder Accord and a V-6 Accord. So how much of that was additional skill on the part of Patrick Bedard, and how much was mathematical adjustment, in the form of temp correction and/or “rollout”?

In 2022, the magazines have standardized on a one-foot rollout allowance. Motor Trend admitted that in their test of a Ludicrous (or was it a Plaid?) Tesla, the car was going 5.9mph before it officially started the test, thanks to rollout allowance. The difference between a 100mph quarter-mile time and a 106mph quarter-mile time is serious business, as noted above, and that’s still true when you’re talking about faster times, such as the 9.4@151 reported by C/D for a Plaid. Edmunds has this to say:

We believe the use of rollout for quarter-mile timed runs is appropriate, as this test is designed to represent an optimum drag strip run that a car owner can replicate at a drag strip. In the spirit of consistency, we also follow NHRA practice when calculating quarter-mile trap speed at the end of the run. So we publish the average speed over the final 66 feet of the quarter-mile run, even though our VBOX can tell us the instantaneous speed at the end of the 1,320-foot course, which is usually faster.

On the other hand, the use of rollout with 0-60 times is inappropriate in our view. For one, 0-60-mph acceleration is not a drag-racing convention. More importantly, it's called ZERO to 60 mph, not 3 or 4 mph to 60 mph, which is what you get when you apply rollout. While it is tempting to use rollout in order to make 0-60 acceleration look more impressive by 0.3 second, thereby hyping both the car's performance and the apparent skill of the test driver, we think it's cheating.

Interestingly, Edmunds also did a drag race with the Plaid vs. a (slightly depowered) 2022 ZX14R and Hayabusa, in which the bikes generally started out behind and either caught the Plaid by the quarter-mile or shortly after. Yet no motorcycle magazine has claimed the kind of trap speeds for a Superbike that the car magazines regularly claim for the Plaid.

(I can tell you from personal experience that my 2015 ZX14R, which is a little faster than a 2022 due to a new EPA-friendly tune, will easily dust a P100D in a short race, usually before hitting 130mph.)

I would feel comfortable suggesting that C/D simply uses the most aggressive mathematical corrections, and that they have done so for a very long time, because doing so is a competitive advantage in the magazine business. And I know that they have a deliberate mathematical advantage over at least one magazine…

The Deliberate Crippling Of R&T Results

Most of you probably know that Car and Driver and Road&Track have had the same owner for a few decades now. Since 1988, in fact. What you perhaps do not know is that the magazines often share the same instrumented tests of new cars. Yet the numbers are never the same. What gives?

It’s simple. The two magazines use a different “correction factor” to produce different results. It’s pure optics — and the optics are always balanced in C/D’s favor, because having the fastest test results is viewed as part of the C/D brand. So you can forget any business about how the correction algorithms accurately adjust to an imaginary 60-degree day with a certain amount of humidity, because were that the case one of the magazines would be deliberately lying. I’d prefer to think that they just happen to disagree on the amount of atmospheric correction that truly reflects reality. That could be the case, right?

In Which The Author Tries To Do It Right

During my previous job, I regularly met with C/D veteran Don Sherman to discuss the creation of a vehicle-testing regime in which we would produce unvarnished results to the highest possible consistency. Don’s opinion was that nobody had bothered to “do it right” in a long time — and after watching a few instances of R&T testing, I agreed with him.

It’s not really the magazines’ fault. Standardized testing is expensive, it’s time-consuming, it is controversial with the automakers, and in the modern era it’s also very hard to get the cars you need to do it. There isn’t a single PR person in the auto business who genuinely wants their cars to be tested accordingly to rigorous methods, but the German and Italian sports-car makers are the very worst in that regard.

In 2022, you’re also competing for access with the influencers, who are far more popular with the PR people and who can be relied upon to tell a precisely delineated story. Why give your car to a magazine for two weeks so they can get all the testing done in good weather? You can give it to Mr. Beast (or, it must be said, Doug DeMuro) and get a lot more eyeballs. Plus there’s no concern about your car being beaten in print by, say, a Corvette or Mustang.

My bosses treated my attempts to begin honest, standardized testing with the same attitude you’d use on a cat who dropped a shit-stained dead mouse on your glass cooktop — but they were more enthusiastic than any of the manufacturers with whom we spoke. Most of them were very blunt about the fact that they wanted to either supervise our instrumented testing or perform it themselves, with us in attendance.

Remember that last sentence, it will be relevant in Part 2.

Is There Any Performance Metric You Can Trust?

The toughest number to outright fake in convincing fashion, and the one that most magazines neglect anyway, is quarter-mile trap speed. It’s not much affected by rollout and temp correction, and it’s also not that reflective of driver skill. When in doubt, use trap speed.

It was my opinion that Hagerty should use half-mile trap speed as a legitimate performance metric, since that would indicate the combination of power, weight, and aero/mechanical drag in a given car better than almost anything else.

“What a stupid idea,” I was told. Oh well. Maybe they were right.

Alternately, you can do what a lot of race series do: look at true measured curb weight and a Dynojet chart. That will at least get you in the neighborhood. But the sad truth is this: unless you were present at a magazine test, you haven’t seen the real numbers from that magazine test. And you never will.

Not That It Matters Much

We already live in a sort of post-truth world when it comes to acceleration numbers. Today’s performance vehicles are so powerful that traction is the limiting factor in most cases up to perhaps 60-70mph. Electric cars are faster than gas-powered cars in acceleration tests, because they can optimize for these tests in a way that no EPA-legal street ICE car can.

(They also need to; a Hellcat is actually a little faster at the end of a 300-mile, tank-draining drive, while a Plaid is rather unhappy in the same conditions.)

Nowadays, quarter-mile times are almost directly correlated with how much money you’re willing to spend. Every automaker sells a dumb-assed 600-horsepower SUV that runs elevens. Many automakers sell a dumb-assed EV that can rip off a ten-second quarter in certain conditions. All of this stuff costs real money. Much of it is absolute trash in daily use.

We have more acceleration available to us than ever before. And yet people are less enthusiastic about cars than they’ve ever been. Is this causation, or merely correlation? Or is there an algorithm somewhere in the Car and Driver offices that already has the whole thing figured out? Maybe it’s not about the numbers after all — but my motorcycle is still faster than yours. I’m pretty sure it is. Cycle World said so, anyway.