Report And Distort Part Two: Lightning Crap

The immense stupidity of comparative magazine lap times

we don’t need to check torque, it’s a press car, it’s fuckin fine, just get in the cars

In Part One of this series we talked about how straight-line performance numbers are fudged. Today, we’ll talk about all the road course data, from Top Gear Stig times to C/D’s famous “Lightning Lap”.

But first, a very big DISCLAIMER: I was only around for some of this stuff. Part of what you will read below is coming to you secondhand. It may be exaggerated. It may not be true. Treat it as a way to look at what you read, not as letter-perfect gospel. It’s not my intent to libel, or even disparage, any particular individual. Thanks.

Okay. Deep breath.

It pains me to say this, and I worked for years in multiple futile attempts to make what I am about to say not true, but: With one critical exception, every single set of racetrack lap times ever published in automotive media is

Completely. Useless.

That Top Gear leaderboard, set by “The Stig”? Meaningless, with margins of error that vastly exceed the differences between most class-competitive vehicles. All those Willow Springs and Laguna Seca times in Motor Trend? Assuming they aren’t outright faked — and I personally believe, without overwhelming evidence, that some of them are — they don’t mean anything at all. The Car and Driver Lightning Lap, invented by Larry Webster in an attempt to expand the magazine’s considerable existing road-test authority? It might be the biggest joke of them all.

“Not so fast, Baruth,” you might be saying. “It’s easy enough to dump on the competition, but what about your track testing, in particular the two years where you were the test driver for Road&Track’s Performance Car Of The Year?”

Reader, I tried, anyway — and I’ll explain what I did in an attempt to make the numbers useful. I’ll also share a few things I learned that might help you make some sense out of what’s published out there. Oh, and I’ll go over that “one critical exception” before I close up for the day.

Let’s start by assuming that autowriters and their tame race drivers are, in fact, great drivers. Forget everything I’ve ever said and everything you’ve ever seen. All those dead-fuckin’-last finishes in vaguely competitive events from jerkoffs as diverse as Jamie “VWVortex” Vondruska (DFL in the VW TDI Cup) and Johnny Lieberman (DFL at Pikes Peak in a FACTORY PORSCHE) and Robin Warner (12th place in a Skip Barber season, driving against total rookies, housewives, and children). Just forget that stuff. Let’s assume that everybody is as great as… oh gosh, SCCA Runoffs winner James Libecco!

Libecco is a major amateur talent who is often two seconds a lap faster than other experienced drivers in identically prepared cars. I don’t think I have ever seen him make a mistake on track. A few weeks ago, when I was busy winning the SCCA Big Bore race flag to flag in my Radical SR8, Mr. Libecco was running the SRF3 spec cars. He led Saturday morning qualifying with a very respectable 1:37.738, as you can see here:

https://speedhive.mylaps.com/Sessions/6855708?user=MYLAPS-GA-5dbd51e393a147be8f5b8155d1ab0d94

That’s pretty darn good for an SRF3. But in the race he pulled out a scarcely credible 1:36.131, more than TWO SECONDS ahead of the rest of the field, as you can see here:

https://speedhive.mylaps.com/Sessions/6855709?user=MYLAPS-GA-5dbd51e393a147be8f5b8155d1ab0d94

James wasn’t the only fellow who went significantly faster in the afternoon than in the morning. What changed? Just the track conditions. The track temperature was about twenty degrees higher in the afternoon. It had rained a few days before, so Mid-Ohio was “green” on Saturday morning. The six track sessions that occurred between qualifying and the race put a lot of rubber on the track.

Libecco didn’t improve as a driver between the sessions— he has thousands of laps here. In fact, he slowed down a bit the next morning, turning a 1:35.690 on the 0.2-mile-shorter Pro layout.

I’m using James and his SRF3 competitors here because these cars are fully and completely understood. The chassis has been raced since 1984, with three different engine packages that are fully sealed. Libecco doesn’t own his car; it’s prepared identically each and every weekend by Alliance Autosports. He always runs new tires, the best of everything. There’s no “development” happening here. Not any more.

And yet… a 1.6 second difference on the same day. Go look at the Top Gear Power Lap Board and try playing around a bit with 1.6-second changes. That’s the difference between the Viper ACR and the Lamborghini Aventador. Also the difference between a Radical SR3 and a BMW M5, of all things.

If a Runoffs winner operating in peak form in a car he knows like the back of his hand can have that kind of gap between consecutive sessions, and his competition can experience the same thing, then how valid, really, are the Top Gear lap times set by any number of “Stigs” across all four seasons of English weather?

The answer, of course, is “not at all”. The confidence interval of those laps has to be more than 1.6 seconds. It could be twice that, or three times that. Taking all the variables of driver, weather, and track condition into account, it could be… hell, it could be four seconds! That’s the gap between the hero-level Ferrari 599 Fiorano and the pussy-assed Porsche Boxster Spyder — and more than the gap between the aforementioned Boxster and a VW GTI.

Any gap of less than four seconds on the Power Lap is potentially meaningless. You can apply the same general thinking to pretty much all the single-lap lists published by magazines. Motor Trend uses a single driver, of course, but that single driver has his distinct limitations. I have also heard it said, without substantiation, that the aforementioned single driver puts in vastly different amounts of effort depending on the automaker involved, often due to the potential of a future business relationship, or lack thereof. I couldn’t tell you.

What Happens When The Journosaurs Get Involved?

Up to now, we’ve only discussed variations between laptimes set by drivers with reasonable resumes and abilities. Throw the average no-drivin’ doofus writer in the mix, and things get hectic in a hurry.

Take the C/D Lightning Lap. Why does it use the “Grand Course” at VIR instead of VIR Full? Because if they used VIR Full, you’d be able to see that most of their times can’t match, say, those of the 200-horse Civic I drove (badly!) in Koni Challenge 13 long years ago.

The in-car videos from Lightning Lap are pure cringe, featuring everything from thumbs continually wrappin’ the steering wheel and just waiting to be broken to… panic-brake maneuvers on the Climbing Esses. It seems impossible to believe that this is the best they can do — and yet it is. You won’t mistake any of the drivers for James Libecco, or even for your humble author.

The automakers watch the videos, too… and they hate them. For years now, many of them have been sending their own people to Lightning Lap in mostly futile efforts to coach C/D personnel into vague acceptability around VIR Grand. I heard a rumor than one automaker spent three days trying to coach K.C. Colwell up to the laptime set by their pro in twenty minutes.

There’s also a huge difference in how various automakers approach these timed events. Some of them send a trailer full of tires, a pro to help out, some data coaches, and on-site mechanics. Others send… the car. During my time at R&T, Ferrari sent us a car with untorqued lug nuts. When a staffer begged to be allowed to just check torque and pressures before driving, a very senior editor threw a fit and told them to “get in the fuckin’ car”, at which point the wheel, ah, mostly departed the vehicle on-track.

Oh well. Journosaurs are very easy to replace; there are a lot of us and we don’t earn much.

The only value I can see in Lightning Lap is this: sometimes the cars break in ways that are potentially informative, as with the early 350Z that lost its brakes.

Recently, C/D invited a bunch of readers to join them for Lightning Lap. A few of them easily beat the magazine’s times in totally stock cars. Oh well.

(A brief digression: The recent trend at various magazines of putting on events for readers is quite possibly the most embarrassing and miserable thing I can imagine in the autowriting space. I don’t know whom I pity more: the spanks who pay huge money to watch people like Eddie Alterman and Elana Scherr talk about nothing in particular, or the small number of genuinely decent people at the magazines who never expected that their jobs would basically devolve to “trained monkey who entertains rich people”. This bullshit can’t stop soon enough to suit me.)

If the margin of error at Top Gear is four seconds, then the margin of error at Lightning Lap has to be three times that much. The track is much longer, and the people involved are much worse at their jobs. Let’s say ten seconds, to be charitable. That’s the gap between a Ford GT and the Porsche (Cayenne) Turbo GT, or the gap between a C6 Z06 and the Audi RS7. Oh, and there’s the fact that just 0.4 seconds separates the LightLap times of the Boxster S and the Cayman R. I’m a bit of a skeptic when it comes to Porsche but… come on. Betcha the Cayman R is ten seconds faster in competent hands.

Bottom line: you don’t need to take Lightning Lap any more seriously than you would take any rap album I would personally write, perform, and produce, and for the same approximate reason.

Horst Is Just Zis Guy…

Last but not least we have the infamous Sport Auto times set by Horst von Saurma. It’s worth reading this interview where he discusses his method, which boils down to “I go about as fast as I feel comfortable going, for one lap.”

Having driven the ‘Ring for time myself, knocking out a hand-timed full course (not bridge to gantry) 7:50 in a Lotus Evora 410 back in 2017 or thereabouts, I am deeply suspicious of any timed sessions on that track. The weather is fearsomely variable, often dipping 10 or 15 degrees from one side of the course to the next. Much of the Ring can best be described as a dyno test periodically interrupted by a chance to bet your literal life on whether or not you need to lift or brake for a given high-speed corner. More than any other track I’ve ever driven — I’ve driven ninety-four different configurations of sixty-three tracks in the past twenty-two years — the ‘Ring is dependent on driver familiarity, risk tolerance, and just plain luck.

Remember, too, that street tires at high speed are exceptionally bad at shedding heat. Let’s say you’re running the ‘Ring for time. Do you heat the tires early in the first thirty corners, knowing you’ll pick up precious seconds at the risk of a big crash later, or do you save the rubber for those high-speed bends later? The number of variables involved is remarkable. In my first day driving the aforementioned Evora, I mistook one corner for another and lifted briefly when I didn’t need to. It cost me perhaps ten seconds. From a single lift of the throttle. Think about that.

Alright, So How Do You Fix It?

Road&Track ran Performance Car Of The Year testing at NCM Motorsports Park for four years. The first year, Larry Webster was the driver, on a variant of the full course. He took it very seriously but struggled a bit to set consistent times on a track he didn’t know well.

When I took over I cut down the course to just the NCM West layout and also reduced the amount of time set aside for testing. We invited manufacturer reps and drivers to observe, including Jim Mero, Chevrolet’s Ring-time engineer.

I couldn’t get rid of all the variables but I thought I did a half-decent job of getting the laps done in similar weather and track conditions. There were a few times I thought I was under-driving the car — this was particularly true of the McLaren 720S, which was painfully short on both front and rear tire grip. But the times generally matched what you’d expect after reading the spec sheets. I always offered the manufacturers a shot at driving the cars themselves, but they did not take it.

(Out of raw ego, but also because it’s possible I was being massively shined on, I should note that on the multiple occasions I’ve shared data with a factory-provided driver, I’ve always managed to do a little better than they did. Who knows why. I suspect it was because we tested at places like NCM and Summit Point, rather than at the big-name tracks with which they’d have been familiar.)

In the fourth year, Kim Wolfkill replaced me with IndyCar driver JR Hildebrand, mostly out of sheer starfucking ignorance. JR didn’t drive the slower cars very well and he flat-out broke an Alfa Romeo — but I think his time in the Senna was better than what I would have gotten, largely because he has much more experience and ability with aero cars. JR was thoughtful, decent, and generous throughout — but he also didn’t have a lot of ability to distinguish between relatively tame street cars, for the same reason you wouldn’t expect Barry Bonds to be a particularly good judge of slow-pitch softball bats.

In recent years, Travis Okulski has been setting laptimes at R&T. He’s a skilled and thoughtful driver who is very good at extracting some useful impressions from a car and conveying them — but he’s also had to work with some crummy tracks, like Lime Rock. (Lime Rock is great for spectating, and great for vintage racing, but not so great for modern cars.)

Truthfully, I’m not sure it’s really possible to get hard, reliable data on trackday performance. Over the course of a day, you put rubber on the track, you improve slightly — or maybe you get tired, and maybe it gets hot enough to affect the engines.

(Side note: During GM’s testing of the C7 ZR1, they tried to see if there was a temperature below which the lower tire grip from cold rubber would offset the power gains from denser air. They tested from 90 degrees down to 30 degrees without finding that temperature. The increased engine performance always outweighed lesser grip. Would the same thing happen in a Miata? Probably not.)

So The Point Is:

I’ve come to believe that the true value in “objective testing” is actually subjective. The process makes you learn things you wouldn’t otherwise figure out. Example: The AMG GTR is perhaps my all-time favorite trackday supercar because it has a multiple-position adjustable knob for the traction control that you can twist between corners. After five laps twiddling that knob, it’s possible to get massive laptime gains from what you’ll learn — and because you can take camber and available traction into account, you can truly maximize each corner exit.

Another example: Brother Bark actually set a better laptime in the current-gen NSX that I could at NCM West. Why? Simple: he’d just won Targa Southland in an NSX and he knew how much he could trust the tricky electric-motor front end to turn the car at high speed. He was something like 1.1 seconds faster in a single big corner, and only lost 0.5s of that elsewhere on track.

The process of setting fast laps in a disciplined fashion makes you an extremely close observer of a vehicle’s behavior. If you’re able to retain and record the insights you get along the way, you can provide the reader with information he can genuinely use to buy the right car for him. This is especially valuable with cars that simply can’t be test-driven to any sort of limit before purchase.

Should I ever find myself in another magazine role again, I’ll do my best to provide detailed information to help the reader buy the right car. No doubt I’ll record some times along the way. They won’t be much more useful than the trash served up by Top Gear and Motor Trend, but at least I’ll be a tiny bit self-aware about the process and the results.

Until then, feel free to use various lap times for bench racing, bull session, Internet arguments, and just plain amusement — but don’t fool yourself into thinking they mean anything. The McLaren Senna is probably faster than the Honda Fit. Probably. But if you put Jonny Liberman behind the wheel of the Senna and Jimmy Libecco in the Fit? Don’t bet your last dollar on that one.