Weekly Roundup: Woo For Its Own Sake At The End Of Tech Days Edition

Alright, let's get it off our chests: When Wednesday happened, when Lady Gaga had un-self-consciously re-enacted The Hunger Games, and when it became apparent to even the dimmest among us that Trump and "Q" were not going to descend into the vaguely Riefenstahl-esque walled-off self-celebration/bad-poetry-slam with a Blackhawk chock-full of pedophile-grabbing grapples like the ones used in the Christian Bale Terminator movie, what was your first thought?

I'll personally admit that my first thought was selfish. I didn't think about the end of the American oil industry, or the promised gun confiscation, or the female athletes whose scholarships just vanished into thin air, or all that business about structuring the economy around issues of racial justice and climate justice. All I could think was: Well, that's the end of the tech biz.

What the American press won't tell you, the Indian press is shouting from the rooftops. Biden is promising "the infusion of hundreds of thousands of visas per year". Let me repeat that:

The infusion of hundreds of thousands of visas per year.

Mr. Biden famously told unemployed coal miners that they should learn to code. I hope none of them listened, because as career advice in the Biden era, "learn to code" will be slightly less useful than "learn to play the accordion". For God's sake, there are only 1.46 million software development jobs in the whole country. Ask yourself a question in the format the meme kids love:

What percentage of software development jobs will be given to new visa holders, and why is it 100?

Like him or loathe him, Trump was good for middle-class American jobs, particularly in tech. I watched pay rates increase by a full third during the first two years of his administration, and had I stayed in tech rather than departing for the editorial lyfe, yo, I could have looked forward to further raises.

If you're wondering why Big Tech mounted such a full-court press against him, now you know. It wasn't to protect America's womyn, nor was it to ensure the dignity of (insert your favorite group here). It was to reset labor costs back to the Obama years, and then some. Just as importantly, it was to take white and Black employees out of these jobs and replace them with people who can be dominated via the iron band of visa control. If you've never worked in tech, you've never seen how that control is used. The visa holders are the first people in the office and the last ones out. They never raise their voices to disagree, they never refuse a task no matter how degrading or unnecessary. It's actually terrible for software development, because without anyone to say "No" you wind up with catastrophically complicated projects. But it makes the bosses feel goooooood.

Whatever. It's done. Elections have consequences. Onwards with The Great Reset, amirite? Nevertheless, this Brave New World will force me to do at least one dangerous thing, and it's this: I have to disagree with Scott Locklin.

Ninety-nine percent of the time, I have no interest in disagreeing with Scott Locklin. Insofar as you can ascertain someone's IQ over the Internet, he is a 3SD typa dude, meaning one of the 13 in 10,000 people with an IQ over 145. He might even be in that 4SD group with your not-so-humble author. It's hard to say. The IQ test isn't reliable at those levels; furthermore, I have long suspected that 4SD IQ is actually a form of mental illness. You're way outside design parameters at that point. I can personally attest that after a few dozen concussions, each one making me stupider by some degree, I'm actually a lot happier than I was as a child when I had perfect and total recall of everything from entire fiction books to the arrangement of the bricks in the window outside my classroom.

Mr. Locklin just wrote something really fascinating, as he always does, and this time it's about woo for its own sake. When Locklin writes "woo", he usually means "emotional stuff, with no basis in fact", such as astrology or personality testing, but in this case he means "software written with no thought to practical use". I'll excerpt a few of the most relevant parts for those of you who don't like reading about computer software:

Once in a while, independent minded programmers demand more. They may or may not be “so fookin smart,” but they think they are. Their day jobs consist of unpleasant plumbing tasks, keeping various Rube Goldberg contraptions functioning and generally eating soylent and larva-burgers and claiming to like it. As such, most programmers long to do something fancy, like develop a web server based on Category Theory, or write a stack of really cool lisp macros for generating ad server callbacks, or add some weird new programming language of dubious utility to an already complex and fragile stack. . Allowing your unicycle-riding silver pants mentat to write the prototype in Haskell to keep him from getting a job at the Hedge Fund may make some HR sense. But if you’re going to rewrite the thing in Java so a bunch of offshore midwits can keep it running, maybe the “adulting” thing to do is just write it in Java in the first place. . One of the valuable things about popular but boring languages is that the code has been traversed many times, and routine stuff you’re likely to use in production is probably well debugged. This isn’t always true, but it’s mostly true. The other benefit to boring languages is people concentrate on the problem, rather than the interesting complexities of the language itself. . If you have a job doing shit with computers, you are presumably solving real world problems which someone pays for. Maybe, you know, you should solve the problem instead of being a unicycle riding silver pants juggling chainsaws.

In the second half of the column, Locklin discusses the addictive nature of complexity in software and computing infrastructure. He writes about "the towers of jelly that are modern OS/language/framework stacks" and, dear reader, let me tell you that, as the kids say, I felt that. At a job I had a while back, our management came up with this profoundly stupid idea for deploying Automated Teller Machine software to the ATMs:

the physical server contains the virtual server which contains the Docker image which contains the Kubernetes image which contains the Docker image which contains the application.

I suspect I only got two responses from anyone who just read that:

People who had their last tech job before 2015, or no tech job at all: uhhhhh wat

Current tech workers: yeah, it sucks, we're doing that at my job too

The tech business has fallen in love with abstractions and layers because it is widely supposed in theory, but never proven in practice, that these architectures can be successfully operated by people with a 110 IQ. They believe this because the idea of this architecture is always sold by someone with a 105 IQ to someone with a 104 IQ, on the hopeful basis that someone just a little bit smarter than both of the idiots in that transaction will be able to effortlessly make it work.

Their goal, of course, is to get rid of the Scott Locklin in the department and replace him with three visa-holders who will be considerably easier to hire, manage, and fire.

Now here's where it gets weirder. Docker and Kubernetes (we say k8s, of course!) were probably originally the private project of someone like Scott. In my brief time working on the pointy end of tech, I learned that all the brightest bulbs were perpetually working on some computationally unlikely project to solve a problem that really only existed in their own heads. At Google they call it "twenty percent time", and it's meant to be a reward for the "eighty percent time" you spend doing what Scott calls, quite rightly, the "unpleasant plumbing tasks".

In the end, you get this kind of hierarchy. The 170 IQ dude has a vague idea about computing. He tells it to the 150IQ dude, who conceptualizes something like Kubernetes. It is then put into practice by 130IQ people, debugged and completed by 120IQ people, and sold to the 104IQ guy by the 105IQ guy on the basis that the 110IQ people can make it work.

The 110IQ people make a giant mess of it, so they find the lone 120IQ person at their visa body shop who can fix it, and he does so behind the scenes, but it isn't fixed well enough to actually work in practice, and that's how you get a Tower Of Jelly.

Let me show it to you:

This is Amazon's recommended architecture for implementing Drupal, a shit-simple CMS that you could install in half an hour on your grandmother's Celeron desktop, from scratch, including all the Linux OS you'd need to run it. This uses at least eight, possibly nine, of Amazon's branded AWS services.

The purpose, as anyone can see, is to drastically increase Amazon's profitability by selling you eight or nine services for every interaction your users have with your Drupal server. That's why Amazon recommends it. Literally. I'm serious. This is Tru-Coat and the extended warranty and the nitrogen for the tires.

If you've ever actually sold Tru-Coat, like I have (well, it was Ziebart, but you get the idea) then you know that the biggest problem with Tru-Coat isn't how much you pay for it but rather how badly it's done. I delivered a fair number of cars that had rustproofing sprayed inside the wheels, thus making the car shake and shudder from the moment it left the dealer.

Similarly, the biggest problem with the Amazon diagram isn't how ridiculous it is, or even how expensive it is, but rather how badly it's implemented in practice by the midwits-and-below who find themselves doing the work. If you're lucky, the whole duct-taped assembly will run like it's on Grandma's Gateway 2000. If you're not lucky, then you get phantom interactions of complexity. Downtime no one can explain. Database behavior that isn't discussed in the manual. And so on.

The careful reader, who has outstanding pattern recognition, might be starting to see a common theme running through all of this, from Locklin's unicycle-rider business to the wacky Drupal diagram, and it's something like this:

There is always a tendency in modern computer stuff, among people of all ability levels, to complicate the project.

Locklin's "mentats" (that's a Dune reference, God bless him) want to write everyday services in new computer languages, many of which are highly optimized for considerably more complex tasks. The "midwits" want to make giant Amazon diagrams. The dummies want to sell each other Russian-nesting-doll strategies. At no point is there ever any thought given to how well something will actually work.

This problem is addressed in Locklin's piece. He draws a distinction between the engineers who make things run, and the everyone else who want to complicate things for their own self-aggrandizement/amusement/resume-padding. Then he suggests that everything important should be left to the engineers.

So where do we disagree, exactly? Only in this. Locklin, in his column, suggests this approximate order of desirability:

Engineers writing stuff in the appropriate language > Midwits using Java > Mentats using Haskell or another hobby language

I'd argue that his middle choice, Midwits using Java, always fails. It has zero chance of being effective in the long run. Whereas the last choice, Mentats using Haskell, at least has a chance of success. It could work.

If we translate this to infrastructure, Locklin's assertion is like this:

Engineers using simple apps on bare metal or VMs > The 110IQ crowd using Docker > Someone from GNU writing a new clustering strategy

Again, the middle option has a 100% chance of failure at some point, and the right side option might succeed. Therefore, I would like to meekly suggest that we re-order both of these, to

Smart people doing tried and true stuff > Really smart people making it up > Non-smart people following a recipe

This not only lines up more closely with my 25 years of professional experience, it also lines up with my personal experience. As you increase the intelligence of a person doing a job, that job is done better --- up to a point, beyond which it is done worse. I believe that this rule is probably absolute, and would apply to theoretical physics as well as it applies to running a forklift, except that we are simply short on the human capital necessary to demonstrate it at scale. If there is a "200IQ" person someone out there, I bet he's working in a fast food restaurant, because he's terrible at every task and fast food is one of those jobs where we accept low performance.

Which brings us back to Wednesday and the likely future effects. One of the downsides of completely outsourcing software and computing infrastructure in this country is that you take the 1.5 million people who might be pretty well-equipped, intellectually speaking, for such a job, and you force them into a "dumber" job. They won't do this "dumb" job as well, because they are on the far side of the curve for that job.

And what about the 1.5 million people who got moved out of their jobs by the more intelligent, better-credentialed ex-engineers? Well, they go to a dumber job, at which they are not as good, and so on. It's a knock-on effect that hurts the whole economy. Everyone's life gets just a little bit worse, more degrading, less interesting to them.

And that is why I come to praise "woo" of this type, not bury it: because it represents the antithesis of forcing everyone into the worst job we can find for them. It holds out the promise for satisfaction, however temporary and however frustrating after the fact. It places a value on intellect, and on intelligent effort. There's something innately human about it. And when it doesn't work, it's no worse than the default choice, which also doesn't work.

What does a world without this kind of "woo" look like? Keeping in mind that it's not just software "woo", it's everything from architecture "woo" to street-sweeping "woo" like you see in Japan. What does it look like if you privilege the midwit over the Real Smart People? Well, everything just gets that much worse. The music is moronic, tuneless, crass. The poetry is without rhyme or reason. The movies are tedious exercises in special effects and jump scares. Every flight is late, every subway is dirty, every business process struggles under the parasitic weight of government and social requirements. We have ugliness shoved down our throats and we are told we must pretend to see it as beautiful. You lose the freedom to say that two plus two equals four.

Winston Smith wrote that "If there is hope, it lies in the proles." That's not true. It never was, not in 1776 and not now. Here's the truth: if there is still hope, it lies in the woo.

* * *

This is a rare Weekly Roundup, because we have a special corner case article to share. In addition to what I wrote for Hagerty --- a complaint about timidity in club racing --- we have a lovely profile of a Riverside Green commenter to share! I sent my protege, Grace, to meet Toly Arutunoff. I know you'll enjoy her story.