Triumph of the Idiots

At my last job I got into several arguments because I would be "too willing" to call some people "idiots". (I'm not completely useless at matters of team management, I didn't shout "idiot" at people's faces, but sometimes shouting "idiot" does wonders for morale - my morale at any rate.)

The people who got offended by this weren't usually the other developers, they (mostly) agreed with me; not always though, but most of the time. The offended were usually those who had been less than two years out of university, in the more woolly roles. It was to be pretty much expected that a sociology graduate tasked with writing "user stories" would see things differently from a cynical senior developer, they mostly took offense because I wasn't "taking into account others opinions". (There is a certain irony here, dismissing someone's opinion for not taking into account someone else's opinion.)

This thinking, that all opinions are equal is very dangerous. Some people are just idiots, that's just the how it is. Somehow the belief that "everyone is entitled to an opinion", which is perfectly true, it's certainly wrong to persecute people over their beliefs, has transformed into "all opinions are equal", which is completely wrong. Dead, dead wrong.

You see I would use the word "idiot" as a synonym for stupid, dense, a bit dim; someone with intelligence below that required to operate efficiently. And by "intelligence" I use a pure definition, intelligence is someone's capacity to learn.

On my plane of existence (which may well turn out to have a population of one), someone who has designed a complex multifaceted publishing system, but then complains about my schema design because "it has too many tables, you can't have more than three tables" is an idiot; especially when the system being replaced failed due to difficulties extending it, caused in no small part by an extremely bad schema. End of!

I don't mind guiding idiots in the right direction, not at all, I'm more than happy to explain why my way is superior. But no, I don't think he's got a valid opinion, and no I definitely do not think it's "worth a go", my way will work fine thanks, we don't need any alternatives and we definitely do not need to try the other idea first.

"It's because they're stupid, that's why, that's the reason everyone does everything"

They don't teach you how to deal with idiots in school, it's one of those missing subjects like financial management. (This is mostly because the people who design the curriculum are idiots.) Although in a way, they do... all the pointless set-piece exercises for science classes, measuring acceleration of a ball rolling downhill, that sort of thing; little did I know how similar that would be to real life.

Invariably we would be split into teams of five, out of this team there would be: 1 person who started work ahead of the others, measuring times, but forgetting to measure distances and therefore making the data useless; 1 person who would pointlessly suggest a better way of operating the stopwatch; 2 people thinking up elaborate ways to trick each other into drinking sulphuric acid; and me, shouting, "hang on, that's not the same start position, we need to write this down, that didn't take two minutes you forgot to reset the stopwatch, someone call an ambulance Bob's face shouldn't be that corroded."

That's pretty much the way most software projects work, in my experience. At my first set of job interviews before I had any real commercial experience, I would get asked about these exercises; the question would invariably be "who was in charge", or "how did you choose the team leader". There was no leader, there was no control, they were chaos. Same as software projects then, people may have titles, but almost never any power (usually because senior management doesn't trust the development team and pulls the rug from under any initiatives).

It was during Job No. 5 that I finally had the epiphany. Until that point I assumed project failures were mainly due to unexpected problems during delivery, after that point I finally realised that most projects were doomed at the outset, usually because people ignored risk - or, more specifically, were blissfully unaware of risks - and that was because of their stupidity. (Of course, if they'd managed to make the projects work, they would have been geniuses, but they never did work.) The only reason these projects even start is because everyone concerned is too stupid to know it will fail.

(I've met many clever contractors who've earned a fortune because of this, you can tell who they are a mile off, they're the ones insisting on an hourly fee rather than daily.)

Once you accept the idiot hypothesis, most of the planet makes sense. Did the credit crunch happen because of an unpredictable set of global macro-economic events, or was it because the senior executives at various banks are simply incapable of understanding the consequences of their actions? (It's become fashionable to condemn traders for their actions, but they knew what they were doing, they made the cash and moved it to the next bubble - oil, gold, etc. - they were behaving rationally, if greedily; it's the management of mortgage lenders who are the idiots, the ones who spent years making bad loans and were too stupid to avoid being left holding the bomb when the clock reached zero.)

And it goes even higher than that, why did the central banks let it happen, and let inflation run away at the same time? Was it because: a) they're the best economists in their field, the fact that they missed it proves the unpredictability of it all; or b) they are professional ladder-climbers sitting with their fingers crossed hoping to land the next job before it all goes wrong?

Intelligent people get squeezed out of organisations, given time. For some reason the combination of intelligence and ladder-climbing almost never co-exists. Senior executives of large companies often get described as being intelligent, but this is just mistaking outward signs of achievement with intelligence. Anyone can make money in a rising market, it's only those who seem to make a company grow when the odds are against them (e.g. Steve Jobs' return to Apple) are worthy of praise; the rest were just fortunate to be sitting on a boat in the rising tide.

Evidence of this can be seen in the competition between any large companies, this phenomenon explains why Microsoft can never quite do what Google does. It explained why IBM could never quite catch up with Microsoft. Etc. It's not that they became too large, or too complacent, it's because they became too stupid. They are collectively idiots. It'll happen to Google too, given time.

Most of the worlds great companies were founded by visionaries, powered by good people. But soon the Dead Sea effect takes hold. An interesting side effect is that growth and/or profits will almost certainly accelerate after this effects reaches a certain point, but only for a limited period. This is due to the "idiots" cutting corners, and therefore costs; this gives certain types of idiots are reputation as gurus, as troubleshooters. But by the time this happens the rot has set, and the company will never recover, although it's only fatal in the most extreme cases, it's more of a permanent disability. There is usually enough legacy assets to allow the company to stumble on indefinitely.

The combination of all of this, that intelligent people are almost never found in the upper ranks of large organisations, creates a lot of the geek/nerd stereotypes, for example that "programmers have no ambition, they just like to code" - or to translate: programmers cannot out stupid the stupid, and end up taking a personal responsibility to make a system work, despite the lack of respect and recognition - that intelligent people somehow get stuck in a rut, and like it.

This just isn't accurate, intelligent people are far more likely to be freelance or work for organisations with flat power structures. The exact sort of places which don't have rigid concepts of organisation, that don't have an "Org Chart" on the wall, the very opposite of being stuck in a rut. (Although being stuck in a rut is a common symptom of "got to get a new job disease".)

Islands of intelligence

The consequence of all this is that I cannot understand how humanity is still here, I really can't. How on earth have we not all been vapourised in a nuclear accident, and why do buildings not fall down more often? And, considering that we are still here, why is my gas bill always wrong - surely reading numbers from a meter is not too much to ask.

The answer is there are clever people out there somewhere, they do exist, they are rationed, that's the problem. I find myself more and more gravitating towards these oasis of anti-denseness. Places like Radio 4, although they have their moments, random history programmes on BBC2 or Channel 4; any form of media that makes a faux-pas of bad research is almost guaranteed to lose me forever (practically all forms of mainstream news, for example).

Today I've been impressed by some news reports from Formula One, in particular one about a new steering wheel. It's so simple it's genius. There's other less simple expressions of genius too, for example look at the rear wing on this picture, it defies simple analysis, it's almost as though it wasn't designed by a human. For one person, or even a large team, to go through every permutation of size of wing, shape of end plate (which is the most unorthodox and impressive part), would take a very long time. It's not known how it is designed, all the teams (McLaren especially) are very secretive, but it looks to me like the end result of several generations of a genetic algorithm. (For comparison, look at the rear wing of the worst car in the pack - completely flat and almost square end panels.)

Just like Test Cricket, no one would invent Formula One in it's current form if it didn't already exist. It's ludicrously expensive, the sum of all the development budgets are in the region of $2bn/year. But it's unique, which is why it's still going, why it managed to survive Michael Schumacher's one-man five-year reign of tedium, and why the current season is more open and interesting than all the "more exciting" forms of racing.

It's also very good for the "there are some clever people after all" factor. Although I used to work with someone who spent an internship at the Arrows F1 team, they weren't full of clever people, if you believe his stories; they were full of "in it for the lifestyle" people, which is probably why they never won a race...

The governing body is aware of the cleverness factor, and the fact that many common road technologies were pioneered in Formula One. Next year introduces rules which limits the teams ability to faff about with new aerodynamic components for every, single, damned, race (at vast expense), but instead opens up a whole new world of experimentation in the form of kinetic energy recovery systems. By focussing all that money on an "environmentally friendly" technology which could be applied elsewhere, the governing body is aiming to put Formula One at the top of a patch of moral high ground.

Development is not going well so far, one factory had a fire, a mechanic has been electrocuted; and there are concerns that a flywheel based approach could disintegrate and send razor-sharp shards into the crowd. But it should ensure that things remain interesting from a technology stand-point for many years to come...

Conclusion

So how does any of this apply to anything, either the world in general, my current predicament, or other people in a similar situation. I don't know, I don't think it does. I'm not going to become a mechanic anytime soon (ever); although I would like to know how the Formula One race modelling tools work. This is the software that's constantly running which tells the teams the optimal times for pit-stops, etc., it updates every two seconds taking into account live timing, positions, weather conditions, and so on. If I had something similar I could plug it into my Betfair bot and make shedloads!

[This was going to start as a couple of interesting anecdotes I wanted to write down, but it's turned into a Paul Graham-esque essay, which then turned into a vaguely connected stream of consciousness; but that's how it goes sometimes, blogging FTW!]