Confronted with the news that the Russians asked the US to look into possible violent activities by Tamerlan Tsarnaev, the elder Boston bomber, you might wonder why he was able to wreak havoc in Boston.

The answer is the same as for any person drifting toward psychopathic behavior or murder — the tell-tale clues are only visible in hindsight.  Probably hundreds of people fit the profile of Tsarnaev, none of whom go on to commit terrorist acts.  The problem is we have no way to tell which one will.

The world is, inherently and irredeemably, complex.   By this I do not mean complicated.  Sending rockets into space is complicated; analyzing the human genome is complicated.  But these remain systems that are analytically simple — you can pull them apart into many small elements, and the total of the system is the sum of those elements.  The problem with the social world is that it is complex in the sense that the big picture does NOT simply emerge from adding up the elements.  In decoding the human genome, there are basically four nucleotide bases that appear as the basic terms in DNA sequencing, and the hard part is to take the millions of bases in a DNA molecule and figure out their order.  We do this by splitting many strands of DNA from an organism into smaller pieces, identifying the sequence in those pieces, looking for overlaps, and then putting the whole thing together from the bits.  Fortunately, every strand of DNA from the same organism has the same sequence (that is, except for random mutations), so get enough strands and enough pieces and enough computing power and you can figure out the sequence of the entire molecule.

Now imagine how much harder this would be if each nucleotide was an individual thinking and acting bit, that could change its order at will, disguise itself or pretend to be another base to fool observers, and in which no two sets of interactions were exactly the same twice.   The whole system we use would be useless.

It would be nice if the analysis of terrorism, or the psychology of individuals, was amenable to accurate predictions, like that of rockets or DNA.  But the phenomena are just different, and always will be.

In fact, people find it immensely hard to deal with situations that involve more than a few variables, especially when those variables can interact with each other and act in non-linear ways (another way of saying you can’t simply add up the bits to get the whole).    This is why interventions to improve society so often fail.

I recently learned one reason our education programs in Afghanistan went so badly.  Citing a study showing that investing in primary education produced the greatest return, better than investing in secondary or tertiary education, USAID decided to put ALL of its educational investment into primary education — after all, that gives the best bang for the buck.  So they set a target of getting ALL Afghan children into primary school.

Yet that program has been in many ways a failure, because schooling is not the same as education.  That is, you can herd children into a newly built school, but if you have inexperienced, barely literate teachers who show up only half the time, very little learning goes on (a recent Center for Global Development study shows that this a common problem in developing countries).

Yet if you seek to expand primary education by millions of children, how can you expect to teach them unless you train hundreds of thousands of teachers?   The math is simple:  if you put 5 million children in school, and you aim for a 50 to 1 student to teacher ratio, you need at least 100,000 teachers, not counting administrators, substitutes, and other support staff.  But how do you get those people if you invest nothing in secondary and tertiary education?  Answer — you don’t.  And your goal of actually providing primary education falls to the practice of setting targets for bodies in classrooms, never mind about actual learning.

Why does such folly occur?  Because to deal with the problem requires thinking in terms of complex systems with many interrelated elements — how many secondary and tertiary teaching graduates do you need, and how fast can they be produced, and how can we adjust the expansion of primary schooling to the availability of qualified teachers?   That is a complex problem; among other things it may take years to determine how many people actually complete a teaching credential among those that sign up (in the US, the drop-out rate from colleges is over 40%, largely due to admitting students who are not prepared to do college work).  So how you can you tell how fast teachers can be produced?

It is much simpler to simply pick a target that CAN be approached in linear fashion – you build a school, get students to attend, and the total number of students schooled is just the sum of those bits.

All too often, companies and governments and individuals in all walks of life make decisions on the basis of habit (this worked before), rules of thumb, easily measured targets, simple analogies, and other approaches that lend themselves to few variables and linear thinking.

You might say that we should then put people in charge who can think in terms of many dimensions, non-linear relations, and complexity.  And you would be right — except for the fact than many such folks are academics who cannot manage people at all, and who master this approach because they only focus on a tiny slice of life and can’t generalize it to real-world problems.  What you need are managers who understand complexity, can harness the judgment of experts to guide their decisions, and then manage people to effectively carry out those decisions.  That combination does not arise very often.

So we should be very glad that for most of the things we do — growing food, building highways, generating electricity — the world responds pretty well to simplifications that involve breaking down complicated problems into lots of smaller simpler problems and that the solutions to the simpler problems can be added up to create the solution we need to the complicated one.

Sadly, that just does not work well for truly complex problems – like picking out in advance, from hundreds or thousands of jihad-influenced individuals, which ones will actually set up a bomb or where and when they will do so.

So don’t be too hard on our homeland security folks — they have done a pretty good job.  And learn to accept that the real, living, breathing world of people is darn hard to manage.  The main thing we can do is always be learning from, and resilient to, the things that we will inevitably miss.