[Klik hier voor de Nederlandse versie van deze post]
The rational individual is an ‘optimization machine’ who chooses the right means to achieve a given goal. A problem is that people turn out to be not rational at all, which has motivated to the construction of optimization machines in order to pursue rationality. The first form of construction is that of computers. The second form is that of smart social structures, such as the free market. However, by removing the ability to make ‘good’ choices from people, moral problems arise. Especially the moral autonomy of individuals is challenged. With that, there are plenty of reasons not to focus too much on any idea of rationality and there are even more reasons to question the way these ideas are materialized in machines and structures.
−
Once upon a time, rationality was what defined us as human beings. After all, animals were not rational. To be fair, most people weren’t either, by but if there was anything rational, it had to be man. This focus on the ratio allowed for the Enlightenment to take off. With our special mind, we were able to free ourselves from all those dogmas that kept us under control. Science, democracy, ethics, all were based on the assumption that man was in control of his destiny thanks to his superior intelligence. And yes, I write ‘ his’ here, because the big thinkers were not that enlightened that they realized that women have the same capacity for thinking as men.
But what exactly makes human thinking rational? First of all, the mind is something that belongs to an individual. Society is no more than a loose gathering of thinking individuals, they have entered into a ‘social contract’ out of self-interest, but in the end, living together remains a necessary evil.
Rational thinking is also seen as something that goes against emotions, because it is believed that emotions are not intentional, but atavistic intuitions. Emotions are primarily a disturbance of thinking, rather than a form of thinking itself.
The separation between ratio and emotions returns in the separation between mind and body, where the body is given the role to obey the assessments of the mind. In this, the mind is increasingly identified with the brain, which is seen as the organ in which all thought takes place.
As a side note, this central position of the brain only seems to get stronger with the rise of brain scans, making neural activity traceable and measurable. This is a self-fulfilling prophecy, because if we think that thinking only takes place in the brain, we will measure it only there and we have become so good in doing tis that we don’t feel the need to look further.
Rational thinking also is a linear process in which ‘input’ is converted into ‘output’. This approach to thinking emphasizes the conscious mind as the central command centre of human action. Consciousness then means that the thinking person knows what he thinks and knows what he wants, he can consider the range of alternative options and therefore decide what the best way is to achieve a certain goal.
This linear process is made possible by the way the brain is conceived by creating ‘internal’ representations of the external world. This vision of representations fits in seamlessly with the conception of this external world as being made up of seperated elements that can be interpreted in a clear way.
−
Rational thinking lends itself fully to utilitarian ethics, in which individualism is combined with the pursuit of the optimization of a certain goal – happiness – given the resources available. In short, the rational brain is above all an optimization machine. But as I have argued elsewhere, our brain does not work rational at all. In fact, our thinking is far from optimal.
To emphasize this last point, Herbert Simon introduced the idea of ‘bounded rationality’, pointing to our inadequate ability to oversee all possible options when we make a certain choice. Instead of an optimizers, humans are ‘satisficers’. They apply ‘heuristics’, shortcuts that allow the full range of options to be brought down to a manageable clear number. Irrevocably, such heuristically-limited thinking leads to suboptimal solutions.
Daniel Kahneman and Amos Tversky point to the many ‘biases’ we have when making decisions. Instead of using ‘system II’, the most rational cognitive part of our brain, we tend to use ‘system I’ – the lazy system that tends to make mistakes.
The derogatory terms ‘bounded’, ‘heuristics’, ‘biases’, they all indicate that natural, human, cognition is seen as the deviation from the norm. It is clear that if we look for rationality, we will not find it in the brain.
−
If our brain is no optimization machine, then we will build one. Enter the computer. It is no wonder that Simon had high expectations of the intelligence of the artificial kind, since this was only limited by Moore’s law: computing power will only expand, which makes convergence to rationality nothing but inevitable.
For Simon and many others with him, the pinnacle of human thinking is the game of chess. The ultimate goal of artificial intelligence was to beat the human with chess. That would prove how rational a machine could be, a quest that was already successful in 1997 when IBM’s Deep Blue won from world champion Gary Kasparov. We had been overtaken.
But why the excitement? After all, computers are precisely built to deal with problems such as chess problems. These are the kind of problems that fit in perfectly with our idea of rationality, in which fixed rules can be applied analytically without being disturbed by any context whatsoever. The kind of problems that we humans are bad at, very bad. Of course a computer wins, just as a calculator is better in arithmetic than a human.
If we then look at a so-called dumb sport, such as football, the reputation of computer fades fast. Even the eleven most advanced robots, each paired with its own supercomputer, will lose to a team of veterans of which the players base their decisions entirely on their lazy ‘system I’.
Ultimately, computers were developed based on the assumption about how the brain works. Namely, that thinking is a process that, uninterrupted by emotions or any external influence, converts input into output. Now that assumption turns out to be wrong, but that does not lead to an adjustment of our image of rationality, on the contrary, it is our cognition that is now being labelled as inferior. This is weird. Why not see the computer as an inferior form of thinking?
−
It doesn’t stop with the computer. More rationality is also sought by designing ‘smart’ social structures to remedy the shortcomings of our inferior brain. So we also build social optimization machines.
The motivation for this is that while individuals, independently of each other, invariably make the wrong decisions, a collective is capable of making the right decisions. This insight was presented by Francis Galton in an article from 1907 called Vox Populi that appeared in Nature. This article describes how the weight of a bull was estimated at a fair. Every individual guess was wrong, but oddly enough, the average of the estimates turned out to be correct. According to Galton, a democratic form of decision making produced more reliable results than you would expect.
To see how unexpected that was, you only have to look at Gustave Le Bon’s work The Crowd that appeared 11 years earlier. For Le Bon, the masses were mostly stupid. Individuals lost their ability to think independently as soon as they became part of a collective. Instead, individuals began to display traits such as “impulsiveness, irritability, incapacity to reason, the absence of judgment and of the critical spirit, the exaggeration of the sentiments, and others besides — which are almost always observed in beings belonging to inferior forms of evolution — in women, savages, and children, for instance”.
Le Bon confirms the conventional rationality model, including its appaling misogyny and racism. Galton’s findings yielded a crack in this model, but Freud’s psychology went much further. Human thinking turned out not to be rational at all, but decisions were made from the deepest layers of our subconscious mind. Numerous empirical findings have directly supported Freud - at least in the sense that psychologists, economists, neurologists have shown that the individual’s thinking is not at all rational.
While man has become irrational, the masses have only become more rational. In 2004, the journalist James Surowiecki published the well-known book Wisdom of the Crowds, in which Galton’s classic article is systematically elaborated. A crowd can be wiser than an individual because in a collective individual shortcomings are levelled out. After Surowiecki, many thinkers have been concerned with how to organize the social collective in such a way that the collective decision is the right one. What is especially important is that people do not influence each other, so that the statistical distribution of judgments among different individuals remains respected.
Now people, weak as they are, tend to be influenced by others – those are the evolutionary shaped heuristics that keep us from being rational. But with the right structures and the right tricks you can bypass these heuristics.
For example, you can ‘nudge’. By designing the right ‘choice architectures’, we can choose ‘better’. By placing fruit at the cash register instead of candy, we opt for the healthy alternative, by making saving for a pension mandatory, we do not spend all our income in one go. It is assumed that the decisions we make are not the ones that we really want, but that choices are imposed on us by our inferior brain. Our ‘true’ choices are the choices we would make if we had been rational. How we then know what is rational is a question that is usually not being asked.
−
That the crowds can be wise is because people disagree with each other – they eliminate each other’s misconceptions. We have known since Socrates that divergent knowledge claims are necessary to become smarter. A dialogue, either between people or between the different voices in your head, shows things from different sides, so that you get a better understanding of a certain issue. The dialogue therefore seems to be a smart structure because it makes us think better as individuals. But that idea is abandoned when it comes to the wise crowd. The collective itself has become smart, the people that make up the crowd remain just as dumb as they already were.
The prime example of a smart social structure that corrects individual biases is that of the free market. In this, individuals pursue their self-interest, individuals who want as much as possible for themselves, without worrying about the fate of others. That’s nice, because by only caring about themselves makes them much less susceptible for external pressures than in other social structures, such as those of a centrally controlled bureaucracy. In such a bureaucracy, the capacity of people to influence others is given free rein. With all the irrationality that follows.
Also here, one sees a remarkable reversal of the rationality model. For Max Weber, who characterizes modernity as ongoing process of increasing rationalization, the bureaucratic organization was precisely the paradigm of this process, because such an organization acts as a rational decision-maker in which the single administration analogous to the individual brain arrives on the basis of the correct information. These decisions are subsequently implemented by the executive units of the organization.
The free market has long been seen as a necessary evil. An unstructured collection of individuals driven by greed. Of course, Adam Smith already showed in the eighteenth century that a free market can lead to an optimal level of general prosperity, but the idea that a bureaucratic organization is more rational has survived until the fall of the Berlin wall, less than 30 years ago. This historical event showed that a centrally controlled system was not able to function properly at all and did not stand a chance in the long term against the adaptability of the free market.
The neo-liberal creed that has been dominant since 1989 states that rationality is present in the market and, in fact, nowhere else. The fact that companies have now become bureaucratic juggernauts that surpass most state organizations in terms of size and power is thereby conveniently ignored. That individuals are crumbled as marginal beings does not seem to harm neoliberal thinking.
−
In summary: science showed the failure of the rational brain, history showed the failure of the rational organization. The computer is the designated replacement of the brain, the market the designated replacement of bureaucracy.
I find these turns strange and, as should be clear by now, also undesirable. As described above, they go at the expense of the human dimension. Either by portraying people as inferior thinkers, or by making the interests of people subordinate to the interests of market organizations. This while the human measure remains the measure of things, at least for me.
There are also other important moral merits that are at stake here, namely our ability to make choices and bear responsibility for these. After all, the Enlightened thinkers granted the individual the ability to make decisions independently, that was not so much an empirical description, it mainly had a moral purpose. After all, if an individual person can make a deliberate choice, it will become possible to hold individuals responsible for their choices. People can be called to account for their decisions because they have been taken on purpose.
Isn’t it a problem then that people don’t turn out to be rational? Not if we rely on Jürgen Habermas’s approach to rationality. For him, a rational decision maker is part of an instrumental rationality that is subservient to a communicative rationality based on the ability of people to reach agreements on norms and interests via deliberative processes.
From the viewpoint of communicative rationality, it is no problem that we do not make our choices consciously, because we may be asked to give reasons for those choices afterwards. It can then be determined whether these reasons are legitimate or whether it would have been too much to ask for these reasons, in the latter case the person in question has proved to be unaccountable. Our legal system constantly examines which reasons are legitimate and where the boundary lies between liability or not. Thus the idea of a rational decision maker has turned out to be an intermediate step in the development of modern institutions, an intermediate step that has proved to be unnecessary and for which we now have much more fine-grained theories.
But still we are stuck with the idea of rationality, which has now been transferred from the individual to machines and social structures. In addition, the moral responsibility for individual choices has been reduced to an instrumental responsibility for the most efficient choices: computer, nudges, the free market, they are all aimed at optimizing choices.
The major problem is that our autonomy is being affected here. After all, we do not make the choices ourselves as individuals, no, those choices are made for us. The proponents of smart systems, often thinkers that have a utilitarian outlook, defend their position by stating that the choices that are being optimized are things like well-being, prosperity, a longer and healthier life. And who doesn’t want that? If an individual autonomy is infringed, it is primarily the autonomy to make stupid choices.
In addition, individual autonomy was based on the idea of a rational decision maker, but that is precisely what an individual is not. Man is not at all autonomous, but a slave to all kinds of physical processes that take place in the brain without being conscious about them. That forwards another reason why it is only valuable to intervene in these processes from outside.
Even the communicative rationality of Habermas is not at stake, because which values need to be optimized is precisely a question that lends itself to a deliberative process to which everyone can contribute.
−
This way objections can be explained away. But in a very naive way. Firstly, values and goals are never completely independent of each other. There is only an analytical distinction, but in the real world your goals are largely determined by the resources you have at your disposal.
Moreover, all technologies and all social structures have intrinsic values. In the case of the optimization machines described here, the most important value is rather obvious, since that is optimization itself: efficiency becomes the central goal that must be pursued in whatever capacity.
In addition, technologies and social structures can handle some values much better than others. In smart systems, for example, everything is usually interpreted in terms of flows that run from one point to another. This is because such flows allow themselves to be optimized quite conveniently.
With this point, we also come to the problem that optimization machines tend to become intertwined: we are increasingly witnessing how ICT-systems and social structures are connected to each other in smart systems. In this, it is not the case that values can be easily adjusted in these interconnected optimization machines, also if new values are called for. In the first place, these systems are adapted to each other by having commensurable forms of input and output, you can’t just ignore that.
This means, for example, that in a ‘smart’ environment you can’t just hang around on the street, you are expected to be on the move, going someplace in the most efficient way possible. Anyone who fiddles around is suspicious, perhaps that person can be sent in the right direction by nudging him. camera surveillance perhaps, street furniture on which you can’t really sit comfortably, devices that make an annoying sound.
−
We cannot simply introduce technical and social optimization machines that only make life better, these types of systems intervene in our ability to make choices as individuals and to be held responsible for those choices. However, the belief in the blessings of ICT and smart social structures such as the free market means that these ethical aspects are hardly discussed. That seems unfair to me and, in any case, it is hardly optimal at all.(Pesch & Ishmaev, 2019)
−
Further reading:
Ensmenger, N. (2012). Is chess the drosophila of artificial intelligence? A social history of an algorithm. Social Studies of Science, 42(1), 5-30. doi:10.1177/0306312711424596
Galton, F. (1907). Vox populi (the wisdom of crowds). Nature, 75(7), 450-451.
Habermas, J. (1985). The theory of communicative action: Volume 2: Lifeword and system: A critique of functionalist reason (Vol. 2). Boston: Beacon press.
Kahneman, D. (2011). Thinking, fast and slow. New York: Farrar, Straus and Giroux.
Le Bon, G. (2017). The crowd: Routledge.
Pesch, U., & Ishmaev, G. (2019). Fictions and frictions: Promises, transaction costs and the innovation of network technologies. Social Studies of Science, 49(2), 264-277. doi:10.1177/0306312719838339
Schreurs, P. (2000). Enchanting Rationality. An Analysis of Rationality in the Anglo-American Discourse on Public Organization. Delft: Eburon.
Simon, H. A. (1997). Administrative behavior. A study of decision-making processes in administrative organization. New York and London: The Free Press.
Solomon, M. (2006). Groupthink versus the wisdom of crowds: The social epistemology of deliberation and dissent. The Southern Journal of Philosophy, 44(S1), 28-42.
Surowiecki, J. (2005). The wisdom of crowds: Anchor.
Thaler, R. H., & Sunstein, C. R. (2009). Nudge: Improving decisions about health, wealth, and happiness: Penguin.
