Tristan Schmurr/Flickr

World News


More is at Stake than just Privacy in China’s Social Credit System

A recent report that Chinese citizens have been banned from buying travel tickets 23 million times as part of the social credit system, together with calls from Chinese policy advisors to ramp up the rollout of the system makes it clear that the system is coming, and it’s coming in force. Since the planning outline of the system was released in June 2014, the social credit system has been a constant fixture in Western media.

Early commentaries on the system focused on the systems intrusive nature, while more recent ones have toned down the Orwellian nature—of at least the current system—and introduced a more nuanced picture. What these commentaries have in common, though, is a more or less one-sided focus on the system’s clash with privacy. There has not been much, if any, discussion whether the system will actually work. Will the system really be able to create a culture of trustworthiness in Chinese society, as desired by the central government? I believe trustworthiness; yes, but trustworthiness? No.

The early implementations of social credit systems in China suggest that the finalised system will make use of a large number of sanctions and rewards to shape citizens’ behaviour. Plausibly, this includes restricted mobility, reputational costs, and reduced opportunity sets in terms of education and employment. These are all external incentives, and when they are introduced into domains in people’s lives previously governed by intrinsic motivation; problems abound. At least if you believe motivation crowding theory.

The basic idea is that the introduction of external incentives, such as money, can undermine and overtake people’s originally intrinsic reasons for doing something, such as giving blood, and lead to a decrease in performance of that activity. I will not go deeper into why this is the case; the important thing to note is that the social credit system could lead to less, and not more, socially desirable behaviour. But ultimately, I don’t think this will be the case.

The reason is simply that the scope of the system is so large that non-compliance is just too costly to bear. If jaywalking determines whether your child can go to a good school, you probably comply even though you think the regulation is a bit silly. In other words, even though your motivation for doing it has changed; you will still do it. So, the social credit system in China will probably be successful in inducing citizens to behave socially desirable: citizens can be relied upon to act in certain ways and there are all kinds of benefits because there is trustworthiness in society. But this ‘trustworthiness-as-reliance’ is not genuine trustworthiness. And this is important for two reasons.

Firstly, because people motivated to undertake certain activities primarily for external rewards will try to game the system. We know this from school accountability policy. Since the system can only monitor and shape what is readily observable and easily measurable, citizens are incentivised to undertake acts that may not be the most conducive to the public interest. Also, in situations that cannot be monitored due to technological constraints; no trustworthiness can be relied upon. And moreover, for activities that are monitored, people will try to manipulate the system—in fact, hacking to improve one’s score has already been observed in China. All of this suggests that the type of trustworthiness that is produced under the social credit system is shaky, to say the least. Whether that will be a problem depends on the exact specification of the final system.

Secondly, and I think more interestingly, is that the change in motivation can be seen as a problem in itself. While people can be relied upon to behave in a certain manner in the new system—as long as the sanctions and rewards are salient, the type of trustworthiness created is far from the fuller sense of trustworthiness where people can be counted on to “act from a trustworthy disposition.” This genuine trustworthiness, if we follow Nancy Nyquist Potter, is a character trait—a virtue—that is undertaken not for its good consequences or because it is required by duty, but because it is intrinsically good and an essential part of a proper human life. By imposing a social credit system, something very meaningful may then be lost in society: citizens’ ability to cultivate their character and live flourishing lives with meaningful relationships.

As China undeniably moves closer towards the introduction of a comprehensive social credit system with far-reaching consequences for how its citizens live their lives, it’s important to appreciate what is at stake. People’s right to privacy is one value. Another is people’s character. There needs to be more engagement with the many ethical implications of the system to provide a more complete picture of its impact; not only for China in the near future, but for similar systems in the West.