Image via Peter Parks / AFP / Getty
This is Niche Culture. In this column, we regularly cover anime, geek culture, and things related to video games. Please leave feedback and let us know if there’s something you want us to cover!
Fast Company’s Mike Elgan has alleged that “Silicon Valley” is collectively building a “social credit” system akin to the one being tested in China.
For those unfamiliar, the social credit system in China was unveiled in 2014, and gradually began numerous tests via eight major companies (including online shopping, banking, social media, and even dating) ready for its full implementation in 2020.
Much like how a financial credit score can be used to determine if someone can be trusted to pay bank a loan, the social credit score’s objective is to determine someone’s “trustworthiness.”
In short, the program “rewards” individuals who act as “good citizens”, while those who act poorly are punished. Those who fail to pay debts, or who are arrested for a crime for example would have their social score reduced.
The Chinese government stated the plan would “allow the trustworthy to roam everywhere under heaven while making it hard for the discredited to take a single step.”
International criticism came from what was deemed being a bad citizen. From government criticism, refusing military service, or practicing Tibetan Buddhism; to bad driving, jaywalking, or even buying too many video games.
The punishments themselves were also criticized, including bans from international flights, trains, hotels, throttling internet speeds, banning you or your children from higher education, being publicly named as a “bad citizen.”
Those who perform “good behavior” (charity, acts of heroism, paying taxes on time, and looking after sick relatives) are likewise rewarded. Travel applications being processed faster, or not needing to play a cash deposit in hotels, “receiving a $50 heating discount every winter, and obtaining more advantageous terms on bank loans.”
A writer at Fast Company, a “business media brand, with an editorial focus on innovation in technology, leadership, world changing ideas, creativity, and design. Written for and about the most progressive business leaders,” wrote an article on the 26th August.
Mike Elgan proposed several major technology focused companies are using similar “behavioral nudging” to the social credit system.
“Many Westerners are disturbed by what they read about China’s social credit system. But such systems, it turns out, are not unique to China. A parallel system is developing in the United States, in part as the result of Silicon Valley and technology-industry user policies, and in part by surveillance of social media activity by private companies.”
First, Elgan points at The New York State Department of Financial Services. He explains how they began utilizing social media at the start of this year to determine life insurance premiums. If an individual acts in a restless manner, the company can alter premiums- but never based on race or disability.
“That Instagram pic showing you teasing a grizzly bear at Yellowstone with a martini in one hand, a bucket of cheese fries in the other, and a cigarette in your mouth, could cost you. On the other hand, a Facebook post showing you doing yoga might save you money.”
Likewise, the company can also use your social media account to see if you lied about taking part in dangerous hobbies.
Elgan continues, discussing a company called PatronScan. While the company sells equipment to help manage bar and restaurant customers (across the US, Canada, Australia, and the UK), they also help detect trouble-makers.
“When customers arrive at a PatronScan-using bar, their ID is scanned. The company maintains a list of objectionable customers designed to protect venues from people previously removed for “fighting, sexual assault, drugs, theft, and other bad behavior,” according to its website. A “public” list is shared among all PatronScan customers. So someone who’s banned by one bar in the U.S. is potentially banned by all the bars in the U.S., the U.K., and Canada that use the PatronScan system for up to a year.”
Nonetheless, Elgan does not that individual owners are free to ignore bans, data on non-offending customers is deleted in 90 days or less, and bars may keep lists “private” and not shared with other bars (though information on “bad customers” is kept for up to five years).
Next, Elgan tackles “sharing economy” based services such as Uber and Airbnb. While both organisations (and similar ones) offer terms in which they can ban a user from their service, Elgan fears the system can be abused.
“Airbnb can disable your account for life for any reason it chooses, and it reserves the right to not tell you the reason. The company’s canned message includes the assertion that “This decision is irreversible and will affect any duplicated or future accounts. Please understand that we are not obligated to provide an explanation for the action taken against your account.” The ban can be based on something the host privately tells Airbnb about something they believe you did while staying at their property. Airbnb’s competitors have similar policies.
It’s now easy to get banned by Uber, too. Whenever you get out of the car after an Uber ride, the app invites you to rate the driver. What many passengers don’t know is that the driver now also gets an invitation to rate you. Under a new policy announced in May: If your average rating is “significantly below average,” Uber will ban you from the service.”
This is somewhat supported by journalist Tim Pool’s own experiences. When discussing a similar article on the New York Post on his YouTube channel, he discussed an alleged incident in which an Airbnb property owner threatened to have Pool’s account shut down with false claims, unless they were paid more.
Pool also discussed another alleged incident in which an Uber driver never picked him up, but claimed he did. Both of these alleged incidents gave Pool little evidence to defend himself.
Elgan’s next point of focus was WhatApp, suggesting you can be banned “if too many other users block you.” While Elgan does cite more serious reasons for being banned (“sending spam, threatening messages, trying to hack or reverse-engineer the WhatsApp app, or using the service with an unauthorized app”), he claims:
“In much of the world, it’s the main form of electronic communication. Not being allowed to use WhatsApp in some countries is as punishing as not being allowed to use the telephone system in America.”
Finally, Elgan lays out his issues with such systems. Answering his own question of “What’s wrong with using new technology to encourage everyone to behave,” he expresses his concern about such systems being punishments outside the law- and so operate with little or no safe-guards.
“The most disturbing attribute of a social credit system is not that it’s invasive, but that it’s extralegal. Crimes are punished outside the legal system, which means no presumption of innocence, no legal representation, no judge, no jury, and often no appeal. In other words, it’s an alternative legal system where the accused have fewer rights.
Social credit systems are an end-run around the pesky complications of the legal system. Unlike China’s government policy, the social credit system emerging in the U.S. is enforced by private companies. If the public objects to how these laws are enforced, it can’t elect new rule-makers.”
Elgan’s proposal is that as private companies have control over “societal privileges,” that it could lead to losing those privileges based on actions taken far-removed from the service.
“If current trends hold, it’s possible that in the future a majority of misdemeanors and even some felonies will be punished not by Washington, D.C., but by Silicon Valley. It’s a slippery slope away from democracy and toward corporatocracy.
In other words, in the future, law enforcement may be determined less by the Constitution and legal code, and more by end-user license agreements.”
Pool also proposes a similar concern at the start of his video, proposing a scenario where if a user is banned from a service (such as Facebook) resulting in them losing access to other services such as their credit-card being declined at a bar.
Comparisons can be made with “Operation Choke Point“, a US law that makes it easier for payment processors and banks to cut-off “high risk” businesses that may be engaged in illegal activity.
However, it has come under scrutiny due to claims of services shutting their service off from independent journalists (under false reports of hate speech or fake news). An anti-trust complaint issued against Patreon was dismissed in May of this year.
What do you think of the wider adoption of a social credit system? Sound off in the comments below!
