One of my Asian friends recently posted a link to a terrifying use of Machine Learning. This is what I call the “dark side” of this field, the use of machine learning by a government to make you behave a certain way.

1984’s Big Brother

China is building its own version of 1984’s Big Brother, a massive scoring system that’s probably a large scale classification algorithm most likely sitting on top of a Big Data structure like Hadoop. I call this an utter and complete abuse of machine learning.

The government hasn’t announced exactly how the plan will work — for example, how scores will be compiled and different qualities weighted against one another. But the idea is that good behavior will be rewarded and bad behavior punished, with the Communist Party acting as the ultimate judge.

Yes, we use classification and scoring system in everyday commerce. Banks use it to grade your credit card worthiness. Businesses use it to determine your propensity to buy their product. This project goes beyond all that.

Social Trust

China wants to learn if you say and do anything to break “social trust.”

They will scrap online sources available and assign you a score. This score will determine your trustworthiness. How they define trustworthiness remains the question.

This can manipulated and gamed. Questionable data quality can lead to misclassification and mistakes and Chinese people will suffer those consequences.

IMHO China’s drive for a harmonious society with a “Asian Big Brother” will only hasten it’s demise.

• AI Machine Learning
• Privacy