This approach, willful or not, has led to its latest scandal, where a previously available API for app developers was harnessed by Trump and Brexit Leave campaign technology provider Cambridge Analytica to pull not just the profile data of 270,000 app users who gave express permission, but of 50 million of those people’s unwitting friends.

Facebook famously changed its motto in 2014 from “Move fast and break things” to “Move fast with stable infra” aka ‘infrastructure’. But all that’s meant is that Facebook’s products function as coded even at enormous scale, not that they’re built any slower or with more caution for how they could be weaponized. Facebook’s platform iconography above captures how it only sees the wrench, then gets shocked by the lightning on the other end.

Sometimes the abuse is natural and emergent, as when people grow envious and insecure from following the highlights of their peers’ lives through the News Feed that was meant to bring people together. Sometimes the abuse is malicious and opportunistic, as it was when Cambridge Analytica used an API designed to help people recommend relevant job openings to friends to purposefully harvest data that populated psychographic profiles of voters so they could be swayed with targeted messaging.

Whether it doesn’t see the disasters coming, makes a calculated gamble that the growth or mission benefits of something will far outweigh the risks, or purposefully makes a dangerous decision while obscuring the consequences, Facebook is responsible for its significant shortcomings. The company has historically cut corners in pursuit of ubiquity that left it, potentially knowingly, vulnerable to exploitation.

And increasingly, Facebook is going to lengths to fight the news cycle surrounding its controversies instead of owning up early and getting to work. Facebook knew about Cambridge Analytica’s data policy violations since at least August 2016, but did nothing but send a legal notice to delete the information.It only suspended the Facebook accounts of Cambridge Analytica and other guilty parties and announced the move this week in hopes of muting forthcoming New York Times and Guardian articles about the issue (articles which it also tried to prevent from running via legal threats.) And since, representatives of the company have quibbled with reporters over Twitter, describing the data misuse as a “breach” instead of explaining why it didn’t inform the public about it for years.

Facebook CEO mark Zuckerberg

“I have more fear in my life that we aren’t going to maximize the opportunity that we have than that we mess something up” Zuckerberg said at a Facebook’s Social Good Forum event in November. Perhaps it’s time for that fear to shift more towards ‘what could go wrong’, not just for Zuck, but the leaders of all of today’s tech titans.

An Abridged List Of Facebook’s Unforeseen Consequences

Here’s an incomplete list of the massive negative consequences and specific abuses that stem from Facebook’s idealistic product development process. [Thanks to user suggestions, we’ve added some more in an upate]:

• Beacon = Your Shopping Revealed – One of of Facebook’s first big miscalculations was of people’s reactions to Beacon back in 2007. The feature let other websites people visited, starting with 44 partners, publish News Feed stories about their browsing and shopping behavior back to their Facebook profile unless they opted out. Users were sometimes unaware of these posts, and the sites also gave Facebook ad targeting data. After privacy complaints, stories of marital affairs being outed, and a class action lawsuit, Zuckerberg eventually apologized and Beacon was shut down.
• Engagement Ranked Feed = Sensationalized Fake News – Facebook built the News Feed to show the most relevant content first so we’d see the most interesting things going on with our closest friends, but measured that relevance largely based on what people commented on, liked, clicked, shared, and watched. All of those activities are stoked by sensationalist fake new stories and partisan exaggeration, allowing slews of them to go viral while their authors earned ad revenue and financed their operations with ad views delivered by Facebook referral traffic. Facebook downplayed the problem until it finally fessed up and is now scrambling to fight fake news.
• Engagement Priced Ad Auctions = Polarizing Ads – Facebook gives a discount to ads that are engaging so as to incentivize businesses to produce marketing materials that don’t bore or annoy users such that they close the social network. But the Trump campaign designed purposefully divisive and polarizing ads that would engage a niche base of his supporters to try to score cheaper ad clicks and more free viral sharing of those ads.

   

  

  Facebook Beacon’s opt-out message about it publishing your browsing habits to the News Feed

   

• App Platform = Game Spam – Another early encounter with unforeseen consequences came in 2009 and 2010 after Facebook launched its app platform. The company expected developers to build helpful utilities that could go viral thanks to special, sometimes automatic posts to the News Feed. But game developers seized on the platform and its viral growth channels, spawning companies like Zynga that turned optimizing News Feed game spam into a science. The constant invites to join games in order to help a friend win overwhelmed the feed, threatening to drown out legitimate communication and ruin the experience for non-gamers until Facebook shut down the viral growth channels, cratering many of the game developers.
• New Privacy Controls = A Push To Share Publicly – In 2010 when Facebook rolled out much-need granular privacy controls, it also baked in a new default of sharing status updates publicly, instead of just with friends. Facing competition for content from Twitter, Facebook aggressively pushed users to accept the new privacy system with a “Recommended” collection of settings which I criticized for directing users to share statuses and photos with everyone. By 2011, Facebook would be forced to sign settlements with the FTC and the European Union’s privacy office promising not to change users’ privacy settings without proper notice and opt-in, and Zuckerberg apologized (again).

   

  

  Facebook’s cavalier recommended privacy controls in 2010

   

• Academic Research = Emotion Tampering – Facebook allows teams of internal and external researchers to conduct studies on its users in hopes of producing academic breakthroughs in sociology. But in some cases these studies have moved from observation into quietly interfering with the mental conditions of Facebookers. In 2012, Facebook data science team members manipulated the number of emotionally positive or negative posts in the feeds of 689,000 users and then studied their subsequent status updates to see if emotion was contagious. Facebook published the research, failing to foresee the huge uproar that ensued when the public learned that some users, including emotionally vulnerable teenagers who could have been suffering from depression, were deliberately shown sadder posts.
• Ethnic Affinity Ad Targeting = Racist Exclusion – Facebook’s ad system previously let businesses target users in “ethnic affinity” groups such as “African-American” or “Hispanic” based on their in-app behavior as a stand in for racial targeting. The idea was likely to help businesses find customers interested in their products, but the tool was shown to allow exclusion of certain ethnic affinity groups in ways that could be used to exclude them from legally protected opportunities such as housing; employment, and loans. Facebook has since disabled this kind of targeting while investigates the situation.

   

  

  Exclusionary ethnic affinity ad targeting, as spotted by ProPublica

   

• Real Name Policy = Enabling Stalkers – For years, Facebook strictly required to use their real names in order to reduce uncivility and bullying facilitated by hiding behind anonymity. But victims of stalking, domestic violence, and hate crimes argued that their abusers could use Facebook to track them down and harass them. Only after mounting criticism from the transgender community and others did Facebook slightly relax the policy in 2015, though some still find it onerous to set up a pseudonym on Facebook and dangerous to network without one.
• Internet.org = International Net Neutrality Concerns – Facebook’s plan to give free Internet access to developing nations has been largely criticized as merely a way to wrap a user growth strategy in a philanthropic bow. Internet.org’s app initially offered a limited set of free services, excluding some competing social networks and featuring Facebook’s products. While some think giving free access to a slice of the web is better than none for those who might not be able to afford it, others see the zero-rating programs as a form of digital colonialism designed to get the poor hooked on a certain company’s version of the Internet. The brand name didn’t help to dispel that. Telecom partners pulled out in India leading Facebook to open the program to all develpers and rename it “Free Basics”. A government ban forced it to withdraw the app from India, though it remains available in 50 countries.
• Self-Serve Ads = Objectionable Ads – To earn money efficiently, Facebook lets people buy ads through its apps without ever talking to a sales representative. But the self-serve ads interface has been repeatedly shown to used nefariously. ProPublica found businesses could target those who followed objectionable user-generated Pages and interests such as “jew haters” and other disturbing keywords on Facebook. And Russian political operatives famously used Facebook ads to spread divisive memes in the United States and pit people against each other and promote distrust between citizens. Facebook is only now shutting down long-tail user-generated ad targeting parameters, hiring more ad moderators, and requiring more thorough political ad buyer documentation.
• Developer Data Access = Data Abuse – Most recently, Facebook has found its trust in app developers misplaced. For years it offered an API that allowed app makers to pull robust profile data on their users and somewhat limited info about their friends to make personalized products. For example, one could show which bands your friends Like so you’d know who to invite to a concert. But Facebook lacked strong enforcement mechanisms for its policy that prevented developers from sharing or selling that data to others. Now the public is learning that Cambridge Analytica’s trick of turning 270,000 users of Dr. Aleksandr Kogan’s personality quiz app into info about 50 million people illicitly powered psychographic profiles that helped Trump and Brexit pinpoint their campaign messages. It’s quite likely that other developers have violated Facebook’s flimsy policies against storing, selling, or sharing user data they’ve collected, and more reports of misuse will emerge.

Each time, Facebook built tools with rosy expectations, only to negligently leave the safety off and see worst-case scenarios arise. In October, Zuckerberg already asked for forgiveness, but the public wants change.

Trading Kool-Aid For Contrarians

The desire to avoid censorship or partisanship or inefficiency is no excuse. Perhaps people are so addicted to Facebook that no backlash will pry them their feeds. But Facebook can’t treat this as merely a PR problem, a distraction from the fun work of building new social features, unless its employees are ready to shoulder the blame for the erosion of society. Each scandal further proves it can’t police itself, inviting government regulation that could gum up its business. Members of congress are already calling on Zuckerberg to testify.

Yet even with all of the public backlash and calls for regulation, Facebook still seems to lack or ignore the cynics and diverse voices who might foresee how its products could be perverted or were conceptualized foolishly in the first place. Having more minorities and contrarians on the teams that conceive its products could nip troubles in the bud before they blossom.

“The saying goes that optimists tend to be successful and pessimists tend to be right” Zuckerberg explained at the November forum. “If you think something is going to be terrible and it is going to fail, then you are going to look for the data points that prove you right and you will find them. That is what pessimists do. But if you think that something is possible, then you are going to try to find a way to make it work. And even when you make mistakes along the way and even when people doubt you, you are going to keep pushing until you find a way to make it happen.”

Zuckerberg speaks at Facebook’s Social Good Forum

That quote takes on new light given Facebook’s history. The company must promote a culture where pessimists can speak up without reprise. Where a seeking a raise, reaching milestones, avoiding culpability, or a desire to avoid rocking the Kool-Aid boat don’t stifle discussion of a product’s potential hazards. Facebook’s can-do hacker culture that codes with caution to the wind, that asks for forgiveness instead of permission, is failing to scale to the responsibility of being a two billion user communications institution.

And our species is failing to scale to that level of digital congregation too, stymied by our insecurity and greed. Whether someone is demeaning themselves for not having as glamorous of a vacation as their acquaintances, or seizing the world’s megaphone to spew lies in hopes of impeding democracy, we’ve proven incapable of safe social networking.

That’s why we’re relying on Facebook and the other social networks to change, and why it’s so catastrophic when they miss the festering problems, ignore the calls for reform, or try to hide their complicity. To connect the world, Facebook must foresee its ugliness and proactively rise against it.