It is widely believed by misinformation researchers that one of the most powerful — if controversial — tools that social media platforms have in combating misinformation from public figures and lesser-known individuals alike is to kick the worst offenders off entirely. But before platforms take that step, they typically follow a more nuanced (and sometimes confusing) system of strike policies that can vary from platform to platform, issue to issue and even case by case. These policies often stay out of the spotlight until a high-profile suspension occurs.
Many misinformation experts agree that social media platforms had to start somewhere, but such policies sometimes suffer from the perception that they were created only after things went wrong. And some critics question whether the confusing nature of these policies is a feature or a bug.
“The most outrageous people, the most controversial people, the most conspiratorial people, are good for business. They drive engagement,” said Hany Farid, a professor at the University of California Berkeley School of Information whose research focuses include misinformation. “So that’s why I think there’s this tug of war — we’re going to slap you on the wrist, you can’t post for a week, and then they come back and of course they do it again.”
Social media companies say the strike policies allow them to balance managing misinformation with educating users about their guidelines, and also ensuring their platforms remain open to diverse viewpoints. They also point to the millions of pieces of problematic content they have removed, and highlight efforts to boost the reach of reliable information to counteract the bad.
“We developed our three strikes policy to balance terminating bad actors who repeatedly violate our community guidelines with making sure people have an opportunity to learn our policies and appeal decisions,” said YouTube spokesperson Elena Hernandez. “We work hard to make these policies as understandable and transparent as possible, and we enforce them consistently across YouTube.”
In a statement, a Twitter spokesperson said: “As the Covid-19 pandemic evolves in the United States and around the world, we continue to iterate and expand our work accordingly. … We’re fully committed to protecting the integrity of the conversation occurring on Twitter, which includes both combatting Covid-19 misinformation through enforcement of our policies and elevating credible, reliable health information.”
Social media strike policies are “designed, in essence, to discourage people from spreading misinformation, but the effect it probably has is negligible,” said Marc Ambinder, the counter-disinformation lead for USC’s Election Cybersecurity Initiative. He added that the policies appear aimed more at average users accidentally posting bad information than strategic, frequent posters of misinformation.
“What we know is that the most effective way the sites can reduce the spread of harmful misinformation is to identify the serial spreaders … and throw them off their platform,” he said.
The strike rules
To make matters more complicated, users accumulate strikes for each issue separately: they get five chances on posting Covid-19 misinformation, and five chances on civic integrity. (For other rules violations, Twitter said it has a range of other enforcement options.)
“Everything is reactionary,” Farid said. “None of this has been thoughtful, and that’s why the rules are such a mess and why no one can understand them.”
Both Facebook and YouTube say they may remove accounts after just one offense for severe violations. YouTube may also remove channels that it determines are entirely dedicated to violating its guidelines. And Facebook said it will remove accounts if a certain percentage of their content violates the company’s policies, or if a certain number of their posts violate policies within a specific window of time, though it doesn’t provide specifics “to avoid people gaming our systems.”
On Facebook and Instagram, it’s somewhat less clear what constitutes a strike. If the company removes content that violates its guidelines (which include prohibitions of misinformation related to Covid-19 and vaccines and voter suppression), it “may” apply a strike to the account “depending on the severity of the content, and the context in which it was shared.” Multiple pieces of violative content may also be removed at the same time and count for a single strike.
“Generally you may get a strike for posting anything which goes against or Community Standards – for example – posting a piece of content which gets reported and removed as hate speech or bullying content,” Facebook said in a statement. Separate from its guidelines enforcement, Facebook works with a team of third-party partners to fact check, label and, in some cases, reduce the reach and monetization opportunities of other content.
In the same month that Twitter began enforcing its civic integrity misinformation policy, Greene received what appears to be her first known strike, with more to follow. Based on Twitter’s policy, Greene’s recent week-long suspension would represent her fourth strike on Covid-19 misinformation, but the company declined to confirm.
According to Twitter’s policy, Greene could be permanently banned from the platform if she violates its Covid-19 misinformation policy again. But the line between spreading misleading information and violating the policy can be murky, highlighting the ongoing challenges with making these policies work in stopping the spread of misinformation to users.
“I don’t necessarily envy the decisions … that the platforms have to make,” USC’s Ambinder said. “But it does seem pretty clear that the volume and the vigilance of misinformation reduces itself in proportion to the number of serial misinformation spreaders who are deplatformed.”