Liable Sources

Liable Sources

If you're wondering why your AI is racist, here's a clue

By J.M. Berger | April 21, 2023

The Washington Post had a barn-burner of a story on the data used to train AI chatbots and generative text apps, based on a paper out of the Allen Institute for Artificial Intelligence. The paper describes the C4 linguistic corpus, which has been used to train major components of the current generation of AI products.

The exact weighting of this corpus in the final product compared to other mitigating factors is not totally clear, but we know that AI applications have learned to be racist and sexist, to such an extent that their creators have had to equip them with “guardrails” to keep them at least somewhat in check.

How do these AIs learn to be racist and sexist? We already pretty much knew the answer. They learned it from watching us—on the Internet. The new Post story adds some important granularity to that question, however, and I am about to add some more. The C4 dataset is a giant bucket of words, scraped from the Internet and then filtered for toxic content. One of the Cs in C4 stands for “clean,” meaning it’s been vetted for racism and other objectionable material. In theory. Another C is for “crawled.” To clarify a point I have seen some confusion about in replies, no one picked the 15 million websites in the dataset; it was generated by a blind web crawl. The sites discussed herein are likely the product of negligence, rather than malicious intent. 

The Post story describes the C4 dataset in some detail and includes a tool for searching the URLs that were scraped to produce it.

Meanwhile, The Post found that the filters failed to remove some troubling content, including the white supremacist site stormfront.org No. 27,505, the anti-trans site kiwifarms.net No. 378,986, and 4chan.org No. 4,339,889, the anonymous message board known for organizing targeted harassment campaigns against individuals.

Sounds bad, right? Well, it’s worse than it sounds.

The rankings cited in the quote above are based on the number of tokens (words and/or short phrases) scraped from each source. To greatly oversimplify, a large-language model contains statistical data about how tokens are typically stacked. For instance, if you look up “best” it might know that “regards” often follows. Generative AIs owe a debt to autocorrect, but they’re much more sophisticated, able to string together thousands of words in a credible manner in response to a prompt. A generative AI includes contextual cues. So if you mention a movie, it might guess that “best” is more likely to be followed by “picture.” When an AI is trained on a website, it doesn’t just learn the words on that site, it learns the relationships among those words—for instance, which adjectives are applied to which racial groups.

The Washington Post, a generalist publication, noted a few well-known extremist and toxic sites that appear in the dataset. As a non-generalist, I immediately used the tool provided with the story to look up sites that are less well-known. Since the rankings reflect “tokens,” I theorized that sites with very large quantities of content would rank higher than even Stormfront or KiwiFarms, which are better known and more consistently used by extremists.

Unfortunately, I was right. The table below consists only of sites that ranked within the top 2 percent of C4 sources according to the number of tokens scraped. Most of these sites, those ranked higher than 150,000, are in the top 1 percent. Odds are you won’t recognize all these names unless you’re working in the field. I picked these sites for various reasons, but they don’t represent the full spectrum of potentially problematic content. This is just a small sample.

This is a deeply problematic list.

At the top of the list is Global Research, a website stuffed with a wide range of conspiracy theories, including anti-vax and pro-Russia content. Here’s what its front page looked like yesterday: