5:00am

Tue December 6, 2011
Technology

How Twitter's Trending Algorithm Picks Its Topics

Originally published on Wed December 7, 2011 2:18 pm

The list of "trending topics" on the right side of Twitter's home page is a coveted spot because millions of people see it. It often reflects what's hot in the news, from the death of Steve Jobs to Kim Kardashian's latest exploits.

Sometimes a topic that seems hot, like Occupy Wall Street, doesn't trend, leading some activists to charge Twitter with censorship. But the complex algorithms that determine trending topics are intended to find what's trending in the moment, and not what's been around for a long time.

Getting a spot on the trending list has become so important that television programs hire consultants to help them get there.

Jason Pollock, a social media consultant, says the singing competition The Voice set the tone for Twitter interaction. On the show, teams of young singers get coached by stars like Christina Aguilera. The Voice became one of the most popular singing competitions on TV this year with the help of Twitter.

Pollock says that as competitors are training for the competition, they tweet with some help from paid staff. They put pound signs in front of catchy phrases to create hashtags, like #TeamNakia. "They had them really operating fast. ... They had hashtags going and they were asking questions and doing polls and really engaging stuff," Pollock says.

When those hashtags hit the trending topics list on Twitter, more people started to watch The Voice on TV.

"If you start trending on Twitter, people on Twitter will turn that show on," Pollock explains. "Everyone is obsessed about the ratings — well, here's a new tool to up your ratings."

#OccupyWallStreet Not On The List

The trending list can also bring attention to your cause. During the tense moments of Occupy Oakland, freelance journalist Susie Cagle was following the Occupy Wall Street hashtag.

"Just looking at how often I was tweeting, let alone how often others were on the same Occupy Oakland hashtag, there have been times where that hashtag is scrolling and updating so quickly that there's no way I can keep up," she says.

But it struck her as odd that Occupy Oakland wasn't on trending topics, even after weeks of protests.

"To not see those things trending a month in was surprising," she says.

On blogs, some Occupy Wall Street activists charged Twitter with censoring its trending topics list. Why were there trending hashtags like #WhatYouFindInLadiesHandbags or #ThingsThirstyPeopleDo?

Twitter's Matt Graves says it's because the protests had been going on for weeks.

"We look at trending topics as a reflection of what people are talking about more right now in this moment, than they were a minute ago, an hour ago or a day ago," Graves says.

And when Graves says "we," he's really talking about Twitter's secret algorithm that sorts through 250 million tweets a day. It's designed to search for the sudden appearance of a topic in large volume.

"That may mean, for example, when Steve Jobs [died], there are different trending topics around that," Graves says. "It may mean on a local level, when there's an earthquake — whether it's in Oklahoma or California — being able to go and see, well, is 'earthquake' trending in Oklahoma right now?"

Getting Used To An Algorithmic Editor

There had been thousands of tweets for Occupy Wall Street regularly over many weeks, so Twitter's algorithms stopped putting it on the trending topics list. In some ways, Twitter's algorithms act like a lot of human news editors who are more interested in the latest news than an ongoing story, says Tarleton Gillespie, a communications professor at Cornell University.

"It's going to be hard for us to think about the involvement of the algorithm in a way that we could understand the involvement of an editor ... [who] might step in and say, 'I don't want to run this story' or 'I think this one's more important than that one,' " Gillespie says. "What does it mean that an algorithm decided that something was more important than something else?"

Gillespie points to the Amazon best-seller list. In that case, it seems like a simple unbiased calculation.

"Naively, you would say, 'Well, the most-selling book is No. 1 and the second-selling book is No. 2,' " he says.

But a couple of years ago, there was a dust-up because all the gay-themed books disappeared from the list. It turns out that Amazon doesn't let any adult books in its best-sellers, and someone accidentally put the gay-themed books in that category.

"It's a curated list," Gillespie says. "It's a list that will never show us if something that they or their publishers had classified as adult would ever show up there."

Everything from restaurant reviews to your friend's baby pictures to your local news is getting served up to you by an algorithm. As much as programmers may think their algorithms will deliver objective results, those calculations may be just as biased as a real human being.

Copyright 2013 NPR. To see more, visit http://www.npr.org/.

Transcript

STEVE INSKEEP, HOST:

If you spend any time on Twitter, the online service will be sure to advise you what's trending. That's the Twitter equivalent of trendy, what people are telling each other about right now. Earlier this morning the trends in the Washington, D.C. area included "Jersey Shore" and Pearl Harbor.

It matters what trends because those little listings become an advertisement encouraging more people to pay attention to that topic. Sometimes a topic that seems like it should be hot, like Occupy Wall Street, does not trend, which has led some activists to charge Twitter with censorship. But as NPR's Laura Sydell reports, it's not about censorship, but about algorithms.

LAURA SYDELL, BYLINE: Getting a spot on Twitter's trending topic list has become so important that television programs hire consultants to help them get there.

BEVERLY MCCLELLAN: And it's such an honor to be singing this song with Christina Aguilera.

JASON POLLOCK: "The Voice" really set the tone.

SYDELL: That's Jason Pollock a social media consultant. "The Voice" became one of the most popular singing competitions on TV with the help of Twitter. Teams of young singers like Raquel Castro get coached by stars like Christina Aguilera for a competition.

UNIDENTIFIED WOMAN ONE: This is an amazing opportunity. I'm only 16 years old and it's kind of nerve wracking because I haven't performed a lot.

UNIDENTIFIED WOMAN TWO: Oh my God.

SYDELL: As competitors are training for the competition, says Pollock, they tweet with some help from paid staff. They put pounds signs in front of important catchy phrases to create hashtags like Team Nakia.

POLLOCK: They had them really operating fast and they were pumping it out and they had hashtags going and they were asking questions and doing polls and really engaging stuff.

SYDELL: And when those hashtags hit the trending topics list on Twitter, more people started watching "The Voice" on TV.

POLLOCK: If you start trending on Twitter, people on Twitter will turn that show on. So, you know, where everyone is obsessed about the ratings, well, here's a new tool to up your ratings.

SYDELL: Or bring attention to your cause.

(SOUNDBITE OF PROTEST)

UNIDENTIFIED MAN: Let them go.

(SOUNDBITE OF PROTEST)

SYDELL: During the tense moments of Occupy Oakland freelance journalist Susie Cagle was following the Occupy Wall Street hastag.

SUSIE CAGLE: Just looking at how often I was tweeting, let alone how often others were on the same Occupy Oakland hastag, there have been times where that hashtag is scrolling and updating so quickly that there's no way I can keep up.

SYDELL: Yet it struck her as odd that Occupy Oakland wasn't on trending topics, even after weeks of protests.

CAGLE: And to not see those things trending a month in was surprising.

SYDELL: On blogs some Occupy Wall Street activists charged Twitter with censoring its trending topics list. Why were there trending hashtags like #What-you-find-in-ladies-handbags or #Things-thirsty-people-do? Twitter's Matt Graves says it's because the protests had been going on for weeks.

MATT GRAVES: We look at trending topics as a reflection of what people are talking about more right now in this moment than they were a minute ago, an hour ago, or a day ago.

SYDELL: And when Graves says we, he's really talking about Twitter's secret algorithm that sort through 250-million tweets a day. It's designed to search for the sudden appearance of a topic in large volume.

GRAVES: And that may mean, for example, when Steve Jobs dies, you know, that there are different trending topics around that. It may mean on a local level when there's an earthquake, whether it's in Oklahoma or California, being able to go and see well, you know, is there earthquake trending in Oklahoma right now.

SYDELL: There had been thousands of tweets for Occupy Wall Street regularly over many weeks, so Twitter's algorithms stopped putting it on trending topics list. In some ways Twitter's algorithms act like a lot of human news editors who are more interested in the latest news than an ongoing story says Tarleton Gillespie, a communications Professor at Cornell University.

TARLETON GILLESPIE: It's going to be hard for us to think about the involvement of the algorithm in a way that we could understand the involvement of the an editor, right, that might step in and say I don't want to run this story or I think this one's more important that that one. What does it mean that an algorithm decided that something was more important than something else?

SYDELL: Gillespie points to the Amazon bestseller list. In that case, it seems like a simple unbiased calculation.

GILLESPIE: Naively you would say well, the most selling book is number one and the second most selling book is number two.

SYDELL: But, a couple of years ago there was a dust up because all the gay books disappeared from the list. It turns out that Amazon doesn't let any adult books in its bestsellers and someone accidentally put the gay books in that category.

GILLESPIE: It's a curated list. It's a list that will never show us if something that they or their publishers had classified as adult would ever show up there.

SYDELL: Everything from restaurant reviews, to your friend's baby pictures, to your local news is getting served up to you by an algorithm. As much as programmers may think their algorithms will deliver objective results, those calculations may be just as biased as a real human being.

Laura Sydell, NPR News, San Francisco.

INSKEEP: And you can follow this program on Twitter. We are @MORNING EDITION and @NPRINSKEEP. Transcript provided by NPR, Copyright NPR.