From the very beginning of Twitter, trending topics has become one of the core features of this popular product. From Twitter trends, you can easily get what’s popular now. There’s no wonder that many companies like to ask candidates to design a trending algorithm in system design interviews.
We’ve covered this topic a briefly in a previous post – How to Design Twitter (Part 2). Since this question is so popular that I decided to write an in-depth article about it.
Twitter tt Explore Twitter Trending Hashtags and statistics with the best experience and download Twitter Videos and Photos. USA Twitter Trends Now - Pat Tillman, LWOP, Nikita, Mayweather, Andrew Yang, Kyle Higashioka, Josh Hawley, SCOTUS, The Founders. Worldwide Argentina Australia Brazil Canada France Germany India Indonesia Israel Italy Japan Mexico Russia Saudi Arabia South Africa Spain Thailand Turkey United Kingdom United States South Korea.
Again, none of us at Gainlo has worked on Twitter trends and different interviewers have their own styles of conducting system design interviews. Therefore, the point of this article is definitely not to give you a standard answer to this question, but give you more ideas about the topic.
Question
Trending Topics Twitter
Since Twitter is so popular, I will only clarify the question briefly here.
As to design a trending algorithm for Twitter, the system should be able to provide a list of topics that are popular at the current moment. I like to make the problem general and a little bit vague since I want to see how a candidate can approach this kind of open-ended question. For instance, there’s no clear definition of what popular means here.
General Ideas
Twitter Trending Hashtags (Topics) & Trends for Worldwide Today. 26987191 Minutes Ago Currently. We store and analyze Twitter data to bring you insights on what’s currently trending and help you keep up with the audience. Find out hot trends, review trend history and browse long-term popular hashtags and topics. Contact Us Find us on Twitter Find us on Facebook. Follow @gdtrends. Help us improve: 💬 Take a survey. From your Topics: Tap Topics in your profile icon menu. Tap Unfollow next to the topic. Note: Up to 100 of your followed Topics are visible at a time in your Topics menu. To see all your followed Topics, and those we think you are interested in based on your activity and profile, visit Your Twitter data.
If you haven’t thought about this problem before, I would recommend you spend at least 15min thinking about it. It’s always better to have your own solution and then compare with others.
Alright, here we go.
A general idea is that let’s use a term (or a word) to represent a topic and it can be a hashtag (like #gainlo) or just a phrase (like Donald Trump). If a term has a huge volume in recent tweets compared to the past, the term should be identified as popular. For example, if millions of people are talking about #gainlo today but in the past only hundreds of people talked about it, #gainlo should definitely be a hot topic at the current moment.
The reason we should compare to past volume is that for some common terms like “Monday” or “weather”, they have a huge volume at any time and shouldn’t be selected as trending in most cases.
To sum up, the basic idea is that for each term, if the ratio of term volume within last several hours and term volume of last X days is high, it will be regarded as a trending topic.
Infrastructure
Of course the trending topics should be displayed instantly, which means we can ask users to wait for an hour so that the system can calculate and rank all the terms. So what the underline infrastructure looks like?
Obviously, the calculation can be costly given the huge amount of tweets everyday. In this case, we can consider using offline pipelines.
More specifically, we can keep several pipelines running in the offline that calculates the ratio of each term and output the results to some storage system. The pipelines may refresh every several hours assuming there’s no big difference between a short period of time. So when a user checks the trending topics from the front end, we can just this user with pre-computed results.
The basic approach is really naive, do you have any ideas to improve it? It can be from any perspective, like improve the system cost, or improve the data quality etc. I’ll illustrate few ideas in the following part of the post.
Absolute term volume
If you simply calculate the ratio as explained above, I’m pretty sure there will be some very weird terms selected. Think about the following scenario, suppose there are only 300 people who tweeted about a weird topic “#gailo-mock-interview” and in the past no one has ever talked about it. The ratio (volume within last few hours / volume within last X days) is 1, which can rank at the top of the list.
Apparently, this is not something we want to show to users. I believe that you’ve already identified the problem here. If the absolute volume is not big enough, some unpopular terms may get picked. You can calculate the ratio like volume within last few hours / (volume within last X days + 10000) so that small volume gets diluted. Or you can use a separate signal as absolute term volume score to combine with the ratio.
Influencers
Another idea is that if some topics are discussed by high profile people, they might be more likely to be interesting and popular.
Trending Topics Twitter Turkey
Eyeshadow for mac. There are many ways to design this algorithm. One approach is to first identify who are high profile users. We can simply use follower number (although there are lots of fake influencers who bought followers). If a topic was tweeted by any influencer, we can count this topic tweeted by multiple times. So just multiply the tweet counts with a parameter based on the popularity of the influencer.
One may argue that we should not give influencers more weight since if a topic is trending, there must be a huge number of normal users talking about it. That might be true. The point here is that you will never know the result until you give it a try. So I’m definitely not saying that this is the right thing to do, but it may be worth to have an experiment.
Personalization
Different people have different taste and interests. We can adjust the trends list according different users. This can be quite complicated since there are so many things you can do to make it personalized.
For instance, you can calculate a relevance score between each topic and the user based on signals including his previous tweets, who he has followed and what tweets he has favorited etc. Then the relevance score can be used together with the trending ratio.
In addition, location should also be a valuable signal. We may even calculate trending topics for each location (maybe city level).
Summary
There are many more interesting topics I haven’t covered yet. For example, a lot of topics just mean the same thing, how does the system dedup them? Are there other signals like users search history can be integrated?
Trending algorithm is absolutely fun and useful. Although we use Twitter as an example, many things we’ve discussed can also be applied to Facebook or other platforms.
The post is written by Gainlo - a platform that allows you to have mock interviews with employees from Google, Amazon etc.
I'd like to learn more
Related posts:
goglobalwithtwitterbanner
Trending On Twitter
- Using Twitter
- Tweets
- Managing your account
- Login and password
- Safety and security
- Security and hacked accounts
- Rules and policies
- Twitter Rules and policies
There are so many good conversations happening on Twitter, it may be hard to find what’s most relevant to you from time to time. That’s where Topics can help.
From your profile menu, on your Home timeline, and in search results, you’ll see prompts to follow various Topics. Simply follow the Topic and we'll personalize your Twitter experience with relevant Tweets, events, and ads. We may also match you with other Topics that we think you are interested in based on your profile and activity, such as the Tweets you view or like.
There are numerous Topics across many categories on Twitter to follow through the Topic browser. We’re working hard to add new Topics almost each week. How do we choose which Topics to offer? They’re based on the volume and health of conversations happening on Twitter. For volume, we look at how much people are Tweeting, Retweeting, replying, and liking Tweets about that Topic.
We want Topics on Twitter to reflect the broader, lasting conversations people have about the events, people, and things they discuss. So we use machine learning to find related Tweets from these conversations. This could mean they Tweet a lot about the Topic or interact a lot with Tweets about the Topic. From there, we find the Tweets that are most interesting to those people, using algorithms, keywords, and additional signals.
We’ve implemented a number of protections to help keep conversations included in a Topic healthy and free from abuse –– including working to not recommend Tweets for a Topic if the engagement is manipulated or spammy.
Topics you follow are public. Anyone who can see your full profile can view the Topics you follow. If your Tweets are protected, only your followers will be able to see your Topics. To protect your Tweets, go to your Privacy and safety settings and select Protect your Tweets.
- You’re in control. Just like with accounts you follow, you can unfollow Topics at any time. We may still occasionally recommend Tweets about a particular Topic, for example, if you continue to like, Tweet, or Retweet about it. When you unfollow a Topic, we will no longer suggest content based on the fact that you followed that Topic.
- Tell us when you’re not interested. We strive to constantly improve your Topics experience and the suggestions we show you. We’ve made it easier for you to indicate when you’re not interested in a Topic we might suggest for you within your Home timeline and Explore tab. Simply tap the Not Interested selection in the suggested Topic prompt.
- How to detach your account from certain interests. You can always opt-out of interests by visiting Interests from Twitter in Your Twitter data. We won’t use that Topic to personalize your experience on Twitter, and we will not recommend it as one you can follow in the future, unless you specifically search for it. Please note that these changes can take some time to take effect across our systems, and you may still see content about a Topic due to other reasons such as following someone who is talking about it.
- We take the time to test. As we continue to build new Topics, including Topics that may involve more complex subjects and interests, we may take a cautious approach by only featuring a limited amount of accounts to start before adding more over time. We do this to test out the Topic and adjust our systems so we’re serving you the most interesting, healthy Tweets. If you are featured in a Topic, you can opt-out by following the instructions below.
How to follow a Topic
From your Home timeline:
- While scrolling through your Home timeline, we may suggest a Topic for you to follow.
- Tap the Follow button next to the suggested Topic to follow.
From your Topics menu:
Go to More and tap on More.
A popup with options will appear.
Click on Topics.
If you are following any Topics, they will appear here. You can select to follow more by clicking the Follow button.
If you are not interested, you can select the button and those Topics will appear in the Not Interested category of Topics. You can always choose to follow these Topics if you change your mind.
How to unfollow a Topic
From your Home timeline:
- In your Home timeline, navigate to a Tweet about a Topic you’re currently following.
- Tap the icon from the top of the Tweet and select Unfollow.
From your Topics:
- Tap Topics in your profile icon menu.
- Tap Unfollow next to the topic.
Note: Up to 100 of your followed Topics are visible at a time in your Topics menu. To see all your followed Topics, and those we think you are interested in based on your activity and profile, visit Your Twitter data.
How to detach your account from certain interests
On the Twitter for iOS and Android app:
- Go to Settings and privacy in your profile icon menu, and then tap Account.
- Below the Data and permissions section, tap Your Twitter data.
- Enter your password and tap Confirm.
- From the Your Twitter data screen, tap Interests and ads data.
- Tap Interests from Twitter.
- Deselect any Interests you’d like to opt out of.
On twitter.com
- Go to Interests from Twitter in Your Twitter data.
- Deselect any Interests you’d like to opt out of.
How to opt-out of being featured in a Topic
On twitter.com
- Go to Interests from Twitter in Your Twitter data.
- Click the Known For tab at the top.
- Deselect any Topics you do not want to be Known For.
How to see the Topics someone follows
You have the option to see the Topics that someone else is following. You’re able to view their Topics if their Tweets are public, or if their Tweets are protected and you've been approved as a follower.
On the Twitter for iOS and Android app, and twitter.com:
- Go to their profile.
- Click or tap the more menu at the top of their profile page.
- Select View Topics.
Bookmark or share this article