Social Analytics companies have been massively using twitter to get insights about whatever data they are interested in for brands, celebrities, etc. and also trending topics. In this tutorial you'll know how to get countries that have trends on Twitter and also get insights about what topics are trending the most and be able to retrieve the url of that tweet and its volume as well.
Here I'll walk you through how you can do that with Python and Tweepy. You can do many other things with Tweepy other than trending topics but in this tutorial I will focus on getting trends. Let's dive in and see what Tweepy can do for us.
What is Tweepy?
Tweepy is an API wrapper for twitter; you can use it to get some data from twitter, to know some information about your account and get some insights about public data.
Tweepy is in PyPy, so you can use pip to install it:
An alternative way to do it, is to install it from its GitHub repo:
Tweepy supports Python 2.7, 3.5, 3.6, 3.7, & 3.8
Twitter Developer Credentials
First thing you need to do is to have a twitter developer account. If this is your first to set up a twitter developer account, it may take time from twitter's side to accept your request. Once your account is accepted, you can click on Developer Portal tab at your developer account and hover over your name and select Apps from the dropdown. There you can create a new app, you'll be asked some questions regarding your app. You should generate your tokens and it's a good practice to save them at password manager like passpack for example; it's a free software for storing and sharing passwords.
Authentication and Authorization
Now, you have four credentials for your app:
- API Key
- API secret key
- Access token
- Access token secret
Let's understand why API keys are different from access tokens. Short answer is API keys are used for authorization, while tokens are for authentication. What does that mean?
Before we know that, we need to know what we should do first in order to do our task. What you want is to let twitter know who you are to give you access to the data that you want. Twitter gives you permissions through authorization and then asks you to generate tokens which know who you claim to be (authentication), this is like logging in twitter account after you've been authorized that you have the privilege to read and/or write access.
It's recommended not to hardcode these credentials in our script so better to have them as environment variables. Here is the script so far:
What we need to do is to let our script know our environment variables. So let's assign them through the command line:
If you are interested in getting available locations that have trends at twitter around the world, you can use Tweepy's method trends_available()
Let's see what we have until now:
Trending Tweets for Specific Country
Let's say we want to get the trending tweets in Egypt. Instead of hardcoding 'Egypt' in the script we can pass it as argument variable.
Tweepy has a method trends_closest() and trends_place() -- combining the two gives us very similar result to what we're looking for, trends_closest just needs a longitude and latitude of the country we want and then it can return the a JSON file that has WOEIDs that we're interested in. WOEID (or Where On Earth IDentifier) is a unique identifier for any feature on earth. For example, WOEID for New York is 2459115 and WOEID for Los Angeles is 2442047. Although both reside at the United States, they both have unique WOE ID.
One note to consider, just install geocoder which is a library that helps you get location relevant information and we need it here to get the longitude and latitude.
Install with Pip
Install with Conda
Make sure when you run the final script to add the argument variable of the country that you want:
Now, you can use Tweepy to get some the latest 50 trending topics on twitter, but make sure of number of requests you have if you're using it extensively .. because you have a limit of 100,000 requests per day. That's according to Twitter on June 19, 2019
Tweepy has a lot more than trending methods, I hope this tutorial is useful and maybe motivational to read more about Tweepy.
Thank you for reaching here!