Do you know the feeling? “I should do <fill in something cool here>”. Well, I was facing this already a time by learning python. I knew that you could do some cool thing with it, but never pushed myself to do it. Until now! Otn appreciation day! Thanks to Mr Oracle Base Tim Hall. Some time ago, he launched the idea of otn appreciation day and of course I added my entry as well. You can find it here.
Tuesday 11 october 2016, my entry was scheduled at 08:30 CEST and seeing all the other blog posts, I soon realised that this would be a very nice bunch of information. I started copy pasting the blogs and links, but … As soon as I started doing something else (work!!! ) I missed some. That brought me the idea of creating a script. The idea was simple, log in to twitter, fetch all tweets hashtagged by ThanksOTN and then filtering out the retweets. Simple huh? Then … how to do it? mmm … let’s take the challenge, I’ll do it in python.
I broke it down into some steps. In order to read a twitter feed you need a twitter application. To do so, surf to https://apps.twitter.com and after loging in, click the create application button.
I only left the callback url blank. For this purpose, we don’t need it. I think, if I do, please let me know.
Then all is done. The next screen will give you an overview about the application you just have created and in the tab “keys and access tokens”, you only have to click on one more button “Create my access token”. So that’s it folks, nothing more to be done at twitter side.
Just record following fields:
- Consumer Key (API Key)
- Consumer Secret (API Secret)
- Access Token
- Access Token Secret
these we need in order to be able to establish a connection with tweepy to twitter.
Then it’s time to write some code! I assume you already have python setup, if not, drop me a mail, comment or tweet and I’ll help you out. So I never ever had done some python scripting so it was googling a bit.
It turned out i’d need tweepy for this task, so it was easy to install. Pip install tweepy was all I needed to enter and confidence was growing, if it’s going to be as easy as this, I’ll be good!
first we need to import some things:
from tweepy import OAuthHandler
Then (at this point) I only needed a main procedure. I call it “main”. maybe obvious but ok 🙂 If you need more procedures, they go right after the import statements.
So the main I created looks like this. The comments are my comment for the blog as well:
#Variables that contains the user credentials to access Twitter API
access_token = "<fill in your own>"
access_token_secret = "<fill in your own>"
consumer_key = "<fill in your own>"
consumer_secret = "<fill in your own>"
# OAuth process, using the keys and tokens
# here we create an auth object which uses the tweepy oauthhandler. There we need to pass the consumer key and secret
# then we need to set the access tokens into the auth object
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
# Creation of the actual interface, using authentication
# here we create the actual connection to twitter and we call it api. It's just a name.
api = tweepy.API(auth)
# search it and display them after removing the RT @'s
# some variables i picked:
query = '#thanksOTN'
max_tweets = 1000
# here i am gathering the tweets into an array and put them in a cursor.
searched_tweets = [status for status in tweepy.Cursor(api.search, q=query).items(max_tweets)]
# this is just "quick'n dirty I figured out in the meanwhile. A better way would be to
# use templates and fill them in this way, but hey... this works :-)
# now I have a resultset (searched_tweets) and I will run over them in a for loop.
for tweet in searched_tweets:
# I'm filtering out the retweets in following line
if "RT @" not in tweet.text:
## the next line in comment. I added an extra if-clause first to only list tweets which contained "OTN Appreciation day:" in the tweet
# later turned out that not everyone put this one in, so I commented it out.
# if "OTN Appreciation Day:" in tweet.text:
# and here i'm printing the user who has sent the tweet
print('<p>Twitter user:',tweet.user.screen_name,'<br />')
# and eventually what he has tweeted. This is also a bit Q'n'd I figured out. but the emoji weren't
# displayed correctly in python 3.4, so this is a was to have them parsed to utf8. I think there
# will be some more efficient ways of doing this, so feel free to comment on this.
tweet_text = str(tweet.text.encode("utf-8") if tweet.text else tweet.text).lstrip('b\'')
# and finally print the tweet.
# and close the webpage
# finally call main
And finally this script was scheduled in crontab every 30 minutes, and redirected to a html file.
I should still create a kind of tokeniser to manipulate the tweet_text in order to make hyperlinks from the links. But hey … that’s something for the future 🙂
So this was my very very very first python script. I think it’s a fun language which I’ll be using more and more.
Comments, advice,… are always welcome!