Sentiment Analysis of news providers Tweets and only display positive articles.

Source: Unsplash

Right now, there appears to be more and more negative news published to the internet everyday. We’re going to look at how to build a web application to filter out all negative articles using Sentiment Analysis. Sentiment Analysis refers to analysing an opinion or feelings about something using data, in this case, tweets from major news companies. The finalised web application can be found here.

The technologies we are going to use for this project include Python, Pandas, Tweepy API, Flask, and Heroku to host our application.

1. Reading Tweets

To start, we are going to need to read tweets from all major news Twitter accounts. Using the Twitter Developer Credentials which can be accessed by any user here: https://developer.twitter.com / and the Tweepy API we can create a Pandas dataframe of all the tweets and other relevant data from several news accounts. For now, we are going to stick with the follow news providers: “washingtonpost”, “ABC”, “SkyNews”, “CNN”, “BBCWorld”, “nytimes”, “NBCNews”, “VICENews”

2. Filtering Tweets Using Sentiment Analysis

We now have 1000+ tweets from several news providers stored within a single dataframe. To filter these based on positivity, we are going to use the TextBlob library. This is a lightweight Python library for measuring the sentiment of text using polarity on a scale of 0 to 1 (0 being very negative and 1 being very positive). We will look to filter our dataframe and discard any tweets that have a polarity of less than 0.6.

3. Displaying this information.

Finally, we will look to display this information within a simple web application. To achieve this, we are going to use Flask, a Python web application framework and Boostrap an open-source CSS framework.

Our Flask application, using Jinja2 syntax will display our newly filtered dataframe with all positive news stories and links to these articles. Bootstrap and other custom HTML/CSS will display this information in a more clear and concise way.

Flask application with table inlcuded.

index.html file for displaying table with Jinja2 syntax.

Our final result that can now run on a local host will look something like this.

4. Hosting our web application

Now that our application is ready to go online we’ll be using Heroku to host the application. Heroku provides free accounts that are perfect for side projects like this. The free version allows for up to five applications to run at one time. To deploy a Flask web app to Heroku we will run the following commands.

First install a web server called Gunicorn. Run the following command within the application folder.

pip install gunicorn

Update the requirements file by running

pip freeze > requirements.txt

Create a new file with Procfile as the name and do not add any extension. Include the python file name and the application name within the file.

web: gunicorn twitterbot:app

Now initialise git within the directory and commit to heroku

git init
git add .
git commit -am "commit1"
git push heroku master

Our application is now posted as a heroku application and can be found here.