Getting data from the web using Python

Hey scientist! How’s it going?
Nice, you downloaded your comma separated values (csv) file. First you needed to search for it somewhere, now you have to indicate to your software where the file is, then you can start to work. Boring, am I right?
Today we’ll see how to access a csv file from the internet using Python! To do that, we’ll use some different libraries. Let’s do this!

To get the csv file from the web, first we need… a csv file. In our tests we’ll use the Yahoo series of historical prices, which I got from Yahoo Finances. This is the address of the series: http://finance.yahoo.com/q/hp?s=yhoo.

There’s a link at the end of the page: “Download to spreadsheet”. But you don’t need to download the little one. Click the link with the right button and select “Copy link address”, or the correspondent option.

copylink

There you go! We’ll use it later on this post.

The complete code of this example follows:

from urllib.request import urlopen
import matplotlib.pyplot as plt
import pandas

url = 'http://real-chart.finance.yahoo.com/table.csv?s=YHOO&d=9&e=17&f=2015&g=d&a=3&b=12&c=1996&ignore=.csv'
yahoo_csv = urlopen(url)

yahoo = pandas.read_csv(yahoo_csv, index_col=0, parse_dates=True)
yahoo.plot(y='Adj Close')

plt.xlabel('Year')
plt.ylabel('Adj Close')
plt.legend().set_visible(False)
plt.show()

Let’s explain it in chunks, ok?

from urllib.request import urlopen
import matplotlib.pyplot as plt
import pandas

Here we import the function urlopen from urllib.request.
It’ll obtain the data from the csv file. We call also pandas, which will transform the csv data into a readable table, and matplotlib.pyplot, to modify the parameters on the resulting plot.

url = 'http://real-chart.finance.yahoo.com/table.csv?s=YHOO&d=9&e=17&f=2015&g=d&a=3&b=12&c=1996&ignore=.csv'
yahoo_csv = urlopen(url)

Remember that address we copied on Yahoo Finances? Now we paste it (right button, “Paste”) on the code as a string, which we’ll attribute to the variable called url. After that, we use urlopen to pass this content to yahoo_csv.

yahoo = pandas.read_csv(yahoo_csv, index_col=0, parse_dates=True)
yahoo.plot(y='Adj Close')

Now we use Pandas to change the content of the obtained csv file into a table and we plot the result. To see the five first lines of that table, type:

yahoo.head()

This is the result:

                 Open       High        Low      Close    Volume  Adj Close
Date                                                                       
2015-10-16  33.639999  33.860001  33.160000  33.369999  12209300  33.369999
2015-10-15  32.419998  33.490002  32.400002  33.480000  19370900  33.480000
2015-10-14  32.279999  32.490002  31.770000  32.090000  11277400  32.090000
2015-10-13  32.560001  32.830002  32.320000  32.340000  12610500  32.340000
2015-10-12  32.709999  33.020000  32.549999  32.860001  12565800  32.860001

Note that we used the column “Adj Close” as the axis y on our plot: yahoo.plot(y=’Adj Close’).

plt.xlabel('Year')
plt.ylabel('Adj Close')
plt.legend().set_visible(False)
plt.show()

Finally, these commands change x and y labels and hide the legend. We saw some of that commands on the series about Python 2D plots, which began here. This is the resulting plot:

serieyahoo

Awesome! A nice, good-looking series!
Today we saw how to use urlopen and how to change a csv file into a pandas table. Also, we saw how to plot one of the table columns. Try to change the value of y on yahoo.plot()! For instance, use yahoo.plot(y=’Open’) or yahoo.plot(y=’Close’).
Thanks scientist! Gigaregards!


Like this? Please comment and share with your friends!
Want to download Programando Ciência codes? Go to our GitHub!
Make a donation for Programando Ciência!
Like us also on Facebook: www.facebook.com/programandociencia
I’m on Twitter! Follow me if you can! @alexdesiqueira

2 thoughts on “Getting data from the web using Python

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s