How to Scrape and Parse Stock Earnings Reports

For years I looked for a simple way to get earnings reports from Wall St. It seems they would change their earning reports on a regular basis and the dates would be inconsistent. Then one day when I wasn’t paying attention, BAM. Earnings report and the stock would take off or crash on the numbers and I would be left holding the bag or missed the boat. You get the point.

It has been my experience that Yahoo Financial calendar provides the most consistent and update earning report for listed companies. (If there any better ones out there please do email or do PR for the repo.)  So now that I have found a source, how to do I make take that information and standardize and normalize it into a database. Obviously once I have that information in a database, I can use that data to purchase calls, puts or evaluate the earnings date based on a host of other information I have available to “cross reference” or analyze.

Luckily, through the power of scraping + python + github anybody can now have standardized earnings date reports and now I can scrape and standard the data and manipulate it to my needs.

In my repo I added the file earnings.py.  This is where the magic occurs..

Just a simple Mysql table to capture date, stocks, eps estimates, etc…

q = "truncate yahoo_earnings"
cursor.execute(q)

Then we have to a little “wonky” stuff with the dates that we are going to query. It seems like weather I’m in PHP or Python, I’m always getting entagled in dealing with DATE or TIME, if anybody can parse those dates better please send at PR.

#set dates
now = datetime.now()
startDate = now.strftime('%b %d %Y %I:%M%p')
print(startDate)
endDate = now + timedelta(days=60)
endDate = endDate.strftime('%b %d %Y %I:%M%p')
print(endDate)
startDate = datetime.strptime(
startDate, '%b %d %Y %I:%M%p')
endDate = datetime.strptime(
endDate, '%b %d %Y %I:%M%p')

Then as you can see from the code it’s pretty simple after that, call the function, parse the data, load it into your database or where ever you need it. Oh I forgot to mention, Make a ton of Money.

All this code can be found in my stock and options API repo, specifically with the Earnings-Scraper/earnings.py file that I’m discuss here. If you want the raw repo from wenboyu2

 

How to get option prices for free – API Yahoo

I don’t know why this was so difficult to find this on google, but eventually I found it in the code of a node js library.

Deep inside the nodejs library is the url to query option data. I don’t think this is a secret but yahoo sure wants it that way with there use of YSQL nonsense.

Regardless, here is the url: https://query1.finance.yahoo.com/v7/finance/options/AAPL

Obviously, AAPL is apple computers. This will return nicely formed JSON data. It will also return all the existing expiration dates at the top of the json object.

If you want a specific date the you just add the ?date=1505433600 to the end of the URL.  The date is converted into unix time, which you can translate here.

https://query1.finance.yahoo.com/v7/finance/options/CMG?date=1505433600

I will use php to parse the json object, drop it into database, which wasn’t that difficult. The key take away is the formation of the URL call to Yahoo. One can use any language to parse the data.

Surprisingly, this is pretty simple.

Leave questions or concerns in the comments. I’m also available for hire.