How to Scrape and Parse Stock Earnings Reports

For years I looked for a simple way to get earnings reports from Wall St. It seems they would change their earning reports on a regular basis and the dates would be inconsistent. Then one day when I wasn’t paying attention, BAM. Earnings report and the stock would take off or crash on the numbers and I would be left holding the bag or missed the boat. You get the point.

It has been my experience that Yahoo Financial calendar provides the most consistent and update earning report for listed companies. (If there any better ones out there please do email or do PR for the repo.)  So now that I have found a source, how to do I make take that information and standardize and normalize it into a database. Obviously once I have that information in a database, I can use that data to purchase calls, puts or evaluate the earnings date based on a host of other information I have available to “cross reference” or analyze.

Luckily, through the power of scraping + python + github anybody can now have standardized earnings date reports and now I can scrape and standard the data and manipulate it to my needs.

In my repo I added the file  This is where the magic occurs..

Just a simple Mysql table to capture date, stocks, eps estimates, etc…

q = "truncate yahoo_earnings"

Then we have to a little “wonky” stuff with the dates that we are going to query. It seems like weather I’m in PHP or Python, I’m always getting entagled in dealing with DATE or TIME, if anybody can parse those dates better please send at PR.

#set dates
now =
startDate = now.strftime('%b %d %Y %I:%M%p')
endDate = now + timedelta(days=60)
endDate = endDate.strftime('%b %d %Y %I:%M%p')
startDate = datetime.strptime(
startDate, '%b %d %Y %I:%M%p')
endDate = datetime.strptime(
endDate, '%b %d %Y %I:%M%p')

Then as you can see from the code it’s pretty simple after that, call the function, parse the data, load it into your database or where ever you need it. Oh I forgot to mention, Make a ton of Money.

All this code can be found in my stock and options API repo, specifically with the Earnings-Scraper/ file that I’m discuss here. If you want the raw repo from wenboyu2


How do you get a job in algorithmic trading?

TL;DR: Looking for employment resources in Finance Programming / Trading algorithms? I’m in a rock and hard place…

I have the ability to write stock trading algorithms, portfolio trading algorithms. Also Cryptocurrencies, Options, and if need be futures and FX.

I blame the great depression for my ability to code, infact I call myself a “self taught depression coder”. Meaning that during the great recession I couldn’t find a job, so I took off my MBA in Finance on my resume and started teaching myself PHP and MySql. I also know Javascript, python, node, etc, to name a few.  So I don’t have the Computer Science Degree or Statistics PhD , but I’m right in the middle with an MBA and proven coding skills.

Currently, I’m a fully employed programmer. Why you might ask? Well because I can’t seem to walk down the street without someone trying to hire me for a programming job for the last few years. The pay is great, the risk is low and generally the stress is low. I have made lot of money programming other peoples ideas. Oddly enough, a lot of them failed, not because of me, but mostly because the ideas where not that great in the first place.  For example, a new paypal system for marijuana or a website to list trailers? Hmm… last time I checked it was free to list trailers on craigslist. Oh well, even though the money is great, there is a limit to how much you can get paid versus boredom and intellectual challenge.

So know I have slowed down any side work and only work on my algorithms at night. My day programming job is not intellectually challenging, so I have to spend my nights writing algorithms to remain intellectually stimulated. I’m also an SEC register online adviser (RIA), but I don’t have any clients as marketing is not necessarily my strong suit.

Basically, I’m looking for people or websites that might help me find a job in the area of quantitative finance / algorithm trading. Not interest at working at one of the banks, or investment firms cause I’m sure they got it all covered. But maybe a smaller shop. The only other stipulation is that I have made my home in Denver, with no desire to move to Co-Location alley (NJ,NY).

I was wondering if any can point me to a website/recruiters/firms/portfolio manager  that specialize in algorithmic trading or if there are an Adviser or RIA that would like to discuss some opportunities.

A click and show resume can be found here.   Some of my trades can be found here.

Drupal Vs. Laravel

A few months ago I posted on HackerNews a question about Laravel vs. Drupal. Although an unofficial poll, One can the overwhelming response in favor of Laravel. However, gather feedback from the Internets wasn’t enough.

A little background, I’m working on a contract for a large organization and the administrator of the contract who has a background in system administration decided to use Drupal 8 as project to prevent people from uploading directly to the production server from Dreamweaver, SCP, FTP. terminal or some archaic method. (There is absolutely no deployment methodology) Prior to the sys admin leaving in July, I asked the sysadmin why he chose Drupal? His response was “Well I programmed in it briefly, like 10 years ago so I think it will help us.” So that was the rational. I started to learn to program in Drupal, I quickly learned that essential it is a pain in the @ss to program in Drupal. In fact it is downright excruciating.

Spending a few weeks trying to implement a simple group permission module that would allow certain writers access to certain articles and deny writers access to different articles, etc. What I would consider very basic content moderation / group access, there only two modules out there, Groups and Organic Groups and neither of them seem to work very well. Most importantly, most developers haven’t moved over to Drupal 8, so there doesn’t seem to be a rush to support these modules.  In addition, I had coded out content moderation / group access features in php / Codeignitior / MVC with no problems in the past.

With the  Sys Admin, who did a little coding 10 years ago, now gone, I saw opportunity to change direction, as I didn’t want to get stuck coding in Drupal. I address the issue with the new administrator and instead of telling him that I was going to switch to a new framework, I thought it was only fair to a present my case. I believe that it is important for administrators or senior leadership to make informed decisions. All to often I have seen developers just tell non technical people what they are going to and then start programming in their flavor of the week coding language. With that said I would put together a presentation on Drupal vs. Laravel and make non technical person to understand. I was excited about this because writing code is fun, but sometimes I like add a little more challenge and this would provide an opportunity.

The key issue really is 40% of the project really setting up a simple CMS system to publish articles. The other 60% of the project is completely unknown. It could be something like converting oracle data and displaying in a custom CMS view.

The one thing I did get to Sys Admin was to get a simple mission statement as to lock him down on the specifics of the project. So here is the presentation:

Mission Statement / Directive: To build a comprehensive Content Management System (CMS) using the Drupal framework to allow different stakeholders and pre approved groups the ability to create appropriate content (news, embeded video, images) and static pages for publication on the website. The public facing will only have read access and all content management (CRUD) will occur behind a protective firewall.

With that here is my presentation:

Laravel vs. Drupal

Problem: Develop a CMS for an unknown project description and make sure the CMS system is adaptable enough that it can implemented across a variety of project / websites that have not been defined.

Current situation: Currently we are attempting to implement Drupal 8, which was was released on Nov 2015. Drupal 8 is significantly different than Drupal 7 and does not have an end of life date. This is important to note that current web developers are not converting sites to Drupal 8 at an accelerated pace because they can continue to use Drupal 7. Hence the development on Drupal 8 and drupal modules has been slower and spotty. Implementing Drupal 7 would be an issue as we would be confronted with an upgrade to Drupal 8 in the future. Using Drupal 7 would only be an effort to kick the can down the road.

Proposed solution: After working with Drupal over the course of a few months, it appears the community is stuck in Drupal 7 and possibly abandoning the platform. Solutions to Drupal 8 issues appear to be sparse. In addition some key modules are in beta. Modules for group permissions, there are only two modules that support this feature. Which really makes this project dependent on the two modules. Permission to manage content is standard with WordPress.

After careful consideration and research. It is recommended we move over to the Laraval / PHP framework without altering our November or new October 1st deadline, maybe even decreasing it. Laraval is a robust framework that allows us to implement the most basic requirements of the “Drupal – RISE – Project” of creating a simple blog/article ability with basic Admin capabilities. The beauty of laravel allow to adapt and build features that we don’t know exist, for example connecting the water levels with mussel data should be able to develop without being bound to constraints that Drupal might place on creating new features. Although all constraints of a Drupal or Laravel can be overcome through programming, Laravel is just built in a way, that you can get up and running faster to implement new features.

PRO Laravel:
-Up and coming community
-Laracast (well documented support and video learning)
-Well received / received in the community
-Eloquent ORM (immediate protection against Mysql attacks)
-Lumen (micro service / API)
-Far greater admin user interface than Drupal
-Follows traditional MVC framework
-LDAP integration
-Super easy documentation:
-Super easy and fast
-Implements a simple Model-View-Controller architecture

CON Laravel:
-New system
-Start from the beginning
-Started in 2012
-Calls it’s programmers, “Web Artisans”

PRO Drupal:
-Work already exists, LDAP, admin
-The traditional and established “work horse”, built in 2001
-Empirical evidence appears that drupal is used for newspapers format.

CON Drupal:
-Open source community seems to be stuck between Drupal 7 and Drupal 8, minimal development
– (Module Derick needs is stuck on Drupal 7)
-Fading use as illustrated in charts below.
-Not traditional MVC environment (although moving towards it)


Stack Overflow:

Drupal Users Group: = 7,066 readers
Laravel Users Group: = 11,028 readers

Epilogue: In the end, I wasn’t able to persuade the new Sys Admin to move over to Laravel, we are stuck with Drupal.  Not sure if I really lost as much as I might not have hit the right selling points with the audience. That can be difficult, when the mission statement is very open and the end goals are not defined.  I did have some competition as another developer presented the case for Drupal.  Any frame work is going to be flawed, but the lack of support in the Drupal 8 community is a serious issue. On the bright side, I was able to gain exposure to Laravel as I got to play around with it for a week or so. Although my background is Codeigniter, Laravel is super fast to pick up super quick to learn.

Leave back here or on hackernews, be interesting to see the communities thoughts.
















The Best or Worst way to decline a technical interview.

Hiring practices in the tech field seem to be topic of discussion on HN. I’m truly, amazed at some of the hiring practices  in the tech field.  Passion and desire are overlooked for technical knowledge. It truly is like no other field… With demand for developers, it kind of shocks me there are not better methods. CTO / Sr. developers should not be the sole gate keeper of hiring developers. (I’m sure that will get me in trouble…  notice that I used the word “sole”, yes they should have some input in the hiring decision.)

Recently (last week, to be exact), I applied for a position Senior Software Development position. I was intrigued by the salary, working remote and some of the leadership opportunities. However, I knew at some point it was going to lead a stereotypical technical interview. (This might come as a shock, but the first question on the tech interview was “Describe Object Oriented Programming?”) Being a self taught programmer, I unfortunately take great pride in knowing that I will fail miserably. I also seem to have lot of desire not to speak the lingo of the tech interview either, as I would rather spend more time learning new code. With that I really never should have taken the interview, as I honestly believe I’m qualified, but don’t really care to have a pop quiz over the phone about what I consider academic programming questions. My work, my side projects, code examples, desire to learn should illustrate that on a resume (7 years of paid programming experience), that was never discussed in the technical interview.

So I went thru the culture fit with a non technical / culture interview with no problems. Then came the technical interview…. as you can see from below it was an epic fail. As laid out below it’s clearly my fault. In fact it might have been comical to some degree. But regardless weather I wanted the job or not, rejection is frustration and annoying.  So I shouldn’t have agreed to the interview in the first place as I knew the outcome. While at the same time I feel extremely comfortable there wasn’t anything in their code that I couldn’t have done. At the end of the day it was my fault for being frustrated and I don’t want to be angry at myself. I would rather save my anger for injustice in the world.

After the interview, I sent the following email to hopefully relieve myself of some my anger. After clicking send, I do feel better… So there is in a opening in Denver, maybe I will get a referral fee…



I’m going to respectfully withdraw my application for employment for HxxxxXxxxx. I don’t feel that meet the technical requirements as laid out in the technical interview.

This might not come as a surprise as the technical interview was a interesting adventure in clearly explaining I don’t know the answers to academic programming questions. Since I’m not a typical programmer and self taught programmer, I believe that might have surprised the interviewer with my willingness to clearly state that I had no experience with a particular type of procedure and politely explain that I didn’t have answer. (I could almost feel the pause on the phone as I answered honestly, that I didn’t know the answer. I also mentioned that I wasn’t going to try to google question while on the phone either, which seems to be a new trend.) Honesty, can be a terrible curse. It’s 4 years since I retired from the Marine Corps it’s concept of honestly and responsibility are still haunting me.

Regardless, please be assured that I have absolute no animosity towards the HxxxxXxxxx or the people that interviewed me. I’m not lashing out or frustrated. The animosity lies completely with myself. Yes, that’s right. The responsibility lies completely with myself that has placed me in a frustrated mood. Unlike our current political / societal environment, it’s all me. I should have never taken this interview as I generally winds up with a stereo-typical technical interview and I know that I’m not going to pass. Which ultimately meaning that, I shouldn’t have applied for the position. Yeap, I should have never taken the interview. I’m writing this email for three reasons 1. Unburden some of my personal frustration. 2. This will also serve as a “Memento” to myself, not to fall into this self inflicted trap I successfully get myself into every few months as I seem to have mild form of amenisa regarding this issue. 3. Sometimes I like to step away from the code and practice some of communication skills and self awareness. For this self inflicted wound, I apologize for expending your time and mine.


Chad Humphrey

Part of the reason I like to post things like this, is I get the see other people reactions on HN. Which can be good and bad, but I’m willing to see things from another perspective.

How to get option prices for free – API Yahoo

I don’t know why this was so difficult to find this on google, but eventually I found it in the code of a node js library.

Deep inside the nodejs library is the url to query option data. I don’t think this is a secret but yahoo sure wants it that way with there use of YSQL nonsense.

Regardless, here is the url:

Obviously, AAPL is apple computers. This will return nicely formed JSON data. It will also return all the existing expiration dates at the top of the json object.

If you want a specific date the you just add the ?date=1505433600 to the end of the URL.  The date is converted into unix time, which you can translate here.

I will use php to parse the json object, drop it into database, which wasn’t that difficult. The key take away is the formation of the URL call to Yahoo. One can use any language to parse the data.

Surprisingly, this is pretty simple.

Leave questions or concerns in the comments. I’m also available for hire.




Why are Tech recruiters so #$%& ?

Your mind is filling in the blanks, but in fact I didn’t say anything negative. What I meant to say is: Why are Tech recruiters so smart?

Regardless, I do have your attention and chances are really good you are a tech recruiter.  With that said I’m looking for work, I’m not your average programmer… I have the ability to think, program, code, algorithms, communicate, lead and provide strategic direction. brogrammers

So there are plenty of Brogrammers, man-children and star trek programmers out there for you to hire, I’m sorry I’m not one of those.

Click here for a resume that does more than list off all the programming languages I have ever worked with, because let’s face it we nobody reads those resumes.

You can’t have more than 10 years of experience on Rails!

This is why programmers dislike account people / business people…

So last week, I was selected for an interview for Sr. Web Developer. The job sounded interesting so I took time out of my day to have phone interview. I  noticed the organization  mostly programmed in Ruby on Rails, but then it said PHP and other languages and I was like okay.  See below for the description.

So the interview went fairly well, I really never address the issue Ruby on Rails, there was also some confusion as to Bootstrap.  According to the project manager Bootstrap is also responsive. (No kidding?) I said that’s great because I use Twitter Bootstrap and that is responsive as well. (I was playing games at that point) Then I was told they where not the same? I don’t know if that was the breaking point, but it was amusing.

Finally, I was  told I would get notification of the outcome of the interview. I think that is fair since I took time out of my day… a certain level of professionalism and courtesy. So I shot a quick email back just making an inquiry if there was a decision since I hadn’t heard anything. I received the following in an email.






Oh well, I will live to fight another day… But then I remember that I follow @DHH on twitter and Ruby is kind of new. (Just an FYI @DHH created Rudy and he did it ~10 years ago. I follow @DHH not because of Rails, mostly because of his libertarian oriented tweets, much like myself.)  So to amuse myself I asked him:

He was kind of enough to reply:



Lead web developer. Love a challenge? Work from anywhere. $100k+ (Denver)

compensation: $100k+
  • Do you stand out in a crowd of software engineers as the “go-to” person?
  • Do you love inventing cutting-edge software systems that launch startups and help forward-thinking businesses grow exponentially?
  • Can you architect, lead and build web development projects that require 2 to 4 programmers for a year or two at a time?
  • Do you have an entrepreneurial streak and want to help our scrappy 15-person software consultancy grow up?/li>

If so, we should talk.

Dolphin Micro partners with unique startups and established businesses like AT&T and Wired to build elegant, robust custom software. This month we’re busy streamlining order fulfillment, helping people learn faster, and helping ambulance dispatchers save lives.

We build mostly in Rails and Django, with the latest client side frameworks like Angular and Bootstrap. If you’ve been programming professionally for around a decade, have a few years of full stack web experience (Rails, Django, PHP, Java Spring, and similar), and love learning, you’ll be a great fit.

What we offer:

  • Competitive benefits package, profit share and paid continuing education
  • Flexible schedule, work from one of our offices (Denver and New York) or from home anywhere in the US

About us:

We continuously strive to make Dolphin Micro a great place to work–interesting enough to be a lifelong career and profitable enough that it’s the last job you’ll ever need.

We have a talented sales team that finds us cool clients and challenging projects, super smart developers that love working in small teams to invent and ship software, and business-savvy project managers that pave the way for our developers and get things done.

Interested in joining? Send us your resume and a brief paragraph on why, with the subject line “sounds like fun.”

* Please note that Dolphin Micro screens all potential employees carefully before extending any job offers. If you are selected for a second interview, you will be asked to consent to a criminal background check. Only those consenting to a background check will be considered for employment.

  • Principals only. Recruiters, please don’t contact this job poster.
  • do NOT contact us with unsolicited services or offers

My Favorite Sample of Code

Sometimes I click thru the job postings to see what’s out there or just to have fun. I happened to run across this:

Interested applicants should reply with their favorite color and programming language. A github account, favorite code sample, or creative drawing will also be accepted.

Passing variables between functions is so much fun, that I should have a favorite line code! Hysterical.

But in order to answer the question for the sake of entertainment, my favorite line of code is on the scroll just to the right. Not because developed a new way to pass variables into functional Ajax arrays (I just made that up), but because the code makes $$.  Honestly,  people that work in web development seem to forget that? They can’t see the forest from the trees, then again business people can’t see the trees from the forest.

Here is the job listing…, I’m going to email this post as my response. I wonder if I will get the job…

In the meantime, this video explains life in corporate America for a coder.