So you want to build a financial trading algorithm?

How to build a Financial Trading Algorithm.

It all starts with an idea… or a little bit of anger. 

Chances are pretty good your algorithm strategy wasn’t developed because your bank account is filled with fiat currency or because you really like your day job. Just like blogging, all you need is an internet connection and resentment. 

Usually the strategy or algorithm comes from losing a lot of money when making trades from your gut feeling or listening to the advice of a taxi driver. Hopefully this isn’t the case, but possibly you were taking trade ideas from Wall Street Bets or it could come from everybody’s favorite Wall Street clown. More often than not great algorithmic trade strategies don’t just happen because you were clicking around on charts and nothing better to do.  

Recently, I was approached by someone to program their algorithm. The idea seemed a little half baked, but possibly could work. Since I have experience in this area and know some of the in and outs of this type of work, I usually put together a brief scope of work. In the end I didn’t get the job but my scope of work turned out to be a pretty good blog post on the basis of build a algorithm. I didn’t get the job because I was probably too expensive and I ask lot of questions, well cause I like to think things thru. 

Phase 1: “Let’s see what sticks”

Build a “Scrubber” that would take stocks/options and process them thru your algorithm. The result would be placed in a database / excel sheet for your further analysis. 

The goal would be to use significant amounts of stocks/options and data attempting to confirm your algorithm thesis and find any significant trends and capitalize on them. The more stock/options that you can run thru the “scrubber” the stronger your algorithm will be. My current  stock algorithm runs over 500 stocks. 

What I need to get the project started: Bullet point description of the algorithm. Need a control method, to ensure the algorithm identifies the correct trades. 

Cost: MP

Phase 2: “What worked, what didn’t and what worked surprisingly well”

Continued development of the algorithm, making changes to the algorithm based on the findings from the data. Such as adding a volatility filter? As this case with most algorithms, and which I believe is the exciting part is discovery. Analysing the data and finding new ways to increase profitability. These would be small changes in the code to “tweak” the algorithm.  

Cost: ~hourly basis

Phase 3: “4 weeks of NFL preseason”

I don’t believe in NFL preseason games! Okay, maybe not 4 weeks. But it’s important to set up a paper trading account at Interactive broker. Make sure that slippage isn’t an issue and there are no other execution issues. Do market orders work better than limit orders? Do you have restrictions on your account (margin limits)  Are orders backing up in the queue? Slippage? Account limitations… meaning you might not have enough money in the account to make 10 trades, but can make 9 trades?

User requirements: server (AWS or computer connected to the internet)

Cost: MP

Phase 4: “Make it Rain”

Live trading and making $$. Set up, training and maintenance. Paper trading will be the major hurdle, but there might be a few unknown items moving into the final phase. From my experience, the difference of paper trading to live trading is pretty much a flick of the switch. 

Cost: MP


The unknown: Human emotion is the biggest hurdle that I have run across. Currently, my algorithm runs 24 hours a day 5.5 days a week. I only know it is working when I check in on my account or I get the IB reset alert on slack. Otherwise, the algo is just turn out trades, regardless of what the president tweets or what wars or civil war we are currently in. It takes a little bit of time to accept the algo is working and all your hard work is doing what it’s supposed to be and you can kick back a relax.  

Most people generally accept that when flying and at a cruising altitude, the pilot has the plane on auto pilot and he is just monitoring controls, not flying the plane. We have a certain faith that the pilot and the autopilot feature are doing what they need to do. The same goes with a financial algorithm, you have to trust that it’s working and that isn’t an easy thing to do at first when money is on the line. 
So that is the basic layout of putting an algorithm together, I’m sure I missed a few things and maybe over exaggerated others. Just because one has made a few successful trades on Robinhood, doesn’t mean you are ready for an automated trading. So with that it takes a lot of work and if you want to work with me the starting price is at a minimum $7,000. In the event you are still interested, start the conversation off with my payment of 7,000 M&Ms and make sure they are all green!

Update 2020: The Best and Worst Stock, Futures and Option Trading APIs

Update 2020: The Best and Worst Stock and Option Trading APIs

My original post is still here, I get a lot questions, so I thought I would update with my current findings.

When picking an API the $64,000 question is  does Charles Schwab (TD Ameritrade) or Morgan Stanley (Etrade) really want to deal with building an out API for their customers once the mergers happen? Meaning is there enough revenue generated from API trading to keep 10 annoying coders with at least 2 dev ops bros who believe they know everything and of course an annoying project manager.  I personally believe these firms kind of see the API as dead weight, with only a few customers and the trend is more getting day traders like Robinhood has?  Keep in mind brokerages generally don’t have the trade fees they used to have.

Td Ameritrade

The TD Ameritrade API is courtesy not a guaranteed feature. (This was told to me by a customer service rep) They don’t necessarily provide support if you send them a question at [email protected] If it’s an easy question they might respond, if it’s a difficult one then they won’t. Reddit / stackoverflow is too fond of TD API. Otherwise their documentation generally is terrible. I currently only use TD API for live option quotes, because I like they can give the entire option chain so I can analyze it on “my  local”. It generally is the worst API that I have ever used. Since it’s more of a courtesy I I  On the flip side I Think or Swim is awesome.

Etrade API

I have poked around the new eTrade API, it appear to be very easy to use. I think they are using standard REST API methods etc. Meaning they are probably using a framework such as api-platform to build it. From what little I can see it seems like it is rather too easy to use and well documented.

Interactive Brokers

Interactive Brokers is what I trade options and futures with, their code has a learning curve, it is not terribly nice and sometimes you just have to wonder about serious WTF. Once you get past that and consider it is really the only game in town, it works just fine. I’m trading futures 24 hours a day, with no problems. IB really focuses on their API and has a dedicated staff, support etc.

etc…

I have not had time to test out Alpaca. I don’t get paid to write these reviews (I wish I did) So since I’m a practitioner, I don’t have an incentive to test out Alpaca or any other API at this time. Although IB is a pain in the butt, it’s working just fine for me.

I ran across this website the other day… Any wants to join forces or pay to front run Robinhood, shoot me an email.  This is some low hanging fruit.

Recommended Books for building trading algorithms

A quick list of books that have shaped and molded my investment / trading algorithmic trading startegies.

Notice I didn’t write 10 books you have to read to make millions of dollars in the market.That just seems a little click-bait/Buzz Feedy. Unfortunately, from a data scientist/ SEO perspective that would have been the more profitable. 

As we all suffer through this quatrine I just put together a list of books that I have read that I believe have shaped my understanding of markets and have molded my investment philosophy. I think bigger picture books such a The Big Short and Black Swan are far more important than technical books.

Creating algorithms and or trading strategies is something that I believe starts with the creative process, maybe just staring at charts for hours or losing a lot of money. I’m lucky, I’m a self taught coder and I can implement my strategies without getting caught up in whether my code is object oriented and if I have the correct classes etc. I always say that the code is the easy part, it’s finding a good strategy that is hard part.

So these are some books that I have enjoyed and maybe you will too as well. If you have any recommendation, please leave them in the comments as I’m always looking for something to read / listen to. 

Books that shaped my investment philosophy or possible continue to shape my philosophy strategy.

The Creature from Jekyll Island: A Second Look at the Federal Reserve by G. Edward Griffin
The Warren Buffet Way by Robert G. Hagstrom, jr.
How Technical Analysis Works by Bruce m. Kamich
The Black Swan By Nassim Nicholas Taleb
Beating The Street By Peter Lynch
The Big Short by Micheal Lewis
Flash Boys by Micheal Lewis
Confession of an Economic Hit Man by John Perkins
A mathematician plays the stock market by John Allen Paulos

Here are some other books, that I have read that but am not giving an a full endorsement. I especially disliked “Random Walk”, but that seems to be Wall Street favorite.

A Random Walk Down Wall Street: The Time-Tested Strategy for Successful Investing by Burton G. Malkiel
AntiFragile By Nassim Nicholas Taleb
Skin In the Game By Nassim Nicholas Taleb

Malcolm Gladwell and Freaknomics books are good audio books and generally pretty easy to consume quickly:
The Tipping Point by Malcolm Gladwell
Blink by Malcolm Gladwell
Outliers by Malcolm Gladwell
What the Dog Saw: and Other Adventures by Malcolm Gladwell
Boomerang by Micheal Lewis
Freakonomics by Steven D. Leviit and Stephan J Dubner

The Intelligent Investor: The Definitive Book on Value Investing by Benjamin Graham

Academic / Text Books
Option Volatility & pricing by Sheldon Natenberg
The Complete Guide to Option Pricing Formulas By Espen Gaarden Haug (This book is great for the formulas)

High-Frequency Trading by Irene Aldridge (Probably outdated and really that good.)

Simple guide to build TWS API on AWS EC2 instance

Simple guide to build TWS API on AWS EC2 instance

Currently, I’m running 2 different Interactive Brokers Trader Work Station (TWS), my own propriety python trading algorithm on an AWS EC2 instance. This is very simple instructions to get you one started, it is not the cure all.

  1. Build an AWS EC2 instance / 18.04 Ubuntu
  2. SSH into the instance
  3. Insert the following code as required

Cover the basics:

Sudo mkdir code && 
cd code && 
sudo apt-get update -y &&
sudo apt-get clean -y &&
sudo apt-get autoremove -y && 
sudo apt-get install mysql-server -y && 
sudo apt-get update -y &&
sudo apt install python3-pip -y && pip3 install pandas matplotlib seaborn plotly pymysql sklearn 

The are two ways to VNC into your EC2 instance. The RealVNC method is little more complicated and it is an enterprise solution.

  1. https://help.realvnc.com/hc/en-us/articles/360003474572-How-do-I-get-started-with-VNC-Connect-on-Linux-
  2. xhost +local:Name_of_your_computer
  3. sudo apt install ./VNC-Server-6.6.0-Linux-x64.deb
  4. tar -xzf vncsetup-helper.tar.gz
  5. sudo ./vncsetup.sh
  6. sudo reboot

If you want a simple VNC method follow this bloggers instructions.

Interactive Brokers

  1. follow instructions: http://interactivebrokers.github.io/
  2. sudu unzip twsapi_macunix.979.01.zip -d /home/user/
  3. cd IBJts/source/pythonclient/
  4. sudo apt-get install python3-setuptools && sudo python3 setup.py install
  5. Download TWS https://www.interactivebrokers.com/en/index.php?f=16040 follow instructions on the screen. (if you want to use IB gateway, which is not as sexy but works fine you can get it here)
  6. git clone your repo

The Best and Worst Stock and Option Trading APIs

Update: 2020 The Best and Worst Stock, Futures and Option Trading APIs

In my quest to program and build my own trading system, I have discovered a lot of conflicting information on the “Internets” about trading APIs and stock and option price quotes.  In the past, I posted on HN news about some of my findings, only to get some great new insights. One thing I can’t find is a simple location for all trading APIs and I have stumble along some rabbit holes when dealing with the APIs, trying to see what works and what is no longer supported. With that said, I will will be launching a General information Git Repository, to hopefully provide links to SDK for trading API and price quote APIs, Etc. I will obviously do pull requests, but my opinions and finding on certain trading systems will be detailed below and on this site.

Continue reading “The Best and Worst Stock and Option Trading APIs”