How to get historical data from Interactive Brokers

Work in progress… details to follow but this some sample code that works

   @printWhenExecuting
    def historicalDataOperations_req(self):
        contract = Contract()
        asset_info = quotes_library.get_asset_info()
        for val in asset_info:
            print(val)
            self.assetId = val['assetId']
            tickerId = val['assetId']
            contract = Contract()
            contract.symbol = val['base_symbol']
            contract.secType = "FUT"
            contract.exchange = val['exchange']
            contract.currency = "USD"
            contract.lastTradeDateOrContractMonth = val['lastTradeDateOrContractMonth']
            self.reqHistoricalData(tickerId, contract, "", "1 D", "1 min", "TRADES", 0, 1, True,[])
      
    
    @iswrapper
    def historicalDataUpdate(self, reqId: int, bar: BarData):
        # print("HistoricalDataUpdate. ReqId:", reqId, "BarData.", bar)
        quotes_library.load_IB_1_min_realTime_bars(reqId,bar)
        
    @iswrapper
    # ! [historicaldataend]
    def historicalDataEnd(self, reqId: int, start: str, end: str):
        super().historicalDataEnd(reqId, start, end)
        print("HistoricalDataEnd. ReqId:", reqId, "from", start, "to", end)
        self.keyboardInterrupt(self)
    # ! [historicaldataend]

Think or Swim on Ubuntu 20.04 / Focal Fossa

It will be interesting to see what happens with Think or Swim and Charles Schwab in the coming years as the two merge. Retail brokerages are slowing consolidating and going by the wayside as Robinhood / options become all the rage.

I use Interactive Brokers for all my trading because I believe hands down they have the best API for all my needs. However, I use TOS in all my chart analysis and developing my algorithmic strategies.

I have written about installing Think or Swim on previous versions of Linux, so I thought I would update to Focal Fossa cause once again TOS directions didn’t work.

Start with some java:

1. install sudo apt install openjdk-8-jre

2. Download the installer

3. cd /Downloads && sudo chmod +x thinkorswim_installer.sh

4. install sh ./thinkorswim_installer.sh

m The installer should start the application for you. The TOS small box will pop up and say that it is installing updates. This seems to take a long time, like 5 minutes? So go do something else for a few. Then you should get a login screen after a while.

Note: If you find any discrepancies or tips please leave them in the comments. I will do my best to post them, as this blog post and the other Ubuntu / TOS get a lot of hits, there are people out there that need hits.

So you want to build a financial trading algorithm?

It all starts with an idea… or a little bit of anger. 

Chances are pretty good your algorithm strategy wasn’t developed because your bank account is filled with fiat currency or because you really like your day job. Just like blogging, all you need is an internet connection and resentment. 

Usually the strategy or algorithm comes from losing a lot of money when making trades from your gut feeling or listening to the advice of a taxi driver. Hopefully this isn’t the case, but possibly you were taking trade ideas from Wall Street Bets or it could come from everybody’s favorite Wall Street clown. More often than not great algorithmic trade strategies don’t just happen because you were clicking around on charts and nothing better to do.  

Recently, I was approached by someone to program their algorithm. The idea seemed a little half baked, but possibly could work. Since I have experience in this area and know some of the in and outs of this type of work, I usually put together a brief scope of work. In the end I didn’t get the job but my scope of work turned out to be a pretty good blog post on the basis of build a algorithm. I didn’t get the job because I was probably too expensive and I ask lot of questions, well cause I like to think things thru. 

Phase 1: “Let’s see what sticks”

Build a “Scrubber” that would take stocks/options and process them thru your algorithm. The result would be placed in a database / excel sheet for your further analysis. 

The goal would be to use significant amounts of stocks/options and data attempting to confirm your algorithm thesis and find any significant trends and capitalize on them. The more stock/options that you can run thru the “scrubber” the stronger your algorithm will be. My current  stock algorithm runs over 500 stocks. 

What I need to get the project started: Bullet point description of the algorithm. Need a control method, to ensure the algorithm identifies the correct trades. 

Cost: MP

Phase 2: “What worked, what didn’t and what worked surprisingly well”

Continued development of the algorithm, making changes to the algorithm based on the findings from the data. Such as adding a volatility filter? As this case with most algorithms, and which I believe is the exciting part is discovery. Analysing the data and finding new ways to increase profitability. These would be small changes in the code to “tweak” the algorithm.  

Cost: ~hourly basis

Phase 3: “4 weeks of NFL preseason”

I don’t believe in NFL preseason games! Okay, maybe not 4 weeks. But it’s important to set up a paper trading account at Interactive broker. Make sure that slippage isn’t an issue and there are no other execution issues. Do market orders work better than limit orders? Do you have restrictions on your account (margin limits)  Are orders backing up in the queue? Slippage? Account limitations… meaning you might not have enough money in the account to make 10 trades, but can make 9 trades?

User requirements: server (AWS or computer connected to the internet)

Cost: MP

Phase 4: “Make it Rain”

Live trading and making $$. Set up, training and maintenance. Paper trading will be the major hurdle, but there might be a few unknown items moving into the final phase. From my experience, the difference of paper trading to live trading is pretty much a flick of the switch. 

Cost: MP


The unknown: Human emotion is the biggest hurdle that I have run across. Currently, my algorithm runs 24 hours a day 5.5 days a week. I only know it is working when I check in on my account or I get the IB reset alert on slack. Otherwise, the algo is just turn out trades, regardless of what the president tweets or what wars or civil war we are currently in. It takes a little bit of time to accept the algo is working and all your hard work is doing what it’s supposed to be and you can kick back a relax.  

Most people generally accept that when flying and at a cruising altitude, the pilot has the plane on auto pilot and he is just monitoring controls, not flying the plane. We have a certain faith that the pilot and the autopilot feature are doing what they need to do. The same goes with a financial algorithm, you have to trust that it’s working and that isn’t an easy thing to do at first when money is on the line. 
So that is the basic layout of putting an algorithm together, I’m sure I missed a few things and maybe over exaggerated others. Just because one has made a few successful trades on Robinhood, doesn’t mean you are ready for an automated trading. So with that it takes a lot of work and if you want to work with me the starting price is at a minimum $7,000. In the event you are still interested, start the conversation off with my payment of 7,000 M&Ms and make sure they are all green!

Update 2020: The Best and Worst Stock, Futures and Option Trading APIs

My original post is still here, I get a lot questions, so I thought I would update with my current findings.

When picking an API the $64,000 question is  does Charles Schwab (TD Ameritrade) or Morgan Stanley (Etrade) really want to deal with building an out API for their customers once the mergers happen? Meaning is there enough revenue generated from API trading to keep 10 annoying coders with at least 2 dev ops bros who believe they know everything and of course an annoying project manager.  I personally believe these firms kind of see the API as dead weight, with only a few customers and the trend is more getting day traders like Robinhood has?  Keep in mind brokerages generally don’t have the trade fees they used to have.

Td Ameritrade

The TD Ameritrade API is courtesy not a guaranteed feature. (This was told to me by a customer service rep) They don’t necessarily provide support if you send them a question at [email protected]. If it’s an easy question they might respond, if it’s a difficult one then they won’t. Reddit / stackoverflow is too fond of TD API. Otherwise their documentation generally is terrible. I currently only use TD API for live option quotes, because I like they can give the entire option chain so I can analyze it on “my  local”. It generally is the worst API that I have ever used. Since it’s more of a courtesy I I  On the flip side I Think or Swim is awesome.

Etrade API

I have poked around the new eTrade API, it appear to be very easy to use. I think they are using standard REST API methods etc. Meaning they are probably using a framework such as api-platform to build it. From what little I can see it seems like it is rather too easy to use and well documented.

Interactive Brokers

Interactive Brokers is what I trade options and futures with, their code has a learning curve, it is not terribly nice and sometimes you just have to wonder about serious WTF. Once you get past that and consider it is really the only game in town, it works just fine. I’m trading futures 24 hours a day, with no problems. IB really focuses on their API and has a dedicated staff, support etc.

etc…

I have not had time to test out Alpaca. I don’t get paid to write these reviews (I wish I did) So since I’m a practitioner, I don’t have an incentive to test out Alpaca or any other API at this time. Although IB is a pain in the butt, it’s working just fine for me.

I ran across this website the other day… Any wants to join forces or pay to front run Robinhood, shoot me an email.  This is some low hanging fruit.