In this article I describe how my backtested algorithm can be used in live algorithmic trading. My broker provides me with the TWS (Trader WorkStation) API which is the solution that I use to build my trading application.
The basic idea of my trading strategy is to rank hundreds of stocks by value (
) in descending order. That way a company with a high PB and PE ratio will land at the bottom. I then short the bottom ranking 30 stocks and go long the top ranking 30 stocks. The algorithm will check if rebalancing is needed and will do so using an interactive mode as a precautionary measure every time it runs. How often it runs is up to you. You can reschedule it to run daily, monthly or quarterly using a simple cron job.
For the value factor I need price and fundamental data. The price data I request from my broker. I have a market data subscription for that. For fundamental data such as earnings and equity I use my own data source. For the later stay tuned as I will be making this fundamental data available sometime in the future.
I used Python to develop this strategy. You might need to signup for a paper account at your broker and make sure they are using TWS. For a full TWS API please refer to the official documentation.
The TWS API Architecture
I will not go into all the API details since the official documentation describes everything pretty well. But I believe it is good to at least describe the idea behind the TWS API architecture. Some of the text below is taken straight out of the official documentation.
First you need to start TWS on your machine to allow for incoming connections. TWS acts as a server to receive requests from the API application (our Python client/strategy) and responds by taking appropriate actions. The first step is for the API client to initiate a connection to TWS on a socket port where TWS is already listening.
Once the TWS is up and running and actively listening for incoming connections we are ready to write our code. We have to implement TWS API’s two major interfaces: EWrapper interface and the EClientSocket.
The EWrapper interface is the mechanism through which the TWS delivers information to the API client application (our strategy). By implementing this interface the client application will be able to receive and handle the information coming from the TWS.
The class used to send messages to TWS is EClientSocket. Unlike EWrapper, this class is not overriden as the provided functions in EClientSocket are invoked to send messages to TWS. To use EClientSocket, first it may be necessary to implement the EWrapper interface as part of its constructor parameters so that the application can handle all returned messages. Messages sent from TWS as a response to function calls in EClientSocket require a EWrapper implementation so they can processed to meet the needs of the API client.
TWS API programs always have at least two threads of execution. One thread is used for sending messages to TWS, and another thread is used for reading returned messages. The second thread uses the API EReader class to read from the socket and add messages to a queue. Every time a new message is added to the message queue, a notification flag is triggered to let other threads know that there is a message waiting to be processed. In the two-thread design of an API program, the message queue is also processed by the first thread. The thread responsible for the message queue will decode messages and invoke the appropriate functions in EWrapper.
In Python TWS API the EReader thread is automatically started upon connection to TWS. There is no need for user to start the reader.
Let us go ahead and implement the EWrapper interface first since that is where the data from TWS will be coming in. The code snippet below does not contain all the methods implemented for EWrapper as my intention is to illustrate the basic idea. The full implementation will be listed at the end of this article.
class Wrapper(EWrapper):
def __init__(self):
wrapper.EWrapper.__init__(self)
def accountSummary(self, reqId: int, account: str, tag: str, value: str, currency: str):
super().accountSummary(reqId, account, tag, value, currency)
global account_summary
account_summary[tag] = value
def accountSummaryEnd(self, reqId: int):
super().accountSummaryEnd(reqId)
global account_summary_end
account_summary_end = True
def tickPrice(self, reqId: TickerId, tickType: TickType, price: float, attrib: TickAttrib):
super().tickPrice(reqId, tickType, price, attrib)
if tickType == TickTypeEnum.CLOSE:
global close_price
close_price = price
def nextValidId(self, orderId: int):
global next_valid_order_id
next_valid_order_id = orderId
It is important to understand that the methods implemented in EWrapper will be called when the respective data (e.g.: account summary) that has been requested by the client will become available. Depending on the data requested the method might get called repeatedly as can be the case with tick data or it can be called as low as once. For the later the end is usually signaled using the counterpart *End method.
Let’s look at our EClient subclass that will be requesting our data from TWS. Again I am not showing the entire client implementation at this stage.
class Client(EClient):
def __init__(self, wrapper):
EClient.__init__(self, wrapper)
def get_close_price(self, symbol):
contract = Contract()
contract.symbol = symbol
contract.secType = "STK"
contract.currency = "USD"
contract.exchange = "SMART"
contract.primaryExchange = "NASDAQ"
global close_price
close_price = None
reqId = self.get_next_request_id()
self.reqMarketDataType(4) # 1 for live, 4 for delayed-frozen data if live is not available
self.reqMktData(reqId, contract, "", False, False, [])
while close_price is None:
sleep(0.1)
self.cancelMktData(reqId)
return close_price
def get_portfolio_value(self):
request_id = int(time())
global account_summary_end
account_summary_end = False
self.reqAccountSummary(request_id, "All", "$LEDGER:ALL")
while not account_summary_end:
sleep(0.1)
global account_summary
return float(account_summary["TotalCashBalance"])
def get_valid_order_id(self):
global next_valid_order_id
next_valid_order_id = None
self.reqIds(numIds=1)
while next_valid_order_id is None:
sleep(0.1)
return next_valid_order_id
Notice how the client receives an EWrapper object in its constructor. In the above snippet see how we are calling the TWS API methods using the keyword self(e.g.: self.reqAccountSummary(...)) from within our methods (e.g.: get_portfolio_value(...)). The API methods are available to us since our client is a subclass of EClient. In order to make it easier to read the code and distinguish between API provided methods and method written by me I will use snake_case as my naming convention for my methods.
Finally we create a new class called Strategy that is both an EClient and EWrapper that will be connecting to TWS.
Once the client is connected, an EReader thread will be automatically created to handle incoming messages and put the messages into a message queue for further process. If you are using the TWS API with other programming languages you might need to pass an EReaderSignal object to the EClientSocket's constructor. This object is used to signal that a message is ready for processing in the queue. In Python the Queue class handles this task directly so we don’t have to pass in any EReaderSignal objects to our client’s constructor.
Looking inside the EClient's connect(..) method we see that the EReader thread is created for us. This thread is used to read from the socket and add messages to the queue.
def connect(self, host, port, clientId): ... self.reader = reader.EReader(self.conn, self.msg_queue) self.reader.start() # start thread ...
We also need to trigger Client::run(), where the message queue is processed in an infinite loop and the EWrapper call-back functions are then called. Let us briefly look inside the run() implementation of EClient. You don’t have to worry about this since this is already made available to you.
def run(self):
"""This is the function that has the message loop."""
try:
while not self.done and (self.isConnected() or not self.msg_queue.empty()):
try:
try:
text = self.msg_queue.get(block=True, timeout=0.2)
if len(text) > MAX_MSG_LEN:
self.wrapper.error(NO_VALID_ID, BAD_LENGTH.code(),
"%s:%d:%s" % (BAD_LENGTH.msg(), len(text), text))
self.disconnect()
break
except queue.Empty:
logger.debug("queue.get: empty")
else:
fields = comm.read_fields(text)
logger.debug("fields %s", fields)
self.decoder.interpret(fields)
except (KeyboardInterrupt, SystemExit):
logger.info("detected KeyboardInterrupt, SystemExit")
self.keyboardInterrupt()
self.keyboardInterruptHard()
except BadMessage:
logger.info("BadMessage")
self.conn.disconnect()
logger.debug("conn:%d queue.sz:%d",
self.isConnected(),
self.msg_queue.qsize())
finally:
self.disconnect()
In order for the client to be able to trigger EWrapper call-back functions we need to provide it with a concrete EWrapper implementation – a Strategy instance in our case. We do that using constructor dependency injection.
class Strategy(Wrapper, Client):
def __init__(self, ipaddress, portid, clientid):
Wrapper.__init__(self)
Client.__init__(self, wrapper=self)
self.connect(ipaddress, portid, clientid)
thread = Thread(target=self.run)
thread.start()
setattr(self, "_thread", thread)
The wrapper methods are then triggered in the infinite loop listed in the run() method above using the Decoder object as soon as a message is available in the queue:
self.decoder.interpret(fields)
The decoder interprets the fields and knows what method to call on the EWrapper object.
That is basically the high level overview of the TWS API architecture. Next we will be focusing on the concrete trading strategy.
The Strategy
The first thing to do is instantiate a new Strategy and connect to TWS:
app = Strategy(ipaddress="localhost", portid=1234, clientid=0)
Next we will be fetching some stocks from our database that we will be processing:
fundamentals = Fundamentals('database.ini')
y = year()
q = last_quarter()
stocks = fundamentals.get_stock_profiles(y, q)
Fundamentals is my proprietary API that communicates with my fundamentals database. I am currently working on making this publicly available in the future. But for now you can assume that this returns a list of StockProfile objects from a MySQL database. Each StockProfile represents a company and has properties such as price_earning, price_book and industry. year() and last_quarter() just return the current year and quarter and are passed in get_stock_profiles(...) to fetch fundamental data for that reporting period.
We will now request the latest close prices for each stock in our list from TWS:
for stock in stocks:
stock.price = app.get_close_price(stock.symbol)
Filter out companies for which we don’t have enough data and companies whose industry is either Financials or Utilities:
def filter_stock_profiles(stock_profiles):
filtered_stock_profiles = list(filter(lambda stock:
(stock.price_earnings is not None)
and (not math.isnan(stock.price_earnings))
and (stock.price_book is not None)
and (not math.isnan(stock.price_book)), stock_profiles))
filtered_stock_profiles = list(filter(lambda stock: (stock.industry is not None) and (
stock.industry != 'Financials') and (stock.industry != 'Utilities'), filtered_stock_profiles))
stocks = filter_stock_profiles(stocks)
Once we have filtered out all the unwanted companies we proceed with ranking them by value in descending order and fetching the top and bottom stocks for which we want to place an order:
def rank_stocks(stock_profiles):
return sorted(stock_profiles, key=lambda stock: 1.0 / (float(stock.price_earnings) + float(stock.price_book)), reverse=True)
stocks = rank_stocks(stocks)
longs = stocks[:MAX_LONG_POSITION_SIZE]
shorts = stocks[-MAX_SHORT_POSITION_SIZE:]
Next we calculate the weights that will be used to rebalance our portfolio later. We want the top ranking stocks to have a heavier weight in our long position and for the shorts we want the lowest ranking companies to carry a greater weight. That is, we buy more stocks of our top companies and sell more of our worst companies. Notice also that I don’t check a stock’s liquidity or if I can actually trade a particular stock. It is probably a good idea to do that in order to diminish market impact. It might also be a good idea to invest less in stocks with very low liquidity. I don’t do that here for the sake of simplicity.
portfolio_value = app.get_portfolio_value()
amount = portfolio_value // TARGET_NUMBER_OF_POSITIONS
my_positions = app.get_positions()
amount_available_for_long = (portfolio_value // 2)
number_of_long_units = (MAX_LONG_POSITION_SIZE * (MAX_LONG_POSITION_SIZE + 1)) / 2
amount_per_long_unit = amount_available_for_long // number_of_long_units
for rank, stock in enumerate(longs):
dollar_amount = (MAX_LONG_POSITION_SIZE - rank) * amount_per_long_unit
amount = dollar_amount // stock.price
stock.portfolio_weight = (amount * stock.price) / portfolio_value
stock.order_amount = amount
amount_available_for_short = (portfolio_value // 2)
number_of_short_units = (MAX_SHORT_POSITION_SIZE * (MAX_SHORT_POSITION_SIZE + 1)) / 2
amount_per_short_unit = amount_available_for_short // number_of_short_units
for rank, stock in enumerate(shorts):
dollar_amount = (rank + 1) * amount_per_short_unit
amount = dollar_amount // stock.price
stock.portfolio_weight = (amount * stock.price) / portfolio_value
stock.order_amount = -1 * amount
Before we go about rebalancing our portfolio we will be exiting any positions that we have in companies that are not included in our ranking.
order_book = [] # Exit positions that are not ranking any more for symbol in my_positions: position = my_positions[symbol] if position.position > 0 and not any(stock.symbol == position.symbol for stock in longs): order = MyOrder(position.symbol) order.description = "Closing long position." order.type = "EXIT" order.amount = -1 * position.position order.currency = position.currency order_book.append(order) for symbol in my_positions: position = my_positions[symbol] if position.position < 0 and not any(stock.symbol == position.symbol for stock in shorts): order = MyOrder(position.symbol) order.description = "Closing short position." order.type = "EXIT" order.amount = -1 * position.position order.currency = position.currency order_book.append(order)
As you can see I am storing any potential orders in a list called order_book and don’t actually place them yet. The reason for that is that I want to display them on screen and ask the user to confirm them before sending them off to TWS for execution. I also don’t want this strategy to continue running if there are any open orders still existent. I do this as a precautionary measure:
if len(order_book) > 0:
confirm = input("Do you want to place the above exit orders (yes/no)? ")
if confirm == 'yes':
open_orders = app.get_open_orders()
if open_orders:
log.debugv("There are pending orders in the system.")
log.debugv(print_open_orders(open_orders))
log.debugv("Not executing. Bye!")
app.disconnect()
exit()
else:
for order in order_book:
if order.currency is not None:
app.order_target_value(symbol=order.symbol, amount=order.amount, currency=order.currency)
else:
app.order_target_value(symbol=order.symbol, amount=order.amount)
else:
log.debugv("Decline execution accepted. Not executing. Bye!")
app.disconnect()
exit()
else:
log.debugv("Portfolio seems up-to-date. Not executing. Bye!")
app.disconnect()
exit()
Once I have exited all my unwanted positions I go ahead and calculate any adjustments that I need to undertake on any current positions and prepare the order for any new positions. Again I am storing any orders in the order_book list and ask the user to confirm them before sending the off to TWS:
order_book = []
log.debugv("No remaining pending orders.")
# Buy/Sell/Adjust
for stock in longs:
if (not stock.symbol in my_positions) or (my_positions[stock.symbol].position == 0):
order = MyOrder(stock.symbol)
order.description = "New long position {0} x {1}".format(stock.order_amount, stock.symbol)
order.type = "NEW"
order.amount = stock.order_amount
order_book.append(order)
else:
position = my_positions[stock.symbol]
target_position = stock.order_amount
if position.position < 0:
stock.order_amount = target_position + abs(position.position)
elif position.position > 0:
if position.position != target_position:
stock.order_amount = target_position - position.position
else:
stock.order_amount = 0
if stock.order_amount != 0:
order = MyOrder(stock.symbol)
order.description = "Adjusting existing long position for {0} from {1} -> {2}".format(
stock.symbol, my_positions[stock.symbol].position, target_position)
order.type = "ADJUSTMENT"
order.amount = stock.order_amount
order_book.append(order)
for stock in shorts:
if (not stock.symbol in my_positions) or (my_positions[stock.symbol].position == 0):
order = MyOrder(stock.symbol)
order.description = "New short position {0} x {1}".format(stock.order_amount, stock.symbol)
order.type = "NEW"
order.amount = stock.order_amount
order_book.append(order)
else:
position = my_positions[stock.symbol]
target_position = stock.order_amount
if position.position > 0:
stock.order_amount = target_position - position.position
elif position.position < 0:
if position.position != target_position:
stock.order_amount = target_position - position.position
else:
stock.order_amount = 0
if stock.order_amount != 0:
order = MyOrder(stock.symbol)
order.description = "Adjusting existing short position for {0} from {1} -> {2}".format(
stock.symbol, my_positions[stock.symbol].position, target_position)
order.type = "ADJUSTMENT"
order.amount = stock.order_amount
order_book.append(order)
In the above code snippet you can see that I am also storing whether a particular trade is caused by a new position or adjustment. It might have been more simple to just exit all positions in my current portfolio and then just go long the top and short the bottom companies but if a position doesn’t change or just needs a minor adjustment this would lead to unnecessary transaction costs. You might even ignore an adjustment if it is too small.
Finally, I again do some checking for any open orders and ask the user to confirm the new portfolio before placing the orders with my broker:
if len(order_book) > 0:
confirm = input("Do you want to place the above orders (yes/no)? ")
if confirm == 'yes':
open_orders = app.get_open_orders()
if open_orders:
log.debugv("There are pending orders in the system.")
log.debugv(print_open_orders(open_orders))
log.debugv("Not executing. Bye!")
app.disconnect()
exit()
else:
for order in order_book:
if order.currency is not None:
app.order_target_value(symbol=order.symbol, amount=order.amount, currency=order.currency)
else:
app.order_target_value(symbol=order.symbol, amount=order.amount)
open_orders = app.get_open_orders()
log.debugv("Waiting for any pending orders to be filled.")
while open_orders:
sleep(0.1)
log.debugv("No remaining pending orders. Exiting. Bye!")
app.disconnect()
exit()
else:
log.debugv("Decline execution accepted. Not executing. Bye!")
app.disconnect()
exit()
else:
log.debugv("Portfolio seems up-to-date. Not executing. Bye!")
app.disconnect()
exit()
else:
log.debugv("Filter resulted in less than %d stocks." % (TARGET_NUMBER_OF_POSITIONS))
The Full Strategy Code
Below you will find the full code for the strategy. You simply have to run it using:
python strategy.py
# Import ibapi deps
from ibapi import wrapper
from ibapi.client import EClient
from ibapi.contract import *
from ibapi.common import BarData
from ibapi.common import TickerId
from ibapi.common import TickAttrib
from ibapi.ticktype import TickType
from ibapi.ticktype import TickTypeEnum
from ibapi.order import *
from ibapi.order_state import OrderState
from threading import Thread
from datetime import datetime
from datetime import timedelta
from time import time
from time import sleep
import pandas as pd
from fundamentalslib.fundamentals import Fundamentals
from fundamentalslib.fundamentals import StockProfile
import random
from termcolor import colored
import math
import io
from logging import getLoggerClass, addLevelName, setLoggerClass, NOTSET
import logging
import logging.handlers
# use epoch as first orderId at api app startup, and increment it with each order, this way it ensures uniqueness and bigger across multiple clientId and each app invocation
next_valid_order_id = 0
MARKET_ID = 19004
TARGET_NUMBER_OF_POSITIONS = 10
MAX_SHORT_POSITION_SIZE = TARGET_NUMBER_OF_POSITIONS // 2
MAX_LONG_POSITION_SIZE = TARGET_NUMBER_OF_POSITIONS // 2
# Create a global DataFrame to insert the data in
historical_data = pd.DataFrame(columns=['Date', 'Open', 'Close', 'High', 'Low'])
positions = {}
position_end = False
open_orders = {}
open_orders_end = False
account_summary_end = False
account_summary = {}
close_price = None
log = None
DEBUG_LEVELV_NUM = 9
class MyLogger(getLoggerClass()):
def __init__(self, name, level=NOTSET):
super().__init__(name, level)
addLevelName(DEBUG_LEVELV_NUM, "STRATEGY_DEBUG")
def debugv(self, msg, *args, **kwargs):
if self.isEnabledFor(DEBUG_LEVELV_NUM):
self._log(DEBUG_LEVELV_NUM, msg, args, **kwargs)
class Position(object):
def __init__(self, symbol=None):
self.symbol = symbol
self.currency = None
self.sec_type = None
self.position = 0
def __str__(self):
return "Symbol: {0} Currency: {1} Position: {2} Sec Type: {3}".format(self.symbol, self.currency, self.position, self.sec_type)
class MyOrder(object):
def __init__(self, symbol=None):
self.symbol = symbol
self.currency = None
self.amount = 0
self.description = None
self.type = None
def __str__(self):
return "Symbol: {0} Currency: {1} Amount: {2} Description: {3} Type: {4}".format(self.symbol, self.currency, self.amount, self.description, self.type)
class Wrapper(wrapper.EWrapper):
def __init__(self):
wrapper.EWrapper.__init__(self)
def historicalData(self, reqId: int, bar: BarData):
global historical_data
historical_data = historical_data.append({'Date': str(bar.date), 'Open': float(bar.open), 'Close': float(
bar.close), 'High': float(bar.high), 'Low': float(bar.low)}, ignore_index=True)
def historicalDataEnd(self, reqId: int, start: str, end: str):
log.debugv("Done downloading historical data (request_id=%s)." % (reqId))
def accountSummary(self, reqId: int, account: str, tag: str, value: str, currency: str):
super().accountSummary(reqId, account, tag, value, currency)
# log.debugv("AccountSummary. ReqId:%s Account:%s Tag:%s Value:%s Currency:%s" % (reqId, account, tag, value, currency))
global account_summary
account_summary[tag] = value
def accountSummaryEnd(self, reqId: int):
super().accountSummaryEnd(reqId)
log.debugv("Done fetching account summary (request_id=%s)." % (reqId))
global account_summary_end
account_summary_end = True
# https://api.lynx.academy/Account&PortfolioData?id=positions
def position(self, account: str, contract: Contract, position: float, avgCost: float):
super().position(account, contract, position, avgCost)
global positions
pos = Position(contract.symbol)
pos.position = position
pos.sec_type = contract.secType
pos.currency = contract.currency
positions[contract.symbol] = pos
def positionEnd(self):
super().positionEnd()
log.debugv("Done fetching positions.")
global position_end
position_end = True
def tickPrice(self, reqId: TickerId, tickType: TickType, price: float, attrib: TickAttrib):
super().tickPrice(reqId, tickType, price, attrib)
if tickType == TickTypeEnum.CLOSE:
global close_price
close_price = price
log.debugv("Got close price %s for request_id=%s." % (close_price, reqId))
def openOrder(self, orderId: int, contract: Contract, order: Order, orderState: OrderState):
log.debugv("Open %s order with ID=%s Symbol=%s State=%s." % (order.action, orderId, contract.symbol, orderState.status))
global open_orders
# https://interactivebrokers.github.io/tws-api/order_submission.html#order_status
if orderState.status == 'PreSubmitted':
open_orders[orderId] = order
elif orderState.status == 'Filled' and orderId in open_orders:
del open_orders[orderId]
elif orderState.status == 'PendingCancel' and orderId in open_orders:
del open_orders[orderId]
def openOrderEnd(self):
super().openOrderEnd()
log.debugv("Done fetching open orders.")
global open_orders_end
open_orders_end = True
def nextValidId(self, orderId: int):
"""
Receives next valid order id.
The LYNXApi.EWrapper.nextValidID callback is commonly used to indicate that the connection is completed
and other messages can be sent from the API client to TWS. There is the possibility that function calls
made prior to this time could be dropped by TWS.
https://api.lynx.academy/Connectivity
"""
log.debugv("Got next_valid_order_id=%d." % orderId)
global next_valid_order_id
next_valid_order_id = orderId
class Client(EClient):
def __init__(self, wrapper):
EClient.__init__(self, wrapper)
def get_historicalData(self, contract, duration="1 M", barSize="1 hour", reqId=1):
log.debugv("Requesting historical data for '%s'." % (contract.symbol))
queryTime = (datetime.today() - timedelta(days=0)).strftime("%Y%m%d %H:%M:%S") # Define the end date of the query
# Here we are requesting historical bar data for the the contract
self.reqHistoricalData(reqId, contract, queryTime, duration, barSize, "MIDPOINT", 1, 1, False, [])
MAX_WAITED_SECONDS = 5
log.debugv("Getting historical data from the server... can take %d second to complete." % MAX_WAITED_SECONDS)
sleep(MAX_WAITED_SECONDS)
global historical_data
return historical_data
def get_close_price(self, symbol):
contract = Contract()
contract.symbol = symbol
contract.secType = "STK"
contract.currency = "USD"
contract.exchange = "SMART"
contract.primaryExchange = "NASDAQ"
global close_price
close_price = None
reqId = self.get_next_request_id()
self.reqMarketDataType(4) # 1 for live, 4 for delayed-frozen data if live is not available
self.reqMktData(reqId, contract, "", False, False, [])
log.debugv("Getting close price for '%s' (request_id=%d) from the server." % (contract.symbol, reqId))
while close_price is None:
sleep(0.1)
self.cancelMktData(reqId)
return close_price
def get_positions(self):
global position_end
position_end = False
self.reqPositions()
while not position_end:
sleep(0.1)
global positions
return positions
def get_portfolio_value(self):
request_id = int(time())
global account_summary_end
account_summary_end = False
self.reqAccountSummary(request_id, "All", "$LEDGER:ALL")
while not account_summary_end:
sleep(0.1)
global account_summary
return float(account_summary["TotalCashBalance"])
def order_target_value(self, symbol, amount, currency="USD"):
if amount == 0:
log.debugv("Error: amount cannot be zero.")
return
contract = Contract()
contract.symbol = symbol
contract.secType = "STK"
contract.currency = currency
contract.exchange = "SMART"
contract.primaryExchange = "NASDAQ"
# Define the order to place
order = Order()
if (amount > 0):
order.action = "BUY"
else:
order.action = "SELL"
order.orderType = "MKT"
order.totalQuantity = abs(amount)
self.place_order(contract, order)
def place_order(self, contract, order):
request_id = self.get_valid_order_id()
log.debugv("%s (%s) %d %s (Order ID=%s)" % (order.action, order.orderType, order.totalQuantity, contract.symbol, request_id))
self.placeOrder(request_id, contract, order)
def get_open_orders(self):
log.debugv("Fetching any open orders.")
global open_orders_end
open_orders_end = False
self.reqAllOpenOrders()
while not open_orders_end:
sleep(0.1)
global open_orders
return open_orders
def get_next_request_id(self):
sleep(2)
request_id = int(time())
log.debugv("Generated the next request_id=%d." % (request_id))
return request_id
def get_valid_order_id(self):
log.debugv("Getting valid order id from the server.")
global next_valid_order_id
next_valid_order_id = None
self.reqIds(numIds=1)
while next_valid_order_id is None:
sleep(0.1)
return next_valid_order_id
class Strategy(Wrapper, Client):
def __init__(self, ipaddress, portid, clientid):
Wrapper.__init__(self)
Client.__init__(self, wrapper=self)
self.connect(ipaddress, portid, clientid)
"""
API programs always have at least two threads of execution.
One thread is used for sending messages to TWS, and another thread is used for reading returned messages.
The second thread uses the API EReader class to read from the socket and add messages to a queue.
Everytime a new message is added to the message queue, a notification flag is triggered to let other threads
now that there is a message waiting to be processed.
The thread responsible for the message queue will decode messages and invoke the appropriate functions in EWrapper.
In the two-thread design of an API program, the message queue is also processed by the first thread.
Once the client is connected, a reader thread will be automatically created to handle incoming messages and put the messages
into a message queue for further process. User is required to trigger Client::run() below, where the message queue is processed
in an infinite loop and the EWrapper call-back functions are automatically triggered.
https://api.lynx.academy/Connectivity
"""
thread = Thread(target=self.run)
thread.start()
setattr(self, "_thread", thread)
def last_quarter():
now = datetime.now()
quarter_no = (now.month - 1) // 3 + 1
return "Q{}".format(quarter_no-1)
def year():
return datetime.now().year
def filter_stock_profiles(stock_profiles):
"""[summary]
Keep only stock profiles whose industry sector is not 'Public Utilities' or 'Finance'.
"""
log.debugv("Filtering stocks.")
filtered_stock_profiles = list(filter(lambda stock:
(stock.price_earnings is not None)
and (not math.isnan(stock.price_earnings))
and (stock.price_book is not None)
and (not math.isnan(stock.price_book)), stock_profiles))
filtered_stock_profiles = list(filter(lambda stock: (stock.industry is not None) and (
stock.industry != 'Financials') and (stock.industry != 'Utilities'), filtered_stock_profiles))
return filtered_stock_profiles
def rank_stocks(stock_profiles):
"""
Ranks stocks by value.
Arguments:
stock_profiles -- A list of stock objects to rank.
"""
return sorted(stock_profiles, key=lambda stock: 1.0 / (float(stock.price_earnings) + float(stock.price_book)), reverse=True)
def print_stocks(stock_profiles, title, limit):
writer = io.StringIO()
writer.write("\n%s\n" % (title))
buf = ('%-4s %-6s %-35s %-25s %25s %25s %25s %25s\n' % ('#', 'Symbol', 'Name', 'Industry', 'PE', 'PB', 'Value', 'Weight'))
writer.write(buf)
writer.write('-' * len(buf))
writer.write("\n")
for i, stock in enumerate(stock_profiles):
name = stock.name[: 30] if stock.name is not None else "N/A"
industry = stock.industry[: 30] if stock.industry is not None else "N/A"
price_earnings = stock.price_earnings if stock.price_earnings is not None else 0.0
price_book = stock.price_book if stock.price_book is not None else 0.0
value = 1.0 / (float(stock.price_earnings) + float(stock.price_book)) if (stock.price_earnings is not None) and (stock.price_book is not None) else 0.0
weight = stock.portfolio_weight
writer.write(
'%-4d %s %-35s %-25s %25s %25.2f %25.3f %25s\n' %
(i + 1, colored("%-6s" % stock.symbol, 'green'),
name,
industry,
price_earnings,
price_book,
value,
("{:.0%}".format(weight))))
limit -= 1
if limit == 0:
break
return writer.getvalue()
def print_positions(positions):
writer = io.StringIO()
writer.write("\nCURRENT POSITIONS\n")
buf = ('%-6s %-25s %-25s %25s\n' % ('Symbol', 'Position', 'Currency', 'Sec Type'))
writer.write(buf)
writer.write('-' * len(buf))
writer.write("\n")
for symbol in positions:
position = positions[symbol]
if position.position != 0:
s = position.symbol if position.symbol is not None else "N/A"
p = position.position
c = position.currency if position.currency is not None else "N/A"
t = position.sec_type if position.sec_type is not None else "N/A"
writer.write('%-6s %-25s %-25s %25s\n' % (s, p, c, t))
return writer.getvalue()
def print_order_book(order_book):
writer = io.StringIO()
writer.write("\nORDER BOOK\n")
buf = ('%-25s %-25s %-25s %-65s\n' % ('Type', 'Symbol', 'Amount', 'Description'))
writer.write(buf)
writer.write('-' * len(buf))
writer.write("\n")
for order in order_book:
t = order.type if order.type is not None else "N/A"
s = order.symbol if order.symbol is not None else "N/A"
a = order.amount
d = order.description if order.description is not None else "N/A"
color = 'green'
if t == 'EXIT':
color = 'red'
elif t == 'ADJUSTMENT':
color = 'yellow'
writer.write('%-25s %-25s %-25s %-65s\n' % ((colored("%-25s" % t, color), s, a, d)))
return writer.getvalue()
def print_open_orders(orders):
writer = io.StringIO()
writer.write("\nOPEN_ORDERS\n")
buf = ('%-25s %-25s %-25s %-25s\n' % ('ID', 'Action', 'Quantity', 'Type'))
writer.write(buf)
writer.write('-' * len(buf))
writer.write("\n")
for order_id in orders:
order = orders[order_id]
a = order.action
q = order.totalQuantity
t = order.orderType
writer.write('%-25s %-25s %-25s %-25s\n' % (order_id, a, q, t))
return writer.getvalue()
def main():
setLoggerClass(MyLogger)
global log
log = logging.getLogger(__name__)
log.setLevel(DEBUG_LEVELV_NUM)
ch = logging.StreamHandler()
fh = logging.handlers.RotatingFileHandler('strategy.log', maxBytes=20000, backupCount=5)
formatter = logging.Formatter('[%(asctime)s]: %(message)s')
fh.setFormatter(formatter)
ch.setFormatter(formatter)
log.addHandler(fh)
log.addHandler(ch)
log.propagate = False
app = Strategy(ipaddress="localhost", portid=1234, clientid=0)
log.debugv("Server version=%s Connection time=%s." % (app.serverVersion(), app.twsConnectionTime()))
if not app.twsConnectionTime():
log.debugv("Could not connect to TWS. Exiting. Bye!")
exit()
fundamentals = Fundamentals('database.ini')
y = year()
q = last_quarter()
log.debugv("Fetching stocks from database for %s/%s." % (y, q))
stocks = fundamentals.get_stock_profiles(y, q)
log.debugv("Fetched %s stocks from database." % (len(stocks)))
for stock in stocks:
stock.price = app.get_close_price(stock.symbol)
log.debugv("Fetched prices for all symbols.")
stocks = filter_stock_profiles(stocks)
if (len(stocks) >= TARGET_NUMBER_OF_POSITIONS):
stocks = rank_stocks(stocks)
longs = stocks[:MAX_LONG_POSITION_SIZE]
shorts = stocks[-MAX_SHORT_POSITION_SIZE:]
# Rebalance
portfolio_value = app.get_portfolio_value()
amount = portfolio_value // TARGET_NUMBER_OF_POSITIONS # don't overweight position sizes if the pickings are slim
my_positions = app.get_positions()
amount_available_for_long = (portfolio_value // 2)
number_of_long_units = (MAX_LONG_POSITION_SIZE * (MAX_LONG_POSITION_SIZE + 1)) / 2
amount_per_long_unit = amount_available_for_long // number_of_long_units
log.debugv("Portfolio value=%s, amount_available_for_long=%s, number_of_long_units=%s, amount_per_long_unit=%s." %
(portfolio_value, amount_available_for_long, number_of_long_units, amount_per_long_unit))
for rank, stock in enumerate(longs):
# To diminish market impact, I invest less in stocks with very low liquidity.
# if data.can_trade(stock):
dollar_amount = (MAX_LONG_POSITION_SIZE - rank) * amount_per_long_unit
amount = dollar_amount // stock.price
stock.portfolio_weight = (amount * stock.price) / portfolio_value
stock.order_amount = amount
amount_available_for_short = (portfolio_value // 2)
number_of_short_units = (MAX_SHORT_POSITION_SIZE * (MAX_SHORT_POSITION_SIZE + 1)) / 2
amount_per_short_unit = amount_available_for_short // number_of_short_units
log.debugv("Portfolio value=%s, amount_available_for_short=%s, number_of_short_units=%s, amount_per_short_unit=%s." %
(portfolio_value, amount_available_for_short, number_of_short_units, amount_per_short_unit))
for rank, stock in enumerate(shorts):
# To diminish market impact, I invest less in stocks with very low liquidity.
# if data.can_trade(stock):
dollar_amount = (rank + 1) * amount_per_short_unit
amount = dollar_amount // stock.price
stock.portfolio_weight = (amount * stock.price) / portfolio_value
stock.order_amount = -1 * amount
log.debugv(print_stocks(longs, "NEW PORTFOLIO - LONGS", MAX_LONG_POSITION_SIZE))
log.debugv(print_stocks(shorts, "NEW PORTFOLIO - SHORTS", MAX_SHORT_POSITION_SIZE))
if any(stock.order_amount == 0 for stock in longs):
log.error("Error: some long stocks have zero order amount. Exiting.")
app.disconnect()
exit()
if any(stock.order_amount == 0 for stock in shorts):
log.error("Error: some short stocks have zero order amount. Exiting.")
app.disconnect()
exit()
log.debugv(print_positions(my_positions))
order_book = []
# Exit positions that are not ranking any more
for symbol in my_positions:
position = my_positions[symbol]
if position.position > 0 and not any(stock.symbol == position.symbol for stock in longs):
order = MyOrder(position.symbol)
order.description = "Closing long position."
order.type = "EXIT"
order.amount = -1 * position.position
order.currency = position.currency
order_book.append(order)
for symbol in my_positions:
position = my_positions[symbol]
if position.position < 0 and not any(stock.symbol == position.symbol for stock in shorts):
order = MyOrder(position.symbol)
order.description = "Closing short position."
order.type = "EXIT"
order.amount = -1 * position.position
order.currency = position.currency
order_book.append(order)
log.debugv(print_order_book(order_book))
if len(order_book) > 0:
confirm = input("Do you want to place the above exit orders (yes/no)? ")
if confirm == 'yes':
open_orders = app.get_open_orders()
if open_orders:
log.debugv("There are pending orders in the system.")
log.debugv(print_open_orders(open_orders))
log.debugv("Not executing. Bye!")
app.disconnect()
exit()
else:
for order in order_book:
if order.currency is not None:
app.order_target_value(symbol=order.symbol, amount=order.amount, currency=order.currency)
else:
app.order_target_value(symbol=order.symbol, amount=order.amount)
else:
log.debugv("Decline execution accepted. Not executing. Bye!")
app.disconnect()
exit()
else:
log.debugv("Portfolio seems up-to-date. Not executing. Bye!")
app.disconnect()
exit()
open_orders = app.get_open_orders()
log.debugv("Waiting for any pending orders to be filled.")
while open_orders:
sleep(0.1)
order_book = []
log.debugv("No remaining pending orders.")
# Buy/Sell/Adjust
for stock in longs:
if (not stock.symbol in my_positions) or (my_positions[stock.symbol].position == 0):
order = MyOrder(stock.symbol)
order.description = "New long position {0} x {1}".format(stock.order_amount, stock.symbol)
order.type = "NEW"
order.amount = stock.order_amount
order_book.append(order)
else:
position = my_positions[stock.symbol]
target_position = stock.order_amount
if position.position < 0:
stock.order_amount = target_position + abs(position.position)
elif position.position > 0:
if position.position != target_position:
stock.order_amount = target_position - position.position
else:
stock.order_amount = 0
if stock.order_amount != 0:
order = MyOrder(stock.symbol)
order.description = "Adjusting existing long position for {0} from {1} -> {2}".format(
stock.symbol, my_positions[stock.symbol].position, target_position)
order.type = "ADJUSTMENT"
order.amount = stock.order_amount
order_book.append(order)
for stock in shorts:
if (not stock.symbol in my_positions) or (my_positions[stock.symbol].position == 0):
order = MyOrder(stock.symbol)
order.description = "New short position {0} x {1}".format(stock.order_amount, stock.symbol)
order.type = "NEW"
order.amount = stock.order_amount
order_book.append(order)
else:
position = my_positions[stock.symbol]
target_position = stock.order_amount
if position.position > 0:
stock.order_amount = target_position - position.position
elif position.position < 0:
if position.position != target_position:
stock.order_amount = target_position - position.position
else:
stock.order_amount = 0
if stock.order_amount != 0:
order = MyOrder(stock.symbol)
order.description = "Adjusting existing short position for {0} from {1} -> {2}".format(
stock.symbol, my_positions[stock.symbol].position, target_position)
order.type = "ADJUSTMENT"
order.amount = stock.order_amount
order_book.append(order)
log.debugv(print_order_book(order_book))
if len(order_book) > 0:
confirm = input("Do you want to place the above orders (yes/no)? ")
if confirm == 'yes':
open_orders = app.get_open_orders()
if open_orders:
log.debugv("There are pending orders in the system.")
log.debugv(print_open_orders(open_orders))
log.debugv("Not executing. Bye!")
app.disconnect()
exit()
else:
for order in order_book:
if order.currency is not None:
app.order_target_value(symbol=order.symbol, amount=order.amount, currency=order.currency)
else:
app.order_target_value(symbol=order.symbol, amount=order.amount)
open_orders = app.get_open_orders()
log.debugv("Waiting for any pending orders to be filled.")
while open_orders:
sleep(0.1)
log.debugv("No remaining pending orders. Exiting. Bye!")
app.disconnect()
exit()
else:
log.debugv("Decline execution accepted. Not executing. Bye!")
app.disconnect()
exit()
else:
log.debugv("Portfolio seems up-to-date. Not executing. Bye!")
app.disconnect()
exit()
else:
log.debugv("Filter resulted in less than %d stocks." % (TARGET_NUMBER_OF_POSITIONS))
if __name__ == "__main__":
main()
Disclaimer: this is a very basic algorithm to illustrate how to interact with the IB API using Python in order to build a fully automated trading algorithm. Please do not invest any real money with this algorithm. Algorithmic trading involves a substantial risk of loss and is not appropriate for everyone. No representation is being made that utilizing the algorithmic trading strategy will result in profitable trading or be free of risk of loss.
Leave a Reply
You must be logged in to post a comment.