Adventures in python and algorithmic trading

[hatemypips]

I am going to be doing something similar this year if I have the time. Unfortunately, what I am undertaking isn't as simple as running some python code. I plan on automating a macro based system which will be monitoring economic data and news feeds and trading off of it. The code will require data streaming, natural language processing, AI with deep learning, and a complex trade management process. I will be doing this in azure because a single pc won't be enough to handle the processing and trade decisions. If Ray Dalio can do it why not me.

Momentum algo? You will have a lot of trouble finding good broker for that, it's not easy money. Tested.

 

[Sharky]

Hi Paul...

ive always used quant / programming mates to do my grunt work ....so huge respect here to you

This allows me the objectivity of focusing on the trading system and not getting lost in the programming detail .....

will watch with interest ........

(dont let the programming get in front of the all important trading strategy)

as a complete Knob on EA programming ....can i ask why python is better than MT4 EA's .....which is where all my projects to date have been ?

N

Thanks NVP! I can totally see why folks would want to get someone to do the 'grunt' work for them, and avoids the danger of getting sidetracked by the tools themselves. However I can also see a lot of upsides to the DIY approach, which by the end of the year I should be in a better position to comment on. Right now I can only draw from my own experience of being a developer and then instructing developers, as I've done in my career outside of trading. The big pluses for me, are:

• You can iterate quicker, since you don't have the overheard of dealing with anyone else
• You can 'speak' the language, so you can be part of, and learn from, that 'community'
• The DIY approach can be a lot more satisfying (but, conversely, a lot more frustrating)
• If you're interested in machine learning and AI, Python has leading open-source libraries like TensorFlow
• You're not limited to just market data, you can incorporate whatever data sets you think might be interesting and Python is great at munging (cleaning up and structuring data)
• Perhaps if there are more barriers to entry and hence less people doing something, then there's a greater chance of gaining a competitive edge?!
• Gaining a background in a general purpose programming language, means you can apply that knowledge outside the narrow scope of trading. Who knows when it might come in useful!

I see you are using a Mac - I use the regular PC
and IB - I use MT4
Looks like we are diverging ??

I don't see whether you use a Mac, PC or Linux as making much of a difference, Python is Python whatever platform you use and nowadays with Docker you can run pretty much anything on anything. I certainly won't be running my algos on Mac, it'll either be on Windows or Linux.

As for MT4, I've never used it myself for automated trading, but you're right there's a big convergence between using proprietary trading languages, versus general languages. Hopefully during the course of this journey, I'll be able to share a better understanding of why you might want to go down a more generalistic path.

Python for trading meetup in London on the 23rd january

https://www.meetup.com/meetup-group-XAbNZRBz/events/256380499/?rv=me2

Nice one! Thanks for sharing TS.. I won't be back in the UK until end of the month, but I'll try and go along to the next one on Feb, 27th: https://www.meetup.com/meetup-group-XAbNZRBz/events/sjbkcqyzdbkc/. The whole competition of the event sounds pretty cool.

Looks like you made a great start there.

Thanks sideways-sid! Comments like this, really help!

I am going to be doing something similar this year if I have the time. Unfortunately, what I am undertaking isn't as simple as running some python code. I plan on automating a macro based system which will be monitoring economic data and news feeds and trading off of it. The code will require data streaming, natural language processing, AI with deep learning, and a complex trade management process. I will be doing this in azure because a single pc won't be enough to handle the processing and trade decisions. If Ray Dalio can do it why not me.

Just from your description you're already ten steps ahead of me, but on first glance I can't see why Python wouldn't be a strong candidate for doing all that. It's only the super high frequency algo traders who obsess over milliseconds where you'd need a compiled language like Java or C++. But sounds really interesting, I hope you'll share your progress.

 

[Trader333]

For those wishing to learn Python there are a couple of books that in Kindle format are free from Amazon so here are the links:

Python Programming Beginners
Python Intermediate Learning

 

[Sharky]

Sunday's always a good day to plan your week ahead. So I've spent a bit of time setting out my goals for this week.

Before sharing those goals I want to mention a couple of the tools I'm using as part of this journey to keep me on track.

I recently started using Trello (https://trello.com) as a task management tool. It's free to use and if like me you prefer to go native, there are apps for my laptop (Mac) and phone (Android). Windows and iOS are covered too. By adding my tasks for the week ahead I don't have to stop and think what to do next, I've already thought about it and can just get on with it.

The other tool I'm using is Evernote (https://evernote.com/) which is a note taking app which like Trello, is free to use and has native versions. I've always been a big fan of nvALT (http://brettterpstra.com/projects/nvalt/) for quickly jotting down anything I think I might want to revisit later, but given the complexity of the journey ahead, I decided that Evernote will be a better option to organise myself, as it offers saved searches, tagging and multi-media features that nvALT just doesn't have. I think it's going to really help to make my journal updates easier to manage and better structured, as I can create posts in draft and over time flesh them out.

So I learnt last week, that progressing without a plan results in little real progress. I didn't set myself any specific tasks, so I ended up doing a lot of googling, and whilst it may have been fun to listen to some algorithmic podcasts, watching a few YouTube videos and reading a bunch of blog posts, it really add up to anything. I'm sure many would-be traders, fall into this trap, quickly realise there's a mountain of information out there, don't know where to start and either give up too quickly or seek solace in the comfort, and cost, of a 'training programme' that promises to be their guiding light and will most probably fail to deliver.

As a great believer in the less is more approach, let's make sure this week is a productive one by focussing on a few tasks that I know are fundamental building blocks for moving forward. As NVP pointed out earlier, I can't let the programming and nuts and bolts, get in the way of the trading strategy, so I need to progress on both. Here's my plan of attack:

• Trading Strategy: Read the first two parts of 'The Financial Times Guide to Options' by Lenny Jordan - https://www.amazon.co.uk/Financial-Times-Guide-Options-Successful/dp/0273736868. I got hold of a copy of this in early December, and I really like Lenny's straightforward and concise approach to explaining options trading and strategies. It's essentially a rebranded second edition of the author's previous book, Options Plain and Simple, which was recommend by @RogerM in the Best Thread: T2W Guide to Trading Options all the way back in 2004! The first two parts consist of 14 short chapters and there are Q&A exercises to complete for each.
  
  
• Programming: Get QTPyLib (https://qtpylib.io/) up and running, and paper trading a very simple automated MA-crossover strategy in my IB account. Last week I spent some time looking around for algorithmic trading platforms and libraries which could get me up and running quickly, which can support paper and live trading with IB, back testing and support options trading; and this open source library seems to fit the bill nicely. The author of the library, Ran Aroussi, has created a four part webinar series on Treading with Python (you read that right) available on his website here: https://aroussi.com/#tag/recording. I've watched the first one and it was excellent. You get to see how easy it is to prototype trading strategies with Python using Jupyter notebooks. I plan to watch the rest of the webinars when I have the time to follow them, step by step. For now, I prefer to get some actual trading under my belt!

 

[FXX]

Momentum algo? You will have a lot of trouble finding good broker for that, it's not easy money. Tested.

it's not a momentum algo, its a news algo. Initially will be used for swing trading but will be enhanced for intraday news trading

 

[FXX]

Just from your description you're already ten steps ahead of me, but on first glance I can't see why Python wouldn't be a strong candidate for doing all that. It's only the super high frequency algo traders who obsess over milliseconds where you'd need a compiled language like Java or C++. But sounds really interesting, I hope you'll share your progress.

it will be a combination of technologies and languages but isn't going to be a high frequency algo. I estimate it will produce between 5 to 10 trades a week. Luckily i am in the tech space and work with data systems in a professional capacity with approx 15 years programming experience across many languages. I have never tried something like this before but i do have a profitable strategy i would like to replicate and maybe even enhance. Ray Dalio said his algo makes better decisions than he does these days and their recent 14% gain last year while most other hedge funds lost is testament to that. I will definitely start a journal on this as it will be an exciting project but i have to get a few exams out the way first as part of my career roadmap so don't have a lot of time to spend on it besides taking notes on how to implement it with some light research.

 

[Trader333]

.....Ray Dalio said his algo makes better decisions than he does these days and their recent 14% gain last year while most other hedge funds lost is testament to that.

If the alogo he has developed has done this over several years then that is impressive however, if it is just one year then it remains to be seen if the same set of criteria used in the algo will consistently produce the same results in my vew.

 

[FXX]

If the alogo he has developed has done this over several years then that is impressive however, if it is just one year then it remains to be seen if the same set of criteria used in the algo will consistently produce the same results in my vew.

its been running for years and is a pure macro algo

 

[Trading Spaces]

look at the jma from www.jurikres.com if you want to program a moving average system.

 

[Sharky]

look at the jma from www.jurikres.com if you want to program a moving average system.

Thanks for the tip TS, I haven't visited that site in years. The JMA indicator you mentioned looks interesting although I don't think they sell a version that would work with Python. That's fine, for the purposes of MA crossover strategy I mentioned, I'm really just looking to get something up and running super quick. It's not an actual system I'd want to trade.

 

[NVP]

Thanks NVP! I can totally see why folks would want to get someone to do the 'grunt' work for them, and avoids the danger of getting sidetracked by the tools themselves. However I can also see a lot of upsides to the DIY approach, which by the end of the year I should be in a better position to comment on. Right now I can only draw from my own experience of being a developer and then instructing developers, as I've done in my career outside of trading. The big pluses for me, are:

• You can iterate quicker, since you don't have the overheard of dealing with anyone else
• You can 'speak' the language, so you can be part of, and learn from, that 'community'
• The DIY approach can be a lot more satisfying (but, conversely, a lot more frustrating)
• If you're interested in machine learning and AI, Python has leading open-source libraries like TensorFlow
• You're not limited to just market data, you can incorporate whatever data sets you think might be interesting and Python is great at munging (cleaning up and structuring data)
• Perhaps if there are more barriers to entry and hence less people doing something, then there's a greater chance of gaining a competitive edge?!
• Gaining a background in a general purpose programming language, means you can apply that knowledge outside the narrow scope of trading. Who knows when it might come in useful!

I don't see whether you use a Mac, PC or Linux as making much of a difference, Python is Python whatever platform you use and nowadays with Docker you can run pretty much anything on anything. I certainly won't be running my algos on Mac, it'll either be on Windows or Linux.

As for MT4, I've never used it myself for automated trading, but you're right there's a big convergence between using proprietary trading languages, versus general languages. Hopefully during the course of this journey, I'll be able to share a better understanding of why you might want to go down a more generalistic path.

understood - and respect from this level 1 dummy programmer ....(well these days anyway)

N

 

[FXX]

If you can get the data you can just calculate the ma in python in real-time without anything special. I think there are even libraries that do this for you and all you need is to feed it data.

 

[Sharky]

If you can get the data you can just calculate the ma in python in real-time without anything special. I think there are even libraries that do this for you and all you need is to feed it data.

Totally. In QTPYlib, it's as simple as:

# compute averages using internal rolling_mean
bars['short_ma'] = bars['close'].rolling_mean(window=10)
bars['long_ma']  = bars['close'].rolling_mean(window=20)

^ Taken from their code example

 

[hatemypips]

it's not a momentum algo, its a news algo. Initially will be used for swing trading but will be enhanced for intraday news trading

Same issue with news algo, I used dozens of brokers to trade news (fx ones) all of them start to delay opening of a trade, telling me its server overload and liquidity issues.

 

[Sharky]

Just a quick update to explain the reason I haven't posted this Sunday was because I was travelling back to the UK from Brazil. So it threw my routine out of the window, but I'm settled back in now, and you can expect an update this weekend.

 

[FXX]

Same issue with news algo, I used dozens of brokers to trade news (fx ones) all of them start to delay opening of a trade, telling me its server overload and liquidity issues.

I trade the news all the time without any issues.

 

[Trader333]

@Sharky Are you intending day-trading options or just wan to have the ability to automatically open and close trades when certain criteria are met ?

 

[Sharky]

@Sharky Are you intending day-trading options or just wan to have the ability to automatically open and close trades when certain criteria are met ?

It's early days, but I'm guessing trades will be held over weeks, rather than days or intraday. The ability to automatically open and close trades with minimal oversight is what I'm really after.

 

[Trader333]

It's early days, but I'm guessing trades will be held over weeks, rather than days or intraday. The ability to automatically open and close trades with minimal oversight is what I'm really after.

Have you decided on what options strategy you may use in broad terms such as directional, volatility etc ?

 

[FXX]

I have put my project on hold for a while. Busy developing a site centralbanktrader.com as a trader fundamental research portal that traders can use to reduce their bigger picture analysis overhead. Will get back to my own auromation maybe in a couple of months. How is yours coming along sharky