Adventures in python and algorithmic trading

NVP

Legendary member
Jun 21, 2004
36,091
1,752
#16
Hi Paul...

ive always used quant / programming mates to do my grunt work ....so huge respect here to you

This allows me the objectivity of focusing on the trading system and not getting lost in the programming detail .....

will watch with interest ........

(dont let the programming get in front of the all important trading strategy)

as a complete Knob on EA programming ....can i ask why python is better than MT4 EA's .....which is where all my projects to date have been ?

N
 

Pat494

Legendary member
Mar 27, 2004
13,191
1,262
#17
I see you are using a Mac - I use the regular PC
and IB - I use MT4
Looks like we are diverging ??
 

FXX

Experienced member
Oct 12, 2017
1,135
181
#20
I am going to be doing something similar this year if I have the time. Unfortunately, what I am undertaking isn't as simple as running some python code. I plan on automating a macro based system which will be monitoring economic data and news feeds and trading off of it. The code will require data streaming, natural language processing, AI with deep learning, and a complex trade management process. I will be doing this in azure because a single pc won't be enough to handle the processing and trade decisions. If Ray Dalio can do it why not me.
 
Likes: 0007

hatemypips

Well-known member
Nov 11, 2014
258
5
#21
I am going to be doing something similar this year if I have the time. Unfortunately, what I am undertaking isn't as simple as running some python code. I plan on automating a macro based system which will be monitoring economic data and news feeds and trading off of it. The code will require data streaming, natural language processing, AI with deep learning, and a complex trade management process. I will be doing this in azure because a single pc won't be enough to handle the processing and trade decisions. If Ray Dalio can do it why not me.

Momentum algo? You will have a lot of trouble finding good broker for that, it's not easy money. Tested.
 

Sharky

Staff member
Nov 4, 2000
5,437
260
#22
Hi Paul...

ive always used quant / programming mates to do my grunt work ....so huge respect here to you

This allows me the objectivity of focusing on the trading system and not getting lost in the programming detail .....

will watch with interest ........

(dont let the programming get in front of the all important trading strategy)

as a complete Knob on EA programming ....can i ask why python is better than MT4 EA's .....which is where all my projects to date have been ?

N
Thanks NVP! I can totally see why folks would want to get someone to do the 'grunt' work for them, and avoids the danger of getting sidetracked by the tools themselves. However I can also see a lot of upsides to the DIY approach, which by the end of the year I should be in a better position to comment on. Right now I can only draw from my own experience of being a developer and then instructing developers, as I've done in my career outside of trading. The big pluses for me, are:
  • You can iterate quicker, since you don't have the overheard of dealing with anyone else
  • You can 'speak' the language, so you can be part of, and learn from, that 'community'
  • The DIY approach can be a lot more satisfying (but, conversely, a lot more frustrating)
  • If you're interested in machine learning and AI, Python has leading open-source libraries like TensorFlow
  • You're not limited to just market data, you can incorporate whatever data sets you think might be interesting and Python is great at munging (cleaning up and structuring data)
  • Perhaps if there are more barriers to entry and hence less people doing something, then there's a greater chance of gaining a competitive edge?!
  • Gaining a background in a general purpose programming language, means you can apply that knowledge outside the narrow scope of trading. Who knows when it might come in useful!
I see you are using a Mac - I use the regular PC
and IB - I use MT4
Looks like we are diverging ??
I don't see whether you use a Mac, PC or Linux as making much of a difference, Python is Python whatever platform you use and nowadays with Docker you can run pretty much anything on anything. I certainly won't be running my algos on Mac, it'll either be on Windows or Linux.

As for MT4, I've never used it myself for automated trading, but you're right there's a big convergence between using proprietary trading languages, versus general languages. Hopefully during the course of this journey, I'll be able to share a better understanding of why you might want to go down a more generalistic path.

Nice one! Thanks for sharing TS.. I won't be back in the UK until end of the month, but I'll try and go along to the next one on Feb, 27th: https://www.meetup.com/meetup-group-XAbNZRBz/events/sjbkcqyzdbkc/. The whole competition of the event sounds pretty cool.

Looks like you made a great start there.
Thanks sideways-sid! Comments like this, really help!

I am going to be doing something similar this year if I have the time. Unfortunately, what I am undertaking isn't as simple as running some python code. I plan on automating a macro based system which will be monitoring economic data and news feeds and trading off of it. The code will require data streaming, natural language processing, AI with deep learning, and a complex trade management process. I will be doing this in azure because a single pc won't be enough to handle the processing and trade decisions. If Ray Dalio can do it why not me.
Just from your description you're already ten steps ahead of me, but on first glance I can't see why Python wouldn't be a strong candidate for doing all that. It's only the super high frequency algo traders who obsess over milliseconds where you'd need a compiled language like Java or C++. But sounds really interesting, I hope you'll share your progress.
 
Likes: Trader333

Sharky

Staff member
Nov 4, 2000
5,437
260
#24
Sunday's always a good day to plan your week ahead. So I've spent a bit of time setting out my goals for this week.

Before sharing those goals I want to mention a couple of the tools I'm using as part of this journey to keep me on track.

I recently started using Trello (https://trello.com) as a task management tool. It's free to use and if like me you prefer to go native, there are apps for my laptop (Mac) and phone (Android). Windows and iOS are covered too. By adding my tasks for the week ahead I don't have to stop and think what to do next, I've already thought about it and can just get on with it.

The other tool I'm using is Evernote (https://evernote.com/) which is a note taking app which like Trello, is free to use and has native versions. I've always been a big fan of nvALT (http://brettterpstra.com/projects/nvalt/) for quickly jotting down anything I think I might want to revisit later, but given the complexity of the journey ahead, I decided that Evernote will be a better option to organise myself, as it offers saved searches, tagging and multi-media features that nvALT just doesn't have. I think it's going to really help to make my journal updates easier to manage and better structured, as I can create posts in draft and over time flesh them out.

So I learnt last week, that progressing without a plan results in little real progress. I didn't set myself any specific tasks, so I ended up doing a lot of googling, and whilst it may have been fun to listen to some algorithmic podcasts, watching a few YouTube videos and reading a bunch of blog posts, it really add up to anything. I'm sure many would-be traders, fall into this trap, quickly realise there's a mountain of information out there, don't know where to start and either give up too quickly or seek solace in the comfort, and cost, of a 'training programme' that promises to be their guiding light and will most probably fail to deliver.

As a great believer in the less is more approach, let's make sure this week is a productive one by focussing on a few tasks that I know are fundamental building blocks for moving forward. As NVP pointed out earlier, I can't let the programming and nuts and bolts, get in the way of the trading strategy, so I need to progress on both. Here's my plan of attack:

  • Trading Strategy: Read the first two parts of 'The Financial Times Guide to Options' by Lenny Jordan - https://www.amazon.co.uk/Financial-Times-Guide-Options-Successful/dp/0273736868. I got hold of a copy of this in early December, and I really like Lenny's straightforward and concise approach to explaining options trading and strategies. It's essentially a rebranded second edition of the author's previous book, Options Plain and Simple, which was recommend by @RogerM in the Best Thread: T2W Guide to Trading Options all the way back in 2004! The first two parts consist of 14 short chapters and there are Q&A exercises to complete for each.

  • Programming: Get QTPyLib (https://qtpylib.io/) up and running, and paper trading a very simple automated MA-crossover strategy in my IB account. Last week I spent some time looking around for algorithmic trading platforms and libraries which could get me up and running quickly, which can support paper and live trading with IB, back testing and support options trading; and this open source library seems to fit the bill nicely. The author of the library, Ran Aroussi, has created a four part webinar series on Treading with Python (you read that right) available on his website here: https://aroussi.com/#tag/recording. I've watched the first one and it was excellent. You get to see how easy it is to prototype trading strategies with Python using Jupyter notebooks. I plan to watch the rest of the webinars when I have the time to follow them, step by step. For now, I prefer to get some actual trading under my belt!
 
Likes: Trader333

FXX

Experienced member
Oct 12, 2017
1,135
181
#25
Momentum algo? You will have a lot of trouble finding good broker for that, it's not easy money. Tested.
it's not a momentum algo, its a news algo. Initially will be used for swing trading but will be enhanced for intraday news trading
 

FXX

Experienced member
Oct 12, 2017
1,135
181
#26
Just from your description you're already ten steps ahead of me, but on first glance I can't see why Python wouldn't be a strong candidate for doing all that. It's only the super high frequency algo traders who obsess over milliseconds where you'd need a compiled language like Java or C++. But sounds really interesting, I hope you'll share your progress.
it will be a combination of technologies and languages but isn't going to be a high frequency algo. I estimate it will produce between 5 to 10 trades a week. Luckily i am in the tech space and work with data systems in a professional capacity with approx 15 years programming experience across many languages. I have never tried something like this before but i do have a profitable strategy i would like to replicate and maybe even enhance. Ray Dalio said his algo makes better decisions than he does these days and their recent 14% gain last year while most other hedge funds lost is testament to that. I will definitely start a journal on this as it will be an exciting project but i have to get a few exams out the way first as part of my career roadmap so don't have a lot of time to spend on it besides taking notes on how to implement it with some light research.
 
Likes: Sharky
Jan 14, 2003
8,447
873
#27
.....Ray Dalio said his algo makes better decisions than he does these days and their recent 14% gain last year while most other hedge funds lost is testament to that.
If the alogo he has developed has done this over several years then that is impressive however, if it is just one year then it remains to be seen if the same set of criteria used in the algo will consistently produce the same results in my vew.
 

FXX

Experienced member
Oct 12, 2017
1,135
181
#28
If the alogo he has developed has done this over several years then that is impressive however, if it is just one year then it remains to be seen if the same set of criteria used in the algo will consistently produce the same results in my vew.
its been running for years and is a pure macro algo
 

Sharky

Staff member
Nov 4, 2000
5,437
260
#30
look at the jma from www.jurikres.com if you want to program a moving average system.
Thanks for the tip TS, I haven't visited that site in years. The JMA indicator you mentioned looks interesting although I don't think they sell a version that would work with Python. That's fine, for the purposes of MA crossover strategy I mentioned, I'm really just looking to get something up and running super quick. It's not an actual system I'd want to trade.