Adventures in python and algorithmic trading

[Surepips]

These threads are exactly why I love amd have started joining these forums. I'm looking to broaden my knowledge on coding in general but specifically algorithmic coding, never used a bot before but have entertained the idea of programming my own for fun. Alot of great reference here. So thanks in advance

 

[Sharky]

Wow, it's been 3.5 years since I started this journal and 1.5 years since I last updated it. Well as you can probably guess, my progress both in terms of python and algorithmic trading got put on hold. Blame having a second child! My free time got super squeezed. Thankfully baby is now a 2 year old and recently started nursery, so I can finally commit my time again to updating the journal, but first I've got some long overdue housekeeping to do with T2W.

Stay tuned! 😅

 

[Sharky]

Housekeeping under way, and in the meantime, a new public commitment, to get a fully automated trading system running in python with real money up and running before the end of 2022. Much work to do, fun times ahead..

 

[Sharky]

Since I was last learning Python I've upgraded to a brand new MacBook Pro (to replace my old iMac / 2015 MacBook Pro setup). Hence need to set up the Python dev environment again. For this I'm following the handy Hacker Codex guide for macOS Monterey here: https://hackercodex.com/guide/python-development-environment-on-mac-osx/ - installing via Homebrew. Should be set up in a jiffy..

 

[Sharky]

Python set up. Since we're going to be spending time in Mac's Terminal, we might as well upgrade to iTerm2. I found a good, up to date, guide here: https://towardsdatascience.com/the-ultimate-guide-to-your-terminal-makeover-e11f9b87ac99. I installed Oh-My-Zsh in order to install the PowerLevel10K theme with a little help from this tutorial: https://www.swtestacademy.com/customize-mac-terminal/ - note all of this is optional, I just like shiny things!

I also took the opportunity to upgrade Visual Studio Code, which is my text editor of choice having previously used Sublime Text.

 

[Trader333]

What made you change from Sublime text to VSC and did you have the free version of Sublime or paid ?

 

[Sharky]

I used the free one for a long time, then finally ponied up for the paid version. I just kept coming across everyone in the blogosphere using VSC as their preferred editor, so I read up on some reasons people were switching eg. https://nickjanetakis.com/blog/switching-to-vscode-from-sublime-text and that convinced me to make the switch. To be quite honest most tools are more than adequate, inc ST - it's what you do with them that counts! If you're starting from scratch I'd say go straight to the most popular tools, as they've all got a learning curve.

 

[Sharky]

Warning to anyone attempting to set up a new M1 MacBook for algorithmic trading (or any sort of data science). You're going to need python packages like pandas and statsmodel, but these can't be installed on M1 Macs simply by using pip, the standard python package manager. After some digging, I found a workaround here: https://stackoverflow.com/questions/67602680/how-to-install-pandas-on-m1-mac

python3 -m pip install virtualenv
virtualenv -p python3.8 venv
source venv/bin/activate
pip install --upgrade pip
pip install numpy cython
git clone --depth 1 https://github.com/pandas-dev/pandas.git
cd pandas
python3 setup.py install

This worked, you can do the same for statsmodel by cloning from https://github.com/statsmodels/statsmodels. In case you need a specific version you can always git checkout a particular version by finding the hash commit, see: https://tysonmaly.com/programming/git/how-to-clone-older-version-in-git/ and then just checkout the version you want before running the 'python3 setup.py install' command above.

Alternatively you can either use rosetta2 or miniforge/miniconda, as one data scientist ended up doing here: https://towardsdatascience.com/new-m1-who-dis-677e085baffd (and which I might end up doing myself).

 

[Sharky]

I've reached the conclusion that running any sort of python open source project with a bunch of package dependencies is most likely to fail on the M1 Mac simply because there aren't that many M1 configured versions available. Thus I think your best bet is to install the regular x86 version, and rely on Rosetta, it's slower but at least you should be able to run everything - which is fine if like me you're just running on your Mac for development purposes. I can always switch to using Docker or my linux laptop if the speed starts to become a problem.

 

[Sharky]

Updated the list of resources on the first port, to remove Quantopian (replaced by Zipline Reloaded) see Andreas Clenow's blog post and added Robert Carver's rather interesting pysystemtrade.

 

[Here]

will you be trading like Carver?

 

[Sharky]

will you be trading like Carver?

I'm seriously considering it. Check out https://qoppac.blogspot.com/2021/12/my-trading-system.html

 

[1nvest]

I'm seriously considering it. Check out https://qoppac.blogspot.com/2021/12/my-trading-system.html

Thanks for the links Sharky. Seems like a great blog. Ive never been a big fan of carver, only read his smart portfolios book. Have you read his systematic trading?...worth reading?

 

[Sharky]

I've got all three of his books (that doesn't mean I've read them all cover to cover though!). Having said that, I definitely think Systematic Trading, or his newest book, Leveraged Trading, would be a better first book for most people. You can decide for yourself which one based on this handy comparison he wrote: https://qoppac.blogspot.com/2019/10/new-book-leveraged-trading.html (based on your account size and experience level).

 

[Sharky]

Following on from my last couple posts, I've made the decision to embrace Robert Carver's pysystemtrade as my algorithmic trading framework. Given his extraordinary generosity, not only do you have a free, open source platform but his complete futures trading system using both trend and non-trend following rules, all explained and dissected across his books and blog. No black box, fully customisable, and built from the ground up in Python. It ticks all my boxes.

Where to start then! So I decided to spend this week brushing up on a number of areas. It's amazing how quickly you forget things, but at least it's a lot quicker to pick up second (or third etc.) time round.

I wanted to accelerate my learning, and whilst in the past I've favoured learning from books (and continue to), the world has moved on, and these days I also really appreciate good quality educational video content, predominately on YouTube, to which I'm happy to stump up for the Premium subscription give how much time I spend on it (and my kids) during the week.

Whilst YouTube is great for finding quick videos on pretty much any topic, it lacks the structure of something like Udemy. I like how courses are broken down into sections and segments and it tracks my progress, even I choose to jump about different sections. I ended up signed up for a Udemy Pro Personal Plan Subscription that gives you a 7 day free trial (after which it's £26.99/month). This gives me access to tons of the best courses on the site and is a great way to throw myself into intensive learning. I'm covering a lot of different areas and I've narrowed down what I consider to be some of the best courses available on each subject on Udemy, based on ratings, what's included in the Pro plan and my own first impressions of each course. From these take your pick or better still combine them to really grasp the concepts learnt...

Basic Linux Administration

The Linux Command Line Bootcamp: Beginner To Power User

Level Up Your Skills And Take Control Of Your Machine, w/ Dozens of Commands, Projects, and Challenges!

www.udemy.com

Linux Mastery: Mastering the Linux Command Line

Learn the Linux Command Line from Scratch and Improve your Career with the World's Most Fun Project-Based Linux Course!

www.udemy.com

Version Control with Git and GitHub

The Git & Github Bootcamp

Master the essentials and the tricky bits: rebasing, squashing, stashing, reflogs, blobs, trees, & more!

www.udemy.com

Docker (optional, but handy!)

Docker & Kubernetes: The Practical Guide [2023 Edition]

Learn Docker, Docker Compose, Multi-Container Projects, Deployment and all about Kubernetes from the ground up!

www.udemy.com

Python (lots to choose from, for now it's enough to get your head around the basic stuff)

The Python Bible™ | Everything You Need to Program in Python

Build 11 Projects and go from Beginner to Pro in Python with the World's Most Fun Project-Based Python Course!

www.udemy.com

100 Days of Code: The Complete Python Pro Bootcamp for 2023

Master Python by building 100 projects in 100 days. Learn data science, automation, build websites, games and apps!

www.udemy.com

Python Bootcamps: Learn Python Programming and Code Training

Learn Python like a Professional Start from the basics and go all the way to creating your own applications and games

www.udemy.com

Complete Python Developer in 2023: Zero to Mastery

How to become a Python 3 Developer and get hired! Build 12+ projects, learn Web Development, Machine Learning + more!

www.udemy.com

The Python Mega Course: Learn Python in 50 Days with 20 Apps

2022 Redone! Take the full course to become a Python programmer, or pick any project to master specific areas of Python.

www.udemy.com

Python - The Practical Guide [2023 Edition]

Learn Python from the ground up and use Python to build a hands-on project from scratch!

www.udemy.com

Python Data Analysis (esp. numpy, pandas, jupyter)

Python for Time Series Data Analysis

Learn how to use Python , Pandas, Numpy , and Statsmodels for Time Series Forecasting and Analysis!

www.udemy.com

Ultimate Pandas and Python Data Analysis (Complete Course)

Analyze data quickly and easily with Python's powerful pandas library! All datasets included --- beginners welcome!

www.udemy.com

Learn Python for Data Science, Structures, Algorithms, Interviews

Learn how to use NumPy, Pandas, Seaborn , Matplotlib , Plotly , Scikit-Learn , Machine Learning, Tensorflow , and more!

www.udemy.com

Complete Machine Learning & Data Science Bootcamp 2023

Learn Data Science, Data Analysis, Machine Learning (Artificial Intelligence) and Python with Tensorflow, Pandas & more!

www.udemy.com

Visual Studio Code (bonus extra - my preferred code editor!)

Learn Visual Studio Code (VSC) for Faster Web Development

Everything you need to know about Visual Studio Code, the most popular editor in Web Development!

www.udemy.com

 

[Trader333]

Whilst not directly related to this topic, Andrea Neagoie also has this Javascript course which I think is maybe the first developer language that he learned.

The Complete Web Developer in 2023: Zero to Mastery

Learn to code and become a Web Developer in 2023 with HTML, CSS, Javascript, React, Node.js, Machine Learning & more!

www.udemy.com

 

[Sharky]

Whilst not directly related to this topic, Andrea Neagoie also has this Javascript course which I think is maybe the first developer language that he learned.

The Complete Web Developer in 2023: Zero to Mastery

Learn to code and become a Web Developer in 2023 with HTML, CSS, Javascript, React, Node.js, Machine Learning & more!

www.udemy.com

It's also included as part of the Pro subscription. I like Neagole's teaching style and it looks like a great course syllabus if you want to get into web development. Python is great for data science, and can be used for web development with say Django and Flask, but node.js seems to be the way to go, server side, since you need to learn javascript anyway.

 

[iq200]

Better books to read on systematic trading are Pardo and multiple books by Bandy.

 

[Sharky]

Better books to read on systematic trading are Pardo and multiple books by Bandy.

I've got a copy of Design, Trading and Optimisation of Trading Systems by Pardo and hope to read it after I get through Leveraged Trading (first) and Systematic Trading both by Carver. For Bandy, any particular recommendation: https://www.amazon.co.uk/Books-Dr-Howard-B-Bandy/s?rh=n:266239,p_27:Dr+Howard+B+Bandy

Rob Carver, has quite a nice list of recommended reading here:

Resources

Not as portable as a Kindle, but much more satisfying Health warning: These are some of my favourite books and websites. They do ...

qoppac.blogspot.com

 

[Sharky]

Well I haven't succeeded in updating my journal in almost a year.

Life got in the way and my free time got squeezed so unfortunately the journal went into a deep freeze.

The good news is that I've got more time again and I plan to pick up where I left off. Starting with dusting off my old linux laptop, so that's exactly what I'm going to do now... stay tuned for further updates!