going from starting the systems once a day to starting them once a week
The ongoing trades didn't wipe me out after all, and I am almost back to my highest level of capital of 46k. That bet was very very unhealthy, but the markets went my way and I got away once again.
So, since I still have a capital, I have been working on my effort to automate my systems so I can start them just once a week (cfr. post from a week ago), rather than every day. This will help me to avoid hanging around TWS too often, and being tempted each time. This feat is almost complete. I've been working on doing it on excel, given that my systems run on TWS in conjunction with excel, via DDE.
The last part I needed to work on is creating a macro that tells me via email if everything is going OK, so I don't have to open a VNC / Teamviewer session to the server, and check how they're doing and be tempted, once again, several times a day. Which still wouldn't be as bad as before, because I am now using IB Gateway, which doesn't have any menus to view charts, quotes, although it does of course allow you to place orders via the excel workbook (or the systems couldn't send them either).
So, 1) I have achieved that I don't
have to run them every day, 2) I have achieved that I am not seeing TWS anymore, but just IB Gateway, and now 3) I have also achieved that I can get notified via email, so I can be serene. But even more, now 4) I have achieved that I will get notified via sms, so if I am on vacation, I don't even have to start my computer to check on emails.
I will now show you all 4 achievements and how they work. It will take more than an hour, but I owe it to my ten readers for caring about my trading.
How I now run systems once a week with IB Gateway (achievements #1 and #2)
Going from IB TWS to IB Gateway
The first thing that needs to be said is dealing with the platform TWS, which turns itself off once a day. And I have an idea why IB didn't allow traders to turn it off. There are sloppy people who'd forget it on for days, needlessly, and that would be a useless waste to the community. There's 2 softwares that deal with this problem: IBController (light but very basic) and TWSStart (heavy but with more options), both excellent. But there's still the problem that TWS, running for days, could cause problems. Also, using one of those two applications, although excellent, will by itself increase the probability of problems (the more programs, the more potential problems). So I instead opted for running IB Gateway, which was meant for automated traders and doesn't turn itself off once a day, but never, and only returns the quotes and historical data which you request via DDE. So it's very light. Only problem is that old problem the Security Token being requested, for which reason I had to temporarily disable that security option, until they finally solve it again. I'll turn it back on in a few months. But for now this part of the problem is solved, as far as I am concerned, by going to IB Gateway, rather than using IB TWS with other programs that disabling its daily shut down.
Dealing with VBA's Application.OnTime method
The second thing that needs to be told is how my systems run and how I used excel to run them (which in turn influenced how they run). They only monitor prices every so often, and if prices are where they should be, the signal to trade is triggered. So most of my systems are time-based entries. If at a certain time of the day, prices are having a given behavior, then the trade is placed. For the few systems that don't exactly do this, I had VBA check where price repeatedly, at 15 minutes intervals, but even in this case, the price-checking is done for an interval that amounts to a few hours in total. The way I had VBA handle this whole thing (time-based and interval-based entries, and exits as well are all time-based) is through the Application.OnTime TimeValue method. Let's say I want to check price at 11 AM and then, depending on the price, trigger the trade or not. Then the code will be like this:
Code:
Application.OnTime TimeValue("11:01:00"), "hg.apertura_HG_ID_02"
This is the case, for real, with my HG second Intra-Day system, which has an interval of checking of price for a few hours around 11 AM (US CT). "Apertura" means "opening" in Italian. This is how I named the macro (VBA procedure) that checks price and, if conditions are there, triggers the trade.
So why did I say all this? Because my biggest and basically only problem in going from a daily start to a weekly start of the systems consisted of dealing with VBA's Application.OnTime TimeValue method. It is usually called "Application.OnTime" method, but I add the TimeValue because that's an important detail:
Code:
Application.OnTime TimeValue("11:01:00"), "hg.apertura_HG_ID_02"
In going from a daily start to a weekly start of my systems, the problem I discovered is that running the code "TimeValue("11:01:00")" will not mean that my procedure "hg.apertura_HG_ID_02" will be triggered every day at 11:01, but only in the first 11:01 excel comes across, whether today (if I start it before 11:01) or tomorrow (if I start it after 11:01).
Now this problem gets even more complex if you think that I am running in the same way the entries and exits for 120 systems. This means at least 240 OnTime events per day, but in fact more, because as I said I have systems that check price for several hours, every 15 minutes. Let's say I have even just 30 of these systems, and that they check price not once but 12 times, that is 11 extra times. That would be 30*11=330 extra OnTime events, on top of the 240 mentioned. So a total of 570 OnTime evens to deal with.
So let's recapitulate. I start my excel file, which triggers 570 OnTime events, to be executed in the next 24 hours. And that would be no problem if I ran my systems on a daily basis - and in fact I've been so meticulous that I haven't had any bugs or problems for years with this. But now, going to a weekly start, what were my options?
Well, in short, I had to place, as the 571st event, a timer for an extra event, to trigger all these 570 events all over again. A "loop" as they call it. And all these programmers on web sites are advising against using "loops", but I see no other options.
I do not trust programmers advising to cancel all (possible) extra events and then run all events again, because I am afraid that if I try cancelling 570 events, something might get away, for whatever reason (maybe excel is tired or busy with something else).
And, since I have a period of several hours when no events are scheduled, why not take advantage of that? I will seize that peaceful moment to start my excel file. So recapitulating, once a week, I run all 570 events, turning the file on
before the first event is scheduled, and, as the last event, I insert an OnTime event that runs the macro again. It is a "loop" but I really see no possibility of it going wrong or having any unexpected side effects. The events cannot be postponed, cannot get away, cannot bounce back to me, and therefore it must be ok to run them again, in a loop, once they're all done.
So, here's the deal with my file. I open my excel file, "Private Sub Workbook_Open()" starts as soon as I open the workbook, because it's placed on the VBA module called "ThisWorkbook". In turn this procedure, along with running other procedures (which are only run once), runs this OnTime event once:
Code:
Application.OnTime Now() + TimeValue("00:02:45"), "schedule.schedule_systems"
By the way, here be sure to see the difference between "OnTime Now() + TimeValue..." and "OnTime TimeValue...". In this case the code says "run it now plus x minutes", whereas in my other 570 OnTime events, it says "run it at x time".
Then, once the procedure "schedule_systems" (placed in the module "schedule") runs (only once a week), in turn it starts my 571 OnTime events, the last one of which is this:
Code:
Application.OnTime TimeValue("15:48:00"), "schedule_systems"
This is a line, at 15:48 (US CT, which is the Time Zone on my server), that triggers the restart of the schedule_systems procedure, as I explained. Ok, so we're done with this part.
How I now get notified by excel via email, and then by sms (achievements #3 and #4)
However... with all this mess, or rather this magnificent and intricate machine, no matter how magnificent already, something could still go wrong, at least until I've been doing some debugging for the next few months.
So I want to make sure to be notified that everything is still running, several times a day, and I want to be notified of my margin, overnight margin, available balance, and balance.
Emails from excel
This is where I got the idea of being sent emails by excel.
So, basically I found this script on the web, customized it (see parts in red), scheduled it as an OnTime event (along with the other roughly 600 OnTime events) to be run every few hours, and there's nothing much else to add:
Code:
Sub email() 'weekly
On Error Resume Next
Const cdoSendUsingPickup = 1 'Send message using the local SMTP service pickup directory.
Const cdoSendUsingPort = 2 'Send the message using the network (SMTP over the network).
Const cdoAnonymous = 0 'Do not authenticate
Const cdoBasic = 1 'basic (clear-text) authentication
Const cdoNTLM = 2 'NTLM
Set objMessage = CreateObject("CDO.Message")
objMessage.Subject = [COLOR="red"]Format(Now(), "hh:mm") & ": r." & Format(s.Cells(29, 10), "hh:mm:ss") & ", m." & Format(s.Cells(21, 2), "##,##0") & ", ov.m." & Format(s.Cells(22, 2), "##,##0") & ", b." & Format(s.Cells(22, 4), "##,##0")[/COLOR]
objMessage.From = [COLOR="red"]"<[email protected]>"[/COLOR]
objMessage.To = [COLOR="red"]"[email protected]" '1 sms alert, by vodafone email to vodafone cell[/COLOR]
objMessage.Cc =[COLOR="red"] "[email protected]" '2 sms alerts, by ifttt.com and google calendar to vodafone cell[/COLOR]
objMessage.Bcc = [COLOR="red"]"[email protected]" '2 sms alerts, by ifttt.com and google calendar to TIM cell[/COLOR]
objMessage.TextBody = [COLOR="red"]"cfr. title"[/COLOR]
'==This section provides the configuration information for the remote SMTP server.
objMessage.Configuration.Fields.Item _
("http://schemas.microsoft.com/cdo/configuration/sendusing") = 2
'Name or IP of Remote SMTP Server
objMessage.Configuration.Fields.Item _
("http://schemas.microsoft.com/cdo/configuration/smtpserver") = [COLOR="Red"]"smtp.gmail.com"[/COLOR]
'Type of authentication, NONE, Basic (Base64 encoded), NTLM
objMessage.Configuration.Fields.Item _
("http://schemas.microsoft.com/cdo/configuration/smtpauthenticate") = cdoBasic
'Your UserID on the SMTP server
objMessage.Configuration.Fields.Item _
("http://schemas.microsoft.com/cdo/configuration/sendusername") = [COLOR="red"]"[email protected]"[/COLOR]
'Your password on the SMTP server
objMessage.Configuration.Fields.Item _
("http://schemas.microsoft.com/cdo/configuration/sendpassword") = [COLOR="Red"]"password"[/COLOR]
'Server port (typically 25)
objMessage.Configuration.Fields.Item _
("http://schemas.microsoft.com/cdo/configuration/smtpserverport") = [COLOR="red"]465[/COLOR]
'Use SSL for the connection (False or True)
objMessage.Configuration.Fields.Item _
("http://schemas.microsoft.com/cdo/configuration/smtpusessl") = True
'Connection Timeout in seconds (the maximum time CDO will try to establish a connection to the SMTP server)
objMessage.Configuration.Fields.Item _
("http://schemas.microsoft.com/cdo/configuration/smtpconnectiontimeout") = 10
objMessage.Configuration.Fields.Update
'==End remote SMTP server configuration section==
objMessage.Send
End Sub
There's been a whole lot of customization to do, so if you use it, make sure to look carefully at all the parts in red.
In the body, I didn't write anything, because I don't want to waste time opening the emails. Everything is in the title: margin values, balances, and other useful things.
Receiving text messages on my cell phone for every email
As you can see from the code, I used 3 emails:
1) email_1 at gmail, as the sender and the first recipient
2) email_2 at gmail as the second recipient,
3) another email at vodafone
These 3 recipients would be useless, because emails always go through with the first gmail recipient, so the only reason I have two extra recipients in place is that I want to get notified via sms of the emails.
The most efficient one is... actually here is the summary of my tests:
Sorry, some parts are in Italian.
So, in this intricate ordeal, I am using 3 emails and 2 cell phone subscriptions: a Vodafone cell phone, and a TIM (Italian provider) cell phone, for which I subscribed yesterday, precisely to test these text messages (it only cost me 5 euros - it is amazing how cheaply one can get a new phone number).
Ok, there are 3 methods I have found out so far, and I will list them in a descending order of efficiency:
1)
SMS notification of email by email account with the cell phone provider method. Excel sends an email to my vodafone email account (which I got by subscribing to a vodafone cell phone card), which in turns sends me an sms (I turned sms notifications on) telling me I got an email and its title. This always works, and it always gets sent within 10 seconds of sending the email with excel. So it is unbeatable. So I guess I could stop here, but I went further, just to double-check things and tried 2 other methods, which turned out to be much worse, but interesting.
2)
Google Calendar method. Excel sends an email to my gmail account, which is filtered and marked with a label (based on the sender) I created, which I called "SMS". Then, Google Calendar, acting on a script which I placed in Google Drive, sends me an sms telling me that I received an email which it found in the folder SMS. It is all described in detail and with the code here:
flowontech.blogspot.com/2013/04/how-to-get-sms-notification-for-emails.html
http://scornik.blogspot.com/2012/04/get-sms-notification-for-emails-at.html
https://sites.google.com/site/scriptsexamples/learn-by-example/other-examples/gmail-filter-sms
Don't try to go to the developers.google.com page link because it never works. They just removed it for good:
https://developers.google.com/apps-script/articles/gmail_filter_sms#section1*
Probably because they don't want to encourage people to be sent sms by google calendar.
Ok, so the problem with this are two:
1) with my tweaking of the various options and parameters (especially the options of Google Drive for the Google Apps script), I still haven't figured out a way to be sent just one text message, and I often get 2 of them. But it has nothing to do with how many times you click "run" on the script. The attached image is not updated with this information, but I've found that running the script 5 times on both, still resulted in one sms being sent (rather than 5). It probably has to do with how many times the email is seen before it gets removed from the folder, but I don't see how I can control that.
2) both my TIM and Vodafone cell phone providers don't like getting sent all these text messages by google, and after about 10 of these messages every 24 hours, they stop sending them to me, until the next day or so.
3)
ifttt.com method.
www.ifttt.com goes into my gmail and sends me a text message each time I received an email from a given sender. This method always works, and somehow my phone providers don't get tired of sending me their messages, maybe because they only send each message once, or maybe because they don't have a rule in place imposing a quota yet, like they definitely have for google (after the quota was exceeded for google, I still got message from yahoo and other internet sms). The only problem with this method is that messages take from (according to my tests) 4 to 25 minutes to reach my cell phone.
If you're interested, go to
www.ifttt.com, get an account, and use this script:
https://ifttt.com/myrecipes/personal/8950446
This is the simplest method to implement, along with my Vodafone email notification via sms.
I will keep all 3 methods, but will only use them when I am on vacation.
To disable text messages, I will:
1) remove the second gmail and the vodafone email, by commenting them out in my vba code.
2) disable the rule on my ifttt.com for my first email account
3) disable the script on google drive, by commenting its content out.
When I am not on holiday, I will only receive the emails at my first gmail account, so I don't have to log in to my remote server and check if everything is working properly.
