Here are the answer your questions:
1. For me its simple, I use MT4 is my main data source because it's free (no subscription fee). The data coverage as well sufficient especially for forex analysis and multimarket analysis (with stock, index, etc) as basis for neural network. Demo account can be use as our paper trading and forward test, while waiting the proper trading strategy. You are correct we can use NST to send the trading order to MT4, but I do not use it because I use all the indicators value as my NN input from MT4.
It should be not too much different either using indicator values from MT4 or NST, but the important point is in MT4 we get the indicator values at current bar while in NST we can get only the previous bar value (please correct me if I am wrong).
Other advantage is if we are using all data, indicator and NN firing in the same platform (MT4), we get benefit to remove any slippages or delay due to calculating process in NST and transferring the signals. That is why I use Neuroshell2 (not Neuroshell trader) to create the neural network dll and call it from MT4.
I get difficulties to use trained NN in NST if the NN inputs are coming from MT4, but it is possible to use indicators from NST and fire them in NST as well.
2. I know the back test result on MT4 not as good as NST, but I should accept it as long I do not have any other tools can be used. For your information, I took the data from trained NN from MT4 on 15 minutes time frame about 2500 data since 1999 (20% out of sample) and I implement to test it on 4H time frame after 2006. For me it can be considered as a full walk forward test. It give me a hints how a robustness of my trained NN (correct me if I am wrong).
Thank you for your sharing idea.
Thanks for Arryex and the other contributors.
Here are a few more bytes and questions on the subject.
Mt4 is indeed a very accessible system with tons of free data providers, huge community and lots publicly available documented experience.
My own problem with MT4 is its computational and programming limitations. For example, even if you manage to program a very fancy indicator that is heavier on resources, it will be extremely slow to backtest. Try testing an EA that includes Corona or other spectral indicators for example. In such cases you will have to give up the available intra-bar data and do with the lower quality MT4 backtesting options. Communicating with other systems is possible but very tedious.
It seems that many of the other alternatives like open quant and multi-charts aren't as prevalent and don't offer this attractive communal experience.
I therefore expect that it should be much harder to develop systems for them but assume that they offer many advantages (any experience of other members on this regard?)
I got to this thread because recently I've been considering NST. From what I understand this is a closed system in the sense that you can't really extract the internals of a trading strategy including its NN components. To me this is an obstacle (which I'm willing to overcome if I can really be convinced that it can generate for me $$ more efficiently). I imagine that this is also one of the attractions of Arryex Neuroshell 2, with which you are able to extract an all inclusive C routine. It also offers a fine control of the network learning process, which is a benefit if you know what you are doing.
One of my concerns regarding NST (including it's connectors to brokers) is that (as far as I understood) it doesn't provide sufficient flexibility to control money management.
Is that correct? Is it possible for example to program there an "EA" that trades according to Kelly fraction?
Your comments please.
I assume that some of these limitations will be alleviated with the forthcoming MT5, but we aren't there yet.