Commercial trading systems
 How to establish if a vendor has a profitable system
There are many commercial trading systems available, but most are not profitable for anyone except the vendor selling them! The best way to determine if a system is truly profitable is to demand to see broker statements for recent trades. Not many people selling systems actually trade their system - you have to wonder why not!? Usually, if someone has written a profitable system they will most likely trade it for all that its worth and NOT sell it.
If they are selling AND trading a system, and you can see profitable results then you can be reasonably confident the system they are selling is good.
 Accounting for slippage and commissions
Other factors to look out for are whether the trading strategy performance reports account for slippage and commissions - when trading S&P e-mini futures, you would want to see roundtrip commissions deducted AND a minimum one tick per side of slippage factored into any backtest results, maybe more depending on the market traded.
Another point to be aware of is if the strategy is using limit orders rather than market orders - limits have a tendancy not to be filled a significant percentage of the time which would render backtesting results invalid regardless of how good they look.
 Curve Fitting and Out of Sample Testing
Using math, you can develop a strategy to do anything you want to. However, if you use TOO much math and optimise your system too precisely to a specific sample of past events (read market prices), then it will almost certainly mean that the strategy won't continue to perform in the future. If you've had some Calculus, then you know that you can use math to specify almost any curve. Therefore, you can optimise code to buy and sell at the most profitable swing high and low for all past market data in your sample. This is called curve-fitting.
Almost every strategy is curve-fit to some degree, but the less curve-fit a system is over a large sample of trades in the past, then the more likely it is to continue performing the same in the future.
Curve fitting is every system buyers worst nightmare because a vendor can optimize his system so that, on paper, it looks like it generates spectacular trading performance. In reality however, an over optimized "curve fit" system will never perform because the specific set of past market occurences that the system was optimized for will never happen in quite the same way again, and as a result the system will invariable fail walking forward- often quite spectacularly.
Out of sample testing is the definitive answer to curve-fitting - its the period of time and the sample of market data AFTER the system was tested - essentially, the system walking forward on fresh, new, live data. The sort of results a system generates on out of sample data are the results you will typically see in live trading, assuming commissions and slippage are factored into the results.
Every system will have losing trades - the ratio of how big the losers are vs winners will determine how much you can expect to allow your account to draw down by. You should also clearly understand how a system manages stop loss orders. We heard about a trading system vended by a company which claims to have an unusually high % of successful trades - however the system basically never closes a losing trading!
 Do your own research!
Careful research is obviously a must before you spend any money. You do not want to spend several thousand pounds, dollars or euro's on a system only to find that it loses your trading capital too!
You can investigate what others have said about commercial systems or vendors by searching through the T2W forum for commercial systems right here: T2W Commercial Systems Forum Other popular forums you can search through are: [Elite Trader] [TradeStation Addons] There is a wealth of information on these sites - if the system you are researching is well established, you can be sure to find some information about it from other users on at least one of these sites.
 3rd Party System evaluation
You can also view systems that are monitored by an independant 3rd party such as Attain Capital - their SystemRank portfolio of tested systems is viewable [here]. Interestingly, many of these have been losing money!
Futures Truth is a magazine published quarterly that is dedicated to testing and reporting on system performance - check them out [here].