I was having breakfast with Scott L out systems tester extraordinaire the other day. Our chice of venue for such outings is a cafe in Brighton. Not because it is half way between our two abodes but rather that it boast a boat ramp. And as everyone knows a boat ramp is the best free entertainment you can have.
Our conversation turned to system design as it often does and Scott had done some interesting work looking at the impact of changing markets on system performance. To illustrate this point he tested a simple generic system applied to the ASX 300, NASDAQ 100, S&P500 and the Russell 2000. It is important to note that exactly the same trading system was applied to each market. The only variable was the market used.
The basic data looks like this.
Interestingly, this system outperformed the market in all markets that it was applied to. This rate of return whilst modest combined with the very low drawdown would put almost every fund manager in the world to shame. But that is a different issue.
What is really interesting is the Monte Carlo results. In testing a system it is important to make certain that the system is truly robust and that the results achieved are not simply dependent upon the order in which the trades were taken. This is done by what is known as a Monte Carlo analysis which reorders the results to see what happens if trades are taken in a different order. In simple terms Monte Carlo analysis makes certain that you simply did not get lucky.
These results show something very interesting in terms of the robustness of a system when applied to different markets. The most compelling feature of this table is the Russell 200o results. During this simulation the average profit for the Russell 2000 was a paltry $20,977 and even more startling is that this system has the potential to actually go broke when applied to this market.
The chart below shows the pronounced difference between the various markets.
The take home message for this particular simulation and for trading in general is that your choice of markets matters.