EdgeWare FastBreak Pro Version 5 - Manual (Page 46)

46
After at lest one generation has completed, go to the Out-of-Sample Results screen
(shown when the Light Bulb tool bar button is clicked). The GA will continue to look for
combinations of parameters that gave good historical performance. However, at some
point, the trading systems will become over-optimized and not be good systems under
future market conditions. The reason we reserve some recent out-of-sample market data
is to test the optimized parameters. We are looking for the point that the systems start to
become over-optimized. Looking at the screen below we can see that the GA is finding
better IS results. The IS ANN column is calculated by taking the ten best systems
available at the end of each generation and using the system parameters to test for returns
in the OS Date range. The results from the top 10 systems are simply averaged. You do
not always see an increase from generation to generation in IS performance (as you do in
this example) because the top 10 systems are based on “adjusted performance” not
absolute performance.
We want to increase the IS performance but not to the point we start hurting the
predictive future performance in the OS period. This is why we watch the OS ANN
column. The OS ANN column is the result of taking the top 10 systems at the end of
each generation and using data in the OS Date Range to calculate the performance for
each system (the 10 performance numbers are averaged together to obtain the average
value that is in OS ANN column). When we begin to see the OS ANN returns start to
drop, this is an indication that over-optimization is starting to occur. Some runs can