EdgeWare FastBreak Pro Version 6.2 - Manual (Page 50)

50
future market conditions. The reason we reserve some recent out-of-sample market data
is to test the optimized parameters. We are looking for the point that the systems start to
become over-optimized. Looking at the screen below we can see that the GA is finding
better IS results.
The IS ANN column is calculated by taking the ten best systems
available at the end of each generation and using the system parameters to test for returns
in the OS Date range. The results from the top 10 systems are simply averaged. You do
not always see an increase from generation to generation in IS performance (as you do in
this example) because the top 10 systems are based on “adjusted performance” not
absolute performance.
We want to increase the IS performance but not to the point we start hurting the
predictive future performance in the OS period. This is why we watch the OS ANN
column. The OS ANN column is the result of taking the top 10 systems at the end of
each generation and using data in the OS Date Range to calculate the performance for
each system (the 10 performance numbers are averaged together to obtain the average
value that is in OS ANN column). When we begin to see the OS ANN returns start to
drop, this is an indication that over-optimization is starting to occur. Some runs can show
a drop in OS performance for a couple of generations, but then new solutions will be
found which start to improve OS results. Therefore, you should not stop the optimization
at the first sign of a decrease in OS performance.