EdgeWare FastBreak Pro Version 5 - Manual (Page 69)

69
Reversing the In-Sample and Out-of-Sample Data Periods
FastBreak Pro allows the OS data period to be earlier than the IS data period. The more
traditional method is to allow the GA to derive parameters then perform walk-forward
testing on the most recent market data. The advantage to the traditional order is to look
at the predictive capability of systems on the most recent history. However, an argument
can be made that it is better to present the most recent data to the GA so that parameters
are optimized on recent history. Testing the systems on older data is used to recognize
when over-optimization begins to occur.
A study was performed to compare the two different methods. One optimization run was
made in the traditional order using an IS period of 6/1/1989 to 12/31/1996. The OS
period was 1/2/1997 to 12/31/1998. A second run was made reversing the IS/OS order
using an IS period of 6/1/1991 to 12/31/1998 and the OS period was 6/1/1989 to
5/3/1991. Our standard Fidelity Select testing family was used. The best systems from
generation 10 of the first run were saved along with the best systems from generation 12
of the second run. These twenty systems were then tested for the period 1/2/1999 to
10/29/1999. Here are the results:
System
Num. IS
OS
Ann.
Return IS
OS
MDD OS IS
Ann.
Return IS
OS
MDD
1 47.8 -13 20.4 -16
2 32.5 -14 20.5 -13
3 36.2 -12 20.5 -13
4 40.2 -11 20.5 -13
5 34.1 -10 35.1 -16
6 37.4 -10 16.8 -14
7 37.4 -10 53.0 -12
8 40.2 -10 14.7 -15
9 47.8 -10 14.5 -14
10 34.1 -10 18.4 -13
Avg 38.8 -11 23.4 -13.9
S&P 13.2 -5 13.2 -5
Family 17.4 -12 17.4 -12
Columns 2 and 3 are results from the traditional date order, with columns 4 and 5
reversing the test order. In this particular study, the traditional date order produced
superior returns in 1999 – 38.8% vs. 23.4% returns, and 11% vs. 13.9% MDD. Both
methods out performed the family average.
This single study does not necessarily prove that the traditional date order is always the
best. We have included the study results to advise you that the date order can be
reversed, and that under other conditions may prove useful. Postscript: The ability to
reverse IS and OS order became very important during the bear markets of 2001 and
2002. This date range was the only significant bear market data in the FastTrack
history. We found that optimization on this recent bear market data improved risk
performance.