Running forward optimization is the method used in finance to determine the best parameters to use in trading strategies. Trading strategies are optimized with sample data for a period of time in a series of data. Data left is reserved for sample testing. A small amount of data is ordered after the sample data is tested with the recorded results. The time window in the sample is shifted forward by the period covered by the outgoing sample test, and the process is repeated. In the end, all recorded results are used to assess the trading strategy.
This means getting the most appropriate/stable parameters from the system and running the system with this parameter using another data segment and these two data segments do not overlap. This is the culmination of the following methods and helps in the creation of a powerful system.
Backtesting uses past data to test the trading system. This is useful because, if the system is not profitable in the past, it's a strong sign that it will not be profitable in the future. This refers to applying the trading system to historical data to verify how the system will perform during the specified time period.
The fore test also known as Walk forward testing is a simulation of real market data on paper only. This means that even if you are moving along a live market, but you are not really putting real money in, but doing a virtual trade in the market to understand better market movements. Therefore, it is also called Paper Trading . Future performance testing is an actual trading simulation and involves following the system logic in the direct market.
Video Walk forward optimization
Overview
One of the biggest problems with system development is that many systems do not survive until the future. There are several reasons for this. The first is that the system is not based on a valid premise. Another is that the test is not audible for reasons such as:
- Lack of toughness in the system due to improper parameters. A system is considered strong if it runs well in every market condition.
- Inconsistent rules and incorrect system testing using 'out of sample' and 'in sample' data.
Advanced Runs Analysis perform optimization on a set of training; test in a period after the set and then roll it all forward and repeat the process. We have several periods outside the sample and see these results combined. The analysis goes on initially discussed by Robert E. Pardo. Walking forward can make the trading model one step ahead. Walking forward is so called, because we have some walking training and testing periods tend to be less frequent overfitting.
Walking forward testing allows us to develop a trading system while maintaining a reasonable 'degree of freedom' . Walk-forward testing brings the idea of ​​'out-of-sample' testing to the next level. This is a specific application of a technique known as Cross-validation. This means taking your data segment to optimize the system, and other data segments to validate. Therefore, here you optimize the data window that says the last 1000 bars, and then test it on the next 200 bars. Then roll it all in front of 200 bar and repeat the process. It gives you a large sample period and allows you to see how stable the system is over time.
Let's say you are considering a strategy around moving averages. You take the first 3 months of data, and find that for that period, the 20-minute moving average is optimal (using tick data). You then validate this rule by assessing its performance for the 4th month (ie earnings, rewards/risk, or other interest stats). Next, you repeat the optimization using data from 2-4 months, and validate it by month 5, and keep repeating it until you reach the end of the data. The performance you get for the month of validation (4-13) is performance outside of your sample.
The basics behind the data used
Prior to backtesting or optimization, one needs to organize the necessary data which is the historical data of a given time period. This segment of historical data is divided into the following two types:
- In-Sample Data : This is the last segment of the market data (historical data) provided for testing purposes. This data is used for any initial and optimized testing and is the original parameter of the system being tested.
- Out-of-Sample Data : This is a backup data set (historical data) that is not part of the in-sample data. This is important because it ensures that the system is tested at other periods of historical data not earlier so it eliminates any bias or influence in system performance checks.
The process is to first develop a trading system using in-sample data and then apply the out-of-sample data to the system. The results of both cases can then be compared and tested.
Maps Walk forward optimization
Description
The concept for walk-forward testing is similar to using the 'in-sample' and 'out-of-sample' testing periods. Instead of optimizing for twenty years of data and using the last four years of data for testing, optimization is done over ten years and the system is tested on the eleventh. After the test is complete, move the entire time window ahead one year and run the test the following year. Find the optimal set of parameters for each 10-year window and use the set of parameters to trade for the next year. Move the time window ahead one year and run the test the next year until all the years in the data series have been tested.
When system performance is evaluated, all one-year windows are consolidated to construct periods outside the sample for each optimal window. Out-of-sample performance is used to assess how good the system is.
Walk-forward testing works like this. Let's say you have twelve years of extended data from 1998 to 2009 for the markets you want to trade. Let's assume that your trading strategy requires at least three years of data for testing and optimization.
To begin, start by developing and optimizing the system using only the first three years of data - in this example, 1998-2000. In these three years of data, try as many ideas as you like and optimize the parameters in various ways you can think of. It's important not to see any data after 2000! When you think you have found the 'Holy Grail' trading system, note the rules for the system with optimum parameters. These rules and optimized parameters will be used later for final testing with new data beginning in 2000.
Slide window time three years ahead a little - say a month. Now, the data you run runs from 2 months to 2 months 2000. Repeat the analysis, including optimization and note the rules and parameters that are optimized. In the last pass, this parameter will be used for the 2nd month of year 2000.
Continue with 'walk forward' and optimize the three-year data period. Record the results for use in the first month after a three-year optimization period. When your data finally runs out in 2009, go back, and test the system for the entire period from 2000 to 2009. Change the rules and parameters every month to use the data you find and record. As a result, you perform tests outside of new samples for each month. System performance for these nine years without sample (108 months without sample) is a much better indication of how the system will work in real time than the performance of every single time period used for optimization.
There is nothing magical about the time assumed - three years for system development and one month for a walk-forward interval. Selecting these two time parameters is a trade-off between the timing of optimization and the statistical validity of the results. In practice, I've found that using about 20% of the optimization period for walk-forward windows works pretty well. Which window sizes work best is also affected by the given system, for different systems, optimal training and window sizes outside of the sample will be different.
If the results for 'out of sample' months look good, continue the process of progressing in real time to find the parameters to be used with real money. Another advantage for this method of system and trading development is that your system will be better adapted to changes in market behavior over time. The market is changing over time - we've all seen systems that have been making money for several years and then stopped working because the market has changed how often these changes affect systems related to the best size for training and out-of-sample sets. Testing into the sample and out-sampling manually as described is useful, but automated testing ahead with automatic parameter selection is the best way to avoid curve adjustments.
Conclusion
For a better understanding, please see an example here.
To evaluate any system, one should check its performance when using "External Data Sample" (test data) and not "Data In Sample" (data used for system optimization). Thus, a test runs forward determines the performance of the optimized system as follows:
- Is that realistic? This is considered realistic if it can be suitable for all test data (or at least for larger test data segments) used. This implies that the system has real-time and strong market characteristics.
- Is it overfitting? If the system is not working properly using test data and seems to match only the opportunity characteristics (not necessarily a part of the test data), the system is considered overfitting. This is not a strong or reliable one and should not be used for trading.
Therefore, out-of-sample data plays an important role in determining the validity and reliability of the system and is a realistic estimate of how the system should work in the real market.
See also
- Re-hack
- Trade strategy
- Optimization (math)
- Overfitting
References
Literature
- Katz, Jeffrey Owen, and McCormick, Donna L. "The Encyclopedia of Trading Strategies." McGraw-Hill, 2000.
- Essential technical analysis: tools and techniques to see market trends By Leigh Stevens
- Encyclopedia of technical market indicators By Robert W. Colby
External links
- Unsampled testing and running forward
Source of the article : Wikipedia
0 Comments