Managing Subjectivity with Objective Functions
Every day, traders must make decisions:
What to trade.
When to enter.
How large a position to take.
When to exit.
Whether the system is healthy.
Discretionary traders acknowledge the subjectivity of their decisions and draw on experience.
Systematic traders use objective functions designed to identify important decision criteria and quantify them.
The underlying assumptions of technical analysis are:
– The markets are not completely efficient.
– There is information in the historical price series that can be used to identify profitable trading opportunities.
– Trading systems can be designed to recognize the patterns and give buy and sell signals.
– Patterns similar to those found in the historical data will be found in future data.
Objective functions are important in two phases of trading systems design:
To rank alternative systems.
To decide the size of the next position.
Ranking alternative systems
A trading system is a set of computations, logic statements, and parameter values that comprise a set a rules that identify profitable trading patterns and give buy and sell signals.
There are an infinite number of possible systems.
In order to make the process manageable, relatively simple systems are designed to focus on specific trading ideas, such as trend following, mean reverting, seasonality, etc.
For any one of these ideas, there are many alternatives. A trend following system might have logic that looks at breakouts, or the crossing of two moving averages, or the projection of a regression. For each of these there are numeric parameters such as the lengths of the moving averages, or magnitude of breakout. There might be multiple rules to exit a position, such as logic, trailing exit, profit target, and / or maximum loss stop.
Designing a trading system is an iterative process of:
Modify the logic and parameters.
Test the performance.
Each set of logic and parameters creates a new trading system – one of the alternatives to be evaluated.
The developer needs to decide which is best.
Best is subjective. The purpose of the objective function is to provide an objective metric that represents the subjectivity of the developer’s definition of best.
Depending on the preference of the trading system developer, determination of best includes evaluation of profitability, risk, trading frequency, holding period, execution efficiency, percentage of winning trades, and any other measurable feature.
A decade ago, the reporting from test runs made by trading system development platforms was limited to basic metrics such as net profit and maximum drawdown. Today, many platforms provide both a wide range of sophisticated metrics and the capability for the developer to design custom metrics. This gives the developer the opportunity to incorporate subjective judgments into a custom objective function – a metric computed using scores and weights for each of the evaluation criteria.
To insure that the objective function accurately reflects the preferences of the developers, it must be calibrated. The calibration process is:
1. Pick a time period and tradable and apply a trading system, generating a set of trade results and equity curve.
2. Perform Step 1 several times, using different logic and different parameter values. Using the same time period and tradable, generate the reports and equity curve for each alternative.
3. Print out the reports and equity curves, one to a page, and lay them out on a table or floor.
4. Arrange them into order according to your subjective preference — from the one you would be most comfortable trading to the least. If necessary, make up and include hypothetical results to fill in gaps in performance and to represent results typical for your trading.
5. Analyze the results, paying particular attention to the features that are important to you. Create a list of these features. Create a measurement scale for each of the features, and assign relative weights to each feature. Set up the measurement scale so higher values are preferred. The sum of all the measurement scores and weights is your objective function.
6. Return to the trade reports and calculate the objective function score for each report. This step should be very easy. If it is not, return to Step 5.
7. If the order the reports were placed in using your subjective judgment is the same as the order of the objective function, you have created an objective function that reflects your subjective preferences. You are done.
8. If the two orderings are different, modify your objective function by adding or removing terms and modifying weights. Return to Steps 5, 6, and 7 until you are satisfied.
When you are satisfied, program your objective function and use it to objectively rank alternatives. Whenever you find that you prefer some alternative other than the one ranked highest, return to the design of your objective function and refine it. The goal is to have a high degree of confidence that trading results that have high objective function scores are results you like, and the systems that produced them are systems you would be confident trading.
I favor functions that reward account growth, reward smoothness, and penalize drawdown. Two well documented functions that include these criteria are:
K-Ratio, described by Lars Kestner and documented in this Investopedia article.
Ulcer Performance Index, described by Peter Martin and documented in this Wikipedia article .
Upon completion of calibration, the designer can be confident that the trading system ranked highest among all the alternatives is the one he prefers. This is useful during the design and testing phases, and is critically important during the validation phase. (I will discuss this in an upcoming article on Walk Forward Testing.)
For more discussion about objective functions, see Quantitative Trading Systems.
Deciding the size of the next position
The trading system that results from the design, testing, and validation provides a single set of trades with single mean, single standard deviation, single terminal wealth, single maximum drawdown.
These results will be repeated as the system is traded only if the future price series is exactly the same as the historical series used during development. In order to estimate profit potential and risk it is important to consider the distribution of potential results. (For more discussion, read the article on Distributions.)
Modeling future performance, including evaluating system health, estimating profit potential and estimating risk is based on:
1. Using the set of trade results that, in the judgment of the developer, best represents the trades that are likely to occur in the future.
2. Using Monte Carlo simulation techniques to create equally likely trade sequences.
3. Analyzing the distributions of profit and drawdown resulting from the trade sequences.
4. Comparing both the magnitude and probability of both the profit potential and drawdown with the trader’s personal tolerance for risk and desire for profit to determine system health and position size.
An objective function formed by taking the ratio of the Compound Annual Rate of Return (CAR) at some confidence level to the Maximum Drawdown (maxDD) at some confidence level is very useful in deciding whether a system is worth trading.
This process, including the software necessary to run the Monte Carlo simulations, is described in detail in Modeling Trading System Performance.
Position size is intimately related to system health and changes dynamically as the synchronization between the logic of the trading system and the price series it is processing changes.
System health must be monitored during trading, and position size revised as necessary.
The correct position size for system that is broken is zero.
Thanks for listening,
Howard
Reprinted with permission Dr. Howard B. Bandy. (http://www.blueowlpress.com)