How to Run a Rank Performance Test

Rank Performance is a powerful tool used to backtest ranking systems and assess how effectively they have predicted future returns for the selected universe of stocks.

The primary use case is to test the ranking system with the universe you intend to use in your live strategy. However, it is also recommended to run tests using unrelated universes to demonstrate that the ranking system's performance is robust. For example, an investor developing a ranking system for US small caps could also test it on U.S. mid/large caps, Canadian stocks, European stocks, and so on. A robust ranking system should perform reasonably well across most markets. 

Rank Performance Settings

All the stocks in the universe are ranked on each rebalance date. The combination of Period and Rebalance Frequency determines the rebalance dates.

For each date, the stock ranks are used do divide the stocks into quantiles, with the Rank Buckets setting determining the number of quantiles.

The Output filters Min Price and Min Liquidity are applied to remove any stocks which do not meet these minimum requirements. Tip: These filters were implemented as a last resort in case a user accidently used a universe containing extremely illiquid stocks which could give invalid results. However, it is better to apply these filters directly within your universe, as applying them here removes stocks from the buckets and can lead to empty or imbalanced quantiles.

Next, the Max Return % modifier is applied, which replaces any return values exceeding the specified cap with the cap value.

The average return for each rebalance period is then calculated for the remaining stocks in each quantile, after which slippage costs are applied. These returns are used to compute annualized returns and other performance statistics.

Annualized Returns by Quantile

The chart shows the average annualized returns for each bucket (quantile). By default, the results are displayed as Excess vs. Benchmark. In this example, the rightmost quantile (out of 20) shows that the stocks ranked in the top 5% returned an additional 6.8% annually relative to the benchmark, while the stocks ranked in the bottom 5% underperformed the benchmark by 16.4%.

The results can also be displayed as Total Returns:

Return Statistics

Average – Compound Annual Growth Rate (CAGR)

Estimate – The projected return of the top quantile, calculated using a regression of the quantile returns.

Avg Stocks – Average number of stocks in each quantile on each date.

Spearman's Rank Correlation Coefficient - Measures the strength and direction of a monotonic relationship between rank and returns.

Pearson Correlation Coefficient - Measures the strength and direction of a linear relationship between rank and returns.

Slope - how much the predicted return changes for each increase in quantile.

X Intercept – point where the regression line crosses the X-axis.

These statistics are shown for the entire period, as well as separately for the first and second halves of the period. Splitting the results into two sub-periods provides a way to verify whether the ranking system has performed consistently over time.

Cumulative Compounded Return

This chart shows the cumulative compounded return, starting from a base value of 100. Several series can be displayed:

• Up to two quantiles of your choice

• The Universe or the Benchmark

• The High quantile minus the Benchmark, or High quantile minus the low quantile

Period Returns for Each Quantile

Ideally, you want to see consistent excess returns for all the periods shown in the graph. The returns displayed correspond to the selected period and are not annualized.

Stocks per Quantile

The average number of stocks per quantile can vary over time if the size of the universe varies.

Using Rank Performance to Test Individual Factors

The Rank Performance page is also useful for testing individual factors to assess how well they have performed historically. The results below are for the EarnYield factor. It is worth noting that the factor performed well overall, except for the top quantile. This suggests that many stocks which appear extremely cheap may be cheap for a reason.

Rank Performance API endpoint

The Portfolio123 API includes a rank_perf endpoint that allows you to run multiple consecutive rank performance tests programmatically. A common use case is to create a spreadsheet containing a list of factors, then use the rank_perf endpoint to run a rank performance test for each formula in the file and save the results to another spreadsheet.

Another example is running rank performance tests on pairs of factors, assigning each factor a 50% weight to evaluate how combinations of factors perform.

Additional information about the rank_perf API endpoint is available here.