Backtest with AIFactorValidation / Cross Validation via K-Fold

Let us say we have the following variables for an AI Factor Model:

And we specify K-fold CV Validation & Training.

What exactly happens during K-fold CV Validation & Training?

K-fold CV (blocked): Divides the dataset into K equally sized blocks and performs cross-validation, ensuring no overlap between training and validation sets:

The AI Factor Model will therefore be able to build the portfolio for 21.6 Years (above: in Orange: Validation Holdout). It builds the portfolio based on predictions of the Validation Training Periods (above: in green: Validation - Training):

K-fold CV (blocked):

• Instead of relying on a single train-test split, K-fold cross-validation splits the dataset into multiple train-test splits.

• K-fold cross-validation tests the model on unseen data multiple times, ensuring that it generalizes well across different portions of the dataset.

• This ensures that the model is evaluated on different subsets of data, providing a more stable and unbiased estimate of performance.

• In K-fold CV, every data point is used for both training and validation at some point, maximizing data efficiency.

• It allows us to use AIFactorValidation to test the entire period for which the data was loaded!

Backtest (out of sample!) with AIFactorValidation in a Portfolio Strategy

This (K-fold CV Validation & Training) also allows us to use AIFactorValidation Predictors (if we save them!) to test (almost) the entire period in Portfolio Strategy for which the data was loaded. AIFactorValidation Predictors get saved here:

For the AIFactorValidation predictor test we can take the Model(s) with the best results:

We click on the ML model (the "Fx" Symbol):

And this window pops up:

Now we copy the String: AIFactorValidation("6MRel + Feature Importance +JJJ K-Fold", "extra trees I") into a ranking system and include the ranking system in a portfolio strategy and can test from 01/04/2003 - 07/27/2024:

The above test remains an "Out of Sample Test" because the Validation Training ("Training Time") and Validation Holdout ("Portfolio Building Time") are strictly separated.

Other specifications of the above portfolio strategy test:
No buy rule. Sell Rule: Rank < 80. Static Weight. Rebalance every week.