A.I. Investing:

Machine Learning Predictions for Individual Stocks and Financial Assets

A Robust Investment Performance Architecture

Machine Learning Stocks Prediction Results and Analysis

Portfolio Backtesting Fiduciary

The chart below shows the annual return predictions of the meta-model for 1,926 U.S. stock exchange-listed stocks with sufficient history and data. These 1,926 stocks are a subset of the Russell 3000 Index, for which we had adequate financial statement history during our research in spring 2022.

Our tests span the period from 2014 to 2021, comparing each stock’s predicted return to its actual return for the same period. Each stock’s quarterly predictions were correlated with its actual returns, providing one data point for the histogram below. A correlation of zero indicates random predictions, while a correlation of one indicates perfect accuracy.

We take extensive measures to mitigate bias. This includes walk-forward out-of-sample validation, separating models into training, testing, and validation sets, thoroughly reviewing feature data, carefully governing feature selection, and prioritizing bias-minimizing objective functions.

For further interpretation, any value with a negative correlation indicates a poor prediction. While 10% of our predictions fall into this category, only 1.5% have a correlation less than -0.25, classifying them as very poor predictions. Therefore, our directional accuracy stands at 90%.

The challenge in conducting such a test lies not in generating attractive results but in ensuring that the returns produced are reliable. Eliminating biases is difficult, which is a common issue in most machine learning projects.

Below is a year-by-year examination of the test results, followed by an aggregate analysis.

What I appreciate about this method of performance evaluation is its clarity in investment terms. Do the actual returns grow steadily? To create these graphs, we classified all predictions into eight octiles. The rightmost octile (bar) represents the top 1/8th of the 1,926 stocks with the highest predicted returns, while the leftmost octile represents the lowest predicted returns. If the actual returns follow this pattern, we know we are adding predictive value.

As you can see, the results are not perfect (never trust perfect AI-based stock predictions! ), but they are sufficient to achieve a realistic and consistent performance advantage.

We tested several variations around the time horizon. In these charts, Time Horizon 4 represents four quarters, or a one-year prediction. Our results were consistent across multiple time horizons, which we have configured as an input variable for generating the predictions. “Window size” refers to the amount of historical data used. A window size of 0 means we use all available data preceding the prediction date. We observed that using more data generally improves results, which is common in machine learning applications.

Year by year Decile performance

you can scroll the years to see each year's predicted vs actual performance

Aggregate performance 2016-2021

For this period, the S&P 500 produced an annual return of 14.75%, and the S&P 1500 produced an annual return of 14.50%.

Portfolios selected from our top decile over the same period would average a return of 28%.

Accordingly, one could judge the performance of the model by buying the top decile and shorting the bottom decile. This long-short, market-neutral portfolio strategy would yield nearly an 8% return.

A.I Investing Performance testing and Analysis
A.I Investing Performance visualization

Results of Market and Sector Models

Scroll through the charts to explore the performance of the sector models. For the sectors, we used a regression of the best-fit sector, irrespective of SIC or S&P classification.

The Future

Portfolio Backtesting - Minimizing Expenses
More data. More features. More models. More results. More learning. This is a machine-learning flywheel. As we continue to build out the model, we expect results to improve.
 

We believe that the combination of stock-specific machine learning model formation within the CAPM prediction architecture provides a real opportunity for the predictions to help deliver better performance for investors across assets, economic conditions, and time.

James Damschroder - Headshot

AUTHOR

James Damschroder

ACKNOWLEDGEMENTS

Special thanks to Marissa Rubb for her steady work on the project, to Professor Diciccio as an academic advisor, Xiaolong Yang, Ph.D., Sr. Lecturer, Sr. Associate Director, MPS Program, and all of the students who have made contributions over the years.

Important Disclosure about A.I., Backtesting and Hypothetical Performance

This portfolio or research is hypothetical. This is a historical simulation of the portfolio performance an investor would have obtained had they invested in the same selections at the beginning of the simulation. This report provides information on how the portfolio holdings would have changed and performed over a certain period.

We have strived to reduce or eliminate potential biases in the process to provide the most accurate assessment of the performance prospects of the strategy. However, it may not be possible for any historical simulation to completely ensure it is free of all biases.

For a more complete understanding of biases and risks when backtesting portfolio strategies, please see “The Gold Standard for Portfolio Backtesting” and “The Seven Deadly Sins of Portfolio Backtesting.

Backtested strategies also run the risk of cherry-picking, which occurs when the author of the backtest creates many variations and presents the most favorable one. This research was not produced in whole or in part by cherry-picking.

This simulation is based on an account with tax-exempt or tax-deferred growth. Taxable accounts will have to pay the appropriate taxes for dividends, interest, and capital gains, which will decrease the performance depicted.

This simulation is not based on actual trading accounts or account composites, which may or may not exist for this strategy and may be materially different, including worse than the performance illustrated above. Past performance is not necessarily indicative of future performance. Performance results, including risk, return, and diversification measures, are not guaranteed to persist in the future.

This historical performance simulation has been adjusted to reflect estimated management fees.

The suitability of this portfolio strategy requires that you have thoughtfully and accurately completed your investor objectives from your accounts’ Investment Policy Statement. Diversification strategies alone cannot assure a successful investment outcome. Strategies offering greater diversification cannot guarantee any reduction in the loss of capital.

Your ability to follow this investment strategy is a risk. Investors often dispose of successful strategies at inopportune times, thus turning potentially profitable strategies into losses.

Portfolio data is taken from sources believed to be accurate; however, there is no warranty or guarantee as to the accuracy or completeness of the data and statistical calculations thereupon. Our performance results are not audited or otherwise approved by any regulatory agency. We regularly perform quality and accuracy tests on our calculations and algorithmic procedures. Portfolio ThinkTank does not furnish investment advice without an investment advisory agreement.

The period selected for analysis may significantly impact the relative attractiveness of the strategy versus another portfolio or benchmark. The author of the strategy controls the default period used to analyze performance, and users may select any desired period from the menu. In general, longer periods, greater diversification, and lower concentrations of holdings result in more credible and persistent performance.