The stock market, or share market, is a chaotic system. It is impossible to predict what will happen in the next day, month, quarter, year, or even decade based on prior data. Because the actors in the system
Chaotic systems are deterministic, nonlinear dynamic systems that exhibit extreme sensitivity to initial conditions, making long-term prediction impossible despite following set rules. The share market is a second order chaotic system.
Second order chaos responds to predictions. So the system not only takes in input, but the actors in the system are trying to guess what all the other actors will do in response to the new data. It’s predictions on predictions on predictions. Because of this, the system is inherently unpredictable over any finite period of time.
The weather is an example of a first order chaotic system. While it is very sensitive to initial conditions and inputs, it does not care if the weatherman predicts the correct outcome. This means we could eventually have perfectly accurate forecasts within a certain time period.
This is not true for the stock market. Because if we had a perfectly accurate AI model that predicted the stock market returns, this would influence stock market returns. Everyone would pile their money into the perfectly accurate AI model, which of course, predicted this, and so the share market would crash, as predicted.
Despite the stock market being a second order chaotic system, there are certain patterns that arise from the noise. One of these patterns is that over long periods of time, the stock market tends to increase more than decrease. With over 100 years of data on the US share market, we know that there was no 20 year period in which the US share market was lower than 20 years previous.
100% of all 20 years periods in the US share market were positive returns. However, this has not been true for all share markets (Japan, Germany, New Zealand for example). And past performance is not indicative of future results.
Stock market randomness is covered in much more detail in the book A Random Walk Down Wall Street by Burton Malkiel. This idea is one of the core investing ideas that forms my investing philosophy and advice at Simple Money NZ.