The University of Chicago School of Economics has trained more than 26 Nobel Prize winners. One of its laboratories, the Becker Friedman Institute of Economicspublished a very interesting article in May of this year on the ability of large language models to analyze accounting financial statements.
Without citing it explicitly, this essay offers more clues to understand to what extent the theory of perfect efficiency of capital markets will cease to be a vision. The accuracy of the forecasts of this type of algorithms increases prodigiously, exceeding the forecasts of investment analysts.
From the beginning of the essay, the three researchers who wrote this article encourage us to understand how this type of algorithm approaches the supreme capacities of the human mind: critical thinking, reasoning and analysis of uncertainties. These are the virtues that distinguish a conventional fundamental financial analysis from a superlative analysis. The model they applied consists of download the balance sheet and income statement of a sample of companies into the algorithm.
Due to the importance of the test, the name of these companies, the precise date of each period, the notes and the report on the financial statements are excluded. With this anonymity, it is possible to avoid the risk that the model used uses its memory, therefore, to predict the past.
The authors use an instruction known as a thought chain, or Chain of thought, (CoT)to try to reproduce the human reasoning of an investment analyst. CoT is a programming method that requires the algorithm to simultaneously generate a detailed, self-contained narrative with its intermediate results, a pseudo form of artificial reasoning. The CoT of this trial was constructed to predict the immediate financial results of 15,401 companiespublished from 1968 to 2021. In addition, financial analysts’ forecasts published between 1983 and 2021 were used in particular from 39,533 observations, coming from 3,152 companies.
Thanks to CoT, the accuracy of the language model has increased from one F1 score from 53% to 60%. F1 score tells us the proportion of false negatives and false positives in the prediction range, the closer it is F1 score at 100%, the more accurate the algorithm, the more true positives and negatives the model will predict.
The essay also highlights the complementarity between its models of linguistic processes, with machine learning algorithms, also called artificial neural networks. These being able to apply specialized tasks, thanks to more specific instructions provided by the programmers. But this virtue limits them when working on more heterogeneous or complex data, as is the case of the financial statements of companies, especially the smallest ones, due to their great sensitivity to the economic cycle. Therefore, the coupling of the two prototypes of algorithms would provide us with a higher level of precision during the asset selection process and the rebalancing process.
To exclude that the correctness of the language model algorithm takes advantage of your memory, The researchers extracted the narrative obtained by the CoT-instructed model about a specific company and then encoded it in another process known as: Bidirectional Encoder Representation of Transformers.. This encoder uses 768 data vectors, loading the results into another artificial neural network, the F1 score of the neural network increased from 63% to 65%. This slight but favorable improvement would confirm that the original narrative convincingly explains the successful predictive ability of the model’s narrative, not its memory.
When the quality of financial statements is poor, financial analysts have an advantage, since their reasoning
Another contribution of the essay is to demonstrate that human intelligence is complementary to large linguistic models. Let us examine two situations that commonly occur, When financial statement quality is poor, investment analysts have an advantage, since their reasoning allows them to access more information about a company than the data that appears in the financial statements. only data source available for the test model.
The second situation is due to the behavioral biases of the human mind, a risk that is currently reduced in models, even if it is likely that gradually, the prejudices of programmers and financial analysts will be inoculated into the algorithms that they program and monitor. After reading this essay published in Chicago, we understand better that Artificial intelligence does not escape human epistemology, at least for a time.
If B. Graham could edit again, The Smart InvestorIt is more than likely that you will enjoy integrating the advances in computing and programming of the last decades. Since the CoT method was created to reproduce human reasoning, it is worth mentioning José Ortega y Gasset, the philosopher of reason in capital letters, and one of his most famous reflections, which fits perfectly with the essay that this article is about: “I am myself and my circumstance.” Ortega spent half his life claiming freedom, overcoming the fear of science, of research, of change, just like his admired Santiago Ramón y Cajal.
