Financial institutions are playing a numbers game in the core of their business: they evaluate risks and opportunities of potential transactions and if the perceived benefits outweigh the associated risks they will agree to the deal. In the case of traditional banking, this is true for the credit approval process, whereas in the case of insurance companies, the correct evaluation of risk factors determines the conditions of an insurance policy. Like weather forecasts demonstrate, assessments of future events like risk factors or return expectations may occasionally be wrong. Still, if the underlying models to assess risk and return are correct on average, a steady profit will be generated. Competition among market participants forces financial institutions to have well-working systems to evaluate and forecast their risk and return metrics. This is where artificial intelligence (AI), a new breed of data modeling and forecasting, appears with unprecedented capabilities and is silently cutting out traditional methods on a daily basis and in manifold ways.
The financial markets are a very special case. In the stock, bond and commodity markets, billions of dollars are turned over every day. These markets are heavily researched and large profits can be made with superior forecasting systems. These systems are driven increasingly by AI, which is capable of pulling together various types of data and transforming it into forecasts to depict where prices are moving.
However, even though AI technology and data are widely and easily available, getting to the point where profits are generated by AI is still a challenging endeavor. Throwing massive amounts of data into a machine learning or AI system usually leads to devastating outcomes. This rather unexpected behavior is mainly due to such statistical effects as overtraining (when an AI model is so flexible that it ‘learns’ statistical fluctuations but not the underlying market mechanics any more) and spurious correlations, in which a connection between two pieces of information seems to exist but actually only does so on a purely random basis. To overcome these pitfalls, human effort and creativity is needed to come up with data selections and AI model blueprints that are well thought through and of reasonable complexity. Subject matter expertise and experience in the field both play a crucial role for success here.
Not only is AI an exciting technology for market speculators and investors but also for producers who need to obtain raw materials in large quantities and are sensitive in their business model to the purchasing price.
Take, as an example, the price for three months of frozen orange concentrate, which can be traded on futures exchanges like the International Exchange (ICE) in Atlanta, Georgia. In former times, smart people would have produced a future price estimate based on their knowledge, gut feeling and the information they had at hand, such as from newspapers. But even in the case of well-informed and experienced experts, it is hard for the human brain to outperform sound mathematical methods on a continuous basis. Evidence for this can be found in the work of Daniel Kahneman, a behavioral psychologist and Nobel Prize laureate. He found that, by nature, it’s a tough job for the human brain to deal with large amounts of quantitative information on a continuous basis. Mathematical methods for financial forecasts have been developed over decades and are continuously disrupted these days by AI. The reason for this continuous replacement is that the early models weren’t made to process vast amounts of different sorts of data – a capability that’s absolutely needed in today’s times. Today, data of all sorts can and should be utilized to come up with a best-of-breed forecasting system.
In the case of frozen orange juice concentrate, the future price will eventually be determined by supply and demand. On the supply side, it appears obvious that the current season’s recent weather and sunshine conditions can be applied to estimate the degree of ripeness of oranges, at least for a certain orange plantation where this information is available. This estimate can be enriched with satellite imagery of orange fields and complemented with past orange production records and weather forecasts, and maybe even the likelihood of big weather events like hurricanes. On the bottom line, this information can be transformed into a supply forecast for this particular orange plantation at a specific point in time. To produce a forecast that reflects global price discovery, this process of transforming data into an orange supply forecast should be repeated for the world’s major orange producing regions. Obviously, this sounds like a tedious job for a group of experts but is exactly the scenario for which AI has its strengths. After having considered the supply side, determining the demand side is a whole other story but follows the same principles. Both the supply and demand forecast can then be used to eventually come up with a future price estimate.
Only advanced machine learning and AI methods are able to digest these amounts of information on a regular and comprehensive level. In the above described example, methods of image recognition, text processing and time series analyses will play significant roles to produce a sound and competitive forecasting system. Today, dedicated AI systems like these are built in various ways and for different applications. However, due to the complexity of the matter, these improvements in forecasting accuracy happen in tiny and silent steps. This stealth mode of gaining ground means it is a real threat to those companies who do not invest in such systems. Once the competition’s intellectual property on forecasting future price movements is advanced enough to constantly outperform the traditional methods, it may be too late to catch up.
Still, human knowledge and experience are very important success factors for the application of AI models. The strong side of an AI system is to transform large amounts of data into accurate predictions on a highly repetitive pattern, but it also uncovers its weak side: An AI system is (at least today) not capable of interpreting singular events such as the U.S. government imposing tariffs or discover general shifts in market conditions, such as the outbreak of a war.
Artificial intelligence is an extremely fast-moving field and the state of the art today may look antiquated five years from now. It is very important for financial institutions to follow closely with these developments and to invest to maintain their competitive edge. The slow-movers in this development might have to deal with some leftover deals on bad margins and won’t get the chance to fight their way back to the top.
