ChatGPT, the powerful language model from OpenAI, continues to pique curiosity regarding its real-world applications. Could it, for example, be a viable day trading tool? Sangheum Cho, a researcher at the World Bank, recently explored this very concept, feeding ChatGPT market sentiment from financial Twitter (“fintwit”) to see if it could consistently predict short-term stock movements.
While his results appear promising at a glance, experienced market analysts see the need for a closer examination of ChatGPT’s capabilities and limitations in this realm.
AI and Market Analysis: Not a New Concept
Using AI to analyze market trends isn’t groundbreaking. Research papers have demonstrated moderate success for AI models generating buy and sell signals based on news headlines. Cho’s experiment is distinct in that it focuses on whether an AI can construct a profitable long-short portfolio driven largely by market sentiment. He utilized Bloomberg and Wall Street Journal tweets as the primary input, all from periods outside of ChatGPT’s training data.
Strong Results, But Questions Linger
Cho’s findings indicate potentially significant returns from the ChatGPT-generated portfolios. However, some analysts point out that the model occasionally suggested contradictory actions – both buying and selling the same stock. This seeming randomness raises questions about the depth of ChatGPT’s true understanding of market dynamics.
A closer look at the portfolios themselves yields further insight. One day in mid-January 2023, ChatGPT’s “buy” list was filled with familiar tech giants, while the “sell” list was dominated by smaller meme stocks and speculative companies. This sort of categorization is not dissimilar from what a human analyst might compile after consuming a steady diet of financial news – reinforcing the impression that ChatGPT mirrors trends without deeper insight.
Hallucinations and Errors: Limits to Consider
It’s also worth noting that ChatGPT occasionally included tickers that don’t exist at all, alongside foreign stocks it had been instructed to exclude. One of our analysts highlights a persistent issue with generative AI, known as “hallucinations” – the fabrication of seemingly plausible information. These glitches become problematic when the intent is to make real-world trading decisions. In this case, is it truly picking Chinese companies due to market insights, or is it simply echoing training data that might reference Thai pop stars?
The need for “sanitizing” the portfolio outputs presents a challenge. Cho’s approach involved running the model multiple times, selecting the most frequent recommendations. This filtering helps, but it raises further questions about ChatGPT’s reliability as a standalone tool.
Potential, Yet Premature
While Cho’s research hints at intriguing possibilities, it’s too early to herald ChatGPT as a comprehensive day trading solution. The model shows promise in responding to market sentiment and could potentially be part of a trader’s broader toolkit. However, the presence of errors and the tendency for the model to reflect the biases present in its feedstock should give seasoned market participants pause.
It’s fair to say, then, that ChatGPT is perhaps better suited for idea generation than for directly informing precise, split-second trading decisions.
