Skip to content

What Is Sentiment Analysis And How Does It Work?

justin freeman
Justin Freeman trader
Updated 6 Oct 2022

Investor sentiment is one of the most important factors that determine asset prices. The way that the mood of traders can influence trading decisions explains why markets aren’t ‘efficient’ and why prices move away from the ‘fair value’ for periods of time. During times of optimism and when the fear of missing out takes control, markets can form bubbles. When panic takes hold, stocks, forex, commodities and crypto can, in the short term at least, overshoot to the downside.

What Is Sentiment Analysis And How To Use It

The upside of incorporating sentiment analysis into trading decisions is that being able to see through the herd mentality can result in trade entry and exit points being optimised. Understanding the way that short-term price moves are out of line with long-term valuations can enhance returns. The difficult part is measuring, or even defining, which elements of investor sentiment are best to consider. Below are some of the ways that sentiment analysis is used to help with trading decisions and the factors to consider when using it as part of a successful strategy.

What Is Sentiment Analysis?

Investor emotions can’t be tracked and interpreted as directly as the numbers found in the financial statements used in fundamental analysis, or the price data used in technical analysis. One approach adopted by those looking to track the mood of the market is to use metrics that indirectly assess the influence that greed and fear are having on a market.

One of the well-known sentiment analysis tools is the CBOE Volatility Index (VIX), which monitors buying and selling activity in options markets. If demand for Put Options increases, this indicates that investors are hedging potential downside risk and a price correction could be imminent. A reduction in demand for Put Options points to investor sentiment being more bullish. Polling traders to establish their view on the markets would be difficult to implement and be open to abuse, but the VIX records the trading activity of market participants and gives an indication on the mood of investors.

The Fear and Greed Index is another popular sentiment analysis tool. It combines data from the VIX and six other sentiment-based metrics and averages out those readings in an effort to measure overall market sentiment. A neat feature of the Fear and Greed Index is that it takes a wide range of data points and converts them into a single number that reports the overall mood of the markets. That number is between 0 and 100, with a reading above 50 being a sign that optimism is the main price driver, and a reading below 50 suggesting that fear is determining price direction.

Tracking market activity is not the only way of trying to establish the nature of investor sentiment. Investment funds that use computer models to track online comments and trade off the back of them have been operating for more than a decade.

Paul Hawtin, the founder of the hedge fund Derwent Capital Markets, went on record as far back as 2012 saying: “For years, investors have widely accepted that financial markets are driven by fear and greed, but we’ve never before had the technology or data to be able to quantify human emotion.” His fund collected data from financial market-related Twitter accounts and ran it through algorithms designed to turn the harvested data into useful information.

Turning new ideas into practical investment decisions isn’t always a straightforward process, but there are two good reasons why the project continues to be pursued. The first is that the area of study is ground-breaking and exposes new ways of getting an edge over the rest of the market. The other is that it is apparently effective. The mood index developed by researchers at Indiana University and the University of Manchester, for example, predicted the Dow’s daily fluctuations with an astounding 87% accuracy.

client sentiment ig

Source: IG

The last and most straightforward sentiment analysis tool has a much longer track record than the above models and ties up the thinking behind how all of the different systems work. Order-book analysis involves simply watching movements in the market and establishing how the different participants are positioned. It has long been used by those looking to gauge if the next move will be up or down.

daily changes in positions ig

Source: IG

Using data relating to the accounts of real traders, brokers such as IG make it possible to establish if the clients on a platform are net long or short, forex, stocks or other assets. The snapshot taken by the IG Sentiment Indicator reports the aggregate positions on its platform, the percentage change over time, and whether market signals are bearish or bullish.

Making Trading Decisions Based on Sentiment Analysis

Traders can therefore use market data-driven sentiment analysis tools such as the VIX, or more qualitative models such as those that scour social media platforms. As with most forms of analysis, developing a hybrid approach and incorporating both methods, and as many indicators as possible, is likely to result in more rounded conclusions. Deciphering market sentiment is only the first part of the process of trading using sentiment analysis.

After establishing investor sentiment, traders are faced with trading with the trend or against it. The former, taking on positions that line up with market momentum, is a tried-and-trusted route for successful investors. Experienced market gurus such as Cathie Wood and Warren Buffett have made fortunes out of mastering the technique and taking the long-term view, but it can also be harder to do than it appears.

The nagging doubt in any momentum trader’s mind is that ‘the trend is your friend, until the bend at the end’. While a reversal is bad news for traders who are in established positions, it can be good news for those looking to get into positions at the start of the next trend.

Whether you are trading with momentum or looking for a reversal, sentiment analysis is a useful tool. Making the right call, at the right time, will depend on also using other research and indicators. These could include breakout strategies, trendline analysis and company analysis, with the general consensus being that the more indicators that point to a particular price move, the stronger the signal.

Advantages of Using Sentiment Analysis

Despite the opaque nature of some of the metrics used in sentiment analysis, the approach does have one particular advantage over other types of analysis: it is a leading indicator.

The VIX, for example, measures trading activity in options that are future-dated instruments. If investors are buying more Puts and driving the VIX index upwards, it is because they expect increased price volatility between now and the options expiry date. They’re buying more insurance to hedge the risk on the rest of their portfolio.

Social media-based sentiment indicators can also be considered forward looking. An algorithm that measures the sentiment regarding an upcoming news announcement, or the trading prospects for next year, captures the thoughts shared by traders on where the market might be going.

The forward-looking nature of sentiment analysis compares well to other forms of analysis such as fundamental, quantitative and technical. Those schools of thought give a clearer picture because they use hard data from government announcements, company reports and price data, but all of that information is historical in nature. It gives a clear indication of what has happened, but less of an idea on what might happen next.

Disadvantages of Using Sentiment Analysis

The approach of the Fear and Green Index is discussed in more detail here. Part of that analysis considers how using a quantitative approach to measuring sentiment still involves arbitrary decisions being made in the design phase when determining what methodology to follow. The seven indicators that go into creating that index are, for example, all given equal weighting, which is a decision that could be questioned.

The qualitative models such as those that scour Twitter for clues in online comments are open to the risk of misinterpreting the true meaning of the posts. One anecdote shared during the infancy of the program was that the models misinterpreted regional expressions and nuances in humour. Tweets made by US-based investors that “This is great!” tended to be aligned with markets rallying, but the British sense of irony meant that the phrase could as equally be used tongue in cheek when markets were crashing.

Another challenge that sentiment indicators face is that if set up to monitor one sector, such as European stocks, then they might not be as effective when applied to another, such as the global crypto markets. European bourses have strict opening times, whereas crypto markets operate on a 24/7 basis, and they attract different types of investors. A broad ‘catch-all’ sentiment analysis model may not be effective given the different characteristics of the different financial markets.

One of the most straightforward sentiment analysis tools is the order-book analysis. At face value, this appears quite robust as it relies on data taken from the accounts of active traders. Even this approach faces the challenge of being able to ensure that the data it uses is appropriate for the circumstances. The positions reported by individual brokers such as IG will relate to their own accounts and so reflect only part of the entire market. It is possible to extrapolate that the accounts at IG are likely to be a realistic sample of the entire market, but the approach does build in an inherent weakness.

Forex markets are notoriously hard to analyse using the order-book approach. Unlike stock exchanges where all trade instructions go through a centralised server, the forex markets are unregulated and order processing is decentralised. This makes establishing the true nature of currency order-book positions a more complicated task.

Final Thoughts

Sentiment analysis is a diverse field that involves a range of different approaches. There are difficulties associated with collating, measuring and interpreting the data due to the core samples being based on the thoughts of investors. These are by definition harder to establish than, for example, the latest unemployment figures from the US.

Despite the challenges faced, strategies that factor in investor sentiment remain popular due to their position as leading indicators. Interpreting the effects of human and market psychology makes intuitive sense, and there is also empirical evidence to suggest that sentiment analysis can be one of the best indicators of future price movements.

Sentiment analysis is also a field that is continuously evolving. There is a feeling that the next effective indicator could be around the corner as improvements in AI, data science and text analytics continue to generate clearer outcomes from what starts out as a relatively unclear data set. Whether you’re an experienced investor or are new to trading, sentiment analysis can form an important element of any strategy. The basics still apply, which include the fact that there are benefits from using any kind of analysis in conjunction with other types of research and that using a trusted broker remains the first rule of any kind of trading.

justin freeman
Justin is an active trader with more than 20-years of industry experience. He has worked at big banks and hedge funds including Citigroup, D. E. Shaw and Millennium Capital Management.