Monitoring & Analyzing Social Media

Five Myths about Automatic Sentiment Analysis

This is cross posted on Alterian’s Engaging Times blog.

Sentiment analysis is a hot topic. If the social media monitoring tool doesn’t have it, there’s criticism. If it does have it then there’s skepticism. So let’s take some time to talk about these five myths:

1. The technology isn’t accurate
2. Sentiment doesn’t take into account cultural differences
3. Positives and negatives cancel each other
4. Can’t identify the influence of those expressing the sentiment
5. Sentiment doesn’t indicate action

1. The technology isn’t accurate

Sentiment analysis using natural language processing. Yes, it is done by a machine and no, it’s not 100 % accurate. The industry estimates that it’s at 70 – 80%. We are very open about that and recommend that it be used as an overview.

In using a tool like Techrigy SM2 for automatic sentiment, customers can see the overall view of thousands of search results. It would take hours to manually review the same amount and one still wouldn’t have an overall sense of the percentage positive vs negative.

SM2 has a customizable dictionary so users have the option of reviewing a sample of results and revising the dictionary. This functionality was added to accommodate the language differences of various verticals. For example “well” in the health industry is a positive thing, but in the energy industry, an oil well has no bearing on sentiment.

2. Sentiment doesn’t take into account cultural differences

One of the first dictionaries that we customized was for our friends down under. Australia has some vernacular that is unique to it’s region. No matter the location, those lexical idiosyncrasies can be added to SM2’s dictionary.

3. Positives and negatives cancel each other

Marta Strickland listed this in her recent blog post: Five Reasons Sentiment Analysis Won’t Ever Be Enough

Many people assume this. In SM2 a search result can be attributed with positive and negative sentiment. In addition, a search result may have 3 positive aspects and 1 negative. This would also be charted as such (3 in the positive column, and 1 in the negative).

This information is then considered along with the length of the post when calculating tone.

4. Can’t identify the influence of those expressing the sentiment

That is possible and it utilizes the power of SM2’s categories.

Let’s consider the negative sentiment around McDonald’s McCafe. We want to see the negative results in order of influence (SM2 refers to it as Popularity).

By creating a category called ‘Neg McCafe Sentiment’ and assigning those results to it, we can then view them in the Demographics report. This will allow you to see the distribution from highest to lowest influence. You can drill in at each level.

And they can also be displayed under ‘View Results’ and displayed in order of Highest Popularity first. This is a nice way to browse them.

5. Sentiment doesn’t indicate action

I agree that it doesn’t indicate it, but I would also argue that automatic sentiment makes it very easy to realize trends/patterns that would otherwise be very difficult to identify.

By reviewing the results under Negative Sentiment in the McCafe example, some trends become apparent. And they fit under the following business objectives:

• feedback on the products – product development
• complaints about customer service – customer service
• irritation with specific ad’s – marketing/advertising
• reference of being diet conscious – marketing research

What do you think about automated sentiment analysis?

Katie Paine has a poll on her blog.

And Andy Beal offers another viewpoint.