Monitoring & Analyzing Social Media

SM2 is not a search engine

We had an interesting conversation here at Techrigy headquarters about our technology with a guest who came by to learn about what we do. And we ran into a common misconception about SM2: That it is some kind of search engine. While it has some search capabilities, search is not its primary function.

Search solves a problem: finding specific answers to queries.

SM2 solves different problems: Finding references to specific terms across social media conversations and content and understanding who is having those conversations, what they’re saying and why. Unlike search, which seeks to supply the best answer(s), SM2 offers up all the possible results and then provides tools for organizing and understanding all of the relevant results.

SM2 has two components. There is a collection system that goes out and collects new social media results on an ongoing basis and stores those results in our Social Media Warehouse. Each result, which might be a blog post, a Tweet, meta data from a YouTube video, etc., is parsed for various data within that result. This data includes public information about the person who created the result such as location, gender, age, etc, any tagging or categorization the user has provided, things like DNS records and IP addresses, URLs, Alexa and Technorati data, etc. Each result can have up to 30 or more data fields in SM2.

The warehouse grows every day. We recently bypassed the 500 million results level and will rapidly hit one billion results as social media participation explodes. Each of those billion results will have multiple data fields which SM2 users can access. Obviously understanding all of this is a challenge. That’s where the second component of SM2 comes in.

The SM2 application front end is a set of tools for discovering conversations and understanding them without having to manually go through them one by one. Set-up gives you the ability to tailor the things you wish to monitor by using keyword phrases, excludes (phrases you do not want to find), whitelists (sources you specifically wish to monitor), etc. Once you run your search, SM2 goes into the warehouse and brings back all the results it finds that match your set-up and analyzes them. After the initial search it continues to bring back results as they are added to the warehouse until you terminate the search.

The analysis tools SM2 provides are extensive, comparable in some ways to web metrics tools like Google Analytics- except that they are very focused on the humans behind the conversations rather than traffic sources and patterns. They look at sentiment, gender, age, location, popularity, trends and themes, categories, etc.

For people who are very used to the search engine model, these differences are a little challenging to grasp. The key lies in understanding how social media differs from the traditional or web 1.0 Internet. Once you grasp that social media is primarily about communication the difference becomes easier to understand.