Basing important strategic decisions on AI-powered social analytics, at least in part, is wise. But only if the analytics are auditable. Otherwise, you’ll do just as well asking a magic 8-ball for answers or relying on a poorly programmed image recognition app! Here’s how to determine the transparency level of your tool-informed insight. And why it matters so very much. And also, importantly, what any of this has to do with hotdogs.
Hotdog vs Not Hotdog
Sometimes, a thing looks really good on the outside – and lots of people appear impressed by it. But its internal workings are abysmal. And with a little poking and prodding, users inevitably make this unpleasant discovery themselves and switch to a more robust (and ROI-viable) option.
Other times, these tools luck out. Maybe the client figures out the social analytics tool in question isn’t the most accurate option on the block. But its company has already invested heavily in the tool, so they are going to make it work. They’ll eventually switch to something better, of course, but for the time being it will have to do.
These ‘less than accurate’ social analytics options are able to offer surface insights. But it is intel that matters little to a brand’s bottom line. It’s comparable to something touted as a next-level image recognition app that’s only able to identify hotdogs . . . and everything that is not a hotdog. Watch:
Yes, it’s really that blatantly awful once discovered. And having social intelligence output that’s less than “insightful” is dangerous.
The Dangers of Shoddy Social Insight
Imagine your brand is cruising along, with tons of positive social insight, and suddenly sales tank. How can this be? You’ve been checking in daily and your numbers are good – great, even!
Your tool misunderstood sarcasm, and a couple of viral posts later, what you thought was love was actually a brewing PR crisis. One that you could’ve quelled had been aware of it from the onset. And something your social analytics really should have alerted you to!
Or maybe it was just a matter of misclassifying alternative spellings, abbreviations, urban slang or common misspellings as positive or negative – or neutral, when they most certainly were not.
Either way, it’s a headache no brand wants. And one superfast way to sort out which tools to steer clear of (and avoid these unpleasant surprises) is the “Black Box” test.
Black Box Methodology
Black Box Methodology is a devious bit of business. On the outside, it looks amazing, offering lots of intel around your audience’s loves, hates and everything in between. You can see which terms are trending in relation to your brand and can see spikes in mentions, sometimes as they happen.
But when you try to spot check the insight for accuracy (which you should be doing, at the least when it comes to understanding accuracy), you can’t.
So although seeing when fluctuations happen is great, without understanding why these fluctuations occur, you’re left with a bunch of pretty – and meaningless – visualizations. You need context.
Many tools claim transparency when they don’t offer it though. And before you’re familiar with how that actually looks, you may be fooled. As you probably guessed, many analytics tools look pretty slick. There are Word Clouds and line graphs and algorithms at play – and it’s a lot to take in initially.
But, while red and green labeling in Word Clouds is certainly a helpful visualization, it becomes less so if you can’t click through and analyze each word to see what is informing its classification – and to see it in context.
So, if you’re unable to click on insight and explore, down to a granular level – and see what, specifically, is supporting that spike in sentiment or that Word Cloud distinction, you’re dealing with a black box. And with insight that may not even apply to your brand.
Unfortunately, even asking about black box methodology isn’t enough. Those selling it ‘as a service’ are skilled in tap dancing around it. But that’s not the only way you can tell, so don’t worry!
Other Transparency Tests
Beyond tracking insight back to its origin, you need tools that communicate with you honestly about their capabilities. Just because they’re struggling capturing a specific piece of insight, doesn’t mean they’re not worth considering. But when they lie about it, it should be a red flag.
At NetBase, we push product updates every few weeks. We do this to keep client ahead of the connectivity curve. And that’s a high bar for competitors to match. Some of them do pretty okay with a good number of things otherwise though.
Be sure to ask tools about their data sources too. Are they capturing every bit of available insight that your brand needs? Don’t make assumptions there. Not just connecting to Twitter, but all varieties of structured and unstructured data.
Can it all be aggregated and analyzed? Will they help you sort out how to connect it with your own propriety data sets? Get a list of what can/can’t happen and check it against competitors.
And be certain to ask specifically about Converged Media capabilities. This is where brands track owned, partnered and earned efforts. And it’s super important to have a good sense of these metrics for any marketing campaigns.
And a big one to check in on revolves around how quickly these tools adapt to online changes, particularly API updates.
Some Tools Struggle with API Updates
The winter of 2018 will go down in history as the “lost holiday season” for a large number of retailers. They were using tools that did not adjust in time to Instagram’s new API (although there was really ample time to do so). And those businesses lost access to insight during this critical shopping period. Ask any potential tool about that – and Google it too, to check. You’ll be surprised.
As part of this process, we do hope you reach out for a demo of our next generation AI-powered social analytics as well! It’s a wise consideration to have on your shortlist. That is, if you want something guaranteed to take you beyond ‘not hotdog.’ And we’re pretty sure you do!
