Text Analysis: Everything you need to know
In a world saturated by information fuelled by the widespread growth of social media channels and live chat, text analysis can help businesses effectively gain customer insight.
Keeping up with all the latest tweets, support tickets, live chat and product reviews, businesses need effective tools to help them understand their performance and improve the overall impact of the brand.
Manual analysis and understanding of large volumes of text are neither scalable nor practical. Natural Language Processing, more commonly known as NLP, has recently emerged as a popular subfield of Computer Science that deals with the analysis of large amounts of text in the natural (human) language.
Text Analysis has been growing as a popular application of NLP, helping businesses deal with high information overload.
But what is text analysis and how exactly does it help businesses deal with vast quantities of qualitative data?
In this blog we'll address those questions. Going into detail about the advantages and types of text analysis.
If you work in CX, support or product teams, by reading this you’ll get an introduction into the world of Natural Language Processing (NLP) and text analysis.
What is text analysis?
Text analysis is defined as the automatic process of understanding and extracting information from large amounts of unstructured text.
Text analysis can be done by both rule-based systems and machine learning-based (ML) systems.
- Rule-based systems use handcrafted rules to detect linguistic features in the text such as part of speech (POS), stopwords and keywords to understand and organize data;
- ML-based approaches learn patterns/rules from data, where the data contains the set of text samples and their corresponding predictions/tags.
The performance of both these systems is bottlenecked by human effort. In the case of rule-based systems, quality of handcrafted rules limits performance. For ML-based systems, accuracy and reliability of the training data limit performance.
Advantages of automated text analysis
Automated text analysis allows businesses to scale and organize vast information effectively from data sources such as tweets, product reviews, support tickets and brand surveys, thus, helping you to leverage human resources to more important tasks.
Given the scale of the data, manual detection and resolution of urgent queries are hard. Thus, Text analysis models that run 24/7 can be used to detect and flag expressions that indicate a negative sentiment or imply urgency, so that a business can take the necessary action.
Since these models predict based on specific criteria, learnt from the information they are trained on, they are more consistent than humans, as they apply the same criteria for any data that is run through it, unlike us humans, who sometimes let personal emotions and beliefs affect our understanding of the text.
Text analysis process
The Cornerstone of all insights that are of value to any firm which plans to understand their customer sentiment better. Traditionally, firms have relied on human effort and NPS to understand customer sentiment towards their product and business. However, this is neither scalable, as the amount of data is huge, nor is it complete, as NPS scores do not take into account feedback on social media channels like Twitter and Facebook. Sentiment Analysis through NLP and Text Mining techniques addresses these issues making it a viable alternative that is easy on both the pocket and time. An example of that would be:
"Great product features, however, the delivery was really late."
So the system would categorise it as: "Product features" - positive, "Delivery" - negative
Topic-Based Sentiment Analysis
Every company wants to understand the different areas where they perform well, need improvement or perform badly.
Text classifiers can be used to identify topics in product/business reviews, their perceived sentiment among customers and store the data in an organized manner accordingly. For example, if a customer review for a delivery app-based company reads as:
“The app offers good value on delivery”
This review can be categorised as value for money. This structured organisation of data helps companies monitor and improve different areas of their business, independently of each other.
Businesses try to understand the performance of a product through customer feedback. Ideally, it would be convenient if a business is notified immediately when the feedback received is a complaint or needs immediate attention. In such cases, NLP based text classifiers help automatically detect the intent of the text.
For example, if a Facebook page for a company receives a comment:
“I've sent 4 emails and none have been replied to by your customer support”
We see that this comment requires some kind of action immediately in order to keep customer satisfaction up and thus, the comment can be immediately flagged as Action needed: High Priority and the company can be notified.
Aspect Based Sentiment Analysis
Often what might be of value to the firms with multiple products or firms with products that offer a wide range of solutions, is to know what products their customers are talking about in addition to knowing their customer’s sentiment about the same. ABSA has been formulated as an NLP problem and the solution developed for it is faster and cheaper compared to monitoring via human efforts.
“I really hated the table purchase from your website. It did not have a good finish”
Here, the aspect table, which is one of the products the company offers, has been spoken about in a negative sense.
In some scenarios, in addition to analysing the performance of the product, companies might have a requirement to also track, for example, the performance of specific brands registered under them. Custom Pattern Extraction models can be used to extract and track these keywords, thus helping companies in brand monitoring.
“Sheraton always provides the best room service”
For example, if a hotel booking website wants to track the performance of specific hotels (here “Sheraton”) under them, this can be done by keeping track of the specific brand name appearing in the reviews.
Businesses can benefit from using all these types of text analysis to gain customer insights from a variety of customer touchpoints, from NPS reviews to customer support tickets.
These insights can be used as evidence to back up your CX or product decisions, detect opportunities or hidden patterns and improve your product and ultimately increase customer loyalty and reduce churn.