Quantitative vs Qualitative CSAT Analysis

Every CSAT survey provides two data points. The qualitative and quantitative.

‍The quantitative part is the actual “CSAT score”. Which is calculated with a simple calculation:

“To calculate CSAT, take the number of customers who rated you 4 or 5 and divide that by the total number of responses and multiply by 100 to get a percentage. For example, if you surveyed 100 customers, and 45 gave a score of 4 or 5, your CSAT score would be 45/100 = 45%.”

How do you know if you’ve got a “good” CSAT score?

You should be wary of benchmarking against non-comparable companies or industries. Instead, set a baseline with your own customers by surveying them and getting a healthy response rate.

Your goal should be to improve upon your own score, not beat industry averages (which are likely false reporting or bias results).

Qualitative CSAT Analysis

The qualitative part of your CSAT results is the free text box.

This is your goldmine of actionable customer insights. Whether it’s NPS, CSAT, CES, or (our favorite) your support conversations, the qualitative free text box provides the best opportunity to understand customer feedback at scale.

We leveraged AI tagging to analyze the CSAT free-text results for the food delivery company pictured below. It allowed us to:

1. Uncover exactly what topics were driving customer sentiment (positive/negative)
2. Track the impact of improvements over time

This customer service analytics project helped with evidence to get buy-in from across the company.

For example, “Missing Ingredients” was a top driver of negative customer sentiment and a new operational project was implemented to reduce it in the future.

Without AI, this can be hard to do at any meaningful scale, but it is possible. Here are three methods for analyzing CSAT feedback.

How do you improve CSAT? “DO SOMETHING with what you learn from your VoC program!”—Nicholas Zeisler, Zeisler CX Consulting

How to Analyze CSAT Feedback At-Scale

If like most support teams, you’re collecting CSAT surveys continuously at the end of every support contact, you’re likely sitting there with an exponentially growing number of results.

How do you overcome this mountain?

We like to break it down into three methods.

1. Doing it manually (with our template)

Depending on the volume of CSAT survey results you have, you might want to take a handful and go through them manually.

We wrote a guide to manual support ticket analysis here. You can follow a similar methodology for CSAT survey free-text fields.

Here's our CSAT analysis template:

The obvious downside to this method is: it takes a long time and you can’t be comprehensive. 

When new CSAT survey results come in every minute, it’s near impossible to stay on top of things.

2. Keyword-based automations

Another method is using the volume of keywords present to indicate what topics are most talked about. For example, in all your CSAT results with a score of “1” how often is the word “rude” mentioned? What other common words are mentioned? 

The “word cloud” is one way to get a high-level understanding of CSAT. But it’s rudimentary and filled with errors.

Flaws of "keyword" based analytics

Imagine you’re sending the above CSAT survey to every customer and you now have 10,000 answers to the question ‘What’s the main reason for your score?’

If one such response was: “I love the t-shirt I bought, but found it difficult to checkout. But I’m giving you a positive rating because it was delivered in just one day ”. 

A simple keyword approach might see the phrase ‘love the t-shirt’ and stop there. Missing altogether that checkout difficulty was an issue or that next-day delivery drove positivity.

A further downside of the keyword approach is that it relies on you. You must query the data, telling it to find ‘X’ keyword. 

But, what if you are missing something? Or a brand new issue has arisen you don’t know about yet? Then you wouldn’t know what to look for in the results.

3. Machine-learning and sentiment analytics tools

In comparison to a keyword tool, machine learning-based CSAT analytics solutions take an intelligent approach to analysis.

They are real-time, like a keyword tool, but the insight is granular and contextual. Instead of ‘love the t-shirt’ it would categorise the result into:

1. Positive: ‘t-shirt’, ‘fast delivery’
2. Negative: ‘difficult to checkout’

So you’d have a detailed breakdown of your entire CSAT feedback process in real-time.

Machine-learning also looks at your data and identifies new phrases as they happen. If it looks like lots of customers are mentioning a new issue, it will surface that and notify you.

Another great reason to use AI.

Our latest innovation at SentiSum is that we link support tickets to CSAT results. So, if a customer leaves a rating score but doesn’t provide a reason, you’ll have context about why that CSAT result was given. 

Check out our CSAT solutions to learn more.

How to act on CSAT insights 

The feedback is pouring in. You’ve got your analytics setup to manage the data and make sense of it. 

Now it’s time to turn insight into action. And it’s time to get others in your organization to do the same.

In our tipsheet, ‘how to sell the value of customer experience’, we detailed 7 tactics you can use to persuade others to use customer insight.

Here are my favourite three:

1. Build quantitative evidence. Modern businesses are data-driven. If you want to bring focus onto the customer, try quantifying the impact others in the business have on them. Use ticket tag insights to show examples like, "200 more people today had this issue compared to yesterday."
2. Make CX easy to understand. Look at all the lingo we use: NPS, CSAT, QA. We must simplify the value of customer experience. For example, try involving your colleagues more: make customer service part of everyone's onboarding process and schedule monthly sessions where different teams answer contacts.
3. Get the right people on board early. People will back your project ideas if they feel they were instrumental in its inception. Win buy-in for customer-centric projects by inviting executives to take part in scoping and identifying opportunities for improvements in the planning stages

In our research, we’ve also found a few more:

1. Close the feedback loop—service recovery

Feedback that falls below a certain threshold should always be followed up on. You can set up alerts that notify an agent that they should reach out to the customer—which is an opportunity to make things right when your customer has had a bad experience.

GetFeedback recently shared a useful guide to making a service recovery plan:

To plan, ask yourself these questions:

• Who will respond to the customer? 
• Within what time frame?
• In what mode (phone, email, in-person)? 
• What will they say/ask (e.g., apologize, ask for more information to get to the root cause, schedule a follow-up call for more details, etc.)? 
• How will you empower your staff to handle these calls? 
• What information do they need to make the call? 
• What is the intended outcome of the follow-up? 
• When and how does the service recovery get escalated? 
• How will you capture the discussion? Will you share best practices with others to learn from? 
• How will you know if the customer is satisfied with the follow-up?
• How will you know if you’ve saved the customer?

The author also notes that this process can be setup for positive feedback, too. If feedback mentions specific employees or raves about a product, then you can route it to the appropriate person or team to give them kudos.

“It’s not all about the bad experiences; share the good and reinforce the types of actions and behaviours that all employees should engage in.”

2. Strategic alignment: Connect it to the roadmap of existing projects 

Insights and their corresponding stories must be shared across the organization and in such a way that people know what to do with it. 

Insights and resultant recommendations have to get into the hands of the right people who will do something with them. The insights need to be shared with those teams or departments with a vested interest in the specific feedback. 

And you’ve got to get the insight into the hands of your executives, as well. Some of the improvements that need to be made are organization-wide and require C-level involvement to ensure the commitment is there for time, funds, and other resources.

Customer insights help to bring the customer into focus. But you can't just throw insight into the air and hope it's caught.

Find out what's important to other departments right now and share customer insight that's targeted to their current projects–that way they are built into existing action, rather than requiring the bandwidth to implement a new project.

How to Measure Customer Satisfaction - FAQs Answered

What metric should be used to measure customer satisfaction?

Each of these are methods of collecting feedback from your customers, which can be used to determine their ‘health’ levels for your business.

Top of the list is CSAT surveys. This is the industry-wide benchmark for customer service teams and we recommend you focus on that to understand the satisfaction of specific interactions.

What is customer satisfaction?

To avoid confusion and create an industry-wide benchmark was created: the CSAT score.

Nathalie Rey Valdivieso: “A measure of the level of service perceived by the customer when working to resolve an issue with the customer service team.”

CSAT is a transactional survey. Which means they investigate the satisfaction associated with a specific interaction or transaction (for example: when they are sent immediately after a customer support interaction).

CSAT definition:

CSAT is a formal measurement of customer satisfaction. The CSAT surrey is one simple question with a scale of 1 to 5 to measure the answers.(Hotjar)

CSAT calculation:

To calculate CSAT, take the number of customers who rated you 4 or 5 and divide that by the total number of responses and multiply by 100 to get a percentage. For example, if you surveyed 100 customers, and 45 gave a score of 4 or 5, your CSAT score would be 45/100 = 45%.

It’s important to include a free text field at the end of this survey to allow the customer to explain why they gave their score. This is vital for understanding the drivers of satisfaction, although most customers won’t take the time to provide extra context like this, when they do the information is often useful.

What’s a good CSAT response rate?

Although there are anomalies and it differs by industry, the average survey response rate is 10-30%. A survey response rate of 50% or more is considered excellent. 

Common mistakes with CSAT surveys

There are a number of flaws in traditional surveys that you have to watch out for.

‍Related read: Why customer feedback surveys suck

Here are the key flaws in CSAT surveys you should avoid:

1. Survey fatigue

Too many surveys! Companies are realising in quickly rising numbers that their customer’s voice is important to the business, which is good, but it also means that customers are constantly bombarded with surveys.

When you send a survey as a CX professional, you’re already wary of the fact that consumers today are suffering from a severe case of survey fatigue. 

Attention is a precious commodity in the 21st century and with surveys (especially post-COVID) being fired at consumers from all angles, we have to face the reality that most of our survey emails are going to be deleted before they are opened. 

There’s not a huge amount we can do to control this (...even if you do throw in a chance to win an amazon voucher).

2. Siloed insight

One of the greatest challenges facing organisations today is dealing with the rising volume of ‘insights’. 

Not only are companies collecting too many surveys from customers, but they’re doing so without knowing or considering that other departments are doing the same.

When customer service teams collect customer satisfaction surveys, they typically do so to understand how successful an individual customer contact was. Whereas the product research team might run surveys to understand areas they can improve a product or feature.

Either team could benefit from the insight the other collected. But it isn’t effectively shared.

3. No context—actionable insights

It’s not easy to make insight actionable. Mostly, teams will gather insight in the best way they know how, and make an informed decision based on it.

But to make large changes happen, everyone needs to be confident that they will impact customer satisfaction and then retention. To be confident, actionability needs to be at the forefront of our insight collection process.

As we recently wrote, “If we provide managers with actionable insights we empower the contact centre (who now directly contributes to continuous improvement) and the entire company. It creates energy: encouraging agile ways of working and making customer-centricity easy.”—How to make your insights actionable.

So it’s mission critical to make CSAT survey insight actionable—which often lies in how you analyze the collected information.

4. Failing to act on VoC

We hear it time and again from our colleagues in the CX space: ‘stop collecting voice of the customer data if you are not going to use it.’

Yet we see it happening all the time. Either the insights are too high volume to analyze or no one wants to listen.

Luckily, we have answers to both. 

We’ve built technology that analyses large volumes of CSAT survey data in seconds, and have interviewed lots of support leaders about how they get insight heard and acted upon.

Why is measuring customer satisfaction important?

“It serves as an indicator to see if you are meeting or exceeding customer expectations. It can provide insights into processes that are not working or are inefficient and lead to long resolution times for the customers.”—Nathalie Rey Valdivieso

Customer satisfaction measurement is important because it helps you understand what is driving ‘customer satisfaction’. Customer satisfaction itself has direct consequences for business growth: happy customers stay with you for longer, spend more money and spread the word about your business further. 

Who wouldn’t want that?

Here are 4 answers to the question ‘why measure CSAT?’

1. Earn an unbeatable competitive advantage

Katie Stabler hits the nail on the head, “with no end of competition, satisfied customers needs to be one of an organisation's differentiators”.

Especially in industries with little differentiation in price and offering, high customer satisfaction can swerve as a unique selling point and help you stand out from the crowd.

Build your CSAT scores into your value proposition and messaging to acquire new customers at a faster pace.

2. Retain more customers

Staying on top of the drivers of customer satisfaction, and tackling the drivers of dissatisfaction, can help you create stronger customer loyalty and retention.

It has been shown empirically that existing customers are more valuable than new customers. They cost less to serve than newly onboarded customers, and do not require costly marketing campaigns to acquire as they renew each year. In short, retention wins over acquisition from a bottom line growth perspective.

In more transactional companies (like in the eCommerce industry), it has been shown that “91% of consumers who are dissatisfied with your products or services will not come back to your company.”

So a clear benefit of customer satisfaction measurement is to reduce lost customers and improve repeat customers.

Knowledge is power, and the more customer insight you can uncover the more power you yield to better serve your customers.

3. Increase lifetime value

Better customer satisfaction is proven to increase customer lifetime value. 

One example of this comes from our customer, schuh. Schuh has seen improvements in business results by providing more convenient delivery options—i.e. next day delivery improved basket size, conversion rate and net promoter scores. (Read the full article here).

Schuh also introduced a video live chat option into their customer service process. Introducing video chat led to "a propensity to spend more money, an ability to convert that was a multiple of the normal website conversion and being absolutely delighted by the interaction."

These examples show the link between positive customer outcome and revenue, which makes it important to measure whether your customers are satisfied or not.

4. Find areas for improvement (in your product or service)

If you measure the drivers customer satisfaction, you’ll have a clear idea of which processes or product features are causing your customers issues.

There’s often a big gap between what you think customers like and dislike, and what they actually like and dislike. In this gap lies your room for improvement.

For example, an analysis of your CSAT survey results might reveal that your ‘refunds process’ is mentioned positively just 22% of the time in the past week. This shows 78% of the time your customers request a refund, they leave negative feedback. 

This insight could help you prioritise fixing refunds so that they’re faster, easier to get, or even automated (which Amazon has pioneered for some items).

‍
Furthermore, product managers are forever looking for ways to improve their product. Collecting feedback about customer satisfaction from surveys and support tickets is a vital source of information to inform product roadmaps.

A company's success relies on the quality of its products, and their ability to fulfill the needs of customers. Which is why innovative companies are measuring product performance based on the volume of product-related support ticket requests—which are a good way to understand the drivers of customer dissatisfaction.

Alex Tune: “It's simple, customers are likely to return if they are satisfied with their experience. Customer satisfaction is earned, but it takes a short time for it to be lost. It is therefore important that when an issue arises a customer's feelings heard, addressed and resolved accordingly in order to keep this satisfaction.”‍

How to Improve Customer Satisfaction 

There have been a million and one articles written about ‘How to improve customer satisfaction’.

So we turned to our expert panel again to get their unique, experience-led opinions on it.

Their responses:

Katie Stabler: “Genuinely hear what your customer and employees are telling you, then act on it - always act.”

Queen Joseph: “In order to improve customer satisfaction, you have to know where your pain points or trouble areas are first and work from the ground up. It could be improving internal processes or teaching your support reps how to write in a more friendlier tone.”

Nathalie Rey Valdivieso: “When receiving a bad customer satisfaction score it is important to follow up with the customer and understand their reasoning behind their score and see what can be done to make the situation better. Sometimes this can actually lead to a change in satisfaction and a good customer experience overall.”

Alex Tune: “Be engaging with your customers. Remember, we need the customer, they don't necessarily need us.”

Adam Ginty: “Put your heart into it, and be authentic, honest and transparent. Empower your first line to make decisions on behalf of customers, challenge processes and procedures and put customers at the heart of everything that you do.”

The common themes here are:

1. Get a clear understanding of what’s driving customer satisfaction
2. Act on your voice of customer insights
3. Creating a culture that empowers staff, continuously improves, and engages with customers.

We’ve covered understanding the drivers of customer satisfaction already, so let’s focus on how to act on CSAT insights and how to build a culture that values the customer.

Bonus: How does language improve customer satisfaction?

Research suggests that the language used in customer service has a significant impact on custoer experience.

We suggest building a lexicon of words and phrases that your team should use or avoid. That way you can use language to shape how customers and agents perceive the interaction.

Certain words and phrases are negative, like "no problem" which implies there was a problem in the first place and "sorry" we could be perceived in various ways depending on culture.

Furthermore, the words used internally to describe customer service (or the customer themselves) can have a large impact. Language changes perception which changes culture and action.

Read our mini-guide to see some examples of customer service language in action.

Can I do CSAT sentiment analytics in Zendesk?

Yes! Our sentiment analytics integration with Zendesk makes it easy to measure and analyze every survey results, support ticket and conversation in an instant.