The Impact of AI on the Post-Sales Customer Experience
At SX Live in Austin last week, the following support and success leaders gathered to discuss the impact of AI on the post-sales customer experience. - Tina Branson Dobie, CCO, Calendly - Jenna Koontz, VP Support, Certinia - Eran Ashkenazi, CCO, SentinelOne - Tim Barnes, VP Financial Services Support, Oracle - [Moderator] Joe Andrews, CMO, SupportLogic Throughout the lively discussion, the panelists shared their insights and experiences with implementing AI in their own organizations. They emphasized the importance of taking a pragmatic, customer-centric approach to AI, and using it to enhance the support experience rather than replace humans.
I first want to welcome everyone to our SX Live event.
A few of you have been with us before,
and we're so happy you could join us again.
My name is Joe Andrews,
and I'm the Chief Marketing Officer,
Support Logic, and the Host and Moderator.
I want to provide some brief context.
Who's familiar with the Gartner hype cycle?
Presumably most of you.
They basically publish a curve
of where emerging technologies are in terms of hype.
This is the peak of hype.
Generative AI is at the very top of that hype cycle.
Now what happens, as we know,
from other technology innovations
is you come through this trough of disillusionment
where realistic expectations are brought in,
and then it sort of normalizes
into a plateau of adoption use.
So it's not hyperbole to say that we are hyped right now
with Generative AI.
In fact, Jenna, who's on our panel,
she was here at this event a year ago,
and we touched on Generative AI very briefly
because it was a brand new topic.
Chat GPT had just launched.
It was brand new.
We were sort of testing it for our personal use,
but there wasn't a lot of focus for real
in the world that we all reside.
That said, right, there is a tremendous opportunity.
This is data that Boston Consulting Group, BCG,
shared in a previous event.
We were actually up in Seattle last week,
and I thought this was incredibly compelling.
14 billion hours per year
that customers spend contacting customer service.
Now that huge number is really hard to visualize.
So I did a quick calculation.
It's like taking every person in the United States.
If they sat down for a full 40 plus hour work week
and they were on the phone with customer service,
that's how much interaction time there would be,
except it's on a global basis.
So there's a ton of opportunity
to improve those interactions and those experiences
that you all, we all, have with our customers
when they're engaged with you as vendors.
Now, the challenge, we talked about the hype cycle,
is that the perception by C staff,
some of us here are in the C suite,
some of us, many of us report to officers of our companies.
And just six months ago, again, this is BCG data,
they reported that more than 50% of the C suite
did not fully understand the implications of JNI,
no surprise, there's an investigation to be done,
and they were discouraging its use.
Now we know about, many years ago was
in the cloud infrastructure space,
and the cloud hype cycle was a protracted event,
and it's still going on.
There's still mainstream cloud adoption challenges going on.
We anticipate this is going to be a long transition,
but we were very pleasantly surprised,
this is data that we just saw from BCG,
where they said, in six months,
the C suite has shifted.
Two in three, now view JNI as the most disruptive technology
in the next five years.
Now again, that sounds hyperbolic,
but one third of these same executives said
they are increasing their investments in AI as a result.
And this survey, you can go see it,
it's their digital acceleration technology index,
and it's a survey I think of 2000 executive,
senior executives at, the largest enterprise companies.
So there's legitimacy here,
it's not hyperbolic to say that there's hype with JNI,
but it's also not hyperbolic to say
that there are real use cases and value and adoption
that are taking place today.
And that's what we're going to drill into.
From our perspective at SupportLogic,
AI impacts every function in the company.
We really started with a focus here
for support executives, managers, team leads,
where you extract signals that might be sitting
in support interactions, unstructured data, right?
A lot of head nods here.
Metadata, pull down menu, severity level type of issue.
Who is the agent assigned?
Well, if you could actually harness
the qualitative information, the sentiment
coming from a customer and be able to push that upstream
to customer success, sales, product and engineering,
be more proactive about fixing the issues
inherent in the products,
you can drive financial metrics, KPIs for the companies,
improved retention, lower operating costs,
who doesn't love that?
So this is becoming very real,
and the benefits of AI are cutting across
every function in the company.
And we believe at SupportLogic, as do many other companies,
now I work in marketing for an AI company.
So I'm prone to say this, right?
But we believe just like 10 years ago,
we would say every company will be a cloud company.
Today, that's not a surprise.
Back then, people shook their heads a little.
Now we believe every company will be an AI company.
Every company will incorporate AI into what they build,
what they deliver, how they serve their customers,
which is why all of us are here today.
So just a couple more slides,
and then we'll bring up our panel.
Where do you start is the question?
So there was other data from BCG,
we just actually published a blog on this,
where approximately 40% were sort of just getting started
or not getting started with AI,
10% are fast and furiously running towards it scale.
And then the big mass in the middle, like 50%,
we're sort of tinkering with it, proof of concept.
Where are folks like in the room,
who is actively driving AI solutions today?
Wow, that's two thirds of the room roughly.
I mean, Jenna, year ago, right?
Very different response, scratching the surface.
Who is sort of tinkering with it?
Testing it out, okay, a few.
Who, it's okay to admit it hasn't started yet,
but has good intentions, okay?
So there's a mix in the room.
We're gonna have a mix of perspectives on our panel as well.
But what BCG said is that there's kind of four areas
where you can start.
The area on the right, number four,
is within the actual support response itself.
It's more tactical, but it's more immediately
realizable in terms of value,
enabling support teams, agents to improve
the customer experience in the moment,
do right by the customer, resolve issues,
better, faster, cheaper, right?
And then you go over to the left,
where you have self-help, enabling customers
to get the resolution that they seek by themselves.
Self-service, a lot of companies for many years in support,
because support was very cost-centered focused,
were really focused on self-service to deflect cases.
But we know if many of us are in complex technology companies,
the element of the human will never go away.
You can push some of the easier to resolve cases
down to self-service, but you wanna still focus
on delivering the best service.
So this is an area of investment.
Self-healing, being able to preemptively improve
and resolve the issue for customers
before they even realize it's an issue,
wouldn't that be amazing?
And with very little effort.
And then finally, the preemptive,
being able to fix product issues proactively
before anyone notices them.
I think some automobiles and maybe some other high priced
products do that to some degree.
But you can imagine this is the hardest area, right?
So most companies are starting here
with the support response space,
being able to improve the life in the moment
for the customer and improve operational efficiency
while you're at it.
So I'm not gonna present to this slide.
It's an I chart, but there are,
again, this is a BCG slide.
They have a lot of amazing content on this space.
And if you look across these four areas,
there are, I think, 36 or 38 different use cases
where they are starting to see value
where companies are actually investing in AI
for the post-sale customer experience.
And what do we see?
This is a support logic slide.
And we see that, again, this is also an I chart.
This is on our website.
But we see three kind of core areas.
Agent assist or agent productivity.
We see quality monitoring and coaching,
being able to auto-QA every single case
in customer interaction and provide feedback
and coaching against a program rubric.
Wouldn't that be fantastic?
And then core support operations efficiency.
Being able to understand customer sentiment,
reduce escalations, intelligently assign the case
to the right support engineer or agent,
be able to better more efficiently route cases,
alert people in the organization,
bring them in to swarm to resolve customer issues.
This is kind of a big area of investment.
So these are kind of the things
that we're going to start to dive into as a panel.
And really would love to make this conversational
and continue this throughout the evening.
And these are some of the support logic customers
that are seeing incredible results using AI.
In the customer experience camp,
you have significant reduction in escalations,
which has a very real operational cost,
but also improves the customer experience.
You have retention and the CFOs love these metrics.
Being able to show we were able to preserve these renewals.
We were able to ensure that customers don't churn.
And then on the operational efficiency,
being able to reduce the time to resolution,
being able to shrink the number of cases,
get support manager productivity time back in the day
so that companies can redeploy those valuable resources
into things that are more proactive
to improve the customer experience.
And I see a lot of heads nodding.
And this is sort of the panacea.
And this is where the AI is really delivering value.
And we would love to have more conversations with you all.
And so I'm going to call everyone up.
We'll start at the bottom here with Tim.
You want to take your place Tim Barnes,
VP Financial Services Support at Oracle.
Next, we'll call up Jenna Coons,
who's the VP of Global Technical Support at Sertinia,
Iran Ashkenazi, the Chief Customer Officer at Sentinel-1.
And then Tina Dobie,
who's the Chief Customer Officer at Calendly.
So we are going to dive into these topics
and engage the audience, bring your best questions.
But let's start with a quick introduction
and an icebreaker.
So Tina, would love, welcome.
Thank you for joining.
I'd love to hear a little bit about your role at Calendly
and what you're focused on,
and maybe a fun fact about yourself.
- Okay.
Hi everyone.
Thank you for having me.
Appreciate it.
Y'all should get started.
It's like a family meal.
- Yeah.
(audience laughs)
- So I'm Tina Dobie.
I am the Chief Customer Officer at Calendly.
I see a familiar face over there.
Hello.
And I've been with the company for about two and a half years.
I was previously at a local company called WP Engine.
Chief Customer Officer there for seven plus years.
So lots of experience working with customers and teams.
I'm responsible for all of the post sales teams,
everything from 24/7 customer support
to customer success for our large enterprise customers.
We have an implementation on-boarding team.
I'm an operations team that manages all of our tech stack
and analytics as well as customer education and training
and enablement that makes everything happen at scale.
So that's a little bit about me.
Has everyone heard of Calendly?
Does everyone know what Calendly is?
Yeah, okay.
Lots of good nods.
That's great.
- You use it for your bookie.
- Good.
Glad to know that.
So fun fact about me.
Hmm.
We were talking about this
and wondering what we were gonna say.
So my husband and I are empty nesters.
We're currently trying to sell our house
and when that happens, we are hoping to be homeless
for about a year and just travel around
and do Airbnb's in different cities
for about a month at a time.
So wish me luck.
The housing market in Austin is not very good right now.
- Yeah, all right.
- All right.
- All right.
- Hi everyone.
My name is Aaron Ashkenazi.
I'm a chief customer officer at Sentinel-1.
Probably less familiar than Calendly.
Sentinel-1 is a cybersecurity company.
Who knows Sentinel-1, by the way?
- Yeah, I think we have even at least one or two customers
in the audience.
Cybersecurity company that was founded about 11 years ago.
I've been there for 10 years.
So I'm in employee number 12.
I've been running post sales in once, you know,
some shape or form since I started.
I didn't have anyone reporting to me the first year.
There's a team of over 430 people today.
So everything really ranging from support to success,
but also managed services, high consulting, training,
you know, professional services.
A lot of different functions there.
Global team obviously.
The company kind of focuses on, you know,
B2B, enterprise, cybersecurity,
input protection, cloud protection.
You talked about cloud journey.
We've had a lot of fun during the pandemic grew significantly
and very fast and went through an IPO process
about three years ago.
So we're listed on NICEI if you wanna check it out.
And a fun fact, I learned to ski when I was 40
and I'm not really good at it.
So, yeah.
- Hello everyone, I'm Jenna Kuntz.
I am the VP of our global technical support organization
at Sertinia, formerly known as Financial Force.
Curious if anyone has ever heard of that company.
- Perfect.
So Sertinia is built on the Salesforce platform.
So much of what I'll talk about today
will be sort of in that lens of the ecosystem.
Is any curious if there's anybody
that also is a Salesforce customer
or maybe even your own application is on the platform?
Okay, perfect, wonderful.
So I've been with Sertinia about three years now.
I've always been part of a support organization.
Kind of made my way through Oracle
and other Motorola back in the day here in Austin.
That was 25 years ago.
Motorola is not here any longer,
which gives you some indication
of how long I've been around.
So, but as part of my organization structure at Sertinia,
we have sort of life cycle of community
as part of my remit, install deployment like TINA.
Frontline support, we're not tiered,
so we're swarming in our company.
But if you were to look at it as a tier,
tier one, tier two, tier three.
And then we have sort of our team
that kind of parachutes in when there's challenges,
as well as support the customer success organization
with things like adoption.
That's our technical success organization
and a dedicated escalation team.
So that is, that's my part at Sertinia.
Fun fact, gosh, I am struggling with that,
but I guess I would say I'm originally from Boston.
I'm a huge Patriots fan.
I don't know if that's fun, but it's definitely a fact.
So there you go.
(audience laughs)
- And maybe controversial.
- Yeah.
- By the way, that's mine too.
Originally Boston.
- Hi, I'm Tim Barnes.
I'm Vice President of Financial Services Support at Oracle.
I've been there for 20, I don't know how many years.
I didn't plan on it when I went there.
I came up through the Austin tech scene
back in the kind of the turn of 2000.
And we're responsible for risk and finance products,
crime and compliance, revenue insurance products,
things like that.
We have a global team in probably more countries
than I could rattle off right now.
I came up through development.
I've only been with support for maybe nine years or so.
So I feel like sometimes it holds me back,
but also a different perspective
from being working with our partners
over on the development side.
So fun fact about me.
And I came prepared for anybody that wants to nerd out.
I was at one of the first Java one conferences,
so this is a Java ring from 1998.
And it helped me buy a cup of coffee.
And that's all I did.
But it's stainless steel and it looks cool
and I still keep it in my drawer.
So that's my fun fact.
That's awesome.
Yeah, thank you everyone for joining.
So I want to start with the AI topic.
That's front and center.
It's funny because last year it was the last topic.
Now it's the first.
It's been quite a year.
We talked a little bit, show of hands here
indicates that two thirds or so are in the process
or actively using it.
So let's talk about where do we see
the benefits in the post sale customer experience.
I mean, we'll start with Jenna,
because we've been on the journey with you
for a good year or so.
And we'd love to hear your perspective on that.
And then everyone else, like where do you see use cases?
Where do you see the biggest opportunities?
Where are your organizations actively investing?
Yeah, great question.
So I think obviously within a support organization
it's all around scale efficiency.
And those are the obvious answers.
And I think as we've been quite thoughtful on our approach
to AI, we actually coined it the pragmatic AI approach
internally, both with our own product and our usage of AI,
but also as we're looking at tools and evaluating tools.
So I think there's a multitude of business cases.
Joe showed a really interesting slide,
I think, on some areas.
But for me, the focus for me is really around,
how do I build an experience and elevate
that experience for our customers
and capitalize on AI doing it?
So I think there's loads of opportunity here.
So I run a global organization.
And I also have an offshore team.
And sometimes there are language barriers.
And sometimes there are challenges
just bringing that continuity in an experience
that you want as a support leader.
So I think as we're evaluating tools and assessing,
we want that human touch.
We want to lead with empathy.
So the irony of using AI to bolster
some of those situations is kind of interesting to me.
But also, yeah, that's definitely a focus area for us
this year in terms of investments
that we're looking to make.
Also for our Knowledge Center Services Knowledge articles,
a brilliant area to also look at
in terms of the efficiency and productivity and consistency
you can get with AI building your knowledge articles
and your Knowledge Center Services and Knowledge Base.
Same with community, there's opportunities there
as customers enter through your stream
and the experience that you're offering right
from the get go with self-serve versus assisted.
So I think, yeah, I could keep going,
but I'll pass the mic.
>> Thanks, John.
How about 10?
>> So yeah, almost end to end, the entire process,
there's opportunities here from the time.
And I know you talked about some of the kind of self-heal
before the issues even happen, but issues will happen.
That's why jobs.
So when they do happen from the time the tickets logged,
how we're interacting with the customer
to gather as much information as possible,
make sure that data is as structured as possible
to get to the agent, to where they can then look at that
and quickly make a decision, or even cross-reference.
You talked about Knowledge Base,
cross-reference other areas that might be similar.
I don't think we've gotten to the point
to where we're so confident in the solution
that we would say, in some rare cases where we would say,
"Here's the solution."
And matter of fact, I'm so confident
close the service requests, right?
Especially with some of our customers
that are doing things like regulatory compliance
or crime and fraud.
We wanna be sure, and that requires human interaction.
And that sort of human interaction that you said,
it's never gonna go away.
We don't expect it to.
To take every step along the process
and make the agent's life easier.
And then when they are interacting with the customer
to make sure it's as high value as possible.
That's what we're really looking for.
We know that they've been sold value, product value.
And now you move to the services side
and you want them to see the value of the service as well.
So I think every point along,
we've identified some sort of opportunity.
And then it just becomes, what do you do first?
How are we gonna build it?
How are we gonna test it?
Do we trust it?
And trying to collaborate with all the different lines
of business to make sure you're doing it the right way.
>> Well said.
Thanks, Tim.
Tina, how about you?
>> Sure.
So I think about these AI technologies
in kind of two separate ways.
And one is really thinking about the internal team's usage
of the tools in order to drive kind of an agent assist
experience such that there's technology right there
that can help them with a customer in a live chat
in the moment.
And our goal is to basically serve our customers
via live chat for as greater than 80%
of all of our interactions.
So having an agent assist tool right there in the moment
is really, really key for us and important.
And I can talk more about it later,
but that's kind of one area.
And then the other area that we have been experimenting with
and have not yet nailed what we're looking for
is in the customer's chatbot experience.
So if you think about I'm a customer,
I come to the website or I'm in the application
and I'm looking for help,
the first place you're gonna go to
because you're authenticated and we know who you are
is you're gonna go to a chatbot.
And that chatbot is going to help you with your questions
and provide you and route you to a level of service
that we deem as the right path based on who you are.
And so we have been testing generative AI
through that chatbot extensively.
And I think it's also okay to say that we haven't found
we haven't found the solution that we're looking for yet.
Our declarative answers,
which are the answers we write for the chatbot,
have about a 92% deflection rate at this point
or a success rate.
And the generative AI solutions that we've been testing
to have not been able to cross that bar yet.
So I also think you have to kind of think about it
in terms of what kind of solution are you offering
if you're a regular e-commerce,
e-commerce product or solution,
that's a lot easier to train a chatbot on
than it is to train on a more technical type of solution.
So I kind of think of those two areas.
There's obviously tons of other,
other areas like translations and all kinds of things.
But I think of it in terms of how we're helping our customers
and what I want to use generative AI for
to be that front door of support, not quite there yet.
And then also the agent assist for internal usage,
which we're having really good luck with.
>> Good to hear.
>> Or on speaking of more technical products.
>> Yeah, actually, I like what Tina said about
like looking at it internally versus externally
because that's actually what we're doing as well.
We have a very complicated product.
It's like B2B operating systems.
Some of it is like kernel based, malware,
all sorts of things that are happening over there.
And it's very, very technical, you know,
in terms of like the support infrastructure,
very different than like kind of a pure SaaS product
because there's a hybrid deployment that includes an agent
in most cases.
And so, you know, we're, honestly,
everywhere I look, I see opportunity.
And so if you go back to that hype kind of chart
that you showed, I think there's so much hype
and we're like trying to figure out really
where do we want to put our eggs in which basket
because there's opportunities here all over the place.
We are experimenting with,
we started with an initial AI based tool
which actually is kind of like connected
to our intranet to some degree.
And that gives you the ability to ask simple questions
and get things like from different knowledge base,
internal knowledge base and/or Slack channels
and other, you know,
powerport presentations and things like that.
And then we're like,
maybe we can do more with that.
And we're starting to look into other areas
primarily because we still don't trust
the hallucination factor, right, of the AI
and the fact of the accuracy that you were talking about.
So we're starting internally for it.
Like let's see what we can do for the agent itself
to help them be a little bit more productive.
The problem there is that it's hard to really understand
how much more productive are you actually making that agent.
But then really the next step is looking at
the AI chatbot, externally chatbot.
We actually did not have any chat as a medium so far
just because of that.
Like we just couldn't really get,
I would say good outcomes from like the standard,
you know, templated responses.
But that might be an opportunity for us.
And then we're looking at so many other areas
like from, you know, churn prediction and mitigation
to like understanding where there's up
so potential with customers and so on.
So it's really like all over the place.
And now I just, I have a relatively small op-steam
that needs to work with IT and I need to like prioritize
and pick and choose, okay, what am I investing in right now?
Let's start with what I feel as the most ROI
and then maybe a call out for the vendors.
You know, when vendors talk to me, I say,
you know, they tell me, do you have like a budget for an AI tool?
And I'll be like, no, I don't actually.
But if you can prove to me, because I have a team
of hundreds of people, you can prove to me
that you can save me, you know, the effort of 10 people
while improving some customer experience,
then yeah, it's easy for me.
I mean, I'll write the check tomorrow.
I change my AOP and I put, you know, from, you know,
from the FTEs I put it in the Opics.
So.
- Good tip for everyone for that.
So let's stay with you, Aaron.
Let's flip the question over and talk about
what's getting in the way.
We've talked a lot about the opportunities
for AI.
There's a lot of them.
But there is still some hype
and there is still some resistance internally.
So what are the challenges you're facing
that are preventing you from moving from us?
- Yeah, well, it's, you know, one thing
that one of my VP sent me was like,
remember there was like, there's an app for that, you know,
like the commercials of Apple.
So apparently there's a website that says,
there's an AI tool for that.
So you write like what you're looking to do.
And apparently there's like, you know,
a couple of like a thousand plus AI tools already there.
I think that one of the problems, like, where do I go?
Like, you know, which tool do I actually invest in?
A lot of the tools are actually promising a lot more
than what they can actually deliver.
And I think that to me, that's like a first problem.
Second thing is that we're already inside a year,
which probably we did not plan for an AI project.
Yet we realized probably in the last couple of months
or I don't know, three to six months that we needed.
And now we need to figure out how we're gonna do it.
That's why I talked about kind of the conversion
between ROI, like instead of investing more in FTEs,
actually getting that from a tool.
I do not have any issue actually
from like executive leadership.
In fact, it's my CEO that's like pushing the entire company
GoAI.
And I have to say one thing about it is that
we are an AI company to some degree in cyber security space.
You know, it was called machine learning.
We didn't know, you know, that, you know,
mature into an AI models eventually.
And also we integrated our own, you know,
we call it an AI analyst or a co-pilot.
We call it purple.
It's basically a capability that's gen AI within our console
that allows an analyst to do their job
without really knowing that much.
Like they could do a threat hunting, for example,
without knowing how to write a query, you know.
And so we're actually going to use some of that technology
into the use case of support,
because in my mind, one of the things that we need to do
is to bring things as close as possible to the customer.
Like if we can, if the chatbot resides within,
you know, the first thing that the customer sees,
maybe even before the community, maybe before the agent,
that's a win, right?
So, but there's a lot of hurdles.
So I'm like, we're still in experimenting, right?
Katja, who wants to add to that?
Like I guess, one of the challenges I think
is the complexity of the space.
I don't think, you know, we fully understand it.
I don't think maybe if we're talking to our peers
that they fully understand it.
And so that hype curve that you showed,
it kind of works to our disadvantage,
because people say, "Well, wait until it's gone past the hype."
But some of it is in many things, it is the name, right?
AI itself makes people think of very complex.
But if you take away the AI and you say,
"I want to automate things.
"I want to industrialize things."
Things we've all been doing for a long time.
And now I want to do it with Gen AI,
or I want to tack on some additional things.
It becomes much more palatable than just saying,
"There's this huge thing that only data scientists
"and PhDs understand."
And I think that would get us past some of those hurdles.
In full disclosure, in the first few years
of the support logic business,
we didn't really tout the AI.
It was just the technology we used to solve problems.
And then really in the last year with this tailwind,
we said, "We kind of have to."
If we don't, we're crazy.
So we sort of jumped on that bandwagon as well,
and to sum it to hype cycle, but so on and so forth.
Tina, has it gotten easier or harder?
You saw the trend data from the last six months
to get your peers in senior leadership on board?
- No, it's not hard.
- No.
It's not hard.
I think we have a lot of people at the company
who are kind of chomping at the bit
and really excited about new technologies.
We're an innovative company.
We track folks who really care about innovation,
and it's kind of, you don't want to call it
the shiny new toy, but it is to a certain extent,
and we've been out evaluating just a number of technologies
and have done proof of concepts
with a number of different companies.
And what I would say is,
like to go back to kind of the barrier question,
don't expect smashing success out of the gate, right?
I would say, like in our case,
we're in a proof of concept right now
with a specific technology,
and we've been able to prove in just a few short weeks
that we can get in a minute,
we can save a minute and a half on every, on average
for every live chat that we do,
and our average live chats are about 16 minutes,
so you can do the math, and the next thing you know,
that adds up when you're doing 30 to 50,000 support requests
in a month, right?
So have realistic expectations, start small,
get a small win, and then it multiplies it as a,
like a multiplier effect,
and that's just for one particular sliver of age and assist,
and so if you think there's way more you can do with it,
and have that mindset, then it's not hard
to get people excited about it,
and you know, there are a lot of companies out there right now
who are happy to do proof of concepts
because the technology is new, and they wanna prove it,
and they want customer testimonials in order to do it,
so you know, as leaders of post-sale support organizations,
use that to your advantage.
- That's a good tip.
Jenna, with the journey you've had over the past year plus,
what are some of the barriers you're still facing?
- I'd say one of the areas that you really wanna consider
is organizational change management with AI,
and really any technology, but I recently had to,
was asked to speak to a company town hall,
and the topic was of course AI,
because that is the topic du jour,
and I didn't consider the impact to my organization,
and maybe even like a support analyst,
listening to me as a leader speak about how we were looking
at it and how we were approaching it,
and a lot of the survey feedback that we got on that
all hands was, "Is my job going away?"
And so we really have to reframe those things,
and just talk about it with intention,
and what it really means for you and your organization.
For us it is not about, it's not a head count reduction exercise,
it is truly about creating opportunities and pockets
for our people to do more high value work,
and again elevating that support experience for our customers,
so that we can free them up from the mundane,
and create those efficiencies and those pockets of opportunity.
So I actually ended up having an all hands,
just try to walk the team through this to say,
"Hey, look, we are pursuing these,
like every other company,
and oh by the way, our own company is employing AI,
and it brought it to market a couple of years ago,
so it's something that you really want to consider.
For those early in the journey,
just think about how your teams may respond to this,
because people will have different feelings about it along the way.
There's the hype and the noise,
but then there's other people that are fearful.
>> Well said, shifting gears a bit,
you mentioned a phrase that is near and dear to my heart,
support experience,
and the fact that we have two support leaders,
and then two chief customer officers here on the panel
is not a coincidence,
we want to get a slightly different viewpoints,
and then talk about the collaborative opportunities,
but first Jenna, let's keep it with you.
What does support experience mean to you and your organization?
>> So I wake up thinking about how our customers
and how they're treated, the technology solutions
that we're bringing to them, how quick we were,
the quality, both from a technical standpoint
and from an engagement standpoint,
what that looks like, holistically,
and the entire journey.
You really have to kind of map that journey
so that, by way of example, we were on a call
about the whole leadership team today together
to talk through our case submission form,
something that simple, but really it's the first experience
your customers are seeing,
and they're making impressions of you right then and there.
So we're looking at it, we're walking down it,
and the team had a lot of really interesting observations,
and quite frankly, a few of them have never even seen
the case submission form, so I was just shocked,
and I thought, this is the kind of stuff,
those details matter, what does the survey say
when it goes out, in that whole entire journey,
you really need to really just look at it.
And then, I think you said end-to-end,
and I agree with that.
>> Tim, what are your thoughts on support experience?
>> Yeah, I mean, I think it comes back to providing
absolutely the best interactions that you can.
We have some really high performers on the team,
and they really want to provide the best service possible,
and so we do want to take those more mundane tasks
and give them the time to provide the best service.
We want our customers to feel like we're their partners,
and they have goals.
Every one of our customers has a different set of goals
that they want to provide to their customers,
they can only do it.
We feel with our products,
and so we want to help them to achieve that.
And the other thing I think is that
anyone that's in the services area,
you really want your customers to become your advocates,
to become your champions.
And I heard someone say that
certainly if someone's going to be a reference for you,
maybe sometimes you have to twist their arm a little bit
in order to be a reference for you,
but they're your champion.
They're going to talk about you when you're not in front of them.
And those are the kind of customers that we want to build,
and we're the ones that are interacting with them
on a very frequent basis after they bought the product.
So that's what we have to strive for.
>> Good point, I saw a stat that said that in B2B,
especially complex, B2B,
more than half of customer interactions
occur through support.
So that your function really is front and center
in many cases and responsible for delivering
an amazing experience.
Let's hear from the chief customer officers in the room.
We've had a decade, we were talking earlier
about your path, customer success,
and then professional services,
but there's been this convergence and collaboration
between customer success support
in really delivering a more holistic, unified customer experience,
post-sale customer experience.
From your perspective, what's working well,
and maybe what are some opportunities for,
what are opportunities for improving the collaboration
between support and success?
Tina, do you want to start?
>> Yeah, so when I think about this,
I think about, you know,
I'm responsible for all the post-sales teams,
and so rather than thinking about how support supports
their customer and customer success supports the customer
and community, which is also under my remit,
you know, the community supports the customer
or in the help center, I don't think about that way.
I think of it in terms of we have a customer
and we need to think about their experience
and where they get help from in the moment doesn't matter.
They just want their problem or their question solved.
And so if you put the customer view first,
then you're going to make decisions
about how the teams support that customer
as opposed to the teams that are coming first.
And so by that I would just mean we architect, you know,
journeys for the customer that we try to make it
as effortless as possible.
So an example of this is we want to make sure
that our teams that are frontline
have the information they need
that will be helpful in the moment.
So for example, if I'm a support agent,
I know because of the integrations
that we've been able to create through our tech stack,
I know if that customer is in there a first 14 days,
which for us is really important
because that means they're on a trial
and we're trying to convert them to a paid plan, right?
If there are a customer that's been with us for two years,
we have, you know, segmentation that we put in place
based on product usage data that tells us
and they put them into a profile of this customer's
at risk for these reasons and, you know,
we have analytics behind it.
And so the agent knows exactly what playbook they need
to run and what questions they need to ask
based on those profiles.
And so we're putting data into the agent's hands.
You know, conversely in the community, for example,
we know when somebody goes to the community
and then we know when they go to support
and then the CSM, the customer success manager,
can see and gain side, for example,
that they had a support interaction yesterday
about this topic and they went to the community
earlier today about that topic.
And so we get kind of this 360 degree view.
And so again, it's about the customer first
and our protecting that journey
and making sure that all of the data is integrated
amongst all of the different technology solutions
that we have so that we can see what's going on
and it's not just, I'm in the community
and I only talked, you know,
I only just done what's happening
with my customers in the community
and I only do support and so I only know what they're up to.
We just try to make it more on the channel.
- Mm-hmm, that's fantastic.
Or with your perspective?
- Yeah, I mean, obviously, I mean,
echoing a lot of what Tina said,
but also I think that there's the realities that,
you know, the groups actually a lot of times
operate within silos.
They all have the best intentions,
but we know also that the road to hell
is paved with that, right?
So we have to make sure that, you know,
and I think this is where technology comes to play,
you know, that technology helps basically
connect those groups together.
And I think that, you know, a lot of times
you see situations and when you think about
a customer journey, there's actually this pyramid
that I constructed and I like to show it
almost in every like sales kick off
and customer event that we do,
which is a take on the muscle pyramid of needs.
So the muscle pyramid of needs, you know,
you start with the basics and you go, you know,
all the way up to like, I don't know,
feeling accomplished, et cetera.
And I took that and I put it in a CS mindset
and I said, well, it's start with,
keep the business running or like,
I like to say, don't break my shit.
So like, if I install a piece of software,
I wanna make sure that it doesn't really ruin anything else.
That's the first thing.
The second thing is that it needs to work,
like do what it's supposed to do.
Like, I don't know, set meetings
or protect my endpoint or whatever, right?
And then you go into like the Assistant and advice piece,
which is like, all the support pieces,
the success, the professional services,
and eventually you get to that end of the pyramid
to meet the delight piece.
That's where the customers becomes your fans.
That's where they start talking about you.
That's where they come to panels, right?
And like, you know, talk about their experience, I guess.
Jenna had a good experience with you guys, right?
So I think that like, I'm trying to convey that,
not just by the way to my teams,
I'm trying to convey that to the rest of the organization.
I go with that to sales.
That's like a pitch I talk to the Salesforce as well.
Because to me, customer experience is, of course,
it's everything we do within my org,
but it's also like the way that
the quoting process looks like,
or it's the way that the sellers work with the customer,
how they interact in case of a escalation, for example.
Instead of like being an amplifier,
they work together with the team and we're one team.
We're not like sales here and this there
and whatever and product and engineering.
So, you know, I'm trying to bring all of that together.
I'm not saying that it's all successful.
I'm saying that that's the vision
and that's where we're aiming in terms of a target.
- Great, thank you for sharing that.
I wanna go back to Tina for one question.
You mentioned it earlier.
Being a product-led growth company,
you talked a little bit about the playbooks
and for helping get customers successful early,
especially I think you mentioned a 14-day trial period.
What are some of the other challenges
that you're facing in terms of that,
delivering that amazing experience early on
being a PLG company?
- Yeah, that's a good question.
So, PLG basically means product-led growth
and it's where the philosophy is that customers
should be able to self-serve basically everything.
And I think there's this healthy tension
between we as a support organization,
we quantifiably know that if we're able to help a customer
in the first 14 days, actually first 30 days,
that we have a conversion rate that is significantly higher
so they convert to a paid plan that's significantly higher
than if they don't talk to support at all.
And the challenge there is that we have
like 1% of customers in their first 30 days
actually talk to support, right?
So, it's a healthy tension between
you wanna be a self-surface product
and at the same time, we know if we can talk to them,
we can get them to convert, right?
And so, I think that's probably the healthiest tension
is it's like, do you wanna have a call to action
in your application to contact support
when you're in your first 30 days or 14 days?
And then do you wanna give other resources after that?
So, it's like this healthy,
I think that's probably the hardest thing to--
- So, you're literally paying off what Tim was saying earlier
where the support experience is actually benefiting
the customer's experience and the conversion
and retention of that customer.
- We have, yeah.
We can quantify the impact.
- That's fantastic.
One more question for you all
and then I wanna open it up for some audience questions.
AI and all this conversation around new technologies
brings the opportunity for skill development.
You all have a diverse set of teams within your functions.
How are you all thinking about training for this new era?
One of the questions we were talking about earlier
is do you hire more generalists or specialists?
Is the role of all being to just become
essentially prompters for the AI?
Or are we thinking about it too simplistically?
How do you think about skill development for your teams?
Tim, do you wanna start?
- Sure.
I think the team that we have right now
that's focused on automation,
which will expand out to be more AI or J&AI
or some of these other tools,
those are gonna be our specialists.
And, but I do think that anyone that's using these tools,
even if it's an internal chat bot
that's just a natural language model,
which is giving suggestions,
either in solutions or helping you look up things,
unified search and things like that or topic clustering.
There's a variety of things you can do internally,
but we can't just say, here's the tool, use it.
They need to have a base understanding.
I think when Cloud, you mentioned Cloud having this explosion
years ago, and you start to then look more
when you're hiring people or when you're training people,
you say I'd like to have somebody
that at least has that base understanding.
You hear it on the sales side all the time,
selling on-prem and selling Cloud very different.
You have to have a base understanding
before you can move forward.
And I think support's gonna be no different.
We have to have everyone trained at a generous level
that's gonna be using these tools,
but the people that are building them
and trying to convince others to invest more in them,
they have to be our specialists.
It's application and special depth.
Jenna?
>> Yeah, same.
I mean, great, great response.
I definitely think it's a hybrid model.
And I definitely think that as you consider tools
that you're introducing, how are you,
we stumbled a little bit when we implemented
support logic in terms of our adoption of the tool
and our preparedness for teaching the tool.
So with support logic sort of predictive,
analysis of, hey, this looks like it's likely to escalate.
You're offered an opportunity to say,
yes, that's true or no, it's not.
And we were a little slow on the uptake,
but I think we've got it, we're cooking on gas now
because it is almost like the accuracy of it is scary good.
So over time, we were able to kind of build
that generalist community that's coming in
and telling it, yes, this is real, this is not.
And our touch points on that are lessening.
But to your point, when I look to the future
with things like co-pilot, yeah,
you absolutely need specialists
for that type of advanced technology.
- Just tangential follow up to that.
What would be helpful to make you get
over that learning curve faster?
- I think it, gosh, that's a great question.
I think we owned our part in that.
I think it was more our readiness and our ability
to really drive it a little quicker.
But it was, once we began to,
it came together pretty quickly, relatively speaking,
it was just a bit of a stumble at the onset.
- I actually think that, just going back to the example
of cloud and on-prem, I think that once this technology
actually really ripens, then it actually should be
that complicated because when I connect to data
service, I don't really know if it's on-prem or cloud.
I mean, there's the saying that cloud
is just someone else's computer, right?
So I think that if you think about it that way,
then eventually it will be very simple and intuitive,
but in that interim, which could take a year or three,
or I don't know, whatever, but we will need to have,
obviously, the data scientists, AI expert types
that are going to do a lot of the training to the models
and kind of fine-tuning, and then you need to make sure
that the general population is actually,
the adoption there is in such a way that they understand
that they have a part in it.
Like, for example, just in a supervised model
when you say yes or nay, right?
Like, is it working for me or not?
Just that thing alone goes back to the data scientists
and go back to the AI team that changes it,
but down the road, it would get to a point where like,
it kind of just works.
It will take us some time, I don't know how much time,
but like, that's my view.
- Fantastic.
- Yeah, I think in the here and now,
the way that we've implemented kind of our agent assist,
that we're doing right now, I think of it kind of even
as a journey within agent assist.
So for example, what we're doing right now is you have,
and this goes to kind of more of the specialist,
I'm sorry, the generalist topic,
but the first kind of step in agent assist
is having the technology do the transitions
between the different topics a customer is asking about.
So it does a salutation, does a greeting,
and then based on how the sentiment comes back
from the customer, it prompts for,
let me understand your question in a kind of,
so it guides the agent not in the answer,
but in what to say next,
from like a greeting and sentiment type of thing.
And that's kind of the first evolution, right?
The second evolution is really thinking about
what's called like custom intense.
So why did a customer reach out to you
for support in the first place?
And so we've been able to map all the reasons
why customers reach out to us with all of our helps
and our content and a tool we have called Guru,
which is an internal kind of knowledge base,
map those together, and then from that,
the technology is surfacing up answers
for the agent to then say, yes, this is the answer,
but you can't, as an agent, you have to know the product,
you have to have been trained on the product
because then you become QA and you become,
you know, you free up your time to be able to work
on cooler, bigger type projects
and spend more time training and up level in your skills.
And so I think that, you know, yes,
the evolution will be more towards specialist in the future,
but at this point, the technology is all about
helping a generalist be just more efficient
in what they're doing and have better quality
in the process.
So generalist and then specialist evolving to that.
Why don't we open it up?
Any questions from the audience?
The question was about any concerns
in exposing case data to the LLMs
and sharing some of that customer information.
So I don't know if anyone wants to think.
- No direct experience right now, yes, certainly concerned
and have done some research around large language models
that can tell lies for the query.
And so certainly if you were to ask a general statement
to a large language model, it may give you something.
But if you say within this context,
and so there are concepts within AI like vectors
that siloed off, right?
Private business data, customer data, things like that,
which you may have use cases for.
We haven't gotten that far down that road yet,
but I think the data science appears to address it.
But I think it's still early in the days, at least for us.
- For us, we did examine that concern.
And it was really a matter of retention, storage,
and some of those areas that for us
alleviated the concern.
The data was not leaving our
system of records, so.
- That was really it for us,
on our factor moving forward.
- We're just being very deliberate
about where the technology is pulling information from.
So for example, we know that we'll get better output
if we structure the input.
So all of our help center articles
have the same kind of construct, taxonomy, et cetera.
And we're not at this point putting case data
through those channels, but it's really more about
the internal help knowledge and customer facing help knowledge
and you structure it appropriately so that, you know.
Yeah.
- I'll add to that, and I'll say,
because we're talking about JNAI,
and I'd like to ask how many here know
that their company have actually a JNAI policy
related to security and privacy.
And this is something that, like for example,
we had to construct and share with the entirety
of the company.
I mean, we're a cybersecurity company
and we're a public company, so that gives us
a lot of scrutiny.
That also means that you have to do a couple of things,
both the tagging, the vectoring,
kind of leaving some data out and minimizing data, et cetera.
But also investing sometimes in, basically, in-house,
LLM enterprise models out of the big kind of AI companies,
which actually cost a lot of money.
So some of these things, if you wanna do them yourself,
especially when I integrate them in your product,
you have to make a huge investment.
And other times, you just have to have a lot of scrutiny,
both on the in and the out, I guess.
And it starts also from having an awareness,
because guess what?
Anyone in the company can go and do a trial
or do something and start pushing data.
And all of a sudden, maybe you got some of your stuff
out there, so that's why it's important to also start
with internal education and policy
from a compliance and privacy standpoint.
>> Very good point.
>> Thanks for sharing that.
>> Any other questions?
>> You need an article for that,
which is where we're moving towards.
And as I mentioned at the top of this time together,
we're using Salesforce Einstein
for a lot of our next generation AI capability.
So in this case, we're keying off of some of that.
We also use a blend of COVEO as part of our tech stack as well,
that also helps, you mentioned agent insights
and some other areas.
So the combination of these tools are helping us
kind of build out consistency in our article creation.
What we're not doing well right now is,
and I'm hoping that we get there with this next level
of what we're introducing with Salesforce,
is reducing article,
like I don't know how you're managing that today,
but over time, things become legacy real fast, right?
You're moving new product versions in and out,
and the articles don't always keep tempo with that.
So that's an area that we're struggling with today as well.
We're trying to be careful in some cases
of not automatically creating new knowledge,
because there is that maintenance aspect of it.
So if it can be as structured as possible,
either in tagging along the way to where you can present
to the agent a draft of an knowledge article,
but then also train in the kind of the diagnostic methodology
of you going through all these steps, does it make sense?
Is there another one like it?
Do we have to add yet another thing to maintain?
So that's really important.
- We'll take one more audience question.
- So I'm hopeful that those are all gonna converge,
and it's gonna be embraced across not just support,
but all the different lines of business,
so I'm excited about that.
- Fantastic, well with that,
I thank you all for joining us.
Thank you all as well, and let's continue.
I think we have an hour plus, more food, cocktails,
yep, thumbs up in the back.
So let's continue the conversation.
- Thanks again.
- Thank you.
(audience applauding)
This is a test comment.
This is a longer test comment to see how this looks if the person decides to ramble a bit. So they're rambling and rambling and then they even lorem ipsum.