The Daily: What GenAI at work taught us this year, and what we can expect from it in 2025

On today’s podcast episode, we discuss why US workers’ adoption of AI might be slowing, how generative AI has helped companies and workers the most, how businesses can best quantify AI productivity gains, and how we expect the technology to help workers do their jobs better in 2025. Tune in to the discussion with Senior Director of Podcasts and host Marcus Johnson, Senior Vice President of Media, Content, and Strategy Henry Powderly and Vice President of Generative AI Dan Van Dyke.

Subscribe to the “Behind the Numbers” podcast on Apple Podcasts, Spotify, Pandora, Stitcher, YouTube, Podbean or wherever you listen to podcasts. Follow us on Instagram.

Episode Transcript:

Marcus Johnson (00:00):

Data collaboration enables innovative companies to uncover powerful new insights that transform customer experiences and fuel business growth. With LiveRamp, marketers get the industry's only interoperable platform for data collection across every cloud, walled garden and media platform. Learn more at liveramp.com.

Henry Powderly (00:20):

So there's this idea that you just kind of turn it on and it's going to do all of these things for you and make your life easier and make everything you do at 2X. I think that's not the case. There's a lot of effort that needs to go into working with these tools, learning how to best communicate with them or finding outputs.

Marcus Johnson (00:42):

Hey, gang. It's Monday, December 9th. Dan, Henry and listeners, welcome to the Behind the Numbers Daily: An EMARKETER podcast, made possible by LiveRamp. I'm Marcus. Today, I'm joined by two fellas. We have with us our VP of all things generative AI. He's currently coming to us from our New York studio. He's Dan Van Dyke.

Dan Van Dyke (00:42):

Hey, good to be here. Thanks, Marcus.

Marcus Johnson (01:04):

Hey, fella. Yeah, thank you so much for hanging out with me. And this other chap I'm going to introduce right now. He's based in Maine. He's our SVP of media content and strategy. We call him Henry Powderly.

Henry Powderly (01:15):

Hey, Marcus. Thanks for the invite.

Marcus Johnson (01:17):

Yes, sir. Thank you both for being here. We start with the fact of the day. Bears don't actually hibernate, so not many animals truly do. And bears are amongst those that do not. So bears enter a lighter state of sleep or rest called torpor, which, like hibernation, is a winter survival tactic and it includes decreased breathing and heart rates, lower metabolic rates and body temperature. And according to the National Forest Foundation, unlike hibernation, torpor is not voluntary and only lasts for shorter periods of time. And during this time, animals are able to wake up quickly to avoid danger or to have a snack.

Dan Van Dyke (01:52):

I think I entered a torpor over Thanksgiving.

Henry Powderly (01:55):

Turkey torpor.

Marcus Johnson (02:00):

You know how unsettling it is though when you wake up after like a 12-hour sleep. Can you imagine a three-month kip?

Dan Van Dyke (02:05):

No.

Marcus Johnson (02:05):

Can you imagine actually hibernating?

Henry Powderly (02:07):

I cannot.

Marcus Johnson (02:08):

You wake up like, "Where am I?" you wake up thinking "Who am I?"

Henry Powderly (02:12):

It's too much.

Marcus Johnson (02:13):

It's too much time to be asleep. Anyway, today's real topic, what Gen AI taught us this year and what we can expect from it in 2025.

(02:26):

All right, gents. So I want to start with a November 12th article from Salesforce. It was noting that despite continued urgency on the part of business leaders and growing interest in AI among employees, AI adoption in the US is cooling. According to the new Slack workforce index, AI adoption rates among US workers have slowed growing from 32 to just 33%. So just one point increase from March to August of this year. However, from January to March of this year, so the first couple of months, adoption was up six points. So big spike in the beginning of the year. It's plateaued a little bit and in the middle of the year, according to this research. Dan, I'll start with you. What is your take on this report saying that AI adoption has slowed amongst the US workers?

Dan Van Dyke (03:11):

Yeah, so I was an analyst before I stepped into the role leading generative AI here. And so my first immediate take is just to dive into the methodology. I'm not convinced that this is anything but statistical noise. Two reasons for that. One is if you look at the global averages, they continue to climb. It's only a couple of geographies including the US and I believe, France where you're starting to see just a minor uptick in adoption from, I believe it was like 32% to 33% in the US and a similar climb in France. And then secondly, it's the weird time periods that could be factoring for this. So Salesforce fielded this in January of 2023, January of 2024, March, and then August also in 2024. And so comparing March to August means that you're going to see some seasonality. People may be using it more heavily in March as compared to August.

(04:08):

So if we still see this plays out again, if you refield this in January or March, I'll believe it's real. But until then, I look at it with skepticism. And then secondly, if it is the case that adoption is slowing, I imagine that one of the key factors is I've heard a lot of research pointing to a gender gap and adoption and I don't really have the expertise to explain or dive into that, something that I want to learn more about. But I've heard that men tend to adopt Gen AI tools more so than women, although I haven't seen that play out across EMARKETER as an organization. So those are the two immediate reactions that come to mind. But Henry, I'm curious for your take.

Henry Powderly (04:50):

Yeah, I might not have been as skeptical about the research but all the points you make are really valid. I do think it's plausible and some of the things that they point out in the article is driving that. The lack of training around AI I think is something that we know teams are still struggling with. This feeling like using AI is cheating. I think that is a real concern among workers. And I think it's important to remember that working with AI is work. And so there's this idea that you just kind of turn it on and it's going to do all of these things for you and make your life easier and make everything you do 2X.

(05:26):

I think that's not the case. There's a lot of effort that needs to go into working with these tools, learning how to best communicate with them or finding outputs. And if you don't have the time to do that at work, and we all know how busy teams are right now, especially most who are working with less resources over time. I can totally understand how there might be a slight pullback in finding the time to learn these tools that they can work for you.

Marcus Johnson (05:53):

Yeah. The point you made about people feeling like they're cheating if they're using this, a large number of folks are saying that almost half of people feeling uncomfortable missing using AI according to the research. It was interesting that they're more uncomfortable emitting because they're using it for certain tasks. So they were saying spell check, summarizing things, fine, writing code, absolutely. But the client emails, Slack messages to colleagues, maybe not so much. I'm wondering, you mentioned the gender gap, Dan, I'm wondering whether there's also part of this is just the adopter gap and also the age gap. So there's an argument maybe to be made that you've already reached all the early adopters and now the early majority will need some more help adopting this. They've got the first movers. You've got apparently twenty-something knowledge workers who are all pretty much using Gen AI tools according to a Google Workplace survey.

(06:36):

But then Slack Senior VP of Research and Analytics, Christina Janzer was saying too much of the burden has been put on workers to figure out how to use AI to ensure adoption of the tech. It's important to lead us not only train workers, but encourage employees to talk about it and experiment with AI out in the open. So I wonder if you've got to do a bit more to get that next wave of people to adopt and maybe that's what we're seeing potentially.

Dan Van Dyke (06:56):

Yeah, it sounds like I've got my work cut out for 2025.

Marcus Johnson (07:01):

You do indeed. A big part of this as well is folks say that they've used it, but how much have you used it? That same Salesforce study was saying 61% of workers have spent less than five hours total learning how to use AI, 30% saying they've had no training at all or experimentation. All this said, if you zoom out and you pointed to there's some seasonality in these numbers, which I think is a really good point, Dan, but our forecast team, we estimate that 37 million Americans enter a prompt into a Gen AI system and use the output in a professional setting at least once a month.

(07:33):

So 37 million, we call them US Gen AI at work users. We think that's going to grow to 50 million next year. That would be 16% of internet users, 43% of all Gen AI users using it for work. So we think those numbers are still going up when you zoom out and look at it year over year. Henry, end of 2024 now, when you look back at the year, what couple of things stand out to you in terms of how Gen AI has been helping workers and companies the most?

Henry Powderly (08:02):

Well, I think the first thing to note is Gen AI hasn't been a constant thing all year that works at a constant level. These tools have updated at a very fast rate and every time I'm amazed at what new capabilities are coming to the platforms like ChatGPT. Claude, I think, has had a really big year in all of the features that it has added. So it's not only that Gen AI has been a tool helping workers throughout the year, but that tool keeps getting better each quarter and changing the game on what you can actually do with it. So I think that's been a really a big change.

(08:36):

I think the experimentation phase is also slowing down a little bit because I think there's some feeling that they didn't get, workers didn't get what they wanted out of it after they spent enough effort at the beginning and now they're just starting to kind of say, "Well, I could do this myself." And anecdotally, I've met people who have told me that, "I really put some effort into using AI to help me do this, and in the end, I was constantly rewriting it or going back and refining the idea." And I think that there's that inertia that comes after that experience after you have that experience so much.

Dan Van Dyke (09:11):

I think those are great points. I would add to that two big breakthroughs that I anticipate in 2025 are one access to more corporate data. So data that lives on intranets could really enhance the quality of outputs. So for instance, I don't know what use case you were talking about that your team was disappointed in, Henry, or whatever colleagues you spoke to, but I have to imagine that if a tool had access to trainings that dictated like, "Here's what good looks like for that particular use case, or examples of great content that it should emulate," that the output would be much stronger.

(09:49):

And secondly, long-term memory is something that I anticipate breakthroughs within. So the painful sort of iterative process of getting the output to be what you want, a lot of those points of feedback are repeated every time you start a new chat. And wouldn't it be wonderful if ChatGPT's memory feature or Claude's was expanded to encompass really a stronger understanding of your preferences? And so those are two areas where I think we'll start to see breakthroughs that convert some of the people who are on the fence, whereas all the talk of agents is really dominating the conversation, rightfully so, but memory, I see as a tremendous opportunity.

Marcus Johnson (10:38):

It's Agentic AI. Is that too early to be talking about 2025?

Dan Van Dyke (10:40):

No, it's not too early at all.

Marcus Johnson (10:42):

No.

Dan Van Dyke (10:42):

And we've already seen, for instance, Claude this year releasing a demo of the ability to operate a computer using its generative AI models and Pydantic releasing new frameworks for developing Gen AI agents as well. So there have been lots of breakthroughs in this space and I think this is an area where you'll start to see competitive battle lines shifting in 2025 as Claude and OpenAI compete to be the platform of choice for developing these agents. However, I really see in terms of driving at work adoption among employees, memory being the key dimension to get everybody over the line and start using the tools.

Henry Powderly (11:24):

Yeah, I completely agree. In fact, my team right now is working on a report on Agentic AI that we're doing in partnership with Salesforce, and Salesforce is doing a lot. Its Agentforce tool is now embedded within its own ecosystem, starting to help with automation around a lot of marketing and sales tasks.

Dan Van Dyke (11:41):

Oh, yeah. It gives some quotes for that one.

Henry Powderly (11:43):

That's right.

Marcus Johnson (11:45):

I was wondering what you guys think of this because we're talking here about a couple of things that was Gen AI shown that it can help companies and workers with the most, but there was a take here basically saying that leadership and employees are not always on the same page. It happens a lot in life, but with this in particular, the Salesforce data is showing a disconnect, the article said, between what leadership wants employees to focus on and what workers expect they'll focus on with the time they've saved by using AI. So once you've used it, hopefully this helps you save time. What are you going to do with that extra time? Executives wanting employees to prioritize upscaling and innovation. Employees expecting to have that time saved by using AI to catch up on busy work and existing projects. Any thoughts there, gents?

Henry Powderly (12:30):

Yeah. To me, this just sounds like an illustration of the classic divide between leadership and staff. Obviously, leadership is going to be focused strategically and looking for ways to build product and build the business, and many times staff are participating in that, but when it comes to looking for ways to help them, it's the administrative stuff that takes up most of their days. And so I totally understand why, and I think it's also interesting when you think about AI and helping it work. There are two buckets to approach that. There's how are you using AI to improve the core output of what your job is meant to do? Is it to create content? Is it to generate sales? Is it to build and monitor software and how is AI going to support that?

(13:14):

But in every role, we have this shared administrative responsibility when it comes to managing email and meetings and notes and developing proposals, and those are the things I think that the survey is kind of pointing to that. Most employees that are working in the pits are really focusing on ways to save time there because that really is where a lot of the pain points currently exist, and I don't think that's a bad thing because if you really want efficiency, if you can gain it in the administrative side, then you want to make the case for how you use that extra time on the strategic side.

Marcus Johnson (13:47):

Yeah. Yeah. Dan, I'll turn to you for this question first. What's been the most overhyped aspect of how Gen AI can help people at work in your opinion?

Dan Van Dyke (13:55):

I think one overhyped aspect is something that Henry touched on, which is that you could just give ChatGPT a simple prompt and it will return something that is good enough. Oftentimes, the case is that you really need to build custom GPTs or quad projects, which is Claude's answer to custom GPTs and custom GPTs by way of explanation or just going into ChatGPT and basically giving it a lot of context about what you're looking to have it do, what good looks like, what bad looks like to avoid, and some examples of finished outputs in order for it to give you something that's workable.

(14:40):

And so the idea that if you prompt it, you're just going to get a great output, simply doesn't measure up. And then secondly, realizing that even with a GPT, it's oftentimes going to be an iterative process, and so what we're seeing emerge is not like AI sort of taking the steering wheel, but to use a sort of tired analogy being a co-pilot and the process is quite iterative. It's very much like working between a manager and an employee or an editor and a writer. It's a back and forth conversation.

Marcus Johnson (15:18):

Henry, anything else come to mind when you think about the most overhyped elements of Gen AI?

Henry Powderly (15:23):

I think the time savings, the productivity, the efficiency side of AI. I think we're still too early to be focused so much on that because of, like I said before, we should be doing the work right now to learn how to use these tools and to experiment with them. We should understand that we're going to develop prompts and they're not going to work and the outputs are not going to be good and we're going to need to refine them, and we're going to need to go back and think about how we're approaching these tools. We're going to need to research all of the tools that are available. The ecosystem is way bigger than ChatGPT and Claude, and I'm seeing so many great new tools emerge that are focused on specialized tasks.

(16:03):

So there's still a lot of work to be done to really integrate and learn this technology that I don't think that the focus really needs to be on the productivity side. Unless you are enrolled in a task or you have a job that is very repeated, something you're working for a call center or in things like a manufacturing, those kind of jobs that are very repetitive, I can see focusing on productivity. But for knowledge work and marketing and some of the areas that we focused on, I still think that's not the right thing to be focused on.

Dan Van Dyke (16:34):

I sort of agree with you that it's not an end, but it is a means to an end that is useful. If you can demonstrate productivity gains in the form of hours saved and you can show that those hours saved are up into the right, then in the absence of being able to point to meaningful cost savings or meaningful revenue gains without having launched a production use case at this point, you can still justify to the C-suite some momentum and make a case for broadening access to tools and developing more trainings and all those needs that you rightfully pointed to that should really be the core focus at this point in time. So I see the two going hand in hand, but I do agree with your point. It's not the end-all, be-all.

Marcus Johnson (17:23):

Yeah. You were talking about productivity gains. Ryan Heath of Axios was noting 79% of leaders believe their company needs to adopt AI to stay competitive, but about 60% are worried that they aren't effectively quantifying productivity gains from AI according to 2024 Work Trend Index Annual Report from Microsoft and LinkedIn. Any other thoughts there on how companies can best quantify productivity gains from AI?

Dan Van Dyke (17:46):

I thought about this a lot and the approach that we've taken is to set up a number of pilots across the organization. We've run about 23 at this point, which are all run by independent leaders. An example of a pilot is our engineering team has one where they're using GitHub Copilot to speed up development workflows and self-reporting. So each member of this, I think it's 14 person pilot, estimates the amount of time that they save every week, and then we can zoom out and say, what is the collective time savings? Which we sort of calculate as estimate the amount of time this would take for you to accomplish this task. In the past, measure the amount of time that took, you now subtract one from the other and you sort of have time savings. And then if you zoom out even further from just one pilot to all the pilots, you can kind of say, "Here's how many hours saved per month. We did month one, month two, month three."

(18:42):

The issue with that approach, though it's not perfect, is that the foundation is still self-reported data. Self-reported data is not perfect by any stretch of the imagination, and so it could be under-reported, it could be over-reported, and if executives want to point holes or poke holes in that, they absolutely can. But in the absence of a perfect metric, I would say it's the best and perfect one that we have at this point in time.

Marcus Johnson (19:09):

Yeah. Henry, I turn to you to close out because Dan had mentioned a couple of things, looking forward to 2025, a couple of areas where he thinks that Gen AI will help folks working lives next year. Anything from you on the future of Gen AI at work?

Henry Powderly (19:23):

Yeah, I think it all ties back to search and how people retrieve and gather information for their jobs because no matter what position you're in, everybody's on Google at multiple points during the day, and I really think things like ChatGPT search and perplexity and the generative approach to gathering information is going to change that a lot. I expect to see Google lose some share. I expect to see some new entrants continue to come, but I think that that's going to be one of the more transformative things.

Marcus Johnson (19:52):

Yeah. I'll close with this because I think it's a really good way of looking at Gen AI moving into next year, recent Wall Street Journal piece, James Milin, a Google and Amazon vet who helped found a research and software company called Workhelix said, "AI is the general purpose technology of our era." And his colleague, Erik Brynjolfsson, professor at Stanford Institute for Human Centered AI and director of the Stanford Digital Economy Lab went on to explain that "General purpose technologies are things like the steam engine, electricity, computers, and these chaps think AI, they have three characteristics. They can improve over time. They're widely used throughout the economy. And third, they spawn complementary innovations that allow you to do new things and create new technologies that you couldn't have before. And that these general purpose technologies are responsible for most of the productivity and economic growth." Right. Gents, thank you so, so much for taking some time today to hang out with me and the listeners. Thank you first to Dan.

Dan Van Dyke (20:47):

Oh, thank you, Marcus. I always appreciate the invite and particularly so when Henry's on the podcast, so appreciate it.

Marcus Johnson (20:53):

Yes. Love having you on. Thank you also to Henry.

Henry Powderly (20:56):

Yeah, that was a lot of fun. Thanks so much.

Marcus Johnson (20:58):

Yes, indeed. Thanks all for being here. Thank you to Victoria, she edits the show, Stu and Sophie, the rest of the podcast team. Thanks to everyone for listening in to the Behind the Numbers Daily: An EMARKETER podcast, made possible by LiveRamp. Tomorrow, you can hang out with Rob Rubin, host of the Behind the Numbers: Banking and Payment Show, where he'll be speaking with our principal analyst, Tiffani Montez, and Victoria Guida, economics correspondent at our sister company, Politico.