Video: [Oct. 7th] Controlling Cost and Maximizing Value: Guidance for Enterprise Admins | Duration: 3216s | Summary: [Oct. 7th] Controlling Cost and Maximizing Value: Guidance for Enterprise Admins | Chapters: Welcome and Introduction (4.28s), Session Overview (81.165s), Spend Limits Strategy (208.67s), Usage Cost Drivers (327.905s), AI Context Best Practices (415.525s), Role-Based Access Control (519.3s), Usage and Spend Limits (843.145s), Role-Based Access Control (1078.93s), Admin Best Practices (1289.335s), Admin Reporting Framework (1461.165s), User Settings & Analytics (1562.915s), Organization Analytics (1731.095s), Usage Analytics & Reporting (1870.655s), Measuring Platform Value (2076.61s), Product Visibility Tools (2255.23s), Q&A and Wrap-Up (2338.165s), Admin Best Practices (2501.25s), Q&A Session (2609.8s), Parent-Child Organizations (2725.15s), Claude Academy Resources (2780.63s), Q&A Wrap-Up (2931.215s), Closing Q&A (3033.435s), Wrap-Up and Next Steps (3083.95s)
Transcript for "[Oct. 7th] Controlling Cost and Maximizing Value: Guidance for Enterprise Admins": Hi, everyone. How are you doing today? Happy Wednesday. I'm excited to present controlling cost and maximizing value today. We're gonna do some house rules, some introductions. Feel free to drop questions in the chat. That's what I wanna hear. At least there's a q and a section. Disclaimer, we have one person working the q and a. We'll we'll do the best that we can to answer all the questions, but they'll also be approving questions that he thinks the rest of the group should see. So, like, keep an eye on the q and a there. You can upvote questions. I'll also, keep a lookout, like, as I'm, like, going through the presentation. Any that I see, like, make sense to answer given my talk track, I will stop, which is further proof that this is not a recording. This is real. I apologize for sounding a little muffled. I was sick last week, so I'm just getting over that. But thank you again for the time. Let's jump into our agenda today. And, of course, if you have it, we will we will be sharing the recording after this presentation. So that's that's probably the most commonly asked question. We will be sending that out, so you can review, and we'll also share some helpful links at the end of the presentation here. So let's jump in to our agenda. So today, we are going to understand the different features, products, use cases, etcetera that could drive your bill using Claude. If you haven't been able to tell, today's session is focused specifically on cost controls and analytics. We have another presentation, an admin webinar. We're hosting one, in a couple weeks here. You can check on our new resources page. That is specifically for configuring Claude. So, like, product levers and features and things that an admin should be aware of to deliver that set products to to your end users. This is focused particularly on usage and cost analytics. So we we... We've split that out into two different sessions. We're also gonna learn a bit about role based access control. Depending on how you set up your enterprise, that is going to be the really important foundation of how you were granting access to users and through extension, monitoring costs for different groups, making sure that one group might have a different spend limit than another based on their use cases and what kind of products they're using, with Claude. Then tracking and reporting, it's really important for admins and owners within Claude enterprise to be able to report, out to the rest of the organization for sponsors, etcetera, of of how people are using the platform, what value they're getting out of it, what are... Is the actual token spend. And I can show you some best practices to get in, like, a groove of automating that process as best as you can. And spoiler alert, Claude will help a lot with that. And then proving out the value. So, like, what's... What is the spend actually producing? And then best practices. For context, I've spent a lot of time deploying Claude at organizations, both quite small and also quite large global organizations with tens of thousands of employees. So if you have questions on best practices, I'm able to lean on previous, experience and share what I think is a best practice. So, like, there's no question that is, like, too out there. Feel free to drop it in the chat or the question and answer, and we'll get to it. Alright. So this is, like, a way that I think about spend limits, and, hopefully, will inform the rest of the conversation we have here. And I I I like this, like, Goldilocks analogy here. Right? There's too tight, which is, like, okay. We don't want our power users getting blocked mid task. We don't want requests going on triaged, so there's the ability within Claude enterprise to ask for usage increases. Right? Because people will just get exhausted. If you keep trying to enter a door and the door is locked any given day, you're gonna stop trying to go through the door. Right? You're like, I'm gonna go find a different door. Right? I'm gonna go use this other product, which maybe isn't unlocking as much value for me. Right? So there's some analytics best tips and tricks that I'll share on the too tight style. Too loose is no limits at all. No one wants to be able to go to their CFO and say, so I actually didn't do my due diligence and set limits. And, everyone's spent a million dollars this quarter, and that's our budget for the entire year. We have controls in place to save you from that potential outcome, and that will be the core foundation as we are going through. So I think this is, like, this just right. It's you taking stock of your organization. Where do different users live and what do different users need? And then based on them using it from a pilot capacity to a wall to wall capacity, right, you're extending it to different populations of users. How are they using it? What do the numbers start looking like? And then how do I build an infrastructure to support revisiting those numbers on some time cadence? I think biweekly or monthly is perfectly fine. And report out, hey. I'm noticing this trend. What should we change given this information? Right? Or maybe you're good. You found your just right and you're good to go and, you know, you just report out, consistently for visibility. Alright. So let's go and talk about what drives a bill. This might be pretty obvious, but I think about, like, what are the things that a user might drive usage up or down. Right? It's the product, whether using chat, Coworker code. It's the model. Are they using a Haiku, Sonnet, Opus, or Fable model? Cost scales up. Effort, you can change the effort on the different model you're using, which drives prices up. If you're using Fablemax, it's gonna be, like, six times the cost of, like, a Fable medium. Right? So you're gonna... You definitely throttle effort there and therefore throttle cost. And then context. You know, are there a lot of tool calls being used? Are, are are there a lot of connectors being used? Is there... Is it set up efficiently? How is the person querying Claude? This one, a little bit less dependent as far as model capability goes. Context is becoming less and less relevant. It really does come down to to model and effort choice and and the product that's being used the harness. But context is like, hey. If I am just, like, being really laissez faire and not querying this correctly, I think this is where AI fluency actually plays quite a big part in, like, building a a culture around this. In that, like, you don't want people just driving up useless queries that that Claude doesn't really know what to do with, so it's just, like, wasting tokens for nothing. Right? So wanna call that out. We're gonna talk about role based access control. So first here, I'm going to share the architecture of what role based access control looks like. So, actually, there's a question coming in. We've got a smaller group, so I'm gonna answer this question. Someone asked in the chat, hey. Can you give some concrete examples around context? What are good practices on creating literacy there? I will tell you the easiest way, and I I want you to internalize this and think about how could I potentially communicate this to my end user audience. What I preach as gospel as far as, like, building AI fluency is treating Claude like an actual colleague. I think there is a paradigm shift, where back in the day when model... I say back in the day, like, it was years ago, like, last year, It's changing very quickly. Where we we viewed AI as an assistant. Right? An assistant, you only have to provide so much context, and it doesn't really matter, like, what you what you give it. Right? It's just like, help me, please. Do this. Right? So just wanna call that out. I think this shift of, hey. We need to treat Claude like a colleague. I need to give instructions. Boris, the inventor of Claude code, actually, was just talking about this the other day, about, like, people are surprised. This is how I query Claude. He's literally just having a conversation with it. And I think there's this, like, people, like, trying to chip away and, like, try to do, like... What about this and this little piece and this little piece? Like, you should just be able to, like, share your vision upfront because I would never catch a colleague in the hallway and be like, debug, please. It'd be like, what are you talking about? Right? So I think that very simple thing actually covers the context piece quite... Is a quite helpful way to think about it. Hopefully, that's helpful. And we can revisit that as we get into, like, analytics as well. Analytics will share a little bit more about that, and I wanna showcase smart reports for everyone on this call. Let's talk about role based access control. So role based access control is going to be the most important thing, most likely, that you were setting up at your enterprise. I think it is less relevant for smaller organizations, but that doesn't mean that smaller organizations should not leverage. And when I say, like, role based access control, what it is is the ability for you to create groups and extend capabilities to the group. So when I say capabilities, I mean, like, access to specific models, access to specific features, and then also extend spend controls, which we're gonna focus on primarily during this session, but extending spend controls to these groups. So for example, I have my base group. This is everyone. And what is driving this is what I am setting at the organization level in my org. So if I turn off Cowork work for the org base, every user is not getting Cowork work at the org level. But if I create a group that is builders and I have Cowork on, I can extend Cowork to the builders and then create another group for folks that... Like, a custom group that don't need Cowork. Right? So this is, like, due diligence that every owner slash admin should do is think about the populations of users. And I I I recommend simplifying it. Do the simple thing that works. Like, AKA technical staff that probably are gonna use Claude code, nontechnical staff where, hey. I'm I'm probably just gonna give them chat and Cowork work and give them up to Opus capabilities. Right? Opus medium is an amazing daily driver. Right? Cost has gone down. Where, like, maybe that technical staff, I'm gonna extend Fable and give them any effort level they want and give them the state of the art models there. So... And you could also see spend limits. So these are all the things that you can control in RBAC, and we're gonna do a demo. This isn't just gonna be me talking at you guys. But the way that it works, architecturally, is you create a group, then you create a role, and you apply that role to the group. So it's like painting the role, and I'll show you what I mean when we go into the actual demo here. There's also model settings to decide on day one, like, what you want the default model to be, what you want sticky to be, like, new chat start on your default, model access, and effort. So we're gonna take a look at that. And then for role based access control, we're gonna take a look at the demo here where we have, like, hey. I can actually set an organization limit. I can set a group limit, or I can send, I could send, individual events as well. There's a question coming in. Hey. Does this work with, Entre ID groups? Yes. It does. Connect to that IDP. Alright. So let's go into the actual demo. I guess I got one more slide here, where as far as limits go, your end users will have, an alert saying, hey. 75% of usage is used. 90% of usage is used. Once you hit a 100%, there's the ability to request more usage through the native, interface that routes it to admins. So I'm gonna show you what that, like, request section looks like. Alright. Let's go into the actual demo instead of slides where you don't understand what I'm talking about, and let's actually put some sets here. So let's choose a window. One moment. Of course, Google changes their... How you share things, which is ridiculous. It was not this this morning. Alright. Share this window. Okay. Folks can see this. Let's go in. Saw a question before I popped in about, like, what's your recommendation on bad actors preventing one user from draining the group pool. I mean, that's AI fluency too, but also note which user is doing that and slap an individual limit on them, I would say, is probably the best thing. And I I think, like, this is just monitoring it. Using the analytics and cost APIs to automate some of that process. I don't have access in this demo environment to, like, run any simulations of, like, a API workflow, but I actually will recommend this. We'll show you how you can get the APIs if you have the permission. In a lot of instances, only the primary owner is able to generate these API keys. But that being said, I really... And please excuse any bias here, but, like, this is just gonna be the easiest way to do it. Use Claude to set up this infrastructure. Like, if you say, like, hey. I have the cost and analytics API. Look at the help documentation. Set up this workflow. That works specifically for me. That's how you get a custom end result. Alright. So let's actually just go through the organization settings so we can look at spend. So we're going through here. It's how we're provisioning the groups. We know all about this, but I wanna talk about how the spend is considered here. So if we go to usage, here you can see how many of your users are blocked right now on spend and how many are near limit. So these are actually reports. It's a demo, instance, so not everyone is using it anywhere near their limit. But, also, what you can see is, like, how much has been spent in the monthly period. Here, I have not set an organization limit. It's unlimited, so maybe I could do, like, a a 10,000. And this is cool. This is for everyone at your org. You can enable or disable limit increase requests. You can see spend limits per individual. And based on the groups that we have created, either via syncing to our IDP or creating custom groups, if I toggle here, I can see all the groups that I have created and the limits per member. Now if I wanna do pooled budget, this is where I set the pooled budget. And, actually, as a clear follow-up I have from the last webinar that I did... That I need to follow-up on, so hold me accountable and share this feedback during this webinar is, like, if you have two values here, which one supersedes the other? I believe it is pool budget, but I do need to confirm that our help materials do do, make that clear. But the difference between these two is it's pretty intuitive. If I have a group and I set a member monthly limit, every individual in the group has this distinct thing. So, like, if I set this to 50, Paul, Jacob, and Sarah all have $50 to spend that month, and they all have their own $50 that is unique to them as an individual. Whereas monthly pooled budget is, John, Jacob, and Sarah all drained from the 10,000. So that's what someone is saying. It's like, how do you prevent a bad actor from doing that? This is the trade off. Right? You working to see, like, usage patterns. Do I go with the monthly limit and make sure that, like, anyone just gets stopped or a pulled budget? And then you can see, again, if you have services like code review or tag or security, how much those spends are doing... Spends are going and, like, setting up specific cost controls for those products within your organization. Someone is asking how do I clear old spend limit requests? If you go to notifications, they are in requests here. So you can use the, the interface here. Or if you have access to create one, you can create going to our API section. It's a compliance API here, but you can you can create other API keys, that are, you know, admin and analytics. And the admin API specifically allows you to programmatically enable or, deny requests. So I see a lot of pings coming in like, hey. I got a lot of requests. I recommend looking into how you can programmatically do this using the admin API. That's where where you can see that. It's easy... Especially at larger organizations, that's kind of, like, a prerequisite. The reason being is no one wants to read through a thousand requests. And if you have a thousand requests at the same time, it's probably worth noting that you should probably take a closer look at your limits. Someone's asking, do services like Claude Tag and Claude Security use individual spend budget? No. Separate. Those have their own separate budgets, that you can set. So, like, for example, like a Claude Tag, you can have a separate budget for a channel that Claude tag operates in. So there's there's granularity based on the individual product you're using. So let me show you what role based access control looks like. So here's all the members of your org, and it shows you where they are assigned and what groups they are members of. Here's all the groups you've created. You can have up to 1,000 groups, although I recommend don't do that. That's impossible to manage. And then the roles that you assign to the groups are here. So here, if I were to click into and edit the role, you can see I can edit the capabilities here. Someone was saying, hey. How do I make it so someone doesn't have access to all the admin stuff, but they could see analytics and billing stuff? So this is where you use role based access control to create a custom admin role, right, like a billing admin, and you would extend this access to can manage, can view, etcetera, based on your preferences and needs. Connectors and models. So, like, you can edit which models are going. Hey. I want the defaults to be Claude Opus five five, my personal favorite model that I use all the time. Hi. Save one change, and you just... Now the builder role defaults to Opus five five. Or maybe you want your nontechnical non builder role to default to Sonnet 5.5 at a medium effort to keep costs in control, perfectly fine. Right? I I I... Also as well, like, we've shared in our five five model releases that a Haiku five five is is getting released in the coming weeks. So if you're an admin and you're thinking about, like, okay. Maybe I want, you know, my scaled users to default to Haiku five five at high Afro levels. There's there's granularity that you can do here using the role based access control system. So this is how we extend capabilities. And then in usage, just going back to this to emphasize it again, this is where we set the per limit group spend controls or pool budget. We could also default to, hey. I wanna set it to a higher group limit or a lower group limit. So if someone's in a... In multiple groups and one has a higher group limit, one has a lower, you get to choose which one it defaults to. Alright. So I see a question. Hey. Can you confirm that the compliance API is actually pulling Claude code use? Yes. It is. It use... It pulls it in, and and Cowork work as well. Great. And, yes, someone is asking, hey. Is there an API to extract user analytics data? Is there an API to extract daily spend data? Correct. Yeah. We have a cost and usage API. If you do not have permission to create it, your primary owner will be able to. So find out your primary owner and request a creation of a key in this API and then ask Claude what you should do with it. Every org is gonna have different cadences that they wanna report on. So it's up to you to, like, design the strategy. I'm here to provide best practices, so, hopefully, it's helpful and not just, like, leaving you upstream, without an answer. But it really does depend on your organization. I don't think there's a best practice. But what we'll see in analytics as we get into it, I guess the best practice that I would recommend is, like, have a monthly cadence where you're pulling this, and you should be analyzing this monthly. Because as we know, settings change, model releases change, products change, and keeping, a pulse on how people are using them is important. Great. So I'm gonna go back to the slides and presentation we had. There, we just covered, how to create, groups and attribute roles and extend different capabilities, and also how to change the spend controls for those groups as well. So let's go back. Here, we covered all of this. We talked a little bit about the APIs and setting that up. If you have any questions of things that I didn't cover or you want me to revisit, please let me know. We'll have time for q and a at the end, and I'll cover that. And I know there's high volume, so apologies in advance. If we can't get to your question today, I'll try my best to read through them and answer in cohesive themes. Alright. These are things... Take a screenshot if you're an admin and you haven't set any of this up yet. Or if you have some... Set some of this up and you're like, that doesn't look super familiar. Strategically, this is what you should be doing. Right? You should pick a default model. You should think about the groups across your organization. I know it's easy to be like, oh, the people will figure it out and they'll subserve. If you wanna do that, do it via pilot motion, but I wouldn't go and be like, let's see what happens if I roll this out worldwide and willy nilly this happens. Like, I think this just also goes to change management best practice. You should be setting these expectations early with your users of how you... How people are gonna use Claude across the organization and what usage they have. And then arm people with the ability to take a look at their personal settings and usage so they can understand how to manage at their best, which I'll actually just... I'm gonna take a moment after this slide to show you, because I'm not sure if folks in the audience, are taking particular attention to this. But we have made huge strides in what individuals are able to see about their research patterns, and you should be able to arm your end users with the ability to check that out. Also, on our side, which I'm at a product on-site right now. So if you have any ideas, send them over. I'll go talk to some PMs on the enterprise side of it. It... It's important to realize that, you know, some of this should be baked in the product, and we get that through alerting and stuff to check usage. There's hyperlinks to check usage, but just worth calling that out. Overrides, org limits, requests. We talk quite a bit about this, but, like, set your strategy, do your due diligence, diligence, and and be intentional of what you're setting here. Of course, you're not gonna have all the answers the first go. We are learning. You are learning. It is a constantly moving fast stream, especially with model capabilities changing. So, this is where it's important to have that infrastructure in place to make sure it can adapt to the change. Alright. When I think about being a responsible admin, I think about these three things. If I have an answer for each of these things, I I should feel pretty comfortable about how I can report this, you know, upwards to, executive sponsor or someone along those lines. Right? So going here, there's adoption. This... I describe as, like, breadth. Right? We have... We bought x amount of seats, and we have 75% of them as weekly active users. That's great. Good. That's, like, the highest level, simplest, like, proxy, or, like, proxy or signal for people using the product. There's depth. How are they using it? Hey. Are people using skills? Are they using connectors? Are they having use cases that I would expect that department to have? This is where we get into more, like, granular reporting and analytics and smart reports, which I'll showcase in a moment. Excuse me. And then we have results. So these are things like ROI, outputs created, running those smart reports I mentioned. So let... Let's... We'll take a look at this analytics... These analytics and do a deep dive. But here's the five ways to see and share the number. Analytics chat. It's gonna be a very helpful friend of yours. Exporting a spend report. Member analytics and analytics API, which I've mentioned, and then compliance API. So it's just, like, always on. Right? And and just calling out. Compliance API used to only cover chat. Now it covers Cowork work and code as well. Cool. I do see a question in here that I think is important to answer for everyone. I asked earlier about whether Claude has enterprise controls for limiting spend per user per month. Sounds like there's no spend control that allows me to say someone has a $200 Fable budget and a $1,000 overall budget. Personally, I would really love that feature. Personally, I would also really love that feature. That is actually an ask that I have open. I work closely with our product team. So guess who I'm gonna go talk to after this webinar about, hey. What's the latest on this? That is actually something that I, am owed. So apologies in advance. PM, who I will not name for their sanity, here on this call, but I will... I... I'm literally here in person with that individual, so I will go talk to them. I'm owed that. So, hey, further evidence that this is a live webinar. So thank you all for the the great questions. I really appreciate it. And also just an aside, if you have feedback for this webinar, let us know. This is a brand new one. This is my first time running this. So thanks all for your patience and attendance today. Let's go through the demo because, of course, I can talk about slides all day. It's great. But the only thing that actually matters is is showing you what it looks like in, production instance. So let's do it. Gonna choose a window. Cool. So here, I'm gonna go... First, I'm gonna show you actually, analytics for an individual. So here's all the, like, settings that an individual has. Just pro tip for, an admin. I know this isn't related to cost controls. It's about to be. But just a pro tip because I can't resist it is, like, you should enable all your end users to think about their settings and actually view them. I think if we were to look at the data, probably a lot of people are not looking at this. And this is where the most important stuff is set. This is things like capabilities. What can I do in Claude? Do I have memory enabled? What does my memory actually say about me? You know, what connectors or skills do I have enabled? Also, just like, if you're thinking about your end users, connectors are literally the most important thing to have set up by your organization. You heard it here first. That is, like, a actual thing. So if you're leaving this call thinking about cost and analytics control, connectors should be part of that just as an FYI for everyone on this. It just drives usage, context that it unlocks and the types of tasks people are able to do with connectors just way better than, like, a vanilla clod with no access to any systems. Alright. But I wanna focus on this usage dashboard because what... This used to not exist. This shows the individual how they are using Claude on a model basis, on a time basis, what are the top skills they are use... Using, etcetera. So this gives individuals the ability to be thoughtful about how they're using Claude. Now as an admin, I can go to organization analytics, and this is where I can go ahead and take a look at how the org at large is using it. And I can ask questions using this chat. I could say something like, who are power users that I should consider extending more usage to? Why? Here's a prompt that may or may may or may not make sense, but what this does is it access... Accesses the analytics natively in Claude. So essentially is using a tool call to, like, the data that is natively built into your Claude enterprise license. So here, it's like, hey. Dan Waters has this many chats. Here's the disk... District session counts. We don't really have that many, like, meaningful users as far as power users go, because it's a demo environment. But you can see all this. You can dive into spend. And this is where, like, treating it like a colleague. You can ask follow-up questions. Like, I actually want more insight into raw spend data. I am reporting this to my CFO, so consider, polish and accuracy. Something along those lines. Right? Here I can go in, ask these questions. Go through the analytics. It's going to keep going. I'm gonna export this. Here we are. Right? And this can help me with my decisions. Right? I could pull this data, put it into a slide, go to Cowork, pull the data out, and be like, okay. Here's all this. Create a deck off of this. I'm gonna report it and do my due diligence. Alright. So analytics chat is extremely helpful. It's a new feature. It did not exist, like, a month ago. What we also have is smart reports as well. I do wanna... Actually, before we get to smart reports, I'm gonna showcase, like, who's using Claude. This is that, like, adoption threshold I was talking about. Like, just generally across the organization, am I doing well? Honestly, if you want to build meaningful AI transformation across your organization, I would really recommend, like, shooting for, like, 80% weekly active users. Claude should be the first thing that people reach for as far as their first line of defense, not to become overly reliant on it, but to, like, lean into AI fluency, have, like, give you a first draft, work with it. It's like the capabilities have just gotten exceptional. Here as we go through... Lost my tab here. You can see specific group spend. Can download this and feed it to Claude, or, like, Coworker code to analyze it. You can download this as well, and you can set routines that that do this. And then you could see adoption level, skill usage. So if you see one skill that is, like, really becoming ubiquitous across your organization, you should advertise said skill. This informs your change management or, like, enablement policies. I've seen organizations with tens of thousands of employees use this very simple skill report to be like, oh my god. All these engineers are using this skill. What does it do? You You get more intel, and you're like, wow. This would save the engineering organization thousands of hours every month, which you can roughly translate into cost savings. Right? I I will tell you this honestly. The deck that I was showcasing before, Claude generated it in, like, five minutes. Not in that I didn't give it any effort or care. It's because I fed it all the resources it needed up front. It generated it. I'm like, this is great. Here's some feedback, and we went to go. Like, you can package that into a skill. I did a presentation for a sales organization, last week. That was essentially, like, what is the most tedious thing that you do? Right? Oh, it's like doing this, like, quarterly business review and creating slides for it. Alright. Package it into a skill. I assume you have a standard for it. Right? You make your skill make it ubiquitous across the org and see that visibility here. So that's that's one way way to reduce cost. Connectors. You can see which connectors are valuable and encourage that folks download those connectors. You You can see how agentic people's work is based on the tool calls. This is where connectors and skills come in. There's, like, a, you know, pretty early, we're we're improving this experience, but, like, an ROI report here based on, you know, PRs, file operations, etcetera. How much is Claude costing? So, like, all these things per product surface, spend by model, all of this can flow into analytics chat, and, also, you can just talk to Claude about it and export it through the API. But this is this is a super helpful place. Smart reports. You can turn this on in your admin settings. It's in, I believe, the usage section. We have, the help center documentation on it. But here, if I were to go into, like, a demo report, here you could set, like, hey. How are specific groups using it? Are they, like, doing technical docs? How much spend is going towards these use cases? See effectiveness. This is, like, so so so granular, and you can run this, export this report, and feed it into Claude. That's where you could do some really, really, really crazy stuff and run it on a cadence and get get this level of granularity. Honestly, we can have an entire webinar dedicated to to smart reports alone. That's where I'll leave you with the encouragement of, like, if it's not enabled at your organization, try ahead and go do it. If if it is, play around fit. And here you have, like, recommendations based off of, like, people are using it natively. Maybe you should draft a skill and advertise it to the specific group. Right? There's a lot here that you can do. Surveys. Hey. Quantitative stuff is great, but qualitative is also amazing. Get people's pulse within Claude. Right? So what this does is it allows you to create custom surveys that show up in people's Claude instance, and I'd encourage them to, like, hey. Fill out the survey of how you're using it, and we'll take the best use cases and operationalize them. So that'll be that'll be great. I see feedback on, like, smart reports appear to be way too in-depth. Will there be more templates for a lighter version of the report? It's a good good call out. I don't know. So I'm gonna go ask some people because this is top of mind. This is actually a great forcing function for me to go have conversations with the product team that is owning this. So little serendipitous attendance here. Great question, though. Like, I think, like, prebaked templates would be a great idea for smart reports. Aside from all the visibility on cost and consumption, how do you recommend we measure value? So it's like this is the ever enduring question. I mean, I recommend using smart reports to maybe, like, surface level look at it. But the best way to do it is, like, see what skills are being used across the organization and what connectors are popular and what groups are using them. And that will actually give you a pretty solid inkling of how people are using it, what value. Because very typically, a skill can be attributed to time saved for a specific use case, at least I think so, and you can work backwards from there, and then see, hey. You know, what other departments, are using this skill or could be using this skill. I think also as far as admins go, a good thing to do is to go talk to leaders across your organization because most of the time, they might not actually know how people are using it or might not actually be fostering a culture around people using this platform meaningfully. So if you have a sales leader and, like, yeah, I actually don't know, it would get them to think about, like, oh, like, how do I make sure that my organization is using this in a way that they should? It's easier said than done, but those are the types of activities that a Claude admin should be going and doing. A roadshow of how people are using it, learn how power users are using it, and then ask the question, hey. If this guy is using it, there's actually not that much of a technical barrier for the rest of the organization to be doing something like this. And I can guarantee you, whatever the the power user at, like, a sales org or marketing org is doing, if everyone else did similar flavored things, it's like a light bulb moment that would just, like, help everyone involved. So I just wanna call that out. And then you have the, product specific visibility. So things like Claude chat, people using chats, are they using projects, what kind of projects are popular, who are the top members, are artifacts using... Being used. Like, artifacts are super important and helpful for, like, driving AI fluency across the org, artifacts by type, etcetera, Claude code. So if you have it connected with GitHub, you can see PRs and and other things as well as here. There's leaderboards, etcetera. Design, Cowork work usage, Claude science, Claude tag code review, etcetera. I can go on, but, like, you you should have visibility into your organization how people are using it. I think the biggest thing, and I will acknowledge this just, like, on the call. It's like, there is so much data. We need to help our admins, and we're working on it. Trust me. Help our admins sift through all of this data and just giving you a report saying, okay. Based on x y z that we're seeing, you should do x y z. Right? Like, it's easier said than done because every single organization is different, and we understand the challenge there. That being said, like, it is something that is top of mind for us. And, share that feedback because I have on-site time with the entire enterprise product team, which is great. But all that to be said, let's... I'm gonna wrap up really quick, and I'm gonna stay on for ten minutes, and we'll do q and a live. I should probably should have mentioned, I see a question, pretty cool stuff. Is all this available for 3P p or 1P p? This is 1P p. This is Claude enterprise product. 3P p is an an entire other telemetry problem in that, like, your cloud provider will be able to provide that through open telemetry, API. But let's go back to the slides real quick. I know this is an action packed session. I acknowledge that. There's a lot to get through. Honestly, webinars could be three hours. This one would be, be a conversation. But there's questions around, like, value. So we do have a value tab in the in the analytics as I just went through. Again, it needs improvement. So if there's feedback, like, honestly, more power to you. We'd, like, share that feedback, put it in here, and we'll report it out. But, you wanna think about, like, you know, cost per task, estimated productivity lift, the time recovered, what it can be automated. I have a task that used to take me, like, an hour. It's just, like, doing, like, a scrape of Slack in my email and putting it into a specific format. It takes me, like, five minutes. It's, like, very low calorie now. So, ostensibly, I'm saving an hour every single week, and then you stack that times, like, five use cases. I don't know. I... Based on how I use Claude, I... My productivity is, like, five ten x. But, again, that's where, like, cost controls need to be set. I think, like, setting this operating loop here is, like, you know, set the defaults at first, monitor, share the results of leadership, get feedback, get those open questions, learn, and then adjust. This should be done, like, on an every quarter basis at least. This is moving too fast. You should not do it, like, every half or whatever. Every year would just be not possible. Every two to four weeks, you should be taking a look at this. Can smart reports help us with frontier overreliance, like taking a Ferrari to the seven eleven? Yes. It can. Great question. You you can see what models are being used for specific use cases. I mean, of course, it's redacted slash generalized to a certain degree, but it should be able to give you that visibility. Alright. So nine links worth bookmarking. That being said, there... You don't have access to the slides, so I'm just gonna put them in the chat. And we can share them and follow-up as well. But that being said, what I recommend... Eric, I need to check if I can check the slides because it's a new presentation, but, we we will take a look at that. But I I shared the the slide, the links in here. My recommendation, if you're an admin, managing this is really, really difficult. I'm not underestimating the the the, like, fine line you sometimes have to walk with managing this, and I have extreme empathy for it. That being said, one pro tip that I can give you is, like, have this conversation in a brainstorming capacity with Claude. Whether you've deployed already or not, it is always worth having this curiosity, to workshop with Claude. You should not feel alone in this process. You should... Again, this is not, like, marketing and overreliance on Claude. You should, like, give it these help center articles. You should go into analytics chat. You should use just general Claude with all of this context given upfront. Figure, hey. What are some other ways that I could be a good admin of Claude? And you will get perspective that will... Some of it you'll already know because, like, how could it know it can't read your mind? Some of it will be really insightful. So I just wanna, like, give that nugget of, like, I... That's my recommendation. If I was a Claude admin, that's what I would be doing, because I I do, like, enablement trainings and stuff, and people ask me all the time, hey. If I'm a customer success manager, how do I use Claude? I... Or, like, I'm I'm a product manager. How do I use Claude? I don't know. Here's some, like, general ones I could think of, but, like, obviously, your job is more specific than just that question. Give it all that specificity and figure out what, like, stones you can unturn. Same same principle applies to being an admin. So let's do questions. I know this is a lot. I'm gonna stay on. Let's see what was the most upvoted. Monthly budget check could be too slow to catch a fast burn. What controls exist to stop one user or runaway agent from spending a month's budget in the first few days? Are there daily or weekly spend alerts or only monthly? Does analytics API give near real time per user spend, or is it lag? It's a really good question. I don't know off the top of my head how real time the API is. It should be near real time. And that is where, like, feed clogged the API documentation. I... Apologies. I don't know off the top of my head, so I don't wanna give you a wrong answer. And especially with something as sensitive as cost, lead you down this, like, oh, like, I trust Mark, and I accidentally... The $100,000 is spent because Mark said this on a webinar. So what I recommend is give it the API documentation, ask these questions, be very thorough, and ask best ways to set this up. You can create your own in house alerting system if it supports that in fact and is near real time. I think it is. Don't take me as 100% truth, but, like, those are things where, like, if you have a question on that, see how you can work with Claude to set it up and, like, say, I have this API key. What are the best ways that I can do it, like, innovatively? Can you assign reviewers per domain, for example? I may not know if someone's request is valuable to their org, but their director may. I hear this one frequently. We don't have it currently, but that's a good takeaway for me to figure out of, like, hey, if a sales manager knows this person is doing good work and needs more budget, let's go ahead and do that. I think I think that's a good call out. So, Chris, apologies. I don't have, like, the crisp answer of, like, yes. Emphatically, you should go do this, but I'm gonna take that one away. Does the platform support parent child organizational hierarchy where a parent organization has overall visibility and governance? Child organizations manage their own spend, the user's budgets, and spend independently. So currently, the infrastructure is the parent organization is like an entity that covers, like, SSL and provisioning, but not spend controls. Spend controls are per org. So if you have a parent org with, like, five child orgs, each child org has their own spend limits. I'm not 100% sure on anything that is being worked on right now, of, like, uniting it under one single banner. But for the record, we'll have the transcript process in these questions intake. I'm gonna make sure for the record. Emphasize that one. I think that would be a very helpful thing, to have. So I'm not gonna speak on behalf of product, but I will... I'll call that one out. Do you have any advice for what sort of curriculum to put in place for small teams starting with Claude? Yes. I do. For the sake of time, I'm gonna make it really easy for everyone on this. I recommend that you go and create or check out academy.claud.com. Now I am working on an admin section for this website. I think I told the last webinar that I would have it done by the end of the week. I apologize. I did not. I got pulled into something else. But I am working on it. And, at least in the meantime, the admin resources should be pretty easy to find just by checking our help center and asking Claude there. But this has, like, super helpful resources for your end users. Recurring webinars, articles, self guided courses that allow them the opportunity to earn badges. Right? So, like, building this AI fluency, like, literally might be as simple as, like, here. Hey. Opus five five on high effort. Here's academy. Can you read this website and help me, understand, like, what I should share with my end users to build productivity and AI fluency? Sure. Let me go take a look. Or just, like, comment manually. Right? You also... If you're on 1P p enterprise, you can log into academy, and your users can log in, and it keeps track of their activity and recommends, like, learning pathways. So, like, if you're wondering, how do I, like, hold my users accountable to learn this? Build a program. If they do a bunch of badges, they get some sort of incentive. Or, like, you can also, in your analytics, see how your end users are leveraging academy. This is a brand new feature. Right? So you can actually see if you distribute this, are people logging in? Are they using it? It's really helpful. The other thing that went live actually just yesterday... Let me make sure that we have it. I'm gonna give it to you folks live because it just, like, was launched today or yesterday. We have, we created a, like, resources website. So let me actually go to events, and we've got events, Claude academy. Actually, share it and follow-up. I don't see where it is. Browse workshops, I believe. So we have, like, a a new webinars page. It might actually not be live right now. Maybe it was looking at the staging environment, but it's gonna be, like, all the webinars that you have and all that access. So, like, just wanted to call that out. But academy is a good place to start. Apologies for that side piece. Alright. Will smart reports be available for HIPAA configurations in the future? We'll work on it. Our HIPAA org, you'll notice that some things are available and other things are not. So, I don't sure... I'm not sure for a 100% date, but just wanna call that out. It's a question around concrete examples around context, creating literacy there. I mean, like, it's... What... I would recommend too as an admin. This isn't specifically tied to cost controls, but, like, plugins are gonna be your best friend to make sure that context is best leveraged for the users. Users, unless in... Explicitly instructed and even if explicitly instructed, are typically not gonna have the wherewithal to be like, I need to go set up my connectors now, and I need to set up these skills now. Like, if you set up a plugin, which for everyone on the call, plugin is a collection of connectors and plugins that are skills that automatically install for specific users or for your org, They just have it out of the box by default. So they already are becoming, like, more empowered users on day one if you think about your plug in strategy, which the other... If you take a look, at our webinars that are coming up, I do a configuring Claude webinar that focuses specifically on things like plugins, connectors, and skills. So is there a way to set up notifications for team members to get when they get closer to their token budget? Right now, automatically, it's 7590%. There is no way to customize this, but I am of the personal belief there should be. So maybe that's another thing that I tackle with product this week. I'm gonna answer one more question. We can wrap up. We're on a usage based Claude enterprise plan and need to split Claude code spend between capitalizable development and production support, often for the same person. Can usage be tagged within a project? Yeah. So there's no no level of granularity there. I think the reason being is that, like, foundationally, you should just be, like, building that understanding with users of, like, your budget should be used x y z. I understand it's easier said... Harder... Like, it's harder to do that in in practice. But, Yeah. I I think that just comes from, like, AI fluency. I'm sure you could do, like, some creative smart reporting here, but I wouldn't know specifically how to do that. But all this to say, we covered a lot of ground today. I know we're wrapping up, probably went by in a flash. Check out those links. Check out academy. That will be helpful for you behind... Being a responsible admin and, like, understanding what are some things that maybe you haven't considered in your Claude ecosystem. So I definitely recommend that there. Work with Claude to help you upskill as an admin. I know that seems self serving I... As a Claude representative, but that being said, like, I honestly use that in my day to day to become a better employee, on the stuff that I'm working on. So I don't see why you wouldn't use it to become a better admin, especially, like, using the product itself to build that fluency. So all this to say, we covered a lot of ground. We covered spend limits via RBAC, extending things like product and model capabilities via RBAC, analytics, and the level of granularity we have there, and then managing things via API. And while that wasn't as hands on as a demo, hopefully, it, like, unlocked visibility and, like, hey. I don't actually use any of those APIs. I need to find my primary owner and request access and build some sort of smart dashboarding or a learning system that is automated and allows me to report this on a monthly cadence. If you get to that monthly report cadence, if you don't have it now, like, take this as a call to action, set it up, use Claude, use the APIs, and you will find yourself in a much better secure position as far as managing how people are using it. So that's what I recommend is, like, the central call to action of this presentation. Thank you all so much for your attendance today because this is a new platform, new webinar. I I wanna add more interactivity in the future so we we we get, like, more perspective and data, of course, but, like, thank you all for for your attendance and great questions. Apologies in retrospect if we weren't able to get to your question, but, hopefully, we covered ground. It's helpful for you to go feel empowered to upscale your cost controls and analytics. So thank you all. Much appreciated. Now I gotta get back to product and say, hey. I learned x y z during this webinar. Let's fix it. So thanks all. Have a great rest of your week, and we'll talk soon. Bye.