Name: Ship Code Faster with Claude Code on Vertex AI
Uploaded: 2026-04-29T14:16:11.747Z
Duration: 1 h 4 min 8 s
Description: Ship Code Faster with Claude Code on Vertex AI

Transcript for "Ship Code Faster with Claude Code on Vertex AI": Hey, everybody. Thank you for joining us here. I'm Roy Arson. I'm a solutions architect here at Anthropic, and I'm joined by Ivan from Google. We're super excited to have you, listen to, our webinar. We're gonna dive, quickly into how to, shift code faster with cloud code on Vertex AI. With that, let's start with the quick intro, Ivan. Yeah. Hi, everyone. Nice to nice to see you all here. I'm Ivan. I'm a developer advocate at Google Cloud working together with, Entropic and Roy. So pretty much excited to be here today. So just a few housekeeping items before we begin. Number one is don't worry if you miss something. We're gonna share recording, at the end of, this, in the next day. We also encourage you to submit questions in the q and a chat. We have folks, also listening in the chat and helping answer questions. And do give us feedback. We are spending, more content, in the webinar form and looking for your feedback to help us address topics that are top of mind. And then so what we're gonna cover today is number one, a quick overview of cloud code. I'm gonna walk you through how to set up cloud code on Vertex AI. Then we're gonna cover, you know, feature availability, differences between different platforms, whether you're running cloud code on Entropic versus on Vertex AI. We wanna make sure that picture is clear. And then Ivan will walk you through, how to to leverage GCP controls from a security and access and governance. So walk you through these best practices, and then we'll close together. So by the end of the session, the goal is for you to be comfortable with setting up cloud code on Vertex AI and using it and managing it. So let's dive, right in. Before we talk about cloud code, let's just do a little bit of, framing. Cloud is a full platform. We provide the models at the bottom. It's you know, this is the Frontier intelligence layer. You know, you're familiar with, models like Sonnet and Haiku and Opus. On top of that, we have the agent capabilities. These are the building blocks like persistent memory, tools like web search, skills, agents, sub agents. You know, and then on top of that, this is the platform that you are the surface area that you are interacting with. Whether it's cloud code or cloud apps, this is where you have the cloud desktop app, or the co work cloud co work. And then you have the cloud developer platform or the cloud console if you're building against our API to build your own products. So we're gonna focus on, you know, this piece right here, cloud code running on Vertex AI. But it's worth noting, you know, where it sits in the broader ecosystem. So what is Cloud Code? Cloud Code is, our, you know, coding agent that lives in your terminal or Versus Code. You give it a task. It fixes it. Refactor the module, write test for the service. It goes on and does it. It reads your code base, makes edits, run commands, shows you everything that it's doing. So it maintains transparency and keeps you in the driver's seat if you if you wish to to be in control. So it's one of the fastest growing products that we've ever seen. Now how fast? In fact, Cloud Code, you know, this is already old statistic now, but it reached 1,000,000,000 in December 2025, which feels like a lifetime with AI. Last month, which is February, it's crossed 2.5, so more than doubled since then. The interesting stat, I think, is that enterprise is over half of that revenue. So in the last six months, enterprise customers, contributed 10 x growth. So developers like you are driving this adoption, and many of this adoption is happening on Vertex AI. So we thank the, you know, our partnership with Google Cloud. Today, we'll show you how to run that cloud code agent on Vertex AI with native GCP, using, GCP off, I'm and cloud monitoring all built in. So one of the most compelling thing about cloud code is that, you know, it augment different aspect of your software development life cycle. So anywhere from exploring an unfamiliar code base to planning architecture, to building features, to debugging CI failures. It uses your team's existing CLI tools so you can focus on the solutions. Things like, you know, CLI, like Git, Docker, BQ, if you work a lot with BigQuery like like myself and Ivan here. So this quick slide, I'm not gonna go through every box here, but, you know, as you can imagine, Entropic uses cloud code heavily. We dogfooded. What's interesting is where we see the value shows up across the entire organization, Not just from an engineering point of view. Product uses it for fast prototyping. Data science builds in notebooks. Marketing automates their ad creative. The pattern is anywhere there's a repetitive technical or knowledge work, Cloud Code can create leverage. So, this webinar is focused on developers and and cloud administrators like yourselves, but it's also worth noting that it's not just for software engineers. So let's dive right into the demo. Let me show you how to actually set this up. Alright. So I'm jumping into my GCP console, aka the cloud console, number one is before any developer touches the terminal, you wanna make sure that you are, you know, set up on the Google Cloud side. So first, you wanna enable Vertex AI, on the project where you are running and, you know, cloud code. Next, jump into the model garden. We're gonna spend a minute here because this is important. Now you wanna enable the different models that you are using. I've already enabled Hiku four dot five. You wanna make sure you enable different models, specifically Hiku, and, you know, Sonnet four dot six or Opus four dot six. Hiku is useful because, that's not what Cloud Code uses for a lot of the housekeeping items. It is a fast cost effective model, so we use it whenever we can to maximize your token efficiency. So, on this, you know, model card, the thing I wanna direct your attention to is the specific, you know, capabilities of this model and when it is deployed. So you can see that, you know, it does offer prompt caching, which is, you know, relevant for what for cloud code. It does offer a global endpoint. So, again, check your model card to see the latest features and the latest version, names. Specific syntax matters as we're gonna see later as we set up things on the cloud code side. So in this case, make sure you copy this, and we're gonna paste it shortly, on the cloud code settings. The yeah. So the global already, you can see here, and this becomes relevant if you decide to use the global endpoint. And then last but not least, you really wanna check your quotas. Quota, just a quick note. In my case, my quota, you can you can access your quota from the I'm admin panel. And then, in this case, I'm zeroing in on the Vertex AI service. This is you have lots of quotas in terms of, the the most important ones are the, you know, the query per second, the request per second, the input token per minute, and the output token per minute. Those are per model, per region. So you can see here, this is my, you know, this is my quota for the global endpoint for CloudSonic for, dot six. And in this case, you know, my, you know, newly provisioned GCP project is 3,000,000 input token per minute, and Haiku is, much higher, 10,000,000 token per, you know, per minute. You wanna budget, you know, as your team grows, you wanna make sure that you have enough quota. You know, a good rule of thumb is a couple 100,000 per, you know, token per minute per developer. That number will vary depending on your usage, obviously, but do plan for it. The good thing is you can keep track of your peak usage, current usage. So and you can set up, sir, you know, usage alerts so you can be on top of it. But it's you know, if you think you're gonna hit that quota, you know, quickly, definitely before you roll it out to a team, file a quota request or talk to your account team. So this is what we have on the Google Cloud side. This is the setup, and it's important to know that all of these of controls are available to you, out of the box. So let's jump into the terminal, on the developer side. So now I'm jumping into the my terminal. In this case, I'm working on a, you know, hypothetical app. It's an inventory management app. I'm building new features. This is my, you know, git branch with new features. I already have cloud installed. You can install it from, a one liner in our docs page. What's important to note is, you know, it leverages your GCP application default credentials or whatever GCP, off flow you're using. So in this case, definitely, number one, make sure you are, logging in. So, you know, I already logged in, but, you know, I wanna show you the, you know, full command for everybody's awareness. And then, you know, this is gonna create your auth, token and cloud code will pick it up from there. The other thing I wanna mention is my environment variable that I set up. You wanna make sure that cloud code is set up to use Vertex AI, and then you point it to your GCP project and then the endpoint. Those are the three important environment variables to set up. In my case, I'm using the global endpoint. You may want to set pin it to a regional endpoint. Like I like I've shown you earlier, the models that I'm using, they support, global, endpoint, but you may wish to pin it to a specific region. So there's a lot of environment variables available for you to configure the way you want. We do recommend you pin model versions. So to prevent, you know, CloudCore picking up a new version that you haven't approved in Vertex AI, that's a common, that's a common issue. So pinning the model version, it's it's, you know, highly recommended. Remember that value I asked you to copy from the model, card? So paste it right here. In this case, anthropic default high q model is, the value that we saw earlier. The one thing I wanna point out as well is I wanna make sure I use the 1,000,000 contacts window. So by default, so I I add the bracket one m for 1,000,000 token. So this is really, what you have to set up on your side. I do have also the cloud code disable experimental betas. So, basically, I'm disabling all the beta headers. The reason being is Vertex AI will reject request that has, you know, headers that it doesn't recognize. So to be on the safe side and to minimize your error rate, you know, currently, it's good to disable that. Let's jump into, cloud here. So I'm just gonna fire cloud session. I'm gonna open, just see the status. You know, I wanna, you know, validate that this is running against the Vertex AI endpoint, and sure enough, it is using the Vertex AI and my project and the region that I set. So this is, you know, you're probably already familiar with this, but let me show you what's going on in the serve in this cloud code. I have MCP server set up. I have, you know, skills, and I also wanna show you my context, what's going on in the context already. So you're not losing features. I have custom agent. I have sub agents, skills, MCP tools. I'm using the 1,000,000 token. All of that is, backed by, you know, every request is backed by Vertex AI. So to generate some, traffic, I'm just gonna request the model to, test some new features that I added to the app. So it's gonna leverage, you know, Playwright in this case to actually, open up the app and test the different features. So with that, I'm just gonna jump back to the GCP console. So one of the things that GCP, Vertex AI ships with are out of the box dashboards. So in this case, there's a Vertex AI model garden dashboard. This is this is aggregating metrics and logs, so they're readily available for you to see, you know, what models are being used, what regions are being hit, what's your throughput. So it's a lot of important visibility that you have available without any instrumentation. So I'm just gonna refresh this just to get the latest, you know, over the last, you know, there's not a whole lot of traffic because this is, you know, just one one developer in this project. But you can see there's a throughput. Let me actually reduce the timeline to the last fifteen minutes. K. So, you know, a lot of visibility and, you know, SRE level, metrics that can you can keep a tab on your latency, QPS, your, error rate, again, to make sure that you know, to validate if you have set up anything wrong on your side when it comes to the, you know, cloud code, and regional endpoint, for example, for the models or if you're using a wrong model. The other thing I wanna point out is you can see the cache read percentage is pretty high. So, again, you can validate that prompt caching is working, even though it's a global endpoint. So this should be good news for a lot of you that traditionally pin to regional, just for caching. Now you could still pin to regional for other reasons like regional residency, and we're gonna talk about that later. And then last but not least, in terms of provision throughput, if that's what you have set up, you can also keep track of utilization of the provision throughput to make sure you are, you know, right sized from a capacity point of view. So this is, you know, I know I spent a bit of time on the Vertex AI model garden dashboard, but the bottom line is without any additional setup, you have, you know, production grade observability on cloud code, you know, with the Vertex AI. The other thing I wanna mention is if you enable the audit logs for Vertex AI, specifically the data audit logs for Vertex AI, you can actually, see down to the, request level. Now you're not gonna see the actual request, you know, the prompt and and response, but you will see just the all the, you know, Vertex AI, model prediction request. So this helps you, you know, drill down to the specific users, that is interacting with Vertex AI and, you know, the trend over time. So as you saw, folks, the setup is just a few environment variables. You tell Cloud Code to use Vertex AI, point it to your project, and then you run cloud. That's it. Your existing GCP I'm governs the access. And one thing is that we didn't go through is you can set up the I'm access control for the specific users to be able to access Vertex AI. So not anyone can, hit that endpoint, without, you know, without your, man you know, access management. So most of the cloud code features are work out of the box. And now that you've seen the setup, let's talk about how, you know, how the picture looks like when it comes to running cloud code on an topic versus Vertex AI. This is a common question. You know, here's the full comparison. The key takeaway is that you have CLI capability, Versus code extension. You lose out on the desktop, web, mobile, Slack experience. But, the CLI capabilities is basically, what, you know, the majority of the features in terms of client side sandboxing, plugins, proxy, support, which Ivan is gonna talk about. Tools like web search are also available. One of the where Vertex differs also is the manage settings are distributed via mobile device management rather than your anthropic enterprise account. Also, user cost control is handled through your gateway, which Ivan is gonna cover. But you also have the GCP project quotas that I talked about earlier. Vertex also gives you something, that is pretty unique is, you know, the native request and response logging and sampling, straight into BigQuery. So that's also, available for you if you wanna audit anything along with the ability to run your own security perimeter in GCP, of course. Ivan's going to show you how you build on top of all of this, in the next section. Ivan, over to you. Thank you, Roy. So now let's move from, how do I set up clock code with Vertex AI on how do I run it, on an enterprise, in an enterprise context. So I've been, I've been working with, several teams that adopt the clock code on Vertex AI, and there are some patterns that, they really make the difference. So for the rest of this presentation, we are gonna cover them. In particular, when, you roll out any AI coding tool, at the enterprise scale, platform teams, they usually look at three main, areas. So the first one is, related to environment controls. So any development tool that has access to Shell needs some kind of execution boundaries. So Cloud Code already handles this on the client side, as you know, and as Roy was also mentioning, like, you know, for example, they ask you approval before to run any bash commands. It ships with a sandboxed bash as well. But in an enterprise context, you may want to have an additional control also on on server side, especially when, you know, compliance or data governance is involved. The other the second, consideration is, related to access. So cloud code works great, with the individual credential as also Roy was, highlighting before. But when, you go, from, you know, five developers to 50 or 100, hundreds of developers, you naturally want to start having a centralized layer. So a unique place that allows you to manage credentials, set, pay, like, a pair development, developer policies, as well as, control budgets. So this is, one of, this is the second and most important consideration, everything related to access. The third and the last one is related to impact. So at some point, once your team is enabled with cloud code, you your leadership or your stakeholder, they may have asked, asked questions like, what's the adoption trend? Like, what are, what are we spending? So you need you already have this data from the operational perspective, but you want to provide this data to, your stakeholders in a way that, you know, they can run their own financial analysis and see, like, the impact of cloud code at enterprise level. So, we we are going to cover these three main aspects, that are these three main consideration, and I will show you how you can kind of implement, you know, architectures, on Google Cloud in order to cover them. But before to do that, let's, let's see the environment that we are going to use. So compared to what Roy just shared, this is the environment that we use for the remaining demos, and it's not a local, setup. It's more it's as you can see, it's an architecture. It's a it's an environment that runs on Google Cloud. So how does it work? It's pretty simple. Developers essentially open a browser and land, on a portal that requires some sign in. So and only the authorized developers can go through this. And then, once he get access, he has a web portal that, will allows the developers to provision an environment a development environment with the the ID, installed, which can be in in our demo will be Visual Studio Code, with all the cloud code integrations or the extension. And, everything we run inside a a on a VM that has no public IP. So all the traffic related to this VM is controlled. The VM connects to two main things. So on one side, we have, the, APIs that allows you to get access to cloud models on Vertex AI. And on the other side, as you can see in this picture, we have, some services that we are going to use with Cloud Code, including MCP servers, SLLM gateways, dashboards at the end. Some of them, they run on serverless function like a cloud run. But everything, like, in terms of authorization, like, everything is at the VM level. So it does a service account and that, you know, is used to invoke, those services. So the next three demos, we are going to use all these pieces in this, representation in order to run our demo. So let's start with the first consideration that we were saying, and, I will show you, the the demo related to it. So the first consideration, as I mentioned, is, the environment control. So we want to have some execution boundaries on the developer on the dev tool, in this case, on the on the clock code. So, again, clock code, very good on client side. It ask approval. It comes with sandboxing on the OS level. So you can use seat belt if you use a macOS or Bubblewrap if you use Linux. But on the enterprise context, you want an additional boundaries on the server side. So when you run the code in in core code, it runs in a completely separate environment that has a its own dedicated compute. So this is very useful, for example, when you have shared development environments or when teams need some specific comply some specific compliant requirements. So this is related to code. The same thing we can apply for web search. So code comes with some web search capabilities that developers can can disable or enable. But at enterprise level, you want to have more governance on these choices. And, usually, you set these, you know, you set these controls at the organization level. So the security team has a better view and has a better, again, control, with respect to these, these capabilities. So how do you implement these two pattern? So the, client, the server side code execution and the control on the search capability of a clock code. So here is, how you can do that on, on Google Cloud. So for the code execution, what you can do is that you can build an MCP server that run on a serverless function like Cloud Run using FastMCP, for example, which is one of the most common framework to build, MCP tools. And each time Cloud Code needs to run a code, it runs this code in an isolated environment. So it calls, what it does, it calls the MCP server through an authenticated proxy, And the MCP server, in this case, will expose several tools that will allow us to create a sandbox, execute the code, and delete the sandbox. So it's pretty straightforward. Under the hood, what the MCP server does, it calls the Vertex AI Agent Engine API. And Vertex AI agent engine is a platform that allows that provides a set of services to build, you know, agentic application. And in this case, you will use the code execution, service. The code execution services provide a sandbox, which is an ephemeral VM managed by Google. It's an isolated environment with no network access. So it supports several languages, and it keeps the states across multiple sec session using the same sandbox. And you can also set some time limit in order to force, you know, the code execution on this into a certain, time, into a certain time. So when the code runs in the sandbox, again, it's running remotely, none on the client side, and, it's, it's isolated. So if something crash, crash on the on the sandbox and not on, your, local environment. From the web search, it's easier. You don't need any, you know, integration with NCP. There is a native integration with of cloud models, and on Vertex AI with the search capabilities. And the cloud code will use the standard web search tool to call the API. But the cool part of the search on Vertex AI is, the governance part. So these searching capabilities is, can be enabled only for for, like, only from the organic, organization, administrators. So it's a it's an org level control. Developers cannot override it. And if your project, for example, run-in a v in a VPC, like, the web the web search will block by design. So, again, these are all ways to give, more control on what can be executed. To give you an idea of, the environment that I just show and, how does it work, let me jump into a demo. Share this out. Okay. So, you should see my screen pretty well. So this is the portal that I was describing in the first, in, like, before. So all you as a developers, I go I I access to this web portal. I sign in, and then I land on this, on this page. So as you can see, this page give me two options. I have the, the the possibility to open my development environment, and we will see later, like, how I can track my usage, my token counts with a a dashboard, a gateway dashboard that I will show you later. But for now, let's focus on we want to see how Cloud Code can execute code remotely with the sandbox environment. So I will jump into my development environment. So this is how, it looks like. So just to give an idea, all the code that we are gonna show you today, it will be available, at the end of this, of this webinar. And so we and today, we're gonna run three demos. So the first one is the sandbox. Now while I'm, while I'm explaining you what, what I've done, let me quickly just ask Claude to execute. So, we will see the results of running the code remotely. Execute. Yeah. One to n. Teardown. So for the sake of, for the sake of the demo, I just created a a few scripts that allows me to run, the entire the entire demo. But, essentially, what I want to show you is that, in the first demo, you will have a setup script. And the setup script, what what it does is essentially, will allows you to connect to the remote MCP server, through gateway that, I'll, that I will create. And, it will allows you, you know, to use the tool available on, on the, on this remote, MCP server to execute the code. So this script, essentially, what it does, it point, you know, it's a configure the new code setting to point to the remote, MCP server to run the code. How I build the MCP server to run the code? As I said, I use a fast API. And, here you can see, the code that I created to, you know, create a sandbox environment and execute the code. So, essentially, as I said in the presentation, we use the Vertex AI agent engine, API. So just to give an idea, this is the operation that will allow me to create a sandbox. And as I said, you can define some configuration to hard stop the code execution on the on the sandbox as well as I have a function here that will allows me to get the sandbox that I created, get the code, that Cloud Code will send as an input. And, eventually, you can also pass some input files that runs on your local, local environment, and, it will execute the code in the isolated, in the isolated environment. So once you create this, MCP server, all you can do is that, you you can package it with a simple, Docker docker image and, deploy on, on Cloud Run. So this is, this is essentially the stuff that you need to cover in order to get an endpoint that will allows you to execute the code, remotely. So now as you can see, Cloud is setting is running the entire process of setting up the connection with the remote, MCP server, and they will run a test. A test is a very simple, Python script that I will run-in the remote environment, and then, I will get back, the results. It gets some time, but, in the meantime that we are waiting, like, I just want to do I want just want to show you how, my MCP server look like on the on Cloud Run. So, this is, this is just a view that you will get so you can observe the logs and all the interaction that Callcode is having with the MyMCP server. So here, I have some past interaction. And, if you are fast enough because I mean, it will create, it will spin up the, remote the sandbox code environment. On the fly, you can also jump in the Vertex AI, agent engine. So this is Vertex AI. And the agent engine, as I said, is the platform that provides several services around, agentic application. And, you will see, like, a sandbox popping up, here, that will, will allows you to to run, to run the code. This is a temporary, environment, as I said, so, it will be eliminated. Now as you see, the proxy the proxy is set. The prompt is sent, and there is, it is, it is running in the in the background. It takes some time, but, I can, guarantee you that the code, will, will get, executed. So, but this is just give you an overview on, how, like, a remote code execution look like. So to recap, you create an MCP server to connect to the Vertex AI agent engine API to, to connect to the, the, code execution environment. Yeah. And this you see? So we get the results. This is the funny, like, cool code that I asked, cloud to to print. And, so you connect with the remote, APIs. You create the sandbox, and then, you you just set up the proxy. You change the configuration file on code side and that's it. From the developer perspective, nothing change. As you can see, it's just some few settings that Roy was also showing you. Okay. So this, shows you how you can quickly set a remote, development environment. So at this point so we we kind of set one of the potential environment controls, that you can, that you can have. Now let's talk about the access. So core code works great with individual, Google Cloud credential. So each developer authenticate with the ADC as Roy was mentioning, and you are ready to run, essentially. And this work very well with the small teams. Right? But while the like, as you grow, you you want to create a like, you you probably need a centralized layer. And the the reason is that because you want to have a one place to manage credential instead, you know, at the machine level or, you want to have you want to set some per developer or per team, a rate limit and budget. You want to have visibility on who is using the model and how much is using it, and the ability to manage all these things in just one place. So that's when, you know, the gateway pattern comes in. Essentially, it's a it's a medium between developers and Vertex developers that want to use cloud on Vertex AI and Vertex AI itself, and it gives you this control, again, without affecting the developer experience, and I will show you in the demo. So how you can set the gateway pattern on on Google Cloud is pretty simple. You one of the most common, gateway, project that you can use is a light LLM, it's open source. Again, you can deploy it on Cloud Run, and, on the server on the developer side, as I said, nothing change. You just need to change an environment, a variable in your clock code setting, the on topic based u URL that points to the, the gateway. And the clock code will work exactly as it was working, before without the gateway. On the platform side or the other side, like, things change because now with the gateway, you have that central, control point. The gateway authenticate to Vertex AI with, its own service account, and, one service account is is configured for one place to manage, the, the access. Every developers as well with the, with the the gateway layer gets a virtual key. So the gateway knows, who made every request and not just, you know, request happen, but this developer, they use this model. It burned these many tokens at this cost. And, the the important thing on the virtual key is that you can set some rate limits. So you, again, you can now control on the API keys and, set some budget caps, if you want to have this kind of controls. The other cool thing of, having a gateway is also that, in particular, a gateway like a light LLM is, it provides, two output. So first of all, you can log, everything that is collected by, the gateway in, a cloud SQL. So a PostgreSQL that will be used to power, to feed, the light LLM dashboard that I'm gonna show you in few minutes. But this dashboard gives you real time view of the request, token counts, cost of the endpoint, and so on. And the second output, which is also important, is that, thanks to light l m, you, you will land on the open, telemetry, tracing. And you can, push this trace directly in Cloud Trace with, a collector. And, again, this will allows you to collect a lot of information that you can then ingest in the query and build custom dashboard. So with that being said, let me jump into the second demo, that I want to show you. And let me share again my screen. Okay. So we are back to our demo environment. And in this case, I want to run the second demo. So the one related to the light at the LAN gateway. So in this case, again, first of all, let's let's ask Claude to run again, the entire demo while I execute d n to n again, while I will explain you, what I did. So, essentially, everything is, in the setup, again. So in this setup, there are three things that I want to highlight. First of all, the and the first thing that we are going to, change is, related to the, proxy. So we create a we create a proxy that will allows us, again, to connect our cloud code from this development environment to the to the, remote, gateway. Then as I said, we create a virtual key. And the virtual key, it allows, the developer to be identified in the gateway. So, again, gives that level of control on the token burn, the API calls that are made, directly. And then, we need to also, you know, reconfigure clock code in a way that can use the gateway. And, again, the beauty of clock code is that you can just, change a setting file as a ROI also was sharing. And, again, as I was saying, in this case, we have the, we just need to change the entropic base URL variable to point now to the, gateway. And, we need to disable the direct connection that our cloud code was having with with Vertex AI. So now all the API calls are made through the gateway, just changing an environment variable. So these are the two things that you need to change. Oh, of course, like, with the this setup file is pretty is pretty simple. And as you see, Claude, we already set up and run, 10 prompts. And why I run these, 10 prompts? Because the beauty of, the beauty of, you know, light LLM is that now we have access to a dashboard that will allows me to see this consumption. So let me quickly find the dashboard. You should see the dashboard. So when, when you join the dashboard, you have several components in the Lide LLM, gateway. I'm not gonna cover all of them. What I really want to show you is that it gives you very detailed, capabilities in terms of observability. So here you have the usage. So in the last few days, I made almost 100 requests. Most of them, they get, they went through. And, this is the amount of, token that I burn, input, output, the cost, by, you have the cost by provider. You can have multiple provider. In this case, we have just Vertex AI. And then if I go on the logs, so these are all my API calls associated to the key, a particular developers that I have that I that I set, and then I can click on each entry and see, like, in details all the request, informations, the metrics. And the response in this case is, is not available. Why? Because we redacted it. So I set in a way that I don't show the response, but here you have all the metadata associated to this request. And, I mean, this is just a just a bit of what you can see in this dashboard that you can go ahead and see the model activities. This is the, you know, the trend over time. This is the model that I called and so on. But this is a very powerful tool in terms of, controls and operations. Okay. So with that being said, let me quickly go here. Okay. So at this point, we cover an environment control and access. We just need to cover the last step, which is related to the impact. So we have the server side execution boundaries. We have the centralized access. Now, as the adoption grow of cloud coding and enterprise contacts, contacts, you may have leadership and stakeholder asking, what does, the usage look like? What's the the cost by team? What are the models that give us the best value and so on? So some of these answers, you can find in the dashboard that I just was showing you. But, usually, these these stakeholders, they want to have a better view of this data because maybe they want to merge those datas with the other financial data. Let's assume that you build a product with cloud code. Right? So cloud code in that sense will represent a cost, and they want to do some cost, revenue analysis. So they need to merge the data of the model consumption with the with those data. So you need an analytic an analytical layer that is fully customizable on their side in order to make these analysis. The good news is on Google Cloud, like, collecting this data and, you know, building this dashboard is pretty simple, and you can do in three steps. First of all, you can leverage the native integration of Cloud Trace to BigQuery. Why Cloud Trace? Because as I said, the gateway will push some of the information that I was showing you in the dashboard on Cloud Trace. So what you can do is that you can collect spans, in a in a trace bucket, and then you can create a dataset linked to this bucket. And, this dataset will give you a read only view, which is related to all spans. So on top of this, read only view, you can run how many SQL views you want. So you can have queries, that will allows you, for example, to count the total token, for, per model and the cost associated to it. You can have, SQL queries that will allow us to calculate the cost by hour and model itself. So this is an important information for the trend, in terms of trend for your financial teams. And all these and once you collect all these data in BigQuery, you can use to a visualization tool like Looker to, you know, build your custom charts. So just to give you a quick sense because I think we are going along with the presentation, I just quickly share how this, look like, at the end in, in BigQuery. So and that you can, you will be able to run your code, by yourself. So, as I said, after you, you leverage the integration from, Cloud Trace to BigQuery, so this is how BigQuery looks like for people that are new. BigQuery is analytical, data warehouse that you can use. So you will get our old span table. And, this table contain essentially all the information that, more or less, it was, was, they were powering the Allied LMM dashboard in the back in the back, of the, like, the l l light LMM dashboard that I was showing you before. So from this data, on top of this view, you can run several queries and create additional views, as I said, some of to run latency analysis, usage per model, and so on. So at the end of the day, you will get, you will get table with this with this kind of row, and, you can, you can get, like, information related to your models, like, in this case. So this is a very simple query, that tells me, like, the number of requests, input, output, token consumed, by models. Of course, like, you need to clean a little bit this data, but it just give you an idea of the information that you can you can leverage. And then from this table, you can, just, use the, you can just use, Looker to, let me see if I can share this. So, yeah, the from the big query table, you you can create a visualization like this one. And, I mean, in this case, they look pretty much similar to, what I was showing you before in the, LIDAR LLM dashboard. But imagine you have other data related to, again, the cost, sorry, the revenue that the product that you're building with Calcode is generating. You can merge this and do very and build very powerful financial reports that gives you an idea of the impact that Calcode has on your, product, life cycle and your in your business. Okay. So with that being said, let me go back on my presentation. The only the last thing that I want to, share with you is, mostly related to, some troubleshooting. So once you start, you know, using call code and Vertex AI, there are some common issue that usually developers face and you want to be aware of. So, two main one are related to some code error. So the four zero four and the four two nine. So the four zero four is related to it's a very simple error to interpret. This model, model, model not found. And, it usually means that either you forgot to enable the model or model guard then. So don't don't forget to go on model guard then find the model that you want and click click enable as Roy was, was mentioning. And, if you already enabled the model, but you're still getting this, it's probably related to, the region, that you are choosing. So be sure that you choose the right region. Some model support global, some model doesn't support, do not support global, and be sure to set the vertex AI region environment variable in the clock code configuration. The other error that is pretty common is the four two nine, which is a quota exceeded. Now with respect to this one, again, it's very important that you set the configuration correctly. As Roy was mentioning, don't forget to set all the all the environment variables that are related to the model, and, for example, use a small model, faster model like, AQ for background work, like summarization rather than big models, you know, as a OPUS. And, and, also, whenever possible, like, again, play with the different region, check, which region are you using, if it's global or not. Usually, the global, the global endpoint, can help because it provides, a widest, quota pool that you can use. But, again, it depends on your configuration. And the last thing that I want to mention is, the, the old, conflict of conflicts that you may face. So Vertex AI, as we talked also in this webinar, handle with the Google Cloud credential. The main thing to watch is that you don't have the you you don't have both the, the the Google the g cloud off or the Google application credential environment variable set at the same time. Just pick one depending on your environment. If you are a VM on a VM as we were I was showing, probably service account will be good. If you are on a laptop, probably the authorization like the one that Roy shared, it works the best. And, also, there is this new feature of GCP refresh that allows you to automatically refreshing the a DC, tokens. So you don't need to, you know, handle, this part for you. So I hope with this, I cover more or less some of the three main enterprise consideration and how to solve them on GCP and also, like, some of the issue, that you may face. Back to you, Roy, to some quick reference and to wrap up. Thank you, Ivan. Folks, that was a lot of ground we covered together. So I'm gonna leave you with this, a quick reference guide. These four steps will help you have a smooth rollout. We try you know, we try to capture as many steps as possible here. But if you think about it, it's about setting up your auth, setting up the quota, configure, and cover. Number one number one is, you know, for the authentication, leverage Google ADC, you know, your default credentials, set GCP auth refresh, which was, just, shipped a couple of weeks ago. Very excited about that because now every time your, token expires, cloud code actually will trigger your off flow. So it will, not interrupt your sessions, you know, regardless of what your off flow is. So you can just point it to that. If you're using a sandbox like Ivan showed you earlier, definitely leverage, you know, the IAP with the VMs. The gateway pattern, if needed, if you wanna centralize authentication, you know, use a central API key or service account credentials, you can, use gateway for that. Next step is the quota, setting up your quotas. Using global endpoint is helpful for maximum availability and and, highest quotas, actually. That's why it's listed there. You wanna use the right model for the task. You know, I showed you Sonnet earlier, as my default coding model. But if you are doing a lot of reasoning and planning, you may wanna use Opus four dot six for that. Again, you can use a gateway pattern as well there if you want to have user level cost control and visibility. The visibility that is provided out of the box, that I shown earlier is telemetry level across the models and and different regions, but it's not it's not at the user level. So if that's what you're looking for, the gateway pattern is helpful. And then configuration is key. Cloud code hierarchical settings model, is very powerful. You have the manage settings, that gives you as organization admins, ability to enforce the tools, permissions, the sandbox policy, MCP allow list, what domains, Cloud Code can visit. A lot of those are, you know, you have fine grain controls to begin with, and you can, at the same time, you have provide you know, developers have flexibility to customize those, settings at the project level or user level, but they cannot override them. And everything that Ivan showed you, from a governance control are additional control provided to you by the platform by GCP. And then last but not least, you know, as Odell tracing as well as part of your governance, export those logs, audit your tool usage, audit, you know, you may wanna, you know, audit, your skills usage, and and things like that. So that's, a lot of questions. You know, I'm sure there's going to be a lot of questions, so definitely keep, you know, firing those away on in the chat. Reach out to either one of us on socials if if you, you know, you have additional questions. We're both active on on LinkedIn. And fine, last but not least, additional resources to throw at you. Everything Ivan showed you is, the demo code is gonna be available for you in addition to the sample app that I showed you earlier. There's a lot of documentation on both sides on cloud code, how to set up, on Vertex AI, but also the manage settings that I talked about, the permissions, and the LLM gateway. And then that link at the end of Vertex AI setup lands you directly into the model garden so you can enable, those models. Yeah. I think with that, we have a wrap. Any other, thoughts, Ivan? No. No. I mean, I think, it it was a great session. I I hope you enjoyed as, as I I I was enjoying it. The only thing is that, yeah, like, with, you can easily use call code on, on Vertex AI, and you have, you can have the control that you need and then the price level. Again, all the demos that we show, they're gonna be available. Just go and try yourself. And, yeah. Thank you all, for joining us, and, I hope you enjoy, this webinar as well.