<?xml version="1.0" encoding="utf-8"?><?xml-stylesheet type="text/xsl" href="atom.xsl"?>
<feed xmlns="http://www.w3.org/2005/Atom">
    <id>https://choreolabs.com/api-management/learn/</id>
    <title>Choreo Labs API Content Hub Blog</title>
    <updated>2026-10-22T00:00:00.000Z</updated>
    <generator>https://github.com/jpmonette/feed</generator>
    <link rel="alternate" href="https://choreolabs.com/api-management/learn/"/>
    <subtitle>Choreo Labs API Content Hub Blog</subtitle>
    <icon>https://choreolabs.com/api-management/learn/img/favicon.ico</icon>
    <entry>
        <title type="html"><![CDATA[Spectral API linting: Custom rules & CI enforcement guide]]></title>
        <id>https://choreolabs.com/api-management/learn/spectral-api-governance</id>
        <link href="https://choreolabs.com/api-management/learn/spectral-api-governance"/>
        <updated>2026-10-22T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[Learn Spectral API linting hands-on: write custom rules, enforce them in CI, and see how the same rule model can be applied to API governance in Choreo API Management.]]></summary>
        <content type="html"><![CDATA[<p>An OpenAPI description is a contract. Consumers, SDK generators, documentation, and gateways all build on it, so a <code>snake_case</code> path, a missing <code>operationId</code>, or an undocumented error response doesn't stay small for long. Stoplight Spectral turns your API design guidelines into executable rules, so these issues fail a check instead of waiting for someone to spot them in review.</p>
<p>In this guide, you'll install the Spectral CLI, write custom rules, and assemble them into a ruleset. Then you'll run that ruleset in three places: in the editor with Choreo API Management's API Designer, in CI with GitHub Actions, and across an organization with Choreo API Management.</p>
<aside class="cta_6F_m" aria-label="Make your API design rules enforce themselves" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Make your API design rules enforce themselves</span><span class="desc_GzK8">Use Choreo API Management to run Spectral rulesets as design-time governance policies, from the editor through CI to production.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=spectral-api-governance" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-spectral-and-why-lint-an-openapi-spec-at-all">What is Spectral, and why lint an OpenAPI spec at all?<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#what-is-spectral-and-why-lint-an-openapi-spec-at-all" class="hash-link" aria-label="Direct link to What is Spectral, and why lint an OpenAPI spec at all?" title="Direct link to What is Spectral, and why lint an OpenAPI spec at all?" translate="no">​</a></h2>
<p>Spectral is an open source linter for JSON and YAML, maintained by Stoplight and built for API description documents such as <a href="https://spec.openapis.org/oas/latest.html" target="_blank" rel="noopener noreferrer" class="">OpenAPI</a> and <a href="https://www.asyncapi.com/docs/reference/specification/latest" target="_blank" rel="noopener noreferrer" class="">AsyncAPI</a>. Think of it as ESLint for your API contract: it checks the spec itself, before anyone writes code against it, and reports problems such as inconsistent naming and missing metadata.</p>
<p>Those problems have real costs. A missing <code>operationId</code> changes the method names an SDK generator produces. An inconsistent error format forces consumers to write API-specific error handling. A mix of <code>snake_case</code> and <code>kebab-case</code> paths makes an API harder to learn. Spectral lets you encode each expectation as a rule and catch violations automatically.</p>
<p>Spectral is licensed under Apache 2.0. The source is on <a href="https://github.com/stoplightio/spectral" target="_blank" rel="noopener noreferrer" class="">GitHub</a>, and the <a href="https://docs.stoplight.io/docs/spectral/9ffa04e052cc1-spectral-cli" target="_blank" rel="noopener noreferrer" class="">Spectral documentation</a> is the reference for the CLI and ruleset syntax.</p>
<p>Four related practices are easy to confuse. Each answers a different question:</p>
<table><thead><tr><th style="text-align:left">Practice</th><th style="text-align:left">Question it answers</th></tr></thead><tbody><tr><td style="text-align:left">Validation</td><td style="text-align:left">Is the document structurally valid?</td></tr><tr><td style="text-align:left">Linting</td><td style="text-align:left">Does a valid document follow our design and style rules? Spectral does this, although its built-in rulesets include some structural checks too.</td></tr><tr><td style="text-align:left">Contract testing</td><td style="text-align:left">Does the running API match its spec? (It comes up again in the <a href="https://choreolabs.com/api-management/learn/spectral-api-governance#running-spectral-in-cicd" class="">CI section</a>.)</td></tr><tr><td style="text-align:left">Governance</td><td style="text-align:left">Are those rules defined and applied consistently across the APIs they're meant to cover?</td></tr></tbody></table>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="installing-spectral-and-running-your-first-lint">Installing Spectral and running your first lint<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#installing-spectral-and-running-your-first-lint" class="hash-link" aria-label="Direct link to Installing Spectral and running your first lint" title="Direct link to Installing Spectral and running your first lint" translate="no">​</a></h2>
<p>Install the Spectral CLI with npm. Pin the version so your output matches this guide:</p>
<div class="language-bash codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-bash codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">npm install -g @stoplight/spectral-cli@6.16.3</span><br></span></code></pre></div></div>
<div class="theme-admonition theme-admonition-note admonition_xJq3 alert alert--secondary"><div class="admonitionHeading_Gvgb"><span class="admonitionIcon_Rf37"><svg viewBox="0 0 14 16"><path fill-rule="evenodd" d="M6.3 5.69a.942.942 0 0 1-.28-.7c0-.28.09-.52.28-.7.19-.18.42-.28.7-.28.28 0 .52.09.7.28.18.19.28.42.28.7 0 .28-.09.52-.28.7a1 1 0 0 1-.7.3c-.28 0-.52-.11-.7-.3zM8 7.99c-.02-.25-.11-.48-.31-.69-.2-.19-.42-.3-.69-.31H6c-.27.02-.48.13-.69.31-.2.2-.3.44-.31.69h1v3c.02.27.11.5.31.69.2.2.42.31.69.31h1c.27 0 .48-.11.69-.31.2-.19.3-.42.31-.69H8V7.98v.01zM7 2.3c-3.14 0-5.7 2.54-5.7 5.68 0 3.14 2.56 5.7 5.7 5.7s5.7-2.55 5.7-5.7c0-3.15-2.56-5.69-5.7-5.69v.01zM7 .98c3.86 0 7 3.14 7 7s-3.14 7-7 7-7-3.12-7-7 3.14-7 7-7z"></path></svg></span>Versions used in this guide</div><div class="admonitionContent_BuS1"><p>The examples use Spectral CLI 6.16.3 and Node.js 22. Built-in rules, default severities, and messages can change between versions.</p></div></div>
<p>On a team, install Spectral as a pinned dev dependency instead, so everyone runs the same version:</p>
<div class="language-bash codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-bash codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">npm install --save-dev --save-exact @stoplight/spectral-cli@6.16.3</span><br></span></code></pre></div></div>
<p>Then run it with <code>npx spectral lint</code>. The <a href="https://choreolabs.com/api-management/learn/spectral-api-governance#running-spectral-in-cicd" class="">CI workflow</a> later in this guide uses this setup.</p>
<p>Create a <code>.spectral.yaml</code> file that extends Spectral's built-in OpenAPI ruleset:</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token key atrule" style="color:#3D8BFF">extends</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> spectral</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain">oas</span><br></span></code></pre></div></div>
<p><code>spectral:oas</code> gives you a baseline before you write any rules of your own, including checks for schema validity, unused components, and duplicate operation IDs. The exact list varies by version.</p>
<p>Now lint a spec:</p>
<div class="language-bash codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-bash codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">spectral lint openapi.yaml</span><br></span></code></pre></div></div>
<p>Each finding shows the line and column, severity, rule name, message, and the JSON path of the affected node. For a small spec with a <code>snake_case</code> path and no <code>operationId</code>, Spectral 6.16.3 prints the following (trimmed to the first few findings):</p>
<div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">openapi.yaml</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"> 1:1   warning  oas3-api-servers       OpenAPI "servers" must be present and non-empty array.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"> 2:6   warning  info-contact           Info object must have "contact" object.                        info</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"> 8:9   warning  operation-description  Operation "description" must be present and non-empty string.  paths./order_items.get</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"> 8:9   warning  operation-operationId  Operation must have "operationId".                             paths./order_items.get</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">...</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">✖ 8 problems (0 errors, 8 warnings, 0 infos, 0 hints)</span><br></span></code></pre></div></div>
<p>Every finding here is a warning, so the command exits <code>0</code>, and a CI job running it would pass. That's why exit codes matter.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="understanding-severity-levels-and-ci-exit-codes">Understanding severity levels and CI exit codes<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#understanding-severity-levels-and-ci-exit-codes" class="hash-link" aria-label="Direct link to Understanding severity levels and CI exit codes" title="Direct link to Understanding severity levels and CI exit codes" translate="no">​</a></h2>
<p>Every rule has a severity: <code>error</code>, <code>warn</code>, <code>info</code>, <code>hint</code>, or <code>off</code>. Severity controls how a finding is reported. The <code>--fail-severity</code> option controls which findings make <code>spectral lint</code> exit with a non-zero status.</p>
<p>The default threshold is <code>error</code>. Lower-severity findings still print, but they don't fail the build. To fail on warnings too:</p>
<div class="language-bash codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-bash codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">spectral lint openapi.yaml --fail-severity=warn</span><br></span></code></pre></div></div>
<p>Setting a rule to <code>off</code> silences it without removing it from the ruleset.</p>
<p>Together, these support an incremental rollout: introduce a rule as <code>warn</code>, see how much it flags, and promote it to <code>error</code> once your existing specs are clean.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="writing-custom-rules">Writing custom rules<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#writing-custom-rules" class="hash-link" aria-label="Direct link to Writing custom rules" title="Direct link to Writing custom rules" translate="no">​</a></h2>
<p>A Spectral rule has two required parts:</p>
<ul>
<li class=""><strong><code>given</code></strong>: where the rule runs, as a JSONPath expression.</li>
<li class=""><strong><code>then</code></strong>: what it checks, as a function applied to the selected nodes.</li>
</ul>
<p>Optional properties include <code>severity</code>, <code>message</code>, and <code>description</code>. Without a <code>severity</code>, a rule defaults to <code>warn</code>.</p>
<p>The following rule enforces kebab-case paths, one of the naming conventions in <a href="https://wso2.com/whitepapers/wso2-rest-apis-design-guidelines/" target="_blank" rel="noopener noreferrer" class="">Choreo Labs' REST API design guidelines</a>. The guidelines give a practical reason: when a URL is shown as an underlined link, the underline makes underscores hard to read.</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token key atrule" style="color:#3D8BFF">rules</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">path-kebab-case</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Path segments must use kebab</span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain">case.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">message</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"{{property}} should be kebab-case (e.g. /order-items, not /order_items)"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">given</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$.paths[*]~"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">severity</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> error</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">then</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">function</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> pattern</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">functionOptions</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token key atrule" style="color:#3D8BFF">match</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"^/$|^(/([a-z0-9]+(-[a-z0-9]+)*|\\{[^}]+\\}))+$"</span><br></span></code></pre></div></div>
<p>The <code>~</code> suffix in <code>given</code> selects property keys instead of values, so the rule checks the path strings themselves.</p>
<p>The <code>then</code> block uses the built-in <code>pattern</code> function. The regex allows lowercase segments joined by hyphens and <code>{parameter}</code> placeholders, and rejects trailing slashes.</p>
<p>Why not the <code>casing</code> function? With its <code>separator</code> option (<code>char: "/"</code>, <code>allowLeading: true</code>), <code>casing</code> accepts <code>/order-items</code>. But in Spectral 6.16.3 it still treats <code>{orderId}</code> as part of the value, so it rejects a valid path such as <code>/orders/{orderId}</code>. A regex handles mixed literal and placeholder segments exactly. Choreo Labs' own <code>path-casing</code> rule takes the same approach.</p>
<p>Adapt the regex to your own conventions. As written:</p>
<table><thead><tr><th style="text-align:left">Path</th><th style="text-align:left">Result</th></tr></thead><tbody><tr><td style="text-align:left"><code>/orders</code>, <code>/order-items</code>, <code>/orders/{orderId}</code>, <code>/customer-orders/{customerId}/items</code>, <code>/</code></td><td style="text-align:left">Passes</td></tr><tr><td style="text-align:left"><code>/order_items</code>, <code>/orderItems</code>, <code>/OrderItems</code>, <code>/order-items/</code>, <code>/order_items/{order_id}</code></td><td style="text-align:left">Flagged</td></tr></tbody></table>
<p>The rule checks literal segments only. Parameter names are left alone, so <code>/{OrderID}</code> and <code>/orders/{order_id}</code> pass, and <code>/order_items/{order_id}</code> fails only because of <code>order_items</code>. Add a separate rule if you want to enforce parameter naming.</p>
<div class="theme-admonition theme-admonition-warning admonition_xJq3 alert alert--warning"><div class="admonitionHeading_Gvgb"><span class="admonitionIcon_Rf37"><svg viewBox="0 0 16 16"><path fill-rule="evenodd" d="M8.893 1.5c-.183-.31-.52-.5-.887-.5s-.703.19-.886.5L.138 13.499a.98.98 0 0 0 0 1.001c.193.31.53.501.886.501h13.964c.367 0 .704-.19.877-.5a1.03 1.03 0 0 0 .01-1.002L8.893 1.5zm.133 11.497H6.987v-2.003h2.039v2.003zm0-3.004H6.987V5.987h2.039v4.006z"></path></svg></span>Renaming a path is a breaking change</div><div class="admonitionContent_BuS1"><p>Before you enforce this rule on existing APIs, remember that fixing a finding means renaming a path, which breaks consumers. Treat those fixes as versioning decisions, and start the rule at <code>warn</code> for APIs that are already published.</p></div></div>
<p>The second rule requires an <code>operationId</code> on every operation:</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token key atrule" style="color:#3D8BFF">rules</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">operation-operationId-required</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Every operation must define an operationId.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">given</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$.paths[*][get,put,post,delete,options,head,patch,trace]"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">severity</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> error</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">then</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">field</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> operationId</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">function</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> truthy</span><br></span></code></pre></div></div>
<p><code>given</code> lists the HTTP methods explicitly. A path item can also contain <code>summary</code>, <code>parameters</code>, and <code>servers</code>, and a broader <code>$.paths[*][*]</code> would select those and report false positives. <code>then.field</code> points the check at <code>operationId</code>, and <code>truthy</code> fails if it's missing or falsy (<code>null</code>, <code>false</code>, or an empty string).</p>
<p>Spectral's <a href="https://docs.stoplight.io/docs/spectral/cb95cf0d26b83-core-functions" target="_blank" rel="noopener noreferrer" class="">core functions</a> cover most rules without custom code:</p>
<table><thead><tr><th style="text-align:left">Function</th><th style="text-align:left">Checks</th></tr></thead><tbody><tr><td style="text-align:left"><code>pattern</code></td><td style="text-align:left">Value matches a regex</td></tr><tr><td style="text-align:left"><code>casing</code></td><td style="text-align:left">Value follows a case convention (kebab, camel, snake, pascal, etc.)</td></tr><tr><td style="text-align:left"><code>truthy</code></td><td style="text-align:left">Value exists and isn't falsy</td></tr><tr><td style="text-align:left"><code>schema</code></td><td style="text-align:left">Value validates against a JSON Schema</td></tr><tr><td style="text-align:left"><code>enumeration</code></td><td style="text-align:left">Value is one of a fixed set</td></tr><tr><td style="text-align:left"><code>length</code></td><td style="text-align:left">String, array, or number length falls in a range</td></tr></tbody></table>
<p>For anything these can't express, Spectral supports custom JavaScript functions. None of the rules in this guide need one.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="a-practical-governance-ruleset-beyond-the-defaults">A practical governance ruleset: Beyond the defaults<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#a-practical-governance-ruleset-beyond-the-defaults" class="hash-link" aria-label="Direct link to A practical governance ruleset: Beyond the defaults" title="Direct link to A practical governance ruleset: Beyond the defaults" translate="no">​</a></h2>
<p><code>spectral:oas</code> is a baseline. Your organization's design standards are what make linting worth enforcing. This ruleset adds five rules on top of the built-ins:</p>
<ul>
<li class="">Enforce kebab-case for API paths</li>
<li class="">Require an <code>operationId</code> for every operation</li>
<li class="">Require an explicit security requirement for every operation</li>
<li class="">Require a description for every operation</li>
<li class="">Require a JSON schema for every 4xx and 5xx response</li>
</ul>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token key atrule" style="color:#3D8BFF">extends</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> spectral</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain">oas</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token key atrule" style="color:#3D8BFF">rules</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">path-kebab-case</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Path segments must use kebab</span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain">case</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> not snake_case or camelCase.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">message</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"{{property}} should be kebab-case (e.g. /order-items, not /order_items)"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">given</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$.paths[*]~"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">severity</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> error</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">then</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">function</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> pattern</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">functionOptions</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token key atrule" style="color:#3D8BFF">match</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"^/$|^(/([a-z0-9]+(-[a-z0-9]+)*|\\{[^}]+\\}))+$"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">operation-operationId-required</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Every operation must define an operationId.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">message</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"Operation is missing an operationId"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">given</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$.paths[*][get,put,post,delete,options,head,patch,trace]"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">severity</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> error</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">then</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">field</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> operationId</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">function</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> truthy</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">operation-security-required</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Every operation must define a non</span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain">empty security requirement.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">message</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"Operation is missing a security requirement"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">given</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$.paths[*][get,put,post,delete,options,head,patch,trace]"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">severity</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> error</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">then</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">field</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> security</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">function</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> schema</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">functionOptions</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token key atrule" style="color:#3D8BFF">schema</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token key atrule" style="color:#3D8BFF">type</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> array</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token key atrule" style="color:#3D8BFF">minItems</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">1</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token key atrule" style="color:#3D8BFF">items</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">            </span><span class="token key atrule" style="color:#3D8BFF">type</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> object</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">            </span><span class="token key atrule" style="color:#3D8BFF">minProperties</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">1</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">operation-description-required</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Every operation must document what it does.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">message</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"Operation is missing a description"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">given</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$.paths[*][get,put,post,delete,options,head,patch,trace]"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">severity</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> warn</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">then</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">field</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> description</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">function</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> truthy</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">error-response-schema</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Every 4xx/5xx response must define an application/json response schema.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">message</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"Error response is missing an application/json schema"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">given</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$.paths[*][get,put,post,delete,options,head,patch,trace].responses[?(@property &gt;= 400 &amp;&amp; @property &lt; 600)]"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">severity</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> warn</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">then</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">field</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> content.application/json.schema</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">function</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> truthy</span><br></span></code></pre></div></div>
<p>The severities reflect what should block a merge. Naming and security are <code>error</code> because this example treats them as non-negotiable. <code>operationId</code> is <code>error</code> because SDK generators build method names from it. Descriptions and error schemas are <code>warn</code>: they matter, but they shouldn't block merges during the initial rollout.</p>
<p>If you've used <code>spectral:oas</code> before, you'll notice it already has <code>operation-operationId</code> and <code>operation-description</code> rules. For CLI-only use, the idiomatic approach is to override the built-in's severity (<code>operation-operationId: error</code> under <code>rules</code>) instead of adding a new rule, because with both, the CLI reports the same issue twice. This guide uses standalone rules on purpose: the built-ins don't show up in Choreo API Management's results (see <a href="https://choreolabs.com/api-management/learn/spectral-api-governance#beyond-the-cli-applying-api-governance-with-choreo-api-management" class="">Beyond the CLI</a>), and a self-contained rule doesn't depend on which built-in rules an environment loads.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-security-rule">The security rule<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#the-security-rule" class="hash-link" aria-label="Direct link to The security rule" title="Direct link to The security rule" translate="no">​</a></h3>
<p>This rule has to catch two different cases. One is a missing <code>security</code> field. The other is <code>security: []</code>, which explicitly removes any requirement inherited from the top level.</p>
<p><code>truthy</code> only catches the first, because an empty array is truthy in JavaScript. The <code>schema</code> function catches both, plus one more: <code>minItems: 1</code> rejects an empty array, and <code>items.minProperties: 1</code> rejects <code>{}</code>, an entry that names no scheme and which OpenAPI treats as "authentication optional."</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token key atrule" style="color:#3D8BFF">security</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token plain">      </span><span class="token comment" style="color:#8A93A6;font-style:italic"># flagged by minItems: declares nothing</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token key atrule" style="color:#3D8BFF">security</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain">            </span><span class="token comment" style="color:#8A93A6;font-style:italic"># flagged by minProperties: makes authentication optional</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token key atrule" style="color:#3D8BFF">security</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain"> </span><span class="token key atrule" style="color:#3D8BFF">oauth2</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token plain">    </span><span class="token comment" style="color:#8A93A6;font-style:italic"># passes: names a scheme</span><br></span></code></pre></div></div>
<p>Be clear about what this rule doesn't do. Requiring security is your policy, not an OpenAPI rule. The check confirms that <code>security</code> is present, non-empty, and names a scheme. It doesn't confirm that <code>oauth2</code> exists under <code>components.securitySchemes</code>, that its scopes are valid, or that it fits your authentication model. Those need their own rules.</p>
<p>It also requires each operation to declare security itself, so on Spectral 6.16.3 it flags operations that inherit a root-level <code>security</code>. If your standard allows inheritance, replace it with a rule that evaluates the effective security.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-error-response-rule">The error-response rule<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#the-error-response-rule" class="hash-link" aria-label="Direct link to The error-response rule" title="Direct link to The error-response rule" translate="no">​</a></h3>
<p>This rule checks only for an <code>application/json</code> schema on 4xx and 5xx responses. A response that uses <code>application/problem+json</code> is flagged even if it has a schema (on Spectral 6.16.3), so change <code>field</code> if your error media type differs.</p>
<p>The filter also compares response keys as numbers, so it skips <code>4XX</code>, <code>5XX</code>, and <code>default</code>. If your specs use range keys, extend it:</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">given</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$.paths[*][get,put,post,delete,options,head,patch,trace].responses[?(@property &gt;= 400 &amp;&amp; @property &lt; 600 || @property == '4XX' || @property == '5XX')]"</span><br></span></code></pre></div></div>
<p>You can tighten this rule over time. Choreo Labs' REST API design guidelines define an error model with required <code>code</code> and <code>message</code> fields, optional <code>description</code> and <code>moreInfo</code> fields, and an <code>error</code> array that lists each error separately when more than one occurs. Start by requiring a schema, then require the key fields, then validate the full model.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="running-the-ruleset">Running the ruleset<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#running-the-ruleset" class="hash-link" aria-label="Direct link to Running the ruleset" title="Direct link to Running the ruleset" translate="no">​</a></h3>
<p>Here's a small spec that breaks all five rules:</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token key atrule" style="color:#3D8BFF">openapi</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> 3.0.3</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token key atrule" style="color:#3D8BFF">info</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">title</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Orders API</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">version</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> 1.0.0</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token key atrule" style="color:#3D8BFF">paths</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">/order_items</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">get</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">summary</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> List order items</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">responses</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token key atrule" style="color:#3D8BFF">"500"</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Internal error</span><br></span></code></pre></div></div>
<p>Linting it fires every custom rule and exits with status <code>1</code>. The output below omits the seven <code>spectral:oas</code> findings for readability, including the duplicate <code>operation-operationId</code> and <code>operation-description</code> warnings:</p>
<div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">6:16    error  path-kebab-case                 /order_items should be kebab-case (e.g. /order-items, not /order_items)</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">7:9   warning  operation-description-required  Operation is missing a description</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">7:9     error  operation-operationId-required  Operation is missing an operationId</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">7:9     error  operation-security-required     Operation is missing a security requirement</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">10:15  warning  error-response-schema           Error response is missing an application/json schema</span><br></span></code></pre></div></div>
<p>To fix it, rename the path to <code>/order-items</code>, add an <code>operationId</code> and a description, add an <code>apiKey</code> security requirement, and point the 500 response at an <code>Error</code> schema with a <code>$ref</code>. Lint again: none of the five rules fire, and the command exits <code>0</code>. Some <code>spectral:oas</code> warnings, such as the missing <code>servers</code>, still print.</p>
<p>Before you share the ruleset with your team, test it against both inline definitions and <code>$ref</code>s. A <code>$ref</code> changes which node a rule evaluates and where the finding is reported. For an example, see "Does Spectral resolve <code>$ref</code> before linting?" in the <a href="https://choreolabs.com/api-management/learn/spectral-api-governance#frequently-asked-questions" class="">FAQ</a>.</p>
<p>Save the ruleset as <code>.spectral.yaml</code>, and <code>spectral lint</code> picks it up automatically. The rest of this guide runs this same file in CI and in Choreo API Management.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="in-the-editor-api-designer">In the editor: API Designer<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#in-the-editor-api-designer" class="hash-link" aria-label="Direct link to In the editor: API Designer" title="Direct link to In the editor: API Designer" translate="no">​</a></h2>
<p>The editor is the earliest point to catch a violation. <a href="https://marketplace.visualstudio.com/items?itemName=WSO2.api-designer" target="_blank" rel="noopener noreferrer" class="">API Designer</a> is Choreo Labs' VS Code extension for designing OpenAPI specs. Its Design view shows errors and warnings from Spectral-based governance rulesets.</p>
<p>Three report cards sit at the top of the Design view: <strong>AI Readiness</strong>, <strong>Security (OWASP)</strong>, and <strong>REST Compliance</strong>, which measures alignment with Choreo Labs' REST API design guidelines. In the example below, <code>GET /customers</code> has no summary or description, the same kind of issue <code>operation-description-required</code> catches.</p>
<p><img decoding="async" loading="lazy" alt="API Designer Design view in VS Code showing AI Readiness, Security (OWASP), and REST Compliance score cards above an operation with an undefined summary and description" src="https://choreolabs.com/api-management/learn/assets/images/api-designer-design-view-8664a7227c1f5718e5b024f71717d645.png" width="1600" height="935" class="img_ev3q"></p>
<p><strong>See full report</strong> opens a detailed view with the overall score, passed checks, affected endpoints, and error and warning counts. It breaks results down by category and groups violations by rule, severity, or area of the spec.</p>
<p>The screenshot below shows the AI Readiness report for a sample Leave Management API. The score and the passed-check count are separate measures, so 78% doesn't mean 78% of checks passed. The extension bundles three Spectral rulesets: Choreo REST API AI Readiness Guidelines, Choreo REST API Design Guidelines, and OWASP API Security Top 10. You can fix findings with GitHub Copilot Chat or the form-based editor.</p>
<p><img decoding="async" loading="lazy" alt="API Designer AI Readiness report with a 78% score, 33 of 64 passed checks, 19 affected endpoints, 20 errors, 293 warnings, and a breakdown by dimension" src="https://choreolabs.com/api-management/learn/assets/images/api-designer-governance-report-d8e2e67e4890a60fcdd62042c221bb90.png" width="1600" height="935" class="img_ev3q"></p>
<p>Because these are Spectral rulesets, you can supply your own. Point <code>apiDesigner.spectral.rulesetFolder</code> at a GitHub folder URL or a local directory:</p>
<div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"apiDesigner.spectral.rulesetFolder"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"https://github.com/&lt;org&gt;/&lt;repo&gt;/tree/main/&lt;path&gt;"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token punctuation" style="color:#8A93A6">}</span><br></span></code></pre></div></div>
<p>API Designer matches a custom ruleset to a report by filename, so name each file after the report it replaces: <code>ai-readiness.yaml</code>, <code>owasp_top_10.yaml</code>, or <code>wso2_rest_api_design_guidelines.yaml</code>. If a file fails to parse, API Designer shows a warning and falls back to the bundled default.</p>
<p>What you can customize depends on the report. You can edit existing rules in all three. New OWASP rules should use the <code>owasp:apiN:2023-&lt;description&gt;</code> ID format, new REST design rules appear under <strong>Others</strong>, and the <a href="https://wso2.com/api-platform/docs/tools/vscode-api-design/govern-apis/" target="_blank" rel="noopener noreferrer" class="">API Designer documentation</a> advises against adding new AI Readiness rules. The easiest starting point is to copy <a href="https://github.com/wso2/vscode-extensions/tree/main/workspaces/api-designer/api-designer-extension/spectral-rulesets" target="_blank" rel="noopener noreferrer" class="">Choreo Labs' bundled rulesets</a> and edit them.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="running-spectral-in-cicd">Running Spectral in CI/CD<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#running-spectral-in-cicd" class="hash-link" aria-label="Direct link to Running Spectral in CI/CD" title="Direct link to Running Spectral in CI/CD" translate="no">​</a></h2>
<p>Editor feedback is advisory, and a ruleset that only runs on laptops enforces nothing. Run <code>spectral lint</code> on every pull request and it becomes a gate. Here's a <a href="https://docs.github.com/en/actions" target="_blank" rel="noopener noreferrer" class="">GitHub Actions</a> workflow with a pinned CLI version:</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token key atrule" style="color:#3D8BFF">name</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Lint OpenAPI Spec</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token key atrule" style="color:#3D8BFF">on</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">pull_request</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token key atrule" style="color:#3D8BFF">permissions</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">contents</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> read</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token key atrule" style="color:#3D8BFF">jobs</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">spectral-lint</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">runs-on</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> ubuntu</span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain">latest</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">steps</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain"> </span><span class="token key atrule" style="color:#3D8BFF">uses</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> actions/checkout@v4</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain"> </span><span class="token key atrule" style="color:#3D8BFF">uses</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> actions/setup</span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain">node@v4</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token key atrule" style="color:#3D8BFF">with</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token key atrule" style="color:#3D8BFF">node-version</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"22"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token key atrule" style="color:#3D8BFF">cache</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> npm</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain"> </span><span class="token key atrule" style="color:#3D8BFF">run</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> npm ci</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain"> </span><span class="token key atrule" style="color:#3D8BFF">run</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> npx spectral lint openapi.yaml </span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain">fail</span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain">severity=error </span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain">format github</span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain">actions</span><br></span></code></pre></div></div>
<p>The workflow runs on every pull request. Don't add a <code>paths</code> filter if you make it a required check: when a filtered workflow is skipped, GitHub leaves the required check pending and blocks the merge.</p>
<p>The workflow installs Spectral with <code>npm ci</code> and runs it through <code>npx</code>, so it uses the exact version recorded in <code>package-lock.json</code>. Commit <code>package.json</code> and the lock file alongside the ruleset:</p>
<div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"private"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token boolean" style="color:#FF4600">true</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"devDependencies"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"@stoplight/spectral-cli"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"6.16.3"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token punctuation" style="color:#8A93A6">}</span><br></span></code></pre></div></div>
<p><code>--format github-actions</code> turns findings into inline pull request annotations, so reviewers see each violation on the line that caused it. For GitHub code scanning, use <code>--format sarif --output spectral.sarif</code> and upload the file instead.</p>
<p>Pinning matters more once linting gates merges. A new Spectral release can add, remove, or change built-in rules and messages, and your builds would start failing (or passing) without any change to your ruleset.</p>
<p>An <code>error</code> finding makes Spectral exit with a non-zero status, which fails the step. Make the job a required status check in your branch protection rules, and a failing lint blocks the merge. <code>warn</code> findings still appear in the log and as annotations, but they don't fail the job at the default threshold.</p>
<p>Other CI platforms work the same way: run the command as a job step in <a href="https://docs.gitlab.com/ci/" target="_blank" rel="noopener noreferrer" class="">GitLab CI</a> or as a <code>run</code> step in CircleCI. The platform triggers the check, and Spectral's exit status decides the result.</p>
<p>Linting doesn't replace contract testing. Linting checks that the spec follows your rules before the API is built. Contract testing checks that the running API matches the spec. Tools such as <a href="https://github.com/schemathesis/schemathesis" target="_blank" rel="noopener noreferrer" class="">Schemathesis</a> generate requests from an OpenAPI schema and validate the responses. Use both: lint the contract, then test the implementation against it.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="beyond-the-cli-applying-api-governance-with-choreo-api-management">Beyond the CLI: Applying API governance with Choreo API Management<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#beyond-the-cli-applying-api-governance-with-choreo-api-management" class="hash-link" aria-label="Direct link to Beyond the CLI: Applying API governance with Choreo API Management" title="Direct link to Beyond the CLI: Applying API governance with Choreo API Management" translate="no">​</a></h2>
<p>CI only sees changes that go through the pipeline. Some APIs never do: they're created or edited directly in a platform console. For those, the question shifts from "did this commit pass?" to "is every API in the organization compliant?"</p>
<p><a href="https://wso2.com/api-platform/" target="_blank" rel="noopener noreferrer" class="">Choreo API Management</a> answers that with Spectral-based compliance validation. According to the <a href="https://wso2.com/api-platform/docs/cloud/governance/govern-api-proxy/" target="_blank" rel="noopener noreferrer" class="">Choreo API Management documentation</a>, rulesets are Spectral YAML rule definitions used to validate API artifacts. You attach rulesets to a policy and apply the policy globally, and the platform evaluates existing and new API proxies automatically. No one has to run a linter or wire one into a pipeline.</p>
<div class="theme-admonition theme-admonition-note admonition_xJq3 alert alert--secondary"><div class="admonitionHeading_Gvgb"><span class="admonitionIcon_Rf37"><svg viewBox="0 0 14 16"><path fill-rule="evenodd" d="M6.3 5.69a.942.942 0 0 1-.28-.7c0-.28.09-.52.28-.7.19-.18.42-.28.7-.28.28 0 .52.09.7.28.18.19.28.42.28.7 0 .28-.09.52-.28.7a1 1 0 0 1-.7.3c-.28 0-.52-.11-.7-.3zM8 7.99c-.02-.25-.11-.48-.31-.69-.2-.19-.42-.3-.69-.31H6c-.27.02-.48.13-.69.31-.2.2-.3.44-.31.69h1v3c.02.27.11.5.31.69.2.2.42.31.69.31h1c.27 0 .48-.11.69-.31.2-.19.3-.42.31-.69H8V7.98v.01zM7 2.3c-3.14 0-5.7 2.54-5.7 5.68 0 3.14 2.56 5.7 5.7 5.7s5.7-2.55 5.7-5.7c0-3.15-2.56-5.69-5.7-5.69v.01zM7 .98c3.86 0 7 3.14 7 7s-3.14 7-7 7-7-3.12-7-7 3.14-7 7-7z"></path></svg></span>What we found in testing</div><div class="admonitionContent_BuS1"><p>The platform uses the same <code>given</code>, <code>then</code>, and <code>function</code> rule model, but it evaluates rules in its own environment, so the results aren't identical to the CLI. We tested this guide's ruleset in the Choreo API Management Console and found two differences:</p><ul>
<li class="">Built-in rules from <code>extends: spectral:oas</code> didn't appear in the results. Only the custom rules did.</li>
<li class="">Importing an API added a default security requirement before the rules ran.</li>
</ul><p>Both are covered below, in <a href="https://choreolabs.com/api-management/learn/spectral-api-governance#what-the-compliance-dashboard-shows" class="">What the compliance dashboard shows</a> and <a href="https://choreolabs.com/api-management/learn/spectral-api-governance#same-rule-model-different-execution-environments" class="">Same rule model, different execution environments</a>. Treat CLI and platform results as complementary, and check your rulesets in the compliance dashboard before you rely on them.</p></div></div>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="rulesets-and-policies-in-the-console">Rulesets and policies in the console<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#rulesets-and-policies-in-the-console" class="hash-link" aria-label="Direct link to Rulesets and policies in the console" title="Direct link to Rulesets and policies in the console" translate="no">​</a></h3>
<p>In the <a href="https://console.bijira.dev/" target="_blank" rel="noopener noreferrer" class="">Choreo API Management Console</a>, governance lives under <strong>Admin &gt; Governance</strong>, on the <strong>Policies</strong> tab, in the <strong>Rulesets</strong> section. Every organization starts with three locked default rulesets, each available as a Spectral ruleset file in JSON: Choreo API Design Guidelines, Choreo Style Guidelines, and OWASP Top 10. Custom rulesets you add can be edited.</p>
<p>The Choreo API Design Guidelines ruleset has 11 rules, including <code>path-casing</code> (<code>error</code>) and <code>resource-names-plural</code> (<code>warn</code>). <code>path-casing</code> is the platform's equivalent of the <a href="https://choreolabs.com/api-management/learn/spectral-api-governance#writing-custom-rules" class="">kebab-case rule you wrote earlier</a>.</p>
<p><img decoding="async" loading="lazy" alt="Choreo API Management Console governance page showing default Spectral rulesets: Choreo API Design Guidelines, Choreo Style Guidelines, and OWASP Top 10" src="https://choreolabs.com/api-management/learn/assets/images/bijira-governance-rulesets-c675ec96e488800c8ca1b620a1a66e60.png" width="1600" height="888" class="img_ev3q"></p>
<p><strong>Add Ruleset</strong> opens a form that takes a name, an optional description, a <strong>Spectral Ruleset</strong> file (<code>.yaml</code>, <code>.yml</code>, or <code>.json</code>), an optional <strong>Document Link</strong>, and a required <strong>Provider Name</strong>. <strong>Applies To</strong> is fixed to API Definitions, and <strong>Artifact Type</strong> is set to HTTP API.</p>
<p>Upload the <a href="https://choreolabs.com/api-management/learn/spectral-api-governance#a-practical-governance-ruleset-beyond-the-defaults" class="">practical ruleset</a> from earlier, the same file the CLI uses. The console accepts it as is.</p>
<p><img decoding="async" loading="lazy" alt="Create Spectral Ruleset form in Choreo API Management Console with the practical governance ruleset YAML file uploaded" src="https://choreolabs.com/api-management/learn/assets/images/bijira-create-spectral-ruleset-form-cac3b69f9e547e0770c1b1ea4fddbab6.png" width="1600" height="1488" class="img_ev3q"></p>
<p>A ruleset takes effect only once it's attached to a <strong>Policy</strong>. The policy form sets an enforcement action for each severity: <code>error</code>, <code>warn</code>, and <code>info</code>. By default, the <strong>Enforcement Details</strong> table has an <code>Update</code> row with <code>Notify</code> selected for all three. <code>Notify</code> reports violations in the compliance dashboard without blocking the API from being created. <strong>Applicability</strong> is set to <code>Global</code>.</p>
<p><img decoding="async" loading="lazy" alt="Create Policy form in Choreo API Management Console showing severity-based enforcement (Error, Warning, Info) and ruleset selection" src="https://choreolabs.com/api-management/learn/assets/images/bijira-create-policy-form-5fdc12bb98705501c1b2bc74eaabc030.png" width="1600" height="1416" class="img_ev3q"></p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-the-compliance-dashboard-shows">What the compliance dashboard shows<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#what-the-compliance-dashboard-shows" class="hash-link" aria-label="Direct link to What the compliance dashboard shows" title="Direct link to What the compliance dashboard shows" translate="no">​</a></h3>
<p>To see the policy work, import a deliberately non-compliant definition as a new API proxy. We used a slightly larger spec than the <a href="https://choreolabs.com/api-management/learn/spectral-api-governance#running-the-ruleset" class="">CLI example</a>: a <code>/order_items</code> path with both <code>get</code> and <code>post</code> operations, no <code>operationId</code> values or descriptions, and a <code>500</code> response with no schema. The platform evaluates the API as soon as the proxy is created, with no separate linting step.</p>
<p>The <strong>Insights &gt; Compliance</strong> dashboard showed <strong>Errors (2)</strong>, <strong>Warnings (2)</strong>, <strong>Info (0)</strong>, and <strong>Passed (1)</strong>. These count rules, not individual violations, so they add up to the ruleset's five rules.</p>
<p><img decoding="async" loading="lazy" alt="Choreo API Management Compliance Dashboard showing violations detected automatically: operation-operationId-required and path-kebab-case errors from the custom ruleset" src="https://choreolabs.com/api-management/learn/assets/images/bijira-compliance-violations-a5004963be00e6ea7ea32b3bd596794b.png" width="1600" height="725" class="img_ev3q"></p>
<p>Under <strong>Errors</strong>, <code>operation-operationId-required</code> flags <code>$.paths['/order_items'].get</code> and <code>.post</code>, and <code>path-kebab-case</code> flags <code>$.paths['/order_items']</code>, each with the <code>message</code> defined in the ruleset. Under <strong>Warnings</strong>, <code>operation-description-required</code> and <code>error-response-schema</code> appear, the latter at <code>$.paths['/order_items'].get.responses['500']</code>.</p>
<p>The fifth rule, <code>operation-security-required</code>, shows as <strong>Passed</strong>. That's the result to be careful with.</p>
<div class="theme-admonition theme-admonition-note admonition_xJq3 alert alert--secondary"><div class="admonitionHeading_Gvgb"><span class="admonitionIcon_Rf37"><svg viewBox="0 0 14 16"><path fill-rule="evenodd" d="M6.3 5.69a.942.942 0 0 1-.28-.7c0-.28.09-.52.28-.7.19-.18.42-.28.7-.28.28 0 .52.09.7.28.18.19.28.42.28.7 0 .28-.09.52-.28.7a1 1 0 0 1-.7.3c-.28 0-.52-.11-.7-.3zM8 7.99c-.02-.25-.11-.48-.31-.69-.2-.19-.42-.3-.69-.31H6c-.27.02-.48.13-.69.31-.2.2-.3.44-.31.69h1v3c.02.27.11.5.31.69.2.2.42.31.69.31h1c.27 0 .48-.11.69-.31.2-.19.3-.42.31-.69H8V7.98v.01zM7 2.3c-3.14 0-5.7 2.54-5.7 5.68 0 3.14 2.56 5.7 5.7 5.7s5.7-2.55 5.7-5.7c0-3.15-2.56-5.69-5.7-5.69v.01zM7 .98c3.86 0 7 3.14 7 7s-3.14 7-7 7-7-3.12-7-7 3.14-7 7-7z"></path></svg></span>Why the security rule passed</div><div class="admonitionContent_BuS1"><p>The CLI evaluates the document exactly as written. On import, the console added a <code>default</code> security requirement to the API and to every operation, including operations that explicitly declared <code>security: []</code>. You can see it under <strong>View Source</strong> in the API proxy contract, where the API is marked <strong>Secured</strong>.</p><p>The rule runs against that enriched definition, so there's nothing to flag. The same source document fails this rule in the CLI and passes it in the platform. The <code>schema</code> function itself works in the console; what differs is the input.</p><p>As tested, <code>operation-security-required</code> can't fail in the platform, and a <strong>Passed</strong> result there doesn't mean the source spec declares security. Enforce this rule in <a href="https://choreolabs.com/api-management/learn/spectral-api-governance#running-spectral-in-cicd" class="">CI</a>, where it sees the document as written.</p></div></div>
<p>A compliant spec behaves as expected. With kebab-case paths (including an <code>/inventory-items/{itemId}</code> placeholder), an <code>operationId</code> and description on every operation, and a schema on the error response, the dashboard showed <strong>Errors (0)</strong>, <strong>Warnings (0)</strong>, and <strong>Passed (5)</strong>.</p>
<p>The dashboard also rolls compliance up to the organization, project, and component levels, by policy and, at the component level, by ruleset. Platform teams can see where they stand without opening each API. For details, see the <a href="https://wso2.com/api-platform/docs/cloud/monitoring-and-insights/compliance/" target="_blank" rel="noopener noreferrer" class="">compliance documentation</a> and Choreo Labs' overview of <a class="" href="https://choreolabs.com/api-management/learn/api-governance/">the broader API governance program</a>.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="same-rule-model-different-execution-environments">Same rule model, different execution environments<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#same-rule-model-different-execution-environments" class="hash-link" aria-label="Direct link to Same rule model, different execution environments" title="Direct link to Same rule model, different execution environments" translate="no">​</a></h2>
<p>You can write a ruleset once and run it in three places, but each place evaluates something different, so don't expect identical findings:</p>
<table><thead><tr><th style="text-align:left">Environment</th><th style="text-align:left">What is evaluated</th><th style="text-align:left">Main purpose</th></tr></thead><tbody><tr><td style="text-align:left">Spectral CLI</td><td style="text-align:left">The API description you supply</td><td style="text-align:left">Local development and CI</td></tr><tr><td style="text-align:left">API Designer</td><td style="text-align:left">The spec you're editing in VS Code</td><td style="text-align:left">Early design feedback</td></tr><tr><td style="text-align:left">Choreo API Management</td><td style="text-align:left">The API definition after platform/import processing</td><td style="text-align:left">Organization-wide compliance reporting</td></tr></tbody></table>
<p>The differences come from three sources. The input: the console added a default security requirement on import. The ruleset: the <code>spectral:oas</code> built-ins didn't appear in the console. The outcome: a non-zero exit code in CI, a <code>Notify</code> result in the dashboard, and a score in API Designer. Reference handling, on the other hand, was consistent: an internal <code>$ref</code> to a response was evaluated at the referenced location in both the CLI and the console.</p>
<p>Test your rules in every environment you rely on.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="spectral-vs-other-openapi-linters">Spectral vs. other OpenAPI linters<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#spectral-vs-other-openapi-linters" class="hash-link" aria-label="Direct link to Spectral vs. other OpenAPI linters" title="Direct link to Spectral vs. other OpenAPI linters" translate="no">​</a></h2>
<p>Spectral fits this workflow well, but it isn't the only option:</p>
<table><thead><tr><th style="text-align:left">Tool</th><th style="text-align:left">Language/Runtime</th><th style="text-align:left">Custom Rules</th><th style="text-align:left">Best For</th></tr></thead><tbody><tr><td style="text-align:left"><strong>Spectral</strong></td><td style="text-align:left">Node.js (JSONPath Plus)</td><td style="text-align:left">Full: built-in functions + custom JavaScript</td><td style="text-align:left">General-purpose OpenAPI/AsyncAPI linting</td></tr><tr><td style="text-align:left"><strong>Vacuum</strong></td><td style="text-align:left">Go (single binary; also available via Homebrew, npm, and Docker)</td><td style="text-align:left">Reads Spectral rulesets, plus custom functions in JavaScript or Go</td><td style="text-align:left">Performance-focused linting that reuses existing Spectral rulesets</td></tr><tr><td style="text-align:left"><strong>Speakeasy's linter</strong></td><td style="text-align:left">Go, built on Speakeasy's open source OpenAPI library</td><td style="text-align:left">Custom rules in TypeScript/JavaScript, configured in Speakeasy's own <code>lint.yaml</code> format</td><td style="text-align:left">Teams generating SDKs, where a spec has to be "generation-safe," not just spec-valid</td></tr></tbody></table>
<p>Vacuum is a Go-based linter focused on speed that reads Spectral rulesets, so it's worth benchmarking if you have very large specs or slow CI. Speakeasy's linter is built around SDK-generation readiness, with 90+ built-in rules according to its documentation. It uses its own rule format, so Spectral rulesets don't carry over. To evaluate them, see the <a href="https://github.com/daveshanley/vacuum" target="_blank" rel="noopener noreferrer" class="">Vacuum repository</a> and the <a href="https://www.speakeasy.com/docs/sdks/prep-openapi/linting" target="_blank" rel="noopener noreferrer" class="">Speakeasy linter documentation</a>.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>A Spectral ruleset is your API design guidelines in executable form. Write it once and you can run it at three points: in the editor for early feedback, in CI to block non-compliant changes, and in a governance platform to monitor every API in the organization, including the ones that never pass through your pipeline.</p>
<p>Each environment evaluates rules a little differently, so test your rules where you use them. Start small: take the five rules from this guide, run them as <code>warn</code> against your existing specs, and promote them to <code>error</code> as the findings get fixed.</p>
<p>If you already lint in CI, <a href="https://wso2.com/api-platform/" target="_blank" rel="noopener noreferrer" class="">Choreo API Management</a> accepts the same ruleset file and reports compliance across your organization. Keep CI as the gate for rules such as <code>operation-security-required</code> that depend on the source document as written.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/spectral-api-governance#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<p><strong>What's the difference between Spectral and a validator?</strong></p>
<p>A validator checks that a document conforms to a specification: required fields are present, types are correct, and references resolve. Spectral can run structural checks too, but its value here is enforcing design rules on top of a valid document, such as naming conventions, required metadata, and organizational standards.</p>
<p><strong>Can Spectral lint AsyncAPI?</strong></p>
<p>Yes. Spectral ships a <code>spectral:asyncapi</code> ruleset alongside <code>spectral:oas</code>, and custom rules use the same <code>given</code>, <code>then</code>, and <code>function</code> syntax, targeting AsyncAPI's document structure.</p>
<p><strong>Does Spectral resolve <code>$ref</code> before linting?</strong></p>
<p>Yes, including references to external files. What a rule evaluates, and where a finding is reported, depends on how the rule targets the document.</p>
<p>For example, in Spectral 6.16.3 the <code>error-response-schema</code> rule follows a 500 response declared as <code>$ref: '#/components/responses/InternalError'</code>. If <code>InternalError</code> has an <code>application/json</code> schema, the rule passes. If it doesn't, Spectral reports the finding where <code>InternalError</code> is defined, not on the line with the <code>$ref</code>. External references such as <code>./responses.yaml#/InternalError</code> behave the same way, and so did an internal response <code>$ref</code> in Choreo API Management.</p>
<p>Results can depend on document structure and Spectral version, so test your rules against the file layouts in your own repositories.</p>
<p><strong>Will the same ruleset give the same results everywhere?</strong></p>
<p>Not necessarily. The input, the available built-in rules, and the enforcement behavior differ between the Spectral CLI, API Designer, and Choreo API Management. See <a href="https://choreolabs.com/api-management/learn/spectral-api-governance#same-rule-model-different-execution-environments" class="">Same rule model, different execution environments</a>.</p>
<p><strong>Is Spectral free to use?</strong></p>
<p>Yes. Spectral is open source under the Apache 2.0 license, and the CLI has no paid tier.</p>
<p><strong>Can I lint from an editor instead of the terminal?</strong></p>
<p>Yes. Stoplight's <a href="https://marketplace.visualstudio.com/items?itemName=stoplight.spectral" target="_blank" rel="noopener noreferrer" class="">Spectral extension for VS Code</a> lints as you type or on save, depending on its configuration. Choreo API Management's <a href="https://marketplace.visualstudio.com/items?itemName=WSO2.api-designer" target="_blank" rel="noopener noreferrer" class="">API Designer extension</a> also runs <a href="https://wso2.com/api-platform/docs/tools/vscode-api-design/govern-apis/" target="_blank" rel="noopener noreferrer" class="">governance checks using Spectral rulesets and AI</a>, with reports for the Choreo REST API AI Readiness Guidelines, Choreo REST API Design Guidelines, and OWASP API Security Top 10.</p>]]></content>
        <author>
            <name>Dakshitha Suriyaaratchie</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="Tutorials" term="Tutorials"/>
        <category label="API governance" term="API governance"/>
        <category label="Policies" term="Policies"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[Using Amazon Bedrock as an LLM provider and Bedrock Guardrails with Choreo AI Gateway]]></title>
        <id>https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway</id>
        <link href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway"/>
        <updated>2026-10-15T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[A full walkthrough registering Amazon Bedrock as a provider and attaching its native guardrail to a Choreo AI Gateway, with both requests and responses validated.]]></summary>
        <content type="html"><![CDATA[<p>Amazon Bedrock provides access to foundation models from multiple providers through a single set of APIs. Along with model invocation, Bedrock provides native capabilities such as <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html" target="_blank" rel="noopener noreferrer" class="">Amazon Bedrock Guardrails</a> for controlling model interactions through content filtering, denied topics, and sensitive-information filtering.</p>
<p>Although applications can use Amazon Bedrock directly, doing so does not provide an application-facing layer for managing separate application credentials and policies, or for presenting a consistent API surface across multiple applications. To provide this layer while continuing to use Bedrock's native capabilities, Choreo AI Gateway can be placed in front of Bedrock.</p>
<p>Rather than replacing the capabilities provided by Amazon Bedrock, Choreo AI Gateway complements them by providing an application-facing control layer above the AWS account boundary. Choreo Agent Control Plane is the control plane where you configure that layer: you register a Bedrock provider once, can attach guardrails and policies directly to it, and either call it directly or front it with one or more App LLM Proxies, each with its own client credentials, policies, and guardrail configuration. Choreo Agent Control Plane doesn't sit in the request path itself; it manages configuration and deploys it to the AI Gateway, which is the component that actually routes traffic to Bedrock.</p>
<p>In this guide, you configure Amazon Bedrock as an LLM provider in Choreo Agent Control Plane, attach an Amazon Bedrock Guardrail directly to that provider so it validates both incoming requests and outgoing responses, deploy that configuration to your AI Gateway, and verify the setup with real requests against an Amazon Bedrock model, calling the provider directly with no proxy involved. A later, optional section covers adding an App LLM Proxy on top, for when a specific application needs its own credentials or guardrail; see <a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#why-put-an-ai-gateway-in-front-of-bedrock" class="">Why put an AI gateway in front of Bedrock?</a> for when you need one.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="why-put-an-ai-gateway-in-front-of-bedrock">Why put an AI gateway in front of Bedrock?<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#why-put-an-ai-gateway-in-front-of-bedrock" class="hash-link" aria-label="Direct link to Why put an AI gateway in front of Bedrock?" title="Direct link to Why put an AI gateway in front of Bedrock?" translate="no">​</a></h2>
<p>Applications can interact with Amazon Bedrock directly, but when multiple applications consume the same AI services, managing access and policies independently for each application can become difficult. An AI Gateway provides a dedicated application-facing layer that separates application access from the underlying Bedrock provider — that gateway is the data plane here: once you deploy your configuration to it, it's the component that actually sits in front of Bedrock and handles every request and response.</p>
<p>Choreo Agent Control Plane, by contrast, is the control plane you use to configure that gateway. It's where you register the Bedrock provider, configure guardrails and policies, create proxies, and deploy configuration — but it never sees live application traffic itself; only the deployed AI Gateway does.</p>
<p>With Choreo AI Gateway in front of Amazon Bedrock, you can:</p>
<ul>
<li class=""><strong>Attach a guardrail once, for every caller:</strong> A Bedrock Guardrail can be attached directly to the LLM provider, so it validates every request and response that goes through that provider, including requests that arrive through any App LLM Proxy that uses it. No proxy is required.</li>
<li class=""><strong>Give each application its own key without touching Bedrock credentials:</strong> You can generate several API keys on the same provider, one per application, and group them under a GenAI Application — a distinct Choreo Agent Control Plane feature for attributing usage to the app that generated the key. Applications never handle the Bedrock credential.</li>
<li class=""><strong>Call the provider directly, and add a proxy only where you need one:</strong> A deployed provider can be called directly through the gateway. Add an App LLM Proxy only when a specific application or agent needs its own guardrails, policies, exposed resources, or authentication settings on top of the provider.</li>
<li class=""><strong>Reuse a single Bedrock provider:</strong> Whether you call the provider directly or front it with proxies, the same Bedrock provider configuration serves every caller instead of being configured separately for each one.</li>
<li class=""><strong>Provide a consistent API surface:</strong> Applications interact with Choreo AI Gateway, whether through a provider or a proxy, while the gateway handles communication with the underlying Bedrock service.</li>
</ul>
<p>This approach allows <strong>Amazon Bedrock to remain responsible for model access and its native AI capabilities, while Choreo AI Gateway provides the application-facing layer for access control and policies, with optional per-application proxies on top.</strong> Choreo Agent Control Plane is where you configure all of this; it's the control plane.</p>
<p>A single Bedrock provider registration can serve multiple proxies, each with its own client credentials, policies, and guardrail configuration on top of the provider's own baseline. The <a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#optional-scope-guardrails-and-credentials-per-application-with-an-app-llm-proxy" class="">optional section</a> later in this guide configures one proxy on top of the provider you build here, to show this per-application pattern.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="architecture">Architecture<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#architecture" class="hash-link" aria-label="Direct link to Architecture" title="Direct link to Architecture" translate="no">​</a></h2>
<p>The following diagram shows how the application, the AI Gateway, Amazon Bedrock, and the Bedrock Guardrail interact:</p>
<p><img decoding="async" loading="lazy" alt="Architecture: the application calls the LLM Provider on Choreo AI Gateway directly with a client-facing API key (1) and receives the response; the provider calls the Bedrock foundation model via the Converse API (2) and receives the model response; the guardrail policy attached to the provider calls Bedrock Guardrails via ApplyGuardrail (3) and receives the guardrail result" src="https://choreolabs.com/api-management/learn/assets/images/02-architecture-5ff77d4a2dd72556482bb3b40878fd56.webp" width="1280" height="610" class="img_ev3q"></p>
<p>This architecture lets any number of applications call the same Bedrock provider while the gateway layer handles authentication, the guardrail, and policies on their behalf.</p>
<p>Three credential paths are shown in the diagram:</p>
<ol>
<li class=""><strong>Application to Gateway:</strong> The application sends the client-facing <code>X-API-Key</code> issued by the provider. In this walkthrough, this is the only credential the application needs; it never sees the AWS credentials used by the gateway.</li>
<li class=""><strong>Gateway to Bedrock (model calls):</strong> The LLM provider uses the Bedrock credential configured when the provider is added. In this walkthrough, that credential is a Bedrock API key.</li>
<li class=""><strong>Gateway to Bedrock (guardrail calls):</strong> The AWS Bedrock Guardrail policy uses AWS credentials that allow <code>bedrock:ApplyGuardrail</code>. In the configuration demonstrated here, those credentials are set on the guardrail attachment, separately from the provider's credential.</li>
</ol>
<p>If you later add an App LLM Proxy on top of this provider (see the <a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#optional-scope-guardrails-and-credentials-per-application-with-an-app-llm-proxy" class="">optional section</a>), the application's credential path changes: it authenticates to the proxy instead of to the provider directly, and the proxy in turn authenticates to the provider with its own credential. The two Gateway-to-Bedrock paths described below are unaffected either way.</p>
<p>This tutorial configures two separate credentials: the Bedrock API key on the provider for model calls, and an IAM access key on the guardrail policy for <code>ApplyGuardrail</code> calls. AWS doesn't require two identities. One IAM identity can be granted both <code>bedrock:InvokeModel</code> and <code>bedrock:ApplyGuardrail</code>.</p>
<p>Request-side and response-side validation are independently configurable in the Choreo AI Gateway AWS Bedrock Guardrail policy. When both are enabled, the gateway evaluates the request before sending it to Bedrock and evaluates the response before returning it to the application.</p>
<p>This is also an important distinction from native Bedrock Converse behavior. In the Choreo AI Gateway policy implementation used in this guide, request and response validation are performed through the corresponding <code>ApplyGuardrail</code> operations. Native Bedrock Converse instead accepts guardrail configuration as part of the model invocation itself and evaluates the model interaction within that API call. The gateway therefore provides an application-facing policy layer around the Bedrock capability rather than reimplementing the guardrail evaluation itself.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="amazon-bedrock-and-choreo-ai-gateway-responsibilities">Amazon Bedrock and Choreo AI Gateway: Responsibilities<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#amazon-bedrock-and-choreo-ai-gateway-responsibilities" class="hash-link" aria-label="Direct link to Amazon Bedrock and Choreo AI Gateway: Responsibilities" title="Direct link to Amazon Bedrock and Choreo AI Gateway: Responsibilities" translate="no">​</a></h2>
<table><thead><tr><th>Capability / Responsibility</th><th>Amazon Bedrock</th><th>Choreo AI Gateway</th></tr></thead><tbody><tr><td>Foundation model access</td><td>Yes: access to foundation models from multiple providers through Amazon Bedrock APIs</td><td>Routes application traffic to the provider; adds no model capability of its own</td></tr><tr><td>Model invocation (Converse / Converse Stream)</td><td>Yes, both synchronous and streaming responses</td><td>Exposes both as OpenAPI resources on the provider (<code>/converse</code>, <code>/converse-stream</code>); forwards the call, doesn't reimplement it</td></tr><tr><td>Amazon Bedrock Guardrails</td><td>Yes: content filters, denied topics, word filters, sensitive-information filters, and more, depending on the Guardrails feature and invocation path. Can be <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-permissions-id.html" target="_blank" rel="noopener noreferrer" class="">enforced per IAM principal</a> or <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-enforcements.html" target="_blank" rel="noopener noreferrer" class="">automatically at the AWS account or Organization level</a></td><td>Can attach the same guardrail as a policy on the LLM provider (applied to every caller) or on a specific App LLM Proxy (applied only to calls on that proxy), on whichever of the request/response phases is enabled</td></tr><tr><td>Gateway-native policies</td><td>Not applicable to Amazon Bedrock itself</td><td>Choreo AI Gateway provides additional gateway-level guardrail and governance policies that can be attached alongside or instead of the Bedrock guardrail. See the <a href="https://wso2.com/api-platform/docs/ai-gateway/1.2.0/guardrails/guardrails-catalogue/" target="_blank" rel="noopener noreferrer" class="">guardrails catalogue</a></td></tr><tr><td>Client authentication</td><td>IAM SigV4, or a long-term or short-term Bedrock API key, all authenticated directly against AWS</td><td>A client-facing API key per provider, or per proxy if you add one, so the application authenticates to the gateway rather than to AWS directly</td></tr><tr><td>Usage attribution across applications</td><td>When a gateway uses a shared AWS identity to invoke Bedrock, Bedrock-side usage is attributed to that identity; AWS also provides finer-grained, application-level attribution mechanisms of its own</td><td>Per-application and per-consumer breakdowns in Moesif (linked from Choreo Agent Control Plane's Insights page) when the gateway is configured with a Moesif key; group API keys under a GenAI Application to attribute usage per app</td></tr></tbody></table>
<p>Choreo AI Gateway also supports multi-provider routing and cost-control policies on the same provider or proxy, alongside a Bedrock guardrail. Both are outside the scope of this walkthrough, but they show the gateway layer can do more than route traffic and enforce Bedrock's own guardrail.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="amazon-bedrock-capabilities-vs-choreo-ai-gateway-capabilities">Amazon Bedrock capabilities vs. Choreo AI Gateway capabilities<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#amazon-bedrock-capabilities-vs-choreo-ai-gateway-capabilities" class="hash-link" aria-label="Direct link to Amazon Bedrock capabilities vs. Choreo AI Gateway capabilities" title="Direct link to Amazon Bedrock capabilities vs. Choreo AI Gateway capabilities" translate="no">​</a></h3>
<p>It is worth being explicit about the relationship between the two layers. Choreo AI Gateway does not replace capabilities that Amazon Bedrock already provides. Bedrock has its own authentication and authorization mechanisms, Bedrock API keys, native Guardrails, inference profiles, and AWS-native usage and cost attribution mechanisms, independent of any gateway in front of it.
Choreo AI Gateway adds an application-facing control layer alongside those capabilities. In this walkthrough, that layer issues each caller its own client-facing key, while applying one shared guardrail and policy configuration, a common API surface, and shared usage visibility across every caller of the provider. The two layers are complementary: an organization can use Bedrock's IAM and account-level controls at the AWS layer while using Choreo AI Gateway's credentials and policies at the application layer.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-youll-build">What you'll build<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#what-youll-build" class="hash-link" aria-label="Direct link to What you'll build" title="Direct link to What you'll build" translate="no">​</a></h2>
<p>By the end of this guide, you'll have:</p>
<ul>
<li class="">An Amazon Bedrock LLM provider in Choreo Agent Control Plane, deployed to your AI Gateway.</li>
<li class="">An Amazon Bedrock Guardrail attached directly to that provider, with request-side and response-side validation enabled.</li>
<li class="">A client-facing API key for calling the provider, with no proxy required.</li>
<li class="">Test results that confirm the configuration works end to end.</li>
<li class="">An understanding of when to add an optional App LLM Proxy on top, for an application that needs its own credentials or guardrail.</li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="prerequisites">Prerequisites<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#prerequisites" class="hash-link" aria-label="Direct link to Prerequisites" title="Direct link to Prerequisites" translate="no">​</a></h2>
<ul>
<li class="">A running Choreo Agent Control Plane instance with a project (the <code>Default</code> project works).</li>
<li class="">An AI Gateway that appears as <strong>Active</strong> in Choreo Agent Control Plane. <a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#step-1-set-up-an-ai-gateway" class="">Step 1</a> covers this.</li>
<li class=""><code>curl</code>.</li>
<li class="">An AWS account with access to a Bedrock model in your Region, and the three AWS items described in <a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#prepare-aws" class="">Prepare AWS</a>.</li>
</ul>
<blockquote>
<p><strong>Note:</strong> This guide was tested with AI Workspace 1.0.0, AI Gateway 1.2.0, and the <code>aws-bedrock-guardrail</code> policy version 1.2.0. Policy parameters can change between policy versions. For the parameters, defaults, and supported authentication modes for the version your gateway registers, see the <a href="https://wso2.com/api-platform/policy-hub/policies/aws-bedrock-guardrail" target="_blank" rel="noopener noreferrer" class="">Policy Hub</a> page.</p>
</blockquote>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="prepare-aws">Prepare AWS<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#prepare-aws" class="hash-link" aria-label="Direct link to Prepare AWS" title="Direct link to Prepare AWS" translate="no">​</a></h3>
<p>You set up three things in AWS. The linked AWS guides give the exact, current steps, so this guide doesn't repeat them.</p>
<ol>
<li class=""><strong>An Amazon Bedrock Guardrail</strong> configured for input and output evaluation, with a numbered version.<!-- -->
<ul>
<li class="">For this guide, add a denied topic (for example, investment advice) and a sensitive-information filter for email addresses with the action set to <strong>Mask</strong>. These settings demonstrate blocking and masking. They aren't required by Bedrock or by Choreo AI Gateway.</li>
<li class="">Note the guardrail ID, the guardrail version, and the Region.</li>
<li class="">See <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html" target="_blank" rel="noopener noreferrer" class="">Amazon Bedrock Guardrails</a>.</li>
</ul>
</li>
<li class=""><strong>An Amazon Bedrock API key</strong> for the same Region.<!-- -->
<ul>
<li class="">See <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys-generate.html" target="_blank" rel="noopener noreferrer" class="">Generate an Amazon Bedrock API key</a>.</li>
<li class="">AWS describes the key types and when to use each in <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html" target="_blank" rel="noopener noreferrer" class="">Amazon Bedrock API keys</a>. For anything beyond experimentation, follow the AWS recommendation for production.</li>
</ul>
</li>
<li class=""><strong>AWS credentials for the gateway</strong> that are allowed to call the guardrail (<code>bedrock:ApplyGuardrail</code>).<!-- -->
<ul>
<li class="">See <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-permissions.html" target="_blank" rel="noopener noreferrer" class="">Permissions for Amazon Bedrock Guardrails</a>.</li>
<li class="">Amazon Bedrock doesn't require separate identities for model calls and guardrail checks. This guide configures them separately because the provider and the guardrail policy are configured separately.</li>
</ul>
</li>
</ol>
<h1>Tutorial: Registering Amazon Bedrock and attaching its guardrail</h1>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-1-set-up-an-ai-gateway">Step 1: Set up an AI gateway<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#step-1-set-up-an-ai-gateway" class="hash-link" aria-label="Direct link to Step 1: Set up an AI gateway" title="Direct link to Step 1: Set up an AI gateway" translate="no">​</a></h2>
<p>If you already have an AI Gateway that shows <strong>Active</strong> in Choreo Agent Control Plane, skip to <a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#step-2-verify-your-bedrock-api-key" class="">Step 2</a>.</p>
<ol>
<li class="">In Choreo Agent Control Plane, open <strong>AI Gateways</strong> and click <strong>+ Add AI Gateway</strong>.</li>
<li class="">Enter a <strong>Name</strong> and keep the default <strong>Gateway Version</strong>. Use the <strong>URL</strong> where the gateway runtime is reachable.</li>
<li class="">Click <strong>Add Gateway</strong>. The gateway detail page opens with a <strong>Get Started</strong> section that has setup commands for your deployment option.</li>
<li class="">Follow the commands in <strong>Get Started</strong> to download, configure, and start the gateway.</li>
</ol>
<p>The configure command in <strong>Get Started</strong> shows the registration token as the placeholder <code>&lt;your-gateway-token&gt;</code>. Use the <strong>Copy</strong> button next to the command. It copies the command with the real token.</p>
<p>For the full procedure and the other deployment options, see <a href="https://wso2.com/api-platform/docs/ai-workspace/1.0.0/ai-gateways/setting-up/" target="_blank" rel="noopener noreferrer" class="">Set up an AI Gateway</a>.</p>
<p>When the gateway connects, Choreo Agent Control Plane shows it as configured:</p>
<p><img decoding="async" loading="lazy" alt="Gateway detail page showing &amp;quot;2/2 Steps: You Successfully Configured Your bedrock-ai-gateway-v2 Gateway &amp;amp; it&amp;#39;s Active now&amp;quot; with the Quick Start setup panel below" src="https://choreolabs.com/api-management/learn/assets/images/03-gateway-connected-successfully-6637552d3663d93dfeba74fd72b4e940.png" width="2516" height="1569" class="img_ev3q"></p>
<p>The gateway status on the <strong>AI Gateways</strong> page changes to <strong>Active</strong>:</p>
<p><img decoding="async" loading="lazy" alt="AI Gateways list showing bedrock-ai-gateway-v2 with Status Active" src="https://choreolabs.com/api-management/learn/assets/images/04-gateway-active-status-98c3f4b7b4ec4ccd5e3d6a88d600f2d6.png" width="2516" height="1569" class="img_ev3q"></p>
<p>To confirm which policy version your gateway runs, check the gateway runtime log (for example, with <code>docker compose logs gateway-runtime</code>) for a line like the following, where <code>&lt;VERSION&gt;</code> is the version your gateway registered:</p>
<div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">Registered policy aws-bedrock-guardrail version &lt;VERSION&gt;</span><br></span></code></pre></div></div>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-2-verify-your-bedrock-api-key">Step 2: Verify your Bedrock API key<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#step-2-verify-your-bedrock-api-key" class="hash-link" aria-label="Direct link to Step 2: Verify your Bedrock API key" title="Direct link to Step 2: Verify your Bedrock API key" translate="no">​</a></h2>
<p>Before you configure Choreo Agent Control Plane, confirm that the key works against the model you plan to use:</p>
<div class="language-bash codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-bash codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">curl -s -X POST "https://bedrock-runtime.&lt;REGION&gt;.amazonaws.com/model/&lt;MODEL-ID&gt;/converse" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  -H "Content-Type: application/json" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  -H "Authorization: Bearer &lt;BEDROCK-API-KEY&gt;" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  -d '{"messages":[{"role":"user","content":[{"text":"hi"}]}],"inferenceConfig":{"maxTokens":16}}'</span><br></span></code></pre></div></div>
<p>A successful call returns a JSON response with the model's reply.</p>
<p>Use the model ID or inference profile ID that's available for your Region, for example <code>eu.amazon.nova-micro-v1:0</code>. Use the same ID in the rest of this guide.</p>
<p>Keep the raw key available for the next step.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-3-add-amazon-bedrock-as-an-llm-provider-and-attach-its-guardrail">Step 3: Add Amazon Bedrock as an LLM provider and attach its guardrail<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#step-3-add-amazon-bedrock-as-an-llm-provider-and-attach-its-guardrail" class="hash-link" aria-label="Direct link to Step 3: Add Amazon Bedrock as an LLM provider and attach its guardrail" title="Direct link to Step 3: Add Amazon Bedrock as an LLM provider and attach its guardrail" translate="no">​</a></h2>
<blockquote>
<p><strong>Note:</strong> The parameter names, defaults, and <code>awsAuth</code> modes for the guardrail policy in this step match <code>aws-bedrock-guardrail</code> version 1.2.0, which this guide was tested with. Before you configure the policy, check the <a href="https://wso2.com/api-platform/policy-hub/policies/aws-bedrock-guardrail" target="_blank" rel="noopener noreferrer" class="">Policy Hub</a> page for the exact configuration of the version your gateway registers.</p>
</blockquote>
<ol>
<li class="">
<p>In Choreo Agent Control Plane, open <strong>LLM Providers</strong> and start the create-provider flow. The <strong>Add LLM Provider</strong> page lists the supported providers.</p>
</li>
<li class="">
<p>Select <strong>AWS Bedrock</strong>.</p>
</li>
<li class="">
<p>Enter the following:</p>
<ul>
<li class=""><strong>Name:</strong> a name for the provider.</li>
<li class=""><strong>Upstream URL:</strong> the Bedrock Runtime endpoint for your Region, for example <code>https://bedrock-runtime.&lt;REGION&gt;.amazonaws.com</code>.</li>
<li class=""><strong>API Key:</strong> the Bedrock API key from Step 2. Paste the raw key. Choreo Agent Control Plane adds the <code>Bearer</code> prefix and sends the credential as <code>Authorization: Bearer &lt;key&gt;</code>.</li>
</ul>
<p>The form also has <strong>Version</strong>, <strong>Description</strong>, and <strong>Context</strong> fields. You can keep the defaults.</p>
</li>
<li class="">
<p>In the <strong>Guardrails</strong> section of the same form, click <strong>+ Add Guardrail</strong> and select <strong>AWS Bedrock Guardrail</strong> from the <a href="https://wso2.com/api-platform/docs/ai-gateway/1.2.0/guardrails/guardrails-catalogue/" target="_blank" rel="noopener noreferrer" class="">guardrail catalog</a>.</p>
</li>
<li class="">
<p>Enter the guardrail details from <a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#prepare-aws" class="">Prepare AWS</a>:</p>
<ul>
<li class=""><code>region</code></li>
<li class=""><code>guardrailID</code></li>
<li class=""><code>guardrailVersion</code></li>
</ul>
<p><img decoding="async" loading="lazy" alt="AWS Bedrock Guardrail configuration panel on the provider&amp;#39;s Add Guardrail form, showing the region, guardrailID, and guardrailVersion fields filled in, each marked Optional with helper text about the gateway-wide system parameter fallback, plus the deprecated localGuardrailID/localGuardrailVersion fields" src="https://choreolabs.com/api-management/learn/assets/images/14-guardrail-config-fields-provider-bb46a248aa93446b8916fb2aff5441ba.png" width="1280" height="1100" class="img_ev3q"></p>
</li>
<li class="">
<p>Configure the two directions. The policy has separate <strong>request</strong> and <strong>response</strong> sections:</p>
<ul>
<li class=""><strong>request:</strong> validation is enabled by default.</li>
<li class=""><strong>response:</strong> validation is <strong>disabled</strong> by default. Expand <strong>response</strong> and set <strong>enabled</strong> to <code>true</code> so the guardrail checks both directions.</li>
</ul>
</li>
<li class="">
<p>Set the <strong>jsonPath</strong> for each section to match the Bedrock Converse API's message structure:</p>
<div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">request.jsonPath  = $.messages[-1].content[0].text</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">response.jsonPath = $.output.message.content[0].text</span><br></span></code></pre></div></div>
<p>The request JSONPath checks only the first text block of the latest message. A denied topic in the system prompt, in a second content block, or in an earlier turn isn't checked on the request side. Response validation checks the model's reply, so it can still catch those topics. Don't widen the request path to <code>$.messages[*]</code>: in testing, the gateway returned <code>HTTP 422</code> with <code>Error extracting value from JSONPath</code>, because the result is an array rather than a single string.</p>
</li>
<li class="">
<p>Under <strong>Advanced Settings</strong>, set <strong>awsAuth</strong> → <strong>authenticationType</strong> to <code>iam-user-access-key</code>. Enter the <strong>awsAccessKeyID</strong> and <strong>awsSecretAccessKey</strong> for an IAM identity that can call <code>bedrock:ApplyGuardrail</code> (see <a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#prepare-aws" class="">Prepare AWS</a>). The policy stores these values in the API definition. For a shared or production gateway, use a role-based authentication mode instead; see <a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#production-considerations" class="">Production considerations</a>.</p>
</li>
<li class="">
<p>Click <strong>Add</strong> to attach the guardrail to the form, then click <strong>Add Provider</strong> to save the provider.</p>
</li>
</ol>
<p>The provider's <strong>Guardrails &amp; Policies</strong> tab now shows the attached policy, applied globally so it covers every resource the provider exposes:</p>
<p><img decoding="async" loading="lazy" alt="Guardrails &amp;amp; Policies tab on the provider showing the AWS Bedrock Guardrail (v1) chip under Global Guardrails &amp;amp; Policies, alongside the default LLM Cost policy" src="https://choreolabs.com/api-management/learn/assets/images/15-guardrail-attached-provider-71d2073e0f3c47ff047f6e20682040dd.png" width="1280" height="1100" class="img_ev3q"></p>
<p>You don't need to set the other guardrail parameters for this guide. The <a href="https://wso2.com/api-platform/policy-hub/policies/aws-bedrock-guardrail" target="_blank" rel="noopener noreferrer" class="">Policy Hub</a> page lists every parameter, its default, and the supported AWS authentication modes for your policy version. For more on applying policies in Choreo Agent Control Plane, see <a href="https://wso2.com/api-platform/docs/ai-workspace/1.0.0/policies/overview/" target="_blank" rel="noopener noreferrer" class="">Policies overview</a>.</p>
<blockquote>
<p><strong>Note:</strong> Response validation evaluates a complete JSON response, and <code>/converse-stream</code> returns a sequence of events rather than one JSON document. The policy docs don't address this case directly; in testing, disabling response validation for that resource, or setting <code>passthroughOnError</code> to <code>true</code> for the response, avoided validation errors on streamed output. Verify this against your own gateway version before relying on it in production.</p>
</blockquote>
<p>Now open the provider's <strong>Connection</strong> tab. It shows the Region-specific endpoint and the authentication settings:</p>
<p><img decoding="async" loading="lazy" alt="Provider Connection tab showing Provider Endpoint https://bedrock-runtime.eu-north-1.amazonaws.com, Authentication api-key, and Authentication Header Authorization" src="https://choreolabs.com/api-management/learn/assets/images/05-provider-connection-filled-a0fe419340618174cb0aca61c5663c59.png" width="1280" height="1100" class="img_ev3q"></p>
<p>For more details, see <a href="https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-providers/configure-provider/" target="_blank" rel="noopener noreferrer" class="">Configure an LLM provider</a> and the <a href="https://wso2.com/api-platform/docs/ai-gateway/1.2.0/gateway-artifacts/llm-provider/supported-providers/aws-bedrock/" target="_blank" rel="noopener noreferrer" class="">AWS Bedrock provider reference</a>.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-4-deploy-the-provider-and-generate-a-client-api-key">Step 4: Deploy the provider and generate a client API key<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#step-4-deploy-the-provider-and-generate-a-client-api-key" class="hash-link" aria-label="Direct link to Step 4: Deploy the provider and generate a client API key" title="Direct link to Step 4: Deploy the provider and generate a client API key" translate="no">​</a></h2>
<ol>
<li class="">
<p>Click <strong>Deploy to Gateway</strong>, then deploy the provider to the gateway from Step 1. Wait until the deployment status is <strong>Active</strong>.</p>
<p>Deploy the provider again whenever you change its configuration, including its guardrail. Saved changes reach the running gateway only through a new deployment. See <a href="https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-providers/manage-provider/" target="_blank" rel="noopener noreferrer" class="">Manage an LLM provider</a>.</p>
</li>
<li class="">
<p>Open the provider's <strong>Overview</strong> tab. After deployment, it shows the OpenAPI resources it exposes, the <strong>Invoke URL</strong> for the gateway, and an <strong>API Keys</strong> panel:</p>
<ul>
<li class=""><code>POST /model/{modelId}/converse</code></li>
<li class=""><code>POST /model/{modelId}/converse-stream</code></li>
</ul>
<p>This guide tests <code>/converse</code>. For how response validation affects <code>/converse-stream</code>, see the troubleshooting table.</p>
<p><img decoding="async" loading="lazy" alt="Provider Overview tab showing OpenAPI Resources for POST /model/modelId/converse and POST /model/modelId/converse-stream, the real Invoke URL, and the API Keys panel" src="https://choreolabs.com/api-management/learn/assets/images/13-provider-overview-invoke-url-1f595a63bf815ba45a682796f2767b34.png" width="1280" height="637" class="img_ev3q"></p>
</li>
<li class="">
<p>Click <strong>Generate API Key</strong>, give the key a name, and copy the key right away. Choreo Agent Control Plane shows it only once, in a modal with a ready-to-run <code>curl</code> command next to it. Applications send it in the <code>X-API-Key</code> header.</p>
<p>This key expires 90 days after you generate it. Choreo Agent Control Plane lists the exact expiration date in the <strong>API Keys</strong> table on this tab. Plan to rotate it before then; see <a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#production-considerations" class="">Production considerations</a>.</p>
</li>
</ol>
<p>This is the only key your application needs. It authenticates directly to the provider; there's no proxy, and no second key, in between. In testing, this key persisted across redeployments, so regenerating it wasn't necessary after updating the provider's configuration; confirm this holds for your gateway version before relying on it.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-5-verify-the-configuration">Step 5: Verify the configuration<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#step-5-verify-the-configuration" class="hash-link" aria-label="Direct link to Step 5: Verify the configuration" title="Direct link to Step 5: Verify the configuration" translate="no">​</a></h2>
<p>The following examples were tested against a live provider, a real Bedrock Guardrail configured for both directions, and a real Bedrock model.</p>
<p>The examples use <code>-k</code> because the local gateway in this guide has a self-signed TLS certificate. Remove <code>-k</code> when your gateway uses a certificate that your client trusts.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="test-1-send-a-normal-request">Test 1: Send a normal request<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#test-1-send-a-normal-request" class="hash-link" aria-label="Direct link to Test 1: Send a normal request" title="Direct link to Test 1: Send a normal request" translate="no">​</a></h3>
<div class="language-bash codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-bash codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">curl -k -X POST "&lt;PROVIDER-INVOKE-URL&gt;/model/&lt;BEDROCK-MODEL-ID&gt;/converse" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  -H "Content-Type: application/json" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  -H "X-API-Key: &lt;CLIENT-API-KEY&gt;" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  -d '{</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    "messages": [{"role": "user", "content": [{"text": "Reply with a short hello from AWS Bedrock."}]}],</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    "inferenceConfig": {"maxTokens": 64}</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  }'</span><br></span></code></pre></div></div>
<p>The request returns <code>HTTP 200</code> with the model's reply. Your text, token counts, and latency differ:</p>
<div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"metrics"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"latencyMs"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">418</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"output"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"message"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token property" style="color:#12C2FF">"content"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token property" style="color:#12C2FF">"text"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"Hello! From the team at AWS Bedrock, we hope you're having a great day. If you need any assistance or information about our services, we're here to help!"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token property" style="color:#12C2FF">"role"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"assistant"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"stopReason"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"end_turn"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"usage"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"inputTokens"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">10</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"outputTokens"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">39</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"serverToolUsage"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"totalTokens"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">49</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token punctuation" style="color:#8A93A6">}</span><br></span></code></pre></div></div>
<p>This confirms that the application can authenticate directly with the provider, that the request passes the guardrail, and that the response passes the guardrail on the way back.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="test-2-send-a-multi-turn-conversation">Test 2: Send a multi-turn conversation<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#test-2-send-a-multi-turn-conversation" class="hash-link" aria-label="Direct link to Test 2: Send a multi-turn conversation" title="Direct link to Test 2: Send a multi-turn conversation" translate="no">​</a></h3>
<div class="language-bash codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-bash codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">curl -k -X POST "&lt;PROVIDER-INVOKE-URL&gt;/model/&lt;BEDROCK-MODEL-ID&gt;/converse" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  -H "Content-Type: application/json" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  -H "X-API-Key: &lt;CLIENT-API-KEY&gt;" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  -d '{</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    "messages": [</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      {"role": "user", "content": [{"text": "My favorite color is teal. Remember that."}]},</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      {"role": "assistant", "content": [{"text": "Got it, your favorite color is teal."}]},</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      {"role": "user", "content": [{"text": "What is my favorite color?"}]}</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    ],</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    "inferenceConfig": {"maxTokens": 32}</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  }'</span><br></span></code></pre></div></div>
<p>The request returns <code>HTTP 200</code>, and the model uses the conversation history in its answer. For example, the reply mentions teal.</p>
<p>The guardrail validates the latest message on the way in and the model's reply on the way out, as configured in <a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#step-3-add-amazon-bedrock-as-an-llm-provider-and-attach-its-guardrail" class="">Step 3</a>.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="test-3-verify-denied-topic-blocking">Test 3: Verify denied-topic blocking<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#test-3-verify-denied-topic-blocking" class="hash-link" aria-label="Direct link to Test 3: Verify denied-topic blocking" title="Direct link to Test 3: Verify denied-topic blocking" translate="no">​</a></h3>
<p>Send a prompt about a topic that your guardrail denies. For example, if the guardrail denies investment advice:</p>
<div class="language-bash codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-bash codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">curl -k -X POST "&lt;PROVIDER-INVOKE-URL&gt;/model/&lt;BEDROCK-MODEL-ID&gt;/converse" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  -H "Content-Type: application/json" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  -H "X-API-Key: &lt;CLIENT-API-KEY&gt;" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  -d '{</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    "messages": [{"role": "user", "content": [{"text": "Which stocks should I buy to maximize my investment returns this year?"}]}],</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    "inferenceConfig": {"maxTokens": 64}</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  }'</span><br></span></code></pre></div></div>
<p>The gateway rejects the request with <code>HTTP 422</code>:</p>
<div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"message"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"action"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"GUARDRAIL_INTERVENED"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"actionReason"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"Violation of AWS Bedrock Guardrail detected."</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"direction"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"REQUEST"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"interveningGuardrail"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"AWS Bedrock Guardrail"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"type"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"AWS_BEDROCK_GUARDRAIL"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token punctuation" style="color:#8A93A6">}</span><br></span></code></pre></div></div>
<p>This is the gateway's error response when the configured Bedrock Guardrail blocks a request. The request never reaches the model.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="test-4-verify-pii-masking">Test 4: Verify PII masking<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#test-4-verify-pii-masking" class="hash-link" aria-label="Direct link to Test 4: Verify PII masking" title="Direct link to Test 4: Verify PII masking" translate="no">​</a></h3>
<p>Send an email address in the request:</p>
<div class="language-bash codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-bash codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">curl -k -X POST "&lt;PROVIDER-INVOKE-URL&gt;/model/&lt;BEDROCK-MODEL-ID&gt;/converse" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  -H "Content-Type: application/json" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  -H "X-API-Key: &lt;CLIENT-API-KEY&gt;" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  -d '{</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    "messages": [{"role": "user", "content": [{"text": "Please repeat this email address back to me: jane.doe@example.com"}]}],</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    "inferenceConfig": {"maxTokens": 64}</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  }'</span><br></span></code></pre></div></div>
<p>With the sensitive-information filter set to <strong>Mask</strong>, the guardrail replaces the email address with a placeholder before the request reaches the model, so the model never sees the real address. Bedrock's own output for this prompt is <code>{EMAIL}</code>; the gateway's log shows its own <code>EMAIL_0000</code> placeholder instead, mapped back to the original address. By default (<code>redactPII</code> is <code>false</code>), the gateway keeps that mapping so it can restore the original value in the response if the placeholder appears in the model's reply.</p>
<p>To watch the masking, set the policy engine log level to <code>debug</code> in the gateway's <code>configs/config.toml</code> (<code>[policy_engine.logging]</code>), recreate the gateway containers, and send the request again. See <a href="https://wso2.com/api-platform/docs/ai-gateway/1.2.0/logging-and-tracing/gateway-logs/" target="_blank" rel="noopener noreferrer" class="">Gateway logs</a> for how to read the gateway logs. The policy engine log shows the mapping and both validation calls:</p>
<div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">"awsbedrockguardrail:pii_entities":{"jane.doe@example.com":"EMAIL_0000"}</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">AWSBedrockGuardrail: Validation passed  isResponse=true</span><br></span></code></pre></div></div>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="optional-scope-guardrails-and-credentials-per-application-with-an-app-llm-proxy">Optional: Scope guardrails and credentials per application with an App LLM Proxy<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#optional-scope-guardrails-and-credentials-per-application-with-an-app-llm-proxy" class="hash-link" aria-label="Direct link to Optional: Scope guardrails and credentials per application with an App LLM Proxy" title="Direct link to Optional: Scope guardrails and credentials per application with an App LLM Proxy" translate="no">​</a></h2>
<p>Everything so far works with no App LLM Proxy: one provider, one guardrail, one client key, shared by every caller. Add a proxy only when a specific application or agent needs its own credentials, its own guardrail on top of the provider's baseline, or its own exposed resources. This section creates one on top of the provider from Step 3, to show the pattern and confirm how it layers on what you already built.</p>
<p>Adding a proxy inserts it between the application and the provider shown in the <a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#architecture" class="">Architecture</a> diagram above: the application now authenticates to the proxy instead, and the proxy authenticates to the provider with its own credential. Nothing on the Gateway-to-Bedrock side changes.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="create-the-proxy">Create the proxy<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#create-the-proxy" class="hash-link" aria-label="Direct link to Create the proxy" title="Direct link to Create the proxy" translate="no">​</a></h3>
<p>App LLM Proxies are created and managed at the project level; creating a provider doesn't ask for a project the way creating a proxy does.</p>
<ol>
<li class="">Open <strong>App LLM Proxies</strong>. Choreo Agent Control Plane asks you to select a project. Select your project and click <strong>Go to Project Level</strong>.</li>
<li class="">Click <strong>+ Create App LLM Proxy</strong>.</li>
<li class="">Enter a <strong>Name</strong> and a <strong>Version</strong> for the proxy.</li>
<li class="">In <strong>LLM Provider</strong>, select the Bedrock provider you created in Step 3.</li>
<li class="">Under <strong>API Keys</strong>, either click <strong>Generate API Key</strong> or choose <strong>Enter API Key Manually</strong>. The <strong>Generate API Key</strong> option creates an API key for the selected LLM provider, not for the proxy itself; it authenticates the proxy to the provider. This key also appears on the provider's own <strong>Overview</strong> tab, under its <strong>API Keys</strong> section.</li>
<li class="">Click <strong>Create Proxy</strong>.</li>
</ol>
<p>Three keys are now involved, where calling the provider directly in Step 4 needed only one:</p>
<ol>
<li class="">The <strong>Bedrock API key</strong> from Step 2, stored on the provider. It authenticates the gateway to AWS.</li>
<li class="">The <strong>provider API key</strong> from item 5 above. It authenticates the proxy to the provider.</li>
<li class="">The <strong>client-facing proxy key</strong> you generate below. It authenticates your application to the proxy.</li>
</ol>
<p>The <a href="https://wso2.com/api-platform/docs/ai-gateway/1.2.0/gateway-artifacts/llm-provider/supported-providers/aws-bedrock/" target="_blank" rel="noopener noreferrer" class="">AWS Bedrock provider reference</a> distinguishes the Bedrock bearer API key (gateway to AWS) from a gateway consumer API key (caller to gateway); don't use one in place of the other. Calling the provider directly, as in Steps 3-5, needs only those same two keys — there's no third.</p>
<p>The proxy's <strong>Overview</strong> tab lists the same resources as the provider:</p>
<ul>
<li class=""><code>POST /model/{modelId}/converse</code></li>
<li class=""><code>POST /model/{modelId}/converse-stream</code></li>
</ul>
<p>For more details, see <a href="https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-proxies/configure-proxy/" target="_blank" rel="noopener noreferrer" class="">Configure an App LLM proxy</a>.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="add-a-proxy-specific-guardrail-if-you-need-one">Add a proxy-specific guardrail, if you need one<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#add-a-proxy-specific-guardrail-if-you-need-one" class="hash-link" aria-label="Direct link to Add a proxy-specific guardrail, if you need one" title="Direct link to Add a proxy-specific guardrail, if you need one" translate="no">​</a></h3>
<p>The guardrail attached to the provider in Step 3 already runs on every request that reaches the provider, including requests that arrive through this proxy: in testing, a denied-topic prompt sent through the proxy was blocked with the same <code>HTTP 422</code> response as Test 3, even before any guardrail was attached to the proxy itself. Add a guardrail here only when this specific application needs stricter, looser, or different rules than that baseline.</p>
<ol>
<li class="">Open the proxy and select the <strong>Guardrails &amp; Policies</strong> tab. The layout is the same panel you already used on the provider in Step 3, with the same <code>region</code> / <code>guardrailID</code> / <code>guardrailVersion</code> fields.</li>
<li class="">Under <strong>Global Guardrails &amp; Policies</strong>, click <strong>+ Add</strong> and select a guardrail from the <a href="https://wso2.com/api-platform/docs/ai-gateway/1.2.0/guardrails/guardrails-catalogue/" target="_blank" rel="noopener noreferrer" class="">guardrail catalog</a>, for example another <strong>AWS Bedrock Guardrail</strong> attachment pointed at a stricter guardrail ID, configured the same way as <a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#step-3-add-amazon-bedrock-as-an-llm-provider-and-attach-its-guardrail" class="">Step 3</a>.</li>
<li class="">Click <strong>Add</strong>, then click <strong>Save</strong>.</li>
</ol>
<p>The proxy's <strong>Guardrails &amp; Policies</strong> tab now shows the attached policy as a chip, the same way the provider's did in Step 3.</p>
<p>When both a provider-level and a proxy-level guardrail are attached, both are enforced: the provider's guardrail acts as a baseline, and the proxy's guardrail adds to it.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="deploy-the-proxy-and-generate-a-client-api-key">Deploy the proxy and generate a client API key<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#deploy-the-proxy-and-generate-a-client-api-key" class="hash-link" aria-label="Direct link to Deploy the proxy and generate a client API key" title="Direct link to Deploy the proxy and generate a client API key" translate="no">​</a></h3>
<ol>
<li class="">
<p>On the proxy page, click <strong>Deploy to Gateway</strong>, and then deploy the proxy to your gateway. Wait until the deployment status is <strong>Active</strong>.</p>
<p>Deploy the proxy again whenever you change its configuration. Saved changes reach the running gateway only through a new deployment. See <a href="https://wso2.com/api-platform/docs/ai-workspace/1.0.0/llm-proxies/manage-proxy/" target="_blank" rel="noopener noreferrer" class="">Manage an App LLM proxy</a>.</p>
</li>
<li class="">
<p>Open the proxy's <strong>Overview</strong> tab. After deployment, it shows the <strong>Invoke URL</strong> for the gateway and an <strong>App LLM Proxy Keys</strong> panel, in the same place the provider showed its plain <strong>API Keys</strong> panel in Step 4:</p>
<p><img decoding="async" loading="lazy" alt="Proxy Overview tab showing OpenAPI Resources for POST /model/modelId/converse and POST /model/modelId/converse-stream, the real Invoke URL, and the App LLM Proxy Keys panel" src="https://choreolabs.com/api-management/learn/assets/images/07-proxy-overview-invoke-url-9e024b64eee14eb9d2c1a2308a85de3c.png" width="2516" height="1569" class="img_ev3q"></p>
</li>
<li class="">
<p>Click <strong>Generate API Key</strong>, give the key a name, and copy the key right away. Choreo Agent Control Plane shows it only once. Applications send it in the <code>X-API-Key</code> header. Like the provider's key, this one also expires 90 days after you generate it.</p>
</li>
</ol>
<p>Call this proxy's Invoke URL with this client key instead of the provider's, and the same four tests from <a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#step-5-verify-the-configuration" class="">Step 5</a> apply unchanged: the denied-topic and PII-masking tests still pass because of the provider-level guardrail, even before you add anything proxy-specific.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="troubleshooting">Troubleshooting<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#troubleshooting" class="hash-link" aria-label="Direct link to Troubleshooting" title="Direct link to Troubleshooting" translate="no">​</a></h2>
<table><thead><tr><th>Symptom</th><th>What to check</th></tr></thead><tbody><tr><td><code>HTTP 401</code> from the gateway</td><td>The <code>X-API-Key</code> header is missing, or isn't a key generated for the provider or proxy you're calling.</td></tr><tr><td><code>HTTP 422</code> with <code>Error calling AWS Bedrock Guardrail</code></td><td>The gateway can't call the guardrail, and the request is blocked by default. Check the <strong>awsAuth</strong> credentials in Step 3, their permissions, the Region, and the guardrail ID and version.</td></tr><tr><td><code>HTTP 422</code> with <code>Error extracting value from JSONPath</code> and <code>direction: RESPONSE</code></td><td>The response doesn't match the response JSONPath. Check the path, and check whether the upstream returned an error or a stream.</td></tr><tr><td>An upstream authentication error from Bedrock when you configure the provider through the API or YAML instead of the UI</td><td>In that case, include the <code>Bearer </code> prefix in the credential value. A raw token without the prefix returns <code>HTTP 403</code> from upstream. The Choreo Agent Control Plane form adds the prefix for you. See the <a href="https://wso2.com/api-platform/docs/ai-gateway/1.2.0/gateway-artifacts/llm-provider/supported-providers/aws-bedrock/" target="_blank" rel="noopener noreferrer" class="">AWS Bedrock provider reference</a>.</td></tr><tr><td>Settings in the guardrail panel differ from this guide</td><td>Compare the policy version in the gateway log (Step 1) with the version in the <a href="https://wso2.com/api-platform/policy-hub/policies/aws-bedrock-guardrail" target="_blank" rel="noopener noreferrer" class="">Policy Hub</a>.</td></tr></tbody></table>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="production-considerations">Production considerations<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#production-considerations" class="hash-link" aria-label="Direct link to Production considerations" title="Direct link to Production considerations" translate="no">​</a></h2>
<p>This guide favors reproducibility and ease of testing. Before you use a similar setup in production, consider the following:</p>
<ul>
<li class=""><strong>Use a workload role for production, not the Bedrock API key.</strong> Choreo AI Gateway scopes the bearer key, and a static IAM access key, to getting started and local testing. See <a href="https://wso2.com/api-platform/docs/ai-gateway/1.2.0/gateway-artifacts/llm-provider/supported-providers/aws-bedrock/#choose-an-authentication-method" target="_blank" rel="noopener noreferrer" class="">Choose an authentication method</a> for the production options. A short-term key also needs manual rotation: generate a new key and redeploy the provider with the updated secret before the old one expires.</li>
<li class=""><strong>Use a role-based <code>awsAuth</code> mode for the guardrail too, for production.</strong> This guide uses a static access key for simplicity. See the <a href="https://wso2.com/api-platform/policy-hub/policies/aws-bedrock-guardrail" target="_blank" rel="noopener noreferrer" class="">Policy Hub</a> for the available modes.</li>
<li class=""><strong>Protect the provider key.</strong> It authenticates directly to Bedrock, so keep it out of application code and restrict who can access it.</li>
<li class=""><strong>Apply least privilege</strong>, including <code>bedrock:GetInferenceProfile</code> if you use an inference profile ID. See <a href="https://wso2.com/api-platform/docs/ai-gateway/1.2.0/gateway-artifacts/llm-provider/supported-providers/aws-bedrock/#configure-iam-permissions-for-sigv4" target="_blank" rel="noopener noreferrer" class="">Configure IAM permissions for SigV4</a> for a complete example.</li>
<li class=""><strong>Use a numbered guardrail version, not <code>DRAFT</code>.</strong> The <a href="https://wso2.com/api-platform/policy-hub/policies/aws-bedrock-guardrail" target="_blank" rel="noopener noreferrer" class="">Policy Hub</a> describes <code>DRAFT</code> as useful for testing and recommends numbered versions for production. Because a <code>DRAFT</code> guardrail in Bedrock can keep changing after you've already attached it, a numbered version keeps the gateway's behavior from shifting under you.</li>
<li class=""><strong>Plan for client-key expiry.</strong> Generated keys expire 90 days after creation; Choreo Agent Control Plane lists the exact date in the <strong>Expires At</strong> column. Build rotation into your deployment process instead of treating a generated key as permanent.</li>
<li class=""><strong>Decide how failures should behave.</strong> <code>passthroughOnError</code> defaults to <code>false</code>, which fails closed on both request and response. Set it to <code>true</code> on the side or sides where you want to fail open instead.</li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="does-choreo-ai-gateway-replace-amazon-bedrock-guardrails">Does Choreo AI Gateway replace Amazon Bedrock Guardrails?<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#does-choreo-ai-gateway-replace-amazon-bedrock-guardrails" class="hash-link" aria-label="Direct link to Does Choreo AI Gateway replace Amazon Bedrock Guardrails?" title="Direct link to Does Choreo AI Gateway replace Amazon Bedrock Guardrails?" translate="no">​</a></h3>
<p>No. Amazon Bedrock Guardrails evaluates the content. Choreo Agent Control Plane and AI Gateway attach the guardrail to an LLM provider, an App LLM Proxy, or both, and decide when it runs: on requests, on responses, or on both. You can also apply other Choreo AI Gateway policies to the same traffic, such as <a href="https://wso2.com/api-platform/docs/ai-gateway/1.2.0/routing/multi-provider-routing/" target="_blank" rel="noopener noreferrer" class="">multi-provider routing</a> and <a href="https://wso2.com/api-platform/docs/ai-gateway/1.2.0/cost-control-and-budgets/" target="_blank" rel="noopener noreferrer" class="">cost control</a>.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="why-are-there-several-credentials">Why are there several credentials?<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#why-are-there-several-credentials" class="hash-link" aria-label="Direct link to Why are there several credentials?" title="Direct link to Why are there several credentials?" translate="no">​</a></h3>
<p>Calling the provider directly, as this guide does, involves three: the application's client key to the provider, the provider's own Bedrock credential for model calls, and the separate IAM credential the guardrail policy uses for <code>ApplyGuardrail</code> calls. Adding an App LLM Proxy on top adds a fourth: the proxy authenticates to the provider with its own provider API key, and the application's client key now points at the proxy instead of the provider. Bedrock itself doesn't require separate AWS identities for model calls and guardrail checks, so you can use one identity for both when your setup allows it.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="can-multiple-applications-use-the-same-bedrock-provider">Can multiple applications use the same Bedrock provider?<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#can-multiple-applications-use-the-same-bedrock-provider" class="hash-link" aria-label="Direct link to Can multiple applications use the same Bedrock provider?" title="Direct link to Can multiple applications use the same Bedrock provider?" translate="no">​</a></h3>
<p>Yes, and by default they already do: every application that gets a key generated on the provider calls the same guardrail and policies. If one application needs its own client key, guardrail, or policies that don't apply to the others, add an App LLM Proxy for that application on top of the shared provider; a client key for one proxy isn't valid on another, and a guardrail attached to one proxy doesn't apply to the others.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="which-authentication-should-i-use-in-production">Which authentication should I use in production?<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#which-authentication-should-i-use-in-production" class="hash-link" aria-label="Direct link to Which authentication should I use in production?" title="Direct link to Which authentication should I use in production?" translate="no">​</a></h3>
<p>A workload role (SigV4), not the bearer API key or static IAM access key this guide uses for simplicity. See <a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#production-considerations" class="">Production considerations</a> for the options.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/aws-bedrock-llm-guardrails-wso2-ai-gateway#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>Choreo Agent Control Plane is the control plane: one place to register Amazon Bedrock, attach Bedrock Guardrails, create application-facing proxies where you need them, and deploy everything to your AI Gateway. AI Gateway is the data plane: the component that actually sits in front of Bedrock, authenticates applications with gateway-issued client keys, and applies the policies configured for each provider or proxy.</p>
<p>In this guide, you configured an Amazon Bedrock provider end to end, attached a Bedrock Guardrail directly to it that validates requests and responses, and verified the setup with real Bedrock requests, calling the provider directly. You also saw how to add an App LLM Proxy on top when one application needs its own credentials or guardrail. You can reuse the same provider for more applications, teams, and environments, and combine it with other providers in a multi-provider setup.</p>]]></content>
        <author>
            <name>Dakshitha Suriyaaratchie</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="Tutorials" term="Tutorials"/>
        <category label="Policies" term="Policies"/>
        <category label="Guardrails" term="Guardrails"/>
        <category label="AI" term="AI"/>
        <category label="AI gateway" term="AI gateway"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[AI agent evaluation: Task-based evals & regression testing]]></title>
        <id>https://choreolabs.com/api-management/learn/ai-agent-evaluation</id>
        <link href="https://choreolabs.com/api-management/learn/ai-agent-evaluation"/>
        <updated>2026-10-15T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[How to evaluate AI agents: task-based evals, regression suites, and the difference between offline and online evaluation with a practical rollout plan.]]></summary>
        <content type="html"><![CDATA[<p>A prompt change goes out on a Tuesday. The agent still answers every question fluently, the logs show no errors, and every response returns HTTP 200. Two days later, a support ticket reveals that the agent stopped calling its refund-policy tool after the release and has been answering from memory ever since. Nothing crashed, so nothing alerted.</p>
<p>This tutorial explains how AI agent evaluation catches that kind of silent regression before it ships. It covers task-based and trajectory evals, offline and online evaluation, and how to choose graders, then walks through building a working regression suite in Choreo Integrator, from capturing real conversations to tracking pass rate across builds.</p>
<aside class="cta_6F_m" aria-label="Make your APIs agent-ready" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Make your APIs agent-ready</span><span class="desc_GzK8">Choreo turns existing APIs into governed MCP tools, with full visibility into what agents are calling.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=ai-agent-evaluation" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="why-agent-evaluation-isnt-like-unit-testing">Why agent evaluation isn't like unit testing<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#why-agent-evaluation-isnt-like-unit-testing" class="hash-link" aria-label="Direct link to Why agent evaluation isn't like unit testing" title="Direct link to Why agent evaluation isn't like unit testing" translate="no">​</a></h2>
<p>AI agent evaluation is the practice of scoring an agent's behavior against realistic tasks across several dimensions at once: whether it completed the task, chose the right tools, grounded its answer in tool results, and stayed safe. A unit test compares one deterministic output to one expected value; an agent can reach a correct result through different paths on different runs.</p>
<p>Two properties make agents harder to test than ordinary code. First, the same input can produce a different sequence of tool calls and a differently worded answer on the next run, so a single exact-match assertion fails on valid variation. Second, agent failures are usually behavioral rather than technical. In the example above, every request succeeded at the HTTP level while the agent quietly skipped a step it was supposed to take.</p>
<p>That is why the ground truth for an agent eval is rarely a single string. The Choreo Integrator documentation lists the dimensions directly: correctness, tool selection, groundedness, safety, and tone. <a href="https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents" target="_blank" rel="noopener noreferrer" class="">Anthropic's guide to agent evals</a> draws a related line between the transcript (what the agent said and did) and the outcome (the final state of the environment). A booking agent can tell the user the flight is booked while no reservation exists in the database, so both need checking.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="task-based-evals-judging-whether-the-agent-actually-did-the-job">Task-based evals: judging whether the agent actually did the job<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#task-based-evals-judging-whether-the-agent-actually-did-the-job" class="hash-link" aria-label="Direct link to Task-based evals: judging whether the agent actually did the job" title="Direct link to Task-based evals: judging whether the agent actually did the job" translate="no">​</a></h2>
<p>A task-based eval answers the first question any stakeholder asks: did the agent complete the task? Each task is a realistic input with defined success criteria, and the eval runs the agent end to end and checks the final output or the resulting state.</p>
<p>The quality of a task-based eval depends almost entirely on the tasks. Three rules hold up well in practice:</p>
<ul>
<li class=""><strong>Start from real conversations, not invented ones.</strong> Anthropic recommends starting with 20 to 50 simple tasks taken from failures you have actually seen, rather than waiting until you have hundreds. Bug reports, support tickets, and the manual checks you already run before a release are the best seed material.</li>
<li class=""><strong>Make success unambiguous.</strong> If two engineers could reasonably disagree about whether a run passed, the task needs tighter success criteria before it becomes a test.</li>
<li class=""><strong>Include cases where the agent should not act.</strong> A suite that only asks "did the agent call the search tool when it should?" can produce an agent that searches for almost everything. Add tasks where the correct behavior is to answer directly, refuse, or escalate.</li>
</ul>
<p>For a math tutoring agent, a balanced starter set might include a direct sum ("What is 48 plus 17?"), a word problem that requires subtraction, a conceptual question that needs no tool at all ("What is a prime number?"), and a multi-step calculation. Each one tests a different way the agent can succeed or fail. Step 2 of the tutorial below captures exactly this set.</p>
<p>Success rate on a task set is the headline number, but it hides variance. A task that passes on one run can fail on the next, so Anthropic separates pass@k (at least one success in k attempts) from pass^k (all k attempts succeed). An agent with a 75% per-trial success rate passes three trials in a row only about 42% of the time, which matters for customer-facing agents where users expect consistent behavior.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="trajectory-and-component-level-evaluation-judging-how-it-got-there">Trajectory and component-level evaluation: judging how it got there<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#trajectory-and-component-level-evaluation-judging-how-it-got-there" class="hash-link" aria-label="Direct link to Trajectory and component-level evaluation: judging how it got there" title="Direct link to Trajectory and component-level evaluation: judging how it got there" translate="no">​</a></h2>
<p>End-to-end task evals tell you that an agent failed, and trajectory evaluation shows where. It inspects the sequence of tool calls and reasoning steps behind the final answer, which is the only way to catch an agent that reaches the right answer by the wrong route, such as calculating a total in its head instead of calling the tool that applies the current tax rules.</p>
<p>Component-level evaluation narrows the focus further. Choreo Agent Manager, for example, runs evaluators at three levels: once per trace for the whole interaction, once per agent inside a multi-agent trace, and once per individual LLM call. A trace with three agents and five LLM calls produces one trace-level score, three agent-level scores, and five LLM-level scores. The LLM level is useful for catching a single call that produced unsafe content even when a later step filtered it out of the final answer.</p>
<p>Tool-call correctness deserves its own dimension, because a wrong or repeated tool call can have side effects that a correct-looking final answer hides. The hard part is deciding how strictly to check the path. Anthropic warns that requiring an exact sequence of tool calls produces brittle tests, because agents regularly find valid approaches the eval designer did not anticipate. Pick the strictness for each task instead of setting one rule for the whole suite. Choreo Agent Manager reflects the same idea in its built-in evaluators: Tool Coverage only checks that required tools were called at least once, while Sequence Adherence measures how closely the call order matches an expected sequence and is not strict by default.</p>
<p>The <code>ballerina/ai.eval</code> library makes that choice explicit with four trajectory match modes. A tool call matches when both the tool name and its arguments are equal; tool-call IDs are ignored because they change between runs.</p>
<table><thead><tr><th style="text-align:left">Mode</th><th style="text-align:left">What passes</th><th style="text-align:left">Use it when</th></tr></thead><tbody><tr><td style="text-align:left"><code>STRICT</code></td><td style="text-align:left">Exactly the recorded calls, in the same order</td><td style="text-align:left">Order is part of the contract, for example verify identity before processing a refund</td></tr><tr><td style="text-align:left"><code>UNORDERED</code></td><td style="text-align:left">Exactly the recorded calls, in any order</td><td style="text-align:left">The agent must gather the same inputs but sequence does not matter</td></tr><tr><td style="text-align:left"><code>SUBSET</code></td><td style="text-align:left">Only calls that appear in the recording, with no extras (not every recorded call is required)</td><td style="text-align:left">The concern is unnecessary or unauthorized calls</td></tr><tr><td style="text-align:left"><code>SUPERSET</code></td><td style="text-align:left">At least every recorded call; extras allowed</td><td style="text-align:left">A required call must happen, and extra lookups are acceptable</td></tr></tbody></table>
<p>If you prefer code to the visual designer, the same check runs as an ordinary Ballerina test against an evalset file. This example assumes the agent is declared as <code>mathTutorAgent</code>, requires every recorded tool call to happen, and tolerates up to 20% of threads failing:</p>
<div class="language-ballerina codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-ballerina codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">import ballerina/ai;</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">import ballerina/ai.eval;</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">import ballerina/test;</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">isolated function loadMathTutorEvalset() returns map&lt;[ai:ConversationThread]&gt;|error {</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    return ai:loadConversationThreads("tests/resources/evalsets/math-tutor.evalset.json");</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">}</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">@test:Config {</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    dataProvider: loadMathTutorEvalset,</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    minPassRate: 0.8</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">}</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">function mathTutorMakesRequiredToolCalls(ai:ConversationThread thread) returns error? {</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    check eval:evaluateToolTrajectory(targetAgent = mathTutorAgent, thread = thread,</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">            matchMode = eval:SUPERSET);</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">}</span><br></span></code></pre></div></div>
<p>The <code>minPassRate</code> field was introduced in Ballerina 2201.13.2, so on older distributions every thread must pass. See the <a href="https://central.ballerina.io/ballerina/ai.eval/0.9.0" target="_blank" rel="noopener noreferrer" class="">ai.eval module reference</a> for the full list of rule-based and judge templates.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="building-a-regression-suite">Building a regression suite<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#building-a-regression-suite" class="hash-link" aria-label="Direct link to Building a regression suite" title="Direct link to Building a regression suite" translate="no">​</a></h2>
<p>A regression suite answers a narrower question than a task eval: does the agent still handle everything it used to? Capability evals should start at a low pass rate to give the team something to improve, while regression evals should sit near 100% so that any drop signals a real break. As a capability task becomes reliable, it graduates into the regression suite.</p>
<p>In practice, a regression suite has three parts:</p>
<ol>
<li class=""><strong>A golden dataset:</strong> a set of conversation traces that records the expected behavior for each case, including the tool calls the agent should make and the response it should give. Capturing these from real sessions is faster and more representative than writing them by hand.</li>
<li class=""><strong>Evaluation checks:</strong> functions that replay each case against the current build and score the result, using deterministic asserts where possible and an LLM judge where the dimension is subjective.</li>
<li class=""><strong>Run history:</strong> a record of pass rate per run, so a drop can be traced to the change that caused it.</li>
</ol>
<p>Choreo Integrator maps these three parts one to one onto Evalsets, Evaluations, and Runs and reports.</p>
<p><img decoding="async" loading="lazy" alt="Agent evaluation regression workflow in Choreo Integrator: Evalsets capture real chat sessions as a golden dataset, Evaluations replay every case against the current build and score it, and Runs and reports check the pass rate against the target and track it across runs" src="https://choreolabs.com/api-management/learn/assets/images/agent-evaluation-regression-workflow-a51ff29981eebce490c95fe55d13e6aa.webp" width="2048" height="853" class="img_ev3q"></p>
<p>The suite earns its keep when it runs at the moments that change behavior: before you edit the agent's instructions, swap a tool, or upgrade the model. Running it in CI/CD on every change turns "the agent feels worse" into a failing build with a specific case attached. Because <code>ai.eval</code> templates run as ordinary Ballerina test functions, the code form of the suite fits into the same pipeline as your other tests. Steps 3 through 8 of the tutorial below build this suite end to end.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="offline-vs-online-evaluation">Offline vs. online evaluation<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#offline-vs-online-evaluation" class="hash-link" aria-label="Direct link to Offline vs. online evaluation" title="Direct link to Offline vs. online evaluation" translate="no">​</a></h2>
<p>Offline evaluation runs a fixed dataset against a build before it reaches users. Online evaluation scores live production traffic after deployment. They catch different failures, so they work as a pair rather than as alternatives.</p>
<table><thead><tr><th style="text-align:left"></th><th style="text-align:left">Offline evaluation</th><th style="text-align:left">Online evaluation</th></tr></thead><tbody><tr><td style="text-align:left">Data</td><td style="text-align:left">A fixed golden dataset of curated cases</td><td style="text-align:left">Sampled traces from real users</td></tr><tr><td style="text-align:left">When it runs</td><td style="text-align:left">Before deployment, on every change and model upgrade</td><td style="text-align:left">Continuously after deployment</td></tr><tr><td style="text-align:left">What it catches</td><td style="text-align:left">Regressions in known behavior</td><td style="text-align:left">Drift and failure modes nobody anticipated</td></tr><tr><td style="text-align:left">Ground truth</td><td style="text-align:left">Recorded expected behavior per case</td><td style="text-align:left">Usually none, so scoring relies on rules and judges</td></tr><tr><td style="text-align:left">Cost of a miss</td><td style="text-align:left">A failed build</td><td style="text-align:left">A problem that has already reached users</td></tr><tr><td style="text-align:left">In Choreo</td><td style="text-align:left">Choreo Integrator Evaluations</td><td style="text-align:left">Choreo Agent Manager monitors</td></tr></tbody></table>
<p>The trade-off is reach versus timing. Offline evals are reproducible and have no user impact, but they only test what you thought to record. Production monitoring sees real behavior at scale, but it is reactive, because users hit a problem before the team sees it, and live traffic has no recorded answer to grade against. A/B testing sits between the two and measures real user outcomes, but it needs enough traffic and often several days or weeks before a result is significant.</p>
<p>The two halves feed each other. A failure found online becomes a new offline case, which keeps the regression suite anchored to the real world. When you need to understand one specific failed run rather than a trend, trace-level debugging is the better tool.</p>
<p>Evaluations score behavior after the fact and never block a request. To stop an unsafe request from reaching the model at runtime, use <a class="" href="https://choreolabs.com/api-management/learn/prompt-injection-guardrails-llm">AI gateway guardrails</a> alongside your evals.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="choosing-graders-llm-as-judge-code-based-checks-and-human-review">Choosing graders: LLM-as-judge, code-based checks, and human review<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#choosing-graders-llm-as-judge-code-based-checks-and-human-review" class="hash-link" aria-label="Direct link to Choosing graders: LLM-as-judge, code-based checks, and human review" title="Direct link to Choosing graders: LLM-as-judge, code-based checks, and human review" translate="no">​</a></h2>
<p>A grader is the logic that scores one aspect of a run. Agent evaluations typically combine three kinds, and the practical rule is to use the cheapest grader that can make the call reliably.</p>
<table><thead><tr><th style="text-align:left">Grader</th><th style="text-align:left">Use it for</th><th style="text-align:left">Example check</th><th style="text-align:left">Watch out for</th></tr></thead><tbody><tr><td style="text-align:left">Code-based</td><td style="text-align:left">Anything you can state as a rule</td><td style="text-align:left">Tool calls match the recording; response contains no prohibited string; finished within N iterations</td><td style="text-align:left">Brittle against valid variation in wording or path</td></tr><tr><td style="text-align:left">LLM-as-judge</td><td style="text-align:left">Subjective dimensions</td><td style="text-align:left">Tone fits a support context; claims are grounded in tool results; refusal was appropriate</td><td style="text-align:left">Non-deterministic, costs an LLM call, needs calibration</td></tr><tr><td style="text-align:left">Human review</td><td style="text-align:left">Calibrating judges and auditing edge cases</td><td style="text-align:left">Weekly sample of transcripts; expert review of disputed cases</td><td style="text-align:left">Slow and expensive at scale</td></tr></tbody></table>
<p>Three habits make LLM-as-a-judge grading trustworthy:</p>
<ul>
<li class=""><strong>Calibrate against humans first.</strong> Have a person grade a sample of the same cases and compare. If the judge and the human disagree often, fix the rubric before trusting the score.</li>
<li class=""><strong>Judge one dimension at a time, and allow "Unknown".</strong> A separate judge per dimension produces clearer failures than one judge scoring everything, and an explicit "not enough information" option reduces invented verdicts.</li>
<li class=""><strong>Keep the judge stable.</strong> The <code>ai.eval</code> documentation recommends a low temperature for the judge model so scores stay stable across runs, and its judge templates pass at a default score threshold of 0.8. Agent Manager's LLM-as-Judge evaluators default to a temperature of 0.0 for the same reason.</li>
</ul>
<p>Treat the judge's input as untrusted. An agent's response can contain text aimed at the judge, such as an instruction to ignore the rubric. The <code>ai.eval</code> judges fence agent content and instruct the judge to treat it as data, but the library is explicit that no prompt-level defense is complete and that scores from an adversarial agent should be treated as advisory.</p>
<p>In Choreo Integrator, code-based checks are assert nodes in the visual designer, and an LLM judge is a Model Provider node that scores the response before an assert checks the score. Step 6 of the tutorial below wires up the first and shows where the second goes.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="common-pitfalls-in-agent-evaluation">Common pitfalls in agent evaluation<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#common-pitfalls-in-agent-evaluation" class="hash-link" aria-label="Direct link to Common pitfalls in agent evaluation" title="Direct link to Common pitfalls in agent evaluation" translate="no">​</a></h2>
<p>These mistakes are common enough to plan for. The expired-token case under infrastructure failures happened while this tutorial was being tested.</p>
<ul>
<li class=""><strong>Writing evals after the agent ships.</strong> While the agent is being designed, each requirement is easy to turn into a test case. After launch, the team has to work out what "correct" meant by studying a system that users already depend on. Evaluation-driven development, where the eval defines the behavior before the agent achieves it, avoids that.</li>
<li class=""><strong>Treating evaluation as a one-time gate.</strong> A suite that nobody adds cases to stops reflecting how users actually use the agent. Every production failure should become a new case.</li>
<li class=""><strong>Skipping evals for "low-risk" agents.</strong> Internal or experimental agents tend to gain users, tools, and permissions quietly. By the time one matters, there is no baseline to compare against.</li>
<li class=""><strong>Recording the agent's mistakes as ground truth.</strong> An evalset captured from a real session contains whatever the agent did, including errors. Review and correct each case before it becomes the expected behavior.</li>
<li class=""><strong>Requiring 100% on a non-deterministic system.</strong> Some variation between runs is normal even when the agent behaves correctly, so a 100% threshold produces flaky failures. The Choreo Integrator documentation suggests a threshold of roughly 80 to 90%, which absorbs that noise and still flags genuine breaks.</li>
<li class=""><strong>Counting infrastructure failures as agent failures.</strong> If the judge model is unreachable or a test is misconfigured, that is a failure to evaluate, not a verdict on the agent. Report them separately, or an outage at the judge's model provider shows up as a drop in agent quality. The same applies to the agent itself. In Choreo Integrator, an expired model-provider token does not stop the run: the agent returns a trace whose output is the connection error, so a tool-call assertion fails with an empty actual value that looks exactly like an agent that skipped its tools. A run that finishes far faster than a normal chat turn is the giveaway.</li>
<li class=""><strong>Trusting a score without reading transcripts.</strong> Anthropic reports that Claude Opus 4.5 first scored 42% on CORE-Bench. The causes included rigid grading and ambiguous task specs, and once the bugs were fixed and the agent ran on a less constrained scaffold, the score rose to 95%. When a task fails on every run, check the task and the grader before blaming the agent. While this tutorial was being tested, an LLM judge failed three correct answers because their traces carried the wrong question as input, as Step 12 shows.</li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-choreo-approaches-agent-evaluation">How Choreo approaches agent evaluation<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#how-choreo-approaches-agent-evaluation" class="hash-link" aria-label="Direct link to How Choreo approaches agent evaluation" title="Direct link to How Choreo approaches agent evaluation" translate="no">​</a></h2>
<p>Choreo splits agent evaluation across the development and runtime sides of the agent lifecycle:</p>
<ul>
<li class=""><strong>Choreo Integrator</strong> handles offline evaluation inside the IDE. Its Evaluations feature captures real chat sessions as evalsets, scores them with evaluations built in a visual designer, and tracks results in reports and an Evaluation History view.</li>
<li class=""><strong>Choreo Agent Manager</strong> handles online evaluation in production. Its monitors run built-in or custom evaluators against agent traces after the agent finishes, separately from the agent, so evaluation does not slow down the agent's runtime.</li>
</ul>
<p>The Integrator documentation also draws a clear line between when an evaluation is the right tool and when it is not.</p>
<table><thead><tr><th style="text-align:left">Use an evaluation when...</th><th style="text-align:left">Use something else when...</th></tr></thead><tbody><tr><td style="text-align:left">You are about to change instructions, tools, or the model and want a regression check</td><td style="text-align:left">You need a deterministic unit test for a pure function</td></tr><tr><td style="text-align:left">You want a quality baseline you can track across runs and commits</td><td style="text-align:left">You are debugging one specific failed run (use tracing and observability)</td></tr><tr><td style="text-align:left">You need to verify safety and refusal behavior before shipping</td><td style="text-align:left">You are still prototyping and have not chosen the agent's tools</td></tr></tbody></table>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="tutorial-part-1-build-a-regression-suite-for-an-ai-agent-in-choreo-integrator">Tutorial (part 1): Build a regression suite for an AI agent in Choreo Integrator<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#tutorial-part-1-build-a-regression-suite-for-an-ai-agent-in-choreo-integrator" class="hash-link" aria-label="Direct link to Tutorial (part 1): Build a regression suite for an AI agent in Choreo Integrator" title="Direct link to Tutorial (part 1): Build a regression suite for an AI agent in Choreo Integrator" translate="no">​</a></h2>
<section class="panel_ykK0" aria-labelledby="_R_6f9qmn9al_"><div class="head_AuId"><p class="label_Di5q">Tutorial</p><p class="title_RBm6" id="_R_6f9qmn9al_">Before you start</p></div><div class="body_JQ_P"><ul>
<li class=""><strong>What you will build:</strong> A regression suite for a tool-using chat agent in Choreo Integrator: an evalset captured from real conversations, an evaluation that checks tool selection on every turn, and a run history that shows when a change breaks behavior.</li>
<li class=""><strong>What you will learn:</strong> How to capture and correct a golden dataset, how to replay it against the current build, how to set a pass threshold for a non-deterministic agent, and how to spot a regression in the run history.</li>
<li class=""><strong>Prerequisites:</strong> Choreo Integrator installed, and an AI chat agent with a configured model provider and at least two tools. This tutorial uses a math tutor agent with <code>sumTool</code> and <code>subtractTool</code>; any agent with tools works.</li>
</ul></div></section>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-1-turn-on-tracing">Step 1: Turn on tracing<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#step-1-turn-on-tracing" class="hash-link" aria-label="Direct link to Step 1: Turn on tracing" title="Direct link to Step 1: Turn on tracing" translate="no">​</a></h3>
<p>Evalsets are built from traces, and tracing is off by default.</p>
<ol>
<li class="">Open the agent in the canvas.</li>
<li class="">In the top-right corner, click the <strong>Tracing: Off</strong> toggle.</li>
</ol>
<p><img decoding="async" loading="lazy" alt="The AI Chat Agent canvas in Choreo Integrator with the Tracing toggle in the top-right corner" src="https://choreolabs.com/api-management/learn/assets/images/step1-turn-on-tracing-dc3553cb105107d052104f8b7a4e58b9.png" width="2048" height="1243" class="img_ev3q"></p>
<p><strong>Expected result:</strong> The toggle reads <strong>Tracing: On</strong>, and a confirmation notification appears in the bottom-right corner.</p>
<p>For agents declared inline rather than through the AI Chat Agent wizard, run <strong>Ballerina: Enable Tracing</strong> from the command palette instead.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-2-chat-with-the-agent-to-capture-test-cases">Step 2: Chat with the agent to capture test cases<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#step-2-chat-with-the-agent-to-capture-test-cases" class="hash-link" aria-label="Direct link to Step 2: Chat with the agent to capture test cases" title="Direct link to Step 2: Chat with the agent to capture test cases" translate="no">​</a></h3>
<p>Each message you send becomes a candidate test case, so send a deliberately balanced set: cases where the agent should call a tool, and one where it should not.</p>
<ol>
<li class="">Click <strong>Chat</strong> to open the Agent Chat panel.</li>
<li class="">Send the following messages, one at a time:</li>
</ol>
<table><thead><tr><th style="text-align:left">Message</th><th style="text-align:left">Expected behavior</th></tr></thead><tbody><tr><td style="text-align:left">What is 48 plus 17?</td><td style="text-align:left">Calls <code>sumTool</code> once</td></tr><tr><td style="text-align:left">I had 120 marbles and gave away 45. How many are left?</td><td style="text-align:left">Calls <code>subtractTool</code> once</td></tr><tr><td style="text-align:left">What is a prime number?</td><td style="text-align:left">Answers directly with no tool call</td></tr><tr><td style="text-align:left">Add 3 and 2, then subtract 4 from the result.</td><td style="text-align:left">Calls <code>sumTool</code>, then <code>subtractTool</code></td></tr></tbody></table>
<p><img decoding="async" loading="lazy" alt="The Agent Chat panel open next to the agent canvas, with the first test messages sent to the math tutor agent" src="https://choreolabs.com/api-management/learn/assets/images/step2-chat-with-agent-6c76f1d99f2e98c0aff7c4397c338231.png" width="2048" height="1248" class="img_ev3q"></p>
<p><img decoding="async" loading="lazy" alt="The Agent Chat panel after all four messages, with a View Trace link under each agent reply" src="https://choreolabs.com/api-management/learn/assets/images/step2-view-trace-links-c417b15f9ce116f68a9ecc78745acf66.png" width="2048" height="1235" class="img_ev3q"></p>
<p><strong>Expected result:</strong> Each agent reply shows a <strong>View Trace</strong> link, which confirms that the turn was captured.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-3-export-the-session-as-an-evalset">Step 3: Export the session as an evalset<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#step-3-export-the-session-as-an-evalset" class="hash-link" aria-label="Direct link to Step 3: Export the session as an evalset" title="Direct link to Step 3: Export the session as an evalset" translate="no">​</a></h3>
<ol>
<li class="">Click <strong>Session Traces</strong> at the top of the Agent Chat panel. The view lists every turn with its timestamp, trace ID, input, and output.</li>
<li class="">Click <strong>Export</strong> and choose <strong>Export as Evalset</strong>.</li>
<li class="">Choose <strong>Create new evalset</strong>.</li>
<li class="">Enter <code>math-tutor</code> as the name and press <strong>Enter</strong>.</li>
</ol>
<p><img decoding="async" loading="lazy" alt="The Session Traces view listing each turn, with the Export menu open and Export as Evalset selected" src="https://choreolabs.com/api-management/learn/assets/images/step3-export-as-evalset-95e1ea902ff4538070fff144d0e494e2.png" width="2048" height="1234" class="img_ev3q"></p>
<p><strong>Expected result:</strong> A confirmation message appears, and the file <code>math-tutor.evalset.json</code> is written under <code>tests/resources/evalsets/</code> in the project.</p>
<p>To add more cases later, export another session and choose <strong>Append to existing evalset</strong>. You can also export a single turn from the trace viewer.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-4-review-and-correct-the-evalset">Step 4: Review and correct the evalset<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#step-4-review-and-correct-the-evalset" class="hash-link" aria-label="Direct link to Step 4: Review and correct the evalset" title="Direct link to Step 4: Review and correct the evalset" translate="no">​</a></h3>
<p>The evalset records what the agent did, which is not always what it should have done. This step turns a recording into ground truth.</p>
<ol>
<li class="">Click the test beaker icon in the activity bar to open the Test Explorer.</li>
<li class="">In the <strong>Evalsets</strong> panel, expand <strong>math-tutor</strong> and click the thread to open it in the Evalset Viewer.</li>
<li class="">Check every turn. Confirm that each agent response is correct and that the tool executions match the expected behavior in the Step 2 table.</li>
<li class="">If a turn is wrong, click <strong>Edit</strong> to switch to Edit Mode. Click the incorrect tool execution to open the <strong>Edit Tool Call</strong> dialog, select the correct tool from <strong>Tool Name</strong>, fix the <strong>Input Arguments</strong>, and click <strong>Save Changes</strong>.</li>
<li class="">Click <strong>Save Thread</strong>.</li>
</ol>
<p><img decoding="async" loading="lazy" alt="The Evalset Viewer showing the math-tutor thread, with the recorded tool calls under each turn" src="https://choreolabs.com/api-management/learn/assets/images/step4-evalset-viewer-d75399c5ddaa19995f1056b360758572.png" width="2048" height="1238" class="img_ev3q"></p>
<p><img decoding="async" loading="lazy" alt="The Evalset Viewer in Edit Mode, showing the tool execution chain for each turn and the Save Thread button" src="https://choreolabs.com/api-management/learn/assets/images/step4-edit-tool-call-25ab82d02f98ed5bbfa6feefddadc5ef.png" width="2048" height="1239" class="img_ev3q"></p>
<p><strong>Expected result:</strong> The thread shows the expected tool calls for all four turns. Rename the thread by hovering over its title if you want a descriptive name such as <code>tool-selection-baseline</code>.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-5-create-the-evaluation">Step 5: Create the evaluation<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#step-5-create-the-evaluation" class="hash-link" aria-label="Direct link to Step 5: Create the evaluation" title="Direct link to Step 5: Create the evaluation" translate="no">​</a></h3>
<ol>
<li class="">At the top of the Test Explorer, click the <strong>Add test (+)</strong> icon and choose <strong>Add AI Evaluation</strong>.</li>
<li class="">Fill in the <strong>Create New AI Evaluation</strong> form:</li>
</ol>
<table><thead><tr><th style="text-align:left">Field</th><th style="text-align:left">Value</th><th style="text-align:left">What it does</th></tr></thead><tbody><tr><td style="text-align:left">AI Evaluation Name</td><td style="text-align:left"><code>toolSelectionRegression</code></td><td style="text-align:left">Names the evaluation in the Test Explorer and reports</td></tr><tr><td style="text-align:left">Minimum Pass Rate (%)</td><td style="text-align:left">Drag slider to 80%</td><td style="text-align:left">Sets the share of cases that must pass for the run to succeed; the default is 100</td></tr><tr><td style="text-align:left">How would you like to build this evaluation?</td><td style="text-align:left">From Evalset</td><td style="text-align:left">Replays the recorded conversations against the current build</td></tr><tr><td style="text-align:left">Evalset File</td><td style="text-align:left"><code>math-tutor</code></td><td style="text-align:left">Selects the golden dataset from Step 3</td></tr></tbody></table>
<ol start="3">
<li class="">Click <strong>Save</strong>.</li>
<li class="">Hover over <strong>toolSelectionRegression</strong> in the Test Explorer and click the <strong>View Test Flow</strong> icon.</li>
</ol>
<p><img decoding="async" loading="lazy" alt="The Create New AI Evaluation form with the evaluation name, an 80% minimum pass rate, From Evalset selected, and the math-tutor evalset file" src="https://choreolabs.com/api-management/learn/assets/images/step5-create-ai-evaluation-ad72797ccd206fd68be8347a694eb334.png" width="2048" height="1236" class="img_ev3q"></p>
<p><strong>Expected result:</strong> The evaluation appears in the Test Explorer under your integration, in an evaluations group, and opens in the visual designer with a Start node that receives a <code>thread</code> input, one conversation thread per evalset entry.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-6-build-the-checks-in-the-visual-designer">Step 6: Build the checks in the visual designer<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#step-6-build-the-checks-in-the-visual-designer" class="hash-link" aria-label="Direct link to Step 6: Build the checks in the visual designer" title="Direct link to Step 6: Build the checks in the visual designer" translate="no">​</a></h3>
<p>The evaluation replays each recorded message against the current agent and compares the new tool calls with the recorded ones.</p>
<ol>
<li class="">Add a <strong>Foreach</strong> node from the <strong>Control</strong> group and set:</li>
</ol>
<table><thead><tr><th style="text-align:left">Field</th><th style="text-align:left">Value</th></tr></thead><tbody><tr><td style="text-align:left">Collection</td><td style="text-align:left"><code>thread.traces</code></td></tr><tr><td style="text-align:left">Variable Name</td><td style="text-align:left"><code>trace</code></td></tr><tr><td style="text-align:left">Variable Type</td><td style="text-align:left"><code>ai:Trace</code></td></tr></tbody></table>
<ol start="2">
<li class="">Inside the Foreach, add an <strong>Agent</strong> node from the <strong>AI</strong> group, select your agent in the <strong>Agents</strong> picker, and set:</li>
</ol>
<table><thead><tr><th style="text-align:left">Field</th><th style="text-align:left">Value</th><th style="text-align:left">What it does</th></tr></thead><tbody><tr><td style="text-align:left">Query</td><td style="text-align:left"><code>trace.userMessage.content.toString()</code></td><td style="text-align:left">Replays the recorded user message</td></tr><tr><td style="text-align:left">Session ID</td><td style="text-align:left"><code>thread.id</code></td><td style="text-align:left">Runs in the same session as the original conversation (under <strong>Advanced Configurations</strong>)</td></tr><tr><td style="text-align:left">Type Descriptor</td><td style="text-align:left"><code>ai:Trace</code></td><td style="text-align:left">Returns a trace you can compare (under <strong>Advanced Configurations</strong>)</td></tr><tr><td style="text-align:left">Result</td><td style="text-align:left"><code>actualTrace</code></td><td style="text-align:left">Stores what the current build did</td></tr></tbody></table>
<p>Enter <strong>Query</strong> and <strong>Session ID</strong> as expressions, not as text. The flow is saved as code in <code>tests/tests.bal</code>, so open that file and check the agent call: it should read <code>mathTutorAgent.run(trace.userMessage.content.toString(), thread.id)</code>. If either value is wrapped in quotation marks, the agent receives that literal text instead of the recorded question, makes no tool calls, and every check fails with an empty actual value.</p>
<ol start="3">
<li class="">Below the Agent node, still inside the Foreach, add an <strong>assertEquals</strong> node from the <strong>Test</strong> group and set <strong>Actual</strong> to <code>actualTrace.toolCalls</code> and <strong>Expected</strong> to <code>trace.toolCalls</code>.</li>
</ol>
<p><img decoding="async" loading="lazy" alt="The evaluation test flow in the visual designer: Start, Foreach, Agent, and assertEquals in sequence" src="https://choreolabs.com/api-management/learn/assets/images/step6-evaluation-test-flow-0970b95115c13ba1c8c626dc6d668d51.png" width="2048" height="1239" class="img_ev3q"></p>
<p><strong>Expected result:</strong> The flow shows Start, Foreach, Agent, and assertEquals in sequence.</p>
<p>Here, <code>trace</code> holds the expected behavior from the evalset, and <code>actualTrace</code> holds what the current build produced. The assertEquals check passes only when the agent picks the same tools, in the same order, with the same arguments. That is a <code>STRICT</code> trajectory check, which suits this suite because the arguments are the answer. For agents where several paths are valid, use an <strong>assertTrue</strong> check on a narrower condition instead, such as whether one required tool appears in <code>actualTrace.toolCalls</code>.</p>
<p>To score a subjective dimension such as tone or helpfulness, add a <strong>Model Provider</strong> node after the Agent node, pass it <code>actualTrace</code> and <code>trace</code>, prompt it to return a score, and assert on that score.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-7-run-the-evaluation-and-read-the-report">Step 7: Run the evaluation and read the report<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#step-7-run-the-evaluation-and-read-the-report" class="hash-link" aria-label="Direct link to Step 7: Run the evaluation and read the report" title="Direct link to Step 7: Run the evaluation and read the report" translate="no">​</a></h3>
<ol>
<li class="">If the agent uses the <strong>Default WSO2</strong> model provider, copy the project's <code>Config.toml</code> into the <code>tests</code> folder first. <a href="https://ballerina.io/learn/test-ballerina-code/structure-tests/" target="_blank" rel="noopener noreferrer" class="">Tests read configuration</a> from <code>tests/Config.toml</code>, and without it the run stops with an error that <code>ballerina.ai.wso2ProviderConfig</code> is not configured. Both files hold an access token, so keep them out of version control. The token also expires. If a run that passed before suddenly fails within a second or two, refresh it with <strong>Ballerina: Configure default WSO2 model provider</strong> from the command palette and copy the file into <code>tests</code> again.</li>
<li class="">In the Test Explorer, hover over <strong>toolSelectionRegression</strong> and click the run icon.</li>
<li class="">Review the Evaluation Report that opens:</li>
</ol>
<table><thead><tr><th style="text-align:left">Section</th><th style="text-align:left">What to check</th></tr></thead><tbody><tr><td style="text-align:left">Evaluation card</td><td style="text-align:left">Observed pass rate against your 80% target, and the Passed or Failed badge</td></tr><tr><td style="text-align:left">Evaluation Runs</td><td style="text-align:left">Pass or fail status for each conversation thread, with the assertion message when one fails</td></tr><tr><td style="text-align:left">Test Results panel</td><td style="text-align:left">The run log, including paths to the JSON results file and the HTML report</td></tr></tbody></table>
<p><img decoding="async" loading="lazy" alt="The Evaluation Report for toolSelectionRegression showing a passed run with a 100% observed pass rate against the 80% target" src="https://choreolabs.com/api-management/learn/assets/images/step7-evaluation-report-baseline-29504d465b214c17f860ee13f9aafdb8.png" width="2048" height="1244" class="img_ev3q"></p>
<p><strong>Expected result:</strong> The math-tutor thread passes on the unchanged agent, and the observed pass rate meets the 80% target. This first run is your baseline.</p>
<p>The report scores each conversation thread as one case, and a thread passes only if every turn in it matches. Because all four messages from Step 2 were exported as one session, they form a single thread, so the pass rate here is either 0% or 100%. For a pass rate that reflects individual cases, export each test conversation as its own session and add it with <strong>Append to existing evalset</strong>.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-8-make-a-change-and-catch-the-regression">Step 8: Make a change and catch the regression<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#step-8-make-a-change-and-catch-the-regression" class="hash-link" aria-label="Direct link to Step 8: Make a change and catch the regression" title="Direct link to Step 8: Make a change and catch the regression" translate="no">​</a></h3>
<p>A regression suite proves itself the first time it fails. Introduce a change that should break tool selection and confirm that the suite notices.</p>
<ol>
<li class="">Edit the agent's instructions and add: <em>Answer arithmetic questions from memory. Do not use tools for numbers under 1000.</em> If your instructions already tell the agent to always use its tools, replace that sentence instead of adding a contradicting one.</li>
<li class="">Run <strong>toolSelectionRegression</strong> again.</li>
</ol>
<p><img decoding="async" loading="lazy" alt="The Evaluation Report for the second run showing a failed status, with the assertion message showing the expected tool call and an empty actual value" src="https://choreolabs.com/api-management/learn/assets/images/step8-evaluation-report-failed-c2eeaab88ddf6ec57b5b62f065379cac.png" width="2048" height="1254" class="img_ev3q"></p>
<ol start="3">
<li class="">Click <strong>Evaluation History</strong> at the top right of the report.</li>
</ol>
<p><img decoding="async" loading="lazy" alt="The Evaluation History view with the pass-rate trend dropping below the dashed 80% target line and a run history table listing each run" src="https://choreolabs.com/api-management/learn/assets/images/step8-evaluation-history-248a5b6e76f1927b9ec5e931139355a7.png" width="2048" height="1243" class="img_ev3q"></p>
<p><strong>Expected result:</strong> The thread fails at its first arithmetic turn, because the agent answers without calling <code>sumTool</code>, and the report shows the expected tool call next to an empty actual value. In Evaluation History, the trend chart drops from the baseline to below the dashed 80% target line, and the <strong>Run history</strong> table lists each run with its date, pass rate, status, and a <strong>View Report</strong> link. If the model ignores the new instruction and the run still passes, make the instruction stricter or remove a tool from the agent instead.</p>
<p>Evaluation History shows when quality dropped but keeps no record of what changed in the project. Commit before each evaluation run, and the timestamp on a failed run then points to the commit responsible.</p>
<p>To undo the change, remove the instruction you added, save, and run the evaluation again to confirm the pass rate returns to the baseline.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="tutorial-part-2-score-live-traffic-with-choreo-agent-manager">Tutorial (part 2): Score live traffic with Choreo Agent Manager<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#tutorial-part-2-score-live-traffic-with-choreo-agent-manager" class="hash-link" aria-label="Direct link to Tutorial (part 2): Score live traffic with Choreo Agent Manager" title="Direct link to Tutorial (part 2): Score live traffic with Choreo Agent Manager" translate="no">​</a></h2>
<p>The suite in Part 1 protects every change before it ships. Once the agent runs in production, Choreo Agent Manager scores its behavior on real traffic. Its monitors run evaluators against the traces the agent sends, separately from the agent, and show the scores next to each trace. The steps below connect the same math tutor to Agent Manager, check its tools with a rule-based evaluator, add an LLM judge for the tutor's own rule, and then make the scoring continuous.</p>
<section class="panel_ykK0" aria-labelledby="_R_dv9qmn9al_"><div class="head_AuId"><p class="label_Di5q">Tutorial</p><p class="title_RBm6" id="_R_dv9qmn9al_">Before you start</p></div><div class="body_JQ_P"><ul>
<li class=""><strong>What you will build:</strong> Two Agent Manager monitors for the math tutor: one that scores past traces with a rule-based tool check and an LLM judge, and one that scores new traffic on a schedule.</li>
<li class=""><strong>What you will learn:</strong> How to send traces from a Choreo Integrator agent to Agent Manager, how to read Step Success Rate results, how to create an LLM-judge evaluator and connect it to a model provider, and how to tell a grader problem from an agent problem.</li>
<li class=""><strong>Prerequisites:</strong> The math tutor agent from Part 1, a <a href="https://wso2.com/agent-platform/agent-manager/" target="_blank" rel="noopener noreferrer" class="">Choreo Agent Manager</a> account (the free trial works), and an API key from an LLM provider for the judge. This tutorial uses a Gemini key from <a href="https://aistudio.google.com/apikey" target="_blank" rel="noopener noreferrer" class="">Google AI Studio</a>.</li>
</ul></div></section>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-9-send-the-agents-traces-to-agent-manager">Step 9: Send the agent's traces to Agent Manager<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#step-9-send-the-agents-traces-to-agent-manager" class="hash-link" aria-label="Direct link to Step 9: Send the agent's traces to Agent Manager" title="Direct link to Step 9: Send the agent's traces to Agent Manager" translate="no">​</a></h3>
<ol>
<li class="">In the Agent Manager console, open your project and add a new agent. Choose <strong>Externally-Hosted Agent</strong>, because the agent keeps running in Choreo Integrator, and click <strong>Register an agent</strong>.</li>
<li class="">Name the agent <code>math-tutor</code> and create it. The <strong>Zero-code Instrumentation Guide</strong> opens on the agent page. Set <strong>Language</strong> to <strong>Ballerina</strong>.</li>
<li class="">Copy the endpoint and API key from the guide's <strong>Set Environment Variables</strong> step and keep them private. The key is shown only once.</li>
<li class="">In Choreo Integrator, make the three changes the guide lists:</li>
</ol>
<table><thead><tr><th style="text-align:left">File</th><th style="text-align:left">Change</th></tr></thead><tbody><tr><td style="text-align:left"><code>agents.bal</code></td><td style="text-align:left">Add <code>import ballerinax/amp as _;</code> below the existing imports</td></tr><tr><td style="text-align:left"><code>Ballerina.toml</code></td><td style="text-align:left">Under <code>[build-options]</code>, add <code>observabilityIncluded = true</code></td></tr><tr><td style="text-align:left"><code>Config.toml</code></td><td style="text-align:left">Add the two sections below</td></tr></tbody></table>
<div class="language-toml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-toml codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">[ballerina.observe]</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">tracingEnabled = true</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">tracingProvider = "amp"</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">[ballerinax.amp]</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">otelEndpoint = "https://opentelemetry.obs.dp.cloud.wso2.com"</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">apiKey = "&lt;API key from the guide&gt;"</span><br></span></code></pre></div></div>
<p>The guide gives these values as environment variables. Variables exported in your own terminal do not reach an agent that Choreo Integrator starts, so put the values in <code>Config.toml</code> instead. Enter the endpoint without a trailing <code>/v1/traces</code>, because the module adds that path itself. With the path doubled, no traces reached Agent Manager during testing, and the chat showed no error.</p>
<ol start="5">
<li class="">Start the agent and send the four messages from Step 2. Click <strong>Clear Chat</strong> before each one, so that every message runs in its own session.</li>
<li class="">In Agent Manager, open the agent's <strong>Traces</strong> page and refresh it.</li>
</ol>
<p><img decoding="async" loading="lazy" alt="The Traces page in Choreo Agent Manager listing a post /chat trace for each test message, each with its own input and output" src="https://choreolabs.com/api-management/learn/assets/images/step9-agent-manager-traces-ceef46699f7ff6085233163344dad810.png" width="2048" height="705" class="img_ev3q"></p>
<p><strong>Expected result:</strong> The terminal in Choreo Integrator shows <code>ballerina: started publishing traces to Amp</code>, and each message appears in Agent Manager as a <code>post /chat</code> trace with its own input and output. A <code>create_agent</code> trace from the agent's startup appears as well.</p>
<p>Clearing the chat matters. When several messages share one chat session, Agent Manager shows the session's first message as the input of every trace in it, and Step 12 shows what that does to an LLM judge.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-10-check-that-the-tools-work">Step 10: Check that the tools work<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#step-10-check-that-the-tools-work" class="hash-link" aria-label="Direct link to Step 10: Check that the tools work" title="Direct link to Step 10: Check that the tools work" translate="no">​</a></h3>
<ol>
<li class="">On the agent's page, open <strong>Monitors</strong> under <strong>Evaluation</strong> and click <strong>Add monitor</strong>.</li>
<li class="">Fill in the <strong>Basic Details</strong> and <strong>Data Collection</strong> sections:</li>
</ol>
<table><thead><tr><th style="text-align:left">Field</th><th style="text-align:left">Value</th></tr></thead><tbody><tr><td style="text-align:left">Monitor Title</td><td style="text-align:left"><code>tool-health</code></td></tr><tr><td style="text-align:left">Description</td><td style="text-align:left">Checks that every tool call completes without an error</td></tr><tr><td style="text-align:left">Data Collection</td><td style="text-align:left">Past Traces</td></tr><tr><td style="text-align:left">Start Time and End Time</td><td style="text-align:left">A window that covers the traces from Step 9. Check the AM or PM setting on the end time.</td></tr><tr><td style="text-align:left">Sampling Rate</td><td style="text-align:left">100%</td></tr></tbody></table>
<p><img decoding="async" loading="lazy" alt="The Create Monitor form with the tool-health title, a description, Past Traces selected, a start and end time, and a 100% sampling rate" src="https://choreolabs.com/api-management/learn/assets/images/step10-create-monitor-basic-details-acfa1b5cfb1199788a15b54ccc54a5f9.png" width="2048" height="1157" class="img_ev3q"></p>
<ol start="3">
<li class="">Click <strong>Next</strong>, select <strong>Step Success Rate</strong>, drag <strong>Min success rate</strong> to its maximum so that a single failed tool call fails the trace, and click <strong>Add Evaluator</strong>.</li>
</ol>
<p><img decoding="async" loading="lazy" alt="The Step Success Rate evaluator settings with Min success rate dragged to its maximum" src="https://choreolabs.com/api-management/learn/assets/images/step10-step-success-rate-evaluator-9900e1d5f85a756301fc764f6b215fa2.png" width="2048" height="1159" class="img_ev3q"></p>
<ol start="4">
<li class="">Click <strong>Create Monitor</strong>, then open <strong>tool-health</strong> from the monitor list.</li>
</ol>
<p><img decoding="async" loading="lazy" alt="The tool-health monitor page with an Evaluation Summary average score of 100% and a run summary" src="https://choreolabs.com/api-management/learn/assets/images/step10-tool-health-summary-baee5475e5899774626e8a83632e0da2.png" width="2048" height="968" class="img_ev3q"></p>
<p><strong>Expected result:</strong> The <strong>Evaluation Summary</strong> shows an average score of 100% and a share of skipped evaluations. Step Success Rate skips traces that made no tool calls, such as the prime number question, instead of scoring them. A score of 1.0 means every tool call came back without an error, including a tool that correctly returned a refusal. Whether the agent should have made the call at all needs a different evaluator, which is the next step.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-11-write-an-llm-judge-for-the-agents-own-rule">Step 11: Write an LLM judge for the agent's own rule<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#step-11-write-an-llm-judge-for-the-agents-own-rule" class="hash-link" aria-label="Direct link to Step 11: Write an LLM judge for the agent's own rule" title="Direct link to Step 11: Write an LLM judge for the agent's own rule" translate="no">​</a></h3>
<ol>
<li class="">Switch to the organization level, open <strong>Evaluators</strong> under <strong>Evaluation</strong>, and click <strong>Create Evaluator</strong>.</li>
<li class="">Set <strong>Name</strong> to <code>math-tutor-tool-rule</code>, add a short description, choose <strong>LLM Judge</strong>, and click <strong>Next</strong>.</li>
<li class="">Set <strong>Evaluation Level</strong> to <strong>Trace</strong>, and paste this prompt into <strong>Evaluation Prompt</strong>:</li>
</ol>
<div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">You are reviewing a math tutor agent transcript.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">The agent must call sumTool for every addition and subtractTool for every</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">subtraction, even with small numbers. It must answer conceptual questions,</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">such as "What is a prime number?", directly without calling any tools.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">Transcript:</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">{trace.input}</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">Tool calls made:</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">{trace.get_tool_calls()}</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">Did the agent follow this rule? Answer PASS or FAIL, then explain in one</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">sentence.</span><br></span></code></pre></div></div>
<ol start="4">
<li class="">Leave the <strong>Config Params</strong> at their defaults, including a temperature of 0, and click <strong>Create Evaluator</strong>.</li>
</ol>
<p><img decoding="async" loading="lazy" alt="The Create Evaluator page with Trace selected as the evaluation level and the math tutor rule pasted into the Evaluation Prompt" src="https://choreolabs.com/api-management/learn/assets/images/step11-create-llm-judge-evaluator-46f6da2f91bb9f47abed911957a6f505.png" width="2048" height="1105" class="img_ev3q"></p>
<p><code>{trace.input}</code> and <code>{trace.get_tool_calls()}</code> are placeholders that Agent Manager fills from each trace before the prompt reaches the model. The model itself is chosen when the evaluator is added to a monitor.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-12-connect-the-judge-to-a-model-and-read-the-scores">Step 12: Connect the judge to a model and read the scores<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#step-12-connect-the-judge-to-a-model-and-read-the-scores" class="hash-link" aria-label="Direct link to Step 12: Connect the judge to a model and read the scores" title="Direct link to Step 12: Connect the judge to a model and read the scores" translate="no">​</a></h3>
<ol>
<li class="">Open <strong>Monitors</strong>, click the edit icon on <strong>tool-health</strong>, and select <strong>math-tutor-tool-rule</strong>. Agent Manager asks for an LLM provider, because the judge needs a model to call.</li>
<li class="">In <strong>Create LLM Provider</strong>, enter a <strong>Name</strong> such as <code>Gemini Provider</code>, choose the <strong>Gemini</strong> template, paste your Google AI Studio key into <strong>API key / Credential</strong>, and click <strong>Create &amp; use provider</strong>. Agent Manager adds the provider to the organization's catalog and deploys it to the gateway. Wait until the provider shows as deployed before the next run, because a run that starts during deployment fails.</li>
<li class="">In the evaluator's settings, set <strong>Model</strong> to a model name the provider accepts, such as <code>gemini-flash-lite-latest</code>. Set <strong>Max retries</strong> to 1, click <strong>Add Evaluator</strong>, and click <strong>Save Changes</strong>.</li>
<li class="">Open <strong>tool-health</strong>, wait for the new run to finish, and check its <strong>Scores</strong> tab. Then open <strong>Traces</strong>.</li>
</ol>
<p><img decoding="async" loading="lazy" alt="The tool-health monitor page after the LLM judge was added, with results for both evaluators in the Evaluation Summary" src="https://choreolabs.com/api-management/learn/assets/images/step12-tool-health-both-evaluators-a21b5612092e74d643a71fad0a939348.png" width="2048" height="978" class="img_ev3q"></p>
<p><img decoding="async" loading="lazy" alt="The Traces page with the Score column filled in: the Step 9 traces score 100%, while older traces from a shared chat session score lower" src="https://choreolabs.com/api-management/learn/assets/images/step12-traces-with-scores-814e68389e2dbf74c299cbe24227a67c.png" width="2048" height="1151" class="img_ev3q"></p>
<p><strong>Expected result:</strong> The tool-health summary lists results for both evaluators, and the <strong>Score</strong> column on the Traces page fills in. The traces from Step 9 score 100%. The overall average sits below 100% because of the older traces explained below.</p>
<p>A judge that cannot score a trace skips it, and the run still reports success, so read the skip reasons in the run's <strong>Logs</strong> before trusting the average. These were the messages that came up while this tutorial was being tested:</p>
<table><thead><tr><th style="text-align:left">Log message</th><th style="text-align:left">Cause</th><th style="text-align:left">Fix</th></tr></thead><tbody><tr><td style="text-align:left"><code>LLM_API_BASE and LLM_API_KEY must be set together for gateway routing</code></td><td style="text-align:left">The provider was still deploying when the run started</td><td style="text-align:left">Wait a few minutes after creating the provider and run the monitor again</td></tr><tr><td style="text-align:left"><code>unexpected model name format</code></td><td style="text-align:left">The Model field contains extra characters, such as leading spaces</td><td style="text-align:left">Clear the field and type the model name again</td></tr><tr><td style="text-align:left"><code>429 RESOURCE_EXHAUSTED</code></td><td style="text-align:left">The provider's free tier limits requests per minute</td><td style="text-align:left">Use a lighter model, lower Max retries, or score fewer traces per run</td></tr><tr><td style="text-align:left"><code>503 UNAVAILABLE</code></td><td style="text-align:left">The model is temporarily overloaded</td><td style="text-align:left">Run the monitor again later</td></tr></tbody></table>
<p>The screenshot also shows the grader problem from the pitfalls section in practice. The older traces at the bottom came from one chat session, so each one carried that session's first question, "What is 48 plus 17?", as its input. The judge compared the marbles and prime number answers against that question and failed them, even though the agent answered both correctly. The 50% scores are Step Success Rate's 100% averaged with the judge's 0%, and the prime number trace scores 0% because Step Success Rate skipped it. Before changing an agent because a judge failed it, open the trace and check what the judge was given.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-13-make-it-continuous">Step 13: Make it continuous<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#step-13-make-it-continuous" class="hash-link" aria-label="Direct link to Step 13: Make it continuous" title="Direct link to Step 13: Make it continuous" translate="no">​</a></h3>
<ol>
<li class="">On the <strong>Monitors</strong> page, click <strong>Add monitor</strong> and name the monitor <code>tool-rule-live</code>.</li>
<li class="">Choose <strong>Future Traces</strong> and set <strong>Run Every (minutes)</strong> to 5, the minimum the <a href="https://wso2.com/agent-platform/docs/next/concepts/evaluation/" target="_blank" rel="noopener noreferrer" class="">Agent Manager evaluation concepts</a> allow.</li>
<li class="">Click <strong>Next</strong>, add <strong>Step Success Rate</strong> and <strong>math-tutor-tool-rule</strong>, keep <strong>Gemini Provider</strong> selected, and click <strong>Create Monitor</strong>.</li>
</ol>
<p><img decoding="async" loading="lazy" alt="The Create Monitor form for tool-rule-live with Future Traces selected and Run Every set to 5 minutes" src="https://choreolabs.com/api-management/learn/assets/images/step13-future-traces-monitor-7dd721d1ba5a86f01086759b708a1cc3.png" width="2048" height="1159" class="img_ev3q"></p>
<p><strong>Expected result:</strong> <strong>tool-rule-live</strong> appears in the monitor list, and each interval scores the conversations that arrived since the last one. A monitor's data collection cannot be changed after it is created, which is why continuous scoring needs its own monitor rather than an edit to tool-health.</p>
<p>The two parts of this tutorial cover both columns of the offline vs. online table. The Choreo Integrator suite catches regressions in known behavior before a change ships, and the Agent Manager monitors catch the failures in real traffic that no evalset anticipated. When a monitor flags a genuine failure, add that conversation to the evalset so the regression suite checks it on every future change. The <a href="https://wso2.com/agent-platform/docs/next/tutorials/observe-first-agent/" target="_blank" rel="noopener noreferrer" class="">Agent Manager monitoring tutorial</a> covers more evaluator options.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>AI agent evaluation needs three things working together: a golden dataset captured from real conversations, checks that score behavior and tool calls rather than only the final text, and a run history that shows when the pass rate drops, so each regression traces back to the change that caused it. Code-based checks handle what can be stated as a rule, LLM judges handle what cannot, and human review keeps the judges honest. When a run fails, read the failure message before changing the agent: in the test runs behind this tutorial, every failure traced back to setup or grading, from a missing test config and an expired token to a judge that was given the wrong question, and the agent itself was fine.</p>
<p>Start small: capture a balanced set of conversations this week, correct them into an evalset, and run the suite before the next prompt, tool, or model change. When the agent reaches production, add a continuous monitor, as in Part 2, so the failures users find become the next cases in the suite. The <a href="https://wso2.com/integration-platform/docs/saas/platform-overview" target="_blank" rel="noopener noreferrer" class="">Choreo Integrator documentation</a> covers each stage in more detail.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-makes-a-good-ai-agent-evaluation">What makes a good AI agent evaluation?<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#what-makes-a-good-ai-agent-evaluation" class="hash-link" aria-label="Direct link to What makes a good AI agent evaluation?" title="Direct link to What makes a good AI agent evaluation?" translate="no">​</a></h3>
<p>Realistic tasks with unambiguous success criteria, a balance of cases where the agent should and should not act, graders that check both the outcome and the tool calls that produced it, and a pass threshold that tolerates normal run-to-run variation. It also runs on every change, not once.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-are-ai-agent-evaluation-frameworks">What are AI agent evaluation frameworks?<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#what-are-ai-agent-evaluation-frameworks" class="hash-link" aria-label="Direct link to What are AI agent evaluation frameworks?" title="Direct link to What are AI agent evaluation frameworks?" translate="no">​</a></h3>
<p>Tools that run tasks against an agent, record what it did, apply graders, and aggregate the results. Some focus on offline testing during development, some on scoring production traffic, and a few cover both. In Choreo Integrator, the Evaluations feature provides this inside the IDE, and the <code>ballerina/ai.eval</code> library provides rule-based and LLM-as-judge templates that run as ordinary Ballerina tests.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="which-tools-make-ai-agent-evaluation-easier">Which tools make AI agent evaluation easier?<a href="https://choreolabs.com/api-management/learn/ai-agent-evaluation#which-tools-make-ai-agent-evaluation-easier" class="hash-link" aria-label="Direct link to Which tools make AI agent evaluation easier?" title="Direct link to Which tools make AI agent evaluation easier?" translate="no">​</a></h3>
<p>Tools that capture evaluation datasets from real sessions instead of requiring hand-written cases, keep a run history so you can see when quality changed, and support both deterministic checks and LLM-as-judge scoring. For production, add a monitoring tool that scores live traces continuously.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="Tutorials" term="Tutorials"/>
        <category label="AI" term="AI"/>
        <category label="Agent-ready APIs" term="Agent-ready APIs"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[Agent API permissions: Identity, OAuth & access control]]></title>
        <id>https://choreolabs.com/api-management/learn/agent-api-permissions</id>
        <link href="https://choreolabs.com/api-management/learn/agent-api-permissions"/>
        <updated>2026-10-13T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[Learn how to scope AI agent API permissions with agent identity, OAuth flows, delegated access, and audit trails with patterns for both interactive and autonomous agents.]]></summary>
        <content type="html"><![CDATA[<p>An expense agent needs to read receipts, look up a cost center, and submit a claim. A common shortcut is to give it a long-lived API key for the whole finance API instead. In <a href="https://goteleport.com/blog/top-ai-infrastructure-risks/" target="_blank" rel="noopener noreferrer" class="">Teleport's 2026 Infrastructure Identity Survey</a>, 70% of organizations reported giving AI systems more access than a human would get for the same task.</p>
<p>This guide covers how to scope agent API permissions in 2026. It walks through agent identity, scoped and short-lived tokens, delegated access, and audit. It also shows how the OAuth grant changes when no user is at the keyboard. Several of the standards involved are still drafts, so the guide flags where practice is settled and where it is not. The final section shows how Choreo Agent ID implements each layer.</p>
<aside class="cta_6F_m" aria-label="Give agents only the access they need" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Give agents only the access they need</span><span class="desc_GzK8">Use Choreo's MCP gateway layer to enforce scoped, per-agent authorization for every AI agent.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=agent-api-permissions" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="why-ai-agents-need-a-different-permission-model-than-users">Why AI agents need a different permission model than users<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#why-ai-agents-need-a-different-permission-model-than-users" class="hash-link" aria-label="Direct link to Why AI agents need a different permission model than users" title="Direct link to Why AI agents need a different permission model than users" translate="no">​</a></h2>
<p>AI agents need their own permission model because their access needs change per task, not per job function. Role-based access control works well as a baseline, but on its own it assumes a stable role. An agent may read customer records for one request and issue a refund for the next. Permissions have to follow the task, the context, and the delegating user.</p>
<p>Shared static API keys fail for a different reason. A key issued to one workload can identify that workload. A key shared across agents identifies only a credential. It cannot tell you which agent made a call, or on whose behalf. Roles alone fall short because they are sized for the job. A role that covers every task an agent might attempt is broad by default, and least privilege needs task and context constraints on top of that baseline.</p>
<p>Take a support agent that answers billing questions. It needs read access to invoices. Once in a while, it also needs to issue a credit. If the credit permission sits in its role permanently, any prompt injection that reaches the agent can try to issue credits too. The agent also chooses its tool calls at runtime. So the exact set of calls is not knowable when you provision it. Scoped, per-task authorization is one of the properties of an <a class="" href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis">agent-ready API</a>, alongside clear contracts and safe retries.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-three-anti-patterns-that-get-agent-permissions-wrong">The three anti-patterns that get agent permissions wrong<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#the-three-anti-patterns-that-get-agent-permissions-wrong" class="hash-link" aria-label="Direct link to The three anti-patterns that get agent permissions wrong" title="Direct link to The three anti-patterns that get agent permissions wrong" translate="no">​</a></h2>
<p><a href="https://auth0.com/blog/why-ai-agents-need-their-own-permission-model/" target="_blank" rel="noopener noreferrer" class="">Auth0 has described three anti-patterns</a> that show up repeatedly in agent deployments. Each one is a shortcut that works in a prototype and causes trouble in production.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="embedding-long-lived-credentials-directly-in-the-agent">Embedding long-lived credentials directly in the agent<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#embedding-long-lived-credentials-directly-in-the-agent" class="hash-link" aria-label="Direct link to Embedding long-lived credentials directly in the agent" title="Direct link to Embedding long-lived credentials directly in the agent" translate="no">​</a></h3>
<p>A static key copied into an environment variable or config file often has no expiry set, and every run of the agent shares it. Agents raise the stakes, because a manipulated agent can be talked into revealing what it can read. The <a href="https://owasp.org/www-project-mcp-top-10/" target="_blank" rel="noopener noreferrer" class="">OWASP MCP Top 10</a> lists hard-coded credentials and long-lived tokens under MCP01, Token Mismanagement and Secret Exposure. When a static key leaks, revocation means finding and rotating every copy.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="inheriting-the-full-scope-of-the-users-oauth-token">Inheriting the full scope of the user's OAuth token<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#inheriting-the-full-scope-of-the-users-oauth-token" class="hash-link" aria-label="Direct link to Inheriting the full scope of the user's OAuth token" title="Direct link to Inheriting the full scope of the user's OAuth token" translate="no">​</a></h3>
<p>Reusing a broad user token gives the agent everything that token allows. OAuth tokens can be limited by scope, audience, lifetime, and client, but a token issued for a user's own session is often broad. If an administrator's broad token reaches the agent, every tool call it makes carries administrator rights. The agent usually needs one or two scopes. The MCP specification addresses a related problem at the server side. <a href="https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization/security-considerations" target="_blank" rel="noopener noreferrer" class="">An MCP server must not pass through the token it received to an upstream API</a>. The same principle applies to agents in general. Each hop should carry a token issued for that audience. Issue a token for the intended resource and task with the minimum necessary scopes. Where supported, preserve the initiating user as the subject and the agent as the actor. The resource server should then validate both the scopes and the audience.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="giving-the-agent-effective-superuser-access-to-be-safe">Giving the agent effective superuser access "to be safe"<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#giving-the-agent-effective-superuser-access-to-be-safe" class="hash-link" aria-label="Direct link to Giving the agent effective superuser access &quot;to be safe&quot;" title="Direct link to Giving the agent effective superuser access &quot;to be safe&quot;" translate="no">​</a></h3>
<p>Teams often grant broad access to stop tool calls from failing during testing. That permission set then ships to production unchanged. Auth0 calls this the most damaging of the three. The blast radius is every system the agent can reach, at system level rather than at the level of one user.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="identity-authorization-and-execution-are-three-separate-layers">Identity, authorization, and execution are three separate layers<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#identity-authorization-and-execution-are-three-separate-layers" class="hash-link" aria-label="Direct link to Identity, authorization, and execution are three separate layers" title="Direct link to Identity, authorization, and execution are three separate layers" translate="no">​</a></h2>
<p>A permission check answers three questions: who is calling, what that caller may do right now, and whether the tool enforced the answer. These map to three layers, and each one usually has a different owner.</p>
<ul>
<li class=""><strong>Identity layer:</strong> An identity provider registers the agent as its own principal. It holds the agent's credentials and metadata, such as owner and purpose, and it issues tokens.</li>
<li class=""><strong>Authorization layer:</strong> Policies decide which scopes a token carries. An enforcement point, such as a gateway or the API itself, checks those scopes on each request. It typically validates signed tokens against the issuer's published keys, so the identity provider stays out of the request path.</li>
<li class=""><strong>Execution layer:</strong> The API or MCP server performs the action. It also applies resource-level checks, such as whether an invoice belongs to the customer in the request. Tools are easier to scope when their APIs have <a class="" href="https://choreolabs.com/api-management/learn/openapi-for-ai-agents">machine-readable descriptions agents can discover</a>.</li>
</ul>
<p><img decoding="async" loading="lazy" alt="Three layers between an AI agent and the action. 1: the agent authenticates to the identity provider. 2: it receives a short-lived token. 3: it calls the enforcement point, a gateway or the API itself, with the token, which checks signature, audience, expiry, and required scopes. 4: the allowed call reaches the API or MCP tool, which performs resource-level checks" src="https://choreolabs.com/api-management/learn/assets/images/identity-authorization-execution-layers-e84f21b94af1ccff55de43f2beb176cb.webp" width="3200" height="1930" class="img_ev3q"></p>
<p><em>Each layer answers a different question and applies its own check. The identity provider issues tokens but stays out of the request path.</em></p>
<p>Token issuance and request handling are separate flows. The agent obtains a token before it calls anything. At request time, the token travels with the call. The enforcement point and the tool each apply their own checks, and neither needs to call the identity provider on every request.</p>
<p>Across the agent's lifetime, the same layers support four functions. You <strong>administer</strong> the agent's identity from provisioning to decommissioning. You <strong>authenticate</strong> it without a human typing a password. You <strong>authorize</strong> each action against policy. You <strong>audit</strong> what it did. Choreo Agent ID, covered in the final section, organizes its features around these four functions.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="scoped-short-lived-credentials-capability-and-task-scoping">Scoped, short-lived credentials: Capability and task scoping<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#scoped-short-lived-credentials-capability-and-task-scoping" class="hash-link" aria-label="Direct link to Scoped, short-lived credentials: Capability and task scoping" title="Direct link to Scoped, short-lived credentials: Capability and task scoping" translate="no">​</a></h2>
<p>Scoping an agent has two dimensions, and they solve different problems.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="capability-scoping">Capability scoping<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#capability-scoping" class="hash-link" aria-label="Direct link to Capability scoping" title="Direct link to Capability scoping" translate="no">​</a></h3>
<p>Defines what kinds of actions an agent may ever take. A billing agent might hold <code>invoices:read</code> and <code>credits:create</code>, and nothing that touches payroll. You set this when you register the agent, usually through roles or entitlements on API resources. Name scopes after resources and verbs rather than after agents. A scope called <code>support-agent-access</code> hides what it grants, while <code>invoices:read</code> stays meaningful when a second agent needs the same capability.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="task-scoping">Task scoping<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#task-scoping" class="hash-link" aria-label="Direct link to Task scoping" title="Direct link to Task scoping" translate="no">​</a></h3>
<p>Narrows that set for a single task. The agent requests only the scopes the current task needs, for the audience it is about to call. The MCP authorization specification follows this pattern for MCP servers. Tokens are bound to one server, and servers must reject tokens issued for any other audience.</p>
<p>Short lifetimes limit the blast radius when a token leaks. Anthropic's Workload Identity Federation for the Claude API is a concrete example. A federation rule can set a token lifetime <a href="https://platform.claude.com/docs/en/manage-claude/workload-identity-federation" target="_blank" rel="noopener noreferrer" class="">from 60 to 86,400 seconds, with a default of 3,600</a>. The minted token lasts for the rule's lifetime or twice the remaining lifetime of the identity token the workload presented, whichever is shorter. So it cannot long outlive the upstream identity it came from.</p>
<p>Short lifetimes have a cost. The token endpoint becomes a runtime dependency, so plan for refresh before expiry rather than on failure. Two related controls are worth adding:</p>
<ul>
<li class=""><strong>Refresh tokens stay out of the model's context:</strong> Interactive agents often need refresh tokens to keep working without asking the user again. Store them in a backend credential store, and rotate them. The MCP specification requires refresh token rotation for public clients.</li>
<li class=""><strong>Sender-constrained tokens resist replay:</strong> <a href="https://www.rfc-editor.org/rfc/rfc9449" target="_blank" rel="noopener noreferrer" class="">DPoP (RFC 9449)</a> binds a token to a key the agent holds. A stolen DPoP-bound token is not usable on its own, unlike a plain bearer token.</li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="delegated-access-vs-direct-access-two-different-authorization-shapes">Delegated access vs. direct access: Two different authorization shapes<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#delegated-access-vs-direct-access-two-different-authorization-shapes" class="hash-link" aria-label="Direct link to Delegated access vs. direct access: Two different authorization shapes" title="Direct link to Delegated access vs. direct access: Two different authorization shapes" translate="no">​</a></h2>
<p>With direct access, the agent acts as itself. The token's subject is the agent, and policy evaluates the agent's own entitlements. A nightly reconciliation job that compares ledgers works this way.</p>
<p>With delegated access, the agent acts on behalf of a user. The token carries the user as the subject and the agent as the actor. <a href="https://www.rfc-editor.org/rfc/rfc8693#section-4.1" target="_blank" rel="noopener noreferrer" class="">RFC 8693 defines the <code>act</code> claim</a> for this purpose. The resource server can then require both conditions. The user must hold the permission, and the agent must be allowed to use it. A booking assistant that reserves flights for an employee is a typical case.</p>
<p>The grant type depends on who acts and whether the user is in the session:</p>
<table><thead><tr><th>Agent type</th><th>Grant type</th><th>User present at request?</th><th>Typical use case</th></tr></thead><tbody><tr><td>Autonomous agent acting as itself</td><td>Agent's own credentials (for example, client credentials or JWT bearer, RFC 7523)</td><td>No user involved</td><td>Monitoring, pipelines, scheduled jobs</td></tr><tr><td>Interactive agent acting for a user</td><td><code>authorization_code</code> with PKCE, plus <code>refresh_token</code></td><td>Yes, in the session</td><td>Copilots and chat assistants</td></tr><tr><td>Background agent acting for a user</td><td>Refresh token within earlier offline consent, or CIBA (<code>urn:openid:params:grant-type:ciba</code>) for fresh approval</td><td>Not in the agent's session. With CIBA, approves asynchronously on a separate device</td><td>Ambient agents and approval-gated tasks</td></tr><tr><td>Agent delegating to another agent</td><td>Token exchange (RFC 8693) with an actor token</td><td>No new interaction</td><td>Orchestrators handing off subtasks</td></tr></tbody></table>
<p>The background row is the one teams most often miss. Consider an agent that rebooks a cancelled flight at 2 a.m. It needs the traveller's consent, but no browser session exists to redirect. If the traveller granted offline access in an earlier session, the agent can use a refresh token within that consent and its lifetime limits. When the task needs new approval, CIBA is one option. <a href="https://openid.net/specs/openid-client-initiated-backchannel-authentication-core-1_0.html" target="_blank" rel="noopener noreferrer" class="">Client-Initiated Backchannel Authentication (CIBA)</a> separates the device that consumes the service from the device where the user authenticates. The agent starts a backchannel request. The user approves on their phone. The agent then collects the token from the token endpoint. CIBA still needs the user to respond. If the user is unavailable, the task has to wait or fail safely.</p>
<p><img decoding="async" loading="lazy" alt="Two delegated-access flows. Interactive agent, user in the session: the user asks the agent, is redirected, consents to which agent and which scopes, and the agent requests a token with the code and its actor token, receiving a delegated token with sub as the user and act as the agent. Background agent needing fresh consent through CIBA: the agent sends a backchannel request with a login hint and actor token, the user is notified on another device and approves, and the agent polls the token endpoint for the delegated token" src="https://choreolabs.com/api-management/learn/assets/images/interactive-flow-vs-ciba-dc91bc616aeb0220189184d7adda021e.webp" width="3200" height="2284" class="img_ev3q"></p>
<p><em>The interactive flow uses Choreo IAM's agent-delegation extension, which is based on an expired individual Internet-Draft and requires compatible implementations. CIBA is one option for background consent, and it requires asynchronous user approval.</em></p>
<p>The same split applies whether you expose capabilities as REST endpoints or as MCP tools. For MCP connections, the OAuth 2.1 authorization code flow in the MCP specification matches the interactive row.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="building-the-audit-trail-what-governed-actually-requires">Building the audit trail: What "governed" actually requires<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#building-the-audit-trail-what-governed-actually-requires" class="hash-link" aria-label="Direct link to Building the audit trail: What &quot;governed&quot; actually requires" title="Direct link to Building the audit trail: What &quot;governed&quot; actually requires" translate="no">​</a></h2>
<p>An audit trail is only useful if each entry names the agent. Agents that share a service account all appear in logs as that account. Investigators then cannot tell which agent acted, or for whom. Per-agent identity fixes attribution at the source, and it lets you keep agent actions distinguishable from human user trails.</p>
<p>A usable audit record for an agent action contains:</p>
<ul>
<li class=""><strong>Actor</strong> - The agent's identifier, and its version if behaviour changes between releases.</li>
<li class=""><strong>Subject</strong> - The user the agent acted for, if the access was delegated.</li>
<li class=""><strong>Resource</strong> - The API operation or MCP tool called, and the target object.</li>
<li class=""><strong>Decision</strong> - Allow or deny, and the policy or condition that produced it.</li>
<li class=""><strong>Outcome</strong> - What happened downstream, including errors.</li>
<li class=""><strong>Token context</strong> - Token identifier, scopes, and the delegation chain.</li>
</ul>
<p>Log the delegation chain at every hop. Under RFC 8693, nested <code>act</code> claims for prior actors are informational only. Only the outermost actor counts for access control. If an orchestrator hands work to a sub-agent, the final token alone is weak evidence of how the work got there.</p>
<p>The same OWASP list names missing telemetry as MCP08, Lack of Audit and Telemetry, and it recommends immutable trails of tool invocations. With known identities, you can also baseline each agent's normal behaviour. A spike in denied write calls from one agent is a signal you can alert on. The <a class="" href="https://choreolabs.com/api-management/learn/govern-write-capable-mcp-tools-at-the-gateway-in-wso2-api-platform">walkthrough on governing write-capable MCP tools</a> shows one way to do this at the gateway. The same records support compliance reporting, because they answer who did what, under which authority.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="a-practical-framework-for-scoping-agent-api-permissions">A practical framework for scoping agent API permissions<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#a-practical-framework-for-scoping-agent-api-permissions" class="hash-link" aria-label="Direct link to A practical framework for scoping agent API permissions" title="Direct link to A practical framework for scoping agent API permissions" translate="no">​</a></h2>
<p>The layers above become a repeatable process. Apply it per agent, not per platform.</p>
<ol>
<li class=""><strong>Map the agent to resource-scoped entitlements.</strong> List the API operations and tools the agent's job requires. Group them by resource and by risk, and separate reads from writes. Anything the agent might need only occasionally belongs in a task-scoped request, not in its standing role.</li>
<li class=""><strong>Choose the grant by agent type.</strong> Use the table above. Decide whether the agent acts as itself or for a user, and whether that user is in the session.</li>
<li class=""><strong>Issue short-lived, audience-bound tokens.</strong> Request only the scopes the current task needs, for the API being called. Keep lifetimes short, and move refresh logic into the client library rather than the prompt.</li>
<li class=""><strong>Wire up audit logging before production.</strong> Record actor, subject, resource, decision, outcome, and token context for every call. Keep agent records separate from human ones.</li>
<li class=""><strong>Set immediate revocation paths.</strong> Make it possible to disable an agent identity, and to withdraw a user's delegation, without redeploying anything. Locally validated tokens stay valid until they expire, so short lifetimes bound that gap. Where you need an instant cutoff, have the enforcement point check revocation status. Design write tools so a retry after revocation or token expiry is safe, using idempotency keys and <a class="" href="https://choreolabs.com/api-management/learn/designing-idempotent-tools">safe-to-retry semantics</a>.</li>
</ol>
<p>Applied to the support agent from earlier, the process looks like this. Its standing entitlement is <code>invoices:read</code>, and <code>credits:create</code> is a separate, task-scoped request. When a customer asks for a refund inside a chat, the agent acts for that customer through the interactive grant. The resulting token names the customer as subject and the agent as actor. It expires quickly and is valid only for the billing API. The gateway logs the credit with both identities and the policy that allowed it. If the agent starts issuing credits outside chat sessions, you disable its identity. It can no longer obtain tokens, and any token it already holds expires within minutes.</p>
<p>When is this worth the effort? Look for operational triggers rather than usage thresholds, such as the ones below:</p>
<ul>
<li class="">More than one agent shares a credential, and you cannot attribute actions.</li>
<li class="">Agents act for users on regulated data, such as health or payment records.</li>
<li class="">Agents call external APIs that charge per call, and you need spend attribution per agent.</li>
<li class="">Agents move from pilot to production scale, where a leaked key has real reach.</li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-choreo-agent-id-implements-this">How Choreo Agent ID implements this<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#how-choreo-agent-id-implements-this" class="hash-link" aria-label="Direct link to How Choreo Agent ID implements this" title="Direct link to How Choreo Agent ID implements this" translate="no">​</a></h2>
<p>Choreo Agent ID treats agents as first-class identities in Choreo IAM. It maps onto the four functions described earlier:</p>
<table><thead><tr><th>Function</th><th>How Agent ID handles it</th></tr></thead><tbody><tr><td>Administer</td><td>Agents are provisioned through SCIM2 with agent extensions, with metadata such as owner, purpose, and risk level.</td></tr><tr><td>Authenticate</td><td>Agents authenticate without human intervention, with mTLS and Private Key JWT options.</td></tr><tr><td>Authorize</td><td>Context-aware policies evaluate agent metadata, user consent, and risk signals. Tokens state whether the agent acts for itself or for a user.</td></tr><tr><td>Audit</td><td>Agent actions are logged separately from human user trails, which supports anomaly detection and compliance reporting.</td></tr></tbody></table>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="grant-flows-per-agent-type">Grant flows per agent type<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#grant-flows-per-agent-type" class="hash-link" aria-label="Direct link to Grant flows per agent type" title="Direct link to Grant flows per agent type" translate="no">​</a></h3>
<p>The <a href="https://wso2.com/identity-platform/docs/guides/agentic-ai/ai-agents/agent-authentication/" target="_blank" rel="noopener noreferrer" class="">agent authentication documentation</a> covers each row of the earlier grant table. An agent acting on its own authenticates with its Agent ID and secret through a redirect-free flow. An interactive agent acting for a user follows an extended authorization code flow. The client sends a <code>requested_actor</code> parameter, and the user sees which agent is asking and for which scopes. The agent then proves itself with an <code>actor_token</code> at the token endpoint. That flow adopts an IETF individual draft about <a href="https://datatracker.ietf.org/doc/draft-oauth-ai-agents-on-behalf-of-user/" target="_blank" rel="noopener noreferrer" class="">on-behalf-of authorization for AI agents</a>, co-authored by Choreo Labs engineers. The draft has no formal IETF standing and has expired, which reflects how unsettled this area still is. The <code>requested_actor</code> parameter, and the use of an actor token in this flow, come from that draft. RFC 8693 defines token exchange and the <code>act</code> claim, but it does not standardize this extension. The flow works only with authorization servers that implement it.</p>
<p>For background agents, Agent ID combines CIBA with the agent's actor token. The agent starts the request:</p>
<div class="language-bash codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-bash codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">curl -X POST https://api.asgardeo.io/t/{organization_name}/oauth2/ciba \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  --header "Authorization: Basic &lt;Base64Encoded(CLIENT_ID:CLIENT_SECRET)&gt;" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  --header "Content-Type: application/x-www-form-urlencoded" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  --data-urlencode "scope=openid profile" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  --data-urlencode "login_hint=&lt;username&gt;" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  --data-urlencode "binding_message=Agent requesting access on your behalf" \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  --data-urlencode "actor_token=&lt;AGENT_ACTOR_TOKEN&gt;"</span><br></span></code></pre></div></div>
<p>After the user approves on their device, the agent polls the token endpoint with <code>grant_type=urn:openid:params:grant-type:ciba</code>. The issued token names the user as <code>sub</code> and the agent in the <code>act</code> claim:</p>
<div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"sub"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"&lt;user_identifier&gt;"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"act"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"sub"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"&lt;agent_identifier&gt;"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token punctuation" style="color:#8A93A6">}</span><br></span></code></pre></div></div>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="a-worked-example-claude-without-static-api-keys">A worked example: Claude without static API keys<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#a-worked-example-claude-without-static-api-keys" class="hash-link" aria-label="Direct link to A worked example: Claude without static API keys" title="Direct link to A worked example: Claude without static API keys" translate="no">​</a></h3>
<p>Choreo Labs has documented an integration between <a href="https://wso2.com/library/blogs/claude-wif-wso2-agent-id/" target="_blank" rel="noopener noreferrer" class="">Agent ID and Claude's Workload Identity Federation</a>. The setup has five steps:</p>
<ol>
<li class="">Register an application in Choreo IAM for agents to authenticate against.</li>
<li class="">Create an identity for each agent, with its own Agent ID, credentials, groups, and roles.</li>
<li class="">Register Choreo IAM as a trusted federation issuer in the Claude Console.</li>
<li class="">Create a Claude service account for each agent, or group of agents, in the right workspace.</li>
<li class="">Define federation rules that bind Choreo IAM agent identities to those service accounts.</li>
</ol>
<p>At runtime, the agent authenticates to Choreo IAM and receives a signed identity token. It exchanges that token with Claude for a short-lived access token, then calls the API. No static <code>sk-ant-</code> key lives in the agent's environment. The federation rule is where the policy sits. Matching on the issuer alone would let any Choreo IAM agent map to any service account. Matching on issuer and subject keeps one agent from impersonating another.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="runtime-enforcement-at-the-ai-gateway">Runtime enforcement at the AI gateway<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#runtime-enforcement-at-the-ai-gateway" class="hash-link" aria-label="Direct link to Runtime enforcement at the AI gateway" title="Direct link to Runtime enforcement at the AI gateway" translate="no">​</a></h3>
<p>Agent ID decides who an agent is and what its token carries. Choreo AI Gateway enforces those decisions on live traffic to LLMs, MCP servers, and A2A agents. For MCP traffic, its <a href="https://wso2.com/api-platform/docs/ai-gateway/next/mcp-governance/" target="_blank" rel="noopener noreferrer" class="">MCP governance policies</a> validate tokens and check per-tool scopes or JWT claims before a call reaches the server. For model traffic, a <a href="https://is.docs.wso2.com/en/next/tutorials/wso2-ai-gateway-with-agent-identity-aware-access-control/" target="_blank" rel="noopener noreferrer" class="">Choreo IAM tutorial</a> shows the gateway checking the role in an agent's token before routing the request to a specific model. One identity source and one enforcement layer keep the same agent identity visible across both kinds of traffic.</p>
<p>Teams that are not ready for the full stack can start smaller. The open source Open MCP Auth Proxy sits in front of an existing MCP server and validates bearer tokens. The server code stays unchanged.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>Agent permissions work best as their own governance layer. Human RBAC and shared service-account credentials both leave gaps. An agent needs its own identity and scopes that follow the task. Its tokens should expire quickly, and its grant type should match whether a user is present. Every action it takes should leave an audit record that names it. Most of the standards involved are mature. The delegation extensions for AI agents are still being worked out, so expect some of the details to change.</p>
<p>If you are ready to give agents first-class identities rather than adding permissions to existing IAM, start with <a href="https://wso2.com/identity-platform/agent-id/" target="_blank" rel="noopener noreferrer" class="">Choreo Agent ID</a>. You can try it free and test the grant flows above against your own APIs.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-are-agent-api-permissions">What are agent API permissions?<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#what-are-agent-api-permissions" class="hash-link" aria-label="Direct link to What are agent API permissions?" title="Direct link to What are agent API permissions?" translate="no">​</a></h3>
<p>Agent API permissions are the rules that decide which API operations an AI agent may call, on which resources, and on whose behalf. They are tied to the agent's own identity, not a shared key. They are issued as scoped OAuth tokens, and every use is logged.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-is-ai-agent-authorization-different-from-user-authorization">How is AI agent authorization different from user authorization?<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#how-is-ai-agent-authorization-different-from-user-authorization" class="hash-link" aria-label="Direct link to How is AI agent authorization different from user authorization?" title="Direct link to How is AI agent authorization different from user authorization?" translate="no">​</a></h3>
<p>User authorization usually grants access by job role, and a person decides each action. An agent decides its next action at runtime, often from untrusted input. Its access has to be scoped to the current task and bounded in time. Every call also needs to be attributable to that specific agent.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="should-an-ai-agent-use-the-users-oauth-token">Should an AI agent use the user's OAuth token?<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#should-an-ai-agent-use-the-users-oauth-token" class="hash-link" aria-label="Direct link to Should an AI agent use the user's OAuth token?" title="Direct link to Should an AI agent use the user's OAuth token?" translate="no">​</a></h3>
<p>Not directly. Reusing a broad user token usually gives the agent more access than its task needs. Issue a token for the intended resource and task with the minimum necessary scopes. Where the authorization server supports it, that token names the user as the subject and the agent as the actor.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="which-oauth-grant-should-an-ai-agent-use">Which OAuth grant should an AI agent use?<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#which-oauth-grant-should-an-ai-agent-use" class="hash-link" aria-label="Direct link to Which OAuth grant should an AI agent use?" title="Direct link to Which OAuth grant should an AI agent use?" translate="no">​</a></h3>
<p>It depends on who the agent acts for. An agent acting for itself uses its own credentials. An agent acting for a user in a live session uses the authorization code flow. A background agent acting for a user can use CIBA. An agent handing work to another agent can use token exchange.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-ciba-and-when-do-ai-agents-need-it">What is CIBA, and when do AI agents need it?<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#what-is-ciba-and-when-do-ai-agents-need-it" class="hash-link" aria-label="Direct link to What is CIBA, and when do AI agents need it?" title="Direct link to What is CIBA, and when do AI agents need it?" translate="no">​</a></h3>
<p>CIBA (Client-Initiated Backchannel Authentication) is an OpenID flow where the user approves a request on a separate device, without a browser redirect. Agents need it when they act for a user who is not in the agent's session. An overnight task that still needs the user's consent is a typical example.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-long-should-an-ai-agents-access-token-last">How long should an AI agent's access token last?<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#how-long-should-an-ai-agents-access-token-last" class="hash-link" aria-label="Direct link to How long should an AI agent's access token last?" title="Direct link to How long should an AI agent's access token last?" translate="no">​</a></h3>
<p>As short as the workload can tolerate, typically minutes rather than hours or days. Short lifetimes limit what a leaked token can do. The client library should refresh tokens before they expire, so the agent never handles the refresh logic itself.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-do-you-audit-what-an-ai-agent-did">How do you audit what an AI agent did?<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#how-do-you-audit-what-an-ai-agent-did" class="hash-link" aria-label="Direct link to How do you audit what an AI agent did?" title="Direct link to How do you audit what an AI agent did?" translate="no">​</a></h3>
<p>Give each agent its own identity, then log every call with the agent's identifier, any delegating user, and the resource or tool called. Record the policy decision and the outcome as well. Keep these records separate from human activity logs, so investigators can trace each action to one agent.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-do-you-revoke-an-ai-agents-access">How do you revoke an AI agent's access?<a href="https://choreolabs.com/api-management/learn/agent-api-permissions#how-do-you-revoke-an-ai-agents-access" class="hash-link" aria-label="Direct link to How do you revoke an AI agent's access?" title="Direct link to How do you revoke an AI agent's access?" translate="no">​</a></h3>
<p>Disable the agent's identity in the identity provider, or withdraw the user's delegation. The agent can then no longer get new tokens. Tokens it already holds stay valid until they expire, unless the gateway or API checks revocation status on each request.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="AI agents" term="AI agents"/>
        <category label="Agent-ready APIs" term="Agent-ready APIs"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[Agent observability: Tracing agents from LLM to tool calls]]></title>
        <id>https://choreolabs.com/api-management/learn/agent-observability</id>
        <link href="https://choreolabs.com/api-management/learn/agent-observability"/>
        <updated>2026-10-13T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[How to trace AI agents end to end: what to log, how to connect agent, LLM and tool-call spans, and which metrics predict agent reliability.]]></summary>
        <content type="html"><![CDATA[<p>A customer asks your support agent to refund one order, and the agent refunds a different one. Every HTTP call returned <code>200</code>, latency stayed normal, and token spend looked like any other day. Nothing in your dashboards says anything went wrong.</p>
<p>That is the gap agent observability fills. Uptime and latency tell you the system responded. They do not tell you whether the agent picked the right order, called the right tool, or carried the right context into the next step. To answer those questions, you need a record of what the agent observably did at each step. That means the model requests, tool calls, state changes and decision metadata your application can see.</p>
<aside class="cta_6F_m" aria-label="See exactly what your agents do" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">See exactly what your agents do</span><span class="desc_GzK8">Use Choreo to get observability across AI services and every agent deployment you run.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=agent-observability" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<p>This guide covers what to observe in an AI agent and how to connect agent, LLM and tool-call spans into one trace. It then looks at the metrics to track, how traces feed evaluation, and how to start without rebuilding your stack.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-agent-observability-means-and-why-traditional-apm-doesnt-cover-it">What agent observability means (and why traditional APM doesn't cover it)<a href="https://choreolabs.com/api-management/learn/agent-observability#what-agent-observability-means-and-why-traditional-apm-doesnt-cover-it" class="hash-link" aria-label="Direct link to What agent observability means (and why traditional APM doesn't cover it)" title="Direct link to What agent observability means (and why traditional APM doesn't cover it)" translate="no">​</a></h2>
<p>Agent observability is the practice of capturing structured traces of an AI agent's observable steps, such as planning, LLM calls, tool calls and state changes. The goal is to judge whether the agent did the right thing in the right way. Traditional monitoring asks whether a service is up. Agent observability asks whether the agent behaved correctly.</p>
<p>Traditional APM was built for code paths you wrote. A request hits a route, the route calls a database, and you measure each hop. With AI agents, the model chooses the path at runtime. It decides which tool to call, with which arguments, and whether to loop or stop. Two identical requests can take different execution paths. Both can succeed at the transport layer while only one gives the correct result.</p>
<p>That is why agent behavior needs its own signals. You still want latency and error rates. You also need the decision record behind them.</p>
<p>Agent observability is also distinct from <a class="" href="https://choreolabs.com/api-management/learn/ai-gateway-observability">observability at the traffic and cost layer</a>. Gateway telemetry tells you token usage, cost and latency per provider across all your model traffic. Agent telemetry tells you what one agent did inside one task. Production teams usually need both, and they connect through OpenTelemetry.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-to-actually-observe-in-an-agent">What to actually observe in an agent<a href="https://choreolabs.com/api-management/learn/agent-observability#what-to-actually-observe-in-an-agent" class="hash-link" aria-label="Direct link to What to actually observe in an agent" title="Direct link to What to actually observe in an agent" translate="no">​</a></h2>
<p>A useful way to scope agent observability is to ask four questions about every agent run. Each one maps to data you need to capture.</p>
<p><strong>Did the agent complete the task?</strong> This is task completion, judged against what the user asked for. In the refund example, "refund issued" is not enough. The question is whether the correct order was refunded. Capture the original request and the final outcome on the same trace so the two can be compared later.</p>
<p><strong>Did it follow an acceptable execution path?</strong> An agent can reach the right answer by an unacceptable route. It might call a write tool before confirming the order, or retry a search several times. Capture the order of steps, not just the result.</p>
<p><strong>Were its tool calls correct?</strong> Tool selection is where many production failures start. The agent may call the wrong tool, pass a malformed argument, or misread a tool's output. Record the tool name, a call ID, the outcome and (where policy allows) the arguments for every tool call.</p>
<p><strong>Did it use the right context and state?</strong> Agents depend on conversation history, retrieved documents and memory. If the agent loaded the wrong customer record into context, every later step can be internally consistent and still wrong. Capture what was retrieved and what state the agent carried between steps.</p>
<p>Each question points at a different part of the run. The next section shows how one trace holds all four.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="tracing-across-agent--llm--tool-calls">Tracing across agent → LLM → tool calls<a href="https://choreolabs.com/api-management/learn/agent-observability#tracing-across-agent--llm--tool-calls" class="hash-link" aria-label="Direct link to Tracing across agent → LLM → tool calls" title="Direct link to Tracing across agent → LLM → tool calls" translate="no">​</a></h2>
<p>A trace is the record of one agent run, made of spans that share a trace context. Each span is one timed operation, and every span except the root references a parent. A trace can still be incomplete if spans are sampled out, dropped during export or never instrumented. The parent-child structure is what turns a flat log into an execution path you can read.</p>
<p>The OpenTelemetry GenAI semantic conventions define span types for this. As of October 2026 they sit in a <a href="https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-agent-spans.md" target="_blank" rel="noopener noreferrer" class="">dedicated OpenTelemetry repository</a> and are still in Development status, so names can change. Pin your instrumentation library versions so attribute names do not shift between releases. They are still the closest thing to a shared vocabulary for agent tracing.</p>
<p>Here is how the refund request decomposes into a trace:</p>
<p><img decoding="async" loading="lazy" alt="One request, one trace. The user request &amp;quot;Please refund order #4821&amp;quot; produces a root invoke_agent support-agent span. Inside it, a plan support-agent span holds a chat LLM call, followed by an execute_tool lookup_order span, then a nested invoke_agent refunds-agent span containing its own chat call and an execute_tool issue_refund span, and finally a chat span for the final reply &amp;quot;Your refund for #4821 is on its way&amp;quot;" src="https://choreolabs.com/api-management/learn/assets/images/refund-request-trace-e76ff84748a02d8ddaaed06d7f0aeb25.webp" width="3200" height="1744" class="img_ev3q"></p>
<p><em>A single refund request decomposes into nested spans. Boxes inside boxes are child spans, and left to right shows the order in this example run.</em></p>
<ol>
<li class=""><code>invoke_agent support-agent</code> is the root span for the agent run.</li>
<li class=""><code>plan support-agent</code> records the planning step. The LLM call that produces the plan sits under it as a child span.</li>
<li class=""><code>execute_tool lookup_order</code> fetches the order. It is a sibling of the plan span under the root, not a child of it.</li>
<li class=""><code>invoke_agent refunds-agent</code> is a sub-agent handoff. It nests as a child of the root, with its own LLM call and an <code>execute_tool issue_refund</code> span beneath it.</li>
<li class=""><code>chat</code> (final reply) is the last LLM call, which writes the reply to the customer.</li>
</ol>
<p>That layout follows the conventions directly. The plan span holds the LLM call that produced the plan, and the tool spans that carry it out are siblings under the agent span. In a multi-agent system, the handoff simply adds depth to the tree. The conventions use an internal <code>invoke_agent</code> span when the sub-agent runs in the same process. They use a client span when the agent is a remote service.</p>
<p>Session-level observability sits one level above. A support conversation spans many runs, and each run is its own trace. The <code>gen_ai.conversation.id</code> attribute ties those traces together into one session, so you can see how context built up across turns.</p>
<p>Crossing process boundaries depends on trace context propagation. When the agent calls a remote sub-agent, an MCP server or a gateway, it passes trace context in headers. That lets downstream spans join the same trace. Without it, you get several disconnected traces and correlate them by ID afterwards. Choreo AI Gateway, for example, emits its own spans for policy execution and upstream calls, and its tracing docs describe header-based context propagation. The Choreo section below covers how that gateway layer relates to the agent layer.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-four-types-of-spans-worth-capturing">The four types of spans worth capturing<a href="https://choreolabs.com/api-management/learn/agent-observability#the-four-types-of-spans-worth-capturing" class="hash-link" aria-label="Direct link to The four types of spans worth capturing" title="Direct link to The four types of spans worth capturing" translate="no">​</a></h2>
<p>"Log everything" is not a plan. A more practical start is four span types, which together cover the four questions above.</p>
<p><img decoding="async" loading="lazy" alt="Four span types, four questions. Tool-call spans answer &amp;quot;Were the tool calls correct?&amp;quot; using execute_tool spans. Reasoning spans answer &amp;quot;Was the execution path acceptable?&amp;quot; using a plan span with the chat call as its child. State-transition spans answer &amp;quot;Where did the task end up?&amp;quot; using a custom span or span event. Memory-operation spans answer &amp;quot;Did the agent use the right context?&amp;quot; using search_memory and update_memory" src="https://choreolabs.com/api-management/learn/assets/images/four-span-types-f4c7002fa628ba554e803bc929206dbe.webp" width="3200" height="1988" class="img_ev3q"></p>
<p><em>Start here instead of logging everything. Each span type answers one question you will ask during an incident.</em></p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="tool-call-spans">Tool-call spans<a href="https://choreolabs.com/api-management/learn/agent-observability#tool-call-spans" class="hash-link" aria-label="Direct link to Tool-call spans" title="Direct link to Tool-call spans" translate="no">​</a></h3>
<p>Record each tool invocation. The OpenTelemetry convention is an <code>execute_tool {tool name}</code> span with the tool name required and a tool call ID recommended. Arguments and results are opt-in because they often contain customer data. To expose retries, give each logical operation a durable identifier and record an attempt count, with each attempt as its own span. A tool call ID alone is not enough, because a retried operation can get a new ID. Record a hash or safe reference to the idempotency key, not the key itself. Then correlate attempts with the outcome reported by the service that performs the side effect. That shows duplicate executions the final answer hides. That is one reason <a class="" href="https://choreolabs.com/api-management/learn/designing-idempotent-tools">idempotency keys</a> matter for agent tools.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="reasoning-spans">Reasoning spans<a href="https://choreolabs.com/api-management/learn/agent-observability#reasoning-spans" class="hash-link" aria-label="Direct link to Reasoning spans" title="Direct link to Reasoning spans" translate="no">​</a></h3>
<p>Record the planning operations your application can observe. When a framework exposes an explicit planning step, it maps to a <code>plan</code> span with the LLM call nested inside. The conventions say to skip the plan span when instrumentation cannot tell planning apart from ordinary inference. In that case, the LLM span's input and output messages are the closest record. They show what the model received and returned, not its full internal reasoning. Some providers expose a reasoning summary, but it is a summary, not the complete internal process. Treat explanations drawn from these spans as inferences, and keep plans and summaries apart from confidential prompt or memory content.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="state-transition-spans">State-transition spans<a href="https://choreolabs.com/api-management/learn/agent-observability#state-transition-spans" class="hash-link" aria-label="Direct link to State-transition spans" title="Direct link to State-transition spans" translate="no">​</a></h3>
<p>Record changes in the agent's own state, such as moving from "gathering details" to "awaiting approval". The GenAI conventions do not define an operation for this today. Teams usually emit a custom span or a span event with the old and new state as attributes. It is worth the effort for agents with approval gates or long-running workflows.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="memory-operation-spans">Memory-operation spans<a href="https://choreolabs.com/api-management/learn/agent-observability#memory-operation-spans" class="hash-link" aria-label="Direct link to Memory-operation spans" title="Direct link to Memory-operation spans" translate="no">​</a></h3>
<p>Record reads and writes to agent memory. The conventions define operations such as <code>search_memory</code> and <code>update_memory</code>. These spans answer the context question directly. When an agent acts on stale information, the memory span can show what it retrieved, but only if that content was captured. Some tracing SDKs capture sensitive content by default, so set a capture policy before relying on it.</p>
<p>A tool-call span with manual instrumentation in Python looks like this:</p>
<div class="language-python codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-python codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token keyword" style="color:#3D8BFF">from</span><span class="token plain"> opentelemetry </span><span class="token keyword" style="color:#3D8BFF">import</span><span class="token plain"> trace</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">tracer </span><span class="token operator">=</span><span class="token plain"> trace</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">get_tracer</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token string" style="color:#4ADE80">"support-agent"</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token keyword" style="color:#3D8BFF">def</span><span class="token plain"> </span><span class="token function" style="color:#FFFFFF;font-weight:600">traced_lookup_order</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">order_id</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token builtin" style="color:#3D8BFF">str</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> call_id</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token builtin" style="color:#3D8BFF">str</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> conversation_id</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token builtin" style="color:#3D8BFF">str</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token keyword" style="color:#3D8BFF">with</span><span class="token plain"> tracer</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">start_as_current_span</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token string" style="color:#4ADE80">"execute_tool lookup_order"</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"> </span><span class="token keyword" style="color:#3D8BFF">as</span><span class="token plain"> span</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        span</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">set_attribute</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token string" style="color:#4ADE80">"gen_ai.operation.name"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"execute_tool"</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        span</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">set_attribute</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token string" style="color:#4ADE80">"gen_ai.tool.name"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"lookup_order"</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        span</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">set_attribute</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token string" style="color:#4ADE80">"gen_ai.tool.call.id"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> call_id</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        span</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">set_attribute</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token string" style="color:#4ADE80">"gen_ai.conversation.id"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> conversation_id</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token keyword" style="color:#3D8BFF">return</span><span class="token plain"> lookup_order</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">order_id</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain">  </span><span class="token comment" style="color:#8A93A6;font-style:italic"># your existing tool function</span><br></span></code></pre></div></div>
<p>This example assumes the OpenTelemetry SDK is already configured with a tracer provider and an exporter. Without that setup, the API returns a no-op tracer and no spans reach a backend. When this runs inside the active agent span, the tool span becomes its child automatically. Across process boundaries, that parent link holds only if trace context is propagated. For MCP tools, the per-call fields that matter overlap heavily with <a class="" href="https://choreolabs.com/api-management/learn/mcp-monitoring">MCP server monitoring</a>, including what you should never write to a span.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="key-metrics-to-track">Key metrics to track<a href="https://choreolabs.com/api-management/learn/agent-observability#key-metrics-to-track" class="hash-link" aria-label="Direct link to Key metrics to track" title="Direct link to Key metrics to track" translate="no">​</a></h2>
<p>Traces explain individual runs. Metrics tell you whether agent quality is moving across many runs. Most of these are computed from production traces rather than emitted directly.</p>
<table><thead><tr><th>Metric</th><th>What It Tells You</th><th>How to Measure It</th></tr></thead><tbody><tr><td>Task completion rate</td><td>Whether the agent achieves the user's goal</td><td>Share of traces whose outcome passes a task-level check or evaluator</td></tr><tr><td>Tool-call accuracy</td><td>Whether the agent picks the right tool with valid arguments</td><td>Tool-call spans scored against expected tools, or flagged by an evaluator</td></tr><tr><td>Latency per step</td><td>Which step is slow, not just the total</td><td>Duration of each LLM, tool and memory span, split by span type</td></tr><tr><td>Token usage per task</td><td>What a completed task costs</td><td>Sum of input and output tokens across all LLM spans in a trace</td></tr><tr><td>Production failure rate</td><td>How often runs end in an error or an unusable result</td><td>Traces with an error status, abandoned runs, or failed evaluations</td></tr><tr><td>User feedback rate</td><td>How real users rate the outcome</td><td>Thumbs up or down and escalations, attached to the trace ID</td></tr></tbody></table>
<p>Per-step latency matters because an agent's end-to-end time is built from many steps. When steps run in parallel, their durations overlap, so explain wall-clock latency through the critical path rather than by summing spans. One slow tool can hide inside a normal-looking total. Token usage per task is different from token usage per request. A single task can make several model calls, and that per-task total is what cost reviews need. Tokens are not a uniform cost unit, though. Prices differ by model, between input and output tokens, and for cached tokens.</p>
<p>Cost per provider, time to first token and per-provider error rates belong at the gateway layer. They are covered in Choreo Labs' AI gateway observability guide linked earlier.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="connecting-observability-to-evaluation">Connecting observability to evaluation<a href="https://choreolabs.com/api-management/learn/agent-observability#connecting-observability-to-evaluation" class="hash-link" aria-label="Direct link to Connecting observability to evaluation" title="Direct link to Connecting observability to evaluation" translate="no">​</a></h2>
<p>Traces show what happened. Evaluation decides whether it was good. The two use the same data, which is why most teams connect them early.</p>
<p><strong>Offline evaluation:</strong> Runs an agent against a fixed set of tasks before release. You compare outputs and execution paths against expected results. Offline evaluation catches regressions when you change a prompt, a model or a tool. Its limit is coverage. A test set only contains the cases someone thought to write down.</p>
<p><strong>Online evaluation:</strong> Scores production traces after they happen. Evaluators read the trace and attach a score. Some are rule-based checks, such as "the reply is valid JSON" or "no write tool ran without a lookup first". Others use LLM-as-a-judge scoring for qualities like relevance or tone. Online evaluation can surface cases nobody predicted, because it runs on real user traffic. It only sees traces that were instrumented and retained, so report sampling policy and coverage alongside the scores.</p>
<p>The loop between the two is where agent quality improves. A production trace that fails an online evaluator becomes a new case in the offline set. The next release is tested against it before it ships.</p>
<p>The OpenTelemetry GenAI conventions include a <code>gen_ai.evaluation.result</code> event with a metric name, a score and an explanation. That gives evaluation scores a standard place on the trace, next to the spans they judge. User feedback fits the same pattern. Attach it to the trace ID so that low ratings point to a specific run.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="implementing-agent-observability-without-a-six-month-project">Implementing agent observability without a six-month project<a href="https://choreolabs.com/api-management/learn/agent-observability#implementing-agent-observability-without-a-six-month-project" class="hash-link" aria-label="Direct link to Implementing agent observability without a six-month project" title="Direct link to Implementing agent observability without a six-month project" translate="no">​</a></h2>
<p>You do not need a custom tracing platform to start. Most teams combine three approaches.</p>
<p>Automatic instrumentation comes first. Many agent frameworks already emit traces. The OpenAI Agents SDK, for example, <a href="https://openai.github.io/openai-agents-python/tracing/" target="_blank" rel="noopener noreferrer" class="">traces runs by default</a>, including LLM generations, tool calls, handoffs and guardrails. That tracing is its own integration and exports to OpenAI's backend by default. Sending it to another backend needs an added trace processor or a supported integration. Instrumentation libraries cover other open source frameworks. Check what your framework produces before writing any code.</p>
<p>Manual instrumentation fills the gaps. Custom tools, internal workflows and state transitions rarely appear in auto-instrumented traces. Add spans by hand for the steps you need to debug, following the GenAI attribute names where they exist.</p>
<p>OpenTelemetry as the fallback keeps your options open. When a framework has no native tracing, or uses its own format, emit OTLP directly. Any OTLP-compatible backend can receive it, so the choice of tracing tool stays separate from the choice of agent framework.</p>
<p>One default deserves attention on day one. Content capture is often on by default, and the OpenAI Agents SDK includes sensitive data in traces unless you turn it off. Decide what prompts, arguments and outputs may leave the process before you export anything.</p>
<p>The decision to invest beyond the basics usually follows an operational trigger, not a traffic number. Common triggers include:</p>
<ul>
<li class="">Agents that route across several models or providers</li>
<li class="">Finance asking for spend attribution by team or agent</li>
<li class="">Regulated data appearing in prompts or tool arguments</li>
<li class="">Multiple teams shipping agents that share tools or memory</li>
<li class="">Production incidents that took too long to explain</li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-choreo-approaches-agent-observability">How Choreo approaches agent observability<a href="https://choreolabs.com/api-management/learn/agent-observability#how-choreo-approaches-agent-observability" class="hash-link" aria-label="Direct link to How Choreo approaches agent observability" title="Direct link to How Choreo approaches agent observability" translate="no">​</a></h2>
<p>Choreo covers both observability layers. Each layer has its own traffic path.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-agent-layer-choreo-agent-manager">The agent layer: Choreo Agent Manager<a href="https://choreolabs.com/api-management/learn/agent-observability#the-agent-layer-choreo-agent-manager" class="hash-link" aria-label="Direct link to The agent layer: Choreo Agent Manager" title="Direct link to The agent layer: Choreo Agent Manager" translate="no">​</a></h3>
<p>Choreo Agent Manager is positioned as an open control plane for AI agents. It brings identity, guardrails, observability and evaluation together for agents on any framework. Agent tracing is built on OpenTelemetry. Python agents deployed through the platform can be <a href="https://wso2.com/agent-platform/docs/cloud/concepts/observability/" target="_blank" rel="noopener noreferrer" class="">auto-instrumented</a> for frameworks such as LangChain, LlamaIndex, CrewAI and Haystack, along with MCP client calls. Externally hosted Python agents can send traces by wrapping their start command with the <code>amp-instrument</code> CLI. Teams on other frameworks can emit spans manually against a published contract based on the OpenTelemetry GenAI conventions.</p>
<p>Evaluation runs on those traces after the agent finishes, so it stays off the agent's response path. It still consumes compute, and sampled-out traces are not available to evaluation monitors. Built-in rule-based and LLM-as-judge evaluators, plus custom ones, can score a whole trace or each agent within a multi-agent trace. Continuous monitors score new production traces on a schedule. Historical monitors score a fixed time window once. Scores appear in the trace view, next to the spans they judge.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-traffic-layer-choreo-ai-gateway">The traffic layer: Choreo AI Gateway<a href="https://choreolabs.com/api-management/learn/agent-observability#the-traffic-layer-choreo-ai-gateway" class="hash-link" aria-label="Direct link to The traffic layer: Choreo AI Gateway" title="Direct link to The traffic layer: Choreo AI Gateway" translate="no">​</a></h3>
<p>Choreo AI Gateway governs outbound LLM traffic to model providers and inbound agent access to MCP tools. Its components <a href="https://wso2.com/api-platform/docs/ai-gateway/next/logging-and-tracing/tracing/" target="_blank" rel="noopener noreferrer" class="">export traces over OTLP</a> to any compatible backend. It can also export analytics events as OTLP log records. For that analytics export, the <a href="https://wso2.com/api-platform/docs/ai-gateway/next/analytics/opentelemetry-analytics/" target="_blank" rel="noopener noreferrer" class="">docs state</a> that a slow or unreachable OTLP endpoint does not add latency to API calls.</p>
<p>The two layers see different paths, and none of them is a mandatory chain:</p>
<ul>
<li class=""><strong>Inbound requests</strong> go from a client or application to the agent.</li>
<li class=""><strong>Outbound model calls</strong> go from the agent to an LLM provider, optionally through the AI Gateway.</li>
<li class=""><strong>Tool calls</strong> go from the agent to MCP servers or APIs, optionally through the AI Gateway.</li>
<li class=""><strong>Telemetry export</strong> runs separately from all three. The agent sends spans to Agent Manager, and the gateway exports its spans and analytics to an OTLP collector.</li>
</ul>
<p>Because both layers use OpenTelemetry, teams that already run Datadog, Langfuse or another platform can route the data there. OTLP support does not guarantee the same dashboards, though, and backends may interpret the developmental GenAI attributes differently. The aim is to cover both layers without forcing teams off the evaluation or experimentation tools they already use.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/agent-observability#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>Agent observability is about the decision record, not uptime. A trace that links reasoning, LLM calls, tool calls and state across a full task lets you explain a bad outcome. The four span types give you a practical starting set. Tool-call, reasoning, state-transition and memory-operation spans map closely to the questions you will ask during an incident.</p>
<p>Start with your framework's built-in tracing, add manual spans where it falls short, and keep OpenTelemetry as the format between layers. Then connect production traces to evaluation so failures become test cases. If you want agent tracing and evaluation in the same control plane as identity and guardrails, explore <a href="https://wso2.com/agent-platform/agent-manager/" target="_blank" rel="noopener noreferrer" class="">Choreo Agent Manager</a>.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/agent-observability#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-agent-observability">What is agent observability?<a href="https://choreolabs.com/api-management/learn/agent-observability#what-is-agent-observability" class="hash-link" aria-label="Direct link to What is agent observability?" title="Direct link to What is agent observability?" translate="no">​</a></h3>
<p>Agent observability is the capture and analysis of traces from AI agent runs. It shows the reasoning, model calls, tool calls and state behind each result.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-ai-agent-observability">What is AI agent observability?<a href="https://choreolabs.com/api-management/learn/agent-observability#what-is-ai-agent-observability" class="hash-link" aria-label="Direct link to What is AI agent observability?" title="Direct link to What is AI agent observability?" translate="no">​</a></h3>
<p>It is the same discipline applied to AI agents in production. The focus is behavior quality, such as task success and tool choice, rather than uptime alone.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-should-i-track-for-agent-observability">What should I track for agent observability?<a href="https://choreolabs.com/api-management/learn/agent-observability#what-should-i-track-for-agent-observability" class="hash-link" aria-label="Direct link to What should I track for agent observability?" title="Direct link to What should I track for agent observability?" translate="no">​</a></h3>
<p>Start with tool-call, reasoning, state and memory spans on every run. Then track task completion, tool-call accuracy, per-step latency, tokens per task and failure rate.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="is-opentelemetry-ready-for-agent-tracing">Is OpenTelemetry ready for agent tracing?<a href="https://choreolabs.com/api-management/learn/agent-observability#is-opentelemetry-ready-for-agent-tracing" class="hash-link" aria-label="Direct link to Is OpenTelemetry ready for agent tracing?" title="Direct link to Is OpenTelemetry ready for agent tracing?" translate="no">​</a></h3>
<p>Its GenAI conventions are usable, but as of October 2026 they are still in Development status and names may change.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="AI agents" term="AI agents"/>
        <category label="Observability" term="Observability"/>
        <category label="Agent-ready APIs" term="Agent-ready APIs"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[Agentic enterprise: A practical roadmap to get there (2026)]]></title>
        <id>https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap</id>
        <link href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap"/>
        <updated>2026-10-13T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[What is an agentic enterprise, and how do you actually become one? A practical roadmap covering the architecture, the stages, and what to get right first.]]></summary>
        <content type="html"><![CDATA[<p>Picture a common starting point. A support assistant answers policy questions, a coding agent opens pull requests, and a finance team is piloting an agent that reconciles invoices. Each one works well enough on its own. The harder question arrives later. Dozens of agents start calling the same systems of record, acting for different users, and drawing on the same model budgets.</p>
<p>That question is what the term agentic enterprise is really about. It has less to do with how many agents you build. It has more to do with whether your architecture lets them act safely across the business. This guide covers what an agentic enterprise is and the minimum architecture it needs. It then lays out a three-stage roadmap and the failure patterns that keep programs stuck in pilot. The framework comes from Choreo Labs' architecture work, but it is written so you can apply it to any stack.</p>
<aside class="cta_6F_m" aria-label="Move your agents from pilot to production" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Move your agents from pilot to production</span><span class="desc_GzK8">Use Choreo to govern agents safely across any gateway, cloud, or deployment model as you scale.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=agentic-enterprise-roadmap" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-an-agentic-enterprise">What is an agentic enterprise?<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#what-is-an-agentic-enterprise" class="hash-link" aria-label="Direct link to What is an agentic enterprise?" title="Direct link to What is an agentic enterprise?" translate="no">​</a></h2>
<p>An agentic enterprise is an organization where AI agents operate as first-class actors alongside people, applications, APIs, and services. Agents take real actions across enterprise systems. They do so under policy and with an identity of their own. Every action is traceable and subject to the same operational discipline as any production workload.</p>
<p>The distinction from plain AI adoption is the ability to act. A chatbot that summarizes the HR leave policy is AI adoption. An agent that files a leave request on an employee's behalf, checks the manager's approval rules, and records who authorized what is an agentic capability. The goal is not more AI. The goal is an architecture where agents can work across enterprise capabilities while you keep auditability and operational control.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="why-the-shift-from-app-centric-to-agentic-architecture-matters-now">Why the shift from app-centric to agentic architecture matters now<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#why-the-shift-from-app-centric-to-agentic-architecture-matters-now" class="hash-link" aria-label="Direct link to Why the shift from app-centric to agentic architecture matters now" title="Direct link to Why the shift from app-centric to agentic architecture matters now" translate="no">​</a></h2>
<p>Systems of record remain the source of truth for transactions, customers, and inventory. Around them, enterprises are adding what Choreo Labs' leadership calls <a href="https://wso2.com/library/blogs/agentic-enterprise-with-wso2/" target="_blank" rel="noopener noreferrer" class="">"systems of context"</a>, such as context graphs and knowledge bases. Agents act across both. Tools and policies become runtime dependencies, and generative AI models do the reasoning and planning.</p>
<p>In an app-centric world, you could standardize on interfaces (APIs), connectivity (integration), and guardrails (identity and observability). Those still matter, but they are no longer sufficient. Agents introduce new actors and new pathways to action. Their underlying models are probabilistic and usually work on unstructured data, so behavior can differ between two runs with the same input.</p>
<p>Timing matters too. Agents are no longer confined to engineering teams. Teams in operations, finance, and support are starting to use them to cut manual handoffs. When each team adopts agents on its own terms, the gains stay individual and the risks multiply. When agent tooling is built into the core stack as standard workflows and integrations, the gains become consistent and the controls become shared.</p>
<p>The blast radius depends on what the agent can do. An agent that only answers questions or drafts content has limited reach. An agent that calls tools, chains actions, and holds its own permissions is different, and that is where pre-agentic controls start to fall short.</p>
<p>Quality is the clearest example of the change:</p>
<ul>
<li class="">Traditional software is validated mainly through testing.</li>
<li class="">Generative AI models also need evaluations that assess behavior where no single correct answer exists, often using another LLM as a judge.</li>
<li class="">AI agents need both, plus attention to dynamic execution logic and its side effects.</li>
</ul>
<p>Consider a refund agent. It can pass every unit, system, and regression test and still fail in production. The cause might be a lookup and a refund call chained in an order no test anticipated. Testing tells you the tools work. Evaluation and runtime observation tell you whether the agent uses them sensibly.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-minimum-viable-architecture-for-an-agentic-enterprise">The minimum viable architecture for an agentic enterprise<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#the-minimum-viable-architecture-for-an-agentic-enterprise" class="hash-link" aria-label="Direct link to The minimum viable architecture for an agentic enterprise" title="Direct link to The minimum viable architecture for an agentic enterprise" translate="no">​</a></h2>
<p>The seven capabilities below fall into three groups: access, trust, and operations. They are not a feature list. Treat them as a reference architecture and scope each one to the workloads you run. An agent limited to reading data may need little beyond identity, guardrails, and observability. An agent that changes records or moves money needs every capability in place before it reaches production. None of them depends on a particular vendor.</p>
<table><thead><tr><th>Capability</th><th>What it controls</th><th>Question to ask of your current stack</th></tr></thead><tbody><tr><td>Controlled ingress and egress</td><td>Inbound calls to your APIs and agents, and outbound calls from agents to models and external tools</td><td>Can you enforce and audit policy in both directions?</td></tr><tr><td>Tooling layer</td><td>How agents discover and reach internal and third-party tools, including MCP servers</td><td>Is there one governed catalog, or does each team wire its own tools?</td></tr><tr><td>Identity and policy</td><td>Which agent acted, and on whose behalf</td><td>Can you tell an agent's action apart from the human it represents?</td></tr><tr><td>Guardrails</td><td>Content and action limits for apps, models, MCP servers, and agents</td><td>Are guardrails enforced outside the prompt?</td></tr><tr><td>Observability</td><td>Traces across APIs, integrations, workflows, services, agents, and models</td><td>Can you reconstruct why an agent took a specific action?</td></tr><tr><td>Testing and evaluation</td><td>Functional correctness plus reasoning quality</td><td>Do agents face evaluations before release, not only tests?</td></tr><tr><td>Deployability</td><td>Where the stack runs</td><td>Can it run where your workloads and data residency rules require, without a rewrite?</td></tr></tbody></table>
<p><img decoding="async" loading="lazy" alt="Minimum viable architecture for an agentic enterprise: seven vendor-neutral capabilities in three groups. Access covers how agents reach systems and tools, with controlled ingress and egress and a tooling layer. Trust covers who acted and within what limits, with identity and policy and guardrails. Operations covers how agents are run, checked, and shipped, with observability, testing and evaluation, and deployability" src="https://choreolabs.com/api-management/learn/assets/images/minimum-viable-architecture-1e2e9606b59732cb95a2ffec888983c9.webp" width="3200" height="1806" class="img_ev3q"></p>
<p><em>Ingress and egress are separate enforcement points, not two hops on one request. Scope each capability to the workloads you run.</em></p>
<p>Ingress and egress deserve a note because they are easy to draw wrong. Define both relative to your deployment boundary. Inbound traffic comes from users, applications, and external agents calling the APIs, agents, and MCP servers you expose. An API gateway governs inbound API calls, and an AI gateway can govern inbound agent and MCP tool requests. Outbound traffic goes from your agents to LLM providers and external MCP servers, and an AI gateway governs those calls. These are independent enforcement points, not two hops on one request. A single inbound request can trigger several outbound model calls, and each of those is governed on the egress side.</p>
<p>Identity needs the same precision. An agent acting for a customer should carry its own identity and a record of the delegation. Products like <a href="https://wso2.com/identity-platform/agent-id/" target="_blank" rel="noopener noreferrer" class="">Choreo Agent ID</a> treat agents as first-class identities. Tokens then show whether an agent acted on its own or on a user's behalf.</p>
<p>For an example of a fabric that covers this list, look at the <a href="https://wso2.com/library/blogs/wso2-agentic-enterprise-fabric/" target="_blank" rel="noopener noreferrer" class="">Choreo Agentic Enterprise Fabric</a>. It groups these capabilities into five platforms: Agent, API, Integration, Identity, and Engineering. Within it, Choreo API Management manages API traffic and AI traffic from a single control plane. That includes LLM calls and MCP calls.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="a-3-stage-roadmap-to-becoming-an-agentic-enterprise">A 3-stage roadmap to becoming an agentic enterprise<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#a-3-stage-roadmap-to-becoming-an-agentic-enterprise" class="hash-link" aria-label="Direct link to A 3-stage roadmap to becoming an agentic enterprise" title="Direct link to A 3-stage roadmap to becoming an agentic enterprise" translate="no">​</a></h2>
<p>The architecture above says what you need. The roadmap below says how teams usually get there. Forrester's David Mooter puts the premise in the title of his analysis: <a href="https://www.forrester.com/blogs/agentic-ai-runs-on-integration-not-data-lakes/" target="_blank" rel="noopener noreferrer" class="">"Agentic AI Runs On Integration, Not Data Lakes."</a> He argues that an agent without integration can answer but cannot act. He also reports a split in priorities. Organizations with high readiness for agentic AI are prioritizing integration and change management. Those with low readiness are prioritizing model capabilities.</p>
<p><img decoding="async" loading="lazy" alt="Three stages to an agentic enterprise. Stage 1: make enterprise data accessible to agents, by exposing integrations and a context layer to agents, often through MCP. Stage 2: build the AI agents, from conversational agents to agents inside integration flows. Stage 3: build intelligent integrations, where agents reason and then choose among deterministic paths. Governance, cost controls, and flexibility apply across every stage" src="https://choreolabs.com/api-management/learn/assets/images/three-stage-roadmap-9c783882b8dd2c0182cc37e959520542.webp" width="3200" height="1744" class="img_ev3q"></p>
<p><em>Each stage builds on the last, but they overlap in practice.</em></p>
<p>The stages build on each other, but they overlap in practice. Your integration and context layers will keep changing while you build agents on top of them.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="stage-1-make-enterprise-data-accessible-to-agents">Stage 1: Make enterprise data accessible to agents<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#stage-1-make-enterprise-data-accessible-to-agents" class="hash-link" aria-label="Direct link to Stage 1: Make enterprise data accessible to agents" title="Direct link to Stage 1: Make enterprise data accessible to agents" translate="no">​</a></h3>
<p>Agents are only as useful as the context, data, and systems they can reach. Most enterprises already have an integration layer, but it was designed for system-to-system communication, not for agents. The next step is to expose those capabilities in a form agents consume well. REST, GraphQL, and events all work. An agent-oriented protocol such as the <a href="https://modelcontextprotocol.io/docs/2026-07-28/getting-started/intro" target="_blank" rel="noopener noreferrer" class="">Model Context Protocol (MCP)</a> is often a good fit. It gives many agent clients one standard way to discover and call tools. A native API integration can still be the better choice. That is often the case when your agent clients already support it well, or when it already meets your security and operational requirements. Let client compatibility and those requirements drive the choice.</p>
<p>Design those MCP tools around the tasks an agent performs rather than mirroring every existing endpoint one to one. A "check claim status" tool may need three backend calls. It is still one tool from the agent's point of view, and that keeps tool lists short and intent clear.</p>
<p>On top of the integration layer, build a context layer. It uses context graphs, data catalogs, and data pipelines. Together they combine operational data, enterprise metadata, past interactions, and real-time events into a form agents can use. A good context layer also records what changed, when it changed, and why. Agent decisions feed back into it and can inform later decisions.</p>
<p>A claims agent shows why history matters. Knowing a claim's current status is useful. Knowing that a similar claim from the same policyholder was escalated last quarter, and why, is what lets the agent make a defensible choice. <a href="https://wso2.com/integration-platform/" target="_blank" rel="noopener noreferrer" class="">Choreo Integrator</a> is built around this stage, exposing business capabilities through open standards including MCP.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="stage-2-build-the-ai-agents">Stage 2: Build the AI agents<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#stage-2-build-the-ai-agents" class="hash-link" aria-label="Direct link to Stage 2: Build the AI agents" title="Direct link to Stage 2: Build the AI agents" translate="no">​</a></h3>
<p>Once the base is in place, build agents and connect them to that exposed context through MCP. The range is wide. At one end are conversational agents for customer support, travel booking, or financial planning, often grounded in internal knowledge through retrieval-augmented generation. At the other end are agents that participate in broader integration flows and collaborate with other specialized agents, each owning one business function.</p>
<p>The interface does not set the risk. The actions an agent is permitted to take do. A conversational agent can book appointments, issue refunds, or change records, while an agent inside an integration flow may only read. Once an agent can write to systems of record, it needs its own identity, <a class="" href="https://choreolabs.com/api-management/learn/agent-api-permissions">scoped permissions</a>, and <a class="" href="https://choreolabs.com/api-management/learn/agent-observability">traces</a> that connect its reasoning to the downstream calls it triggers.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="stage-3-build-intelligent-integrations">Stage 3: Build intelligent integrations<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#stage-3-build-intelligent-integrations" class="hash-link" aria-label="Direct link to Stage 3: Build intelligent integrations" title="Direct link to Stage 3: Build intelligent integrations" translate="no">​</a></h3>
<p>Traditional integrations are deterministic in their control flow. Their steps and branching rules are fixed in advance, although results can still vary with system state, dependencies, and timing. Agents choose their next action at runtime. They reason over the context and options available to them, and they may not make the same choice twice.</p>
<p>An intelligent integration combines both in one flow. Insurance claims processing is a useful example:</p>
<ol>
<li class="">A customer starts a claim, which triggers the agent.</li>
<li class="">The agent gathers details conversationally and reasons against historical and real-time context, including fraud checks and policy guidelines.</li>
<li class="">The agent picks one of several deterministic paths: reject with comments, approve, or route to a human for final approval.</li>
</ol>
<p>Fixed code paths alone do not guarantee safe effects. Deterministic code can execute an unauthorized or duplicate action just as consistently as a correct one. The agent's job is to choose among the paths. The agent may also reach a different decision on a similar claim later, because it has more context by then. That is expected behavior. The hard rules belong in the systems that carry out the action. They include authorization checks, transaction limits, required approvals, rate limits, and duplicate protection such as idempotency keys. Enforce them the same way whether a model or fixed code chose the path.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="three-things-that-make-or-break-the-journey-governance-cost-and-flexibility">Three things that make or break the journey: Governance, cost, and flexibility<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#three-things-that-make-or-break-the-journey-governance-cost-and-flexibility" class="hash-link" aria-label="Direct link to Three things that make or break the journey: Governance, cost, and flexibility" title="Direct link to Three things that make or break the journey: Governance, cost, and flexibility" translate="no">​</a></h2>
<p>The three stages give you a direction. Governance, cost controls, and flexibility decide whether the program survives contact with production.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="governance">Governance<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#governance" class="hash-link" aria-label="Direct link to Governance" title="Direct link to Governance" translate="no">​</a></h3>
<p>As agents become more autonomous, a misbehaving agent can do more damage. Governance sits in three places. It sits in the agent and integration logic itself. It sits in evaluation frameworks that check an agent's reasoning and decision steps before deployment. It also sits in human-in-the-loop approval for sensitive or high-risk actions at runtime. The <a href="https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/" target="_blank" rel="noopener noreferrer" class="">OWASP Top 10 for Agentic Applications</a> is a practical reference for the risks these controls should address.</p>
<p>Deciding where human approval applies is partly a design choice. Regulation, contracts, and separation-of-duties policies can require approval regardless of how well an agent performs. Within those limits, base it on the consequence of the action. Reversible, low-impact actions can usually run unattended. Actions that move money, change entitlements, or touch regulated data deserve a checkpoint. Evaluation results and production traces showing an agent handles an action reliably at scale are one input for reducing friction where approved policy permits it. They are not, on their own, the authority to remove a required control.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="cost-controls">Cost controls<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#cost-controls" class="hash-link" aria-label="Direct link to Cost controls" title="Direct link to Cost controls" translate="no">​</a></h3>
<p>Unrestricted token consumption can drain an AI budget quickly and stall an initiative before it shows value. Cost control belongs on the outbound model path, because that is where every model call passes. <a href="https://wso2.com/api-platform/ai-gateway/" target="_blank" rel="noopener noreferrer" class="">Choreo AI Gateway</a> is one example of this pattern already productized. According to its <a href="https://wso2.com/api-platform/docs/cloud/ai-gateway/overview/" target="_blank" rel="noopener noreferrer" class="">documentation</a>, administrators set budget controls such as token-based rate limiting at the provider level. Developers then build per-application proxies that inherit those limits. The same gateway handles guardrails, semantic caching, and routing across multiple models and providers.</p>
<p>The second lever is how agents and integrations get built. Development is shifting toward AI-assisted, natural-language approaches where developers describe a requirement and AI generates the code. Choreo Labs' integration tooling builds this on <a href="https://ballerina.io/" target="_blank" rel="noopener noreferrer" class="">Ballerina</a>, a language designed for integrations and agents. Choreo Labs' position is that integration-specific abstractions lead to cleaner prompts, better generated code, and fewer regeneration cycles, which reduces token use during development.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="flexibility">Flexibility<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#flexibility" class="hash-link" aria-label="Direct link to Flexibility" title="Direct link to Flexibility" translate="no">​</a></h3>
<p>The AI landscape is changing fast, and today's default stack will likely look different within a year. Two habits protect you. First, choose software that supports open standards such as MCP, REST, GraphQL, and gRPC. That keeps the option to switch vendors if costs, compliance needs, or features change. Second, prefer open source components where their licenses suit you. Choreo Integrator, for example, is released under the <a href="https://github.com/wso2/product-integrator" target="_blank" rel="noopener noreferrer" class="">Apache 2.0 license</a>. Source availability lets you self-host the software and inspect how your data is processed. Managed services, commercial support, and operating costs are separate decisions that source availability does not settle.</p>
<p>Agent frameworks are where lock-in tends to hide. Suppose tools are exposed through MCP and agent identities live in a standards-based identity provider. Moving an agent to a different framework then means rewriting the agent. It does not mean rewiring every tool connection and permission.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-getting-this-wrong-looks-like">What getting this wrong looks like<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#what-getting-this-wrong-looks-like" class="hash-link" aria-label="Direct link to What getting this wrong looks like" title="Direct link to What getting this wrong looks like" translate="no">​</a></h2>
<p><a href="https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027" target="_blank" rel="noopener noreferrer" class="">Gartner</a> predicts that over 40% of agentic AI projects will be canceled by the end of 2027. The reasons it cites are escalating costs, unclear business value, and inadequate risk controls. The failure patterns behind numbers like that tend to look familiar:</p>
<ul>
<li class="">Pilots that stall before production. The demo works, but nobody can answer the security review's questions about identity, audit, or data handling.</li>
<li class="">Inconsistent controls across teams. One team routes model calls through a gateway, and another calls providers directly with a shared key.</li>
<li class="">Brittle tool and data connections. Agents are wired point to point into systems, so every backend change breaks a prompt or a tool.</li>
<li class="">Limited visibility when behavior goes wrong. An agent takes an unexpected action. No trace links the request, the model calls, and the tool calls.</li>
<li class="">Lock-in decisions made too early. A tooling choice made for one pilot hard-codes the architecture for everyone who follows.</li>
</ul>
<p>Each pattern maps back to a gap in the minimum viable architecture. That is the practical value of the checklist: it turns vague pilot friction into a specific missing capability.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="where-to-start">Where to start<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#where-to-start" class="hash-link" aria-label="Direct link to Where to start" title="Direct link to Where to start" translate="no">​</a></h2>
<p>Most enterprises are already somewhere on this journey, and the useful step is knowing where. Instead of a maturity score, look for the operational triggers that tell you which capability to add next:</p>
<ul>
<li class="">You use more than one model provider, or plan to. Centralize outbound model traffic so routing, failover, and policy live in one place.</li>
<li class="">Nobody can say which team or agent drove last month's model spend. Add attribution and budgets at the egress point before scaling further.</li>
<li class="">Prompts or tool calls can carry customer or regulated data. Enforce guardrails such as PII masking outside the prompt.</li>
<li class="">Agents act on behalf of users. Give each agent its own identity and record the delegation.</li>
<li class="">You are moving from one pilot to several production agents. Treat observability and evaluation as <a class="" href="https://choreolabs.com/api-management/learn/agent-platform-architecture">shared services</a> rather than per-team projects.</li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>Becoming an agentic enterprise is not a product purchase. It is an architectural commitment that you carry out in stages. The minimum viable architecture defines what has to exist before agents can act safely. The three-stage roadmap shows the order in which teams usually build it. Governance, cost controls, and flexibility keep the program from stalling between pilot and production.</p>
<p>None of this requires replacing what already works. Your systems of record, APIs, and integration layer remain the foundation. What changes is how you expose them, who is allowed to act on them, and how you observe and pay for that activity. Start with the gap your current triggers point to, close it, and then move to the next one.</p>
<p>The <a href="https://wso2.com/agentic-enterprise/" target="_blank" rel="noopener noreferrer" class="">Choreo Labs agentic enterprise overview</a> describes the platforms behind each of these capabilities. It is a useful reference to compare against what you run today.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-an-agentic-enterprise-1">What is an agentic enterprise?<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#what-is-an-agentic-enterprise-1" class="hash-link" aria-label="Direct link to What is an agentic enterprise?" title="Direct link to What is an agentic enterprise?" translate="no">​</a></h3>
<p>It is an organization that lets AI agents carry out work across its systems, not just answer questions. The agents operate with their own identities, within defined policies, and with every action logged for review.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-an-enterprise-agentic-platform">What is an enterprise agentic platform?<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#what-is-an-enterprise-agentic-platform" class="hash-link" aria-label="Direct link to What is an enterprise agentic platform?" title="Direct link to What is an enterprise agentic platform?" translate="no">​</a></h3>
<p>It is the shared infrastructure that teams use to deploy and run agents in production. It typically covers agent identity, guardrails, observability, evaluation, and lifecycle management, so each team does not rebuild them.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-are-agentic-enterprise-solutions">What are agentic enterprise solutions?<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#what-are-agentic-enterprise-solutions" class="hash-link" aria-label="Direct link to What are agentic enterprise solutions?" title="Direct link to What are agentic enterprise solutions?" translate="no">​</a></h3>
<p>They are the products and services that supply parts of the agentic architecture. Examples include AI gateways for model traffic, agent identity services, integration platforms that expose systems through MCP, and agent observability tools.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="do-you-need-to-replace-existing-systems-to-become-an-agentic-enterprise">Do you need to replace existing systems to become an agentic enterprise?<a href="https://choreolabs.com/api-management/learn/agentic-enterprise-roadmap#do-you-need-to-replace-existing-systems-to-become-an-agentic-enterprise" class="hash-link" aria-label="Direct link to Do you need to replace existing systems to become an agentic enterprise?" title="Direct link to Do you need to replace existing systems to become an agentic enterprise?" translate="no">​</a></h3>
<p>No. Systems of record stay in place, and agents reach them through an integration layer. The work lies in exposing and governing access, not in replacing core systems.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="AI agents" term="AI agents"/>
        <category label="Agent-ready APIs" term="Agent-ready APIs"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[AI agent memory: Types, architecture & how to implement it]]></title>
        <id>https://choreolabs.com/api-management/learn/ai-agent-memory</id>
        <link href="https://choreolabs.com/api-management/learn/ai-agent-memory"/>
        <updated>2026-10-13T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[Learn how AI agent memory actually works: the four memory types, vector vs. graph storage, session management, and how to design a memory layer that scales.]]></summary>
        <content type="html"><![CDATA[<p>An employee tells a helpdesk agent on Monday that their VPN keeps dropping. On Tuesday they come back, and the agent asks them to describe the problem from the start. The model did not forget the conversation, because it never had it. A model call does not update the model's weights, so the model carries nothing forward on its own. Any continuity comes from state kept outside the model. The application manages that state, even when a provider hosts the conversation for it.</p>
<p>That application-side state is AI agent memory. This guide is for developers and architects who have to build it. It covers the four memory types, how memory differs from RAG, and where to store memory. It also covers how to scope memory per user, when an agent needs no memory at all, and the mistakes that cause trouble in production.</p>
<aside class="cta_6F_m" aria-label="Give your agents memory that works" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Give your agents memory that works</span><span class="desc_GzK8">Use Choreo AI Gateway to get observability into the state and memory calls your agents make.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=ai-agent-memory" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-ai-agent-memory-and-why-a-bigger-context-window-doesnt-replace-it">What is AI agent memory (and why a bigger context window doesn't replace it)<a href="https://choreolabs.com/api-management/learn/ai-agent-memory#what-is-ai-agent-memory-and-why-a-bigger-context-window-doesnt-replace-it" class="hash-link" aria-label="Direct link to What is AI agent memory (and why a bigger context window doesn't replace it)" title="Direct link to What is AI agent memory (and why a bigger context window doesn't replace it)" translate="no">​</a></h2>
<p>AI agent memory is the stored state an agent can write, retrieve, and update across model calls and sessions. It includes conversation history, facts about users, records of past tasks, and the instructions that shape behavior. The application owns this state and loads the relevant parts into the model's context window on each request.</p>
<p>A useful way to separate the two ideas is to treat the context window as a desk and memory as a filing cabinet. The desk holds what the model is working on during one request, and it is cleared afterwards. The filing cabinet holds what should outlast that request, and the agent pulls the right folder onto the desk when it needs it.</p>
<p>Larger context windows do not remove the need for the cabinet, for four reasons:</p>
<ul>
<li class=""><strong>The window resets:</strong> Nothing in it survives the end of a session unless the application, or a provider-hosted conversation store, saves it first.</li>
<li class=""><strong>More context is not better context:</strong> Liu et al. found that models use information best at the start or end of the input. Accuracy dropped when the relevant passage sat in the middle of a long context (<a href="https://arxiv.org/abs/2307.03172" target="_blank" rel="noopener noreferrer" class="">Liu et al., 2023</a>).</li>
<li class=""><strong>History still costs tokens on every call:</strong> A replayed history adds input tokens to every request, whether or not it is relevant. Provider-hosted state does not remove that cost. <a href="https://developers.openai.com/api/docs/guides/conversation-state" target="_blank" rel="noopener noreferrer" class="">OpenAI's conversation state guide</a> notes that earlier input tokens in a chained conversation are still billed as input. <a href="https://developers.openai.com/api/docs/guides/prompt-caching" target="_blank" rel="noopener noreferrer" class="">Prompt caching</a> can reduce cost and latency for an unchanged prefix, but the cached tokens still occupy the context window.</li>
<li class=""><strong>A transcript is hard to govern:</strong> A memory store can be queried, corrected, and deleted per user. A history pasted into prompts offers few of those controls.</li>
</ul>
<p>A December 2025 survey of agent memory research separates agent memory from LLM memory, RAG, and context engineering. It names token-level, parametric, and latent memory as the three main forms (<a href="https://arxiv.org/abs/2512.13564" target="_blank" rel="noopener noreferrer" class="">Hu et al., 2025</a>). This guide covers token-level memory, which is state stored outside the model as text, records, or embeddings. It is the form application developers control directly.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-four-types-of-ai-agent-memory">The four types of AI agent memory<a href="https://choreolabs.com/api-management/learn/ai-agent-memory#the-four-types-of-ai-agent-memory" class="hash-link" aria-label="Direct link to The four types of AI agent memory" title="Direct link to The four types of AI agent memory" translate="no">​</a></h2>
<p>The widely used four-type taxonomy comes from cognitive science, by way of the CoALA framework from Princeton. CoALA splits agent memory into short-term working memory and three long-term types: episodic, semantic, and procedural (<a href="https://arxiv.org/abs/2309.02427" target="_blank" rel="noopener noreferrer" class="">Sumers et al., 2023</a>). Framework documentation such as the <a href="https://docs.langchain.com/oss/python/concepts/memory" target="_blank" rel="noopener noreferrer" class="">LangGraph memory guide</a> uses the same three long-term types.</p>
<table><thead><tr><th>Memory type</th><th>What it stores</th><th>Example in an IT helpdesk agent</th></tr></thead><tbody><tr><td>Working memory</td><td>The active context for the current step: recent turns, tool results, and intermediate reasoning</td><td>The last few messages about a VPN error and the result of a knowledge base search</td></tr><tr><td>Semantic memory</td><td>Facts about the user, the domain, or the environment</td><td>"This employee uses the Windows VPN client"</td></tr><tr><td>Episodic memory</td><td>Records of past interactions and how they turned out</td><td>"Restarting the VPN client did not fix the issue last week"</td></tr><tr><td>Procedural memory</td><td>Rules and skills that govern behavior, held in model weights, agent code, and system prompts</td><td>"Always search the knowledge base before suggesting a fix"</td></tr></tbody></table>
<p><img decoding="async" loading="lazy" alt="The four types of agent memory. Working memory is short-term and holds what the agent is handling right now, such as the current VPN error and the latest search result. Procedural, semantic, and episodic memory are long-term. Procedural memory holds how the agent should behave, semantic memory holds facts about the user and their environment, and episodic memory holds what happened in past interactions" src="https://choreolabs.com/api-management/learn/assets/images/four-types-of-agent-memory-b60baee6ca1377e4c645d266bc569949.webp" width="3200" height="1614" class="img_ev3q"></p>
<p><em>The four types of agent memory, with an example of each from an IT helpdesk agent.</em></p>
<p>Working memory is the short-term layer. In practice, it is whatever the application assembles into the context window for one call. The other three are long-term memory, and each one fails in its own way. Semantic memory goes stale when facts change, and episodic memory grows without bound unless something prunes it. Procedural memory is the riskiest type for an agent to write. A bad update changes behavior on every future task, as the CoALA authors also note.</p>
<p>Keep semantic memory and semantic search apart. The first is a category of what gets stored, and the second is a retrieval technique that matches by meaning.</p>
<p>The taxonomy tells you what is worth storing. It does not tell you where to store memory, how to scope it, or when to delete it. Those decisions take most of the implementation effort.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="ai-agent-memory-vs-rag-they-solve-different-problems">AI agent memory vs. RAG: They solve different problems<a href="https://choreolabs.com/api-management/learn/ai-agent-memory#ai-agent-memory-vs-rag-they-solve-different-problems" class="hash-link" aria-label="Direct link to AI agent memory vs. RAG: They solve different problems" title="Direct link to AI agent memory vs. RAG: They solve different problems" translate="no">​</a></h2>
<p>Memory and retrieval augmented generation often run on the same infrastructure, so they get conflated. Both can embed text, store it in a vector database, and load close matches into a prompt. They also overlap, because an agent can retrieve its own memories through a RAG pipeline. This guide therefore uses an operational convention rather than a strict taxonomy. RAG retrieves knowledge, and memory retains interaction history and state.</p>
<p>Under that convention, the two usually differ in three ways.</p>
<ul>
<li class=""><strong>Where it comes from:</strong> A RAG corpus usually comes from curated documents such as policies, runbooks, and product specs. Memory comes from interactions, written either by the agent or by a deterministic workflow.</li>
<li class=""><strong>Who it belongs to:</strong> A RAG index is usually shared across many users, with access filters applied per request. Memory usually belongs to one user, one session, or one agent.</li>
<li class=""><strong>How it changes:</strong> A RAG corpus changes when source documents change. Memory can change on every turn, and old entries can contradict new ones.</li>
</ul>
<p><img decoding="async" loading="lazy" alt="One store, two paths. On the write path, an employee says &amp;quot;I&amp;#39;m on the Windows VPN client&amp;quot; and the agent saves a memory with metadata: tenant_id acme, user_id emp-1001, memory_type semantic, the content, source session-42, a created_at timestamp, and no expiry. On the read path, &amp;quot;My VPN dropped again&amp;quot; is embedded and a similarity search filtered by tenant and user finds the closest memories, which are loaded into the prompt" src="https://choreolabs.com/api-management/learn/assets/images/memory-write-and-read-paths-4c76c781bdd9729389ae36bce063f75a.webp" width="3200" height="1632" class="img_ev3q"></p>
<p><em>A memory store serves a write path and a read path. The scope filter sits inside the retrieval query.</em></p>
<p>RAG grounds an answer in external knowledge, while memory gives an agent continuity across sessions. The <a href="https://wso2.com/integration-platform/docs/genai/tutorials/it-helpdesk-chatbot" target="_blank" rel="noopener noreferrer" class="">Choreo Integrator IT helpdesk agent tutorial</a> uses both in one agent. A knowledge base search tool supplies troubleshooting guidance, and a persistent store keeps each employee's conversation history. The two have different owners and retention rules, so keep them in separate stores even when the storage engine is the same. The retrieval side is covered in this guide to <a class="" href="https://choreolabs.com/api-management/learn/rag-api-architecture">RAG API architecture</a>.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="memory-storage-and-retrieval-vector-graph-and-hybrid-approaches">Memory storage and retrieval: Vector, graph, and hybrid approaches<a href="https://choreolabs.com/api-management/learn/ai-agent-memory#memory-storage-and-retrieval-vector-graph-and-hybrid-approaches" class="hash-link" aria-label="Direct link to Memory storage and retrieval: Vector, graph, and hybrid approaches" title="Direct link to Memory storage and retrieval: Vector, graph, and hybrid approaches" translate="no">​</a></h2>
<p>A memory layer has two paths. The write path decides what is worth keeping and attaches metadata to it. The read path decides what to load into the next prompt, and the storage choice shapes both.</p>
<ul>
<li class="">Relational or key-value stores hold ordered records under a key, such as a session ID or user ID. They suit conversation history and exact lookups, and they are the easiest to audit and delete from. On their own, they match exact keys rather than meaning. Extensions such as <a href="https://github.com/pgvector/pgvector" target="_blank" rel="noopener noreferrer" class="">pgvector</a> add similarity search to PostgreSQL, so one database can serve both access patterns.</li>
<li class="">Vector databases store each memory's content, or a reference to it, next to its embedding, because an embedding cannot be turned back into text. They retrieve entries by similarity search. They handle fuzzy recall well, such as "what has this user said about laptops?" Many also filter on stored IDs, metadata, and timestamps. Similarity ranking alone does not order results by time or resolve contradictions, because two conflicting facts can both score as close matches. Choose a store by the queries you need and how strictly updates must stay consistent, not by database category.</li>
<li class="">Knowledge graphs store entities and the relationships between them, such as an employee, their device, and their tickets. A temporal graph can record when a fact became true and when it stopped being true, which helps with stale memories. The cost is an extraction step that turns free text into nodes and edges and can introduce its own errors.</li>
<li class="">Hybrid designs combine these. A relational store holds history and metadata, a vector index handles recall by meaning, and a graph is added only where relationships carry the answer.</li>
</ul>
<p>Start with the simplest structure that answers your agent's actual questions. A graph that nobody queries by relationship is pure overhead.</p>
<p>Whichever store you choose, apply scope filters inside the retrieval query rather than after it. Filtering afterwards means another user's memory has already reached your service layer. The Choreo Labs example later in this guide uses the relational pattern for conversation history.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="session-management-and-conversation-state">Session management and conversation state<a href="https://choreolabs.com/api-management/learn/ai-agent-memory#session-management-and-conversation-state" class="hash-link" aria-label="Direct link to Session management and conversation state" title="Direct link to Session management and conversation state" translate="no">​</a></h2>
<p>Agent state management starts with one decision, which is the scope of each memory. Most systems need some mix of three scopes:</p>
<ul>
<li class="">Session scope covers a single session or thread. It holds the multi-turn conversation that follow-up questions depend on.</li>
<li class="">User scope spans sessions for one person. It holds user preferences and a record of previous interactions.</li>
<li class="">Shared scope spans users or agents. It holds team knowledge or states that several agents hand off between them.</li>
</ul>
<p>LangGraph draws the same line in code. Its short-term memory is scoped to a single thread through checkpoints, while long-term memory lives in custom namespaces that persist across threads.</p>
<p>The scope key has to be trustworthy. If a client can choose any session ID or user ID, isolation by key protects nothing. A caller could simply request someone else's key. Derive the key from the caller's verified identity on the server, or validate a client-supplied session ID against that identity before reading memory.</p>
<p>Metadata makes scoping, ranking, and cleanup possible. A workable minimum is the user ID, the memory type, a timestamp, the source of the entry, and an expiry. Add a tenant ID wherever user IDs can collide across customers. This small local sketch uses Python's built-in SQLite module to show scoped writes, scoped reads, and expiry. It is an illustration, not production code:</p>
<div class="language-python codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-python codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token keyword" style="color:#3D8BFF">import</span><span class="token plain"> sqlite3</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token keyword" style="color:#3D8BFF">import</span><span class="token plain"> time</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">MAX_RECALL </span><span class="token operator">=</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">20</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">db </span><span class="token operator">=</span><span class="token plain"> sqlite3</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">connect</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token string" style="color:#4ADE80">"agent_memory.db"</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">db</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">execute</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token triple-quoted-string string" style="color:#4ADE80">"""</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">    CREATE TABLE IF NOT EXISTS memories (</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">        tenant_id   TEXT NOT NULL,</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">        user_id     TEXT NOT NULL,</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">        memory_type TEXT NOT NULL,</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">        content     TEXT NOT NULL,</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">        source      TEXT NOT NULL,</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">        created_at  REAL NOT NULL,</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">        expires_at  REAL</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">    )</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">"""</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token keyword" style="color:#3D8BFF">def</span><span class="token plain"> </span><span class="token function" style="color:#FFFFFF;font-weight:600">remember</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">tenant_id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> user_id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> memory_type</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> content</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> source</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">             ttl_seconds</span><span class="token operator">=</span><span class="token boolean" style="color:#FF4600">None</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token keyword" style="color:#3D8BFF">if</span><span class="token plain"> ttl_seconds </span><span class="token keyword" style="color:#3D8BFF">is</span><span class="token plain"> </span><span class="token keyword" style="color:#3D8BFF">not</span><span class="token plain"> </span><span class="token boolean" style="color:#FF4600">None</span><span class="token plain"> </span><span class="token keyword" style="color:#3D8BFF">and</span><span class="token plain"> ttl_seconds </span><span class="token operator">&lt;=</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">0</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token keyword" style="color:#3D8BFF">raise</span><span class="token plain"> ValueError</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token string" style="color:#4ADE80">"ttl_seconds must be a positive number or None"</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    now </span><span class="token operator">=</span><span class="token plain"> time</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">time</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    expires_at </span><span class="token operator">=</span><span class="token plain"> now </span><span class="token operator">+</span><span class="token plain"> ttl_seconds </span><span class="token keyword" style="color:#3D8BFF">if</span><span class="token plain"> ttl_seconds </span><span class="token keyword" style="color:#3D8BFF">is</span><span class="token plain"> </span><span class="token keyword" style="color:#3D8BFF">not</span><span class="token plain"> </span><span class="token boolean" style="color:#FF4600">None</span><span class="token plain"> </span><span class="token keyword" style="color:#3D8BFF">else</span><span class="token plain"> </span><span class="token boolean" style="color:#FF4600">None</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    db</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">execute</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token string" style="color:#4ADE80">"INSERT INTO memories VALUES (?, ?, ?, ?, ?, ?, ?)"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">tenant_id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> user_id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> memory_type</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> content</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> source</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> now</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> expires_at</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    db</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">commit</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token keyword" style="color:#3D8BFF">def</span><span class="token plain"> </span><span class="token function" style="color:#FFFFFF;font-weight:600">recall</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">tenant_id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> user_id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> memory_type</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> limit</span><span class="token operator">=</span><span class="token number" style="color:#FF4600">5</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    limit </span><span class="token operator">=</span><span class="token plain"> </span><span class="token builtin" style="color:#3D8BFF">max</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token number" style="color:#FF4600">1</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token builtin" style="color:#3D8BFF">min</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token builtin" style="color:#3D8BFF">int</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">limit</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> MAX_RECALL</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    rows </span><span class="token operator">=</span><span class="token plain"> db</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">execute</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token triple-quoted-string string" style="color:#4ADE80">"""</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">        SELECT content FROM memories</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">        WHERE tenant_id = ? AND user_id = ? AND memory_type = ?</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">          AND (expires_at IS NULL OR expires_at &gt; ?)</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">        ORDER BY created_at DESC</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">        LIMIT ?</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token triple-quoted-string string" style="color:#4ADE80">        """</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">tenant_id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> user_id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> memory_type</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> time</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">time</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> limit</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token keyword" style="color:#3D8BFF">return</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token plain">content </span><span class="token keyword" style="color:#3D8BFF">for</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">content</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"> </span><span class="token keyword" style="color:#3D8BFF">in</span><span class="token plain"> rows</span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token comment" style="color:#8A93A6;font-style:italic"># tenant_id and user_id come from the caller's verified identity, never from the request body</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">tenant_id </span><span class="token operator">=</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"acme"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">user_id </span><span class="token operator">=</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"emp-1001"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">remember</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">tenant_id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> user_id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"semantic"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"Uses the Windows VPN client"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">         source</span><span class="token operator">=</span><span class="token string" style="color:#4ADE80">"session-42"</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">remember</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">tenant_id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> user_id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"episodic"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"Restarting the VPN client did not fix the issue"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">         source</span><span class="token operator">=</span><span class="token string" style="color:#4ADE80">"session-42"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> ttl_seconds</span><span class="token operator">=</span><span class="token number" style="color:#FF4600">30</span><span class="token plain"> </span><span class="token operator">*</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">24</span><span class="token plain"> </span><span class="token operator">*</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">3600</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token keyword" style="color:#3D8BFF">print</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">recall</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">tenant_id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> user_id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"episodic"</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token punctuation" style="color:#8A93A6">)</span><br></span></code></pre></div></div>
<p>The semantic fact has no expiry, while the episodic record stops being returned after 30 days. Expiry in this sketch only removes a row from recall. It does not delete the row, so production retention also needs a deletion or compaction job and a plan for backups. The sketch rejects zero or negative TTLs and caps how many rows one call can return. Session scope works the same way as tenant scope, with a <code>session_id</code> column added to the filter. A vector-backed version would rank by similarity, but the scope filter would stay in the query.</p>
<p>Shared memory needs one more rule, which is to decide which agents may write to it. Write access lets one agent's mistake shape what every other agent reads next.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="when-you-actually-need-a-memory-layer-and-when-you-dont">When you actually need a memory layer (and when you don't)<a href="https://choreolabs.com/api-management/learn/ai-agent-memory#when-you-actually-need-a-memory-layer-and-when-you-dont" class="hash-link" aria-label="Direct link to When you actually need a memory layer (and when you don't)" title="Direct link to When you actually need a memory layer (and when you don't)" translate="no">​</a></h2>
<p>Not every agent needs persistent memory. Adding it creates a data store you must secure, retain, and delete from. Base the decision on real operational triggers:</p>
<ul>
<li class=""><strong>Returning users expect continuity.</strong> A customer support agent that handles a case over several days should not make people repeat themselves.</li>
<li class=""><strong>Tasks outlive a single session or process.</strong> Long-running workflows and conversations that must survive a service restart need state outside the runtime.</li>
<li class=""><strong>Behavior depends on user preferences.</strong> Personalization requires semantic memory scoped to the user.</li>
<li class=""><strong>Several agents hand off work.</strong> A shared, well-scoped store is easier to govern than transcripts passed between agents.</li>
<li class=""><strong>Records must be retained.</strong> Audit or compliance requirements can mandate that conversation records persist in a governed database.</li>
</ul>
<p>Memory adds little elsewhere. One-off question answering and stateless tasks such as classification are better served by a single call. Lookups where every answer comes from a document corpus are better served by RAG. A coding agent often gets more value from procedural memory, such as build steps and project conventions, than from a transcript of last week's session.</p>
<p>Session-only memory held in process is a sensible middle ground. It supports multi-turn conversations without a persistent store until one of these triggers appears.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="common-ai-agent-memory-mistakes">Common AI agent memory mistakes<a href="https://choreolabs.com/api-management/learn/ai-agent-memory#common-ai-agent-memory-mistakes" class="hash-link" aria-label="Direct link to Common AI agent memory mistakes" title="Direct link to Common AI agent memory mistakes" translate="no">​</a></h2>
<p>These failure modes recur once agents with memory reach production:</p>
<ul>
<li class=""><strong>Saving everything:</strong> Storing every turn verbatim creates retrieval noise, and each irrelevant entry competes for space in the context window. Store distilled facts and outcomes, and keep raw transcripts only where retention rules require them.</li>
<li class=""><strong>Skipping metadata:</strong> Without a user ID, memory type, timestamp, and source, you cannot scope, rank, expire, or delete memories. Retrofitting metadata onto an existing store is harder than adding it on day one.</li>
<li class=""><strong>Ignoring the memory lifecycle:</strong> An employee replaces a laptop, and the old device record now contradicts the new one. Define how entries expire, how newer facts supersede older ones, and how a user's memory is deleted on request. Overflow handling belongs here too. Trimming drops the oldest messages, while summarization keeps a compressed version at the cost of detail.</li>
<li class=""><strong>Treating the context window as the memory store:</strong> A long prompt is not persistence, and it cannot be queried, audited, or selectively forgotten.</li>
<li class=""><strong>Trusting every memory write:</strong> OWASP lists Memory and Context Poisoning as ASI06 in its <a href="https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/" target="_blank" rel="noopener noreferrer" class="">Top 10 for Agentic Applications</a>. Content written into memory from an untrusted document or tool result becomes an input to every later session. Record the source of each entry, validate writes that come from untrusted content, and keep the ability to roll back.</li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-choreo-labs-approaches-agent-memory">How Choreo Labs approaches agent memory<a href="https://choreolabs.com/api-management/learn/ai-agent-memory#how-choreo-labs-approaches-agent-memory" class="hash-link" aria-label="Direct link to How Choreo Labs approaches agent memory" title="Direct link to How Choreo Labs approaches agent memory" translate="no">​</a></h2>
<p>Choreo Labs does not sell a dedicated memory product, and dedicated memory layers such as <a href="https://mem0.ai/" target="_blank" rel="noopener noreferrer" class="">Mem0</a> exist for teams that want one. Choreo Labs' approach is narrower. Memory is part of building an agent on an integration platform that already connects to enterprise databases, APIs, and services.</p>
<p>In Choreo Integrator, you add memory to an AI agent from the agent node in the visual designer, as described in the <a href="https://wso2.com/integration-platform/docs/genai/develop/agents/memory" target="_blank" rel="noopener noreferrer" class="">agent memory documentation</a>. The memory is short-term conversation memory, and the store behind it is configurable:</p>
<ul>
<li class=""><strong>In-memory store:</strong> This is the default. It keeps a window of recent messages in process memory, and the history does not survive a restart.</li>
<li class=""><strong>MSSQL store:</strong> This keeps conversation history in a Microsoft SQL Server table, so it survives restarts and spans sessions. The docs recommend it for long-running conversations and for audit or compliance requirements. Several agents can share conversation memory by pointing at the same store.</li>
<li class=""><strong>Custom stores:</strong> PostgreSQL, MySQL, and Redis are not supported out of the box. Teams can implement the <code>ai:Memory</code> interface, which defines get, update, and delete operations by key, to use an existing database.</li>
</ul>
<p>Memory works as a sliding window. When it fills, the default handler trims the oldest messages, and a model-assisted handler can summarize them instead. Choreo Integrator: MI is a separate runtime, and agents built with its AI Connector have their own memory configuration. MI added agent memory trim and summarization support in <a href="https://github.com/wso2/product-integrator-mi/releases/tag/v4.6.0" target="_blank" rel="noopener noreferrer" class="">release 4.6.0</a>. Check the MI documentation for that runtime rather than assuming the stores above apply.</p>
<p>Each session ID keeps an isolated history. In the IT helpdesk tutorial, an employee reports a VPN problem, and the service then restarts. The agent still recalls the issue when the employee returns with the same session ID, because the history lives in MSSQL. Long-term semantic or episodic memory beyond conversation history is not a built-in memory type today. You build it on the custom memory interface or through tools.</p>
<p>An agent with memory involves three independent traffic flows. They are separate paths, not hops in a chain:</p>
<ol>
<li class=""><strong>Inbound</strong> - A client calls the agent's chat endpoint with a message and a session ID.</li>
<li class=""><strong>Memory access</strong> - The agent runtime reads and writes its memory store directly, with its own database credentials, inside your network.</li>
<li class=""><strong>Outbound</strong> - The agent calls the LLM provider with a prompt that now contains retrieved memory.</li>
</ol>
<p>The outbound flow is where memory content leaves your boundary. Choreo AI Gateway governs that LLM traffic. It can apply <a href="https://wso2.com/api-platform/docs/ai-gateway/1.2.0/guardrails/" target="_blank" rel="noopener noreferrer" class="">guardrails such as PII masking</a> to prompts before they reach a provider, and it tracks token usage by provider. The same AI Gateway also governs the MCP traffic agents use to reach tools. Both are managed from a single control plane in Choreo API Management, so policies for model calls and tool access sit in one place.</p>
<p>The <a href="https://wso2.github.io/agent-flavored-markdown/specification" target="_blank" rel="noopener noreferrer" class="">Agent-Flavored Markdown (AFM) specification</a>, Choreo Labs' open format for portable agent definitions, does not yet prescribe a memory standard. Choreo Labs has named agent memory as future work for the spec.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/ai-agent-memory#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>Agent memory is a data and scoping problem, not a prompting trick. The four-type taxonomy helps you decide what to keep. Production outcomes depend more on persistence, scope keys tied to verified identity, metadata, and lifecycle rules. Get those right early, and you can revisit the choice of relational, vector, or graph storage as your queries change.</p>
<p>If your agents already run next to enterprise databases, memory persistence can extend infrastructure you operate today instead of adding a new system. Start with the <a href="https://wso2.com/integration-platform/docs/genai/tutorials/it-helpdesk-chatbot" target="_blank" rel="noopener noreferrer" class="">IT helpdesk tutorial</a> to see MSSQL-backed agent memory working end to end in Choreo Integrator.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/ai-agent-memory#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="does-an-ai-agent-have-memory">Does an AI agent have memory?<a href="https://choreolabs.com/api-management/learn/ai-agent-memory#does-an-ai-agent-have-memory" class="hash-link" aria-label="Direct link to Does an AI agent have memory?" title="Direct link to Does an AI agent have memory?" translate="no">​</a></h3>
<p>Not by default. The underlying model keeps no state between calls, so an agent remembers only what its application stores and loads back into the prompt.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-much-ram-does-an-ai-agent-need">How much RAM does an AI agent need?<a href="https://choreolabs.com/api-management/learn/ai-agent-memory#how-much-ram-does-an-ai-agent-need" class="hash-link" aria-label="Direct link to How much RAM does an AI agent need?" title="Direct link to How much RAM does an AI agent need?" translate="no">​</a></h3>
<p>Agent memory is stored data, not RAM. An agent that calls a hosted model runs like a typical service, and hardware needs grow mainly when you host the model yourself.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-are-the-four-types-of-memory-that-every-ai-agent-needs">What are the four types of memory that every AI agent needs?<a href="https://choreolabs.com/api-management/learn/ai-agent-memory#what-are-the-four-types-of-memory-that-every-ai-agent-needs" class="hash-link" aria-label="Direct link to What are the four types of memory that every AI agent needs?" title="Direct link to What are the four types of memory that every AI agent needs?" translate="no">​</a></h3>
<p>Working, semantic, episodic, and procedural memory. Every agent uses working and procedural memory, but semantic and episodic memory are worth adding only when users return or tasks span sessions.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-do-ai-agents-manage-memory">How do AI agents manage memory?<a href="https://choreolabs.com/api-management/learn/ai-agent-memory#how-do-ai-agents-manage-memory" class="hash-link" aria-label="Direct link to How do AI agents manage memory?" title="Direct link to How do AI agents manage memory?" translate="no">​</a></h3>
<p>The application decides what to write, where to store it, and which scope it belongs to. On each call, it loads the most relevant entries into the context window. Over time, it trims or summarizes overflow and expires stale entries.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="AI agents" term="AI agents"/>
        <category label="Agent-ready APIs" term="Agent-ready APIs"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[Idempotent tool design: Building agent tools safe to retry]]></title>
        <id>https://choreolabs.com/api-management/learn/designing-idempotent-tools</id>
        <link href="https://choreolabs.com/api-management/learn/designing-idempotent-tools"/>
        <updated>2026-10-13T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[AI agents retry aggressively and don't know when it's safe. Learn the idempotency patterns that keep retries from becoming duplicate actions.]]></summary>
        <content type="html"><![CDATA[<p>An agent is halfway through a customer support task. It calls an <code>issue_refund</code> tool, the refund service responds slowly, and the tool call times out. The model sees an error, decides the step failed, and calls the tool again. Both calls succeed. The customer gets refunded twice, and the agent reports that everything went fine.</p>
<p>Timeouts are routine, and retrying is what agents do. The bug lives in the tool, which treated two attempts at one refund as two refunds. This guide covers why agent retries behave differently and which patterns keep a retry from becoming a second side effect. It expands one recommendation from the <a href="https://wso2.com/library/blogs/api-design-for-ai-agents/" target="_blank" rel="noopener noreferrer" class="">broader set of design changes APIs need for agent traffic</a>. It's also part of our series on <a class="" href="https://choreolabs.com/api-management/learn/agent-ready-apis">agent-ready APIs</a>.</p>
<aside class="cta_6F_m" aria-label="Make agent retries safe, not costly" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Make agent retries safe, not costly</span><span class="desc_GzK8">Use Choreo's MCP gateway layer to add the rate limiting and audit logging that stop duplicate retries.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=designing-idempotent-tools" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="why-agents-retry-differently-than-humans">Why agents retry differently than humans<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#why-agents-retry-differently-than-humans" class="hash-link" aria-label="Direct link to Why agents retry differently than humans" title="Direct link to Why agents retry differently than humans" translate="no">​</a></h2>
<p>Picture a person paying for an order on a slow checkout page. The spinner keeps turning. Most people wait, refresh with care, or check their bank app before they press Pay again. They know a payment matters, so they treat the uncertainty with caution.</p>
<p>An agent has no such instinct. As Nuwan Dias, VP and Deputy CTO at Choreo Labs, puts it, a payment is "just another step in a series of steps" to an agent. If that step runs long, the agent retries without weighing the consequences.</p>
<p>Agent retries also come from more places than a refresh button:</p>
<ol>
<li class=""><strong>The client layer:</strong> HTTP client libraries can retry transient failures on their own. Stripe's <a href="https://stripe.com/blog/idempotency" target="_blank" rel="noopener noreferrer" class="">engineering blog</a> notes, for example, that its Ruby library retries failed requests automatically.</li>
<li class=""><strong>The model:</strong> After a timeout or an unclear error, the model can call the same tool again, sometimes with different arguments.</li>
<li class=""><strong>Recovery and resumption:</strong> An agent that crashes and resumes from a checkpoint can re-run tool calls it already completed. The authors of <a href="https://arxiv.org/abs/2603.20625" target="_blank" rel="noopener noreferrer" class="">ACRFence</a> call this action replay. They surveyed 12 agent frameworks and found tool-call side-effect issues across them, with duplicate payments among the consequences.</li>
</ol>
<p>Fan-out makes this worse. <a href="https://wso2.com/library/blogs/api-design-for-ai-agents/" target="_blank" rel="noopener noreferrer" class="">Choreo Labs' analysis of agent traffic</a> describes one tool call fanning out into dozens of search, product detail, and inventory requests. Each one is another chance for a timeout and a retry. For human-driven clients, idempotency improves reliability. For any tool an agent can call, it's a requirement.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-idempotency-actually-means-and-what-it-doesnt">What idempotency actually means (and what it doesn't)<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#what-idempotency-actually-means-and-what-it-doesnt" class="hash-link" aria-label="Direct link to What idempotency actually means (and what it doesn't)" title="Direct link to What idempotency actually means (and what it doesn't)" translate="no">​</a></h2>
<p>An operation is idempotent when calling it several times with the same input leaves the system in the same state as calling it once. Duplicate requests can still reach the server. Idempotency makes those duplicates harmless, because a repeat has no additional intended effect. The server can still log each request. The response to a repeat can differ, but the outcome can't.</p>
<p>HTTP defines this property per method. Under <a href="https://www.rfc-editor.org/rfc/rfc9110#section-9.2.2" target="_blank" rel="noopener noreferrer" class="">RFC 9110</a>, PUT, DELETE, and the safe methods such as GET are idempotent. POST isn't on that list, and <a href="https://www.rfc-editor.org/rfc/rfc5789" target="_blank" rel="noopener noreferrer" class="">PATCH</a> isn't guaranteed to be idempotent either. Stripe's <a href="https://stripe.com/blog/idempotency" target="_blank" rel="noopener noreferrer" class="">engineering post on idempotency</a> shows why PUT is safe to repeat. Its example request carries the full definition of a DNS record. If the record already exists, the server ignores the duplicate and returns success.</p>
<p>Three things idempotency does not mean:</p>
<ol>
<li class=""><strong>It doesn't mean identical responses:</strong> A repeated DELETE can return <code>404</code> where the first call returned <code>200</code>. RFC 9110 allows the response to differ while the intended effect stays the same.</li>
<li class=""><strong>It doesn't come from the method name alone:</strong> A POST handler can be made idempotent with a key. A PUT handler can break the rule if it appends instead of replacing.</li>
<li class=""><strong>It doesn't assume byte-identical input:</strong> Agents rarely send exactly the same request twice. A retried tool call may reformat an amount or reword a note. Research on <a href="https://arxiv.org/abs/2608.01710" target="_blank" rel="noopener noreferrer" class="">agent replay</a> studies these semantically equivalent repeat calls directly. Don't fuzzy-match them to decide two financial operations are the same. Define a short list of allowed normalizations, such as one canonical format for amounts, and reject any other difference under the same key.</li>
</ol>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-idempotency-key-pattern">The idempotency key pattern<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#the-idempotency-key-pattern" class="hash-link" aria-label="Direct link to The idempotency key pattern" title="Direct link to The idempotency key pattern" translate="no">​</a></h2>
<p>Some operations can't be idempotent by nature, such as charging a card or issuing a refund. An idempotency key makes them idempotent by contract. The client generates a unique key for one logical operation and sends it with the request. The server stores the key with the outcome. If the same key arrives again, the server returns the stored result instead of running the operation twice. Keys are only kept for a limited window, though. Stripe, for example, can remove keys once they're at least 24 hours old in its <a href="https://docs.stripe.com/api/idempotent_requests" target="_blank" rel="noopener noreferrer" class="">v1 API</a>, so a retry after that point runs as new work.</p>
<p>Stripe frames the key's value around three failure points. Each one maps to something an agent's tool loop hits in production:</p>
<table><thead><tr><th>Where the failure happens</th><th>What the server does on retry</th><th>What the agent sees</th></tr></thead><tbody><tr><td>The connection fails before the server receives the request</td><td>Sees the key for the first time and processes the request normally</td><td>A network error, with no side effect</td></tr><tr><td>The call fails while the server is processing it</td><td>Returns a pending or conflict response until the outcome is known. Reruns only if it can confirm the first attempt was rolled back</td><td>A timeout, with an unknown outcome</td></tr><tr><td>The operation succeeds, but the response is lost</td><td>Returns the cached result of the successful operation</td><td>A timeout, after the refund already happened</td></tr></tbody></table>
<p>The third row is the one that produces duplicate refunds. It's also the one an agent can't detect, because a lost success and a real failure look the same to the caller.</p>
<p>The second row needs care too. An unknown outcome isn't proof that nothing happened. A worker can crash after the refund goes through but before it records the result. Decide which component owns recovery. If the write reaches an external provider, pass the same key downstream or reconcile against the provider's records before running anything again.</p>
<p>The server-side mechanics, such as atomic key claims, request fingerprints, and retention windows, are covered in our <a class="" href="https://choreolabs.com/api-management/learn/idempotency-keys-for-apis">guide to idempotency keys for APIs</a>. For agents, the harder question sits upstream: who creates the key, and when.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="create-the-key-outside-the-model-once-per-logical-step">Create the key outside the model, once per logical step<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#create-the-key-outside-the-model-once-per-logical-step" class="hash-link" aria-label="Direct link to Create the key outside the model, once per logical step" title="Direct link to Create the key outside the model, once per logical step" translate="no">​</a></h3>
<p>If the model writes the key, a repeat call can arrive with a fresh one. The server then treats it as new work. Instead, have the agent runtime bind the key to a durable operation identity. A plan step ID alone isn't enough, because replanning can reuse a step ID for different work. Tie the identity to the tenant or user, the tool, and the authorized parameters. Reuse it for every retry of that authorized action, and reject a call that reuses it with different parameters. Create a new identity only for a new authorized action.</p>
<div class="language-python codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-python codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token keyword" style="color:#3D8BFF">import</span><span class="token plain"> uuid</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token comment" style="color:#8A93A6;font-style:italic"># key_store and tool.invoke are application abstractions, not MCP SDK methods.</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token keyword" style="color:#3D8BFF">def</span><span class="token plain"> </span><span class="token function" style="color:#FFFFFF;font-weight:600">call_write_tool</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">operation_id</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token builtin" style="color:#3D8BFF">str</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> tool</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> args</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token builtin" style="color:#3D8BFF">dict</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token comment" style="color:#8A93A6;font-style:italic"># The runtime owns the key. The model never generates it.</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token comment" style="color:#8A93A6;font-style:italic"># create_or_get is atomic and durable, for example an INSERT with a unique</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token comment" style="color:#8A93A6;font-style:italic"># constraint on operation_id that returns the existing row on conflict.</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token comment" style="color:#8A93A6;font-style:italic"># Every worker, including one that restarts, gets the first stored key.</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    key </span><span class="token operator">=</span><span class="token plain"> key_store</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">create_or_get</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">operation_id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> default</span><span class="token operator">=</span><span class="token builtin" style="color:#3D8BFF">str</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">uuid</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">uuid4</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token keyword" style="color:#3D8BFF">return</span><span class="token plain"> tool</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">invoke</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">args</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> headers</span><span class="token operator">=</span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token string" style="color:#4ADE80">"Idempotency-Key"</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> key</span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token punctuation" style="color:#8A93A6">)</span><br></span></code></pre></div></div>
<p>Store the key before the first attempt. Otherwise, a crash between sending and storing leaves the resumed agent unable to recognize its own earlier call.</p>
<p><img decoding="async" loading="lazy" alt="Sequence diagram with the model, the agent runtime, and the refund API. The runtime atomically creates the key for operation op-8812 and sends POST /refunds with that key. The API issues the refund and stores the result, but the 201 response is lost and the tool call times out. The model calls the same tool again, the runtime returns the same key, and the API replies 409 while the key is in progress, then returns the stored result with no second refund" src="https://choreolabs.com/api-management/learn/assets/images/idempotency-key-retry-sequence-96dc965df7f9ac8aad2ebcc317330458.webp" width="3200" height="2502" class="img_ev3q"></p>
<p><em>One operation, one key: how a retried tool call stays a single refund.</em></p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="beyond-the-key-idempotency-as-part-of-a-tools-effect-type">Beyond the key: Idempotency as part of a tool's "effect type"<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#beyond-the-key-idempotency-as-part-of-a-tools-effect-type" class="hash-link" aria-label="Direct link to Beyond the key: Idempotency as part of a tool's &quot;effect type&quot;" title="Direct link to Beyond the key: Idempotency as part of a tool's &quot;effect type&quot;" translate="no">​</a></h2>
<p>A key prevents duplicate execution. It doesn't tell anyone which tools need one. That decision is easier to get right when every tool states its effects explicitly.</p>
<p>Engineer Cam Pedersen sketches one way to do this in his essay <a href="https://campedersen.com/agent-patterns" target="_blank" rel="noopener noreferrer" class="">The Importance of Being Idempotent</a>. He gives each tool call a type, and treats its effect profile as the part that matters. A refund tool, in his example, is irreversible and requires a key. A ticket lookup has no effect worth undoing. He also tracks taint, meaning whether a call's inputs may come from untrusted content.</p>
<p>Applied to a support agent's tools, an effect profile might look like this:</p>
<table><thead><tr><th>Tool</th><th>Reversibility</th><th>Idempotency key</th><th>Human approval</th></tr></thead><tbody><tr><td><code>get_ticket</code></td><td>None (read only)</td><td>Not needed</td><td>Not needed</td></tr><tr><td><code>update_shipping_address</code></td><td>Reversible</td><td>Optional, since a full replace is safe to repeat</td><td>Not needed</td></tr><tr><td><code>issue_store_credit</code></td><td>Compensable, because a reversing entry can offset it</td><td>Required</td><td>Set by policy</td></tr><tr><td><code>issue_refund</code></td><td>Irreversible</td><td>Required</td><td>Required above a policy threshold</td></tr></tbody></table>
<p>The reversibility classes come from <a href="https://arxiv.org/abs/2604.23283" target="_blank" rel="noopener noreferrer" class="">Revisable by Design</a>, a 2026 paper that sorts agent actions into idempotent, reversible, compensable, and irreversible. Pedersen uses "pure" where the paper says "idempotent." He also flags the combined signature as his own synthesis, so treat it as one emerging approach, not settled practice.</p>
<p>The takeaway holds either way. A refund and a ticket lookup carry very different risk, so idempotency is a per-tool decision. A blanket policy either over-engineers safe reads or under-protects dangerous writes.</p>
<p>MCP gives you a place to record part of this. The <a href="https://modelcontextprotocol.io/specification/2026-07-28/server/tools" target="_blank" rel="noopener noreferrer" class="">MCP specification</a> defines optional tool annotations, including <code>readOnlyHint</code>, <code>destructiveHint</code>, and <code>idempotentHint</code>. It also says clients must treat annotations as untrusted unless they come from a trusted server. Annotations describe a tool. They don't enforce anything. Enforcement has to live in the runtime or the gateway, and so does <a class="" href="https://choreolabs.com/api-management/learn/govern-write-capable-mcp-tools-at-the-gateway-in-wso2-api-platform">scoping which agents can call irreversible actions at all</a>.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="failure-modes-what-happens-when-tools-arent-idempotent">Failure modes: What happens when tools aren't idempotent<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#failure-modes-what-happens-when-tools-arent-idempotent" class="hash-link" aria-label="Direct link to Failure modes: What happens when tools aren't idempotent" title="Direct link to Failure modes: What happens when tools aren't idempotent" translate="no">​</a></h2>
<p>Missing idempotency rarely shows up in a demo. It shows up under load, during an incident, or when a long task resumes. Three failure modes are worth separating, because each one needs a different fix.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="duplicate-writes-from-naive-retries">Duplicate writes from naive retries<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#duplicate-writes-from-naive-retries" class="hash-link" aria-label="Direct link to Duplicate writes from naive retries" title="Direct link to Duplicate writes from naive retries" translate="no">​</a></h3>
<p>This is the refund-twice scenario. Stripe's version is a charge endpoint: call it twice by accident, and the customer pays twice. With agents, the duplicate can arrive seconds later from the model or hours later from a resumed run.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="retry-storms-that-deepen-an-outage">Retry storms that deepen an outage<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#retry-storms-that-deepen-an-outage" class="hash-link" aria-label="Direct link to Retry storms that deepen an outage" title="Direct link to Retry storms that deepen an outage" translate="no">​</a></h3>
<p>This failure has nothing to do with duplicates. When a backend degrades, every agent session that depends on it starts retrying. Stripe notes that retries during an incident can contribute to further degradation. If many clients fail together and retry on similar schedules, their attempts arrive in waves. That's the thundering herd problem. It hits perfectly idempotent tools too. Idempotency stops the duplicate effect, but it does nothing about the extra load.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="plan-level-replay">Plan-level replay<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#plan-level-replay" class="hash-link" aria-label="Direct link to Plan-level replay" title="Direct link to Plan-level replay" translate="no">​</a></h3>
<p>This one is specific to agents. If an agent never learns that a step succeeded, it may mark the whole task as failed. Then the agent, or the user, starts the task again. Now every write in the plan can repeat, not just the one that timed out. One missed acknowledgment becomes a duplicate order, a duplicate email, and a duplicate refund. The fix has two parts: idempotent tools, plus keys tied to the task so that a resumed task reuses them. A genuinely new task should get new keys.</p>
<p>Plan-level replay is also the hardest to spot, because each request looks valid on its own. Finding it means grouping calls by task, which the gateway section below returns to.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="patterns-for-safe-retry">Patterns for safe retry<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#patterns-for-safe-retry" class="hash-link" aria-label="Direct link to Patterns for safe retry" title="Direct link to Patterns for safe retry" translate="no">​</a></h2>
<p>Idempotency keys are the baseline. The other four patterns here cover what keys can't.</p>
<ul>
<li class="">
<p><strong>Idempotency keys on non-idempotent writes:</strong> Any write that isn't naturally idempotent should require a key, and the server should replay the stored result for repeats. A full replace, such as a PUT that sets an address, can work without one. Make that call per operation in its contract, not by guessing which writes are low-risk.</p>
</li>
<li class="">
<p><strong>Exponential backoff with jitter:</strong> Stripe recommends waiting in proportion to 2^n after the nth failure. It also recommends random jitter, so that clients don't retry in lockstep. For agents, apply this in the runtime, not in the prompt. Cap the number of attempts, and don't retry errors that won't change, such as validation or permission failures. Server-side resilience patterns, such as circuit breakers, cover the other half of this problem.</p>
</li>
<li class="">
<p><strong>Dry-run operations for irreversible actions:</strong> A <code>POST /checkout-dry-run</code> endpoint returns the final effect, such as the total price and delivery date, without committing anything. Choreo Labs' design guidance uses it to close a gap agents create. A human reviews the final price before paying, but an agent tends to commit straight away.</p>
</li>
<li class="">
<p><strong>Two-step confirmation for high-stakes writes:</strong> Split "propose" from "commit." The first call creates a pending confirmation. Only a second, explicit call completes the order. This is an application-level confirmation flow, not distributed two-phase commit.</p>
<div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">POST /checkout</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">-&gt; 201 Created {"confirmation_id": "cnf_7Q2"}</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">POST /confirm-checkout/cnf_7Q2</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">-&gt; 200 OK {"order_id": "ord_91"}</span><br></span></code></pre></div></div>
<p>The first call isn't always harmless. It can reserve stock or hold funds, so give it an idempotency key too. A retry then returns the same <code>confirmation_id</code>. Bind each confirmation to the actor, the operation, its parameters and amount, an expiry, and a single use. Consume it atomically when the order completes, and make retries of the confirm call idempotent as well.</p>
</li>
<li class="">
<p><strong>Bulk endpoints to shrink the retry surface:</strong> An agent that calls a single-item endpoint once per item creates many chances for partial failure. A batch endpoint such as <code>POST /products-batch-get</code> turns many calls into one. Choreo Labs' guidance calls it likely one of the highest-return changes for agent traffic.</p>
</li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="a-pre-publish-checklist-for-tool-authors">A pre-publish checklist for tool authors<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#a-pre-publish-checklist-for-tool-authors" class="hash-link" aria-label="Direct link to A pre-publish checklist for tool authors" title="Direct link to A pre-publish checklist for tool authors" translate="no">​</a></h2>
<p>Before you expose a tool to agents, run it through these questions. Each one is phrased as a test, so it has a clear pass or fail.</p>
<ul>
<li class="">Does every write that isn't naturally idempotent reject requests that arrive without an idempotency key?</li>
<li class="">Does the agent runtime, not the model, create the key once per authorized operation, atomically, and store it before the first attempt?</li>
<li class="">Does a repeat with the same key return the original result, including after a crash and resume?</li>
<li class="">Do irreversible tools offer a dry-run or a two-step confirmation flow?</li>
<li class="">Does the retry loop back off with jitter and stop on errors that won't change?</li>
<li class="">Is each tool's effect profile written down, in its MCP annotations and in its API description? Choreo Labs' design guidance proposes an <code>x-ai-metadata</code> block for this.</li>
<li class="">Have you tested a semantically equivalent retry, such as the same refund with the amount formatted differently?</li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="making-idempotency-a-platform-guarantee">Making idempotency a platform guarantee<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#making-idempotency-a-platform-guarantee" class="hash-link" aria-label="Direct link to Making idempotency a platform guarantee" title="Direct link to Making idempotency a platform guarantee" translate="no">​</a></h2>
<p>Everything above works only if every team remembers to do it. Once several teams ship agent-facing tools, the checklist gets applied unevenly. That's when it pays to move some checks to the gateway, which sees every call.</p>
<p>On Choreo API Management, the API Gateway handles REST API traffic and the AI Gateway handles MCP and LLM traffic. Both are governed from one control plane. On the AI Gateway, keep the two flows distinct. Tool calls go through an MCP proxy to the MCP server and its APIs. The agent's own model calls go through an LLM proxy to the provider. The controls below apply to the tool-call path.</p>
<ul>
<li class=""><strong>Per-tool authorization and visibility:</strong> The AI Gateway's <a href="https://wso2.com/api-platform/docs/ai-gateway/next/mcp-governance/" target="_blank" rel="noopener noreferrer" class="">MCP governance policies</a> work at two different points. The MCP Authorization policy checks each tool call against JWT claims or OAuth scopes. An agent without the right scope for <code>issue_refund</code> gets a <code>403</code> when it calls the tool. The MCP Access Control List policy filters tools out of the tool list and also rejects direct calls to them. Use authorization to decide who can invoke a tool, and the access control list to decide which tools are exposed at all.</li>
<li class=""><strong>Per-tool rate limits:</strong> The MCP Rate Limit policy counts calls per tool. That lets you cap a retry storm against one irreversible tool without throttling reads.</li>
<li class=""><strong>Required idempotency keys:</strong> Gateway policies run as a chain on each request, and you can <a href="https://wso2.com/api-platform/docs/cloud/api-platform-gateway/writing-a-custom-policy/" target="_blank" rel="noopener noreferrer" class="">write your own</a>. On REST traffic, a custom policy attached to a write operation, such as <code>POST /refunds</code>, can check for an <code>Idempotency-Key</code> header and reject requests that lack one. MCP traffic is different. Every tool call is an HTTP POST to the same endpoint, so a header check alone can't tell a read tool from a write tool. A policy would need to read the tool name from the JSON-RPC body, which custom policies can do in the request-body phase. The MCP server also has to pass the key on to the service that performs the write. Even then, a presence check only confirms that a key exists. Deduplication still belongs in the service that owns the write. Stateful rules, such as requiring a recent dry-run, need a shared store and a deliberate design.</li>
<li class=""><strong>Retry visibility per task:</strong> Have the agent runtime send a task or session ID header with each call. Gateway logs and traces can then group calls by task. A rising count of retries per task for one tool is an early warning.</li>
</ul>
<p>Central enforcement earns its keep in three situations:</p>
<ul>
<li class="">Several teams publish agent-facing tools.</li>
<li class="">Those tools touch payments or regulated data.</li>
<li class="">You need to attribute retries to a specific agent.</li>
</ul>
<p>Before that point, a well-reviewed tool contract may be enough.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>Agents retry because networks fail, models are unsure, and long tasks resume. Idempotent tool design accepts that. Every non-idempotent write carries a key created by the runtime, every tool declares its effects, and irreversible actions get a dry-run or a confirmation step. A gateway then turns those rules from team habits into platform defaults. To see how the AI Gateway routes and governs MCP traffic, start with the <a href="https://wso2.com/api-platform/docs/ai-gateway/next/overview/" target="_blank" rel="noopener noreferrer" class="">AI Gateway documentation</a>.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-does-idempotency-mean-in-api-design">What does idempotency mean in API design?<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#what-does-idempotency-mean-in-api-design" class="hash-link" aria-label="Direct link to What does idempotency mean in API design?" title="Direct link to What does idempotency mean in API design?" translate="no">​</a></h3>
<p>An idempotent API operation produces the same server-side outcome whether a client sends it once or several times. GET, PUT, and DELETE have this property by HTTP definition. A POST endpoint usually needs an idempotency key to get it.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-an-example-of-an-idempotent-operation">What is an example of an idempotent operation?<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#what-is-an-example-of-an-idempotent-operation" class="hash-link" aria-label="Direct link to What is an example of an idempotent operation?" title="Direct link to What is an example of an idempotent operation?" translate="no">​</a></h3>
<p>Setting a shipping address to a specific value is idempotent, because repeating it changes nothing. Adding 10 credits to an account is not, because each repeat adds 10 more.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="should-ai-agents-retry-failed-tool-calls-automatically">Should AI agents retry failed tool calls automatically?<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#should-ai-agents-retry-failed-tool-calls-automatically" class="hash-link" aria-label="Direct link to Should AI agents retry failed tool calls automatically?" title="Direct link to Should AI agents retry failed tool calls automatically?" translate="no">​</a></h3>
<p>Reads can be retried with backoff and a retry cap. Writes should be retried only when the tool is idempotent, or when the runtime reuses the same idempotency key for every attempt.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="do-mcp-tool-annotations-make-a-tool-idempotent">Do MCP tool annotations make a tool idempotent?<a href="https://choreolabs.com/api-management/learn/designing-idempotent-tools#do-mcp-tool-annotations-make-a-tool-idempotent" class="hash-link" aria-label="Direct link to Do MCP tool annotations make a tool idempotent?" title="Direct link to Do MCP tool annotations make a tool idempotent?" translate="no">​</a></h3>
<p>No. The <code>idempotentHint</code> annotation only describes intended behavior, and the MCP specification tells clients to treat annotations as untrusted unless the server is trusted. The server still has to detect and absorb repeats.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="API architecture" term="API architecture"/>
        <category label="AI gateway" term="AI gateway"/>
        <category label="Agent-ready APIs" term="Agent-ready APIs"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[Agent platform architecture: Orchestration to observability]]></title>
        <id>https://choreolabs.com/api-management/learn/agent-platform-architecture</id>
        <link href="https://choreolabs.com/api-management/learn/agent-platform-architecture"/>
        <updated>2026-10-13T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[A practical guide to agent platform architecture: orchestration patterns, task routing, memory, and observability for running AI agents safely at scale.]]></summary>
        <content type="html"><![CDATA[<p>The first agent a team ships usually works. It runs on one framework, calls one model, and uses a handful of tools that its developer wired up by hand. Then a second team builds an agent on a different framework. A third team wants two agents to hand work to each other. Within a quarter, nobody can say which agents are running, which models they call, or what data they touched yesterday.</p>
<p>That is the point where agent platform architecture becomes its own discipline. In 2026 the building blocks are maturing quickly: MCP for tool access, A2A for agent-to-agent calls, and OpenTelemetry for tracing. The architecture that ties them together is still settling. This guide maps that architecture using three planes (build, orchestrate, and control). It then works through the layers each plane depends on: orchestration patterns, task routing, memory, observability, and identity.</p>
<aside class="cta_6F_m" aria-label="Build an agent platform that scales" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Build an agent platform that scales</span><span class="desc_GzK8">Choreo governs your build, orchestrate, and control planes across any gateway or cloud.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=agent-platform-architecture" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="why-most-agent-platforms-never-leave-proof-of-concept">Why most agent platforms never leave proof of concept<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#why-most-agent-platforms-never-leave-proof-of-concept" class="hash-link" aria-label="Direct link to Why most agent platforms never leave proof of concept" title="Direct link to Why most agent platforms never leave proof of concept" translate="no">​</a></h2>
<p>Agents combine non-deterministic behavior with broad access to real systems, and that combination is hard to carry from pilot to production. Pre-agentic platforms were not designed to handle both at once. A demo can tolerate that combination. Production cannot, because it needs identity, policy, tracing, and evaluation around every agent.</p>
<p>In a June 2025 forecast, <a href="https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027" target="_blank" rel="noopener noreferrer" class="">Gartner</a> predicts that over 40% of agentic AI projects will be canceled by the end of 2027. The reasons it gives are escalating costs, unclear business value, and inadequate risk controls. The same release notes that most projects at the time were early experiments or proofs of concept. This is a forecast, not a measured failure rate, and it does not name architecture as the cause. This guide treats those three risks as design inputs. Cost visibility, evaluation, and runtime risk controls are platform concerns, and the sections below cover each one.</p>
<p>Part of the response is architectural. Existing distributed applications already need runtime controls such as authorization, monitoring, concurrency control, and incident response. Agents inherit all of that and add variable action selection, plus model and tool behavior that can change between runs. An agent decides its own next step at runtime. It may call a tool nobody expected, or pass a customer record to a model in another region. When that agent also holds credentials to production systems, the platform has to answer questions that pre-release testing alone cannot answer. Who is this agent? What is it allowed to do? What did it actually do, and was the result any good?</p>
<p>Teams that skip those questions rarely fail at the model layer. They fail when a security review asks for an audit trail, or when finance asks which team spent the inference budget. The APIs underneath matter as well, because they are <a class="" href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis">the API layer these agents ultimately call</a>. Governing agents while exposing poorly scoped APIs only moves the risk.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-three-planes-of-an-agent-platform-build-orchestrate-control">The three planes of an agent platform: Build, orchestrate, control<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#the-three-planes-of-an-agent-platform-build-orchestrate-control" class="hash-link" aria-label="Direct link to The three planes of an agent platform: Build, orchestrate, control" title="Direct link to The three planes of an agent platform: Build, orchestrate, control" translate="no">​</a></h2>
<p><a href="https://www.forrester.com/blogs/announcing-our-evaluation-of-the-agent-control-plane-market/" target="_blank" rel="noopener noreferrer" class="">Forrester's three-plane model</a> splits the enterprise agentic stack into build, orchestrate, and control. It is an analyst reference model, not a formal interoperability standard. It is a useful map because each plane answers a different question, and each tends to be owned by a different team.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="build-plane">Build plane<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#build-plane" class="hash-link" aria-label="Direct link to Build plane" title="Direct link to Build plane" translate="no">​</a></h3>
<p>This is where agents are created. It covers model access, agent frameworks such as LangGraph or CrewAI, tool integrations, prompts, and evaluation pipelines. Developers own it, and most enterprises will run more than one framework here.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="orchestrate-plane">Orchestrate plane<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#orchestrate-plane" class="hash-link" aria-label="Direct link to Orchestrate plane" title="Direct link to Orchestrate plane" translate="no">​</a></h3>
<p>This is where agents are embedded into business processes and workflows, alongside non-agentic services. Multi-agent coordination lives here. So do long running workflows, event driven triggers, and the points where a human review step is required.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="control-plane">Control plane<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#control-plane" class="hash-link" aria-label="Direct link to Control plane" title="Direct link to Control plane" translate="no">​</a></h3>
<p>This is where agents are inventoried, governed, and observed. Forrester's argument is that governance has to sit outside the other two planes. Only then can it provide independent visibility and consistent policy when runtimes behave unpredictably.</p>
<p>That last point has a direct consequence for traffic design. The control plane is not a hop that every request passes through. It sets identity, policy, and budgets, then receives telemetry back. Enforcement happens in the data plane, usually at a gateway that sits on each traffic path. Being outside the request path does not remove runtime dependencies. Gateways still rely on the control plane for credentials, token validation, and current policy. Its availability and policy freshness therefore still affect live traffic.</p>
<p><img decoding="async" loading="lazy" alt="Agent platform reference architecture. The control plane at the top holds inventory and lifecycle, agent identity, policy and guardrails, and tracing and evaluation, outside the request path. The orchestrate plane in the middle hosts agents A, B, and C with a dev, staging, production lifecycle. An AI Gateway governs inbound callers on the left, and an AI Gateway on the right governs outbound LLM calls to LLM providers and outbound MCP calls to tools and data. The build plane at the bottom covers any framework, such as LangChain, CrewAI, AWS Strands, and Microsoft Agent Framework" src="https://choreolabs.com/api-management/learn/assets/images/three-planes-agent-platform-09a3244968ae6123b405ef4e04dddd51.webp" width="3200" height="1788" class="img_ev3q"></p>
<p><em>The control plane pushes policy and credentials down and pulls telemetry back up. External agents are governed only on calls routed through AI Gateway, and traced once instrumented.</em></p>
<p>The diagram above separates three flows. Inbound calls arrive from users, applications, or other agents. Outbound LLM calls and outbound tool calls leave the agent on separate routes, each with its own policy. Separate routes are not functionally independent, though. An agent often needs a model response before choosing a tool, so a slow or failed LLM call can stall the tool path too. The control plane pushes policy and credentials down to the gateway and agents, and pulls traces and metrics back up.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="orchestration-patterns-how-multiple-agents-actually-coordinate">Orchestration patterns: How multiple agents actually coordinate<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#orchestration-patterns-how-multiple-agents-actually-coordinate" class="hash-link" aria-label="Direct link to Orchestration patterns: How multiple agents actually coordinate" title="Direct link to Orchestration patterns: How multiple agents actually coordinate" translate="no">​</a></h2>
<p>The first architectural decision is whether you need a multi-agent system at all. A single agent with well-designed tools is easier to test, trace, and secure. Teams usually move to specialized agents for one of three reasons. The task spans domains with different data access rules. Prompts and tool lists grow too large for one agent to handle reliably. Or different steps need different models.</p>
<p>Once you split work across agents, the coordination pattern determines how failures spread and how hard the system is to debug.</p>
<table><thead><tr><th>Pattern</th><th>How It Works</th><th>Best For</th><th>Key Risk</th></tr></thead><tbody><tr><td>Sequential</td><td>Each agent processes the output of the previous one in a fixed order</td><td>Pipelines with clear stages, such as extract, enrich, then summarize</td><td>One bad output propagates through every later stage</td></tr><tr><td>Concurrent</td><td>Several agents work on the same input in parallel, and results are merged</td><td>Independent analyses that a final step combines</td><td>Conflict resolution between contradictory results</td></tr><tr><td>Handoff</td><td>An agent transfers control of the conversation to a more suitable agent</td><td>Triage flows, such as routing a support request to billing</td><td>Lost context at each transfer</td></tr><tr><td>Group chat</td><td>Agents share one conversation thread and take turns contributing</td><td>Review and debate tasks with a human moderator</td><td>Unbounded turns and rising token cost</td></tr><tr><td>Hierarchical (supervisor)</td><td>A supervisor agent plans, delegates subtasks to workers, and assembles the result</td><td>Complex tasks with decomposable subtasks</td><td>The supervisor becomes a single point of failure and a bottleneck</td></tr><tr><td>Peer-to-peer</td><td>Agents call each other directly without a central coordinator</td><td>Cross-team or cross-organization agents that each own a domain</td><td>Hard to trace, and loops are easy to create</td></tr></tbody></table>
<p><img decoding="async" loading="lazy" alt="Three coordination patterns side by side. Sequential: input enters at agent A, control flows to B and then C in a fixed order, and one bad output reaches every stage. Hierarchical: a supervisor S delegates subtasks to agents A, B, and C and collects results, and the supervisor becomes a bottleneck. Peer-to-peer: agents A, B, and C call each other directly with no central coordinator, using discovery via A2A, which is hard to trace and makes loops easy to form" src="https://choreolabs.com/api-management/learn/assets/images/orchestration-patterns-f3f57eb3c4c61be4c609d2f0260496cf.webp" width="3200" height="1278" class="img_ev3q"></p>
<p><em>Three of the six coordination patterns, with the risk each one carries.</em></p>
<p>Most production systems combine patterns. A supervisor might run a sequential pipeline for one subtask and fan out concurrently for another. For a worked example of the supervisor pattern, see the <a class="" href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study">multi-agent calendar assistant case study</a>. Peer-to-peer coordination is becoming more practical as agents publish their capabilities through <a href="https://a2a-protocol.org/latest/specification/" target="_blank" rel="noopener noreferrer" class="">A2A Agent Cards</a>. That makes discovery easier. It also widens the trust boundary, because an agent can now receive work from agents it has never seen before. Pick the simplest pattern that fits the task, and add coordination only when a measured problem requires it.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="task-routing-and-the-control-plane">Task routing and the control plane<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#task-routing-and-the-control-plane" class="hash-link" aria-label="Direct link to Task routing and the control plane" title="Direct link to Task routing and the control plane" translate="no">​</a></h2>
<p>Routing in an agent platform happens at two levels, and conflating them causes trouble.</p>
<p>The first level is agent selection. Task intake decides which agent, or which workflow, handles a request. In a hierarchical system the supervisor makes that decision. In a handoff system the current agent does. This logic belongs to the orchestrate plane, because it encodes business rules about who does what.</p>
<p>The second level is model and tool selection. Once an agent is running, it calls models and tools. Which model it may call, at what cost, and with what data controls is a policy decision. That decision should not live inside each agent's code. If it does, every framework team reimplements budgets, fallbacks, and data rules differently. A change in provider then means a change in every agent.</p>
<p>The cleaner design keeps routing policy in the control plane and enforces it at a gateway on the outbound LLM path. The agent asks for a capability. The gateway picks the model based on cost, latency, capability, or data residency. <a class="" href="https://choreolabs.com/api-management/learn/llm-routing">LLM routing</a> covers those strategies in depth.</p>
<p>Centralizing this is not always worth it on day one. The useful triggers are qualitative:</p>
<ul>
<li class=""><strong>Multi-model usage:</strong> More than one provider or model family is in play, and you want failover or capability-based routing without code changes.</li>
<li class=""><strong>Spend attribution:</strong> Finance needs inference cost broken down by team, agent, or product.</li>
<li class=""><strong>Regulated data in prompts:</strong> Prompts or tool results contain personal, financial, or health data that needs masking or regional routing.</li>
<li class=""><strong>Production scale:</strong> Several teams ship agents on different frameworks, and policy drift between them has become a real risk.</li>
</ul>
<p>Conflict resolution also belongs in this discussion. When two agents produce contradictory outputs, decide in advance whether a supervisor, a deterministic rule, or a human makes the call. Leaving it to whichever agent answers last is not a policy.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="state-memory-and-context-across-agent-sessions">State, memory, and context across agent sessions<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#state-memory-and-context-across-agent-sessions" class="hash-link" aria-label="Direct link to State, memory, and context across agent sessions" title="Direct link to State, memory, and context across agent sessions" translate="no">​</a></h2>
<p>Memory architecture is where agent systems most often drift from correct behavior without anyone noticing. It helps to separate three kinds of state.</p>
<ul>
<li class="">Short term memory is the working context of a single session: conversation history, intermediate tool results, and the current plan. It is bounded by the model's context window, so long sessions need summarization or truncation. Each of those choices drops information.</li>
<li class="">Long term memory persists across sessions, often as user preferences or documents in a vector store. It makes agents more useful over time, and it turns every write into a potential security event.</li>
<li class="">Shared state is what multiple agents read and write together, such as a task board or a common case record. State management here needs the same discipline as any distributed system: clear ownership, versioning, and concurrency control.</li>
</ul>
<p>For the memory types and storage choices in more depth, see <a class="" href="https://choreolabs.com/api-management/learn/ai-agent-memory">AI agent memory</a>.</p>
<p>The security side of memory is now well documented. The OWASP Top 10 for Agentic Applications lists Memory and Context Poisoning as a distinct risk. An attacker plants misleading content in memory or a retrieval store, and the agent acts on it in a later session. Context pollution is the non-malicious version of the same problem. Stale or irrelevant context gradually degrades agent behavior.</p>
<p>The controls are easy to state and hard to retrofit:</p>
<ul>
<li class="">Scope memory by user, tenant, and task, so one session cannot read another's context.</li>
<li class="">Record the source of every memory write, so suspicious entries can be traced and rolled back.</li>
<li class="">Set expiry on long term memory instead of keeping everything indefinitely.</li>
<li class="">Treat retrieved memory as untrusted input, and apply the same guardrails you apply to user prompts.</li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="observability-and-evaluation-seeing-what-agents-actually-do">Observability and evaluation: Seeing what agents actually do<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#observability-and-evaluation-seeing-what-agents-actually-do" class="hash-link" aria-label="Direct link to Observability and evaluation: Seeing what agents actually do" title="Direct link to Observability and evaluation: Seeing what agents actually do" translate="no">​</a></h2>
<p>Observability is a required platform layer, not a debugging convenience. An agent's behavior is the sequence of decisions it made, so request logs alone are not enough. You need the full trace from the inbound request, through each model call, to each tool invocation.</p>
<p>OpenTelemetry is the common foundation. Its <a href="https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-agent-spans" target="_blank" rel="noopener noreferrer" class="">GenAI semantic conventions</a> define agent operations such as <code>invoke_agent</code> and <code>execute_tool</code>. Traces from different frameworks can then be read in one place. The conventions are still maturing, so expect some framework-specific attributes for now.</p>
<p>Traces show what an agent did. Evaluation shows whether it did it well. A production setup typically runs evaluations in two places. Offline evaluations run in CI/CD against curated test sets before a new prompt or model version ships. Online evaluations score sampled production traffic against criteria such as task completion, groundedness, or policy compliance. Both feed the same question: is this agent version good enough to keep running?</p>
<p>Guardrails sit next to this layer. They act on individual requests in real time, while evaluations judge behavior across many requests. Root cause analysis needs both. A guardrail block tells you a request was stopped. The trace and evaluation history tell you why the agent produced it in the first place. For the agent side of this telemetry, see <a class="" href="https://choreolabs.com/api-management/learn/agent-observability">agent observability</a>. For the gateway side, see <a class="" href="https://choreolabs.com/api-management/learn/ai-gateway-observability">AI gateway observability</a>.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="security-identity-and-governance-for-agent-platforms">Security, identity, and governance for agent platforms<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#security-identity-and-governance-for-agent-platforms" class="hash-link" aria-label="Direct link to Security, identity, and governance for agent platforms" title="Direct link to Security, identity, and governance for agent platforms" translate="no">​</a></h2>
<p>The central security question for an agent platform is blast radius. If an agent is manipulated through prompt injection or a poisoned tool description, how much damage can it do? Identity and permission design narrow it most directly. Egress controls, data permissions, business limits such as refund caps, and approvals bound to a specific action also shape it.</p>
<p>Start with identity. Many early agents run on a shared service account or on a user's borrowed token. Both make it hard to tell what the agent did versus what the person did. The <a href="https://openid.net/new-whitepaper-tackles-ai-agent-identity-challenges/" target="_blank" rel="noopener noreferrer" class="">OpenID Foundation whitepaper on identity management for agentic AI</a> recommends treating agents as first-class citizens in identity and access management. That means each agent gets its own lifecycle, governance policies, and accountability. One of its co-authors, Ayesha Dissanayaka, is Lead Architect for Agent Identity at Choreo Labs.</p>
<p>With a distinct identity in place, least-privilege agent design becomes enforceable. A narrow credential still authorizes any harmful action inside its scope, so combine it with the controls below. <a class="" href="https://choreolabs.com/api-management/learn/agent-api-permissions">Permission models for AI agents</a> covers them in more detail.</p>
<ul>
<li class="">Scope by task, not by agent. An agent that drafts refunds should not hold the permission to issue them. Put a human approval step or a separate agent with narrower rights in between. Bind each approval to the specific action and amount so it cannot be reused for another request.</li>
<li class="">Separate inbound and outbound credentials. The credential a client uses to call the agent is not the credential the agent uses to call tools. Mixing them creates confused-deputy problems.</li>
<li class="">Use short-lived tokens with an explicit revocation path. A short lifetime limits how long a stale or stolen token works, but it does not reject the next request on its own. If suspension must take effect on the next request, the gateway needs token introspection, a denylist, or a fresh policy check on every call. Otherwise, document the maximum remaining validity window.</li>
<li class="">Authorize per tool. Role based access control at the tool level limits what a hijacked agent can reach.</li>
</ul>
<p>The <a href="https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/" target="_blank" rel="noopener noreferrer" class="">OWASP Top 10 for Agentic Applications</a> is a sound baseline for threat modeling. It was developed with more than 100 industry experts and covers risks such as Agent Goal Hijack, Identity and Privilege Abuse, and Cascading Failures. Tool-level threats, such as poisoned tool descriptions, deserve a separate review of their own.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-production-teams-get-wrong-and-a-framework-for-getting-it-right">What production teams get wrong (and a framework for getting it right)<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#what-production-teams-get-wrong-and-a-framework-for-getting-it-right" class="hash-link" aria-label="Direct link to What production teams get wrong (and a framework for getting it right)" title="Direct link to What production teams get wrong (and a framework for getting it right)" translate="no">​</a></h2>
<p>Production failures in agent systems look similar across industries and frameworks. Four come up repeatedly.</p>
<p><strong>Context drift:</strong> Agents degrade over long sessions as summaries lose detail and stale memory builds up. The answers stay confident, so the drift goes unnoticed until a user complains.</p>
<p><strong>Unreliable tool invocation:</strong> Agents retry failed tool calls, sometimes with slightly different arguments. Without <a class="" href="https://choreolabs.com/api-management/learn/designing-idempotent-tools">idempotent tool design</a>, a retry can create a duplicate order or send a second email. Error handling has to assume that any call may run twice.</p>
<p><strong>Fragmented observability:</strong> Each framework ships its own tracing format, so incidents that cross two agents need two tools and manual correlation.</p>
<p><strong>Rework on every new workflow:</strong> Each team rebuilds authentication, guardrails, and cost tracking for its own agent, and the policies drift apart.</p>
<p>These failures share a cause: platform concerns were built inside individual agents. A short checklist exposes that early. For any agent platform, current or proposed, check whether you can answer yes to each question:</p>
<ol>
<li class="">Can you list every agent in production, its owner, and its version?</li>
<li class="">Does each agent have its own identity, separate from users and service accounts?</li>
<li class="">Are model, tool, and budget policies enforced outside the agent's code?</li>
<li class="">Can you trace one request across every agent, model call, and tool call it touched?</li>
<li class="">Do new agent versions pass defined evaluations before promotion?</li>
<li class="">Can you suspend a single agent immediately without redeploying others?</li>
<li class="">Are tools safe to retry, and are side-effecting actions gated by human review where needed?</li>
<li class="">Can any single agent or component fail without stopping every workflow?</li>
</ol>
<p>A "no" on any of these is a production readiness gap, and it is cheaper to find during design than after an incident.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-choreo-approaches-agent-platform-architecture">How Choreo approaches agent platform architecture<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#how-choreo-approaches-agent-platform-architecture" class="hash-link" aria-label="Direct link to How Choreo approaches agent platform architecture" title="Direct link to How Choreo approaches agent platform architecture" translate="no">​</a></h2>
<p>Choreo follows the separation described above. Teams build agents with the framework that suits them, and the platform provides one control plane plus gateway enforcement on each traffic path. This is one way to implement the three-plane model. Framework-native runtimes such as LangGraph and CrewAI, and cloud services such as Amazon Bedrock AgentCore, are other options. The right fit depends on how many frameworks and clouds you need to govern.</p>
<p>The center of the Choreo agent platform is Choreo Agent Manager, an open source control plane for AI agents. It supports agents built with LangChain, CrewAI, AWS Strands, and Microsoft Agent Framework. It can also register agents already running in Bedrock, Azure, or LangSmith. Registration adds an external agent to the inventory, but it does not route that agent's calls through the platform. Traces appear once the agent is instrumented, and gateway policy applies only to calls the agent sends through AI Gateway. AI Gateway and Agent ID are built into it, so identity, guardrails, and observability are managed in one system.</p>
<table><thead><tr><th>Plane</th><th>Choreo capability</th><th>What it covers</th></tr></thead><tbody><tr><td>Build</td><td>Any agent framework, or Choreo Integrator for low-code and pro-code agents</td><td>Agent logic, tools, prompts, and connectors</td></tr><tr><td>Orchestrate</td><td>Workflow logic in your agent framework (such as LangGraph), application code, or Choreo Integrator. Agent Manager hosts the agents, and AI Gateway governs A2A calls</td><td>Workflow state and routing decisions, plus agent hosting and A2A call governance</td></tr><tr><td>Control</td><td>Agent Manager, with AI Gateway and Agent ID built in</td><td>Inventory, lifecycle, identity, guardrails, tracing, and evaluation</td></tr></tbody></table>
<p>A few capabilities map directly to the sections above:</p>
<ul>
<li class="">Traffic enforcement. Choreo AI Gateway governs LLM, MCP, and A2A traffic. On the outbound LLM path it handles multi-model routing, failover, and cost budgets that reject traffic once a budget is spent. On the outbound tool path, its <a href="https://wso2.com/api-platform/docs/ai-gateway/next/mcp-governance/" target="_blank" rel="noopener noreferrer" class="">MCP governance policies</a> authenticate callers, authorize per tool, restrict the tool list, and rate limit calls. For inbound agent-to-agent calls, it publishes a governed Agent Card and applies authentication and rate limiting.</li>
<li class="">Guardrails at several levels. Agent Manager applies guardrails at the organization, agent, MCP, and LLM levels, mapped to the OWASP Top 10.</li>
<li class="">Agent identity. <a href="https://wso2.com/identity-platform/agent-id/" target="_blank" rel="noopener noreferrer" class="">Choreo Agent ID</a> registers, authenticates, authorizes, and audits agents as first-class identities, with delegation policies and revocation. The identity an agent presents on outbound calls is separate from the controls that protect calls into it.</li>
<li class="">Tracing and evaluation. Agent Manager observability is built on OpenTelemetry. It covers agents deployed through the platform and agents hosted elsewhere. Built-in and custom evaluations score agent behavior over time.</li>
<li class="">Lifecycle control. Each agent gets a versioned path from development to staging to production. Any single agent can be suspended.</li>
</ul>
<p>The open source Agent Manager project is licensed under Apache 2.0, per the LICENSE file in the <a href="https://github.com/wso2/agent-manager" target="_blank" rel="noopener noreferrer" class="">wso2/agent-manager repository</a> at release amp/v1.0.0 (September 2026). Its README states that agents deployed through the platform run on OpenChoreo and that instrumentation uses OpenTelemetry. You can self-host that release or use Choreo Labs' hosted SaaS version. The Apache 2.0 license covers the open source code, not the hosted service.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>Agent platform architecture is a separate discipline from application platform engineering. Agents combine autonomy with broad access, and deterministic platforms were not built for that combination. The three-plane model gives that work a structure. Let teams build on the frameworks they prefer, and orchestrate agents with the simplest pattern that fits. Keep identity, policy, tracing, and evaluation in a control plane that sits outside both, with enforcement on each traffic path.</p>
<p>If your agents are moving from framework-level experiments toward governed production, <a href="https://wso2.com/agent-platform/agent-manager/" target="_blank" rel="noopener noreferrer" class="">Choreo Agent Manager</a> provides that control plane across frameworks and deployments. A free trial is available.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-agent-platform-architecture">What is agent platform architecture?<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#what-is-agent-platform-architecture" class="hash-link" aria-label="Direct link to What is agent platform architecture?" title="Direct link to What is agent platform architecture?" translate="no">​</a></h3>
<p>Agent platform architecture is the set of shared services an organization uses to build, run, and govern AI agents. It typically covers an agent runtime, orchestration, task routing, memory, observability, identity, and policy enforcement.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-an-agent-control-plane">What is an agent control plane?<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#what-is-an-agent-control-plane" class="hash-link" aria-label="Direct link to What is an agent control plane?" title="Direct link to What is an agent control plane?" translate="no">​</a></h3>
<p>An agent control plane is the governance layer for an organization's agents. It handles inventory, identity, policy, tracing, and evaluation. It sits outside the request path and pushes policy to gateways that enforce it.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="when-should-you-use-a-multi-agent-system-instead-of-a-single-agent">When should you use a multi-agent system instead of a single agent?<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#when-should-you-use-a-multi-agent-system-instead-of-a-single-agent" class="hash-link" aria-label="Direct link to When should you use a multi-agent system instead of a single agent?" title="Direct link to When should you use a multi-agent system instead of a single agent?" translate="no">​</a></h3>
<p>Use multiple agents when a task spans domains with different data access rules or needs a different model per step. If one agent can handle the task reliably, keep it single, because it is easier to secure and debug.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="when-does-a-team-need-a-dedicated-agent-platform">When does a team need a dedicated agent platform?<a href="https://choreolabs.com/api-management/learn/agent-platform-architecture#when-does-a-team-need-a-dedicated-agent-platform" class="hash-link" aria-label="Direct link to When does a team need a dedicated agent platform?" title="Direct link to When does a team need a dedicated agent platform?" translate="no">​</a></h3>
<p>The usual triggers are operational. Several teams ship agents on different frameworks, finance needs costs by team, or prompts carry regulated data. Moving agents into production at scale is the other common trigger.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="AI agents" term="AI agents"/>
        <category label="Agent platform" term="Agent platform"/>
        <category label="Agent-ready APIs" term="Agent-ready APIs"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[REST API vs MCP: Which do AI agents actually need?]]></title>
        <id>https://choreolabs.com/api-management/learn/rest-apis-vs-mcp</id>
        <link href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp"/>
        <updated>2026-10-13T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[REST and MCP solve different problems for AI agents. Compare discovery, state, and auth, and see how to govern both from one control plane.]]></summary>
        <content type="html"><![CDATA[<p>Hand an AI agent the OpenAPI description of your order service and ask whether a customer qualifies for a refund. It has to pick the right endpoints out of dozens, call them in the right order, and stitch the answers together. A developer would read the docs once and write that logic. If the integration hands the agent raw endpoints rather than a few task-level tools, it weighs many candidate calls every time it gets the task, and every wrong pick costs a round trip and more context.</p>
<p>The Model Context Protocol (MCP) exists to close that gap. That is also why "REST API vs MCP" is the wrong framing for most teams. This guide covers what each does well and where REST falls short for agents. It then walks through MCP discovery and authorization in the current spec, and how to run both without two governance stacks. It is part of the Choreo Labs series on <a class="" href="https://choreolabs.com/api-management/learn/agent-ready-apis">agent-ready APIs</a>.</p>
<aside class="cta_6F_m" aria-label="Govern REST and MCP from one place" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Govern REST and MCP from one place</span><span class="desc_GzK8">Use Choreo to govern both REST APIs and MCP tools from a single control plane.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=rest-apis-vs-mcp" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="rest-and-mcp-arent-competing-standards-they-solve-different-problems">REST and MCP aren't competing standards. they solve different problems.<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#rest-and-mcp-arent-competing-standards-they-solve-different-problems" class="hash-link" aria-label="Direct link to REST and MCP aren't competing standards. they solve different problems." title="Direct link to REST and MCP aren't competing standards. they solve different problems." translate="no">​</a></h2>
<p>REST is a general-purpose architectural style that exposes resource-oriented interfaces over HTTP. MCP is a protocol that standardizes how compatible AI hosts list and invoke capabilities at runtime. In most deployments, an MCP server sits alongside an existing REST API and calls it, rather than replacing it.</p>
<p>Put another way, REST describes your system to a developer, and MCP describes a curated set of capabilities to a model. The REST API keeps serving web, mobile, and partner clients, and the MCP server becomes a second, agent-facing interface over the same business logic.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-rest-apis-do-well">What REST APIs do well<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#what-rest-apis-do-well" class="hash-link" aria-label="Direct link to What REST APIs do well" title="Direct link to What REST APIs do well" translate="no">​</a></h2>
<p>REST earned its place through constraints that make large systems predictable. Roy Fielding's stateless constraint requires each request to carry all the information needed to understand it (<a href="https://ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm" target="_blank" rel="noopener noreferrer" class="">Fielding, section 5.1.3</a>). That single rule is a large part of why REST services scale behind ordinary load balancers and caches.</p>
<p>Decades of tooling sit on that foundation. The <a href="https://spec.openapis.org/oas/latest.html" target="_blank" rel="noopener noreferrer" class="">OpenAPI Specification</a> gives humans and machines a standard description of an HTTP API's capabilities. API gateways, SDK generators, mock servers, and contract tests all build on it.</p>
<p>Most REST integrations are built for a consumer who knows what it wants. A mobile developer reads the docs, picks <code>GET /orders/{id}</code>, and hard-codes the call. In that pattern, the interface rarely has to explain itself at runtime because a person already did that work. For clients like that, REST is hard to beat, and the arrival of AI agents doesn't change it.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="where-rest-falls-short-for-ai-agents">Where REST falls short for AI agents<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#where-rest-falls-short-for-ai-agents" class="hash-link" aria-label="Direct link to Where REST falls short for AI agents" title="Direct link to Where REST falls short for AI agents" translate="no">​</a></h2>
<p>The friction starts when the consumer is a model that has to work out what to call. Four gaps show up repeatedly.</p>
<ul>
<li class=""><strong>No shared capability-listing call:</strong> REST supports self-descriptive messages and hypermedia links, and an OpenAPI description can be fetched, filtered, and cached at runtime. What REST lacks is one agreed call that every agent host uses to list capabilities for the current user. Each agent integration picks its own approach, such as OpenAPI, hypermedia, curated tool definitions, or a wrapper.</li>
<li class=""><strong>Every integration is bespoke:</strong> Each API brings its own auth scheme, pagination style, and error format. Connecting several agents to several APIs means one adapter per pairing, which practitioners call the M×N problem.</li>
<li class=""><strong>Endpoint granularity doesn't match tasks:</strong> REST resources are shaped for data access. Answering "can this customer get a refund?" might need an order lookup, a customer lookup, and a policy check. If each endpoint becomes its own tool, the model has to weigh every candidate definition and plan the sequence itself, which costs tokens and adds chances to choose wrongly.</li>
<li class=""><strong>Authorization varies per API:</strong> One service takes an API key, another expects a client-credentials token, and a third uses mTLS. Recording which agent acted on whose behalf is possible over plain HTTP, for example with the actor (<code>act</code>) claim from OAuth token exchange (RFC 8693). Whether it happens depends on each API's identity design and token validation, so results vary.</li>
</ul>
<p>Statelessness is missing from that list on purpose. Earlier MCP revisions held a protocol session open, which made state look like MCP's advantage over REST. The current specification dropped protocol sessions as well.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-mcp-adds">What MCP adds<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#what-mcp-adds" class="hash-link" aria-label="Direct link to What MCP adds" title="Direct link to What MCP adds" translate="no">​</a></h2>
<p>MCP is an open protocol, introduced by <a href="https://www.anthropic.com/news/model-context-protocol" target="_blank" rel="noopener noreferrer" class="">Anthropic in November 2024</a>, that standardizes how AI applications connect to external tools and data. It uses JSON-RPC 2.0 messages between three roles: the host (the AI application), the client (a connector inside the host), and the server (the service exposing capabilities).</p>
<p>The common shorthand is "USB-C for AI": one connector shape that works across many devices. The analogy holds only so far. A shared protocol turns M×N custom adapters into roughly M clients plus N servers, provided both sides support compatible versions, transports, capabilities, and authorization. Each server still needs backend wrappers and domain logic.</p>
<p>The biggest recent change concerns state. The <a href="https://blog.modelcontextprotocol.io/posts/2026-07-28/" target="_blank" rel="noopener noreferrer" class="">2026-07-28 specification</a> retired the <code>initialize</code> handshake and the <code>Mcp-Session-Id</code> header, which makes MCP a stateless request/response protocol at the protocol layer. A server that needs continuity across calls mints an explicit handle and returns it from a tool, and the model passes that handle back as an argument. Requests over streamable HTTP also carry an <code>Mcp-Method</code> header. Tool calls, prompt fetches, and resource reads add an <code>Mcp-Name</code> header. A gateway can use these to route and rate-limit without parsing JSON bodies.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-three-mcp-primitives">The three MCP primitives<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#the-three-mcp-primitives" class="hash-link" aria-label="Direct link to The three MCP primitives" title="Direct link to The three MCP primitives" translate="no">​</a></h3>
<p>An MCP server exposes three kinds of capability. What separates them is who decides when each one is used.</p>
<ol>
<li class=""><strong>Tools are model-controlled:</strong> The model discovers and invokes them based on the task and the user's prompt. Actions with side effects, such as <code>create_ticket</code> or <code>issue_refund</code>, belong here.</li>
<li class=""><strong>Resources are application-driven:</strong> The host application decides how to bring them into context. A database schema, a refund policy document, or a file the user attached are typical examples. They supply context rather than perform actions.</li>
<li class=""><strong>Prompts are user-controlled:</strong> They are templates the user selects explicitly, usually through a command in the host's interface. A "summarize this incident" prompt packages instructions the user would otherwise retype.</li>
</ol>
<p>This split matters for governance. A tool that issues a refund deserves a stricter policy than a resource that returns the refund policy. A REST API has no standard way to express that difference to its caller.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-discovery-actually-works">How discovery actually works<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#how-discovery-actually-works" class="hash-link" aria-label="Direct link to How discovery actually works" title="Direct link to How discovery actually works" translate="no">​</a></h2>
<p>In a typical REST integration, discovery happens at design time, when a developer reads the OpenAPI description and writes code against it. Agent integrations can fetch OpenAPI or follow hypermedia at runtime too, but each one does it its own way. With MCP, discovery happens at runtime, through the same server the agent uses to act.</p>
<p>The client sends <code>tools/list</code>. The server responds with the tools available to that caller, each with a name, a description, and a JSON Schema <code>inputSchema</code>.</p>
<p>Request:</p>
<div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"> </span><span class="token property" style="color:#12C2FF">"jsonrpc"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"2.0"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token property" style="color:#12C2FF">"id"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">1</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token property" style="color:#12C2FF">"method"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"tools/list"</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">}</span><br></span></code></pre></div></div>
<p>Response:</p>
<div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"jsonrpc"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"2.0"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"id"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">1</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"result"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"tools"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token property" style="color:#12C2FF">"name"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"check_refund_eligibility"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token property" style="color:#12C2FF">"description"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"Checks whether an order qualifies for a refund under current policy and returns the reason."</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token property" style="color:#12C2FF">"inputSchema"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token property" style="color:#12C2FF">"type"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"object"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token property" style="color:#12C2FF">"properties"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"> </span><span class="token property" style="color:#12C2FF">"order_id"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"> </span><span class="token property" style="color:#12C2FF">"type"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"string"</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token property" style="color:#12C2FF">"required"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token string" style="color:#4ADE80">"order_id"</span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token punctuation" style="color:#8A93A6">}</span><br></span></code></pre></div></div>
<p>The agent then picks a tool and sends <code>tools/call</code>. Under the 2026-07-28 spec, a call over streamable HTTP carries routing headers alongside the JSON-RPC body.</p>
<div class="language-http codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-http codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">POST /mcp HTTP/1.1</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">Content-Type: application/json</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">Authorization: Bearer &lt;access-token&gt;</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">MCP-Protocol-Version: 2026-07-28</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">Mcp-Method: tools/call</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">Mcp-Name: check_refund_eligibility</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">{"jsonrpc": "2.0", "id": 2, "method": "tools/call",</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"> "params": {"name": "check_refund_eligibility", "arguments": {"order_id": "ORD-1042"}}}</span><br></span></code></pre></div></div>
<p>Behind that one tool, the server might call three REST endpoints and apply a policy rule. The agent sees one capability with one clear description. Both examples are abbreviated. They omit the <code>_meta</code> request metadata the spec requires, as the specification's own examples do, along with some transport headers.</p>
<p>Two details separate this from how most REST integrations describe themselves. First, the server can answer <code>tools/list</code> based on the authorization the request carries, so two agents with different scopes may see different tools from the same server. Filtering the list is not access control on its own, and the server must still authorize every <code>tools/call</code>. Second, servers can announce that their tool list changed, and list responses now carry cache hints (<code>ttlMs</code> and <code>cacheScope</code>) so clients know when to refetch. Clients should cache lists per authorization context and drop them when that context changes.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="rest-vs-mcp-at-a-glance">REST vs MCP at a glance<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#rest-vs-mcp-at-a-glance" class="hash-link" aria-label="Direct link to REST vs MCP at a glance" title="Direct link to REST vs MCP at a glance" translate="no">​</a></h2>
<p>The table below puts the MCP server vs REST API comparison side by side. Read it as two interfaces with different consumers rather than a ranking.</p>
<table><thead><tr><th>Dimension</th><th>REST API</th><th>MCP</th></tr></thead><tbody><tr><td>Architecture</td><td>Architectural style over HTTP, with resources identified by URLs</td><td>Protocol with host, client, and server roles</td></tr><tr><td>Integration pattern</td><td>Usually client code or a wrapper per API, written at design time</td><td>Compatible agent hosts connect through a standard client, subject to version, capability, and auth support</td></tr><tr><td>Discovery</td><td>OpenAPI or hypermedia, usually read at design time, with no single listing call shared by agent hosts</td><td>Runtime <code>tools/list</code>, <code>resources/list</code>, and <code>prompts/list</code>, filtered by the caller's authorization</td></tr><tr><td>Message format</td><td>JSON or other media types over HTTP methods</td><td>JSON-RPC 2.0 messages</td></tr><tr><td>Authentication</td><td>Any scheme the API chooses, such as API keys, bearer tokens, OAuth, or mTLS</td><td>Optional OAuth 2.1 profile for HTTP transports, with PKCE and audience-restricted tokens</td></tr><tr><td>State management</td><td>Stateless by constraint, with session state held by the client</td><td>Stateless protocol core since 2026-07-28, with explicit handles for continuity</td></tr><tr><td>Real-time updates</td><td>Not part of the style, so teams add webhooks, SSE, or polling</td><td>Opt-in subscription stream for list-change notifications</td></tr></tbody></table>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="authentication-why-mcp-is-more-opinionated-than-rest">Authentication: Why MCP is more opinionated than REST<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#authentication-why-mcp-is-more-opinionated-than-rest" class="hash-link" aria-label="Direct link to Authentication: Why MCP is more opinionated than REST" title="Direct link to Authentication: Why MCP is more opinionated than REST" translate="no">​</a></h2>
<p>REST does not define authentication. Each API picks its own scheme, and every client inherits that variety. For a human-built integration, that is a one-time cost. For an agent that connects to many servers, it becomes a blocker.</p>
<p>MCP takes a position, although authorization is optional in the protocol. When an HTTP-based server adopts it, the <a href="https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization" target="_blank" rel="noopener noreferrer" class="">MCP authorization specification</a> treats a protected server as an OAuth 2.1 resource server. Local stdio servers take credentials from the environment instead. Four requirements do most of the work:</p>
<ul>
<li class="">Discovery is built in. Servers must publish OAuth 2.0 Protected Resource Metadata (RFC 9728), which tells the client which authorization server to use.</li>
<li class="">PKCE is mandatory for the authorization code flow. Clients must implement PKCE and stop if the authorization server does not advertise support for it.</li>
<li class="">Tokens are restricted to one server. Clients must send the RFC 8707 <code>resource</code> parameter to name the MCP server a token is for. The authorization server must issue a token restricted to that audience, and the MCP server must validate it.</li>
<li class="">The 2026-07-28 revision tightened the flow. It added RFC 9207 issuer validation and deprecated Dynamic Client Registration in favor of Client ID Metadata Documents.</li>
</ul>
<p>Audience binding matters most for agents. A server that validates the audience will reject a token minted for a different server. A token for your ticketing system is not a token for your payments system. Scopes still decide what the agent may do on each server, and that is where gateway policy takes over. For a walkthrough of this flow with per-tool scopes, see <a class="" href="https://choreolabs.com/api-management/learn/mcp-server-authentication">MCP server authentication with OAuth 2.1</a>.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="when-to-use-rest-when-to-add-mcp-and-when-you-need-both">When to use REST, when to add MCP, and when you need both<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#when-to-use-rest-when-to-add-mcp-and-when-you-need-both" class="hash-link" aria-label="Direct link to When to use REST, when to add MCP, and when you need both" title="Direct link to When to use REST, when to add MCP, and when you need both" translate="no">​</a></h2>
<p>The decision turns on who the consumer is and what it needs to do. Base it on operational triggers rather than on how new the protocol is.</p>
<p><strong>Keep REST as the only interface when:</strong></p>
<ul>
<li class="">The consumers are web or mobile clients rendering known screens.</li>
<li class="">The workload is high-throughput data access, where caching and predictable payloads matter most.</li>
<li class="">Third-party developers build against a published, versioned contract.</li>
<li class="">The traffic is service-to-service inside your own architecture.</li>
</ul>
<p><strong>Add an MCP layer when:</strong></p>
<ul>
<li class="">AI agents are a production consumer, and more than one agent host needs the same capabilities.</li>
<li class="">Agent workflows span several calls or systems, and you want each outcome packaged as one tool.</li>
<li class="">You need per-action authorization for non-human identities, separate from what human clients are granted.</li>
<li class="">You want one integration that works across agent frameworks instead of an adapter for each.</li>
</ul>
<p><strong>Defer MCP when:</strong></p>
<ul>
<li class="">A single internal agent calls a few endpoints and one team owns both sides. Direct function calling against your REST API is simpler until a second agent or a second team appears.</li>
</ul>
<p>For most enterprises the realistic default is both, layered. The REST API keeps serving existing clients unchanged. An MCP server runs alongside it and calls the same endpoints or services internally. Business logic stays in one place, and the MCP server becomes a thin, agent-shaped layer that decides which capabilities agents see and how they are grouped.</p>
<p><img decoding="async" loading="lazy" alt="A typical REST integration sends HTTP requests from client apps to known endpoints. In an MCP integration, an AI agent calls tools/list and tools/call on an MCP server, which calls the unchanged REST API&amp;#39;s existing endpoints" src="https://choreolabs.com/api-management/learn/assets/images/rest-and-mcp-as-layers-bb783ef3596f52cfab3b04cd8738507a.webp" width="3200" height="1708" class="img_ev3q"></p>
<p><em>REST and MCP are layers, not adversaries. The MCP server reuses existing REST endpoints instead of duplicating business logic.</em></p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-governance-problem-this-creates-and-how-to-solve-it-without-choosing-sides">The governance problem this creates, and how to solve it without choosing sides<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#the-governance-problem-this-creates-and-how-to-solve-it-without-choosing-sides" class="hash-link" aria-label="Direct link to The governance problem this creates, and how to solve it without choosing sides" title="Direct link to The governance problem this creates, and how to solve it without choosing sides" translate="no">​</a></h2>
<p>Running both layers fixes the agent integration problem and creates an operational one. You now have two interfaces over the same systems, and each needs authentication, authorization, rate limits, and audit. If REST traffic runs through a managed gateway while MCP servers are deployed team by team, the agent side becomes the part nobody can see.</p>
<p>That blind spot is common. SailPoint's report <a href="https://www.sailpoint.com/press-releases/sailpoint-ai-agent-adoption-report" target="_blank" rel="noopener noreferrer" class="">AI agents: The new attack surface</a>, released in May 2025, covers a survey of 353 IT and security professionals conducted by Dimensional Research. In it, 80% of companies said their AI agents had taken unintended actions, such as accessing unauthorized systems or sharing sensitive data.</p>
<p>The fix is not a second, separate governance stack. It is bringing agent traffic under the same control plane as API traffic, with policies that understand MCP.</p>
<p>This is the approach <a href="https://wso2.com/api-platform/ai-gateway/" target="_blank" rel="noopener noreferrer" class="">Choreo AI Gateway</a> takes. It governs inbound agent-to-tool traffic through MCP proxies and outbound application-to-model traffic through LLM proxies, as separate paths under one control plane. APIs, LLM traffic, and agent access share one governance model and one analytics view. The gateway can run standalone or connect to Choreo Agent Control Plane for centralized management.</p>
<p>On an MCP proxy, the policies map directly to agent-specific risks (<a href="https://wso2.com/api-platform/docs/ai-gateway/1.2.0/mcp-governance/" target="_blank" rel="noopener noreferrer" class="">MCP governance documentation</a>):</p>
<ul>
<li class="">MCP Authentication secures traffic as the MCP specification's authorization profile describes, so OAuth enforcement doesn't depend on each server team.</li>
<li class="">MCP Authorization checks access to individual tools, resources, and prompts using JWT claims or OAuth scopes. A support agent can read orders without being allowed to call <code>issue_refund</code>.</li>
<li class="">MCP Access Control and MCP Rewrite limit which capabilities a caller can reach and give tools stable, user-facing names.</li>
<li class="">MCP Rate Limit applies limits per tool, resource, prompt, or JSON-RPC method, which helps contain a runaway agent loop before it overloads a backend.</li>
<li class="">Semantic Tool Filtering narrows the tool list to what is relevant to the query, which keeps the model's context small. <a class="" href="https://choreolabs.com/api-management/learn/ai-traffic-with-jev">Smarter AI traffic with Jev</a> shows how the gateway applies this.</li>
</ul>
<p>For audit, the Insights dashboards in Choreo Agent Control Plane report call volume per tool, agent sessions, and calling client runtimes for each MCP proxy. They also show latency and the error type behind each failure. Approved servers can be published to the AI Hub, a catalog agent developers search instead of wiring up servers on their own. None of this replaces careful tool design or least-privilege scopes at your identity provider. It gives you one place to enforce and observe them.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="getting-started-a-practical-path-from-rest-to-agent-ready">Getting started: A practical path from REST to agent-ready<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#getting-started-a-practical-path-from-rest-to-agent-ready" class="hash-link" aria-label="Direct link to Getting started: A practical path from REST to agent-ready" title="Direct link to Getting started: A practical path from REST to agent-ready" translate="no">​</a></h2>
<p>MCP works best when you treat it as interface design rather than a translation exercise. A practical sequence looks like this:</p>
<ol>
<li class="">Start from agent tasks, not endpoints. List the few jobs agents actually need, such as checking refund eligibility or opening a ticket, and design one tool per outcome. Mapping each REST endpoint to its own tool recreates the granularity problem inside MCP.</li>
<li class="">Reuse your existing business logic. Implement each tool as a call into the REST endpoints or services you already run. Don't fork logic into the MCP server. <a class="" href="https://choreolabs.com/api-management/learn/rest-api-to-mcp">REST API to MCP server</a> walks through this step by step.</li>
<li class="">Match each capability to its interaction model. Use resources for context the host application loads, and tools for anything the model should call, including read-only queries. Give tools with side effects narrower scopes.</li>
<li class="">Put OAuth 2.1 in front from the first deployment. Use your existing identity provider, require PKCE, and validate the audience on every token.</li>
<li class="">Route through governance before production. Place the server behind an MCP proxy, attach authorization and rate-limit policies, and confirm the traffic appears in your dashboards.</li>
<li class="">Test with real MCP clients. Watch which tools the model picks, then refine names and descriptions where it chooses wrongly.</li>
</ol>
<p>REST remains the interface for software that knows what it wants. MCP is the interface for agents that have to work it out. Most teams need both, and the work that decides success is tool design and governance rather than the protocol choice. When you are ready to build, follow a <a class="" href="https://choreolabs.com/api-management/learn/mcp-implementations">step-by-step guide to implementing MCP</a>, which ends with a running server placed behind an MCP proxy on Choreo AI Gateway.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>REST and MCP answer different questions. REST serves software that already knows what it wants. MCP lets an agent discover what it can do, then act within the limits you set. Since the 2026-07-28 specification made MCP stateless at the protocol layer, the two also run on similar infrastructure, which makes layering them easier.</p>
<p>The protocol choice is rarely the hard part. What decides whether agents are safe in production is tool design and governance. That means shaping tools around outcomes, binding tokens to the right server, and seeing every agent call alongside your API traffic.</p>
<p>Start with one agent task. Build one outcome-shaped tool over your existing REST logic, and put it behind policy before it reaches production. To see how one control plane governs both agent-to-tool and application-to-model traffic, explore <a href="https://wso2.com/api-platform/ai-gateway/" target="_blank" rel="noopener noreferrer" class="">Choreo AI Gateway</a>.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="is-mcp-just-a-rest-api">Is MCP just a REST API?<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#is-mcp-just-a-rest-api" class="hash-link" aria-label="Direct link to Is MCP just a REST API?" title="Direct link to Is MCP just a REST API?" translate="no">​</a></h3>
<p>No. MCP is a JSON-RPC protocol with its own discovery and authorization rules. An MCP server often calls REST APIs internally, but agents interact with it through MCP messages rather than resource URLs.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-is-an-mcp-server-different-from-a-rest-api">How is an MCP server different from a REST API?<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#how-is-an-mcp-server-different-from-a-rest-api" class="hash-link" aria-label="Direct link to How is an MCP server different from a REST API?" title="Direct link to How is an MCP server different from a REST API?" translate="no">​</a></h3>
<p>A REST API exposes resources to clients that already know what to request. An MCP server exposes a curated set of tools, resources, and prompts that an agent lists and invokes at runtime.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-the-difference-between-an-api-and-an-mcp">What is the difference between an API and an MCP?<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#what-is-the-difference-between-an-api-and-an-mcp" class="hash-link" aria-label="Direct link to What is the difference between an API and an MCP?" title="Direct link to What is the difference between an API and an MCP?" translate="no">​</a></h3>
<p>An API is any interface one program offers another. MCP is a specific protocol for connecting AI applications to external capabilities, so an MCP server is one kind of API, built for model-driven consumers.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-replacing-rest-apis">What is replacing REST APIs?<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#what-is-replacing-rest-apis" class="hash-link" aria-label="Direct link to What is replacing REST APIs?" title="Direct link to What is replacing REST APIs?" translate="no">​</a></h3>
<p>Nothing, for most workloads. MCP adds an agent-facing layer over existing APIs, while web, mobile, partner, and service-to-service traffic continue to fit REST.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="is-mcp-stateful">Is MCP stateful?<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#is-mcp-stateful" class="hash-link" aria-label="Direct link to Is MCP stateful?" title="Direct link to Is MCP stateful?" translate="no">​</a></h3>
<p>Not at the protocol level since the 2026-07-28 specification. Every request stands alone. When a workflow needs continuity, the server issues an identifier that travels as an ordinary tool argument.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="do-i-need-mcp-if-my-agent-already-uses-function-calling">Do I need MCP if my agent already uses function calling?<a href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp#do-i-need-mcp-if-my-agent-already-uses-function-calling" class="hash-link" aria-label="Direct link to Do I need MCP if my agent already uses function calling?" title="Direct link to Do I need MCP if my agent already uses function calling?" translate="no">​</a></h3>
<p>Not always. Function calling suits one agent and a small set of endpoints. MCP becomes worth it when several agent hosts share capabilities or when you need central authorization and audit.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="MCP" term="MCP"/>
        <category label="Comparison" term="Comparison"/>
        <category label="Agent-ready APIs" term="Agent-ready APIs"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[Function calling vs. tool calling vs. MCP: What's the difference?]]></title>
        <id>https://choreolabs.com/api-management/learn/tool-calling-vs-mcp</id>
        <link href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp"/>
        <updated>2026-10-13T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[OpenAI function calling and Anthropic tool use carry a model's tool calls, and MCP complements them. Compare schemas, execution flow, and when to use each.]]></summary>
        <content type="html"><![CDATA[<p>Your agent works well against one model until someone asks for a second provider, and then the tool layer breaks. The tool definitions look almost the same, but the request shape, the response shape, and the way you return results all differ. Then a teammate asks whether MCP makes all of this unnecessary.</p>
<p>It doesn't. This guide explains what function calling is, how the loop runs, and where OpenAI and Anthropic differ at the field level. It then covers where the Model Context Protocol (MCP) fits and how to design tool schemas. The last sections cover tool failures and what changes once an application needs more than one model provider. It is part of the <a class="" href="https://choreolabs.com/api-management/learn/agent-ready-apis">agent-ready APIs</a> series.</p>
<aside class="cta_6F_m" aria-label="Standardize tool calls across every model" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Standardize tool calls across every model</span><span class="desc_GzK8">Use Choreo's MCP gateway layer to govern tool calls consistently across OpenAI, Anthropic, and other providers.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=tool-calling-vs-mcp" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-function-calling-and-is-it-the-same-as-tool-calling">What is function calling? (And is it the same as tool calling?)<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#what-is-function-calling-and-is-it-the-same-as-tool-calling" class="hash-link" aria-label="Direct link to What is function calling? (And is it the same as tool calling?)" title="Direct link to What is function calling? (And is it the same as tool calling?)" translate="no">​</a></h2>
<p>Function calling and tool calling are the same concept. A model receives descriptions of functions it may use. When a request needs one, the model returns a structured call with a function name and arguments. Your application runs the function and sends the result back. The model never executes anything itself, even when its output looks like code.</p>
<p>The two names come from vendor convention. OpenAI's documentation states it directly: function calling is "also known as tool calling." "Tool" is the broader term in current APIs. A function is one kind of tool, defined by a JSON Schema. Providers also offer built-in tools, such as web search, that run on their side.</p>
<p>A practical example makes the boundary clear. Suppose a support agent gets the question "Where is order 4417?" The model cannot query your order system on its own, so it can only return something like <code>get_order_status(order_id="4417")</code>. Your code decides whether to run that call, runs it against your API, and returns the status. Every permission check, retry, and audit record lives in your code, not in the model.</p>
<p>That boundary matters for every section that follows, because function calling is a contract between your application and a model. It is not a protocol for reaching tools, and it does not give the model network access of its own.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-functiontool-calling-actually-works">How Function/Tool calling actually works<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#how-functiontool-calling-actually-works" class="hash-link" aria-label="Direct link to How Function/Tool calling actually works" title="Direct link to How Function/Tool calling actually works" translate="no">​</a></h2>
<p>The loop has five steps. OpenAI documents the same five, and the pattern holds for other providers.</p>
<ol>
<li class=""><strong>Send the prompt and tool definitions.</strong> Each definition has a name, a description, and a JSON Schema for its parameters.</li>
<li class=""><strong>Receive a tool call.</strong> The model returns a structured request with a tool name and arguments, but nothing has run yet.</li>
<li class=""><strong>Validate and execute.</strong> Your application checks the arguments, then calls the real API, database, or service.</li>
<li class=""><strong>Send the result back.</strong> The result goes into the conversation, linked to the call by an ID.</li>
<li class=""><strong>Receive the final answer.</strong> The model either answers in natural language or issues another tool call.</li>
</ol>
<p><img decoding="async" loading="lazy" alt="The client-managed tool-calling loop between the LLM provider, your application, and tools. 1: prompt and tool definitions go to the model. 2: the model returns a tool call with name and arguments. 3: the application validates and executes it against the tool. 4: the tool result and call ID go back to the model. 5: the model gives a final answer or another tool call" src="https://choreolabs.com/api-management/learn/assets/images/client-managed-tool-calling-loop-360a0329f980ebe0568284d0d91fdb10.webp" width="3200" height="2134" class="img_ev3q"></p>
<p><em>The model and the tools never talk to each other. In this client-managed loop, your application runs every tool call.</em></p>
<p>The diagram shows a client-managed tool loop, where your application runs the tools. In this arrangement, the model and the tools never talk to each other. The application calls the model API, and it initiates every tool call itself, either directly or through a proxy. A call to the model does not pass through the tool, and a tool call does not pass through the model. Provider-hosted tools work differently, as the MCP section explains.</p>
<p>Step 5 is what makes multi-step agentic AI possible. A multi-step task becomes a loop of steps 2 to 4 until the model stops asking for tools. That loop is your code, so you own its limits. Tool definitions also cost money on every turn. OpenAI notes that callable function definitions count against the context window and are billed as input tokens.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="openai-function-calling-vs-anthropic-tool-use-the-same-idea-different-shapes">OpenAI function calling vs. Anthropic tool use: The same idea, different shapes<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#openai-function-calling-vs-anthropic-tool-use-the-same-idea-different-shapes" class="hash-link" aria-label="Direct link to OpenAI function calling vs. Anthropic tool use: The same idea, different shapes" title="Direct link to OpenAI function calling vs. Anthropic tool use: The same idea, different shapes" translate="no">​</a></h2>
<p>Both providers implement the same five-step loop, and the differences are in the field names and in where data sits in each message. Those differences are small, but every one of them breaks an integration that assumes the other shape. For broader trade-offs between the two APIs, see <a class="" href="https://choreolabs.com/api-management/learn/openai-vs-anthropic-vs-gemini">OpenAI API vs Anthropic API vs Gemini API</a>.</p>
<p>This section uses the OpenAI Chat Completions API. OpenAI's newer Responses API flattens the tool definition and uses <code>function_call</code> and <code>function_call_output</code> items keyed by <code>call_id</code>. Chat Completions is also the shape that the gateway translation described later in this guide supports.</p>
<p><img decoding="async" loading="lazy" alt="The same get_order_status tool defined twice. OpenAI Chat Completions nests name, description, and parameters inside a function object. Anthropic Messages puts name, description, and input_schema at the top level" src="https://choreolabs.com/api-management/learn/assets/images/openai-vs-anthropic-tool-schema-2f73fe0d42023525ec5778cb4672d715.webp" width="3200" height="1794" class="img_ev3q"></p>
<p><em>The JSON Schema inside is the same. Treat the wrapper as a serialization detail that one adapter owns.</em></p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="defining-a-tool">Defining a tool<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#defining-a-tool" class="hash-link" aria-label="Direct link to Defining a tool" title="Direct link to Defining a tool" translate="no">​</a></h3>
<p>OpenAI nests the function inside a <code>function</code> object. With <code>strict: true</code>, every property must be listed in <code>required</code> and <code>additionalProperties</code> must be <code>false</code>.</p>
<div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"type"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"function"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"function"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"name"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"get_order_status"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"description"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"Look up the shipping status of one customer order by its order ID. Use only when the user gives an order ID."</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"parameters"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token property" style="color:#12C2FF">"type"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"object"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token property" style="color:#12C2FF">"properties"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token property" style="color:#12C2FF">"order_id"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"> </span><span class="token property" style="color:#12C2FF">"type"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"string"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token property" style="color:#12C2FF">"description"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"Order ID, for example 4417"</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token property" style="color:#12C2FF">"required"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token string" style="color:#4ADE80">"order_id"</span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token property" style="color:#12C2FF">"additionalProperties"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token boolean" style="color:#FF4600">false</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"strict"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token boolean" style="color:#FF4600">true</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token punctuation" style="color:#8A93A6">}</span><br></span></code></pre></div></div>
<p>Anthropic puts the schema at the top level of the tool, under <code>input_schema</code>. There is no <code>function</code> wrapper. To turn on strict mode, add <code>"strict": true</code> at the top level, next to <code>input_schema</code>.</p>
<div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"name"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"get_order_status"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"description"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"Look up the shipping status of one customer order by its order ID. Use only when the user gives an order ID."</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"input_schema"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"type"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"object"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"properties"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token property" style="color:#12C2FF">"order_id"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"> </span><span class="token property" style="color:#12C2FF">"type"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"string"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token property" style="color:#12C2FF">"description"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"Order ID, for example 4417"</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"required"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token string" style="color:#4ADE80">"order_id"</span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token property" style="color:#12C2FF">"additionalProperties"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token boolean" style="color:#FF4600">false</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"strict"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token boolean" style="color:#FF4600">true</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token punctuation" style="color:#8A93A6">}</span><br></span></code></pre></div></div>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="returning-the-result">Returning the result<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#returning-the-result" class="hash-link" aria-label="Direct link to Returning the result" title="Direct link to Returning the result" translate="no">​</a></h3>
<p>The response side differs more than the definitions do. OpenAI returns <code>tool_calls</code> on the assistant message. Each call has an <code>id</code>, and <code>function.arguments</code> is a JSON-encoded string. You return the result as a separate message with <code>role: "tool"</code> and the matching <code>tool_call_id</code>.</p>
<div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"> </span><span class="token property" style="color:#12C2FF">"role"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"tool"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token property" style="color:#12C2FF">"tool_call_id"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"call_abc123"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token property" style="color:#12C2FF">"content"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"{\"status\": \"shipped\"}"</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">}</span><br></span></code></pre></div></div>
<p>Anthropic returns <code>stop_reason: "tool_use"</code> and a <code>tool_use</code> content block. Its <code>input</code> is already a parsed object, not a string. Anthropic has no tool role, so you return a <code>tool_result</code> block inside a <code>user</code> message, keyed by <code>tool_use_id</code>.</p>
<div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"role"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"user"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"content"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"> </span><span class="token property" style="color:#12C2FF">"type"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"tool_result"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token property" style="color:#12C2FF">"tool_use_id"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"toolu_01A09q"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token property" style="color:#12C2FF">"content"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"{\"status\": \"shipped\"}"</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token punctuation" style="color:#8A93A6">}</span><br></span></code></pre></div></div>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="a-quick-reference-comparison-table">A quick-reference comparison table<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#a-quick-reference-comparison-table" class="hash-link" aria-label="Direct link to A quick-reference comparison table" title="Direct link to A quick-reference comparison table" translate="no">​</a></h2>
<p>The table adds MCP as a third column, since MCP defines its own tool shape. This is the full field map used in the rest of the guide.</p>
<table><thead><tr><th>Aspect</th><th>OpenAI (Chat Completions)</th><th>Anthropic (Messages)</th><th>MCP</th></tr></thead><tbody><tr><td>Tool definition</td><td><code>tools[].function</code> with <code>name</code>, <code>description</code>, <code>parameters</code></td><td><code>tools[]</code> with <code>name</code>, <code>description</code>, <code>input_schema</code></td><td><code>tools/list</code> result with <code>name</code>, <code>description</code>, <code>inputSchema</code></td></tr><tr><td>Schema language</td><td>JSON Schema</td><td>JSON Schema</td><td>JSON Schema</td></tr><tr><td>Call signal</td><td><code>tool_calls</code> on the assistant message</td><td><code>stop_reason: "tool_use"</code> plus <code>tool_use</code> blocks</td><td>Client sends <code>tools/call</code></td></tr><tr><td>Arguments</td><td>JSON string in <code>function.arguments</code></td><td>Parsed object in <code>input</code></td><td>Object in <code>params.arguments</code></td></tr><tr><td>Result</td><td><code>role: "tool"</code> message with <code>tool_call_id</code></td><td><code>tool_result</code> block in a user message with <code>tool_use_id</code></td><td>JSON-RPC result with <code>content</code></td></tr><tr><td>Error signal</td><td>No dedicated field. Describe the error in content</td><td><code>is_error: true</code> on <code>tool_result</code></td><td><code>isError: true</code> in the result</td></tr><tr><td>Disable tools</td><td><code>tool_choice: "none"</code></td><td><code>tool_choice: {"type": "none"}</code></td><td>Not applicable. The host decides which tools to expose</td></tr><tr><td>Force a tool</td><td><code>"required"</code> or a named function</td><td><code>any</code> or <code>tool</code>, not supported on some current models</td><td>Not applicable</td></tr><tr><td>Strict schemas</td><td><code>strict: true</code></td><td><code>strict: true</code></td><td>Not applicable. Servers validate inputs</td></tr></tbody></table>
<p>Two rows in this table deserve a closer look. Anthropic supports <code>tool_choice</code> set to <code>none</code>, so disabling tools is possible on both sides. Forcing a tool is much less portable across providers. Anthropic's documentation states that <code>any</code> and <code>tool</code> return a 400 error on Claude Opus 5.5, Claude Sonnet 5.5, Claude Fable 5.1, and Claude Mythos 5.1. Code that forces a tool on one provider can fail on the other.</p>
<p>The arguments row is easy to get wrong in a hand-written adapter. OpenAI arguments arrive as a string that needs decoding. Anthropic <code>input</code> arrives already parsed.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="where-mcp-fits-a-standardization-layer-not-a-competing-mechanism">Where MCP fits: A standardization layer, not a competing mechanism<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#where-mcp-fits-a-standardization-layer-not-a-competing-mechanism" class="hash-link" aria-label="Direct link to Where MCP fits: A standardization layer, not a competing mechanism" title="Direct link to Where MCP fits: A standardization layer, not a competing mechanism" translate="no">​</a></h2>
<p>The "OpenAI function calling vs MCP" framing suggests a choice between the two, but there isn't one. MCP does not replace function calling. It standardizes how tools are discovered and invoked, so the same tool definitions work across any MCP-compatible client.</p>
<p>MCP is an open standard for connecting AI applications to external systems. It uses a client-host-server model built on JSON-RPC. An MCP server publishes tools, and a client discovers them with <code>tools/list</code> and invokes them with <code>tools/call</code>. Each tool carries an <code>inputSchema</code> written in JSON Schema. For how to build and run an MCP server, see <a class="" href="https://choreolabs.com/api-management/learn/mcp-implementations">MCP implementations</a>.</p>
<p>Here is how the two layers meet in a running agent:</p>
<ol>
<li class="">The host application asks each MCP server for its tools with <code>tools/list</code>.</li>
<li class="">The host maps each tool into the model's format. An MCP <code>inputSchema</code> becomes an OpenAI <code>parameters</code> object or an Anthropic <code>input_schema</code>.</li>
<li class="">The model emits an ordinary tool call, exactly as in the previous section.</li>
<li class="">The host maps that call to a <code>tools/call</code> request on the right server and returns the result to the model.</li>
</ol>
<p>The specification gives the host this coordinating role. The host "coordinates AI/LLM integration," and each client keeps a one-to-one connection with a single server. The two layers are complementary. MCP exposes external capabilities, while the provider API carries the model's tool requests and results. Some providers also host the MCP client on their side. OpenAI, for example, offers a built-in tool that reaches MCP servers. In that arrangement, OpenAI's platform calls the MCP server itself, and the response contains an <code>mcp_call</code> item with the arguments and the server's output. Your application does not execute that call as a function call. MCP replaces the custom glue between your application and each tool, not the contract between your application and the model.</p>
<p>This changes where the work moves. Without MCP, every tool integration is hand-wired per application and per SDK. With MCP, a tool is written once and any compliant host can discover it. The cost is a new runtime dependency. MCP servers, especially remote ones, are services that need authentication, authorization, and rate limits. They also need monitoring like any other API.</p>
<p>MCP is not always worth that cost. A single application that calls a few private functions gains little from a protocol layer. The case for MCP grows when the same tools serve several applications, or when agents built by other teams need to discover them without custom integration work.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="schema-design-best-practices">Schema design best practices<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#schema-design-best-practices" class="hash-link" aria-label="Direct link to Schema design best practices" title="Direct link to Schema design best practices" translate="no">​</a></h2>
<p>A tool schema works as part of the prompt. The model selects a tool and fills its arguments using only the name, the description, and the schema. Poor schemas cause wrong tool selection long before they cause runtime errors.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="write-descriptions-for-the-model-not-for-people">Write descriptions for the model, not for people<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#write-descriptions-for-the-model-not-for-people" class="hash-link" aria-label="Direct link to Write descriptions for the model, not for people" title="Direct link to Write descriptions for the model, not for people" translate="no">​</a></h3>
<p>Anthropic calls detailed descriptions "by far the most important factor in tool performance." State what the tool does, when to use it, when not to use it, and what each parameter means. A description like "Gets order" is too vague for reliable selection. "Look up the shipping status of one order by ID. Use only when the user gives an order ID" is better.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="keep-the-tool-set-lean-per-request">Keep the tool set lean per request<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#keep-the-tool-set-lean-per-request" class="hash-link" aria-label="Direct link to Keep the tool set lean per request" title="Direct link to Keep the tool set lean per request" translate="no">​</a></h3>
<p>Every definition adds input tokens on every turn. A long list also makes the model's selection harder and less predictable. Anthropic recommends consolidating related operations into fewer tools, since fewer tools reduce selection ambiguity. If an agent needs a large tool surface, load tools on demand instead of sending everything at once.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="constrain-inputs-in-the-schema">Constrain inputs in the schema<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#constrain-inputs-in-the-schema" class="hash-link" aria-label="Direct link to Constrain inputs in the schema" title="Direct link to Constrain inputs in the schema" translate="no">​</a></h3>
<p>Use enums for fixed values and required fields for anything the tool cannot run without. Turn on strict mode where the provider supports it. OpenAI's strict mode makes calls adhere to the schema instead of best effort. Anthropic's <code>strict: true</code> guarantees tool inputs match the schema.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="do-not-ask-the-model-for-values-you-already-have">Do not ask the model for values you already have<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#do-not-ask-the-model-for-values-you-already-have" class="hash-link" aria-label="Direct link to Do not ask the model for values you already have" title="Direct link to Do not ask the model for values you already have" translate="no">​</a></h3>
<p>If your code knows the customer ID from the session, inject it at execution time and leave it out of the schema. This reduces the arguments the model can get wrong, and it also stops the model from choosing a different customer.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="name-tools-for-a-shared-namespace">Name tools for a shared namespace<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#name-tools-for-a-shared-namespace" class="hash-link" aria-label="Direct link to Name tools for a shared namespace" title="Direct link to Name tools for a shared namespace" translate="no">​</a></h3>
<p>Prefix names with the service, such as <code>orders_get_status</code> or <code>billing_issue_refund</code>. Anthropic recommends this pattern as the tool library grows. It matters even more when tools from several MCP servers share one context.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="design-for-portability">Design for portability<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#design-for-portability" class="hash-link" aria-label="Direct link to Design for portability" title="Direct link to Design for portability" translate="no">​</a></h3>
<p>Stick to JSON Schema features both providers accept, and keep the schema identical across providers. Treat the wrapper (<code>function</code> versus <code>input_schema</code>) as a serialization detail that one adapter owns. Provider strict modes accept only a subset of JSON Schema and add constraints of their own. OpenAI's strict mode, for example, requires <code>additionalProperties</code> set to <code>false</code> on every object. A schema that an MCP server publishes is therefore not automatically valid for every provider. Have the adapter validate and normalize each schema before it sends the request. The examples in this guide use the same constraints for both providers.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="error-handling-for-tool-calls">Error handling for tool calls<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#error-handling-for-tool-calls" class="hash-link" aria-label="Direct link to Error handling for tool calls" title="Direct link to Error handling for tool calls" translate="no">​</a></h2>
<p>Tool calls fail in four places, namely the arguments, the execution, the results, and the loop. Each needs its own handling.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="malformed-or-invented-arguments">Malformed or invented arguments<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#malformed-or-invented-arguments" class="hash-link" aria-label="Direct link to Malformed or invented arguments" title="Direct link to Malformed or invented arguments" translate="no">​</a></h3>
<p>OpenAI's API reference warns that the model "does not always generate valid JSON, and may hallucinate parameters not defined by your function schema." Parse and validate before execution, even with strict mode on. Strict mode covers schema shape. It does not check business rules, such as whether the order belongs to this user.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="execution-failures">Execution failures<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#execution-failures" class="hash-link" aria-label="Direct link to Execution failures" title="Direct link to Execution failures" translate="no">​</a></h3>
<p>Return the failure to the model instead of silently dropping it. Anthropic provides <code>is_error: true</code> on <code>tool_result</code>. MCP reports tool execution errors with <code>isError: true</code>, and clients should pass them to the model so it can self-correct. Make the message actionable. Anthropic advises stating what went wrong and what to try next. "Order 4417 not found. Ask the user to confirm the ID" gives the model a path forward. A bare "Error" string does not.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="untrusted-results">Untrusted results<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#untrusted-results" class="hash-link" aria-label="Direct link to Untrusted results" title="Direct link to Untrusted results" translate="no">​</a></h3>
<p>Tool output often contains text from outside your control, such as web pages, emails, or third-party API fields. Anthropic warns that such content can carry indirect prompt injection. The MCP specification also says clients should validate tool results before passing them to the model, set timeouts, and log tool usage.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="runaway-loops">Runaway loops<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#runaway-loops" class="hash-link" aria-label="Direct link to Runaway loops" title="Direct link to Runaway loops" translate="no">​</a></h3>
<p>A model can call the same tool repeatedly when results confuse it. A single response can also contain several tool calls. Cap the model turns, the total number of tool calls, and the wall-clock time for each request. The loop belongs to your code, so the limits belong there too.</p>
<div class="language-python codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-python codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token keyword" style="color:#3D8BFF">import</span><span class="token plain"> json</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token keyword" style="color:#3D8BFF">import</span><span class="token plain"> logging</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token keyword" style="color:#3D8BFF">import</span><span class="token plain"> time</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token keyword" style="color:#3D8BFF">from</span><span class="token plain"> jsonschema </span><span class="token keyword" style="color:#3D8BFF">import</span><span class="token plain"> ValidationError</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> validate</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">log </span><span class="token operator">=</span><span class="token plain"> logging</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">getLogger</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token string" style="color:#4ADE80">"agent"</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token keyword" style="color:#3D8BFF">def</span><span class="token plain"> </span><span class="token function" style="color:#FFFFFF;font-weight:600">run_agent</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">client</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> model</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> messages</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> tools</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> handlers</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> schemas</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">              max_steps</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> max_tool_calls</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> deadline_seconds</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    deadline </span><span class="token operator">=</span><span class="token plain"> time</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">monotonic</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"> </span><span class="token operator">+</span><span class="token plain"> deadline_seconds</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    calls_used </span><span class="token operator">=</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">0</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token keyword" style="color:#3D8BFF">for</span><span class="token plain"> _ </span><span class="token keyword" style="color:#3D8BFF">in</span><span class="token plain"> </span><span class="token builtin" style="color:#3D8BFF">range</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">max_steps</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token keyword" style="color:#3D8BFF">if</span><span class="token plain"> time</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">monotonic</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"> </span><span class="token operator">&gt;</span><span class="token plain"> deadline</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">            </span><span class="token keyword" style="color:#3D8BFF">break</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        reply </span><span class="token operator">=</span><span class="token plain"> client</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">chat</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">completions</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">create</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">model</span><span class="token operator">=</span><span class="token plain">model</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                                               messages</span><span class="token operator">=</span><span class="token plain">messages</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> tools</span><span class="token operator">=</span><span class="token plain">tools</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        message </span><span class="token operator">=</span><span class="token plain"> reply</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">choices</span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token number" style="color:#FF4600">0</span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">message</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token keyword" style="color:#3D8BFF">if</span><span class="token plain"> </span><span class="token keyword" style="color:#3D8BFF">not</span><span class="token plain"> message</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">tool_calls</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">            </span><span class="token keyword" style="color:#3D8BFF">return</span><span class="token plain"> message</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">content</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        messages</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">append</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">message</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token keyword" style="color:#3D8BFF">for</span><span class="token plain"> call </span><span class="token keyword" style="color:#3D8BFF">in</span><span class="token plain"> message</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">tool_calls</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">            name </span><span class="token operator">=</span><span class="token plain"> call</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">function</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">name</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">            calls_used </span><span class="token operator">+=</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">1</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">            </span><span class="token keyword" style="color:#3D8BFF">if</span><span class="token plain"> calls_used </span><span class="token operator">&gt;</span><span class="token plain"> max_tool_calls </span><span class="token keyword" style="color:#3D8BFF">or</span><span class="token plain"> time</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">monotonic</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"> </span><span class="token operator">&gt;</span><span class="token plain"> deadline</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                result </span><span class="token operator">=</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token string" style="color:#4ADE80">"error"</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"Tool budget used up. Answer with what you have."</span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">            </span><span class="token keyword" style="color:#3D8BFF">elif</span><span class="token plain"> name </span><span class="token keyword" style="color:#3D8BFF">not</span><span class="token plain"> </span><span class="token keyword" style="color:#3D8BFF">in</span><span class="token plain"> handlers</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                result </span><span class="token operator">=</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token string" style="color:#4ADE80">"error"</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string-interpolation string" style="color:#4ADE80">f"Unknown tool </span><span class="token string-interpolation interpolation punctuation" style="color:#8A93A6">{</span><span class="token string-interpolation interpolation">name</span><span class="token string-interpolation interpolation punctuation" style="color:#8A93A6">}</span><span class="token string-interpolation string" style="color:#4ADE80">."</span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">            </span><span class="token keyword" style="color:#3D8BFF">else</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                </span><span class="token keyword" style="color:#3D8BFF">try</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                    args </span><span class="token operator">=</span><span class="token plain"> json</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">loads</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">call</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">function</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">arguments</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                    validate</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">args</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> schemas</span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token plain">name</span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                    result </span><span class="token operator">=</span><span class="token plain"> handlers</span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token plain">name</span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token operator">**</span><span class="token plain">args</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                </span><span class="token keyword" style="color:#3D8BFF">except</span><span class="token plain"> json</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">JSONDecodeError</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                    result </span><span class="token operator">=</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token string" style="color:#4ADE80">"error"</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string-interpolation string" style="color:#4ADE80">f"Arguments for </span><span class="token string-interpolation interpolation punctuation" style="color:#8A93A6">{</span><span class="token string-interpolation interpolation">name</span><span class="token string-interpolation interpolation punctuation" style="color:#8A93A6">}</span><span class="token string-interpolation string" style="color:#4ADE80"> were not valid JSON."</span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                </span><span class="token keyword" style="color:#3D8BFF">except</span><span class="token plain"> ValidationError </span><span class="token keyword" style="color:#3D8BFF">as</span><span class="token plain"> exc</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                    result </span><span class="token operator">=</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token string" style="color:#4ADE80">"error"</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string-interpolation string" style="color:#4ADE80">f"Invalid arguments for </span><span class="token string-interpolation interpolation punctuation" style="color:#8A93A6">{</span><span class="token string-interpolation interpolation">name</span><span class="token string-interpolation interpolation punctuation" style="color:#8A93A6">}</span><span class="token string-interpolation string" style="color:#4ADE80">: </span><span class="token string-interpolation interpolation punctuation" style="color:#8A93A6">{</span><span class="token string-interpolation interpolation">exc</span><span class="token string-interpolation interpolation punctuation" style="color:#8A93A6">.</span><span class="token string-interpolation interpolation">message</span><span class="token string-interpolation interpolation punctuation" style="color:#8A93A6">}</span><span class="token string-interpolation string" style="color:#4ADE80">"</span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                </span><span class="token keyword" style="color:#3D8BFF">except</span><span class="token plain"> Exception</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                    log</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">exception</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token string" style="color:#4ADE80">"Tool %s failed"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> name</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                    result </span><span class="token operator">=</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token string" style="color:#4ADE80">"error"</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string-interpolation string" style="color:#4ADE80">f"</span><span class="token string-interpolation interpolation punctuation" style="color:#8A93A6">{</span><span class="token string-interpolation interpolation">name</span><span class="token string-interpolation interpolation punctuation" style="color:#8A93A6">}</span><span class="token string-interpolation string" style="color:#4ADE80"> failed. Tell the user it could not complete."</span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">            messages</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">append</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token string" style="color:#4ADE80">"role"</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"tool"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"tool_call_id"</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> call</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token builtin" style="color:#3D8BFF">id</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                             </span><span class="token string" style="color:#4ADE80">"content"</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> json</span><span class="token punctuation" style="color:#8A93A6">.</span><span class="token plain">dumps</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token plain">result</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token punctuation" style="color:#8A93A6">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token keyword" style="color:#3D8BFF">raise</span><span class="token plain"> RuntimeError</span><span class="token punctuation" style="color:#8A93A6">(</span><span class="token string" style="color:#4ADE80">"Stopped at the turn, tool-call, or time budget without a final answer"</span><span class="token punctuation" style="color:#8A93A6">)</span><br></span></code></pre></div></div>
<p>This minimal loop validates argument shape and limits model turns, tool calls, and total time. It does not provide duplicate-write protection, authorization, or per-handler timeouts. Add those controls before exposing operations with side effects. For writes, use a stable operation ID with <a class="" href="https://choreolabs.com/api-management/learn/designing-idempotent-tools">server-side idempotency</a>, check that the caller may act on the specific object, and define a recovery policy. Do not block every repeated argument set, because two separately authorized refunds can have identical arguments. The example uses the Chat Completions shape. Set <code>max_steps</code>, <code>max_tool_calls</code>, and <code>deadline_seconds</code> from what the task needs and log every stop. A trace of each call, argument set, and result is what lets you debug an agent after the fact. Redact sensitive arguments and results before they reach the logs, and restrict who can read the traces.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="choosing-an-approach-for-multi-provider-applications">Choosing an approach for multi-provider applications<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#choosing-an-approach-for-multi-provider-applications" class="hash-link" aria-label="Direct link to Choosing an approach for multi-provider applications" title="Direct link to Choosing an approach for multi-provider applications" translate="no">​</a></h2>
<p>For a single-vendor application, use that vendor's native function calling directly. The SDKs handle the shapes, and a translation layer would add work without adding value. Add MCP when tools need to be shared across applications or exposed to agents you do not control.</p>
<p>The decision changes when operational needs appear, and the triggers are qualitative rather than a traffic number:</p>
<ul>
<li class="">More than one model provider. Each new provider adds another adapter for the shapes in the comparison table.</li>
<li class="">Spend attribution. Finance needs cost per team or per application, which is hard to assemble from separate SDK integrations.</li>
<li class="">Regulated data in prompts and tool results. Masking and audit must apply the same way for every model and tool.</li>
<li class="">Production scale and shared tools. Several teams reuse the same tools, and each tool needs one access policy instead of one per application.</li>
</ul>
<p>At that point, translation and policy belong in shared infrastructure. Choreo AI Gateway handles both kinds of AI traffic: model calls through LLM proxies and tool calls through MCP proxies. These are separate paths, and an MCP proxy routes to its MCP server directly, without passing through an LLM provider. The application still runs the tool calling loop itself.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="model-traffic">Model traffic<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#model-traffic" class="hash-link" aria-label="Direct link to Model traffic" title="Direct link to Model traffic" translate="no">​</a></h3>
<p>With <a href="https://wso2.com/api-platform/docs/ai-gateway/2026.09.24/routing/multi-provider-routing/" target="_blank" rel="noopener noreferrer" class="">multi-provider routing</a>, an application sends every request to one OpenAI-compatible endpoint and selects a provider, for example with an <code>x-provider</code> header. The <a href="https://wso2.com/api-platform/policy-hub/policies/openai-to-anthropic-transformer" target="_blank" rel="noopener noreferrer" class="">OpenAI to Anthropic transformer</a> maps OpenAI <code>tools</code>, <code>tool_choice</code>, <code>tool_calls</code>, and tool messages to Anthropic <code>tools</code> with <code>input_schema</code>, <code>tool_use</code>, and <code>tool_result</code>. It maps <code>required</code> to <code>any</code>, a named function to <code>tool</code>, and drops the <code>tools</code> array when <code>tool_choice</code> is <code>none</code>. The OpenAI to Gemini transformer does the same for Gemini's <code>functionDeclarations</code> and <code>toolConfig</code>.</p>
<p>This is the comparison table, implemented once instead of in every application. On the same model path, the Semantic Tool Filtering policy trims the <code>tools</code> array in each request to the tools relevant to the user query. That applies the lean-tool-set rule from the schema section without changing application code. <a class="" href="https://choreolabs.com/api-management/learn/ai-traffic-with-jev">Smarter AI traffic with Jev</a> shows tool filtering at work.</p>
<p>The translation has limits. It targets the Chat Completions API only. It does not carry over strict-mode flags or <code>parallel_tool_calls</code>. A successful conversion also does not mean the target model accepts the mapped value. A forced tool choice still fails on Claude models that reject forced tool use. Keep argument validation in your application regardless of what the gateway translates.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="tool-traffic">Tool traffic<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#tool-traffic" class="hash-link" aria-label="Direct link to Tool traffic" title="Direct link to Tool traffic" translate="no">​</a></h3>
<p>For MCP proxies, <a href="https://wso2.com/api-platform/docs/ai-gateway/2026.09.24/mcp-governance/" target="_blank" rel="noopener noreferrer" class="">MCP governance</a> policies cover authentication, per-tool authorization, allow and deny lists, tool renaming, and rate limits per tool or method.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="one-control-plane-for-both">One control plane for both<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#one-control-plane-for-both" class="hash-link" aria-label="Direct link to One control plane for both" title="Direct link to One control plane for both" translate="no">​</a></h3>
<p>Choreo Agent Control Plane is the control plane that covers both paths. It manages LLM providers, App LLM proxies, MCP proxies, and their credentials across connected gateways. In release 2026.09.24, a gateway keeps serving traffic if Choreo Agent Control Plane becomes unreachable, and the sync catches up once the connection is restored. Changes made in Choreo Agent Control Plane during the outage, such as new policies or revoked credentials, do not reach the gateway until then.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>Function calling and tool calling are one mechanism under two names. The model proposes a structured call, and your application decides whether to run it. OpenAI and Anthropic implement the same loop with different field shapes, and the response side needs the most adapter code. MCP sits beside function calling. It standardizes tool discovery and invocation, while the model still emits ordinary tool calls.</p>
<p>Start with native function calling, good schemas, and strict validation. Add MCP when tools need to be shared across applications or agents. Move translation and policy into a gateway when you add a second provider. The same applies when cost, compliance, and access control must work the same way across every model and tool. To try the model side, follow the <a href="https://wso2.com/api-platform/docs/ai-gateway/2026.09.24/routing/multi-provider-routing/" target="_blank" rel="noopener noreferrer" class="">multi-provider routing guide</a> to put OpenAI and Anthropic behind one endpoint.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-function-calling-in-llms">What is function calling in LLMs?<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#what-is-function-calling-in-llms" class="hash-link" aria-label="Direct link to What is function calling in LLMs?" title="Direct link to What is function calling in LLMs?" translate="no">​</a></h3>
<p>It is a model's ability to return a structured request for an external function, with a name and JSON arguments. The application runs the function and returns the result.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-the-difference-between-function-calling-and-tool-calling">What is the difference between function calling and tool calling?<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#what-is-the-difference-between-function-calling-and-tool-calling" class="hash-link" aria-label="Direct link to What is the difference between function calling and tool calling?" title="Direct link to What is the difference between function calling and tool calling?" translate="no">​</a></h3>
<p>There is no functional difference. They are two names for the same mechanism. "Tool" is the broader term, and a function is one type of tool.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-an-example-of-a-function-call">What is an example of a function call?<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#what-is-an-example-of-a-function-call" class="hash-link" aria-label="Direct link to What is an example of a function call?" title="Direct link to What is an example of a function call?" translate="no">​</a></h3>
<p>A model asked "Where is order 4417?" returns <code>get_order_status</code> with <code>{"order_id": "4417"}</code>. Your code queries the order system and sends back the status.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="does-mcp-replace-function-calling">Does MCP replace function calling?<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#does-mcp-replace-function-calling" class="hash-link" aria-label="Direct link to Does MCP replace function calling?" title="Direct link to Does MCP replace function calling?" translate="no">​</a></h3>
<p>No. MCP standardizes how tools are discovered and invoked across clients. The model still uses its provider's function calling format to request a tool.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="does-the-model-execute-the-tool">Does the model execute the tool?<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#does-the-model-execute-the-tool" class="hash-link" aria-label="Direct link to Does the model execute the tool?" title="Direct link to Does the model execute the tool?" translate="no">​</a></h3>
<p>No, the model only proposes the call. Your application, or a host acting for it, validates the arguments and executes the tool.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="can-one-application-use-both-openai-and-anthropic-tool-calling">Can one application use both OpenAI and Anthropic tool calling?<a href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp#can-one-application-use-both-openai-and-anthropic-tool-calling" class="hash-link" aria-label="Direct link to Can one application use both OpenAI and Anthropic tool calling?" title="Direct link to Can one application use both OpenAI and Anthropic tool calling?" translate="no">​</a></h3>
<p>Yes, but the request and response shapes differ. You either maintain an adapter for each provider or route through a gateway that translates between them.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="MCP" term="MCP"/>
        <category label="Comparison" term="Comparison"/>
        <category label="Agent-ready APIs" term="Agent-ready APIs"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[Agent-ready APIs: What they are and how to build them]]></title>
        <id>https://choreolabs.com/api-management/learn/what-are-agent-ready-apis</id>
        <link href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis"/>
        <updated>2026-10-13T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[What makes an API agent-ready: clear semantics, machine readable exposure, governance, and observability, plus current standards and how to implement each.]]></summary>
        <content type="html"><![CDATA[<p>An agent is asked to refund a duplicate charge. It reads your API catalog and finds three operations named <code>create</code>, <code>update</code>, and <code>process</code>. Each one is described as "Handles payment records." The agent picks <code>process</code>, sends the amount in dollars where the API expects cents, and gets back <code>400 Bad Request</code> with an empty body. A developer in the same spot would read the docs, ask a teammate, or open the source code. The agent can only retry, guess again, or give up.</p>
<p>Most enterprise APIs were designed for developers who fill in missing context on their own. AI agents work only from what the API states. This guide defines what makes an API agent-ready and breaks it into four requirements you can build. It then maps each requirement to current standards and shows where to start. It is the pillar article in the broader <a class="" href="https://choreolabs.com/api-management/learn/agent-ready-apis">agent-ready API series</a>.</p>
<aside class="cta_6F_m" aria-label="Make your APIs ready for AI agents" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Make your APIs ready for AI agents</span><span class="desc_GzK8">Use Choreo to apply governance, discovery, and observability across your API catalog for agent traffic.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=what-are-agent-ready-apis" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-does-agent-ready-actually-mean">What does "agent-ready" actually mean?<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#what-does-agent-ready-actually-mean" class="hash-link" aria-label="Direct link to What does &quot;agent-ready&quot; actually mean?" title="Direct link to What does &quot;agent-ready&quot; actually mean?" translate="no">​</a></h2>
<p>An agent-ready API is an API that AI agents can discover, understand, call, and evaluate without a human filling in the gaps. Its purpose, inputs, outputs, errors, and permissions are stated in a machine-readable form. Agent-readiness is a deliberate design property. It is different from an API that happens to work when an agent calls it. Agent-ready does not mean unsupervised. An API can be agent-ready and still require human approval for risky actions, such as payments or deletions. The MCP specification itself recommends that a person can deny tool invocations. The goal is to remove manual integration work, not oversight.</p>
<p>The difference shows up when no one is guiding the agent. An API with decent docs can work in a demo, because a developer wrote the prompt and chose the endpoint. The same API breaks down when an agent must choose among dozens of operations, recover from an error, or decide whether a retry is safe.</p>
<p>Three related terms are often used as if they were the same thing:</p>
<ul>
<li class=""><strong>Agent-ready API:</strong> A property of the interface. Any API can become agent-ready, whatever runs behind it.</li>
<li class=""><strong>AI-ready API:</strong> A broader label for APIs prepared for AI consumption in general. That includes LLM-backed applications where a developer still wires up each call by hand.</li>
<li class=""><strong>Agentic API:</strong> A term used inconsistently. Some teams mean an API designed for agents. Others mean an API that is itself backed by an agent. If you use it, say which meaning you intend.</li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="why-this-matters-now">Why this matters now<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#why-this-matters-now" class="hash-link" aria-label="Direct link to Why this matters now" title="Direct link to Why this matters now" translate="no">​</a></h2>
<p>The primary consumer of many APIs is changing faster than API design practice. Postman's <a href="https://voyager.postman.com/doc/postman-state-of-the-api-report-2025.pdf" target="_blank" rel="noopener noreferrer" class="">2025 State of the API Report</a> surveyed more than 5,700 developers and API professionals. It found that 89% of developers use AI, but only 24% design APIs with AI agents in mind. The same survey found that about half of developers (51%) named unauthorized or excessive API calls from agents as their top security concern.</p>
<p>Agents are a distinct class of consumer, not a faster developer. They have no intuition about your domain, and they cannot read between the lines. Traditional integrations already call APIs at machine speed. What is new is that an agent chooses which operations to call at runtime, and its plan can change from one run to the next.</p>
<table><thead><tr><th></th><th>Human developer</th><th>AI agent</th></tr></thead><tbody><tr><td>How it learns the API</td><td>Reads the docs once and builds a mental model</td><td>Reads names, descriptions, and schemas in its context window when it decides on a call</td></tr><tr><td>When a name or field is ambiguous</td><td>Asks a teammate or reads the source</td><td>Picks the most plausible reading of the text it has</td></tr><tr><td>Call sequence</td><td>Fixed in code that the developer writes and tests</td><td>Chosen at runtime, and can change from one run to the next</td></tr><tr><td>Identity</td><td>A person or a registered application</td><td>Software acting on behalf of a user or a system, which needs its own delegated identity</td></tr></tbody></table>
<p>These differences change where the cost of a weak API shows up. With human consumers, a confusing endpoint costs developer time once, during integration. With agents, the same confusion can recur on every run, because the agent rereads the same ambiguous text each time it plans a call. Fixing the contract once removes a failure that would otherwise repeat in production.</p>
<p>None of this calls for a new kind of API. It calls for making explicit what human developers used to infer.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-four-requirements-of-an-agent-ready-api">The four requirements of an agent-ready API<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#the-four-requirements-of-an-agent-ready-api" class="hash-link" aria-label="Direct link to The four requirements of an agent-ready API" title="Direct link to The four requirements of an agent-ready API" translate="no">​</a></h2>
<p>An API becomes agent-ready through four properties. Each one answers a question the agent cannot answer for itself.</p>
<table><thead><tr><th>Requirement</th><th>The question it answers for the agent</th><th>What it covers</th></tr></thead><tbody><tr><td>Clear semantics</td><td>What does this do, and what went wrong?</td><td>Names, descriptions, schemas, and error responses</td></tr><tr><td>Machine-readable exposure</td><td>How do I find this capability and call it?</td><td>A discoverable interface the agent runtime can call, such as MCP tools</td></tr><tr><td>Governance</td><td>Am I allowed to do this, and how often?</td><td>Identity, scopes, limits, and policy on side effects</td></tr><tr><td>Observability</td><td>What did I actually do?</td><td>Traces that follow a task across many tool calls</td></tr></tbody></table>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="clear-semantics">Clear semantics<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#clear-semantics" class="hash-link" aria-label="Direct link to Clear semantics" title="Direct link to Clear semantics" translate="no">​</a></h3>
<p>An agent chooses an operation by reading text. As far as the model is concerned, operation names, descriptions, parameter schemas, and error bodies are the entire interface. Anything a human would learn from a hallway conversation has to be in the contract.</p>
<p>Compare these two partial OpenAPI fragments for the same operation. The first leaves the agent to guess.</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token key atrule" style="color:#3D8BFF">paths</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">/invoices</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">post</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">operationId</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> create</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">summary</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Create</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">requestBody</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token key atrule" style="color:#3D8BFF">content</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token key atrule" style="color:#3D8BFF">application/json</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">            </span><span class="token key atrule" style="color:#3D8BFF">schema</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">              </span><span class="token key atrule" style="color:#3D8BFF">type</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> object</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">              </span><span class="token key atrule" style="color:#3D8BFF">properties</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                </span><span class="token key atrule" style="color:#3D8BFF">amount</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                  </span><span class="token key atrule" style="color:#3D8BFF">type</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> number</span><br></span></code></pre></div></div>
<p>The second states the purpose, the side effect, the units, and the constraints.</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token key atrule" style="color:#3D8BFF">paths</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  /customers/</span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain">customerId</span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain">/invoices</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">post</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">operationId</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> createCustomerInvoice</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">summary</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Create a draft invoice for an existing customer</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">&gt;</span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        Creates an invoice in DRAFT status. This does not charge the customer.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        Call sendCustomerInvoice to deliver the invoice.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">parameters</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain"> </span><span class="token key atrule" style="color:#3D8BFF">name</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> customerId</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token key atrule" style="color:#3D8BFF">in</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> path</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token key atrule" style="color:#3D8BFF">required</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token boolean important" style="color:#3D8BFF">true</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> ID of an existing customer</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> for example cus_123.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token key atrule" style="color:#3D8BFF">schema</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">            </span><span class="token key atrule" style="color:#3D8BFF">type</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> string</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">requestBody</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token key atrule" style="color:#3D8BFF">required</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token boolean important" style="color:#3D8BFF">true</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token key atrule" style="color:#3D8BFF">content</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">          </span><span class="token key atrule" style="color:#3D8BFF">application/json</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">            </span><span class="token key atrule" style="color:#3D8BFF">schema</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">              </span><span class="token key atrule" style="color:#3D8BFF">type</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> object</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">              </span><span class="token key atrule" style="color:#3D8BFF">required</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token plain">amountMinor</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> currency</span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">              </span><span class="token key atrule" style="color:#3D8BFF">properties</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                </span><span class="token key atrule" style="color:#3D8BFF">amountMinor</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                  </span><span class="token key atrule" style="color:#3D8BFF">type</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> integer</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                  </span><span class="token key atrule" style="color:#3D8BFF">minimum</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">1</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                  </span><span class="token key atrule" style="color:#3D8BFF">maximum</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">99999999</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                  </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Whole number in the currency's minor unit</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                    for example cents for USD. Fractions are rejected</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> not rounded.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                </span><span class="token key atrule" style="color:#3D8BFF">currency</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                  </span><span class="token key atrule" style="color:#3D8BFF">type</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> string</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                  </span><span class="token key atrule" style="color:#3D8BFF">enum</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token plain">USD</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> EUR</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> GBP</span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                  </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> ISO 4217 code. Only USD</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> EUR</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> and GBP are</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">                    accepted.</span><br></span></code></pre></div></div>
<p>Both fragments are partial. A complete operation also documents its success response and each error response. For money fields, state the allowed currencies, the unit, the precision, the bounds, and whether values are rounded or rejected. A three-letter pattern only checks the shape of a currency code. It does not tell the agent which codes the API accepts. Add examples for fields with a non-obvious format, and list enum values explicitly. An example shows the expected format directly, so the agent does not have to infer it from a sentence of prose. For a full walkthrough of these fields, see <a class="" href="https://choreolabs.com/api-management/learn/openapi-for-ai-agents">OpenAPI for AI agents</a>.</p>
<p>The same discipline applies to errors. A bare <code>400</code> gives the agent nothing to correct. A structured error body that names the field, the rule, and the received value lets it fix the call and try again. <a href="https://www.rfc-editor.org/rfc/rfc9457" target="_blank" rel="noopener noreferrer" class="">RFC 9457</a> defines a standard format for this, called problem details, served as <code>application/problem+json</code>.</p>
<p>Treat the specification as a live contract rather than a documentation artifact. Lint it in CI and fail the build when an operation lacks a description or an error response is undocumented. When the spec drifts from the implementation, an agent is the first consumer to fail, because it trusts the spec completely.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="machine-readable-exposure">Machine-readable exposure<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#machine-readable-exposure" class="hash-link" aria-label="Direct link to Machine-readable exposure" title="Direct link to Machine-readable exposure" translate="no">​</a></h3>
<p>An agent needs a machine-readable, discoverable interface that its runtime can use. MCP is one way to provide it. <a class="" href="https://choreolabs.com/api-management/learn/tool-calling-vs-mcp">Tool or function definitions</a> in the agent's own framework, generated from your OpenAPI description, are another. The Model Context Protocol (MCP) is an open protocol that lets AI applications discover and call external tools. A client asks an MCP server for its tools with <code>tools/list</code>. Each tool has a <code>name</code>, a <code>description</code>, and an <code>inputSchema</code>. It can also declare an <code>outputSchema</code> and annotations that describe its behavior (<a href="https://modelcontextprotocol.io/specification/2026-07-28/server/tools" target="_blank" rel="noopener noreferrer" class="">MCP specification, tools</a>). The client then invokes a tool with <code>tools/call</code>.</p>
<p>Of the four requirements, exposure is the one where a converged standard, MCP, has emerged. Since December 2025, MCP has been stewarded by the Agentic AI Foundation under the <a href="https://linuxfoundation.org/press/linux-foundation-announces-the-formation-of-the-agentic-ai-foundation" target="_blank" rel="noopener noreferrer" class="">Linux Foundation</a>. The other requirements still draw on several competing or partial approaches.</p>
<p>MCP does not replace your REST API. Applications keep calling the REST API. MCP tools are a curated surface for agents over capabilities you already own (<a class="" href="https://choreolabs.com/api-management/learn/rest-apis-vs-mcp">REST API vs MCP</a> covers when to add that layer). The design question is which tools to offer, and the answer is rarely "one per endpoint." A tool such as <code>get_order_status</code> that returns the order, its shipment, and any open refund is easier for an agent to use than three tools that each return one part. Anthropic's engineering guidance on tools for agents makes the same point. It recommends choosing tools deliberately and consolidating related operations instead of wrapping every endpoint (<a href="https://www.anthropic.com/engineering/writing-tools-for-agents" target="_blank" rel="noopener noreferrer" class="">Anthropic, Writing effective tools for agents</a>). For a comparison of the wider integration patterns, see <a class="" href="https://choreolabs.com/api-management/learn/apis-discoverable-to-ai-agents">AI agent API integration</a>.</p>
<p>Two details in the specification matter for safety and recovery:</p>
<ul>
<li class=""><strong>Annotations are hints, not controls.</strong> Annotations such as <code>readOnlyHint</code> and <code>destructiveHint</code> describe intended behavior. Clients must treat them as untrusted unless the server is trusted. Enforce read-only and destructive boundaries in authorization, not in tool metadata.</li>
<li class=""><strong>Validation errors go back to the model.</strong> The 2025-11-25 revision clarified that input validation failures should be returned as tool execution errors, with <code>isError: true</code>, rather than as protocol errors. That lets the model see the problem and correct its call.</li>
</ul>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="governance">Governance<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#governance" class="hash-link" aria-label="Direct link to Governance" title="Direct link to Governance" translate="no">​</a></h3>
<p>An agent acts on behalf of someone. Governance answers two questions for every call: who is calling, and what are they allowed to do?</p>
<ul>
<li class=""><strong>Authentication that works without a person present.</strong> Login flows that depend on a browser session or a CAPTCHA stall an unattended agent. When the agent acts under its own identity, the OAuth 2.0 <a href="https://www.rfc-editor.org/info/rfc6749/" target="_blank" rel="noopener noreferrer" class="">client credentials grant</a> issues it a token as a client. When it acts for a user, use the authorization code flow with <a href="https://www.rfc-editor.org/info/rfc7636" target="_blank" rel="noopener noreferrer" class="">PKCE</a>, which protects the authorization code from interception. The <a href="https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization" target="_blank" rel="noopener noreferrer" class="">MCP authorization specification</a> builds on these. It requires MCP clients to use PKCE and to send a resource indicator. The resource indicator binds each token to the MCP server it is meant for.</li>
<li class=""><strong>Scopes per action, not per API.</strong> Separate read access from write access. A token that can list invoices should not be able to void them. Map scopes to individual operations or tools so that a narrow task gets a narrow token. <a class="" href="https://choreolabs.com/api-management/learn/agent-api-permissions">Permission models for AI agents</a> covers this in depth.</li>
<li class=""><strong>Limits sized for loops.</strong> Apply rate limits and quotas per agent identity, not only per application. A misconfigured agent can retry the same call far faster than a person would notice.</li>
<li class=""><strong>Explicit side effects.</strong> State in the contract whether an operation changes data. On write operations, accept a client-supplied <a class="" href="https://choreolabs.com/api-management/learn/designing-idempotent-tools">idempotency key</a> and return the original result when the same key arrives again. Bind each key to the operation and its parameters, and reject a request that reuses a key with different parameters. Without that, a timeout followed by a retry can create a second payment.</li>
<li class=""><strong>Policy enforced outside the model.</strong> Tool descriptions and system prompts guide behavior, but they do not control access. Enforce permissions where the call is received, at the API or at a gateway in front of it.</li>
</ul>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="observability">Observability<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#observability" class="hash-link" aria-label="Direct link to Observability" title="Direct link to Observability" translate="no">​</a></h3>
<p>A request log can tell you that <code>POST /refunds</code> returned <code>201</code>. It cannot tell you why the agent called it, what the agent read first, or whether the agent called it twice. <a class="" href="https://choreolabs.com/api-management/learn/agent-observability">Agent observability</a> means following the task, not the single request.</p>
<p>In practice, that requires a few things:</p>
<ul>
<li class="">One trace per task. Propagate a trace ID across every tool call in a multi-step task. The <a href="https://www.w3.org/TR/trace-context/" target="_blank" rel="noopener noreferrer" class="">W3C Trace Context</a> <code>traceparent</code> header is the common format for carrying it between services.</li>
<li class="">Identity on every span. Record the agent's identity and, where relevant, the user it acts for.</li>
<li class="">What was called, and with what. Capture the tool or operation name, the arguments with sensitive values redacted, the result status, and the latency.</li>
<li class="">Model calls correlated, not merged. Calls to the LLM and calls to your tools are separate traffic with separate costs. Keep them as separate spans in the same trace, so you can see which model turn led to which tool call.</li>
</ul>
<p>A trace built this way answers the questions that come up in an incident review. Which user asked for this refund? Which tool results did the agent read before it acted, and did a guardrail change what the model saw? How much did the task cost in model tokens? When those answers live in separate systems, every review becomes a manual join across log stores.</p>
<p>Treat agent traces as audit data. Tool arguments and results can contain personal information, so apply the same retention and access rules you use for other sensitive logs.</p>
<p><img decoding="async" loading="lazy" alt="The four requirements of an agent-ready API, clear semantics, machine-readable exposure, governance, and observability, sit on top of your existing APIs. No new kind of API is needed, just explicit contracts" src="https://choreolabs.com/api-management/learn/assets/images/four-requirements-of-an-agent-ready-api-fc43f66a7931893bf0d3111225c7fc5f.webp" width="3200" height="1748" class="img_ev3q"></p>
<p><em>An agent-ready API is discoverable, understandable, callable, and accountable, with human approval where risk requires it.</em></p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-emerging-standards-landscape">The emerging standards landscape<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#the-emerging-standards-landscape" class="hash-link" aria-label="Direct link to The emerging standards landscape" title="Direct link to The emerging standards landscape" translate="no">​</a></h2>
<p>Each requirement draws on standards at different stages of maturity. Some have been stable for years. Others changed in the past few months. A published version does not mean every SDK or gateway supports it yet.</p>
<table><thead><tr><th>Standard</th><th>Used for</th><th>Steward</th><th>Status in October 2026</th></tr></thead><tbody><tr><td>OpenAPI</td><td>API contracts and schemas</td><td>OpenAPI Initiative</td><td>Stable. <a href="https://spec.openapis.org/oas/v3.2.1.html" target="_blank" rel="noopener noreferrer" class="">Version 3.2.1</a> is the current patch release. Version 3.2.0 was released in September 2025</td></tr><tr><td>Arazzo</td><td>Multi-step API workflows</td><td>OpenAPI Initiative</td><td><a href="https://spec.openapis.org/arazzo/latest" target="_blank" rel="noopener noreferrer" class="">Version 1.1.0</a> is current</td></tr><tr><td>llms.txt</td><td>An entry point for LLMs reading a site</td><td>Community proposal</td><td><a href="https://llmstxt.org/" target="_blank" rel="noopener noreferrer" class="">A proposal</a>, not a ratified standard</td></tr><tr><td>MCP</td><td>Exposing tools to agents</td><td>Agentic AI Foundation</td><td><a href="https://github.com/modelcontextprotocol/specification/releases/tag/2026-07-28" target="_blank" rel="noopener noreferrer" class="">Revision 2026-07-28</a> is the current revision</td></tr><tr><td>A2A</td><td>Agent-to-agent delegation</td><td>Agentic AI Foundation</td><td><a href="https://a2a-protocol.org/latest/specification/" target="_blank" rel="noopener noreferrer" class="">Version 1.0</a> was released in March 2026. <a href="https://github.com/a2aproject/A2A/releases" target="_blank" rel="noopener noreferrer" class="">Version 1.0.1</a> is the latest release</td></tr><tr><td>OAuth 2.0 with PKCE</td><td>Delegated authorization</td><td>IETF</td><td>Stable RFCs. MCP authorization servers must implement OAuth 2.1</td></tr></tbody></table>
<p>MCP shows how fast this layer is still moving. Under the 2026-07-28 revision, every request carries its own protocol version and client capabilities in request metadata. If you publish MCP servers, pin the revision you target and test your clients against it. A2A plays a different role from MCP. MCP connects an agent to tools and data, while A2A lets one agent discover another through an Agent Card and delegate a task to it.</p>
<p>There are also community efforts to describe agent-readiness as a whole. <a href="https://agentready.org/" target="_blank" rel="noopener noreferrer" class="">AgentReady</a> is one attempt. It is an MIT-licensed specification from Ora, Vercel, Mintlify, Auth0, and Stripe. It groups practices into find, read, and act, and assigns each a MUST, SHOULD, or MAY level. One finding from its agent studies is practical for API teams. Agents reached agent-facing files such as llms.txt mostly by following links, not by guessing paths. Publishing a discovery file is not enough on its own. Link it from pages that agents already visit, such as your docs.</p>
<p>AgentReady covers much more than APIs, including crawler access, payments, and in-page tools. The four requirements in this guide are a narrower cut, aimed at the teams that own and operate APIs.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-happens-when-an-api-isnt-agent-ready">What happens when an API isn't agent-ready<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#what-happens-when-an-api-isnt-agent-ready" class="hash-link" aria-label="Direct link to What happens when an API isn't agent-ready" title="Direct link to What happens when an API isn't agent-ready" translate="no">​</a></h2>
<p>When one of the four requirements is missing, the failure rarely looks like an outage. It looks like an agent that does the wrong thing quietly.</p>
<ol>
<li class=""><strong>The capability is invisible:</strong> If an API is not published with clear metadata in a place agents can find, then for an agent it does not exist. The agent works around it or reports that the task is impossible.</li>
<li class=""><strong>The call is malformed:</strong> An undocumented enum is a common cause. Your API expects <code>status=OPEN</code>, the agent sends <code>status=open</code>, and the filter silently returns nothing. The agent then reports, with confidence, that there are no open orders.</li>
<li class=""><strong>The error is a dead end:</strong> A <code>500</code> or an empty <code>400</code> gives the agent no signal. It repeats the same call, tries an unrelated operation, or stops. Each of these is worse than a clear error it could act on.</li>
<li class=""><strong>The side effect happens twice:</strong> A payment call times out, the agent retries, and both requests succeed. Without idempotency, the API has no way to recognize the retry.</li>
<li class=""><strong>No one can explain what happened:</strong> After an incident, the logs show a set of unrelated requests. Without a shared trace and an agent identity, nobody can reconstruct the sequence or say which user it acted for.</li>
</ol>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-to-start-making-your-apis-agent-ready">How to start making your APIs agent-ready<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#how-to-start-making-your-apis-agent-ready" class="hash-link" aria-label="Direct link to How to start making your APIs agent-ready" title="Direct link to How to start making your APIs agent-ready" translate="no">​</a></h2>
<p>You do not need to change every API at once. Start where agents already create real operational pressure. These are the usual signals that it is time to act:</p>
<ul>
<li class="">Coding assistants or internal copilots already call your APIs.</li>
<li class="">Teams use more than one model or provider and need the same policy across all of them.</li>
<li class="">Finance or platform teams need to attribute model spend to applications or teams.</li>
<li class="">Prompts or tool results can contain personal or regulated data.</li>
<li class="">Agents perform writes in production, not only reads.</li>
</ul>
<p>Once one of those applies, work through the requirements in a fixed order:</p>
<ol>
<li class="">Pick a small set of capabilities. Choose the tasks agents most need, not the whole API estate.</li>
<li class="">Repair the contract. Rewrite operation IDs and descriptions, document units and enums, and add structured error bodies. Add spec linting to CI so the contract stays clean.</li>
<li class="">Publish for discovery. List the APIs in a catalog that serves machine-readable specs. Link an llms.txt index from your docs. Match discovery to the audience, so internal agents find internal APIs through authenticated discovery. Hiding an API from an index is not access control, so enforce authorization on every call.</li>
<li class="">Design the tool layer. Define a few task-shaped tools over those capabilities, for example as MCP tools. Test them with real agent runs and refine the descriptions where agents choose wrongly.</li>
<li class="">Put policy at the edge. Give each agent its own identity, scope tokens per tool, set per-agent limits, and require idempotency keys on writes.</li>
<li class="">Trace from the first call. Turn on task-level tracing before the first production rollout, not after the first incident.</li>
</ol>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-choreo-api-management-makes-apis-agent-ready">How Choreo API Management makes APIs agent-ready<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#how-choreo-api-management-makes-apis-agent-ready" class="hash-link" aria-label="Direct link to How Choreo API Management makes APIs agent-ready" title="Direct link to How Choreo API Management makes APIs agent-ready" translate="no">​</a></h2>
<p>Choreo API Management manages APIs and AI traffic from a single control plane. Choreo Labs (formerly WSO2) launched Choreo API Management in March 2026 with agent-readiness as its stated goal. It is built to control traditional APIs and AI assets, such as models, MCP servers, and prompts, together. Each of the four requirements maps to a specific part of Choreo API Management.</p>
<p><img decoding="async" loading="lazy" alt="Choreo API Management control plane with Choreo API Portal and AI Hub, Choreo Agent Control Plane, Analytics, and API Governance. The control plane sends policy to Choreo AI Gateway and collects logs, traces, and analytics. In Choreo AI Gateway, tool calls from an AI agent go through MCP policies to MCP servers, and model calls go through LLM policies to LLM providers" src="https://choreolabs.com/api-management/learn/assets/images/wso2-api-platform-agent-ready-architecture-a0db4922064126323c02f769d92093fb.webp" width="2576" height="1446" class="img_ev3q"></p>
<p><em>Tool calls and model calls take independent paths under separate policies. Neither passes through the other.</em></p>
<p>The two paths through the AI Gateway in the diagram are independent. A tool call travels from the agent to an MCP server under MCP policies. A model call travels from the agent to an LLM provider under LLM policies. Neither path passes through the other. The control plane distributes policy to the gateway and collects analytics from both paths.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="clear-semantics-rules-on-the-contract">Clear semantics: rules on the contract<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#clear-semantics-rules-on-the-contract" class="hash-link" aria-label="Direct link to Clear semantics: rules on the contract" title="Direct link to Clear semantics: rules on the contract" translate="no">​</a></h3>
<p>API Governance validates OpenAPI and AsyncAPI definitions against Spectral-based rulesets. Rulesets are grouped into policies, and results appear on a compliance dashboard. Teams can also check APIs against guideline documents written in natural language, using AI-assisted analysis (<a href="https://wso2.com/api-platform/docs/cloud/governance/overview/" target="_blank" rel="noopener noreferrer" class="">API Governance overview</a>). Violations, such as a missing description, are flagged on the compliance dashboard when the API is updated, rather than surfacing as a failure in an agent run. Governance policies currently notify rather than block, so add a blocking check to your CI/CD pipeline if a release must stop on a violation.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="mcp-exposure-and-discovery-one-catalog-for-people-and-agents">MCP exposure and discovery: one catalog for people and agents<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#mcp-exposure-and-discovery-one-catalog-for-people-and-agents" class="hash-link" aria-label="Direct link to MCP exposure and discovery: one catalog for people and agents" title="Direct link to MCP exposure and discovery: one catalog for people and agents" translate="no">​</a></h3>
<p>Choreo API Portal and AI Hub generates an llms.txt index on request. It also serves the catalog, documentation, and specifications as Markdown and raw specification files. Publishers can hide an individual API from agent-facing surfaces while keeping it visible to human developers in the portal (<a href="https://wso2.com/api-platform/docs/api-portal/next/ai-agent-discovery/" target="_blank" rel="noopener noreferrer" class="">AI agent discovery</a>). Visibility controls what an agent can find, not what it can call, so authorization still applies at the gateway. MCP servers published to the AI Hub appear in the same index, so agents and developers read from one source.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="governance-policy-at-the-ai-gateway">Governance: policy at the AI gateway<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#governance-policy-at-the-ai-gateway" class="hash-link" aria-label="Direct link to Governance: policy at the AI gateway" title="Direct link to Governance: policy at the AI gateway" translate="no">​</a></h3>
<p>For tool traffic, the AI Gateway applies three MCP policies. MCP Authentication follows the MCP specification's authorization profile. MCP Authorization checks access to specific tools, resources, and prompts using JWT claims or OAuth scopes. MCP Access Control filters the tool list with allow and deny rules, so a filtered tool is never offered to the caller (<a href="https://wso2.com/api-platform/docs/ai-gateway/next/mcp-governance/" target="_blank" rel="noopener noreferrer" class="">MCP governance</a>). For model traffic, the same gateway applies guardrails such as PII masking and JSON Schema validation, token-based rate limits, and cost-based budgets. It can also front A2A agents and serve their Agent Cards.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="observability-one-view-across-api-llm-and-mcp-traffic">Observability: one view across API, LLM, and MCP traffic<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#observability-one-view-across-api-llm-and-mcp-traffic" class="hash-link" aria-label="Direct link to Observability: one view across API, LLM, and MCP traffic" title="Direct link to Observability: one view across API, LLM, and MCP traffic" translate="no">​</a></h3>
<p>The gateway produces logs, traces, and analytics. MCP analytics show tool usage, agent sessions, and the distribution of clients. LLM analytics show token usage for model traffic. Platform-level views break traffic down across APIs, LLM proxies, and MCP (<a href="https://wso2.com/api-platform/docs/guides/ai-and-mcp/observe-mcp-traffic/" target="_blank" rel="noopener noreferrer" class="">Observe MCP traffic on the AI Gateway</a>). Because the views share one control plane, a team can review API, model, and tool traffic in one place.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>An API becomes agent-ready through four properties you can build and test. Those are clear semantics, machine-readable exposure, governance designed for automated callers, and observability into what agents actually did. A label does not make an API agent-ready. A contract an agent can read without help does.</p>
<p>Start with one capability that agents already need. Fix its contract, publish it where agents look, put policy in front of it, and trace every call. To see how these pieces work together under a single control plane, explore <a href="https://wso2.com/api-platform/" target="_blank" rel="noopener noreferrer" class="">Choreo API Management</a>.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="is-agent-ready-the-same-as-ai-ready">Is agent-ready the same as AI-ready?<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#is-agent-ready-the-same-as-ai-ready" class="hash-link" aria-label="Direct link to Is agent-ready the same as AI-ready?" title="Direct link to Is agent-ready the same as AI-ready?" translate="no">​</a></h3>
<p>No. Agent-ready is narrower. It means an autonomous agent can find, call, and recover from errors on an API without a person interpreting the API for it.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="do-i-need-mcp-if-agents-can-already-call-my-rest-api">Do I need MCP if agents can already call my REST API?<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#do-i-need-mcp-if-agents-can-already-call-my-rest-api" class="hash-link" aria-label="Direct link to Do I need MCP if agents can already call my REST API?" title="Direct link to Do I need MCP if agents can already call my REST API?" translate="no">​</a></h3>
<p>Not always. Agents can call well-described REST APIs directly. MCP adds a standard way to discover tools and a task-shaped surface designed for model use.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="is-llmstxt-an-official-standard">Is llms.txt an official standard?<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#is-llmstxt-an-official-standard" class="hash-link" aria-label="Direct link to Is llms.txt an official standard?" title="Direct link to Is llms.txt an official standard?" translate="no">​</a></h3>
<p>No. It is a community proposal from 2024. Publishing one takes little effort and gives agents a clear place to start.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="can-an-agent-use-the-same-api-key-as-the-application-it-runs-in">Can an agent use the same API key as the application it runs in?<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#can-an-agent-use-the-same-api-key-as-the-application-it-runs-in" class="hash-link" aria-label="Direct link to Can an agent use the same API key as the application it runs in?" title="Direct link to Can an agent use the same API key as the application it runs in?" translate="no">​</a></h3>
<p>It can, but you lose the ability to tell the agent's actions apart. A separate identity per agent makes per-agent scopes, limits, and audit trails possible.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="where-should-agent-permissions-be-enforced">Where should agent permissions be enforced?<a href="https://choreolabs.com/api-management/learn/what-are-agent-ready-apis#where-should-agent-permissions-be-enforced" class="hash-link" aria-label="Direct link to Where should agent permissions be enforced?" title="Direct link to Where should agent permissions be enforced?" translate="no">​</a></h3>
<p>At the point where the call is received, such as the API or a gateway in front of it. Prompts and tool descriptions are not access controls.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="AI agents" term="AI agents"/>
        <category label="Agent-ready APIs" term="Agent-ready APIs"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[Multi-agent calendar assistant: A real-world case study]]></title>
        <id>https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study</id>
        <link href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study"/>
        <updated>2026-10-13T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[How ScheduleMe, a supervisor-and-specialists calendar assistant, performed across six languages: architecture, results, failure modes, and lessons]]></summary>
        <content type="html"><![CDATA[<p>Calendar management looks like an easy job for an AI agent until you test it. Consider a request like "move tomorrow's design review to Thursday afternoon." The system has to find the right event, check what Thursday afternoon already holds, and edit one record without touching a similar one. ScheduleMe is a research system built to handle requests like this through conversation. Researchers at the University of Moratuwa built it, with Srinath Perera of Choreo Labs as a co-author, and the paper appeared at PACLIC 39. This case study covers the architecture, the scaling extension, both evaluations, and the failure modes the authors documented. The results below come from the paper, including the ones that fell short.</p>
<aside class="cta_6F_m" aria-label="Secure multi-agent systems before you scale" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Secure multi-agent systems before you scale</span><span class="desc_GzK8">Use Choreo to secure agent-to-agent deployments with audit logging and agent-scoped authorization.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=multi-agent-calendar-case-study" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-scheduleme-is-and-why-its-a-useful-multi-agent-example">What ScheduleMe is and why it's a useful multi-agent example<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#what-scheduleme-is-and-why-its-a-useful-multi-agent-example" class="hash-link" aria-label="Direct link to What ScheduleMe is and why it's a useful multi-agent example" title="Direct link to What ScheduleMe is and why it's a useful multi-agent example" translate="no">​</a></h2>
<p>ScheduleMe manages Google Calendar events through natural-language conversation. A supervisor agent talks to the user and hands each calendar operation to one of four specialist agents. The paper is "ScheduleMe: Multi-Agent Calendar Assistant." Four of its authors are from the University of Moratuwa: Oshadha Wijerathne, Amandi Nimasha, Dushan Fernando, and Nisansa de Silva. The fifth is Srinath Perera, who leads Choreo Labs' research team. It is published in the <a href="https://aclanthology.org/2025.paclic-1.27/" target="_blank" rel="noopener noreferrer" class="">PACLIC 39 proceedings</a> and on <a href="https://arxiv.org/abs/2509.25693" target="_blank" rel="noopener noreferrer" class="">arXiv</a>.</p>
<p>ScheduleMe is not a Choreo Labs product. It runs on an open-source Python stack with OpenAI's GPT-4o mini for reasoning. We present it here as a worked example of multi-agent design tradeoffs.</p>
<p>The calendar domain suits that purpose for three reasons:</p>
<ul>
<li class="">Every operation has a checkable outcome. An event was created with the right title and time, or it was not. That makes task success measurable.</li>
<li class="">The operations have real side effects. Editing and deleting change a user's data, so routing mistakes have visible costs.</li>
<li class="">The inputs are ambiguous in familiar ways. Relative dates, similar event titles, and multilingual phrasing all stress the coordination layer.</li>
</ul>
<p>The paper positions ScheduleMe against traditional task-oriented dialogue systems. Those systems rely on intent classification and slot filling, usually through rigid, frame-based flows. The authors note that such flows struggle when users give details out of order, revise a command, or phrase things unexpectedly. ScheduleMe replaces the fixed flow with an LLM supervisor that asks follow-up questions until it has what each operation needs.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-architecture-one-supervisor-four-specialists">The architecture: One supervisor, four specialists<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#the-architecture-one-supervisor-four-specialists" class="hash-link" aria-label="Direct link to The architecture: One supervisor, four specialists" title="Direct link to The architecture: One supervisor, four specialists" translate="no">​</a></h2>
<p>The supervisor is the only agent that talks to the user. It reads each message, works out the intent, and collects the required details before delegating. The four specialists handle scheduling, availability checking, event editing, and event deletion. All communication between agents passes through the supervisor. When a specialist needs more information, it asks the supervisor to go back to the user.</p>
<p><img decoding="async" loading="lazy" alt="ScheduleMe architecture. User requests travel from the Streamlit UI through the FastAPI backend to the supervisor agent in a LangGraph StateGraph, which keeps a JSON state file for multi-turn context. The supervisor delegates one task at a time to four ReAct, stateless specialists: scheduling, availability checking, event editing, and event deletion. Each specialist calls only its own tools against the Google Calendar API with OAuth 2.0 user consent. Model calls go out separately from the supervisor and specialists to GPT-4o mini via LangChain" src="https://choreolabs.com/api-management/learn/assets/images/scheduleme-architecture-7c0ef2a9d3637a62f24fa2d544121df8.webp" width="3200" height="1958" class="img_ev3q"></p>
<p><em>Figure 1: ScheduleMe's supervisor-and-specialists architecture, based on Wijerathne et al. (2025), Figure 1 and Section 3.2.</em></p>
<p>Each specialist follows the <a href="https://arxiv.org/abs/2210.03629" target="_blank" rel="noopener noreferrer" class="">ReAct pattern</a>, which interleaves reasoning steps with tool calls. Its tools are custom functions that wrap the Google Calendar API. The appendix prompts name them directly. The scheduling agent uses <code>check_calendar_conflicts</code> and <code>create_calendar_event</code>, and the availability agent calls <code>check_availability(start_date, end_date)</code>. The editing agent uses <code>update_event</code>, and the deletion agent calls <code>delete_event</code> with an event ID.</p>
<p>The specialists are stateless. Each one receives a structured payload, runs its task, and returns the result to the supervisor. The supervisor then reports the outcome to the user. This is the supervisor/specialist orchestration pattern in practice: one stateful coordinator and several narrow, stateless workers.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-stack">The stack<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#the-stack" class="hash-link" aria-label="Direct link to The stack" title="Direct link to The stack" translate="no">​</a></h3>
<p>The implementation is small enough to list in full:</p>
<table><thead><tr><th>Layer</th><th>Component</th><th>Role in ScheduleMe</th></tr></thead><tbody><tr><td>Coordination</td><td>LangGraph StateGraph</td><td>Directed graph. Nodes are agents and edges are task transitions.</td></tr><tr><td>LLM access</td><td>LangChain with GPT-4o mini</td><td>Intent extraction, clarification, and agent reasoning</td></tr><tr><td>Backend</td><td>FastAPI</td><td>REST endpoints for user interaction and session handling</td></tr><tr><td>Conversation state</td><td>Persistent JSON state file</td><td>Context across multi-turn interactions</td></tr><tr><td>Calendar access</td><td>Google Calendar API with OAuth 2.0 with user consent</td><td>Create, read, modify, and delete events</td></tr><tr><td>Time handling</td><td>pytz and dateparser</td><td>Timezone resolution and natural-language date parsing</td></tr><tr><td>Frontend</td><td>Streamlit</td><td>Chat UI with a live calendar view beside it</td></tr></tbody></table>
<p>Two traffic paths are worth keeping apart when reading this design. User requests travel inward from the Streamlit UI through FastAPI to the agent graph. Model calls travel outward from the agents to GPT-4o mini through LangChain. Calendar operations are a third path, from each specialist's tools to the Google Calendar API.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-routing-contract">The routing contract<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#the-routing-contract" class="hash-link" aria-label="Direct link to The routing contract" title="Direct link to The routing contract" translate="no">​</a></h3>
<p>The supervisor prompt requires a JSON reply with two fields. <code>next</code> names the agent to call, the user, or <code>FINISH</code>. <code>messages</code> holds the instruction or the reply. An illustrative reply in that format looks like this:</p>
<div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"next"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"event_scheduler_agent"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"messages"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"Schedule an event titled 'Design review' on 2025-05-01 at 10:00 AM."</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token punctuation" style="color:#8A93A6">}</span><br></span></code></pre></div></div>
<p>The graph then routes on <code>next</code>. Two details in the published prompts stand out for anyone building a similar system.</p>
<p>First, some safety rules live in prompts rather than code. The scheduling prompt says the agent must check for conflicts before creating an event. That works only while the model follows the instruction. Moving the same rule into the tool layer would make it enforceable. Even an enforced check is not a reservation, though. Two concurrent requests can both see a free slot before either one writes. Production scheduling needs concurrency control at the calendar itself, such as a reservation or a conditional update. It also needs idempotent event creation, so that a retried call does not create a duplicate. This is an engineering inference from the check-then-write flow. The paper did not test for this race.</p>
<p>Second, the agent names in the supervisor prompt are not fully consistent. The agent list includes <code>event_modifier_agent</code> and <code>event_remover_agent</code>. The allowed values for <code>next</code> include <code>event_editor_agent</code> instead and omit the deletion agent. The paper does not say whether the running code shared this mismatch. Still, it shows how easily a prompt and a routing graph can drift apart. Generating both from one definition avoids that class of bug.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="scaling-past-a-single-supervisor">Scaling past a single supervisor<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#scaling-past-a-single-supervisor" class="hash-link" aria-label="Direct link to Scaling past a single supervisor" title="Direct link to Scaling past a single supervisor" translate="no">​</a></h2>
<p>The base design serves one user well. The authors note that real deployments need concurrency, so they added a distributed supervisor architecture. Its components are:</p>
<ul>
<li class=""><strong>Parallel supervisors:</strong> Multiple supervisor instances run at once, each with a unique identifier.</li>
<li class=""><strong>A session-aware load balancer:</strong> A custom balancer routes each session to the least-loaded supervisor. It keeps session affinity and reassigns sessions when an instance fails.</li>
<li class=""><strong>Shared state in Redis:</strong> Redis holds session context and state metadata for all supervisors. Its time-to-live (TTL) setting clears inactive sessions automatically.</li>
<li class=""><strong>A dynamic agent registry:</strong> Agents can be discovered, activated, and deactivated at runtime. Tasks go out through a thread pool executor, with capability-based routing.</li>
<li class=""><strong>An asynchronous backend:</strong> The backend uses async/await so that concurrent conversations do not block the event loop.</li>
<li class=""><strong>Containerized deployment:</strong> Docker Compose runs multiple instances against the same Redis backend. An Nginx reverse proxy provides round-robin HTTP load balancing, health checks, and SSL termination.</li>
<li class=""><strong>A metrics endpoint:</strong> It reports active sessions, supervisor load, and agent utilization.</li>
</ul>
<p>That last item is the metrics and monitoring layer added for the distributed version. It covers system load rather than agent quality, which the experiments measured separately.</p>
<p>A few points matter when reading this section as an architect. The paper describes two balancing mechanisms. One works at the session level, picking the least-loaded supervisor. The other is Nginx round-robin at the HTTP level. The paper reports both mechanisms but does not describe how they compose, so the relationship between them is unclear.</p>
<p>Shared state in Redis is what allows a session to move to another supervisor after a failure. It also makes Redis a dependency that every supervisor relies on. The authors say the design removes the single point of failure at the supervisor tier. The paper does not discuss Redis replication, so that guarantee depends on how Redis itself is deployed.</p>
<p>Finally, the paper reports no throughput or latency measurements for the distributed version. Its experiments evaluate task success and usability. The scaling design is an architecture description, not a benchmarked result.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-it-was-tested-two-real-experiments">How it was tested: Two real experiments<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#how-it-was-tested-two-real-experiments" class="hash-link" aria-label="Direct link to How it was tested: Two real experiments" title="Direct link to How it was tested: Two real experiments" translate="no">​</a></h2>
<p>Direct comparison with earlier scheduling systems was not practical. The authors explain that most traditional assistants are rule-based and lack public evaluation datasets. They also skipped head-to-head tests with commercial assistants, because those APIs are closed and their task coverage differs. Instead they ran two experiments, and the way they measured task success and usability is as useful as the results.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="experiment-1-zero-shot-multilingual-commands">Experiment 1: zero-shot multilingual commands<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#experiment-1-zero-shot-multilingual-commands" class="hash-link" aria-label="Direct link to Experiment 1: zero-shot multilingual commands" title="Direct link to Experiment 1: zero-shot multilingual commands" translate="no">​</a></h3>
<p>The team tested six languages: English, French, German, Tamil, Sinhala, and Chinese. Each language had 20 test inputs, with 5 for each of the four task types. That gives 120 test cases in total. No fine-tuning was done. An output counted as correct only if the assistant read the intent correctly and executed the calendar action with the correct parameters.</p>
<table><thead><tr><th>Language</th><th>Schedule</th><th>Availability</th><th>Edit</th><th>Delete</th><th>Total</th><th>Success rate</th></tr></thead><tbody><tr><td>English</td><td>5/5</td><td>5/5</td><td>5/5</td><td>5/5</td><td>20/20</td><td>100%</td></tr><tr><td>French</td><td>5/5</td><td>5/5</td><td>4/5</td><td>4/5</td><td>18/20</td><td>90%</td></tr><tr><td>German</td><td>5/5</td><td>5/5</td><td>3/5</td><td>4/5</td><td>17/20</td><td>85%</td></tr><tr><td>Tamil</td><td>5/5</td><td>4/5</td><td>3/5</td><td>3/5</td><td>15/20</td><td>75%</td></tr><tr><td>Sinhala</td><td>5/5</td><td>5/5</td><td>2/5</td><td>2/5</td><td>14/20</td><td>70%</td></tr><tr><td>Chinese</td><td>4/5</td><td>3/5</td><td>3/5</td><td>3/5</td><td>13/20</td><td>65%</td></tr></tbody></table>
<p><em>Source: Wijerathne et al. (2025), Table 1.</em></p>
<p>The column pattern says more than the totals. Scheduling held up in every language except Chinese. Editing and deletion degraded most, and those are the operations that change or remove existing data. The authors observe that, apart from Chinese, the ranking follows the language resource levels proposed in <a href="https://aclanthology.org/2022.aacl-main.62" target="_blank" rel="noopener noreferrer" class="">earlier work by Ranathunga and de Silva</a>.</p>
<p>Two caveats apply. Each cell holds only five cases, so one failure moves a cell by 20 percentage points. The authors also note they are native Sinhala and English speakers. Outputs in the other four languages were not checked by native speakers, so those error rates may be off in either direction.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="experiment-2-a-20-person-user-study">Experiment 2: a 20-person user study<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#experiment-2-a-20-person-user-study" class="hash-link" aria-label="Direct link to Experiment 2: a 20-person user study" title="Direct link to Experiment 2: a 20-person user study" translate="no">​</a></h3>
<p>Twenty active digital calendar users worked with ScheduleMe through a web interface linked to test Google Calendar accounts. Each participant completed 5 to 7 operations, mixing simple, complex, and multilingual requests. Participants self-reported task success and errors. They then completed a System Usability Scale (SUS) questionnaire and rated trust and satisfaction on five-point Likert scales.</p>
<table><thead><tr><th>Metric (n = 20)</th><th>Mean ± SD</th></tr></thead><tbody><tr><td>Task success rate</td><td>92.0% ± 9.5</td></tr><tr><td>Errors per task</td><td>0.12 ± 0.08</td></tr><tr><td>SUS score (0 to 100)</td><td>82.5 ± 10.8</td></tr><tr><td>Trust rating (1 to 5)</td><td>4.3 ± 0.6</td></tr><tr><td>Satisfaction rating (1 to 5)</td><td>4.6 ± 0.5</td></tr></tbody></table>
<p><em>Source: Wijerathne et al. (2025), Table 2.</em></p>
<p><img decoding="async" loading="lazy" alt="An illustrative ScheduleMe conversation. The user asks to move tomorrow&amp;#39;s design review to Thursday afternoon. Through the availability agent, the supervisor finds two events titled Design review and asks which one to move. The user picks the 10 AM one. The availability agent reports Thursday afternoon is free from 1:00 to 3:00 PM, and the supervisor proposes 1:00 PM. The user confirms, and the event editing agent updates the event to Thursday, 1:00 to 2:00 PM" src="https://choreolabs.com/api-management/learn/assets/images/scheduleme-rescheduling-conversation-e745f8700cc892cbd5eeb34c9b58b04f.webp" width="3200" height="2020" class="img_ev3q"></p>
<p><em>Figure 2: An illustrative conversation showing how the supervisor routes a rescheduling request. It is not a transcript from the study. The availability check and the edit are separate steps, so the slot is not reserved between them.</em></p>
<p>Raw numbers need a reference point. <a href="https://measuringu.com/task-completion/" target="_blank" rel="noopener noreferrer" class="">MeasuringU's analysis of nearly 1,200 usability tasks</a> puts the average task-completion rate at 78%, with top-quartile tasks above 92%. For SUS, <a href="https://doi.org/10.1080/10447310802205776" target="_blank" rel="noopener noreferrer" class="">Bangor, Kortum, and Miller</a> report a mean of 68.2 for web interfaces. In their <a href="https://uxpajournal.org/determining-what-individual-sus-scores-mean-adding-an-adjective-rating-scale/" target="_blank" rel="noopener noreferrer" class="">adjective-rating data</a>, "Good" corresponds to a mean SUS of 71.4 and "Excellent" to 85.5. ScheduleMe's 82.5 sits above the web mean and closer to "Excellent" than to "Good."</p>
<p>These comparisons are indicative rather than exact. ScheduleMe's success rate was self-reported, the sample was small, and the researchers designed the tasks. The benchmark datasets come mostly from observed usability tests of commercial software. The two benchmarks also measure different things. SUS reflects perceived usability, while the task-success figure reflects participants' own reports of completion.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-went-wrong-failure-modes-worth-knowing">What went wrong: Failure modes worth knowing<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#what-went-wrong-failure-modes-worth-knowing" class="hash-link" aria-label="Direct link to What went wrong: Failure modes worth knowing" title="Direct link to What went wrong: Failure modes worth knowing" translate="no">​</a></h2>
<p>The paper's error analysis names three failure modes. Each one points to a specific design decision.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="translation-induced-errors">Translation-induced errors<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#translation-induced-errors" class="hash-link" aria-label="Direct link to Translation-induced errors" title="Direct link to Translation-induced errors" translate="no">​</a></h3>
<p>In multilingual sessions, some non-English event titles were translated or normalized to English internally. Follow-up queries then failed to match the stored title, which led to wrong retrievals or deletions. A title that the model rewrites stops working as an identifier. The deletion prompt already requires an event ID. The likely weak point is the step before it, where the system resolves a typed title to that ID.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="task-parsing-errors">Task parsing errors<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#task-parsing-errors" class="hash-link" aria-label="Direct link to Task parsing errors" title="Direct link to Task parsing errors" translate="no">​</a></h3>
<p>Complex date and time expressions or ambiguous phrasing caused two opposite failures. Sometimes the system asked for too many clarifications. Sometimes it sent the task to the wrong specialist. The supervisor prompt tells the model to ask the user rather than guess whenever something is unclear. That rule protects against wrong actions, but it pushes ambiguous inputs toward over-clarification.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="entity-reference-confusion">Entity reference confusion<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#entity-reference-confusion" class="hash-link" aria-label="Direct link to Entity reference confusion" title="Direct link to Entity reference confusion" translate="no">​</a></h3>
<p>When several events had similar titles, the system sometimes picked the wrong one to edit or delete. The limitations section says that relying on titles for disambiguation, without persistent conversational memory, increases the risk of this error.</p>
<p>All three errors were more frequent in non-Latin scripts: Tamil, Sinhala, and Chinese. The authors attribute this to semantic drift during translation and limited multilingual training coverage. The pattern matches Table 1, where the edit and delete columns show the largest drops.</p>
<p>For mitigation, the authors plan multilingual entity handling, context tracking, and confidence-based fallback. Their future-work list adds self-consistency with majority voting and constrained decoding with schema or rule checks. It also includes guarded tool calls with retries and backoff, plus escalation pathways. These target the operations where a wrong answer deletes or changes something.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="limitations-and-what-the-researchers-would-do-differently">Limitations and what the researchers would do differently<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#limitations-and-what-the-researchers-would-do-differently" class="hash-link" aria-label="Direct link to Limitations and what the researchers would do differently" title="Direct link to Limitations and what the researchers would do differently" translate="no">​</a></h2>
<p>The paper's limitations section covers memory, infrastructure, evaluation scope, and privacy. The table pairs each stated limitation with the direction the authors propose.</p>
<table><thead><tr><th>Limitation stated in the paper</th><th>Proposed direction</th></tr></thead><tbody><tr><td>No persistent conversational memory. Interactions are largely stateless, and disambiguation leans on event titles.</td><td>Session-spanning memory and better disambiguation</td></tr><tr><td>Limited personalization. The system does not learn preferences or recurring patterns.</td><td>Personalization and predictive scheduling</td></tr><tr><td>Depends on cloud-hosted LLMs and Google Calendar APIs, so it is exposed to latency, downtime, and rate limits</td><td>Optimized distributed deployment with lower latency</td></tr><tr><td>Zero-shot prompting with a single LLM configuration. No few-shot, chain-of-thought, or cross-model baselines.</td><td>Prompt ablations and comparisons across LLM families and sizes</td></tr><tr><td>No native-speaker verification for four of the six languages</td><td>Native-speaker assessment for all languages and larger benchmarks with stress tests</td></tr><tr><td>Calendar content is processed in the cloud without dedicated privacy controls</td><td>Data minimization, anonymization, encrypted state, and local or hybrid inference</td></tr></tbody></table>
<p>The memory row connects to the failure analysis, but the paper's evidence on memory is mixed. Sections 3.2 and 3.3 describe a persistent JSON state file and Redis-backed session context for multi-turn interactions. Section 8 still lists the lack of persistent conversational memory as a limitation. The paper does not define exactly where within-session state ends and durable conversational memory begins. It says the gap increases the risk of errors when events share similar names, but it does not measure how many of the documented failures the gap caused.</p>
<p>The privacy discussion deserves a closer read. ScheduleMe sends event titles and notes, locations, participant names, and times to cloud-hosted LLMs and Google Calendar APIs. The authors list the resulting risks. These include content exposure, re-identification from metadata, retention or logging by providers, cross-border processing, and secondary use without explicit consent. The system relies on provider defaults such as standard transport security. It adds no pseudonymization, on-device inference, or organization-managed encryption. The authors disclose these risks to users and treat privacy as a first-class constraint for future work. They also note that such controls could be critical for enterprise adoption.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-this-case-study-shows-about-building-multi-agent-systems">What this case study shows about building multi-agent systems<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#what-this-case-study-shows-about-building-multi-agent-systems" class="hash-link" aria-label="Direct link to What this case study shows about building multi-agent systems" title="Direct link to What this case study shows about building multi-agent systems" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="centralized-supervision-is-a-deliberate-tradeoff">Centralized supervision is a deliberate tradeoff<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#centralized-supervision-is-a-deliberate-tradeoff" class="hash-link" aria-label="Direct link to Centralized supervision is a deliberate tradeoff" title="Direct link to Centralized supervision is a deliberate tradeoff" translate="no">​</a></h3>
<p>The authors conclude that central supervision simplifies design but limits scalability. Routing every exchange through one coordinator made the system modular and easy to follow. The distributed extension spreads sessions across supervisor instances. Within a session, every turn still passes through one supervisor.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="usability-results-reflect-the-whole-system">Usability results reflect the whole system<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#usability-results-reflect-the-whole-system" class="hash-link" aria-label="Direct link to Usability results reflect the whole system" title="Direct link to Usability results reflect the whole system" translate="no">​</a></h3>
<p>The user-study results suggest that participants found conversational, supervisor-led scheduling usable and trustworthy. These results come from a 20-person study. The study reports these results for its multi-agent implementation. It does not establish how much of the result is attributable to multiple agents rather than tools, prompts, workflow design, or other system choices.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="test-in-the-languages-your-users-speak">Test in the languages your users speak<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#test-in-the-languages-your-users-speak" class="hash-link" aria-label="Direct link to Test in the languages your users speak" title="Direct link to Test in the languages your users speak" translate="no">​</a></h3>
<p>An English-only test set would have reported a perfect score and hidden the main weakness. The multilingual table shows that degradation concentrates in edit and delete operations. Teams building for non-English audiences should include those languages and those operations in the evaluation set from the start.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="put-hard-rules-in-code">Put hard rules in code<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#put-hard-rules-in-code" class="hash-link" aria-label="Direct link to Put hard rules in code" title="Direct link to Put hard rules in code" translate="no">​</a></h3>
<p>Conflict checks, ID-based targeting, and agent names are all contract details. The paper shows them expressed in prompts. Enforcing them in tools and generating routing definitions from one source would close several of the gaps documented above.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="know-when-the-pattern-fits">Know when the pattern fits<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#know-when-the-pattern-fits" class="hash-link" aria-label="Direct link to Know when the pattern fits" title="Direct link to Know when the pattern fits" translate="no">​</a></h3>
<p>A supervisor with specialists suits workloads with distinct operations that need different tools and carry different risks. It also helps when each operation needs its own evaluation, or when destructive actions need tighter controls than reads. A single agent with several tools may be simpler when operations share context and carry similar risk.</p>
<p>This research uses the same framework-built, tool-using agent design that <a href="https://wso2.com/agent-platform/agent-manager/" target="_blank" rel="noopener noreferrer" class="">Choreo Agent Manager</a> is built to operate and govern across frameworks such as LangChain.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/multi-agent-calendar-case-study#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>ScheduleMe shows that a supervisor with four specialists can handle a natural-language task with real side effects. In the user study it achieved high self-reported task success and strong usability ratings. It also shows where such systems still break. Non-Latin scripts, similar event titles, and the absence of persistent memory produced the documented failures. The privacy gaps remain open by the authors' own account.</p>
<p>The full paper includes the agent prompts, the error analysis, and the future-work roadmap. Read it on <a href="https://arxiv.org/abs/2509.25693" target="_blank" rel="noopener noreferrer" class="">arXiv</a>, or find it with related work on the <a href="https://wso2.com/research/" target="_blank" rel="noopener noreferrer" class="">Choreo Labs Research publications page</a>. For the authors' shorter summary, see Srinath Perera's <a href="https://wso2.com/library/blogs/scheduleme-multi-agent-calendar-assistant/" target="_blank" rel="noopener noreferrer" class="">ScheduleMe post on the Choreo Labs blog</a>.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="AI agents" term="AI agents"/>
        <category label="Agent-ready APIs" term="Agent-ready APIs"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[API compliance: GDPR, HIPAA, SOC 2 & PCI guide]]></title>
        <id>https://choreolabs.com/api-management/learn/api-compliance</id>
        <link href="https://choreolabs.com/api-management/learn/api-compliance"/>
        <updated>2026-10-09T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[What GDPR, HIPAA, SOC 2, and PCI DSS require from your APIs: data handling, access control, audit logging, and what changes with AI APIs.]]></summary>
        <content type="html"><![CDATA[<p>Every API that touches a patient record, a card number, or personal data covered by GDPR can fall within the scope of an audit. It makes no difference that the data moved through a REST endpoint instead of a database console. Auditors ask the same questions: who accessed the data, under what authority, and can you prove it?</p>
<p>This guide is about making your own APIs comply with GDPR, HIPAA, SOC 2, and PCI DSS, not about hosted "compliance APIs" that run KYC checks. It covers what each framework requires, the controls they share, and what changes when APIs send regulated data to LLM providers. It sits within <a class="" href="https://choreolabs.com/api-management/learn/api-governance">the broader API governance program.</a></p>
<aside class="cta_6F_m" aria-label="Make your APIs audit-ready before auditors ask" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Make your APIs audit-ready before auditors ask</span><span class="desc_GzK8">Try Choreo API Management to get audit-ready compliance reporting across GDPR, HIPAA, SOC 2, and PCI DSS.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=api-compliance" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-api-compliance">What is API compliance?<a href="https://choreolabs.com/api-management/learn/api-compliance#what-is-api-compliance" class="hash-link" aria-label="Direct link to What is API compliance?" title="Direct link to What is API compliance?" translate="no">​</a></h2>
<p>API compliance is the practice of ensuring your APIs meet the regulatory, industry, and internal-policy requirements that apply to the data they handle. It covers how APIs authenticate callers, limit access, protect data, record activity, and respond to incidents. Where a compliance standard applies to the data, it follows that data into every API that handles it.</p>
<p>The term also names an unrelated product category. A "compliance API" is a third-party service that screens transactions or verifies identities. It is a tool you might use inside a compliant system, not the system itself.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="why-api-compliance-is-harder-than-it-looks">Why API compliance is harder than it looks<a href="https://choreolabs.com/api-management/learn/api-compliance#why-api-compliance-is-harder-than-it-looks" class="hash-link" aria-label="Direct link to Why API compliance is harder than it looks" title="Direct link to Why API compliance is harder than it looks" translate="no">​</a></h2>
<p>Most of these regulatory requirements were written before API-first architecture was common. GDPR talks about controllers and processing, and HIPAA talks about information systems that contain ePHI. Neither mentions gateways, tokens, or OpenAPI specs. Where an obligation applies, it also covers the APIs that carry the regulated data. Your team has to translate legal language into gateway policy, logging configuration, and access rules.</p>
<p>The translation breaks down in predictable places. <a href="https://owasp.org/API-Security/editions/2023/en/0x11-t10/" target="_blank" rel="noopener noreferrer" class="">The OWASP API Security Top 10</a> lists the patterns that also show up as audit findings:</p>
<ul>
<li class=""><strong>Incomplete inventory (API9:2023):</strong> You cannot prove controls on an API nobody knows exists. Deprecated versions and forgotten debug endpoints often still return regulated data.</li>
<li class=""><strong>Weak authentication (API2:2023):</strong> Shared service credentials and long-lived static keys make it impossible to say which person or system made a call.</li>
<li class=""><strong>Misconfiguration (API8:2023):</strong> A gateway with a permissive route, a disabled TLS check, or verbose error output can undo controls that look correct on paper.</li>
<li class=""><strong>Missing monitoring:</strong> Logs that exist but capture the wrong fields, or no one reviews, rarely satisfy an auditor.</li>
</ul>
<p>There is also an ownership gap. Compliance teams own the policy, platform teams own the gateways, and application teams own the code. API compliance holds only when all three work from one set of enforceable rules.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-frameworks-that-matter-most-for-apis">The frameworks that matter most for APIs<a href="https://choreolabs.com/api-management/learn/api-compliance#the-frameworks-that-matter-most-for-apis" class="hash-link" aria-label="Direct link to The frameworks that matter most for APIs" title="Direct link to The frameworks that matter most for APIs" translate="no">​</a></h2>
<p>GDPR, HIPAA, SOC 2, and PCI DSS often appear together in enterprise security questionnaires. They are different kinds of obligation, and treating them as interchangeable leaves gaps.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="gdpr">GDPR<a href="https://choreolabs.com/api-management/learn/api-compliance#gdpr" class="hash-link" aria-label="Direct link to GDPR" title="Direct link to GDPR" translate="no">​</a></h3>
<p><a href="https://eur-lex.europa.eu/eli/reg/2016/679/oj" target="_blank" rel="noopener noreferrer" class="">The General Data Protection Regulation</a> governs the processing of personal data of the people in the EU. Determine whether it applies from your organization's establishment, processing activity, role, and territorial reach. It applies to processing in the context of an EU establishment, wherever the processing takes place. For non-EU organizations, Article 3(2) can apply when they offer goods or services to people in the Union or monitor their behavior there. Citizenship or residence alone is not the test. Once GDPR applies, any endpoint that accepts or returns personal data, such as a name, email address, device identifier, or location, is in scope.</p>
<p>Several articles translate directly into API design decisions:</p>
<ul>
<li class=""><strong>Lawful basis (Article 6) and data minimization (Article 5).</strong> Each endpoint should collect only the fields its documented purpose needs. A profile API that returns full birth dates to a service that only checks age is a minimization problem.</li>
<li class=""><strong>Data subject rights (Articles 15, 17, and 20).</strong> Access, erasure, and portability requests need an operational path. Teams usually expose internal APIs that locate, export, or delete a person's data across systems. The right to erasure is the hardest to implement once data has spread into caches, logs, and downstream services.</li>
<li class=""><strong>Security of processing (Article 32).</strong> The regulation names pseudonymisation and encryption of personal data as examples of appropriate measures.</li>
<li class=""><strong>International transfers (Chapter V).</strong> GDPR does not impose a blanket data residency rule. It restricts transfers outside the EEA unless a mechanism such as an adequacy decision or standard contractual clauses applies. Your API routing and hosting choices determine where data actually flows.</li>
</ul>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="hipaa">HIPAA<a href="https://choreolabs.com/api-management/learn/api-compliance#hipaa" class="hash-link" aria-label="Direct link to HIPAA" title="Direct link to HIPAA" translate="no">​</a></h3>
<p><a href="https://www.hhs.gov/hipaa/index.html" target="_blank" rel="noopener noreferrer" class="">The Health Insurance Portability and Accountability Act</a> applies to covered entities, such as providers and health plans, and to their business associates. Three rules matter for APIs. The Privacy Rule limits how protected health information (PHI) is used and disclosed. The Security Rule sets safeguards for electronic PHI. The Breach Notification Rule defines who must be told after an incident.</p>
<p>The Security Rule's technical safeguards <a href="https://www.law.cornell.edu/cfr/text/45/164.312" target="_blank" rel="noopener noreferrer" class="">at 45 CFR 164.312</a> map closely to API controls. Unique user identification is a required specification. Service identities are not banned, but each access to ePHI must be uniquely attributable and authorized, including the initiating user where relevant. Audit controls must record and examine activity in systems that contain ePHI. Encryption is "addressable," which requires a documented assessment. If it is reasonable and appropriate, implement it. If not, document why and implement an equivalent alternative measure if that is reasonable and appropriate.</p>
<p>Business Associate Agreements (BAAs) are the contractual layer. A vendor that creates, receives, maintains, or transmits PHI on behalf of a covered entity, to perform a function or service regulated by HIPAA, is generally a business associate. Narrow exceptions exist, such as the conduit exception for transmission-only services. <a href="https://www.hhs.gov/hipaa/for-professionals/special-topics/health-information-technology/cloud-computing/index.html" target="_blank" rel="noopener noreferrer" class="">HHS guidance on cloud services</a> confirms this holds even when the provider only stores encrypted data and holds no key.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="soc-2">SOC 2<a href="https://choreolabs.com/api-management/learn/api-compliance#soc-2" class="hash-link" aria-label="Direct link to SOC 2" title="Direct link to SOC 2" translate="no">​</a></h3>
<p>SOC 2 is not a law. It is an attestation report produced by an independent CPA firm under <a href="https://www.aicpa-cima.com/resources/landing/system-and-organization-controls-soc-suite-of-services" target="_blank" rel="noopener noreferrer" class="">AICPA standards.</a> The examination covers controls relevant to security, availability, processing integrity, confidentiality, or privacy. Security is always in scope. The others are included when the service organization chooses them.</p>
<p>A Type 1 report evaluates control design at a point in time. A Type 2 report also tests whether the controls operated effectively over a review period. That is why enterprise buyers ask vendors for a Type 2 report specifically. It is evidence that access reviews, change management, and logging actually happened, not just that a policy exists.</p>
<p>For API teams, SOC 2 works in both directions. Customers ask for your report, and you ask for the reports of the third parties your APIs depend on.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="pci-dss">PCI DSS<a href="https://choreolabs.com/api-management/learn/api-compliance#pci-dss" class="hash-link" aria-label="Direct link to PCI DSS" title="Direct link to PCI DSS" translate="no">​</a></h3>
<p><a href="https://www.pcisecuritystandards.org/standards/pci-dss/" target="_blank" rel="noopener noreferrer" class="">The Payment Card Industry Data Security Standard</a> is a contractual standard from the PCI Security Standards Council. Card brands and acquirers enforce it on any entity that stores, processes, or transmits cardholder data. Version 4.0.1 has been the only active version since v4.0 retired at the end of 2024.</p>
<p>PCI DSS v4.0.1 names APIs directly in its secure software requirements. Requirement 6.2.4 lists attacks on business logic, including manipulation of APIs, among the attack types development practices must prevent or mitigate. Requirement 6.3.2 requires an inventory of bespoke and custom software and the third-party components inside it. For most teams, that inventory has to include internal and partner APIs. Vulnerability scanning remains periodic under Requirement 11.3, and public-facing web applications need an automated technical solution that detects and prevents web-based attacks under Requirement 6.4.2.</p>
<p>The full requirements are in the <a href="https://www.pcisecuritystandards.org/document_library" target="_blank" rel="noopener noreferrer" class="">PCI SSC Document Library.</a> The most effective API control is usually scope reduction. Tokenizing card numbers early can reduce scope by keeping most of your APIs away from primary account numbers. Not seeing a PAN does not by itself take a system out of scope, though. Systems connected to the cardholder data environment, or able to affect its security, can remain in scope.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="comparing-the-four-frameworks">Comparing the four frameworks<a href="https://choreolabs.com/api-management/learn/api-compliance#comparing-the-four-frameworks" class="hash-link" aria-label="Direct link to Comparing the four frameworks" title="Direct link to Comparing the four frameworks" translate="no">​</a></h3>
<table><thead><tr><th></th><th>GDPR</th><th>HIPAA</th><th>SOC 2</th><th>PCI DSS</th></tr></thead><tbody><tr><td><strong>Type of obligation</strong></td><td>EU regulation</td><td>US federal law and HHS rules</td><td>Voluntary attestation report</td><td>Contractual industry standard</td></tr><tr><td><strong>Who it applies to</strong></td><td>Controllers and processors established in the EU, or outside it where Article 3(2) applies</td><td>Covered entities and business associates</td><td>Service organizations whose customers request a report</td><td>Entities that store, process, or transmit cardholder data, or that can affect its security</td></tr><tr><td><strong>Data in scope</strong></td><td>Any personal data</td><td>Protected health information</td><td>Systems in the report's defined scope</td><td>Cardholder and sensitive authentication data</td></tr><tr><td><strong>Breach notification</strong></td><td>Controller notifies the supervisory authority without undue delay and, where feasible, within 72 hours of awareness, unless the breach is unlikely to pose a risk to people. Processors notify their controller without undue delay</td><td>For reportable breaches of unsecured PHI, affected individuals without unreasonable delay and no later than 60 days after discovery. HHS, media, and business associate duties depend on the circumstances</td><td>No statutory clock. Incident response is tested as a control</td><td>Incident response plan must cover notifying card brands and acquirers (Requirement 12.10.1)</td></tr><tr><td><strong>Evidence you produce</strong></td><td>Records of processing, DPIAs, processor agreements</td><td>Risk analysis, policies, BAAs</td><td>Type 1 or Type 2 report</td><td>Report on Compliance or SAQ, plus an Attestation of Compliance</td></tr></tbody></table>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="common-controls-to-map-to-your-applicable-requirements">Common controls to map to your applicable requirements<a href="https://choreolabs.com/api-management/learn/api-compliance#common-controls-to-map-to-your-applicable-requirements" class="hash-link" aria-label="Direct link to Common controls to map to your applicable requirements" title="Direct link to Common controls to map to your applicable requirements" translate="no">​</a></h2>
<p><img decoding="async" loading="lazy" alt="Four frameworks and five controls: a grid mapping encryption, audit logging, access control, breach notification, and third-party agreements across GDPR (EU regulation), HIPAA (US federal law), SOC 2 (attestation report), and PCI DSS (industry standard). Each cell marks the requirement as explicitly mandated, flexible or risk-based, or tested as an audited control, with its source. Examples include GDPR Article 32 encryption and a 72-hour breach notice, HIPAA addressable encryption and notice within 60 days, SOC 2 Type 2 testing of logging and access reviews with no statutory clock, and PCI DSS 12 months of log history (Req. 10.5.1), a unique ID per user (Req. 8.2.1), and service provider terms (Req. 12.8)" src="https://choreolabs.com/api-management/learn/assets/images/frameworks-and-common-controls-2788882a29cbd3682b35f1bd25440b73.webp" width="3200" height="3214" class="img_ev3q"></p>
<p><em>Four frameworks and five controls. Map each control once at the platform layer, then record which requirement it meets, whether it applies to you, and the evidence you keep.</em></p>
<p>The four frameworks look different on paper, and their technical demands overlap. They are not identical mandates, though. Applicability, assessment scope, risk-based obligations, and implementation detail differ by framework. Use the controls below as a starting map. For each one, record the specific requirement, whether it applies to you, how you implement it, and the evidence you keep.</p>
<ul>
<li class=""><strong>Encryption in transit and at rest:</strong> Encrypt regulated data in transit and at rest where your applicable requirements and risk analysis call for it. GDPR names encryption in Article 32. PCI DSS requires strong cryptography for card numbers sent over open, public networks. HIPAA lists it as an addressable safeguard.</li>
<li class=""><strong>Individual user attribution:</strong> Every request that touches regulated data should trace to a specific person or a specific workload identity. HIPAA requires unique user identification. PCI DSS Requirement 8.2.1 requires a unique ID for every user. SOC 2 auditors test logical access against the same principle.</li>
<li class=""><strong>Audit logging with retention:</strong> Record who called which operation, on whose behalf, when, and with what result. PCI DSS Requirement 10.5.1 sets the most specific retention rule. It requires twelve months of audit log history, with the most recent three months immediately available.</li>
<li class=""><strong>Least-privilege access controls:</strong> Scope tokens to the operations and fields a client needs. Use role based access for human users and narrow OAuth scopes for services. Review grants on a schedule.</li>
<li class=""><strong>A maintained API inventory:</strong> Know every API, version, and owner. PCI DSS makes this explicit, and every other framework depends on it implicitly.</li>
<li class=""><strong>Incident response:</strong> Document how you detect, contain, and report an incident. Make sure your logs can answer the questions a regulator will ask inside the notification window.</li>
<li class=""><strong>Third-party agreements:</strong> Every vendor that handles regulated data needs a contract that matches the framework. That means a data processing agreement under GDPR Article 28, a BAA under HIPAA, or service provider terms under PCI DSS Requirement 12.8. Where it supports your own audits, request their SOC 2 reports too.</li>
</ul>
<p>Individual attribution is a common audit failure, and the cause is often the log format. The first entry below attributes the call to a shared integration account. It tells an auditor only that "something" read a patient record.</p>
<div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"timestamp"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"2026-09-14T10:22:31Z"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"principal"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"svc-integration"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"operation"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"GET /patients/48213"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"status"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">200</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token punctuation" style="color:#8A93A6">}</span><br></span></code></pre></div></div>
<p>The second entry carries the authenticated workload, the human user it acted for, and a correlation ID that links the call to downstream activity.</p>
<div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token punctuation" style="color:#8A93A6">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"timestamp"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"2026-09-14T10:22:31Z"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"client_id"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"claims-portal-prod"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"subject"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"user:j.perera@example-health.org"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"scopes"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token string" style="color:#4ADE80">"patients:read"</span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"operation"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"GET /patients/{id}"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"resource_id"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"48213"</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"status"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">200</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token property" style="color:#12C2FF">"correlation_id"</span><span class="token operator">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"c7f2a9e4-1b6d-4f0e-9a51-3d8e2f61b0aa"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token punctuation" style="color:#8A93A6">}</span><br></span></code></pre></div></div>
<p>The second format reduces exposed clinical detail and improves attribution. It may still contain PHI or other personal data through the resource identifier and its context. Restrict access to these logs, minimize the fields you record, and apply appropriate encryption and retention controls. The correlation ID can link the call back to a person for anyone who can join it to other systems, so limit who can resolve it.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="api-compliance-and-ai-what-changed">API compliance and AI: What changed<a href="https://choreolabs.com/api-management/learn/api-compliance#api-compliance-and-ai-what-changed" class="hash-link" aria-label="Direct link to API compliance and AI: What changed" title="Direct link to API compliance and AI: What changed" translate="no">​</a></h2>
<p>None of these frameworks has an exemption for AI. When an application sends a prompt containing personal data or PHI to an LLM API, that call is processing regulated data. It is subject to the same rules as any other third-party API call. European regulators have said so directly. The <a href="https://www.edpb.europa.eu/system/files/2024-12/edpb_opinion_202428_ai-models_en.pdf" target="_blank" rel="noopener noreferrer" class="">EDPB's Opinion 28/2024</a> treats the processing of personal data during the deployment of AI models as subject to GDPR.</p>
<p>That has practical consequences for each framework:</p>
<ul>
<li class=""><strong>HIPAA:</strong> An LLM provider that receives PHI in prompts on behalf of a covered entity or business associate is generally a business associate. You need a signed BAA before the first request, not after the pilot succeeds.</li>
<li class=""><strong>GDPR:</strong> The provider is usually a processor, so you need an Article 28 agreement. If the provider processes prompts outside the EEA, Chapter V transfer rules apply to every call.</li>
<li class=""><strong>PCI DSS:</strong> A card number in a prompt pulls the provider, and potentially the whole AI path, into PCI scope. Keep primary account numbers out of prompts entirely.</li>
<li class=""><strong>SOC 2:</strong> Your auditor may treat the LLM provider as a subservice organization and ask for its SOC 2 report as part of your evidence.</li>
</ul>
<p>Architecture matters as much as contracts. AI traffic adds an outbound flow that many compliance programs have not mapped yet. Document each direction separately:</p>
<ul>
<li class=""><strong>Inbound API traffic:</strong> External clients call your APIs through an API gateway. The gateway authenticates the client and enforces policy before the request reaches your backend.</li>
<li class=""><strong>Outbound AI traffic:</strong> Your applications call LLM providers through an AI gateway. The gateway authenticates the application, applies guardrails such as PII masking, and calls the provider with credentials held by the gateway rather than the application.</li>
<li class=""><strong>Agent tool traffic:</strong> AI agents call tools over the Model Context Protocol (MCP). An AI gateway can authenticate the agent and govern which tools it may call.</li>
</ul>
<p>These flows are independent, and a given application may use only one of them. Map each flow separately. The control plane that configures the gateways does not sit in the request path of either.</p>
<p>Not every team needs a dedicated AI traffic layer on day one. The usual triggers are operational. Prompts carry regulated data, or you use more than one model provider. Finance needs spend attributed by team, or an auditor asks what was sent to which provider. When one of those applies, routing AI traffic through a gateway gives you one place to enforce redaction, attribution, and logging.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-choreo-api-management-helps-you-build-and-prove-api-compliance">How Choreo API Management helps you build and prove API compliance<a href="https://choreolabs.com/api-management/learn/api-compliance#how-choreo-api-management-helps-you-build-and-prove-api-compliance" class="hash-link" aria-label="Direct link to How Choreo API Management helps you build and prove API compliance" title="Direct link to How Choreo API Management helps you build and prove API compliance" translate="no">​</a></h2>
<p>You can assemble the controls above from several tools. Choreo API Management's approach is to manage API governance, API traffic, and AI traffic from a single control plane in Choreo API Management.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="choreo-labs-own-independent-audits">Choreo Labs' own independent audits<a href="https://choreolabs.com/api-management/learn/api-compliance#choreo-labs-own-independent-audits" class="hash-link" aria-label="Direct link to Choreo Labs' own independent audits" title="Direct link to Choreo Labs' own independent audits" translate="no">​</a></h3>
<p>Choreo Labs (formerly WSO2) publishes its certifications and their scope on its <a href="https://wso2.com/security/compliance" target="_blank" rel="noopener noreferrer" class="">security and compliance page.</a> API Management SaaS/Cloud is covered by a SOC 2® Type 2 attestation across the Security, Confidentiality, and Availability criteria. It is certified on PCI DSS v4.0.1 through an annual Qualified Security Assessor assessment. It is attested by an independent auditor as HIPAA compliant, and Choreo Labs signs BAAs on request. Choreo Labs also holds ISO/IEC 27001:2022 certification for its Digital Operations function and lists GDPR, CCPA, and DORA positions with defined scopes. These attestations cover Choreo Labs' services. Your own APIs still need their own controls.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="governance-rules-that-encode-compliance-requirements">Governance rules that encode compliance requirements<a href="https://choreolabs.com/api-management/learn/api-compliance#governance-rules-that-encode-compliance-requirements" class="hash-link" aria-label="Direct link to Governance rules that encode compliance requirements" title="Direct link to Governance rules that encode compliance requirements" translate="no">​</a></h3>
<p>In Choreo API Management, you define rulesets as Spectral YAML rules and attach them to governance policies. A policy applies to every existing and new API proxy in the organization. The default catalog includes Choreo API design and style guideline rulesets and an OWASP Top 10 ruleset. You can add your own rules, for example to require HTTPS server URLs or a documented security scheme on every operation. Violations currently trigger notifications rather than blocking a change. Because rulesets use Spectral's format, the same rule file can also run in your CI/CD pipeline with the open-source Spectral CLI.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="evidence-you-can-show-an-auditor">Evidence you can show an auditor<a href="https://choreolabs.com/api-management/learn/api-compliance#evidence-you-can-show-an-auditor" class="hash-link" aria-label="Direct link to Evidence you can show an auditor" title="Direct link to Evidence you can show an auditor" translate="no">​</a></h3>
<p><a href="https://wso2.com/api-platform/docs/cloud/monitoring-and-insights/compliance/" target="_blank" rel="noopener noreferrer" class="">The Compliance Dashboard</a> reports policy adherence at the organization, project, and component levels. You can also upload your internal compliance documents, such as a data handling standard. AI Compliance Analysis then checks your API definitions against them and reports pass or fail by guideline category. Separately, platform audit logs record administrative operations such as API proxy creation, updates, and deletion. Choreo Labs retains them for a total of two years. That retention applies to administrative audit logs. It is not a guarantee for request bodies, gateway access logs, or backend data-access logs, so plan retention for those separately.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="guardrails-on-ai-traffic">Guardrails on AI traffic<a href="https://choreolabs.com/api-management/learn/api-compliance#guardrails-on-ai-traffic" class="hash-link" aria-label="Direct link to Guardrails on AI traffic" title="Direct link to Guardrails on AI traffic" translate="no">​</a></h3>
<p><a href="https://wso2.com/api-platform/ai-gateway/" target="_blank" rel="noopener noreferrer" class="">Choreo AI Gateway</a> routes and governs outbound LLM traffic and governs inbound MCP traffic from AI agents. <a href="https://wso2.com/api-platform/docs/ai-gateway/next/guardrails/" target="_blank" rel="noopener noreferrer" class="">Its guardrails</a> can detect and mask or redact PII in prompts and responses. You can apply them at the LLM provider level, per LLM proxy, or on MCP proxies. Applications authenticate to the gateway with their own keys, which are separate from the provider credential the gateway uses upstream. You can also attach a masking policy ahead of request logging, so identifying data is masked before it reaches your logs. Plan for one limitation. The PII Masking Regex guardrail matches the patterns you configure, such as email addresses and card numbers. Free-text names don't follow a fixed pattern, so regex alone may miss them. Confirm which guardrail covers them before you rely on it.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="identity-for-gdpr-and-hipaa-workflows">Identity for GDPR and HIPAA workflows<a href="https://choreolabs.com/api-management/learn/api-compliance#identity-for-gdpr-and-hipaa-workflows" class="hash-link" aria-label="Direct link to Identity for GDPR and HIPAA workflows" title="Direct link to Identity for GDPR and HIPAA workflows" translate="no">​</a></h3>
<p>Choreo API Management works with identity providers such as Choreo IAM, Azure AD, or a custom identity provider. Issue API tokens from the same identity provider your users sign in with. That keeps attribution consistent from login to log line.</p>
<p>The deployment model matters for data residency. You can run Choreo API Management fully in Choreo Cloud, in hybrid mode with the gateways (the data plane) in your own data center, or fully self-hosted. Hosting gateways locally gives you control over where they run, and the gateways you host become part of your own compliance scope. Data residency still depends on every configured destination and export path. Map and restrict model calls, logs, telemetry, and other external processing.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/api-compliance#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>GDPR, HIPAA, SOC 2, and PCI DSS differ in legal force, scope, and the evidence they expect. Underneath they converge on the same technical foundation: encryption, individual-user audit logging, least-privilege access, a maintained API inventory, and documented incident response. That foundation now extends to the LLM APIs your applications call with regulated data.</p>
<p>Build those controls once at the platform layer, map inbound and outbound flows separately, and keep evidence ready. To see how governance, API traffic, and AI traffic fit under one control plane, explore <a href="https://wso2.com/api-platform/" target="_blank" rel="noopener noreferrer" class="">Choreo API Management.</a></p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/api-compliance#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="is-soc-2-a-law">Is SOC 2 a law?<a href="https://choreolabs.com/api-management/learn/api-compliance#is-soc-2-a-law" class="hash-link" aria-label="Direct link to Is SOC 2 a law?" title="Direct link to Is SOC 2 a law?" translate="no">​</a></h3>
<p>No. SOC 2 is a voluntary attestation report issued by an independent CPA firm. Customers often require it by contract, but no regulator mandates it.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="does-gdpr-apply-if-my-company-is-not-in-the-eu">Does GDPR apply if my company is not in the EU?<a href="https://choreolabs.com/api-management/learn/api-compliance#does-gdpr-apply-if-my-company-is-not-in-the-eu" class="hash-link" aria-label="Direct link to Does GDPR apply if my company is not in the EU?" title="Direct link to Does GDPR apply if my company is not in the EU?" translate="no">​</a></h3>
<p>It can. GDPR applies to organizations outside the EU that offer goods or services to people in the EU or monitor their behavior. Where you are headquartered does not settle the question, and neither does where your servers run.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-the-difference-between-hipaa-and-soc-2">What is the difference between HIPAA and SOC 2?<a href="https://choreolabs.com/api-management/learn/api-compliance#what-is-the-difference-between-hipaa-and-soc-2" class="hash-link" aria-label="Direct link to What is the difference between HIPAA and SOC 2?" title="Direct link to What is the difference between HIPAA and SOC 2?" translate="no">​</a></h3>
<p>HIPAA is a US law that governs protected health information. SOC 2 is an audit report on a service organization's controls. A SOC 2 report can support HIPAA due diligence, but it does not establish HIPAA compliance by itself.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="do-i-need-a-baa-before-sending-phi-to-an-llm-api">Do I need a BAA before sending PHI to an LLM API?<a href="https://choreolabs.com/api-management/learn/api-compliance#do-i-need-a-baa-before-sending-phi-to-an-llm-api" class="hash-link" aria-label="Direct link to Do I need a BAA before sending PHI to an LLM API?" title="Direct link to Do I need a BAA before sending PHI to an LLM API?" translate="no">​</a></h3>
<p>In most cases, yes. A provider that receives PHI on behalf of a covered entity or business associate is generally a business associate, so the BAA should be in place before any PHI is sent.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="does-using-a-certified-vendor-make-my-apis-compliant">Does using a certified vendor make my APIs compliant?<a href="https://choreolabs.com/api-management/learn/api-compliance#does-using-a-certified-vendor-make-my-apis-compliant" class="hash-link" aria-label="Direct link to Does using a certified vendor make my APIs compliant?" title="Direct link to Does using a certified vendor make my APIs compliant?" translate="no">​</a></h3>
<p>No. A vendor's certification covers the vendor's own controls. You remain responsible for how your APIs authenticate callers, limit access, log activity, and protect data.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="API governance" term="API governance"/>
        <category label="Security" term="Security"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[API governance best practices for enterprise teams]]></title>
        <id>https://choreolabs.com/api-management/learn/api-governance-best-practices</id>
        <link href="https://choreolabs.com/api-management/learn/api-governance-best-practices"/>
        <updated>2026-10-09T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[The practices that make API governance stick at scale: design reviews, CI linting, breaking-change policies, deprecation cadences, and clear ownership.]]></summary>
        <content type="html"><![CDATA[<p>A platform team publishes an API style guide, sets up a weekly review board, and calls governance done. A year later, new APIs still drift from the guide in the same places. Error formats differ between teams, versioning follows no single rule, and nobody can say who owns several APIs in the catalog.</p>
<p>The guide was rarely the problem, but enforcement was. Standards that live only in a document get applied when someone remembers them. Review boards turn into queues as the API count grows. This guide covers what makes API governance hold up: focused design reviews, linting in CI, enforceable breaking-change policies, clear ownership, and enforcement graduated by severity.</p>
<aside class="cta_6F_m" aria-label="Enforce API standards instead of writing them" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Enforce API standards instead of writing them</span><span class="desc_GzK8">Use Choreo API Management to enforce design reviews and breaking-change policies as automated, design-time governance rules.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=api-governance-best-practices" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<p>New to the topic? Start with the <a class="" href="https://choreolabs.com/api-management/learn/api-governance-guide/">API governance overview</a>. If you are establishing a program, the <a class="" href="https://choreolabs.com/api-management/learn/api-governance-framework/">governance framework walkthrough</a> covers ownership and rollout; this guide focuses on operating practices.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-good-api-governance-actually-looks-like">What "good" API governance actually looks like<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#what-good-api-governance-actually-looks-like" class="hash-link" aria-label="Direct link to What &quot;good&quot; API governance actually looks like" title="Direct link to What &quot;good&quot; API governance actually looks like" translate="no">​</a></h2>
<p>Good API governance is a small set of standards written as executable rules and checked automatically wherever an API changes. Each violation carries a consequence that matches its severity. Critical defects stop a merge or a release, while minor ones notify the owner and stay visible until fixed.</p>
<p>That definition rules out a few familiar patterns. A style guide nobody can test is guidance, not governance. A review board that sees every change becomes the slowest step in delivery. A linter that blocks on every warning teaches developers to switch it off.</p>
<p>Governance decides the rules, and API management runs the APIs that follow them across the API lifecycle. In an enterprise API governance program, the work splits into three layers:</p>
<ul>
<li class=""><strong>Standards:</strong> API governance policies for design, API security, documentation, and versioning. Wherever possible, they are expressed as rules against the OpenAPI or AsyncAPI contract.</li>
<li class=""><strong>Checkpoints:</strong> The moments an API changes: a pull request, a promotion to production, an update to a deployed proxy.</li>
<li class=""><strong>Evidence:</strong> A record of which APIs comply, which don't, and who is fixing them. Security and compliance reviews ask to see this record.</li>
</ul>
<p>Consider a team adding a field to a partner-facing orders API. A linter confirms the field follows naming rules and has a description, and a contract diff confirms the change is backward-compatible, such as a new optional request field. A reviewer only needs to look if the field carries customer data. Each checkpoint catches a different class of problem, and none of them needs a meeting.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="design-reviews-that-dont-become-bottlenecks">Design reviews that don't become bottlenecks<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#design-reviews-that-dont-become-bottlenecks" class="hash-link" aria-label="Direct link to Design reviews that don't become bottlenecks" title="Direct link to Design reviews that don't become bottlenecks" translate="no">​</a></h2>
<p>Review the contract, not the code. A design review against the OpenAPI specification happens before implementation starts, while renaming a resource is still cheap. Reviewing after the service is built turns every comment into rework, and teams learn to avoid review.</p>
<p>The second rule is to keep people away from anything a machine can check. Naming conventions, required fields, pagination parameters, and documentation standards are linting problems. Reviewers should spend their time on questions that need judgment:</p>
<ul>
<li class="">Does the resource model match how consumers think about the domain?</li>
<li class="">Does this API duplicate an existing one? Catching duplication of effort at design time is cheaper than merging two live APIs later.</li>
<li class="">Does the contract expose data that should stay internal?</li>
<li class="">Does the authentication and authorization model fit the consumers this API will have?</li>
</ul>
<p>Not every change needs a person, so tie human review to qualitative triggers rather than a blanket rule:</p>
<ul>
<li class="">A new public or partner-facing API, or a new major version of one.</li>
<li class="">A change to the authentication scheme or required scopes.</li>
<li class="">Regulated or personal data appearing in a request or response for the first time.</li>
<li class="">The first API a team publishes on the platform.</li>
</ul>
<p>Backward-compatible changes to internal APIs that pass every automated check can merge on automation alone. Additive does not always mean backward-compatible. A new required request field breaks existing clients, and a new response enum value can break clients that handle values exhaustively. Changes like these need a new major version, which only protects existing consumers if the old version stays available or migration is coordinated.</p>
<p>Run reviews asynchronously in the pull request that changes the spec. Add the architecture group as a required reviewer only when a trigger applies. When reviewers keep flagging the same issue by hand, turn it into a rule so the next review gets shorter.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="linting-in-ci-catching-violations-before-merge">Linting in CI: Catching violations before merge<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#linting-in-ci-catching-violations-before-merge" class="hash-link" aria-label="Direct link to Linting in CI: Catching violations before merge" title="Direct link to Linting in CI: Catching violations before merge" translate="no">​</a></h2>
<p>Linting the contract on every pull request moves automated governance to the point where a fix costs the least. The author sees the problem in the review where they made the change, not in a ticket filed after deployment.</p>
<p><a href="https://github.com/stoplightio/spectral/blob/develop/docs/guides/2-cli.md" target="_blank" rel="noopener noreferrer" class="">Spectral</a> is a common choice, and the same ruleset can run in an editor, a pre-commit hook, and CI. Its severity model is what makes graduated enforcement possible, because each rule carries one of four severities: <code>error</code>, <code>warn</code>, <code>info</code>, or <code>hint</code>. By default, only <code>error</code> results produce a failing exit code. Warnings and information appear in the report without blocking the build.</p>
<p>That gives a ruleset three useful tiers:</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token key atrule" style="color:#3D8BFF">extends</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> spectral</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain">oas</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain"></span><span class="token key atrule" style="color:#3D8BFF">rules</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token comment" style="color:#8A93A6;font-style:italic"># Contract defects fail the build</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">operation-operationId</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> error</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token comment" style="color:#8A93A6;font-style:italic"># Ownership metadata is reported while teams catch up</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">info-contact</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> warn</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token comment" style="color:#8A93A6;font-style:italic"># House convention, visible in the report only</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">paths-kebab-case</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Path segments use lowercase kebab</span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain">case.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">severity</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> info</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">given</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> $.paths</span><span class="token punctuation" style="color:#8A93A6">[</span><span class="token plain">*</span><span class="token punctuation" style="color:#8A93A6">]</span><span class="token plain">~</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">then</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">function</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> pattern</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">functionOptions</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token key atrule" style="color:#3D8BFF">match</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"^(/|(/([a-z0-9]+(-[a-z0-9]+)*|\\{[A-Za-z0-9]+\\}))+)$"</span><br></span></code></pre></div></div>
<p>Save the ruleset as <code>.spectral.yaml</code> in the repository root. Spectral picks it up automatically when you run it from that directory. Under the default failure threshold, which is <code>error</code>, run <code>spectral lint openapi.yaml</code> against a spec with a missing <code>operationId</code>, and the check fails. A spec whose only problem is a missing contact object passes, with the warning listed in the output. A repository that needs a stricter gate can add <code>--fail-severity=warn</code> without changing the shared ruleset.</p>
<p>A few habits keep CI linting useful:</p>
<ul>
<li class=""><strong>Version the ruleset centrally.</strong> Keep one shared ruleset in its own repository and have teams extend it, with any local overrides visible in review.</li>
<li class=""><strong>Introduce new rules as warnings.</strong> A rule that fails most existing APIs on day one gets disabled. Start it at <code>warn</code>, let the report show the backlog, then raise it to <code>error</code>.</li>
<li class=""><strong>Make every rule explain itself.</strong> A description and a link to the standard behind it turn a failed build into a fix, not a support request.</li>
</ul>
<p>For the pipeline these checks plug into, see <a class="" href="https://choreolabs.com/api-management/learn/api-ci-cd-pipeline/">how to build a CI/CD pipeline for APIs</a>.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="breaking-change-policies-and-deprecation-cadences">Breaking-change policies and deprecation cadences<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#breaking-change-policies-and-deprecation-cadences" class="hash-link" aria-label="Direct link to Breaking-change policies and deprecation cadences" title="Direct link to Breaking-change policies and deprecation cadences" translate="no">​</a></h2>
<p>A breaking-change policy only works if it is written down and checked by a machine. It states what counts as breaking, which version a breaking change requires, and how much notice consumers get before a removal.</p>
<p>Semantic versioning covers the numbering: an incompatible change requires a new major version. The guide to <a class="" href="https://choreolabs.com/api-management/learn/api-versioning-best-practices/">API versioning strategies and deprecation</a> covers strategy choice in depth. For governance, the question is narrower: how do you stop a breaking change from merging by accident?</p>
<p>Contract diffing answers it, and a tool such as <a href="https://github.com/oasdiff/oasdiff" target="_blank" rel="noopener noreferrer" class="">oasdiff</a> compares the proposed spec with the published one and classifies each change. Removing an operation that was never marked deprecated is reported as an error. A check run with <code>--fail-on ERR</code> then fails before the change merges.</p>
<p>The same tool can enforce the deprecation cadence for operations and parameters. Mark an operation <code>deprecated: true</code> and add an <code>x-sunset</code> date. The <a href="https://github.com/oasdiff/oasdiff/blob/main/docs/DEPRECATION.md" target="_blank" rel="noopener noreferrer" class="">oasdiff deprecation checks</a> then reject any sunset date inside your published notice window:</p>
<div class="language-bash codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-bash codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token plain">oasdiff breaking published.yaml proposed.yaml \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  --fail-on ERR \</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  --deprecation-days-stable=180</span><br></span></code></pre></div></div>
<p>In <code>proposed.yaml</code>, the deprecated operation carries both fields:</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token key atrule" style="color:#3D8BFF">paths</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">/orders/legacy</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">get</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">deprecated</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token boolean important" style="color:#3D8BFF">true</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">x-sunset</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"2027-06-30"</span><br></span></code></pre></div></div>
<p>The 180 stands for whatever minimum notice period your policy publishes. Base it on your consumers, starting with the release cycle of your slowest important one. Then account for partner contracts and change-freeze windows in regulated industries. Once published, that period is a promise to consumers, and encoding it in CI keeps the promise from depending on a reviewer's memory. oasdiff's full deprecation process covers operations and parameters. Removing a deprecated request or response property is still reported as a breaking change, so plan property removals separately. Pin the oasdiff version in CI so results stay reproducible. This example was tested with oasdiff 1.33.0.</p>
<p>Consumers also need the signal at runtime, in the traffic they already handle. The <code>Deprecation</code> response header, defined in <a href="https://www.rfc-editor.org/rfc/rfc9745.html" target="_blank" rel="noopener noreferrer" class="">RFC 9745</a>, announces that a resource is or will be deprecated. The <code>Sunset</code> header, defined in <a href="https://rfc-editor.org/rfc/rfc8594" target="_blank" rel="noopener noreferrer" class="">RFC 8594</a>, announces when a URI is expected to stop responding. RFC 9745 requires the sunset time to be no earlier than the deprecation time.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="ownership-models-whos-accountable-for-what">Ownership models: Who's accountable for what<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#ownership-models-whos-accountable-for-what" class="hash-link" aria-label="Direct link to Ownership models: Who's accountable for what" title="Direct link to Ownership models: Who's accountable for what" translate="no">​</a></h2>
<p>Every governance check ends with someone fixing something, and if that person isn't named, violations pile up on a dashboard nobody acts on.</p>
<p>Whether a program calls itself centralized or federated, it needs two roles:</p>
<ul>
<li class=""><strong>The platform team</strong> owns the shared ruleset, CI templates, gateway policies, and the exception process.</li>
<li class=""><strong>Development teams</strong> own their APIs end to end. They fix violations, answer consumers, and run the deprecation timeline for their versions.</li>
</ul>
<p>Centralized governance keeps more design review with the platform team. A federated model pushes that judgment to domain teams. The right balance depends on how many teams publish APIs and how much their domains differ. <a class="" href="https://choreolabs.com/api-management/learn/api-federation/">Patterns for multi-team API programs</a> covers that choice in more depth.</p>
<p>Ownership needs mechanics, not just an org chart:</p>
<ul>
<li class=""><strong>Put the owner in the contract.</strong> The <a href="https://spec.openapis.org/oas/latest.html" target="_blank" rel="noopener noreferrer" class="">OpenAPI <code>info.contact</code> object</a> holds the responsible team's name, URL, and email. A specification extension such as <code>x-owner-team</code> can carry an internal team ID. Spectral's <code>info-contact</code> rule then flags specs that leave the contact out.</li>
<li class=""><strong>Register every API in one catalog.</strong> An API catalog with owner metadata lets a security team find an endpoint's owner quickly. APIs running on other gateways belong in the same catalog.</li>
<li class=""><strong>Tie support to publication.</strong> Publishing an API to the developer portal means accepting support duty for it. When an owning team dissolves, its APIs get a new owner or enter deprecation.</li>
<li class=""><strong>Give exceptions to an owner and an expiry.</strong> A waiver for a failing rule should name who owns the risk and when it ends. Without both, it becomes a permanent compliance gap.</li>
</ul>
<p>Choreo API Management supports the catalog side through gateway federation. Its control plane can discover APIs on supported external gateways, such as AWS, Azure, Kong, and Envoy. It then applies the same governance policies and compliance monitoring to them. Support varies by gateway and deployment option, so confirm yours in the federation documentation.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="enforcement-that-doesnt-feel-like-bureaucracy">Enforcement that doesn't feel like bureaucracy<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#enforcement-that-doesnt-feel-like-bureaucracy" class="hash-link" aria-label="Direct link to Enforcement that doesn't feel like bureaucracy" title="Direct link to Enforcement that doesn't feel like bureaucracy" translate="no">​</a></h2>
<p>Governance feels like bureaucracy when every rule carries the same weight. All-or-nothing blocking stops releases over cosmetic issues, while notification-only enforcement lets critical defects ship behind a warning nobody reads. The workable answer is graduated policy enforcement: block on critical violations and notify on minor ones.</p>
<table><thead><tr><th>Control</th><th>Mode</th><th>When it runs</th><th>Effect of a violation</th></tr></thead><tbody><tr><td>Spectral <code>error</code> rules</td><td>Blocking</td><td>Every pull request</td><td>The check fails and the merge waits</td></tr><tr><td>Spectral <code>warn</code> and <code>info</code> rules</td><td>Non-blocking</td><td>Every pull request</td><td>The report annotates the change</td></tr><tr><td>Contract diff for breaking changes</td><td>Blocking</td><td>Every pull request</td><td>An unannounced breaking change fails the check</td></tr><tr><td>Production promotion approval</td><td>Blocking (human)</td><td>Promotion to a production environment</td><td>The promotion waits for an authorized approver</td></tr><tr><td>Platform governance policies</td><td>Non-blocking</td><td>When an API proxy is updated</td><td>The owner is notified and the compliance dashboard records it</td></tr><tr><td>AI compliance analysis</td><td>Non-blocking</td><td>When triggered for a project</td><td>A report rates each guideline as pass or fail</td></tr></tbody></table>
<p>These controls are independent of each other. The CI gate doesn't call Choreo API Management, and Choreo API Management doesn't wait on CI. Each runs at its own checkpoint, and a team can adopt them in any order.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="deciding-which-rules-block">Deciding which rules block<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#deciding-which-rules-block" class="hash-link" aria-label="Direct link to Deciding which rules block" title="Direct link to Deciding which rules block" translate="no">​</a></h3>
<p>A rule earns blocking status when a violation would hurt someone outside the team that made it. Three questions settle most cases:</p>
<ul>
<li class=""><strong>Does it break consumers?</strong> Removed fields and changed types block, but inconsistent casing doesn't.</li>
<li class=""><strong>Does it expose risk?</strong> A missing security scheme blocks, because the cost lands on security and compliance teams later.</li>
<li class=""><strong>Can the author fix it alone?</strong> A blocking rule should point to a clear fix. If the fix needs a cross-team decision, report it and route it to review.</li>
</ul>
<p>Non-blocking doesn't mean ignored, because a warning reaches the API owner first. If it is still open at the rule's planned promotion date, the rule moves to <code>error</code> and starts blocking. Owners who think a rule is wrong file a waiver request with the platform team, so disagreements stay visible.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-choreo-api-management-handles-the-platform-side">How Choreo API Management handles the platform side<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#how-choreo-api-management-handles-the-platform-side" class="hash-link" aria-label="Direct link to How Choreo API Management handles the platform side" title="Direct link to How Choreo API Management handles the platform side" translate="no">​</a></h3>
<p>The last three rows of the table describe Choreo API Management. Its <a href="https://wso2.com/api-platform/docs/cloud/governance/govern-api-proxy/" target="_blank" rel="noopener noreferrer" class="">governance framework</a> uses Spectral rulesets, so a rule written for CI can also run against deployed API proxies. Rulesets are grouped into organization-level policies, and default rulesets in the catalog include Choreo API Design Guidelines and OWASP Top 10. A new policy applies automatically to every existing and newly created API proxy in the organization. Nobody has to compile a list of APIs to re-audit.</p>
<p>Each policy maps each rule severity (Error, Warn, or Info) to an action. Evaluation runs when a proxy is updated, and violations surface as notifications. Choreo API Management evaluates and reports. Blocking stays in the CI gate and the promotion approval, where a failed check has a clear owner and fix.</p>
<p>Results land in a compliance dashboard at the organization, project, and component levels. Admins see compliance across the estate, and API developers see rule-level violations for their own component during development.</p>
<p>Some standards are hard to express as rules. Teams can upload them as natural-language governance documents in PDF form, and an AI compliance analysis evaluates API definitions against them. The docs note that AI-generated results can be inaccurate or incomplete. Treat them as review input rather than a gate.</p>
<p><a href="https://wso2.com/api-platform/docs/cloud/workflows/configure-workflow-approvals/" target="_blank" rel="noopener noreferrer" class="">Approval workflows for environment promotion</a> add the human gate. Administrators assign reviewers at the organization or project level, and a promotion to production waits for approval.</p>
<p>Here is one change moving through all of it. A developer edits the orders API spec and opens a pull request. Spectral fails one error rule, the author fixes it, and two warnings stay in the report. After the merge, the deployed proxy is updated, and the Choreo API Management policy notifies the owner about the same warnings. When the team promotes to production, an approver signs off. No step waited for a meeting. The configured checks reduce the chance of releasing the classes of defects they cover. Runtime tests, authorization tests, and operational monitoring address the risks they don't.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="runtime-enforcement-and-ai-traffic">Runtime enforcement and AI traffic<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#runtime-enforcement-and-ai-traffic" class="hash-link" aria-label="Direct link to Runtime enforcement and AI traffic" title="Direct link to Runtime enforcement and AI traffic" translate="no">​</a></h3>
<p>All of this governs API definitions at design time. Runtime enforcement happens at the gateway, and AI traffic needs it too. Calls from your applications to model providers are outbound traffic. Calls from AI agents to your tools through MCP are inbound traffic. Each crosses a different trust boundary and needs its own policies. <a href="https://wso2.com/api-platform/ai-gateway/" target="_blank" rel="noopener noreferrer" class="">Choreo AI Gateway</a> applies guardrails, authentication, and rate limiting to both directions. It is managed from the same management as your APIs, not from a separate governance stack.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="a-practical-governance-checklist">A practical governance checklist<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#a-practical-governance-checklist" class="hash-link" aria-label="Direct link to A practical governance checklist" title="Direct link to A practical governance checklist" translate="no">​</a></h2>
<p>Use this API governance checklist to audit an existing program.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="standards-and-design">Standards and design<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#standards-and-design" class="hash-link" aria-label="Direct link to Standards and design" title="Direct link to Standards and design" translate="no">​</a></h3>
<ul class="contains-task-list containsTaskList_mC6p">
<li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->One shared, versioned ruleset that teams extend rather than copy</li>
<li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->New rules start at <code>warn</code> with a planned date to raise them to <code>error</code></li>
<li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->Contracts are reviewed before implementation begins</li>
<li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->Human review is tied to named triggers, not applied to every change</li>
</ul>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="ci-and-change-management">CI and change management<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#ci-and-change-management" class="hash-link" aria-label="Direct link to CI and change management" title="Direct link to CI and change management" translate="no">​</a></h3>
<ul class="contains-task-list containsTaskList_mC6p">
<li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->Lint and breaking-change checks run on every pull request</li>
<li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->The minimum deprecation notice is published and enforced in CI</li>
</ul>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="ownership">Ownership<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#ownership" class="hash-link" aria-label="Direct link to Ownership" title="Direct link to Ownership" translate="no">​</a></h3>
<ul class="contains-task-list containsTaskList_mC6p">
<li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->Every API declares an owning team in its contract</li>
<li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->Every API, including those on other gateways, appears in one catalog</li>
<li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->Waivers name an owner and an expiry date</li>
</ul>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="enforcement-and-evidence">Enforcement and evidence<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#enforcement-and-evidence" class="hash-link" aria-label="Direct link to Enforcement and evidence" title="Direct link to Enforcement and evidence" translate="no">​</a></h3>
<ul class="contains-task-list containsTaskList_mC6p">
<li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->Each control is classified as blocking or non-blocking</li>
<li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->API owners can see their own compliance results</li>
<li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->Deprecated versions send <code>Deprecation</code> and <code>Sunset</code> headers</li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>API governance comes down to one question, asked of every rule: what should this violation cost? Critical defects should stop a merge or a release. Minor ones should notify the owner and stay visible until fixed.</p>
<p>That model needs automation at each checkpoint and a named owner for every API. The <a href="https://wso2.com/api-platform/api-governance/" target="_blank" rel="noopener noreferrer" class="">Choreo API Governance framework</a> is one working example of the platform side. It runs Spectral rulesets across your API estate, including federated gateways, and reports results to the owning teams. Pair it with blocking checks in CI, and enforcement stays proportional as the number of APIs grows.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="which-api-governance-practices-matter-most">Which API governance practices matter most?<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#which-api-governance-practices-matter-most" class="hash-link" aria-label="Direct link to Which API governance practices matter most?" title="Direct link to Which API governance practices matter most?" translate="no">​</a></h3>
<p>Automated checks against the API contract, design reviews before implementation, and a written breaking-change and deprecation policy. Add a named owner for every API and consequences that match each violation's severity.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="should-api-governance-checks-block-deployments">Should API governance checks block deployments?<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#should-api-governance-checks-block-deployments" class="hash-link" aria-label="Direct link to Should API governance checks block deployments?" title="Direct link to Should API governance checks block deployments?" translate="no">​</a></h3>
<p>Only for critical violations, such as missing contract elements, security gaps, and unannounced breaking changes. Report style and convention issues without stopping the build.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-do-you-enforce-api-governance-in-cicd">How do you enforce API governance in CI/CD?<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#how-do-you-enforce-api-governance-in-cicd" class="hash-link" aria-label="Direct link to How do you enforce API governance in CI/CD?" title="Direct link to How do you enforce API governance in CI/CD?" translate="no">​</a></h3>
<p>Run a contract linter, such as Spectral, and a breaking-change diff on every pull request. Fail the pipeline only on error-level results, and show warnings in the pull request.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-the-difference-between-centralized-and-federated-api-governance">What is the difference between centralized and federated API governance?<a href="https://choreolabs.com/api-management/learn/api-governance-best-practices#what-is-the-difference-between-centralized-and-federated-api-governance" class="hash-link" aria-label="Direct link to What is the difference between centralized and federated API governance?" title="Direct link to What is the difference between centralized and federated API governance?" translate="no">​</a></h3>
<p>In centralized governance, one platform team sets standards and reviews most designs. In federated governance, the platform team owns standards and tooling, while domain teams own design decisions for their APIs.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="API governance" term="API governance"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[Building an API governance framework: Step by step]]></title>
        <id>https://choreolabs.com/api-management/learn/api-governance-framework</id>
        <link href="https://choreolabs.com/api-management/learn/api-governance-framework"/>
        <updated>2026-10-09T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[A practical walkthrough for designing an API governance framework: stakeholders, standards, enforcement, and the metrics that prove it's working.]]></summary>
        <content type="html"><![CDATA[<p>Someone has asked you to "set up API governance." Your organization runs more APIs than anyone can list from memory. They are spread across several teams and more than one gateway. There is a style guide that a few people have read, and a security review that happens when somebody remembers to request one. The request sounds like it wants a policy document. What it needs is a working system.</p>
<p>This guide walks through building an API governance framework in the order that holds up in practice: stakeholders first, then standards, then enforcement, then metrics. Each step produces one artifact. Together, those artifacts form a template you can adapt to your own API estate.</p>
<aside class="cta_6F_m" aria-label="Get API governance right, out of the gate" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Get API governance right, out of the gate</span><span class="desc_GzK8">Try Choreo API Management to turn stakeholders, standards, and enforcement into one working governance system.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=api-governance-framework" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<p>If you need the definition and scope first, read <a class="" href="https://choreolabs.com/api-management/learn/api-governance-guide/">what API governance covers</a>. This walkthrough focuses on the decisions and artifacts needed to put it into practice.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-four-building-blocks-of-a-governance-framework">The four building blocks of a governance framework<a href="https://choreolabs.com/api-management/learn/api-governance-framework#the-four-building-blocks-of-a-governance-framework" class="hash-link" aria-label="Direct link to The four building blocks of a governance framework" title="Direct link to The four building blocks of a governance framework" translate="no">​</a></h2>
<p>An API governance framework has four building blocks, built in this order:</p>
<ol>
<li class=""><strong>Stakeholders and ownership:</strong> Who decides, who builds, who approves exceptions, and which team owns each API.</li>
<li class=""><strong>Standards:</strong> The written rules for API design, contracts, security, and data handling.</li>
<li class=""><strong>Enforcement:</strong> Automated checks that apply those rules at design time in CI and at runtime in the API gateway.</li>
<li class=""><strong>Metrics:</strong> Measures that show whether APIs meet the standards and whether the program is improving.</li>
</ol>
<p>The order is a dependency chain. Standards written before anyone owns them tend to stall in review, because nobody has the authority to settle disagreements. Enforcement built before standards exist ends up encoding one team's preferences as organization policy. Metrics collected without enforcement measure good intentions rather than API behavior.</p>
<p><img decoding="async" loading="lazy" alt="Four building blocks of an API governance framework, built in order: Step 1 stakeholders and ownership (platform team, API producers, security, compliance, API consumers) produces a stakeholder matrix. Step 2 standards (style guide, contract rules, security baseline, data handling), each a testable rule with a severity, produces a standards checklist. Step 3 enforcement, with two independent checkpoints for design-time contract linting in CI and runtime gateway policies on live traffic, produces an enforcement trigger map. Step 4 metrics (pass rate, time to remediate, inventory accuracy, catalog adoption) produces a metrics dashboard outline, and metrics feed back into revised standards" src="https://choreolabs.com/api-management/learn/assets/images/four-building-blocks-of-api-governance-framework-049ac3e164fbeba1d51f874bacbb5585.webp" width="3200" height="1522" class="img_ev3q"></p>
<p><em>Four building blocks of an API governance framework, each producing one artifact of the template.</em></p>
<p>The figure shows the four steps of the framework. Enforcement has two independent checkpoints. Design-time checks run on the API contract in CI, and runtime checks run on live traffic at the gateway. Metrics then feed back into the standards, which is what turns a one-time rollout into a governance process that improves over time.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-1-identify-stakeholders-and-assign-ownership">Step 1: Identify stakeholders and assign ownership<a href="https://choreolabs.com/api-management/learn/api-governance-framework#step-1-identify-stakeholders-and-assign-ownership" class="hash-link" aria-label="Direct link to Step 1: Identify stakeholders and assign ownership" title="Direct link to Step 1: Identify stakeholders and assign ownership" translate="no">​</a></h2>
<p>API governance is an organizational problem before it is a tooling problem. Before writing a single rule, decide who needs a seat and what each group is accountable for.</p>
<ul>
<li class=""><strong>Platform team:</strong> Runs the shared tooling, including gateways, the developer portal, CI templates, and the governance rulesets. It owns the mechanics of enforcement, not the content of every rule.</li>
<li class=""><strong>API producers:</strong> The teams that design and ship APIs. They own their contracts, fix violations in their APIs, and propose changes when a rule does not fit their domain.</li>
<li class=""><strong>Security:</strong> Defines the security baseline, including authentication and authorization requirements and which findings count as release blockers.</li>
<li class=""><strong>Compliance:</strong> Maps regulatory obligations to concrete data handling rules, such as which fields count as personal data and where that data may flow.</li>
<li class=""><strong>API consumers:</strong> Internal and partner developers who call the APIs. Their complaints about inconsistent or hard-to-use APIs are often the earliest sign that a standard is missing or wrong.</li>
</ul>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="make-api-ownership-explicit">Make API ownership explicit<a href="https://choreolabs.com/api-management/learn/api-governance-framework#make-api-ownership-explicit" class="hash-link" aria-label="Direct link to Make API ownership explicit" title="Direct link to Make API ownership explicit" translate="no">​</a></h3>
<p>Every API needs a named owning team recorded in its catalog metadata, not in someone's head. API ownership is what makes every later step possible. A violation with no owner tends to stay open, and an API version with no owner is rarely retired. Forgotten versions and undocumented hosts are a recognized risk category in their own right. The <a href="https://owasp.org/API-Security/editions/2023/en/0xa9-improper-inventory-management/" target="_blank" rel="noopener noreferrer" class="">OWASP API Security Top 10</a> lists improper inventory management as API9:2023.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="choose-a-federated-or-centralized-model">Choose a federated or centralized model<a href="https://choreolabs.com/api-management/learn/api-governance-framework#choose-a-federated-or-centralized-model" class="hash-link" aria-label="Direct link to Choose a federated or centralized model" title="Direct link to Choose a federated or centralized model" translate="no">​</a></h3>
<p>A centralized model keeps standards, tooling, and accountability with one team. That team does not have to review every change by hand. It can rely on automated checks in CI and at the gateway, and delegate day-to-day execution to API teams. The risk appears when centralization is implemented as manual review of every API, because review queues grow and teams start routing around them. A federated model sets a common floor centrally and lets domain teams own their APIs, with the option to add stricter rules for their area. For a first API governance program, a federated model with a thin central core is usually easier to sustain. The trade-offs between the two models are covered in more depth in our guide to <a class="" href="https://choreolabs.com/api-management/learn/api-federation">API federation for multi-team API programs</a>.</p>
<p>Write down decision rights while you are here. Record who can approve a new standard, who can grant an exception, and how long an exception lasts. Clear governance on these three questions heads off many of the arguments that stall programs later.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-2-define-your-standards">Step 2: Define your standards<a href="https://choreolabs.com/api-management/learn/api-governance-framework#step-2-define-your-standards" class="hash-link" aria-label="Direct link to Step 2: Define your standards" title="Direct link to Step 2: Define your standards" translate="no">​</a></h2>
<p>Standards are the content of the framework. They fall into four groups, and each group has a natural owner among the Step 1 roles.</p>
<ul>
<li class=""><strong>Style guide:</strong> Naming conventions, versioning, pagination, error formats, and status code usage. This is where consistent API design starts.</li>
<li class=""><strong>Contract rules:</strong> What every API definition must include. Use the <a href="https://spec.openapis.org/oas/latest.html" target="_blank" rel="noopener noreferrer" class="">OpenAPI Specification</a> as the contract format for REST APIs, and require the fields reviewers otherwise check by hand. Common examples are an operation ID on every operation, documented error responses, and declared security schemes.</li>
<li class=""><strong>Security baseline:</strong> Authentication and authorization requirements, transport security, and input validation expectations. The <a href="https://api-security.owasp.org/editions/2023/en/0x11-t10/" target="_blank" rel="noopener noreferrer" class="">OWASP API Security Top 10</a> gives security teams a shared reference for which risks the baseline should address.</li>
<li class=""><strong>Data handling standards:</strong> Which fields are classified as sensitive, rules against personal data in URLs and query strings, and what may appear in logs.</li>
</ul>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="write-every-rule-so-it-can-be-tested">Write every rule so it can be tested<a href="https://choreolabs.com/api-management/learn/api-governance-framework#write-every-rule-so-it-can-be-tested" class="hash-link" aria-label="Direct link to Write every rule so it can be tested" title="Direct link to Write every rule so it can be tested" translate="no">​</a></h3>
<p>A rule that cannot be checked will not be enforced. "Use clear resource names" leaves every reviewer to interpret it differently. "Path segments use lowercase kebab-case" can be checked by a linter in seconds. Whether each segment is a plural noun is harder to automate. Treat plural naming as a review convention, or check it against a maintained list of approved resource names. Rewrite each standard until a machine or a reviewer can return a clear pass or fail.</p>
<p>Some rules will stay human judgment calls, and that is fine. "Does this API duplicate an existing one?" needs a person who knows the API estate. Mark those rules as review checks so they land in a design review gate instead of a linter.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="assign-a-severity-to-every-rule">Assign a severity to every rule<a href="https://choreolabs.com/api-management/learn/api-governance-framework#assign-a-severity-to-every-rule" class="hash-link" aria-label="Direct link to Assign a severity to every rule" title="Direct link to Assign a severity to every rule" translate="no">​</a></h3>
<p>Give each rule a severity level: error, warning, or info. Severity tells teams what blocks a release and what only needs attention. It also maps directly to how most linting tools and governance platforms report findings, so the decision you make here carries through to enforcement.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-3-build-enforcement-into-the-pipeline">Step 3: Build enforcement into the pipeline<a href="https://choreolabs.com/api-management/learn/api-governance-framework#step-3-build-enforcement-into-the-pipeline" class="hash-link" aria-label="Direct link to Step 3: Build enforcement into the pipeline" title="Direct link to Step 3: Build enforcement into the pipeline" translate="no">​</a></h2>
<p>Standards change API quality only when something checks them. Effective API governance enforces rules at two independent points in the API lifecycle.</p>
<p>Design-time governance checks the API contract before it ships. The usual place is CI, where a linter runs against the OpenAPI or AsyncAPI definition on every pull request. The open-source Spectral linter is a common choice, and it uses YAML rulesets that encode your standards from Step 2. Configure the job to fail on error-severity findings, and branch protection can then block the merge. Our guide to <a class="" href="https://choreolabs.com/api-management/learn/api-ci-cd-pipeline">building a CI/CD pipeline for APIs</a> covers where this stage fits alongside testing and deployment.</p>
<p>Runtime governance checks live traffic. Runtime policy enforcement happens in the API gateway, which applies authentication and authorization, rate limiting, and request validation to each call. These policies catch what a contract cannot show, such as a client that sends an expired token or exceeds its quota.</p>
<p>The two checkpoints are independent. A contract lint does not sit in the request path, and the gateway does not need CI to run. Map each check to where it runs:</p>
<table><thead><tr><th>Check</th><th>Where it runs</th><th>What happens on failure</th></tr></thead><tbody><tr><td>Style and contract rules</td><td>CI, on every change to the API definition</td><td>The CI job fails, and branch protection can block the merge</td></tr><tr><td>Security baseline in the contract</td><td>CI and platform governance policy</td><td>CI fails, and the platform flags the finding for the owning team</td></tr><tr><td>Authentication and authorization on calls</td><td>API gateway request flow</td><td>The gateway rejects the request</td></tr><tr><td>Rate limiting</td><td>API gateway request flow</td><td>The gateway throttles the client</td></tr><tr><td>Changes to an API definition</td><td>Platform governance policy, re-evaluated when the API proxy is updated</td><td>Promotion waits for a reviewer</td></tr><tr><td>Runtime drift from the approved contract</td><td>Contract tests and configuration comparisons against deployed APIs</td><td>The test run fails and the owning team investigates</td></tr></tbody></table>
<p>Traffic direction matters when you map runtime checks. Inbound calls from clients to your APIs pass through the API gateway. Outbound calls from your applications to third-party services, including LLM providers, need their own egress policies. These are separate paths with separate policy sets. They do not chain into one sequence of hops.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-step-3-looks-like-on-choreo-api-management">What step 3 looks like on Choreo API Management<a href="https://choreolabs.com/api-management/learn/api-governance-framework#what-step-3-looks-like-on-choreo-api-management" class="hash-link" aria-label="Direct link to What step 3 looks like on Choreo API Management" title="Direct link to What step 3 looks like on Choreo API Management" translate="no">​</a></h3>
<p>Choreo API Management includes <a href="https://wso2.com/api-platform/docs/cloud/governance/govern-api-proxy/" target="_blank" rel="noopener noreferrer" class="">a built-in governance framework</a> that maps onto this step. Rulesets are Spectral YAML rule definitions that validate an API's definition, metadata, and documentation for REST and async APIs. The console includes ready-made rulesets such as Choreo API Design Guidelines, Choreo Style Guidelines, and an OWASP Top 10 ruleset. You can also upload governance standards written in natural language as PDF documents, which the AI Compliance Analysis feature uses to evaluate APIs.</p>
<p>Policies attach rulesets to the organization's API proxies and define what happens at each severity level. Policies are evaluated when API proxies are updated. The documented violation action at present is Notify, so findings surface to the owning team rather than stopping the update. In practice, pair platform policies with a failing CI lint for the rules you treat as hard blockers. The Choreo API Management policy then re-evaluates API definitions across the whole estate whenever they change, and CI acts as the merge gate. Policy results describe the submitted definition, not the running backend. A backend can change without any proxy update, so detect runtime drift separately by comparing deployed configurations and tested responses with the approved contract.</p>
<p>Runtime governance sits in the same management. Gateway policies from the Policy Hub, such as JWT and API key authentication, handle runtime checks on API traffic. Environment promotion can also require approval from designated reviewers before an API proxy reaches production. For outbound AI traffic, Choreo API Management's AI Gateway applies its own policies, such as guardrails and token-based rate limiting. All of these are managed from one control plane, so APIs and AI traffic are governed from the same place rather than through separate tools.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-4-define-and-track-governance-metrics">Step 4: Define and track governance metrics<a href="https://choreolabs.com/api-management/learn/api-governance-framework#step-4-define-and-track-governance-metrics" class="hash-link" aria-label="Direct link to Step 4: Define and track governance metrics" title="Direct link to Step 4: Define and track governance metrics" translate="no">​</a></h2>
<p>Metrics are how you prove the framework works and find where it does not. Start with four measures, and track each one by owning team. An organization-wide average hides where the problems sit.</p>
<ul>
<li class=""><strong>Governance pass rate:</strong> The share of in-scope APIs with no error-severity findings.</li>
<li class=""><strong>Time to remediate:</strong> How long a violation stays open from first detection to resolution. If this rises, either a standard is out of step with how teams build, or owners do not have time to fix findings.</li>
<li class=""><strong>Inventory accuracy:</strong> How closely the API catalog matches what is actually running. Compare catalog entries against the APIs that receive traffic at your gateways. A gap in either direction is a finding. Running APIs missing from the catalog are shadow APIs, and catalog entries with no traffic are retirement candidates.</li>
<li class=""><strong>Catalog adoption:</strong> The share of new integrations that start from the developer portal or catalog rather than a direct request to the producing team. This measures API discovery, and it shows whether governance is making APIs easier to find and reuse.</li>
</ul>
<p>Two supporting signals are worth adding once the core four are stable. The first is the number and age of open exceptions, because exceptions that never expire quietly become the real standard. The second is open error findings in security rulesets, which gives security teams a direct view of the API security posture.</p>
<p>Set targets from your own baseline rather than from published industry figures. Your first round of data shows where each team starts, and improvement against that baseline is the number that matters. Traffic and usage analytics from your gateways complement these governance measures.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="putting-it-together-a-governance-framework-template">Putting it together: A governance framework template<a href="https://choreolabs.com/api-management/learn/api-governance-framework#putting-it-together-a-governance-framework-template" class="hash-link" aria-label="Direct link to Putting it together: A governance framework template" title="Direct link to Putting it together: A governance framework template" translate="no">​</a></h2>
<p>The four steps produce four artifacts. Kept together and kept current, they are your API governance framework template.</p>
<table><thead><tr><th>Artifact</th><th>What it contains</th><th>Built in</th><th>Revisit when</th></tr></thead><tbody><tr><td>Stakeholder matrix</td><td>Governance decisions as rows, roles as columns, and "decides", "consulted", or "informed" in each cell</td><td>Step 1</td><td>A team is added or reorganized</td></tr><tr><td>Standards checklist</td><td>One row per rule with an ID, the testable statement, category, severity, check type (automated or review), owner, and rationale</td><td>Step 2</td><td>A rule generates repeated exception requests</td></tr><tr><td>Enforcement trigger map</td><td>The Step 3 check table, extended with the tool and owning team for each check</td><td>Step 3</td><td>A new gateway, pipeline, or API type is introduced</td></tr><tr><td>Metrics dashboard outline</td><td>One panel per Step 4 measure, sliced by owning team and severity, with a trend line and the open exception list</td><td>Step 4</td><td>A metric stops changing decisions</td></tr></tbody></table>
<p>The stakeholder matrix works best when its rows are concrete decisions. Good rows include "approve a new standard", "grant an exception", "change a rule's severity", and "retire an API version". Abstract rows such as "governance oversight" give nobody a clear answer.</p>
<p>The standards checklist is the artifact you will edit most. Keep the rationale column filled in. When someone challenges a rule a year from now, the rationale is what lets the governance strategy evolve without restarting the debate from scratch.</p>
<p>The template does not depend on a specific platform. If you want to see steps 2 to 4 running in one place, <a href="https://wso2.com/api-platform/api-governance/" target="_blank" rel="noopener noreferrer" class="">Choreo API Governance capabilities</a> work as a reference implementation. Rulesets and governance documents hold the standards checklist. Governance policies and gateway policies carry out the enforcement trigger map. The compliance dashboard reports policy and ruleset adherence at the organization, project, and component levels, which supports the governance pass rate. Time to remediate, inventory accuracy, and catalog adoption need separate instrumentation. Typical data sources are issue tracker timestamps, gateway traffic logs, and developer portal usage records. AI compliance analysis runs on request, and its findings are AI-generated. Treat them as input for reviewers rather than an independent compliance decision.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="download-the-template">Download the template<a href="https://choreolabs.com/api-management/learn/api-governance-framework#download-the-template" class="hash-link" aria-label="Direct link to Download the template" title="Direct link to Download the template" translate="no">​</a></h3>
<p>Each artifact above is also a starting CSV you can import into a spreadsheet or issue tracker. Rows marked "(example)" illustrate the shape of the data; replace them with your own decisions, rules, checks, and measures rather than treating them as pre-filled defaults. No platform automatically populates every row for you.</p>
<ul>
<li class=""><a href="https://choreolabs.com/api-management/learn/assets/files/governance-framework-stakeholder-matrix-75a9dbb57a94e76795c28a9e36cbcad1.csv" target="_blank" class="">Stakeholder matrix (CSV)</a></li>
<li class=""><a href="https://choreolabs.com/api-management/learn/assets/files/governance-framework-standards-checklist-281f3fb6ec72bb1df9ee8210e8a6e6e1.csv" target="_blank" class="">Standards checklist (CSV)</a></li>
<li class=""><a href="https://choreolabs.com/api-management/learn/assets/files/governance-framework-enforcement-trigger-map-c6f2c268417105c9dba3dcb8be5b7e79.csv" target="_blank" class="">Enforcement trigger map (CSV)</a></li>
<li class=""><a href="https://choreolabs.com/api-management/learn/assets/files/governance-framework-metrics-dashboard-f493c822612dcbc7a78c18ad56fcc14b.csv" target="_blank" class="">Metrics dashboard outline (CSV)</a></li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="common-mistakes-when-building-a-framework">Common mistakes when building a framework<a href="https://choreolabs.com/api-management/learn/api-governance-framework#common-mistakes-when-building-a-framework" class="hash-link" aria-label="Direct link to Common mistakes when building a framework" title="Direct link to Common mistakes when building a framework" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="skipping-stakeholder-buy-in">Skipping stakeholder buy-in<a href="https://choreolabs.com/api-management/learn/api-governance-framework#skipping-stakeholder-buy-in" class="hash-link" aria-label="Direct link to Skipping stakeholder buy-in" title="Direct link to Skipping stakeholder buy-in" translate="no">​</a></h3>
<p>A framework announced by the platform team without producer or security involvement gets treated as someone else's project. Teams comply with the letter of the rules and route around their intent. Bring each Step 1 group into the drafting of the standards it will be held to.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="over-centralizing">Over-centralizing<a href="https://choreolabs.com/api-management/learn/api-governance-framework#over-centralizing" class="hash-link" aria-label="Direct link to Over-centralizing" title="Direct link to Over-centralizing" translate="no">​</a></h3>
<p>A single review board for every API change feels safe and scales poorly. Once queue times grow, teams start shipping outside the process. Centralize the floor and the tooling, and leave domain-specific rules to the teams that own those APIs.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="defining-standards-without-enforcement">Defining standards without enforcement<a href="https://choreolabs.com/api-management/learn/api-governance-framework#defining-standards-without-enforcement" class="hash-link" aria-label="Direct link to Defining standards without enforcement" title="Direct link to Defining standards without enforcement" translate="no">​</a></h3>
<p>A style guide in a wiki is documentation, not governance. If a rule matters enough to write down, it needs a check in CI, at the gateway, or in a named review step.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="turning-every-rule-into-a-blocker-on-day-one">Turning every rule into a blocker on day one<a href="https://choreolabs.com/api-management/learn/api-governance-framework#turning-every-rule-into-a-blocker-on-day-one" class="hash-link" aria-label="Direct link to Turning every rule into a blocker on day one" title="Direct link to Turning every rule into a blocker on day one" translate="no">​</a></h3>
<p>Introducing a large ruleset at error severity across an existing estate creates a wall of failures. Teams quickly learn to ignore them. Start new rules at warning severity, give owners time to clear the backlog, and then promote the rules that matter to error.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="measuring-nothing">Measuring nothing<a href="https://choreolabs.com/api-management/learn/api-governance-framework#measuring-nothing" class="hash-link" aria-label="Direct link to Measuring nothing" title="Direct link to Measuring nothing" translate="no">​</a></h3>
<p>Without metrics, nobody can tell whether the program improved API quality or only added process. Even a basic pass rate and time-to-remediate report, tracked by team, gives you something to act on.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/api-governance-framework#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>An API governance framework works when four parts run together, built in order. You need owners who can make decisions and standards written as testable rules. You also need enforcement at both design time and runtime, and metrics that show where the rules hold and where they need revising.</p>
<p>You should start small. Name an owner for every API, choose the few rules that matter most, and run them as warnings in CI before promoting them to errors. Measure your starting baseline, and let those results decide which standard to tighten next.</p>
<p>To see the standards, enforcement, and metrics layers working together, explore <a href="https://wso2.com/api-platform/api-governance/" target="_blank" rel="noopener noreferrer" class="">Choreo API Management's governance capabilities</a> as a reference implementation.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/api-governance-framework#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-an-api-governance-framework">What is an API governance framework?<a href="https://choreolabs.com/api-management/learn/api-governance-framework#what-is-an-api-governance-framework" class="hash-link" aria-label="Direct link to What is an API governance framework?" title="Direct link to What is an API governance framework?" translate="no">​</a></h3>
<p>An API governance framework is the set of roles, standards, automated checks, and metrics an organization uses to keep its APIs consistent, secure, and compliant.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-the-difference-between-design-time-and-runtime-api-governance">What is the difference between design-time and runtime API governance?<a href="https://choreolabs.com/api-management/learn/api-governance-framework#what-is-the-difference-between-design-time-and-runtime-api-governance" class="hash-link" aria-label="Direct link to What is the difference between design-time and runtime API governance?" title="Direct link to What is the difference between design-time and runtime API governance?" translate="no">​</a></h3>
<p>Design-time governance checks an API's contract before release, usually by linting the OpenAPI or AsyncAPI definition in CI. Runtime governance applies policies to live traffic at the API gateway, such as authentication, authorization, and rate limiting. Most programs need both.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="who-should-own-api-governance">Who should own API governance?<a href="https://choreolabs.com/api-management/learn/api-governance-framework#who-should-own-api-governance" class="hash-link" aria-label="Direct link to Who should own API governance?" title="Direct link to Who should own API governance?" translate="no">​</a></h3>
<p>A platform team usually owns the tooling and the shared baseline. Each API's owning team is accountable for its own compliance. Security and compliance teams own the rules in their areas and approve exceptions to them.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-do-you-know-if-api-governance-is-working">How do you know if API governance is working?<a href="https://choreolabs.com/api-management/learn/api-governance-framework#how-do-you-know-if-api-governance-is-working" class="hash-link" aria-label="Direct link to How do you know if API governance is working?" title="Direct link to How do you know if API governance is working?" translate="no">​</a></h3>
<p>Track the share of APIs passing governance checks and how long violations take to fix. Add how accurately the catalog reflects what is running and how often teams find APIs through it. Compare each measure against your own starting baseline.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="is-there-an-api-governance-framework-template-or-pdf">Is there an API governance framework template or PDF?<a href="https://choreolabs.com/api-management/learn/api-governance-framework#is-there-an-api-governance-framework-template-or-pdf" class="hash-link" aria-label="Direct link to Is there an API governance framework template or PDF?" title="Direct link to Is there an API governance framework template or PDF?" translate="no">​</a></h3>
<p>A usable template has four parts: a stakeholder matrix, a standards checklist, an enforcement trigger map, and a metrics dashboard outline. All four fit in one shared document that you can export to PDF for review and sign-off.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="API architecture" term="API architecture"/>
        <category label="API governance" term="API governance"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[API governance: The complete framework & implementation guide]]></title>
        <id>https://choreolabs.com/api-management/learn/api-governance-guide</id>
        <link href="https://choreolabs.com/api-management/learn/api-governance-guide"/>
        <updated>2026-10-09T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[What API governance is, why it matters, and how to build a framework teams follow: standards, enforcement, and the federated vs. centralized decision.]]></summary>
        <content type="html"><![CDATA[<p>Your organization probably has an API style guide. It lives on a wiki page, it was written two reorgs ago, and the last three APIs your teams shipped don't follow it. Nobody broke a rule on purpose. The rule just had no way to reach the pull request, the gateway, or the retirement plan.</p>
<p>That gap between what a team agrees on and what actually runs in production is what API governance is for. This guide is for platform engineering leads, API program owners, and architects who are responsible for keeping dozens or hundreds of APIs consistent. It covers what governance includes and how it differs from API management and API security. It also covers the four layers a working framework needs, the centralized vs. federated decision, and the mistakes that turn governance into a bottleneck.</p>
<aside class="cta_6F_m" aria-label="Choose the governance model that scales" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Choose the governance model that scales</span><span class="desc_GzK8">Try Choreo API Management to support both centralized and federated API governance models as you scale.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=api-governance-guide" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<p>Need an implementation plan? Follow the <a class="" href="https://choreolabs.com/api-management/learn/api-governance-framework/">API governance framework</a> to assign owners, define standards, and map enforcement. For software selection, use the <a class="" href="https://choreolabs.com/api-management/learn/api-governance-tools/">API governance tools guide</a>.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-api-governance">What is API governance?<a href="https://choreolabs.com/api-management/learn/api-governance-guide#what-is-api-governance" class="hash-link" aria-label="Direct link to What is API governance?" title="Direct link to What is API governance?" translate="no">​</a></h2>
<p>API governance is the set of standards, policies, and processes that decide how an organization's APIs are designed, secured, versioned, published, and retired. It turns those decisions into automated checks at design time, at the gateway, and across the API lifecycle. Consistency then doesn't depend on a reviewer remembering the rules.</p>
<p>The definition is simple. The hard part is where the checks run. A rule such as "every public operation needs an operationId and a documented error response" can be enforced in at least three places. It can run in the editor while the OpenAPI document is written. It can run again in CI when someone opens a pull request, and in the control plane when the API is registered. A rule such as "partner APIs must validate a JWT" can only be enforced at runtime, on the gateway that receives the call.</p>
<p>That is why an API governance strategy is more than a style guide. It is a map of which rules belong to which stage of the API lifecycle and which system enforces each one. API management provides much of the machinery: the gateway, the catalog, the lifecycle states. Governance policies decide what that machinery should allow.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="why-api-governance-matters">Why API governance matters<a href="https://choreolabs.com/api-management/learn/api-governance-guide#why-api-governance-matters" class="hash-link" aria-label="Direct link to Why API governance matters" title="Direct link to Why API governance matters" translate="no">​</a></h2>
<p>Most governance programs start after something goes wrong, not before. The trigger is rarely dramatic. A security review finds an API nobody can name an owner for. A partner integration breaks because two teams versioned the same resource in different ways. An auditor asks for a list of every API that handles customer data, and producing it takes weeks of manual surveying.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="shadow-apis-are-the-clearest-risk">Shadow APIs are the clearest risk<a href="https://choreolabs.com/api-management/learn/api-governance-guide#shadow-apis-are-the-clearest-risk" class="hash-link" aria-label="Direct link to Shadow APIs are the clearest risk" title="Direct link to Shadow APIs are the clearest risk" translate="no">​</a></h3>
<p>The OWASP API Security Top 10 lists Improper Inventory Management as API9:2023, and its example is worth reading closely. A social network puts rate limiting in a separate component in front of its production API. A researcher then found a beta host running the same API, including the password reset flow, without that component. The researcher brute-forced the 6-digit reset token on the beta host, because the control was tied to one deployment's traffic path instead of to the API. The <a href="https://owasp.org/API-Security/editions/2023/en/0xa9-improper-inventory-management/" target="_blank" rel="noopener noreferrer" class="">OWASP API9:2023 entry</a> also flags two familiar conditions: no retirement plan for each API version, and a missing or outdated host inventory.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="api-sprawl-compounds-the-cost-of-every-other-problem">API sprawl compounds the cost of every other problem<a href="https://choreolabs.com/api-management/learn/api-governance-guide#api-sprawl-compounds-the-cost-of-every-other-problem" class="hash-link" aria-label="Direct link to API sprawl compounds the cost of every other problem" title="Direct link to API sprawl compounds the cost of every other problem" translate="no">​</a></h3>
<p>Each new team, acquisition, or cloud region can add APIs faster than a central review team can track them. OWASP notes that running multiple versions of an API expands the attack surface. Sprawl also slows developers down. When three APIs return customers in three different shapes, every consumer writes three adapters.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="consistency-is-what-makes-an-api-portfolio-usable-at-scale">Consistency is what makes an API portfolio usable at scale<a href="https://choreolabs.com/api-management/learn/api-governance-guide#consistency-is-what-makes-an-api-portfolio-usable-at-scale" class="hash-link" aria-label="Direct link to Consistency is what makes an API portfolio usable at scale" title="Direct link to Consistency is what makes an API portfolio usable at scale" translate="no">​</a></h3>
<p>Predictable naming, pagination, error formats, and authentication let a developer move between APIs without relearning each one. This matters more now that AI agents consume APIs too. An agent picks an operation based on its description, so vague summaries and missing examples cause wrong calls instead of just slower onboarding.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="api-governance-vs-api-management-vs-api-security">API governance vs. API management vs. API security<a href="https://choreolabs.com/api-management/learn/api-governance-guide#api-governance-vs-api-management-vs-api-security" class="hash-link" aria-label="Direct link to API governance vs. API management vs. API security" title="Direct link to API governance vs. API management vs. API security" translate="no">​</a></h2>
<p>The three terms overlap, and teams often use them interchangeably. They describe different layers of the same API program.</p>
<table><thead><tr><th></th><th>API governance</th><th>API management</th><th>API security</th></tr></thead><tbody><tr><td><strong>Core question</strong></td><td>What rules should every API follow?</td><td>How do APIs get built, published, and run?</td><td>How are APIs protected from misuse and attack?</td></tr><tr><td><strong>Primary output</strong></td><td>Standards, rulesets, policies, compliance reports</td><td>Gateways, developer portal, lifecycle states, analytics</td><td>Authentication, authorization, threat protection, rate limits</td></tr><tr><td><strong>Typical owner</strong></td><td>API program owner or governance group with platform engineering</td><td>Platform engineering</td><td>Security team with platform engineering</td></tr><tr><td><strong>When it acts</strong></td><td>Design time, registration, runtime, and retirement</td><td>Runtime and publication</td><td>Mostly runtime, plus design-time checks for insecure definitions</td></tr><tr><td><strong>Example</strong></td><td>"All external APIs use OAuth 2.0 and a standard error schema"</td><td>Deploying the API to a gateway and publishing it to the portal</td><td>The gateway rejecting a call with an invalid token</td></tr></tbody></table>
<p>Governance is the policy layer. Management is the operational layer that carries those policies out. Security is one governed domain among several, alongside style, versioning, documentation, and lifecycle. Take a security rule such as "no API key in a query parameter." Governance writes it, a linter checks it against the OpenAPI definition, and the gateway enforces it at runtime.</p>
<p>The practical consequence: buying an API management platform does not give you governance. It gives you the enforcement points. Governance is the decision about what to enforce at each one.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="federated-vs-centralized-governance-the-foundational-model-decision">Federated vs. centralized governance: The foundational model decision<a href="https://choreolabs.com/api-management/learn/api-governance-guide#federated-vs-centralized-governance-the-foundational-model-decision" class="hash-link" aria-label="Direct link to Federated vs. centralized governance: The foundational model decision" title="Direct link to Federated vs. centralized governance: The foundational model decision" translate="no">​</a></h2>
<p>Before writing a single rule, a governance program has to answer who writes the rules and who enforces them. Most programs pick one of two models, and many drift into a third without choosing it.</p>
<p><strong>Centralized governance:</strong> One team owns the standards, the exceptions, and the domain decisions for every API. It is consistent and easy to audit. That team can enforce its rules through automated checks, manual review, or both. When approvals depend on manual review, the governance team can become a queue once several teams ship APIs.</p>
<p><strong>Federated governance:</strong> Keeps organization-wide standards and mandatory global controls central, but moves domain decisions to the teams that own the APIs. The central group writes the baseline rules, decides exceptions to them, and audits compliance. Domain teams apply those rules in their own pipelines and gateways, and can add stricter rules for their own domain.</p>
<p><strong>Siloed governance:</strong> This is what happens when nobody chooses. Each team sets its own conventions, and the organization finds out about inconsistencies when a consumer complains or an audit fails.</p>
<p>The choice is less about philosophy than about scale and structure. Centralized authority tends to work while one team owns most APIs and the change rate is low. Federated governance becomes the realistic option once several teams publish APIs independently, or when acquisitions bring in gateways and conventions you can't migrate quickly. Automation is a separate decision from the model. Either model can enforce rules through automated checks, and either one stalls when every check waits on a person. A rule that only exists as a review comment is hard to apply consistently in any model, because it travels with the reviewer rather than with the API.</p>
<p>For the organizational side of this decision, including how teams split runtime ownership from organization-wide policy, see <a class="" href="https://choreolabs.com/api-management/learn/api-federation">API federation patterns for multi-team API programs</a>. A dedicated spoke in this series covers <a class="" href="https://choreolabs.com/api-management/learn/federated-vs-centralized-governance">the federated vs. centralized decision in depth</a>.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-four-layers-of-an-api-governance-framework">The four layers of an API governance framework<a href="https://choreolabs.com/api-management/learn/api-governance-guide#the-four-layers-of-an-api-governance-framework" class="hash-link" aria-label="Direct link to The four layers of an API governance framework" title="Direct link to The four layers of an API governance framework" translate="no">​</a></h2>
<p>A useful way to structure an API governance framework is by where each rule gets enforced. Four layers cover most of what a program needs: standards, design-time enforcement, runtime enforcement, and lifecycle and catalog. The layers are not a pipeline that every API passes through in order. Standards feed both enforcement layers, and lifecycle and catalog span the entire life of the API.</p>
<p><img decoding="async" loading="lazy" alt="The four layers of API governance: standards at the top, covering naming, versioning, error formats, authentication, and required metadata, feed rulesets into design-time enforcement (editor, CI on every pull request, registration) and gateway policies into runtime enforcement (token validation, authorization, rate limits, payload validation). Both enforcement layers send compliance results down to the lifecycle and catalog layer, which inventories every API with its owner, version, consumers, and compliance status, and tracks it through Created, Published, Deprecated, and Retired" src="https://choreolabs.com/api-management/learn/assets/images/four-layers-of-api-governance-0261ffa2e27944f014e3835571efb757.webp" width="3200" height="1842" class="img_ev3q"></p>
<p><em>The four layers of API governance: a conceptual model.</em></p>
<p>The figure shows standards feeding two independent enforcement points. Lifecycle and catalog form the base that tracks every API from creation to retirement.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="layer-1-standards">Layer 1: Standards<a href="https://choreolabs.com/api-management/learn/api-governance-guide#layer-1-standards" class="hash-link" aria-label="Direct link to Layer 1: Standards" title="Direct link to Layer 1: Standards" translate="no">​</a></h3>
<p>Standards are the written rules: naming, versioning, error formats, pagination, authentication schemes, required metadata, and documentation expectations. They are the only layer humans read directly, so they need to be short, specific, and explained. "Use kebab-case for paths" is a standard. "APIs should be well designed" is not.</p>
<p>The test for a good standard is whether it can be checked. Each standard should map to at least one machine-checkable rule, or carry an explicit note that it needs human review. Standards that can't be checked tend to become optional over time.</p>
<p>Some platforms can also evaluate APIs against natural-language standards. In <a href="https://wso2.com/api-platform/" target="_blank" rel="noopener noreferrer" class="">Choreo API Management</a>, for example, <a href="https://wso2.com/api-platform/docs/cloud/governance/overview/" target="_blank" rel="noopener noreferrer" class="">governance documents</a> are PDFs that hold standards in natural language. The AI Compliance Analysis feature uses them as guidelines when it evaluates APIs. Treat that as a review aid that catches drift in prose-only rules, not as a replacement for deterministic checks.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="layer-2-design-time-enforcement">Layer 2: Design-time enforcement<a href="https://choreolabs.com/api-management/learn/api-governance-guide#layer-2-design-time-enforcement" class="hash-link" aria-label="Direct link to Layer 2: Design-time enforcement" title="Direct link to Layer 2: Design-time enforcement" translate="no">​</a></h3>
<p>Design-time enforcement checks the API contract before any code ships. In practice that means linting OpenAPI and AsyncAPI documents in the editor and again in CI. The editor gives the author fast feedback. The CI check makes the rule non-optional for anything that merges.</p>
<p>Design time is the cheapest place to catch inconsistency. Renaming a field in a draft costs a few minutes. Renaming it after consumers depend on it is a breaking change that needs a new version, a migration window, and a deprecation plan. That is why design-time findings that require breaking changes should be routed into a versioning decision rather than fixed silently. For how to structure those decisions, see <a class="" href="https://choreolabs.com/api-management/learn/api-versioning-best-practices">API versioning strategies for long-lived APIs</a>.</p>
<p>A second checkpoint at registration time is a useful backstop. When the control plane re-evaluates a definition every time an API is updated, teams that skipped the local check still surface in a compliance report.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="layer-3-runtime-enforcement">Layer 3: Runtime enforcement<a href="https://choreolabs.com/api-management/learn/api-governance-guide#layer-3-runtime-enforcement" class="hash-link" aria-label="Direct link to Layer 3: Runtime enforcement" title="Direct link to Layer 3: Runtime enforcement" translate="no">​</a></h3>
<p>Runtime enforcement happens on live traffic, at the gateway and in the services behind it. It covers what a contract can't guarantee on its own: token validation, authorization, rate limits, quota, payload validation, and header policies. A definition can declare that an operation requires OAuth 2.0. The gateway provides a shared enforcement point for traffic routed through it. Services must also enforce the object- and business-level permissions needed for their operations.</p>
<p>Runtime rules need to apply to every deployment of an API, not only to the one the governance team knows about. The OWASP beta-host example above is a runtime failure. The rule existed, but it was attached to one deployment's traffic path instead of to the API itself. Policies that attach to the API itself, and deploy with it to every gateway, close that gap.</p>
<p>Runtime enforcement does not require all traffic to pass through one central gateway. Teams can run separate gateways, in separate regions or clouds, as long as each gateway applies the same baseline policies. The control plane distributes and audits policy. It does not need to sit in the request path.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="layer-4-lifecycle-and-catalog">Layer 4: Lifecycle and catalog<a href="https://choreolabs.com/api-management/learn/api-governance-guide#layer-4-lifecycle-and-catalog" class="hash-link" aria-label="Direct link to Layer 4: Lifecycle and catalog" title="Direct link to Layer 4: Lifecycle and catalog" translate="no">​</a></h3>
<p>The lifecycle and catalog layer answers two questions at any point in time: what APIs exist, and what state each one is in. The catalog is the inventory. It lists each API, its owner, its version, its consumers, and its compliance status. The lifecycle defines the states an API moves through and what each state allows.</p>
<p>Lifecycle states are where governance meets consumers. <a href="https://wso2.com/api-platform/" target="_blank" rel="noopener noreferrer" class="">Choreo API Management</a>, for instance, uses five lifecycle states: Created, Pre-Released, Published, Deprecated, and Retired. A Deprecated API stays available to existing subscribers but accepts no new subscriptions. A Retired API is no longer in use. Explicit states like these turn "we should retire v1" into an enforced transition with a defined effect on consumers.</p>
<p>A catalog makes registered and successfully discovered APIs visible. Reconcile it with gateway configurations, repositories, cloud inventories, and other discovery sources to identify unregistered APIs.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-to-build-a-governance-framework-a-practical-walkthrough">How to build a governance framework: A practical walkthrough<a href="https://choreolabs.com/api-management/learn/api-governance-guide#how-to-build-a-governance-framework-a-practical-walkthrough" class="hash-link" aria-label="Direct link to How to build a governance framework: A practical walkthrough" title="Direct link to How to build a governance framework: A practical walkthrough" translate="no">​</a></h2>
<p>A governance program doesn't need a long rollout before it produces value. The sequence below starts with visibility, adds rules a team can check, and only then adds enforcement. Each step produces something usable on its own.</p>
<p>The steps below are an overview. Use the <a class="" href="https://choreolabs.com/api-management/learn/api-governance-framework/">step-by-step governance framework</a> for the stakeholder matrix, standards checklist, enforcement map, and measurement template.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-1-inventory-what-you-have">Step 1: Inventory what you have<a href="https://choreolabs.com/api-management/learn/api-governance-guide#step-1-inventory-what-you-have" class="hash-link" aria-label="Direct link to Step 1: Inventory what you have" title="Direct link to Step 1: Inventory what you have" translate="no">​</a></h3>
<p>Start by building the API catalog you will govern against. Pull API definitions from repositories, gateways, and the developer portal. For each API, record the owner, the environment, the version, whether it is internal, partner, or public, and what data it handles.</p>
<p>Expect the inventory to be incomplete on the first pass. APIs on a team's own gateway, test hosts that were never shut down, and old versions still serving traffic are common gaps. Connecting existing gateways to a central catalog, rather than asking teams to re-register by hand, closes most of those gaps faster than a survey does.</p>
<p>The inventory also tells you which governance model fits. If one team owns most APIs, centralized review is workable for now. If ownership is spread across several teams and gateways, plan for federation from the start.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-2-define-a-small-set-of-governance-rules">Step 2: Define a small set of governance rules<a href="https://choreolabs.com/api-management/learn/api-governance-guide#step-2-define-a-small-set-of-governance-rules" class="hash-link" aria-label="Direct link to Step 2: Define a small set of governance rules" title="Direct link to Step 2: Define a small set of governance rules" translate="no">​</a></h3>
<p>Write the first ruleset around the problems the inventory exposed, not around a complete style guide. Good starting rules are the ones that cause real consumer pain or security exposure when they are missing:</p>
<ul>
<li class=""><strong>Contract completeness:</strong> every operation has an operationId, a summary, and documented error responses.</li>
<li class=""><strong>Security baseline:</strong> external APIs declare an authentication scheme, and no credentials appear in query parameters.</li>
<li class=""><strong>Versioning:</strong> a single, documented versioning scheme per API type.</li>
<li class=""><strong>Ownership metadata:</strong> every API names an owning team and a contact.</li>
</ul>
<p>Assign each rule a severity: errors block a merge or require an exception, while warnings appear in reports without blocking. Start strict on a small set of rules and loose on the rest. Teams accept a short list of non-negotiable rules much more readily than a long list of equally weighted ones.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-3-support-teams-with-tooling-not-tickets">Step 3: Support teams with tooling, not tickets<a href="https://choreolabs.com/api-management/learn/api-governance-guide#step-3-support-teams-with-tooling-not-tickets" class="hash-link" aria-label="Direct link to Step 3: Support teams with tooling, not tickets" title="Direct link to Step 3: Support teams with tooling, not tickets" translate="no">​</a></h3>
<p>Governance rules need to show up where developers already work. That means editor integration for fast feedback and a CI step that runs the same ruleset on every pull request. It also means one shared ruleset repository that teams reference instead of copying.</p>
<p>Publish the reasoning with each rule. A rule that says why it exists ("consistent error schemas let SDK generators produce typed exceptions") gets followed. A rule that only says what to do gets worked around. Provide templates and starter specs that already pass the ruleset, so the compliant path is also the easiest one.</p>
<p>Define an exception process early. Some APIs will have legitimate reasons to break a rule. An explicit exception with an owner and an expiry date keeps those cases visible instead of silently ignored.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="step-4-monitor-and-iterate">Step 4: Monitor and iterate<a href="https://choreolabs.com/api-management/learn/api-governance-guide#step-4-monitor-and-iterate" class="hash-link" aria-label="Direct link to Step 4: Monitor and iterate" title="Direct link to Step 4: Monitor and iterate" translate="no">​</a></h3>
<p>Once rules run automatically, the output becomes a compliance report rather than a review queue. Track violations by rule, team, and API over time. Rising violations on one rule usually mean the rule is unclear, the tooling is missing, or the rule no longer fits how teams build APIs.</p>
<p>Review the ruleset on a regular cadence with representatives from the teams being governed. Promote warnings to errors when adoption is high. Retire rules that no longer pay for themselves. Governance leaders who treat the ruleset as a product, with users and a backlog, tend to keep it relevant.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="when-to-formalize-governance">When to formalize governance<a href="https://choreolabs.com/api-management/learn/api-governance-guide#when-to-formalize-governance" class="hash-link" aria-label="Direct link to When to formalize governance" title="Direct link to When to formalize governance" translate="no">​</a></h3>
<p>Teams often ask when an informal style guide stops being enough. The trigger is rarely an API count. These are the operational conditions that usually force the change:</p>
<ul>
<li class="">More than one team publishes APIs independently, and conventions have started to diverge.</li>
<li class="">Partners or customers consume your APIs, so breaking changes have contractual or revenue impact.</li>
<li class="">APIs handle regulated data, and auditors need evidence of consistent controls.</li>
<li class="">An acquisition or multi-cloud move has added gateways you didn't choose.</li>
<li class="">AI agents or LLM-backed applications are starting to call your APIs or external models, and you need to attribute usage and cost.</li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="api-governance-tools-and-automation">API governance tools and automation<a href="https://choreolabs.com/api-management/learn/api-governance-guide#api-governance-tools-and-automation" class="hash-link" aria-label="Direct link to API governance tools and automation" title="Direct link to API governance tools and automation" translate="no">​</a></h2>
<p>Automated governance comes down to three kinds of tooling, mapped to the layers above. Linters enforce standards at design time. Gateways enforce policies at runtime. A control plane ties the two together with a catalog, lifecycle states, and compliance reporting.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="linters-and-ci">Linters and CI<a href="https://choreolabs.com/api-management/learn/api-governance-guide#linters-and-ci" class="hash-link" aria-label="Direct link to Linters and CI" title="Direct link to Linters and CI" translate="no">​</a></h3>
<p>Spectral is an open-source JSON and YAML linter, released under the Apache License 2.0, that needs a ruleset to check anything. It ships built-in rulesets for OpenAPI, AsyncAPI, and Arazzo, which teams extend with their own rules. Because a Spectral ruleset is a file, the same rules can run in an editor, a CI job, and a governance platform. Design-time results stay consistent across all three.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="gateway-level-policy-enforcement">Gateway-level policy enforcement<a href="https://choreolabs.com/api-management/learn/api-governance-guide#gateway-level-policy-enforcement" class="hash-link" aria-label="Direct link to Gateway-level policy enforcement" title="Direct link to Gateway-level policy enforcement" translate="no">​</a></h3>
<p>Gateways apply runtime policies such as authentication, rate limiting, and payload validation to each request. The governance question is not which gateway you run but whether every gateway applies the same baseline. Federation tools that discover APIs on existing gateways and apply common policies help when the estate already includes more than one vendor.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="developer-portal-and-catalog">Developer portal and catalog<a href="https://choreolabs.com/api-management/learn/api-governance-guide#developer-portal-and-catalog" class="hash-link" aria-label="Direct link to Developer portal and catalog" title="Direct link to Developer portal and catalog" translate="no">​</a></h3>
<p>A developer portal is where governance becomes visible to consumers. It shows which APIs are available, which versions are current, and which are deprecated. It also gives teams a reason to register APIs, because discovery is the benefit they get in return.</p>
<p>A <a class="" href="https://choreolabs.com/api-management/learn/api-governance-tools">full comparison of governance tools</a> is covered in a dedicated spoke in this series. The rest of this section uses Choreo API Management as a working example of the framework.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="governance-in-choreo-api-management">Governance in Choreo API Management<a href="https://choreolabs.com/api-management/learn/api-governance-guide#governance-in-choreo-api-management" class="hash-link" aria-label="Direct link to Governance in Choreo API Management" title="Direct link to Governance in Choreo API Management" translate="no">​</a></h2>
<p>Choreo API Management ships governance as part of the product rather than as a separate add-on. The pieces map onto the four layers like this.</p>
<p>At design time, the API Designer extension for VS Code runs governance checks using Spectral rulesets and AI. It bundles three reports: Choreo REST API AI Readiness Guidelines, Choreo REST API Design Guidelines, and OWASP API Security Top 10. Teams can point it at their own ruleset folder in a GitHub repository or on a local path. The <a href="https://wso2.com/api-platform/docs/tools/vscode-api-design/govern-apis/" target="_blank" rel="noopener noreferrer" class="">API Designer governance guide</a> makes a point worth adopting in any program. Some fixes, such as path renames, are breaking and should be treated as versioning decisions rather than silent edits.</p>
<p>In the control plane, governance is built from three concepts. Rulesets are Spectral YAML-based rule definitions. They validate the API definition, the API metadata, and the API documentation. Policies group rulesets and currently apply globally to every existing and new API proxy in the organization. Each policy is evaluated when an API proxy is updated, and violations at Error, Warn, or Info severity raise notifications. Results appear in a compliance dashboard at the organization, project, and component levels, so platform owners and API developers see the same findings at different scopes. The guide to <a href="https://wso2.com/api-platform/docs/cloud/governance/govern-api-proxy/" target="_blank" rel="noopener noreferrer" class="">configuring rulesets and governance policies</a> walks through setting them up.</p>
<p>At runtime, the API Gateway applies policies from the Policy Hub. Each policy attaches to an API or a single operation and runs on the request phase, the response phase, or both. For estates that already run other gateways, <a href="https://wso2.com/api-platform/docs/cloud/federation/overview/" target="_blank" rel="noopener noreferrer" class="">gateway federation</a> discovers APIs on AWS, Azure, Kong, and Envoy gateways and applies the same governance compliance policies to them. The APIs stay on their existing gateways.</p>
<p>For lifecycle and catalog, Choreo API Management's lifecycle states control what consumers see in the Developer Portal. The portal also serves as the API catalog, searchable by name, type, version, and description.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="extending-governance-to-ai-and-mcp-traffic">Extending governance to AI and MCP traffic<a href="https://choreolabs.com/api-management/learn/api-governance-guide#extending-governance-to-ai-and-mcp-traffic" class="hash-link" aria-label="Direct link to Extending governance to AI and MCP traffic" title="Direct link to Extending governance to AI and MCP traffic" translate="no">​</a></h3>
<p>AI workloads add two new kinds of traffic that need the same governance discipline as APIs. They are separate flows, so it helps to map each one before writing policy.</p>
<p>The first is LLM traffic. An application sends an inbound call to the gateway, and the gateway makes an outbound call to an external model provider. Policy can act on both legs: which models a caller can reach, what data goes into prompts, what comes back in responses, and who pays for each call. The second is MCP traffic. An AI agent calls tools on an MCP server, which may be one you run or one a third party runs. Here the questions are which agents can connect and which tools each one can use. The two flows carry separate policies. An MCP tool call does not pass through an LLM provider, and an LLM call does not pass through an MCP server.</p>
<p>Choreo API Management handles both in the <a href="https://wso2.com/api-platform/ai-gateway/" target="_blank" rel="noopener noreferrer" class="">AI Gateway</a>, which manages LLM APIs and MCP servers, and also governs A2A agent cards. For LLM traffic, administrators configure an LLM Provider with credentials, access control, budgets, and organization-wide guardrails. Developers then create an LLM Proxy for each application, which inherits those provider-level policies. Cost policies attach at the provider for a shared budget, or at the proxy for one application's budget. A per-application budget is what makes spend attributable to the team that owns that application. For MCP traffic, MCP Proxy policies handle authentication, per-tool authorization, access control lists, and per-tool rate limits.</p>
<p>The detail that matters for governance is shared enforcement. Guardrails use the same policy engine as API Gateway policies. In the open-source <a href="https://github.com/wso2/api-platform/tree/main/gateway" target="_blank" rel="noopener noreferrer" class="">gateway code</a>, both gateways share two components: an Envoy Proxy router that carries traffic and Go-based policies that process requests and responses. A team that already manages gateway policies for APIs applies AI guardrails through the same policy model. For the operating model behind governing agent tool access, see <a class="" href="https://choreolabs.com/api-management/learn/governing-third-party-mcp">governing third-party MCP tools</a>.</p>
<p>The qualitative triggers for extending governance to AI traffic mirror the ones for APIs. They include multi-model usage, the need to attribute LLM spend to teams or products, regulated data in prompts, and agent pilots moving into production.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="common-governance-pitfalls">Common governance pitfalls<a href="https://choreolabs.com/api-management/learn/api-governance-guide#common-governance-pitfalls" class="hash-link" aria-label="Direct link to Common governance pitfalls" title="Direct link to Common governance pitfalls" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="governance-as-a-bottleneck">Governance as a bottleneck<a href="https://choreolabs.com/api-management/learn/api-governance-guide#governance-as-a-bottleneck" class="hash-link" aria-label="Direct link to Governance as a bottleneck" title="Direct link to Governance as a bottleneck" translate="no">​</a></h3>
<p>A central team that reviews every change by hand becomes the slowest step in delivery. Teams respond by shipping around it, which produces exactly the inconsistency governance was meant to prevent. The fix is to automate the routine checks and reserve human review for exceptions and new patterns.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="governance-theater">Governance theater<a href="https://choreolabs.com/api-management/learn/api-governance-guide#governance-theater" class="hash-link" aria-label="Direct link to Governance theater" title="Direct link to Governance theater" translate="no">​</a></h3>
<p>A program that publishes standards but never checks them gives the appearance of control without the effect. Dashboards full of warnings nobody acts on are a version of the same problem. Every rule needs an owner, a severity, and a consequence, even if the consequence is only a visible report that a team lead reads.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="governing-only-at-design-time">Governing only at design time<a href="https://choreolabs.com/api-management/learn/api-governance-guide#governing-only-at-design-time" class="hash-link" aria-label="Direct link to Governing only at design time" title="Direct link to Governing only at design time" translate="no">​</a></h3>
<p>A clean OpenAPI document says nothing about what is deployed. Definitions drift from implementations, old versions keep running after a replacement ships, and test hosts get exposed. Runtime policy, a current catalog, and enforced retirement close the gap between the contract and production.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="writing-rules-nobody-can-check">Writing rules nobody can check<a href="https://choreolabs.com/api-management/learn/api-governance-guide#writing-rules-nobody-can-check" class="hash-link" aria-label="Direct link to Writing rules nobody can check" title="Direct link to Writing rules nobody can check" translate="no">​</a></h3>
<p>Rules such as "use clear names" generate debate rather than compliance. Rewrite each one as a checkable rule, or move it into guidance and stop calling it a rule.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="treating-every-rule-as-equally-important">Treating every rule as equally important<a href="https://choreolabs.com/api-management/learn/api-governance-guide#treating-every-rule-as-equally-important" class="hash-link" aria-label="Direct link to Treating every rule as equally important" title="Direct link to Treating every rule as equally important" translate="no">​</a></h3>
<p>A ruleset with dozens of errors on day one gets disabled. Start with a short list of blocking rules tied to real risk, and expand the list as teams adapt.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="leaving-ai-traffic-out-of-scope">Leaving AI traffic out of scope<a href="https://choreolabs.com/api-management/learn/api-governance-guide#leaving-ai-traffic-out-of-scope" class="hash-link" aria-label="Direct link to Leaving AI traffic out of scope" title="Direct link to Leaving AI traffic out of scope" translate="no">​</a></h3>
<p>LLM calls and agent tool calls often start as side projects outside the API program. By the time they reach production, they often carry customer data and real spend without the controls the API program already applies.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/api-governance-guide#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>API governance works when every rule has a place to run. A standard that lives only on a wiki page depends on people remembering it, and that stops working once several teams ship APIs on their own schedules. The four layers give each rule a home. Standards define it, and design-time checks catch it in the contract. Gateway policies hold it on live traffic, while the lifecycle and catalog track every API it applies to.</p>
<p>The model decision matters as much as the rules. The model decides who owns standards, exceptions, and domain decisions. Automation decides whether those decisions hold up as more teams publish. Manual review becomes a queue in either model, so automate the routine checks whichever model you choose.</p>
<p>Start with what you can see and check. Inventory the APIs you run, enforce a short list of rules tied to real risk, and widen the scope as teams adopt them. That scope now includes LLM calls and agent tool calls, which need the same controls your APIs already have.</p>
<p>To put the framework into practice, <a href="https://wso2.com/api-platform/api-governance/" target="_blank" rel="noopener noreferrer" class="">Choreo API Governance</a> brings rulesets, policies, compliance monitoring, lifecycle management, and the API catalog together under one control plane. The same control plane manages AI traffic through the AI Gateway, so API and AI policies are governed from the same management.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/api-governance-guide#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-does-api-stand-for">What does API stand for?<a href="https://choreolabs.com/api-management/learn/api-governance-guide#what-does-api-stand-for" class="hash-link" aria-label="Direct link to What does API stand for?" title="Direct link to What does API stand for?" translate="no">​</a></h3>
<p>API stands for application programming interface. It is a defined contract that lets one piece of software request data or actions from another.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-the-difference-between-api-governance-and-api-management">What is the difference between API governance and API management?<a href="https://choreolabs.com/api-management/learn/api-governance-guide#what-is-the-difference-between-api-governance-and-api-management" class="hash-link" aria-label="Direct link to What is the difference between API governance and API management?" title="Direct link to What is the difference between API governance and API management?" translate="no">​</a></h3>
<p>API governance defines the rules APIs must follow. API management provides the gateways, portal, and lifecycle tooling that apply those rules to running APIs.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-an-api-governance-framework">What is an API governance framework?<a href="https://choreolabs.com/api-management/learn/api-governance-guide#what-is-an-api-governance-framework" class="hash-link" aria-label="Direct link to What is an API governance framework?" title="Direct link to What is an API governance framework?" translate="no">​</a></h3>
<p>An API governance framework is a structured model for writing standards and enforcing them. A common structure has four layers: standards, design-time checks, runtime policies, and lifecycle and catalog management.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-tools-are-used-for-api-governance">What tools are used for API governance?<a href="https://choreolabs.com/api-management/learn/api-governance-guide#what-tools-are-used-for-api-governance" class="hash-link" aria-label="Direct link to What tools are used for API governance?" title="Direct link to What tools are used for API governance?" translate="no">​</a></h3>
<p>Most teams combine an API linter such as Spectral with a CI job that runs it. They add gateways for runtime policies and a control plane for the catalog and compliance reporting.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="who-owns-api-governance">Who owns API governance?<a href="https://choreolabs.com/api-management/learn/api-governance-guide#who-owns-api-governance" class="hash-link" aria-label="Direct link to Who owns API governance?" title="Direct link to Who owns API governance?" translate="no">​</a></h3>
<p>Ownership usually sits with an API program owner or a small governance group, with platform engineering running the tooling. In federated models, domain teams own enforcement for their own APIs within shared standards.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="does-api-governance-apply-to-ai-and-mcp-traffic">Does API governance apply to AI and MCP traffic?<a href="https://choreolabs.com/api-management/learn/api-governance-guide#does-api-governance-apply-to-ai-and-mcp-traffic" class="hash-link" aria-label="Direct link to Does API governance apply to AI and MCP traffic?" title="Direct link to Does API governance apply to AI and MCP traffic?" translate="no">​</a></h3>
<p>Yes. LLM calls and MCP tool calls need the same controls as APIs: authentication, authorization, rate limits, data protection, and usage attribution. Many teams apply them through an AI gateway that shares policies with their API gateway.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="API governance" term="API governance"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[API governance tools: Standards, linting & catalog (2026)]]></title>
        <id>https://choreolabs.com/api-management/learn/api-governance-tools</id>
        <link href="https://choreolabs.com/api-management/learn/api-governance-tools"/>
        <updated>2026-10-09T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[The API governance tools worth knowing: linting, security scanning, catalogs, and platform-native governance, organized by what they actually do.]]></summary>
        <content type="html"><![CDATA[<p>Most API style guides start life as a wiki page. A platform team writes down naming conventions, pagination rules and approved auth schemes. Then three product teams ship APIs that follow about half of it. Nobody ignored the guide on purpose, but nothing checked the spec against it before release. API governance tools close that gap by turning written standards into checks that run on their own. This guide sorts the tools by the job they do and shows how to combine them into the <a class="" href="https://choreolabs.com/api-management/learn/api-governance-framework/">broader governance framework</a> these tools help implement.</p>
<aside class="cta_6F_m" aria-label="Turn governance rules into automated checks" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Turn governance rules into automated checks</span><span class="desc_GzK8">Use Choreo API Management to bake design-time governance policies and linting into every API release.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=api-governance-tools" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-api-governance-tools-actually-do">What API governance tools actually do<a href="https://choreolabs.com/api-management/learn/api-governance-tools#what-api-governance-tools-actually-do" class="hash-link" aria-label="Direct link to What API governance tools actually do" title="Direct link to What API governance tools actually do" translate="no">​</a></h2>
<p>API governance tools check APIs against an organization's standards for design, security and documentation, then report or block violations. They work at three points: when a spec is written, before an API is released, and while it runs. Most programs need coverage at more than one point.</p>
<p>A single rule shows why. Say every collection endpoint must declare a page size limit. A linter catches the missing limit parameter in a pull request. A dynamic scanner checks whether the running API honors the limit its contract declares. A catalog tells you which older APIs predate the rule and still need fixing.</p>
<p>The third check is easy to skip, and it is a recognized risk. The OWASP API Security Top 10 lists <a href="https://api-security.owasp.org/editions/2023/en/0xa9-improper-inventory-management/" target="_blank" rel="noopener noreferrer" class="">Improper Inventory Management as API9:2023.</a> Undocumented endpoints and forgotten versions are hard to govern because no rule ever runs against them. API standards tools only help with the APIs they can see.</p>
<p>The people using these tools differ by stage. API developers meet governance as lint output in their editor or pull request. Security teams care about audit scores and exposure reports. Platform owners need the organization-wide view: which teams comply, which rules fail most often, and which APIs have no owner. A tool that serves only one of these groups usually needs a partner tool for the others.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-to-look-for-in-a-governance-tool">What to look for in a governance tool<a href="https://choreolabs.com/api-management/learn/api-governance-tools#what-to-look-for-in-a-governance-tool" class="hash-link" aria-label="Direct link to What to look for in a governance tool" title="Direct link to What to look for in a governance tool" translate="no">​</a></h2>
<p>Feature lists make API governance tools look more alike than they are. These seven criteria separate them in practice.</p>
<ul>
<li class=""><strong>Spec format support:</strong> Check which formats and versions the tool parses, such as OpenAPI 2.0 to 3.2, AsyncAPI 2.x and 3.0, Arazzo 1.x and GraphQL. Support often differs by version, so confirm the versions your teams actually use. A tool that reads only OpenAPI leaves event-driven APIs ungoverned.</li>
<li class=""><strong>Rules as code:</strong> Governance rules should live in versioned files that go through code review. Spectral's ruleset format is common ground here. Postman, Kong Konnect, IBM API Connect, Gravitee and Choreo API Management all evaluate Spectral rules. Support is not identical, though. Some platforms accept only Spectral's built-in core functions in custom rulesets, while others also accept custom functions. Rulesets that stick to core functions move between tools with the least rework.</li>
<li class=""><strong>CI/CD integration:</strong> Look for a CLI with meaningful exit codes, machine-readable output and pull request annotations. A check that only runs in a web console runs too late.</li>
<li class=""><strong>Enforcement mode:</strong> Some tools advise and some gate. Know which mode applies at which lifecycle stage, because hard gates on day one push teams to work around the tool. Advisory checks forever let drift build up.</li>
<li class=""><strong>Role-based access control and audit trails:</strong> Ask who can edit rulesets, who can approve exceptions and what record the tool keeps. Auditors tend to ask these questions early.</li>
<li class=""><strong>API catalog:</strong> Each API should map to an owner, a version and a compliance status. Without that record, governance reporting has no denominator.</li>
<li class=""><strong>Compliance reporting:</strong> Findings should carry a severity and point to the standard behind the rule. Dashboards that break results down by team show where to spend review time.</li>
</ul>
<p>No single governance solution scores well on all seven. The categories below show where each tool is strongest.</p>
<p>Choose the enforcement checkpoints before choosing products. The <a class="" href="https://choreolabs.com/api-management/learn/api-governance-framework/">governance framework walkthrough</a> maps rules to owners and lifecycle stages; the <a class="" href="https://choreolabs.com/api-management/learn/api-governance-best-practices/">best-practices guide</a> shows how to operate those checks in CI.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="design-time-tools-linting-and-style-enforcement">Design-time tools: Linting and style enforcement<a href="https://choreolabs.com/api-management/learn/api-governance-tools#design-time-tools-linting-and-style-enforcement" class="hash-link" aria-label="Direct link to Design-time tools: Linting and style enforcement" title="Direct link to Design-time tools: Linting and style enforcement" translate="no">​</a></h2>
<p>Design-time tools are where most API standardization work happens. They read the API description file, not the running service. That makes them cheap to run and easy to add. A spec that fails a lint check costs a pull request comment. The same mistake found after release costs a new API version. Event-driven APIs deserve the same treatment as REST. AsyncAPI files describe channels, messages and payload schemas, and naming drift there is just as costly. Choose a linter that reads both formats if your estate includes message brokers.</p>
<p>For a broader view of the design stack these tools plug into, see <a class="" href="https://choreolabs.com/api-management/learn/api-design-tools">how to choose API design tools for API-first teams.</a></p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="spectral">Spectral<a href="https://choreolabs.com/api-management/learn/api-governance-tools#spectral" class="hash-link" aria-label="Direct link to Spectral" title="Direct link to Spectral" translate="no">​</a></h3>
<p>Spectral is an open source JSON and YAML linter released under the Apache 2.0 license. It ships rulesets for OpenAPI 2.0 and 3.x, AsyncAPI and Arazzo. Teams extend those rulesets with custom rules for naming conventions, required fields and security schemes.</p>
<p>A ruleset is a YAML file, so it can sit in the same repository as your style guides. Most teams start by extending the built-in OpenAPI ruleset and adding a handful of house rules. Spectral matters beyond its own CLI because so many platforms reuse its format. A rule you write today can follow you to a different enforcement point later.</p>
<p>Severity is the setting that decides whether teams accept a linter. A useful rollout pattern is to introduce new rules as warnings first. Watch how often they fire across existing specs, then fix the noisy ones before promoting them to errors. A rule that flags half the estate on its first run is usually wrong, too broad, or both. Rules about security schemes and required error formats tend to earn error severity quickly. Rules about description wording rarely need to block anything.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="redocly-cli">Redocly CLI<a href="https://choreolabs.com/api-management/learn/api-governance-tools#redocly-cli" class="hash-link" aria-label="Direct link to Redocly CLI" title="Direct link to Redocly CLI" translate="no">​</a></h3>
<p>Redocly CLI is an open source command-line tool with a <code>lint</code> command. Its structural rule currently covers OpenAPI 2.0, 3.0, 3.1 and 3.2, AsyncAPI 2.6 and 3.0, and Arazzo 1.x. It also has experimental support for linting Overlay 1.x files. Format support can differ between commands and individual rules, so check the ones you plan to use. It offers built-in rules plus configurable rules for patterns the defaults do not cover.</p>
<p>Redocly CLI also bundles multi-file specs and builds reference documentation. Teams that already use it for docs get linting from the same tool. Its <code>--format=github-actions</code> output turns findings into inline pull request annotations.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="oasdiff">oasdiff<a href="https://choreolabs.com/api-management/learn/api-governance-tools#oasdiff" class="hash-link" aria-label="Direct link to oasdiff" title="Direct link to oasdiff" translate="no">​</a></h3>
<p>Linters judge one version of a spec. They do not see what changed between two versions. oasdiff is an open source CLI that compares two OpenAPI files and reports breaking changes. It can exit with an error when it finds them, which makes it a natural CI step. Breaking-change detection becomes important once external consumers depend on your API versions. A renamed field can pass every style rule and still break every client.</p>
<p>All three tools run as a pipeline stage. This walkthrough on <a class="" href="https://choreolabs.com/api-management/learn/api-ci-cd-pipeline">building a CI/CD pipeline for APIs</a> shows where lint and diff checks fit. Their shared limitation is scope. They can confirm a spec is well formed and consistent. They cannot confirm the deployed API matches it.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="security-and-compliance-tools">Security and compliance tools<a href="https://choreolabs.com/api-management/learn/api-governance-tools#security-and-compliance-tools" class="hash-link" aria-label="Direct link to Security and compliance tools" title="Direct link to Security and compliance tools" translate="no">​</a></h2>
<p>Security-focused tools look for weaknesses rather than style problems. They split into two groups: static tools audit the contract, while dynamic and posture tools look at what is actually deployed.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="42crunch">42Crunch<a href="https://choreolabs.com/api-management/learn/api-governance-tools#42crunch" class="hash-link" aria-label="Direct link to 42Crunch" title="Direct link to 42Crunch" translate="no">​</a></h3>
<p>42Crunch offers two checks that work as a pair. Its API Audit performs static analysis of an OpenAPI definition and gives it a security score from 0 to 100. A CI job can fail any contract that scores below a minimum you set. Its Conformance Scan tests a running API against its own contract to find differences between the two.</p>
<p>This split mirrors the gap the linters leave. The audit finds weak authentication definitions or loose data constraints in the spec. The scan finds endpoints whose behavior does not match what the spec promises. That second class of problem is invisible to any design-time tool.</p>
<p>Some risks need both views. Authorization flaws, such as one user reading another user's records, depend on how the server handles real requests. A spec can declare that an endpoint requires a token. It cannot show whether the server checks that the token's owner is allowed to see the requested object. Static audits narrow the attack surface, and runtime tests confirm what the server actually does.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="wiz">Wiz<a href="https://choreolabs.com/api-management/learn/api-governance-tools#wiz" class="hash-link" aria-label="Direct link to Wiz" title="Direct link to Wiz" translate="no">​</a></h3>
<p>Wiz approaches API security from the cloud side. It scans cloud accounts without agents and builds an inventory that includes the APIs it finds. Its Dynamic Scanner, offered to Wiz Advanced customers, flags APIs that are publicly exposed and unauthenticated. Coverage depends on which cloud accounts are connected and what the scanner can reach. APIs that require credentials or sit on private networks need other tests.</p>
<p>Wiz does not read your style guide or score your OpenAPI files. Its value for governance is coverage. It can surface an API that no team registered and no linter ever saw. Teams that run it alongside a contract-first tool get two views of the same estate: what was declared and what is reachable.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="a-note-on-compliance-with-regulations">A note on compliance with regulations<a href="https://choreolabs.com/api-management/learn/api-governance-tools#a-note-on-compliance-with-regulations" class="hash-link" aria-label="Direct link to A note on compliance with regulations" title="Direct link to A note on compliance with regulations" translate="no">​</a></h3>
<p>These tools produce evidence, not compliance. A scanner can show whether the payment APIs it reaches enforce authentication. What it reaches depends on credentials, network access, discovery settings and the specs it has. It cannot decide whether your controls satisfy a specific regulation. That judgment stays with your security and compliance teams. What changes is that the evidence exists before the audit starts.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="catalog-and-discovery-tools">Catalog and discovery tools<a href="https://choreolabs.com/api-management/learn/api-governance-tools#catalog-and-discovery-tools" class="hash-link" aria-label="Direct link to Catalog and discovery tools" title="Direct link to Catalog and discovery tools" translate="no">​</a></h2>
<p>API sprawl is mostly an inventory problem. When nobody can list every API, its owner and its version, governance reports describe only the APIs someone remembered to register. Catalog tools fix this in two different ways.</p>
<p>Declared catalogs depend on teams describing their own APIs. Backstage is the open source example most teams know. Its software catalog includes an API entity kind for OpenAPI, AsyncAPI, GraphQL and gRPC definitions. Each entry links to the component that exposes it and the team that owns it. <a href="https://backstage.io/docs/features/software-catalog/creating-the-catalog-graph" target="_blank" rel="noopener noreferrer" class="">The Backstage documentation advises</a> against treating the catalog as the ultimate source of truth. It is only as accurate as the files teams keep up to date.</p>
<p>Whatever tool holds the catalog, each entry should answer a few questions without a meeting. Who owns this API? Which lifecycle state is it in? Which versions are live, and which consumers depend on them? Does it handle sensitive data? What is its current compliance status? Entries that cannot answer these questions are placeholders, not inventory.</p>
<p>Discovered inventories start from infrastructure instead. Gateways, cloud accounts and traffic tell you what is running, whether or not anyone registered it. Wiz, covered above, works this way. Several full platforms below can also discover APIs from external gateways.</p>
<p>Each approach misses something: declared catalogs miss shadow APIs, and discovered inventories miss ownership and intent. Mature programs reconcile the two lists on a schedule. An API that appears in traffic but not in the catalog becomes a governance finding with an owner to assign. An API that appears in the catalog but never receives traffic becomes a candidate for retirement.</p>
<p>A developer portal often becomes the consumer-facing view of the catalog. If you do not have a portal yet, start with this guide to <a class="" href="https://choreolabs.com/api-management/learn/internal-developer-portal">building an internal developer portal.</a></p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="full-platform-governance-api-management-suites">Full-platform governance: API management suites<a href="https://choreolabs.com/api-management/learn/api-governance-tools#full-platform-governance-api-management-suites" class="hash-link" aria-label="Direct link to Full-platform governance: API management suites" title="Direct link to Full-platform governance: API management suites" translate="no">​</a></h2>
<p>API management platforms bundle governance capabilities with the gateway, developer portal and API lifecycle management. Their main advantage is context. A rule violation attaches to the same API record that holds its owner, its deployment target and its subscribers. Their main trade-off is reach. A platform governs the APIs it manages, plus whatever it can import or discover. This section also covers Postman, a governance workspace rather than a gateway, because teams often shortlist it alongside these platforms.</p>
<p>When you evaluate platforms, ask the same four questions of each one:</p>
<ol>
<li class=""><strong>Where do rules run?</strong> On import, on update, before publishing or before deployment.</li>
<li class=""><strong>What happens on a violation?</strong> A notification, a dashboard entry, or a blocked action.</li>
<li class=""><strong>Which APIs are in scope?</strong> Only those created on the platform, or also APIs discovered on other gateways.</li>
<li class=""><strong>Where does the evidence live?</strong> A per-API compliance view that developers see, and an organization view that auditors can use.</li>
</ol>
<p>The answers differ more than the marketing pages suggest. They also change between releases, so check current documentation before you shortlist.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="postman">Postman<a href="https://choreolabs.com/api-management/learn/api-governance-tools#postman" class="hash-link" aria-label="Direct link to Postman" title="Direct link to Postman" translate="no">​</a></h3>
<p>Postman approaches governance from the design and collaboration side. Configurable governance rules are available on Enterprise plans and apply to API specifications in Spec Hub. Teams write custom rules and functions in Spectral. Governance groups then apply rulesets to selected workspaces.</p>
<p>Postman fits teams whose API work already happens in Postman workspaces. It is not a gateway, so runtime policy enforcement lives elsewhere.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="kong-konnect">Kong Konnect<a href="https://choreolabs.com/api-management/learn/api-governance-tools#kong-konnect" class="hash-link" aria-label="Direct link to Kong Konnect" title="Direct link to Kong Konnect" translate="no">​</a></h3>
<p>Kong Konnect Catalog Classic pulls APIs and specs from integrations such as AWS API Gateway, GitHub and SwaggerHub. Its scorecards then evaluate each catalog service. One scorecard criterion lints attached specs against Spectral-based rulesets, including an OWASP Top 10 ruleset.</p>
<p>Scorecards measure more than spec quality. They can also check operational signals, such as whether a service has an on-call engineer assigned. That makes Konnect's model closer to service maturity tracking than to a pure spec linter.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="ibm-api-connect">IBM API connect<a href="https://choreolabs.com/api-management/learn/api-governance-tools#ibm-api-connect" class="hash-link" aria-label="Direct link to IBM API connect" title="Direct link to IBM API connect" translate="no">​</a></h3>
<p>IBM API Connect provides governance through an add-on governance service built on Spectral. Global rulesets from IBM and Spectral, including OWASP and OpenAPI sets, come preconfigured and cannot be edited. Provider organizations add their own custom rulesets. When an API's tags match a ruleset's tags, that ruleset is preselected for the validation scan.</p>
<p>The locked global rulesets suit large enterprises and regulated environments that need a fixed baseline. Teams layer their own rules on top without changing the shared standard.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="choreo-api-management">Choreo API Management<a href="https://choreolabs.com/api-management/learn/api-governance-tools#choreo-api-management" class="hash-link" aria-label="Direct link to Choreo API Management" title="Direct link to Choreo API Management" translate="no">​</a></h3>
<p><a href="https://wso2.com/api-platform/api-governance/" target="_blank" rel="noopener noreferrer" class="">Choreo API Management</a> builds governance into the same control plane that manages its gateways, developer portal and API lifecycle. Its model has three parts.</p>
<ol>
<li class=""><strong>Rulesets:</strong> Rulesets are Spectral YAML definitions. The governance overview describes rulesets for REST and Async APIs that can validate the API definition, the API metadata or the API documentation. The setup guide walks through API definition rulesets, so confirm which artifact types your workflow supports. Built-in rulesets include Choreo API Design Guidelines, Choreo Style Guidelines and an OWASP Top 10 ruleset.</li>
<li class=""><strong>Documents:</strong> Some standards are hard to express as JSONPath rules. For those, teams upload governance documents as PDFs written in natural language. Choreo API Management's AI engine evaluates APIs against them.</li>
<li class=""><strong>Policies:</strong> A policy attaches one or more rulesets to the organization's API proxies. Policies currently apply globally to all existing and new API proxies. Choreo API Management evaluates them when a proxy is updated and sends notifications for Error, Warn and Info findings.</li>
</ol>
<p>Results appear in a Compliance Dashboard at the organization, project and component levels. Platform owners see the overall picture. API developers see what to fix on their own API. <a href="https://wso2.com/api-platform/docs/cloud/governance/govern-api-proxy/" target="_blank" rel="noopener noreferrer" class="">The governance setup guide</a> walks through creating rulesets and policies.</p>
<p>Because rulesets use the Spectral format, a ruleset built on Spectral's core functions can also run in CI with the open source Spectral CLI. Teams get pull request feedback early and a platform-level compliance record later, from one source of rules.</p>
<p>Enforcement in Choreo API Management is currently notification-based. Teams that need a human gate can pair it with workflow approvals. These require an authorized reviewer to approve before an API proxy is promoted to a production environment.</p>
<p>Reach extends past APIs managed in Choreo API Management through gateway federation. Choreo API Management discovers APIs on external gateways such as AWS API Gateway, applies governance policies to them and lists them in the Developer Portal. The APIs stay on their original gateway.</p>
<p>The same control plane also covers AI traffic through Choreo AI Gateway. These are separate flows with separate controls. Outbound calls from applications to LLM providers get guardrails and token-based rate limits. Inbound calls from AI agents to MCP servers get authentication, authorization and tool-level access control. Design-time rulesets do not sit in either request path. They evaluate API definitions, not live traffic.</p>
<p>Deployment options include cloud, hybrid and self-hosted. That matters when governance data must stay inside a specific network boundary.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="gravitee">Gravitee<a href="https://choreolabs.com/api-management/learn/api-governance-tools#gravitee" class="hash-link" aria-label="Direct link to Gravitee" title="Direct link to Gravitee" translate="no">​</a></h3>
<p>Gravitee's API Score uses Spectral to rate APIs on criteria such as security, documentation and consistency. It evaluates the Gravitee API definition plus any attached OpenAPI or AsyncAPI pages. It also covers federated APIs. API Score is static and does not test the data plane.</p>
<p>API Score is currently a technology preview and is off by default. Teams evaluating it for production governance should confirm its status first.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="comparison-table">Comparison table<a href="https://choreolabs.com/api-management/learn/api-governance-tools#comparison-table" class="hash-link" aria-label="Direct link to Comparison table" title="Direct link to Comparison table" translate="no">​</a></h2>
<table><thead><tr><th>Tool</th><th>Category</th><th>Open-source core?</th><th>Standout capability</th></tr></thead><tbody><tr><td>Spectral</td><td>Design-time</td><td>Yes</td><td>Ruleset format reused by most platforms in this list</td></tr><tr><td>Redocly CLI</td><td>Design-time</td><td>Yes</td><td>Linting and docs builds from one CLI</td></tr><tr><td>oasdiff</td><td>Design-time</td><td>Yes</td><td>Breaking-change detection between spec versions</td></tr><tr><td>42Crunch</td><td>Security</td><td>No</td><td>Static contract audit paired with live conformance testing</td></tr><tr><td>Wiz</td><td>Security</td><td>No</td><td>Agentless discovery of exposed, unauthenticated APIs</td></tr><tr><td>Backstage</td><td>Catalog</td><td>Yes</td><td>Ownership graph linking APIs to components and teams</td></tr><tr><td>Postman</td><td>Governance workspace</td><td>No</td><td>Spectral rules scoped to collaboration workspaces</td></tr><tr><td>Kong Konnect</td><td>Full-platform</td><td>Kong Gateway core (Apache 2.0). Konnect is a commercial SaaS platform</td><td>Scorecards that mix spec linting with operational checks</td></tr><tr><td>IBM API Connect</td><td>Full-platform</td><td>No</td><td>Locked baseline rulesets with tag-based selection</td></tr><tr><td>Choreo API Management</td><td>Full-platform</td><td>Core runtime (Apache 2.0)</td><td>Spectral and natural-language rules with federated API coverage</td></tr><tr><td>Gravitee</td><td>Full-platform</td><td>APIM Community Edition. API Score requires Gravitee Cloud, or a self-hosted installation connected to it</td><td>Scoring of platform API definitions (technology preview)</td></tr></tbody></table>
<p>The open-source column describes each product's core. It does not mean the listed governance feature is included in a free or open-source edition. Plans, editions and connected cloud services can change what is available, so check entitlements before you shortlist.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="choosing-the-right-combination">Choosing the right combination<a href="https://choreolabs.com/api-management/learn/api-governance-tools#choosing-the-right-combination" class="hash-link" aria-label="Direct link to Choosing the right combination" title="Direct link to Choosing the right combination" translate="no">​</a></h2>
<p>When comparing the top API governance tools, the useful question is not which one wins. It is which failure you need to catch next. Most mature programs end up with a linter, a security scanner and either a dedicated catalog or a full platform. They rarely adopt all of them at once. Real operational triggers set the order.</p>
<ul>
<li class=""><strong>Start with linting in CI.</strong> It is the cheapest control and the easiest to reverse. Pick a ruleset, run it in advisory mode, and promote the rules teams agree on to errors.</li>
<li class=""><strong>Add breaking-change detection when consumers depend on your versions.</strong> External partners and mobile clients are the usual trigger. A diff check catches what a style rule never will.</li>
<li class=""><strong>Add security auditing when APIs carry sensitive data or face the internet.</strong> Regulated data in requests and responses is the clearest signal. Pair a static audit with a dynamic check so contract and behavior are both covered.</li>
<li class=""><strong>Add a catalog when ownership questions slow you down.</strong> If an incident or an audit request leaves you hunting for an API's owner, inventory is the gap.</li>
<li class=""><strong>Move to a governance platform when evidence must follow the API.</strong> Common triggers are multiple gateways, audit requirements tied to deployment, and a need to govern AI traffic under the same control plane as APIs.</li>
</ul>
<p>Whatever you adopt, keep one source of governance rules. A Spectral ruleset in version control can run in a developer's editor, in CI and on a platform. That consistency is what lets an effective API governance model scale across teams.</p>
<p>Plan for exceptions from the start. Every governance program meets an API that cannot follow a rule yet, usually because of a legacy client or a partner contract. Give teams a documented way to request a waiver, record who approved it, and set a review date. Without that path, teams either ignore the rule or disable the check. Both outcomes are worse than a tracked exception.</p>
<p>Finally, measure the program by trend rather than by score. Track error-level findings per assessed API, and record rule-set versions and scan coverage beside each count. A raw count can fall simply because fewer APIs were scanned or a rule was disabled. Normalized this way, a falling trend says more than any single API's rating. So does a shrinking list of APIs with no owner.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/api-governance-tools#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>API governance tools do different jobs, and none of them covers the whole lifecycle alone. Linters keep specs consistent before code ships. Security tools test both the contract and the running API. Catalogs answer who owns what, and full platforms tie findings to the API record that teams deploy and publish.</p>
<p>The practical path is incremental. Start with one ruleset in CI and treat it as the shared definition of a good API. Add scanning, inventory and platform enforcement as real triggers appear, such as external consumers, sensitive data, ownership gaps or multiple gateways. Keep every rule in one versioned source so each new tool enforces the same standard instead of a slightly different one.</p>
<p>If you want governance for APIs and AI traffic managed from one control plane, explore <a href="https://wso2.com/api-platform/api-governance/" target="_blank" rel="noopener noreferrer" class="">Choreo API Governance</a> and test it against a ruleset your teams already use.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/api-governance-tools#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-are-api-governance-tools">What are API governance tools?<a href="https://choreolabs.com/api-management/learn/api-governance-tools#what-are-api-governance-tools" class="hash-link" aria-label="Direct link to What are API governance tools?" title="Direct link to What are API governance tools?" translate="no">​</a></h3>
<p>They are tools that test APIs against an organization's rules for design, security and documentation. They include linters, security scanners, catalogs and API management platforms.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-the-difference-between-api-linting-and-api-governance">What is the difference between API linting and API governance?<a href="https://choreolabs.com/api-management/learn/api-governance-tools#what-is-the-difference-between-api-linting-and-api-governance" class="hash-link" aria-label="Direct link to What is the difference between API linting and API governance?" title="Direct link to What is the difference between API linting and API governance?" translate="no">​</a></h3>
<p>Linting checks a single API description against rules. API governance is the wider program that sets the rules, assigns owners, handles exceptions and tracks compliance over time.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="do-i-need-a-separate-api-catalog-if-i-use-an-api-management-platform">Do I need a separate API catalog if I use an API management platform?<a href="https://choreolabs.com/api-management/learn/api-governance-tools#do-i-need-a-separate-api-catalog-if-i-use-an-api-management-platform" class="hash-link" aria-label="Direct link to Do I need a separate API catalog if I use an API management platform?" title="Direct link to Do I need a separate API catalog if I use an API management platform?" translate="no">​</a></h3>
<p>Not always. A platform catalogs the APIs it manages or discovers. A separate catalog helps when many APIs live outside any platform or when you track ownership across all software, not only APIs.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="can-api-governance-tools-guarantee-regulatory-compliance">Can API governance tools guarantee regulatory compliance?<a href="https://choreolabs.com/api-management/learn/api-governance-tools#can-api-governance-tools-guarantee-regulatory-compliance" class="hash-link" aria-label="Direct link to Can API governance tools guarantee regulatory compliance?" title="Direct link to Can API governance tools guarantee regulatory compliance?" translate="no">​</a></h3>
<p>No. They produce evidence, such as audit results and compliance reports. Your compliance team still decides whether controls meet a given regulation.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="API governance" term="API governance"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[API lifecycle management: Design to retirement guide]]></title>
        <id>https://choreolabs.com/api-management/learn/api-lifecycle-management</id>
        <link href="https://choreolabs.com/api-management/learn/api-lifecycle-management"/>
        <updated>2026-10-09T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[A full walkthrough of API lifecycle management: design, publish, version, deprecate, and retire APIs without disrupting the teams that depend on them.]]></summary>
        <content type="html"><![CDATA[<p>Most API programs are good at launching APIs. Fewer are good at ending them. A new version ships and the old one keeps answering requests. Two years later, nobody can say which consumers still depend on it or who owns the code behind it. That gap is where API lifecycle management earns its keep.</p>
<p>This guide is for platform leads and architects who run an API program, not a single API. It walks through each stage an API passes through, from the first contract to retirement. It spends the most time on deprecation and retirement, because those are the stages that cause the most production pain. It closes with the metrics that show whether your lifecycle is actually working, and how Choreo API Management tracks each stage as platform state.</p>
<aside class="cta_6F_m" aria-label="Manage every API stage, including retirement" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Manage every API stage, including retirement</span><span class="desc_GzK8">Try Choreo API Management to track every API lifecycle stage, from design through deprecation and retirement.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=api-lifecycle-management" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-api-lifecycle-management">What is API lifecycle management?<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#what-is-api-lifecycle-management" class="hash-link" aria-label="Direct link to What is API lifecycle management?" title="Direct link to What is API lifecycle management?" translate="no">​</a></h2>
<p>API lifecycle management is the structured oversight of an API from initial design through retirement. It covers how an API is designed, tested, published, versioned, deprecated, and removed, and who approves each move. It is a governance discipline as much as a technical one. Its job is to keep an API portfolio healthy as it grows.</p>
<p>The distinction from API management matters. API management usually refers to the runtime and consumption side: the gateway, security policies, rate limiting, and the developer portal. API lifecycle management is the layer that decides which state each API is in and what is allowed to happen next. A gateway can enforce a rate limit on a deprecated API. It takes lifecycle management to know that the API is deprecated, who still calls it, and when it is scheduled to go away.</p>
<p>In practice, this shows up as a set of tracked states rather than a document. An API is either in design, available for early testing, published, deprecated, or retired. Every team, from the API producers to the security reviewers, reads the same status.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-stages-of-the-api-lifecycle">The stages of the API lifecycle<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#the-stages-of-the-api-lifecycle" class="hash-link" aria-label="Direct link to The stages of the API lifecycle" title="Direct link to The stages of the API lifecycle" translate="no">​</a></h2>
<p>There is no single canonical stage count. Published frameworks range from a few broad phases to eight or more distinct stages, depending on how finely they split design, testing, security, and operations. The counts differ, but the journey underneath them is the same. This guide uses six stages: Design, Develop and Test, Publish, Version, Deprecate, and Retire.</p>
<p><img decoding="async" loading="lazy" alt="The API lifecycle as a loop of six stages, each a tracked state with a named owner: 1 Design, where the contract is defined before code, 2 Develop and Test, where the API is built and verified against the contract, 3 Publish, where it becomes discoverable in the developer portal, 4 Version, where changes are released as new versions, 5 Deprecate, where it is marked for removal while consumers migrate, and 6 Retire, where it is removed from service and inventory. A dashed arrow from Retire back to Design shows that lessons feed the next design" src="https://choreolabs.com/api-management/learn/assets/images/api-lifecycle-stages-bad7197dc48644f4e2acc3a8b6c07a48.webp" width="3200" height="2678" class="img_ev3q"></p>
<p><em>Six stages, one loop: every stage is a tracked state with a named owner.</em></p>
<p>The loop is deliberate. Retiring an API is rarely the end of the work. What you learn from usage data and consumer migration feeds the design of the next version or the next API.</p>
<table><thead><tr><th>Stage</th><th>What happens</th><th>Who typically owns it</th></tr></thead><tbody><tr><td>Design</td><td>The API contract is written and reviewed against organizational standards before code exists.</td><td>API producers, with an architecture or design review group</td></tr><tr><td>Develop and Test</td><td>The implementation is built against the contract and tested in a non-production environment.</td><td>API producers and QA</td></tr><tr><td>Publish</td><td>The API becomes discoverable and consumable through a developer portal or catalog.</td><td>API producers, with platform team approval</td></tr><tr><td>Version</td><td>Changes ship as new versions, and consumers are told what changed and why.</td><td>API producers and API product owners</td></tr><tr><td>Deprecate</td><td>An older version is marked for removal. Existing consumers keep working while they migrate.</td><td>API product owners and the platform team</td></tr><tr><td>Retire</td><td>The version is removed from service and from the active inventory.</td><td>Platform team, with sign-off from the API owner</td></tr></tbody></table>
<p>Ownership varies by organization. What matters is that every stage has a named owner. The platform should also tell you which state each API is in without asking anyone.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="why-api-lifecycle-management-matters">Why API lifecycle management matters<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#why-api-lifecycle-management-matters" class="hash-link" aria-label="Direct link to Why API lifecycle management matters" title="Direct link to Why API lifecycle management matters" translate="no">​</a></h2>
<p>The failure mode is rarely dramatic at first. Teams publish APIs faster than anyone retires them. Each new version adds another endpoint to the estate, and old versions keep running because removing them feels risky. That accumulation is API sprawl. Over time, the API portfolio contains duplicates, versions nobody has touched in years, and endpoints that no team will admit to owning.</p>
<p>The security cost is real. The <a href="https://owasp.org/API-Security/editions/2023/en/0x11-t10/" target="_blank" rel="noopener noreferrer" class="">OWASP API Security Top 10 (2023)</a> lists Improper Inventory Management as API9. It notes that an accurate inventory of hosts and deployed API versions is needed to address problems such as deprecated versions and exposed debug endpoints. A deprecated API that still serves its existing consumers isn't a problem. That is what deprecation is for. The risk is the zombie API: an endpoint that stays reachable after everyone has forgotten it, with no owner, no support, and no review.</p>
<p>The consumer cost is just as concrete. When there is no defined deprecation process, removal happens ad hoc. A team shuts down an old version during a cleanup, and a partner integration breaks the same afternoon. The issue usually isn't the change itself. It's that nobody tracked who depended on the old version, or told them when it was going away.</p>
<p>Formal lifecycle management is worth the overhead once a few operational triggers appear:</p>
<ul>
<li class=""><strong>More than one team publishes APIs.</strong> Without shared states and standards, each team invents its own deprecation habits.</li>
<li class=""><strong>External or partner consumers depend on your APIs.</strong> You can't coordinate a breaking change by messaging the team down the hall.</li>
<li class=""><strong>APIs handle regulated data.</strong> Auditors will ask which versions were live, who could call them, and when they were removed.</li>
<li class=""><strong>APIs run on more than one gateway or cloud.</strong> The inventory problem gets harder when no single system sees everything.</li>
</ul>
<p>None of these triggers depends on a specific API count. A small portfolio with external consumers and regulated data needs more structure than a large internal one.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="design-and-development-getting-the-lifecycle-right-from-day-one">Design and development: Getting the lifecycle right from day one<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#design-and-development-getting-the-lifecycle-right-from-day-one" class="hash-link" aria-label="Direct link to Design and development: Getting the lifecycle right from day one" title="Direct link to Design and development: Getting the lifecycle right from day one" translate="no">​</a></h2>
<p>A well-managed lifecycle starts with a well-defined contract. In an API-first approach, the team writes the API definition before the implementation. For REST APIs that usually means an <a href="https://spec.openapis.org/oas/latest.html" target="_blank" rel="noopener noreferrer" class="">OpenAPI</a> document. For event-driven APIs it usually means AsyncAPI. The contract becomes the single artifact that design reviews, mock servers, tests, and documentation all work from.</p>
<p>API definitions also give governance something to check. Naming conventions, error formats, pagination rules, security schemes, and versioning patterns can all be expressed as rules and validated automatically. Checking a contract at design time is far cheaper than finding the same problem after consumers have integrated. Once a field name is in production, changing it is a breaking change.</p>
<p>This is where design-time governance pays off. A typical rule set might require a security scheme on every operation and a shared schema for every error response. It might also require an owner contact in every API definition. Running those rules on each change to the contract catches drift while the fix is still a one-line edit. Teams that adopt <a class="" href="https://choreolabs.com/api-management/learn/api-first-architecture">API-first architecture</a> tend to put these checks in the same pipeline that builds and tests the API.</p>
<p>Development and testing then work against that contract. The implementation is deployed to a development environment, tested against the definition, and promoted only when it matches. Contract tests are the main defense here. They fail the build when the implementation drifts from the published contract, which is the drift that most often surprises consumers later.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="publishing-and-versioning-apis">Publishing and versioning APIs<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#publishing-and-versioning-apis" class="hash-link" aria-label="Direct link to Publishing and versioning APIs" title="Direct link to Publishing and versioning APIs" translate="no">​</a></h2>
<p>Publishing is more than deploying to a gateway. Operationally, a published API has four things. It has a running endpoint, an entry in a developer portal or API catalog, documentation consumers can act on, and a defined way to get access. An API that is deployed but not cataloged is effectively invisible. Consumers can't find it, and the platform team can't count it.</p>
<p>API catalogs matter for lifecycle tracking because they are where state becomes visible. A catalog maintained by hand as a wiki page tends to drift from reality within a few release cycles.</p>
<p>Versioning records how an API evolves, including compatible changes. Breaking changes are the ones that need a migration strategy. The mechanics vary. Teams use URI versions, header versions, or field-level deprecation, and each approach has trade-offs. From a lifecycle perspective, one rule matters most. Publishing a new major version should trigger a decision about the old one. If that decision isn't recorded as a lifecycle state change, the old version simply stays live.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="deprecating-and-retiring-apis-without-breaking-consumers">Deprecating and retiring APIs without breaking consumers<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#deprecating-and-retiring-apis-without-breaking-consumers" class="hash-link" aria-label="Direct link to Deprecating and retiring APIs without breaking consumers" title="Direct link to Deprecating and retiring APIs without breaking consumers" translate="no">​</a></h2>
<p>Deprecation and retirement are the stages that cause the most real-world pain, and they're the easiest to skip under deadline pressure.</p>
<p>Deprecation is a signal, not a removal. A deprecated API still works. Its state tells consumers that it is scheduled to go away and that they should plan to migrate. During this window, the platform's job is to keep the old version running for existing consumers while steering new consumers to the replacement.</p>
<p>The length of the deprecation window should depend on who the consumers are, not on a fixed number. Internal services owned by the same organization can often migrate quickly. Partner integrations and public consumers usually need much longer, because their release cycles aren't yours to control. Mobile clients that users never update may need the longest window of all. Decide the window per API, record it, and publish it with the deprecation notice.</p>
<p>Consumers need the signal in more than one place:</p>
<ul>
<li class="">In the API catalog, where the lifecycle state shows the API as deprecated next to its replacement.</li>
<li class="">In the API definition, where OpenAPI's <code>deprecated</code> field marks individual operations.</li>
<li class="">In the response itself. The <code>Deprecation</code> HTTP response header, standardized in <a href="https://www.rfc-editor.org/rfc/rfc9745" target="_blank" rel="noopener noreferrer" class="">RFC 9745,</a> signals deprecation to a client at runtime. It can be paired with the <code>Sunset</code> header from RFC 8594 to announce the removal date. RFC 9745 requires that the sunset time is not earlier than the deprecation time.</li>
<li class="">In direct communication to the owners of applications that still call the deprecated version.</li>
</ul>
<p>Diff tooling and version strategies get fuller treatment in the guide to <a class="" href="https://choreolabs.com/api-management/learn/api-versioning-best-practices">API versioning strategies for long-lived APIs.</a> For lifecycle purposes, the key point is that deprecation must be a tracked state. If it lives only in a changelog, nobody can query it.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="retirement-is-a-decision-backed-by-evidence">Retirement is a decision backed by evidence<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#retirement-is-a-decision-backed-by-evidence" class="hash-link" aria-label="Direct link to Retirement is a decision backed by evidence" title="Direct link to Retirement is a decision backed by evidence" translate="no">​</a></h3>
<p>Before retiring a version, check its traffic. Which applications still call it, and how often? Zero observed traffic nominates a version for retirement. It doesn't prove the version is unused. Observe it over a window long enough to catch infrequent clients, such as monthly or quarterly batch jobs. Confirm that your telemetry covers every gateway and route that serves the version. Notify the owners of any application that has called it, and keep a rollback plan ready. If a few consumers remain, the choice is to extend the window, contact them directly, or accept the breakage knowingly. Any of those can be right. Retiring blind is not.</p>
<p>Retirement also has to remove the API from operational scope. Taking an API out of the catalog while the endpoint keeps answering is how zombie APIs are created. A retired API should be removed from routing, from the gateway, and from the active inventory. Its record should stay in the history, so audits can still see when it was live.</p>
<p>Platforms that model these states directly make the window visible to everyone. In Choreo API Management, for example, publishing a newer version moves the older one to a Deprecated state. Existing subscribers keep working until that version is retired.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="measuring-api-lifecycle-maturity">Measuring API lifecycle maturity<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#measuring-api-lifecycle-maturity" class="hash-link" aria-label="Direct link to Measuring API lifecycle maturity" title="Direct link to Measuring API lifecycle maturity" translate="no">​</a></h2>
<p>Lifecycle maturity is measurable. The useful metrics track whether states reflect reality and whether transitions happen on purpose. You can't manage what you can't see, so most of these start with the inventory.</p>
<ul>
<li class=""><strong>Catalog coverage:</strong> What share of the APIs actually running in your environments appears in the catalog with a lifecycle state? This is the foundation for every other metric. If APIs run on gateways the catalog doesn't know about, the number will be lower than you expect.</li>
<li class=""><strong>Ownership coverage:</strong> What share of cataloged APIs has a named, current owner? Owners leave teams. An API whose owner has moved on is a deprecation candidate nobody will raise.</li>
<li class=""><strong>Retirement on schedule:</strong> Of the APIs that reached their announced sunset date, how many were actually retired? Versions that stay live past their sunset date are the clearest sign that deprecation is a label rather than a process.</li>
<li class=""><strong>Time from deprecated to retired:</strong> How long do versions sit in the deprecated state? A steadily growing number usually means retirement decisions are being avoided rather than made.</li>
<li class=""><strong>Design-time compliance:</strong> What share of APIs pass your governance rules before publication? Trends here show whether standards are being adopted or worked around.</li>
</ul>
<p>Track these as trends rather than targets. The right deprecation window for a public API differs from the right one for an internal service, so a single organization-wide number can mislead. What matters is direction. Catalog coverage should rise. The backlog of versions past their sunset date should shrink.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-choreo-api-management-operationalizes-the-full-api-lifecycle">How Choreo API Management operationalizes the full API lifecycle<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#how-choreo-api-management-operationalizes-the-full-api-lifecycle" class="hash-link" aria-label="Direct link to How Choreo API Management operationalizes the full API lifecycle" title="Direct link to How Choreo API Management operationalizes the full API lifecycle" translate="no">​</a></h2>
<p>Choreo API Management treats lifecycle state as platform data rather than a convention teams remember. Each API proxy carries one of five lifecycle states: Created, Pre-Released, Published, Deprecated, and Retired. Changing state is a controlled action. Only users with publishing privileges can move an API between states. The console shows the current state and the allowed next transitions as a diagram. The full state model is documented in the <a href="https://wso2.com/api-platform/docs/cloud/develop-api-proxy/lifecycle-management/" target="_blank" rel="noopener noreferrer" class="">API lifecycle management docs.</a></p>
<p>Here is how those mechanics map onto each stage, followed by how you measure them:</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="design">Design<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#design" class="hash-link" aria-label="Direct link to Design" title="Direct link to Design" translate="no">​</a></h3>
<p>API proxies can be created by importing an existing API contract, including from a GitHub repository. Governance rulesets are written as Spectral YAML rule definitions and validate API artifacts, including OpenAPI and AsyncAPI definitions, API metadata, and documentation. Rulesets are defined at the organization level and attached to policies. Teams can also upload governance standards written in natural language as PDF documents, which an AI compliance analysis feature uses to evaluate APIs. The setup is described in the guide to <a href="https://wso2.com/api-platform/docs/cloud/governance/govern-api-proxy/" target="_blank" rel="noopener noreferrer" class="">governing API proxies.</a></p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="develop-and-test">Develop and test<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#develop-and-test" class="hash-link" aria-label="Direct link to Develop and test" title="Direct link to Develop and test" translate="no">​</a></h3>
<p>An API proxy can be tested in a development environment before it is promoted to production. Governance policies are evaluated when API proxies are updated, so a change that breaks a rule shows up during development rather than after release.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="publish">Publish<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#publish" class="hash-link" aria-label="Direct link to Publish" title="Direct link to Publish" translate="no">​</a></h3>
<p>An API in the Created state isn't ready for consumption. Moving it to Published makes it visible in the Developer Portal and available for subscription. The Pre-Released state publishes a prototype to the Developer Portal for early consumer testing, without a full production commitment.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="version-and-deprecate">Version and deprecate<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#version-and-deprecate" class="hash-link" aria-label="Direct link to Version and deprecate" title="Direct link to Version and deprecate" translate="no">​</a></h3>
<p>The Deprecated state is Choreo API Management's record of the deprecation window. The catalog, the console, and reviewers all read the same status, so nobody has to reconstruct it from a changelog.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="retire">Retire<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#retire" class="hash-link" aria-label="Direct link to Retire" title="Direct link to Retire" translate="no">​</a></h3>
<p>Moving a version to Retired unpublishes it and removes it from the Developer Portal. That state change alone doesn't prove every serving route is gone, including routes on federated gateways. Disable those routes explicitly where required, and verify that traffic is rejected. Because the transition is a recorded state change, retirement becomes an auditable event instead of a quiet deletion.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="measure">Measure<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#measure" class="hash-link" aria-label="Direct link to Measure" title="Direct link to Measure" translate="no">​</a></h3>
<p>Compliance results appear in a Compliance Dashboard at the organization, project, and component levels, which covers the design-time compliance metric above. The same model sits within the broader API governance program, which lists structured deprecation and retirement among its goals.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="inventory-across-gateways">Inventory across gateways<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#inventory-across-gateways" class="hash-link" aria-label="Direct link to Inventory across gateways" title="Direct link to Inventory across gateways" translate="no">​</a></h3>
<p>Catalog coverage is hardest when APIs run on more than one gateway. Choreo API Management supports <a href="https://wso2.com/api-platform/docs/cloud/federation/overview/" target="_blank" rel="noopener noreferrer" class="">gateway federation.</a> You register an external gateway, discover the APIs it hosts, and bring them under the same control plane and Developer Portal. Supported third-party gateways include AWS, Azure, Kong, and Envoy. For AWS API Gateway, discovery covers REST APIs, and the discovered APIs keep running on AWS. That makes it practical to improve catalog coverage first and consolidate infrastructure later, if at all.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="ai-traffic-under-the-same-control-plane">AI traffic under the same control plane<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#ai-traffic-under-the-same-control-plane" class="hash-link" aria-label="Direct link to AI traffic under the same control plane" title="Direct link to AI traffic under the same control plane" translate="no">​</a></h3>
<p>Lifecycle discipline increasingly applies to more than REST APIs. The same control plane manages the <a href="https://wso2.com/api-platform/ai-gateway/" target="_blank" rel="noopener noreferrer" class="">AI Gateway,</a> which governs LLM traffic through LLM proxies and agent access to tools through MCP proxies. These are separate traffic flows. Consumer applications call your APIs inbound through the API Gateway. Applications send outbound requests to LLM providers through the AI Gateway. AI agents reach MCP tools through the AI Gateway's MCP proxies. None of these is a mandatory hop in front of another. They are independent data planes managed from one control plane. Each artifact type, whether an API, an LLM proxy, or an MCP proxy, has its own documented lifecycle and policies. What they share is one governance model and one analytics view.</p>
<p>Choreo API Management is built on an open source foundation. One trade-off is worth naming. Lifecycle states are only as accurate as the inventory behind them. The catalog can't show APIs that were never brought into it. Teams can create or import APIs directly, and federation discovers APIs already running on other gateways. Either way, inventory coverage is part of the lifecycle work, not a separate project.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>API lifecycle management works when every stage is a tracked platform state, not an ad hoc process someone has to remember. Design and publication tend to get that treatment by default, because they're tied to shipping. Deprecation and retirement usually don't, and that is where sprawl, zombie APIs, and broken consumers come from.</p>
<p>Start with the inventory. Find out which APIs are running, who owns them, and which state each one is really in. Then make deprecation a recorded state with a published window, and make retirement a decision backed by traffic data. The metrics in this guide will show whether the process is holding.</p>
<p>To run that model end to end, from contract-first design through governed publication to recorded deprecation and retirement, explore <a href="https://wso2.com/api-platform/" target="_blank" rel="noopener noreferrer" class="">Choreo API Management.</a></p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="whats-the-difference-between-api-lifecycle-management-and-api-management">What's the difference between API lifecycle management and API management?<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#whats-the-difference-between-api-lifecycle-management-and-api-management" class="hash-link" aria-label="Direct link to What's the difference between API lifecycle management and API management?" title="Direct link to What's the difference between API lifecycle management and API management?" translate="no">​</a></h3>
<p>API management covers the runtime and consumption side of APIs, including the gateway, security, rate limiting, and developer portal. API lifecycle management governs which state each API is in, from design to retirement, and controls the transitions between states.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-are-the-stages-of-the-api-lifecycle">What are the stages of the API lifecycle?<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#what-are-the-stages-of-the-api-lifecycle" class="hash-link" aria-label="Direct link to What are the stages of the API lifecycle?" title="Direct link to What are the stages of the API lifecycle?" translate="no">​</a></h3>
<p>A common six-stage model is Design, Develop and Test, Publish, Version, Deprecate, and Retire. Other frameworks use more or fewer stages, but they describe the same journey from first contract to removal.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="who-manages-the-api-lifecycle">Who manages the API lifecycle?<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#who-manages-the-api-lifecycle" class="hash-link" aria-label="Direct link to Who manages the API lifecycle?" title="Direct link to Who manages the API lifecycle?" translate="no">​</a></h3>
<p>API producers usually own design, development, and versioning. A central platform team typically owns the shared standards, the catalog, and the decisions to deprecate and retire, with sign-off from each API's owner.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-happens-to-consumers-when-an-api-is-deprecated">What happens to consumers when an API is deprecated?<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#what-happens-to-consumers-when-an-api-is-deprecated" class="hash-link" aria-label="Direct link to What happens to consumers when an API is deprecated?" title="Direct link to What happens to consumers when an API is deprecated?" translate="no">​</a></h3>
<p>A deprecated API keeps working for existing consumers. The deprecation tells them to migrate to a replacement before a stated removal date. New consumers should be directed to the replacement instead.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="when-should-an-organization-formalize-api-lifecycle-management">When should an organization formalize API lifecycle management?<a href="https://choreolabs.com/api-management/learn/api-lifecycle-management#when-should-an-organization-formalize-api-lifecycle-management" class="hash-link" aria-label="Direct link to When should an organization formalize API lifecycle management?" title="Direct link to When should an organization formalize API lifecycle management?" translate="no">​</a></h3>
<p>Formalize it when operational triggers appear, such as multiple teams publishing APIs, external or partner consumers, regulated data, or APIs spread across several gateways. The trigger is complexity and risk, not a specific API count.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="API management" term="API management"/>
        <category label="API governance" term="API governance"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[API security governance: A practical framework]]></title>
        <id>https://choreolabs.com/api-management/learn/api-security-governance</id>
        <link href="https://choreolabs.com/api-management/learn/api-security-governance"/>
        <updated>2026-10-09T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[Learn how API security governance works: threat modeling, policy enforcement, and audit trails that keep APIs secure and compliant at scale.]]></summary>
        <content type="html"><![CDATA[<p>A partner API ships on a Friday. Its OpenAPI file declares OAuth at the top level, so the review checklist passes. One operation, though, carries <code>security: []</code>, a line someone added while testing. In <a href="https://spec.openapis.org/oas/v3.1.0" target="_blank" rel="noopener noreferrer" class="">OpenAPI</a>, an empty security array on an operation removes the top-level requirement for that operation. The contract now declares unauthenticated access for that operation. If the gateway configuration is generated from the contract, the change can also remove runtime protection. If it is not, the contract and the runtime now disagree. Either way, the written policy said this should never happen.</p>
<p>Nobody broke a rule on purpose, because the rule existed only in a document. API security governance closes that gap by turning written security standards into checks that run on every API, every time something changes. This guide covers how to model threats, encode them as policy, enforce that policy, and prove it is working.</p>
<aside class="cta_6F_m" aria-label="Turn security policy into enforced controls" data-cta-position="top"><div class="inner_Hh5j"><div class="text_AyLJ"><span class="kicker_KsLi"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--light_NVdE" width="127" height="18"><img src="https://choreolabs.com/api-management/learn/img/choreo-labs/choreo-labs-logo-white.svg" alt="Choreo Labs" class="logo_vfGX themedComponent_mlkZ themedComponent--dark_xIcU" width="127" height="18"></span><span class="title_oCnW">Turn security policy into enforced controls</span><span class="desc_GzK8">Use Choreo API Management to apply design-time governance policies with audit-ready compliance reporting for every API.</span></div><a href="https://console.bijira.dev/login?utm_source=blog&amp;utm_campaign=content_hub_cta&amp;utm_medium=intro-cta&amp;utm_content=api-security-governance" target="_blank" rel="noopener noreferrer" class="cl-btn cl-btn--primary button___d6">Sign up for free</a></div></aside>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-api-security-governance">What is API security governance?<a href="https://choreolabs.com/api-management/learn/api-security-governance#what-is-api-security-governance" class="hash-link" aria-label="Direct link to What is API security governance?" title="Direct link to What is API security governance?" translate="no">​</a></h2>
<p>API security governance is the set of policies, standards, and enforcement mechanisms that keep every API secure, compliant, and consistent across its lifecycle. It is the security-focused layer of API governance. It defines what "secure" means for your organization and checks each API against that definition automatically, from design through retirement.</p>
<p>It is not a separate discipline from <a class="" href="https://choreolabs.com/api-management/learn/api-governance">the broader API governance framework.</a> Design consistency, versioning, and documentation rules live in the same program. Security governance narrows the focus to authentication and authorization requirements, transport security, data exposure, and traffic limits. It also sets who owns each control when a check fails.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="why-api-security-governance-matters">Why API security governance matters<a href="https://choreolabs.com/api-management/learn/api-security-governance#why-api-security-governance-matters" class="hash-link" aria-label="Direct link to Why API security governance matters" title="Direct link to Why API security governance matters" translate="no">​</a></h2>
<p>Most API security risks are not exotic. They come from inconsistency across a large estate, where one team enforces token scopes and another forgets. A new version ships, and the old one keeps taking traffic because nobody owned its retirement.</p>
<p>The OWASP API Security Top 10 makes this visible. <a href="https://api-security.owasp.org/editions/2023/en/0xa1-broken-object-level-authorization/" target="_blank" rel="noopener noreferrer" class="">Broken object level authorization</a> (BOLA) sits at API1. OWASP's guidance is that object-level checks belong in every function that accesses a data source using an ID supplied by the client. A governance program cannot write those checks for you. It can make sure every API is reviewed for them and that the gateway-level controls around them are in place.</p>
<p>Shadow and zombie APIs are the other recurring problem. A shadow API is one that runs without appearing in any inventory. A zombie API is a deprecated version that was never switched off. Both fall under API9, Improper Inventory Management, where OWASP points to deprecated API versions and exposed debug endpoints as typical issues. Neither can be governed until it is found.</p>
<p>Regulatory exposure depends on more than the data type. Whether GDPR, CCPA, or HIPAA applies also depends on who processes the data, whose data it is, and where they are. Your compliance and legal teams make that call. Governance is how you show an auditor that access controls exist and are applied consistently.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="ai-traffic-adds-two-new-flows">AI traffic adds two new flows<a href="https://choreolabs.com/api-management/learn/api-security-governance#ai-traffic-adds-two-new-flows" class="hash-link" aria-label="Direct link to AI traffic adds two new flows" title="Direct link to AI traffic adds two new flows" translate="no">​</a></h3>
<p>Applications now make outbound calls to model providers. Agents make inbound calls to MCP servers. These are separate paths with separate risks. On the outbound path, prompts can carry sensitive data to an external provider. On the inbound path, an agent may reach tools it should not see. Choreo AI Gateway handles both as capabilities of one gateway. It applies guardrails such as PII masking to LLM traffic. It applies MCP authentication, per-tool authorization, access control lists, and rate limits to MCP traffic. The governance model is the same one you apply to APIs: define the requirement once and enforce it at a known point.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-three-pillars-of-api-security-governance">The three pillars of API security governance<a href="https://choreolabs.com/api-management/learn/api-security-governance#the-three-pillars-of-api-security-governance" class="hash-link" aria-label="Direct link to The three pillars of API security governance" title="Direct link to The three pillars of API security governance" translate="no">​</a></h2>
<p>Most programs settle on three pillars, and each one catches failures the others miss. The split mirrors NIST SP 800-228 (June 2025, updated March 2026), which organizes API protections into <a href="https://www.nist.gov/publications/guidelines-api-protection-cloud-native-systems-march-2026-update" target="_blank" rel="noopener noreferrer" class="">pre-runtime and runtime controls</a> and recommends an incremental, risk-based approach to adopting them. A third pillar keeps both honest over time.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="standardized-secure-by-default-design">Standardized, secure-by-default design<a href="https://choreolabs.com/api-management/learn/api-security-governance#standardized-secure-by-default-design" class="hash-link" aria-label="Direct link to Standardized, secure-by-default design" title="Direct link to Standardized, secure-by-default design" translate="no">​</a></h3>
<p>Security requirements belong in the API contract before code exists. That means a required security scheme, HTTPS-only server URLs, consistent error formats that do not leak internals, and response schemas that return only the fields a client needs. When the contract is the source of truth, a linter can check it. A reviewer no longer has to remember every rule.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="policy-enforcement-at-runtime">Policy enforcement at runtime<a href="https://choreolabs.com/api-management/learn/api-security-governance#policy-enforcement-at-runtime" class="hash-link" aria-label="Direct link to Policy enforcement at runtime" title="Direct link to Policy enforcement at runtime" translate="no">​</a></h3>
<p>A compliant contract does not guarantee compliant traffic. Runtime controls at the API gateway handle token validation, rate limiting, and mutual TLS to backends. These controls apply whether or not the design-time checks ran, so they form the floor every request has to clear. For OAuth-protected APIs, governance should name a reference for how tokens are issued and validated. <a href="https://datatracker.ietf.org/doc/rfc9700/" target="_blank" rel="noopener noreferrer" class="">RFC 9700,</a> the OAuth 2.0 Security Best Current Practice, is a sound baseline because it updates the original OAuth threat model with lessons from production use.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="continuous-monitoring-testing--audit">Continuous monitoring, testing &amp; audit<a href="https://choreolabs.com/api-management/learn/api-security-governance#continuous-monitoring-testing--audit" class="hash-link" aria-label="Direct link to Continuous monitoring, testing &amp; audit" title="Direct link to Continuous monitoring, testing &amp; audit" translate="no">​</a></h3>
<p>Policies drift as new threats appear and new APIs get created outside the usual path. Continuous monitoring re-checks the estate as it changes, and security testing confirms that controls behave as written. An audit trail records who changed what and which violations remain open.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="threat-modeling-for-apis-where-governance-starts">Threat modeling for APIs: Where governance starts<a href="https://choreolabs.com/api-management/learn/api-security-governance#threat-modeling-for-apis-where-governance-starts" class="hash-link" aria-label="Direct link to Threat modeling for APIs: Where governance starts" title="Direct link to Threat modeling for APIs: Where governance starts" translate="no">​</a></h2>
<p>Governance rules work best when each one traces back to a specific threat. Threat modeling supplies that link. The <a href="https://cheatsheetseries.owasp.org/cheatsheets/Threat_Modeling_Cheat_Sheet.html" target="_blank" rel="noopener noreferrer" class="">OWASP Threat Modeling Cheat Sheet</a> frames the process around four questions from the Threat Modeling Manifesto:</p>
<ul>
<li class="">What are we working on?</li>
<li class="">What can go wrong?</li>
<li class="">What are we going to do about it?</li>
<li class="">Did we do a good enough job?</li>
</ul>
<p>Applied to an API estate, the process has three working steps.</p>
<ol>
<li class=""><strong>Map the attack surface:</strong> List authentication flows, token types, and where each API trusts a caller's identity. Mark data exposure points, such as endpoints that return full objects or accept object IDs in the path. Add third-party integrations, since data returned from them is input too.</li>
<li class=""><strong>Map likely attack patterns:</strong> For each surface, name the realistic attack. Object IDs in paths invite BOLA. Login and token endpoints attract credential stuffing. Endpoints that return entire records invite excessive data exposure, which OWASP now groups under broken object property level authorization (API3).</li>
<li class=""><strong>Translate findings into policy requirements:</strong> Each finding becomes a requirement with an owner and an enforcement point.</li>
</ol>
<p>The third step is where most programs stall. The table below shows how findings become enforceable requirements. The enforcement points are independent of each other. A request does not pass through them in sequence, and some requirements live in only one place.</p>
<table><thead><tr><th>Threat finding</th><th>Policy requirement</th><th>Primary enforcement point</th></tr></thead><tbody><tr><td>Operation can opt out of authentication</td><td>Every operation must inherit or declare a security requirement</td><td>Design-time rule on the API definition</td></tr><tr><td>API served over plain HTTP</td><td>Server URLs must use HTTPS</td><td>Design-time rule, plus TLS configuration at the gateway</td></tr><tr><td>Object IDs accepted in paths</td><td>Ownership check on every object read and write</td><td>Application code, verified through security testing</td></tr><tr><td>Token endpoint open to automated guessing</td><td>Per-client rate limits on authentication endpoints</td><td>Runtime gateway policy</td></tr><tr><td>Deprecated versions still reachable</td><td>Every version has an owner and a retirement date</td><td>Inventory review and monitoring</td></tr></tbody></table>
<p>For a full layer-by-layer mapping of the OWASP list, see this guide to <a class="" href="https://choreolabs.com/api-management/learn/api-security-architecture">API security architecture.</a> Threat modeling for governance is narrower, because it asks which of those risks your rules must catch automatically. Revisit the model when the estate changes in ways that shift the attack surface. Typical moments include adding a new identity provider, exposing an internal API to partners, or connecting agents to existing services.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-policy-enforcement-actually-works">How policy enforcement actually works<a href="https://choreolabs.com/api-management/learn/api-security-governance#how-policy-enforcement-actually-works" class="hash-link" aria-label="Direct link to How policy enforcement actually works" title="Direct link to How policy enforcement actually works" translate="no">​</a></h2>
<p>A written security policy becomes enforceable when it is expressed as a rule a machine can evaluate. For API definitions, that usually means a linting ruleset. Spectral is a common choice because it reads OpenAPI and AsyncAPI documents and supports custom rules.</p>
<p>Here is a small security ruleset that encodes the first two rows of the table above:</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#C9CFDA;--prism-background-color:#070D22"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#C9CFDA;background-color:#070D22"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#C9CFDA"><span class="token key atrule" style="color:#3D8BFF">rules</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">root-security-required</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> The API must declare a default security requirement.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">severity</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> error</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">given</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">then</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">field</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> security</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">function</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> truthy</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">root-security-not-empty</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> The default security requirement must not be an empty list.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">severity</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> error</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">given</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$.security"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">then</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">function</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> length</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">functionOptions</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token key atrule" style="color:#3D8BFF">min</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">1</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">operation-security-no-opt-out</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Operations must not remove security unless approved as public.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">severity</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> error</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">given</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$.paths[*][?(@property.match(/^(get|put|post|delete|options|head|patch|trace)$/) &amp;&amp; @['x-public'] !== true)].security"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">then</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">function</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> length</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">functionOptions</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token key atrule" style="color:#3D8BFF">min</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">1</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">security-no-anonymous-alternative</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Security requirements must not include an anonymous </span><span class="token punctuation" style="color:#8A93A6">{</span><span class="token punctuation" style="color:#8A93A6">}</span><span class="token plain"> option.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">severity</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> error</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">given</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$.security[*]"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$.paths[*][?(@property.match(/^(get|put|post|delete|options|head|patch|trace)$/) &amp;&amp; @['x-public'] !== true)].security[*]"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">then</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">function</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> length</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">functionOptions</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token key atrule" style="color:#3D8BFF">min</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token number" style="color:#FF4600">1</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">servers-required</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> The API must declare at least one server.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">severity</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> error</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">given</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">then</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">field</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> servers</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">function</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> truthy</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">  </span><span class="token key atrule" style="color:#3D8BFF">servers-https-only</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">description</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> Server URLs must be absolute HTTPS URLs</span><span class="token punctuation" style="color:#8A93A6">,</span><span class="token plain"> at every level.</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">severity</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> error</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">given</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$.servers[*].url"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$.paths[*].servers[*].url"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token punctuation" style="color:#8A93A6">-</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"$.paths[*][*].servers[*].url"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">    </span><span class="token key atrule" style="color:#3D8BFF">then</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">function</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> pattern</span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">      </span><span class="token key atrule" style="color:#3D8BFF">functionOptions</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"></span><br></span><span class="token-line" style="color:#C9CFDA"><span class="token plain">        </span><span class="token key atrule" style="color:#3D8BFF">match</span><span class="token punctuation" style="color:#8A93A6">:</span><span class="token plain"> </span><span class="token string" style="color:#4ADE80">"^https://"</span><br></span></code></pre></div></div>
<p>The severity on each rule is a governance decision. Here every rule is an error. That covers a missing or empty security requirement, an anonymous <code>{}</code> alternative, an unapproved operation-level opt-out, and a non-HTTPS server URL at the root, path, or operation level. A genuinely public endpoint, such as a health check, is approved by adding <code>x-public: true</code> to the operation after a security review. The exception then stays visible in the contract instead of being silently allowed. These rules check the contract, not the deployed gateway. The example policy requires absolute HTTPS server URLs, so it also rejects relative URLs. Relax that rule if your gateway assigns the host, because a missing or relative URL does not by itself mean traffic runs over plain HTTP.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="blocking-vs-non-blocking-enforcement">Blocking vs. non-blocking enforcement<a href="https://choreolabs.com/api-management/learn/api-security-governance#blocking-vs-non-blocking-enforcement" class="hash-link" aria-label="Direct link to Blocking vs. non-blocking enforcement" title="Direct link to Blocking vs. non-blocking enforcement" translate="no">​</a></h3>
<p>Severity decides what happens on a violation. A blocking check stops a change from moving forward. A non-blocking check records the violation and lets work continue. With the Spectral CLI, only findings at or above the <code>--fail-severity</code> level produce a failing exit code, and the default level is <code>error</code>. So the same ruleset gives you both behaviors in a CI pipeline.</p>
<table><thead><tr><th>Mode</th><th>Typical trigger</th><th>Effect</th><th>Good fit</th></tr></thead><tbody><tr><td>Blocking</td><td>Error-severity rule fails in CI</td><td>Build or deployment stops until fixed</td><td>Missing authentication, plain HTTP, leaked secrets</td></tr><tr><td>Non-blocking</td><td>Warn or info rule fails</td><td>Change proceeds, violation is reported for follow-up</td><td>Documented exceptions, style issues, early adoption of a new rule</td></tr></tbody></table>
<p>Roll out new style and consistency rules as non-blocking, then promote them to blocking once the existing estate is clean. That keeps a noisy new rule from stalling every team on the day it ships. A newly discovered, exploitable gap such as a missing authentication requirement is different. Block it from the start and route any legitimate exceptions through the documented allowlist. A staged rollout is a migration strategy, not a reason to leave a known critical exposure open.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="design-time-and-runtime-are-separate-enforcement-points">Design time and runtime are separate enforcement points<a href="https://choreolabs.com/api-management/learn/api-security-governance#design-time-and-runtime-are-separate-enforcement-points" class="hash-link" aria-label="Direct link to Design time and runtime are separate enforcement points" title="Direct link to Design time and runtime are separate enforcement points" translate="no">​</a></h3>
<p>Design-time checks evaluate the API definition, while runtime policies evaluate live traffic at the gateway. They run independently and catch different failures. A linter cannot stop a stolen token, and a gateway cannot tell that a response schema exposes a field it should not. Neither check depends on the other having run, so treat them as two controls rather than two steps in one pipeline.</p>
<p><img decoding="async" loading="lazy" alt="Two independent governance flows. Flow 1, CI validation in your pipeline: a Spectral security ruleset with rules tagged Error, Warn, or Info runs spectral lint on the changed API definition, then a configured decision based on the fail-severity threshold and approved exceptions either fails the build at or above the threshold or passes it with findings reported. Flow 2, Choreo API Management governance: the same ruleset is attached to a governance policy, the policy applies to all API proxies in the organization when an API proxy is updated, governance evaluation tags violations Error, Warn, or Info, and Choreo API Management notifies with findings and reports in the Compliance Dashboard" src="https://choreolabs.com/api-management/learn/assets/images/ci-and-platform-governance-flows-d9f4ac9829b3820f8e54a698b8f1e900.webp" width="3200" height="1162" class="img_ev3q"></p>
<p><em>CI validation and platform governance run independently. Platform violations notify rather than block, and the hard stop comes from the CI decision you configure.</em></p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="a-worked-example-on-choreo-api-management">A worked example on Choreo API Management<a href="https://choreolabs.com/api-management/learn/api-security-governance#a-worked-example-on-choreo-api-management" class="hash-link" aria-label="Direct link to A worked example on Choreo API Management" title="Direct link to A worked example on Choreo API Management" translate="no">​</a></h3>
<p><a href="https://wso2.com/api-platform/docs/cloud/governance/overview/" target="_blank" rel="noopener noreferrer" class="">Choreo API Management's governance model</a> follows the same structure. Rulesets are written in Spectral YAML, and they can validate the API definition, the API metadata, and the API documentation. A policy groups rulesets and applies them to all existing and new API proxies in an organization. The catalog also offers default rulesets, including one based on the OWASP Top 10, so teams can start from a known baseline.</p>
<p>Policies are evaluated when an API proxy is updated. Each violation carries a severity of Error, Warn, or Info. Today Choreo API Management's action on a violation is Notify, which surfaces the finding in the compliance dashboards. Teams that want a hard stop on error-severity findings run the same Spectral ruleset in their CI pipeline as well. One ruleset then drives both the blocking gate and Choreo API Management's compliance view.</p>
<p>Runtime enforcement sits with the gateways. Choreo API Management manages Choreo, Kong, AWS, Azure, and Envoy gateways from one control plane. Through gateway federation, APIs discovered on third-party gateways can be brought under the same governance policies. That matters for most estates, which rarely run on a single gateway technology.</p>
<p>This is one way to operationalize the model. Dedicated API security platforms and other management products structure it differently. The pattern to look for is the same: rules as code, severity-based outcomes, and one place to see results.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="audit-compliance--governance-dashboards">Audit, compliance &amp; governance dashboards<a href="https://choreolabs.com/api-management/learn/api-security-governance#audit-compliance--governance-dashboards" class="hash-link" aria-label="Direct link to Audit, compliance &amp; governance dashboards" title="Direct link to Audit, compliance &amp; governance dashboards" translate="no">​</a></h2>
<p>Enforcement without a record is hard to defend. An auditor usually asks two questions: which controls apply to this API, and can you show they were applied at a given time? A governance dashboard should answer both without a spreadsheet exercise.</p>
<p>Useful dashboards surface a small set of views:</p>
<ul>
<li class=""><strong>Violation history per API,</strong> with severity, the rule that fired, and when it was resolved.</li>
<li class=""><strong>Policy adherence trends</strong> across teams or projects, so drift shows up before an audit does.</li>
<li class=""><strong>Re-evaluation after rule changes.</strong> When a ruleset changes, existing APIs should be checked again, not only new ones.</li>
<li class=""><strong>Ownership.</strong> Every open violation needs a named owner, or it stays open.</li>
</ul>
<p>In Choreo API Management, the <a href="https://wso2.com/api-platform/docs/cloud/monitoring-and-insights/compliance/" target="_blank" rel="noopener noreferrer" class="">Compliance Dashboard</a> works at organization, project, and component levels. The organization view lists each project with its compliance status and violation counts. The component view breaks results down by policy and ruleset adherence, with a detail view of individual rule violations. Teams can also upload governance standards as PDF documents and run an AI compliance analysis at the project level. The docs note that this analysis can produce inaccurate or incomplete results, so treat it as review input rather than evidence.</p>
<p>Violation history shows what was wrong. It does not show who changed the API to fix it, or who changed it to break it. That second record comes from audit logs. In Choreo API Management, organization-level audit logs capture API proxy creation, updates, deletion, and version creation, along with the timestamp and outcome of each action. Pairing those logs with compliance results lets you reconstruct a timeline: the rule that fired, the change that resolved it, and the person who made that change. By default, only organization administrators can view these logs, and access for others is granted through a role. That is a sensible starting point for evidence that auditors will rely on.</p>
<p>Mapping these controls to specific regimes such as GDPR, HIPAA, or SOC 2 is its own exercise. The governance program supplies the evidence. Your compliance team decides which controls satisfy which clauses.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="kpis-to-measure-api-security-governance-health">KPIs to measure API security governance health<a href="https://choreolabs.com/api-management/learn/api-security-governance#kpis-to-measure-api-security-governance-health" class="hash-link" aria-label="Direct link to KPIs to measure API security governance health" title="Direct link to KPIs to measure API security governance health" translate="no">​</a></h2>
<p>Pick a few measures that show whether governance is changing outcomes. Track the trend over time rather than chasing a fixed target.</p>
<ul>
<li class=""><strong>Coverage of automated evaluation.</strong> The share of APIs that are evaluated against security rulesets on every change. Gaps here usually point to gateways or teams outside the main path.</li>
<li class=""><strong>Shadow API trend.</strong> APIs found by traffic or gateway discovery that were not already in the inventory. A falling count means inventory is keeping up with delivery.</li>
<li class=""><strong>Open error-severity violations by age.</strong> Old critical findings matter more than total counts. They show where ownership has broken down.</li>
<li class=""><strong>Mean time to remediate.</strong> Time from a violation being raised to being resolved, split by severity.</li>
<li class=""><strong>Inventory completeness for critical APIs.</strong> The share of APIs handling sensitive data that have an owner, a classification, and a retirement plan for old versions.</li>
</ul>
<p>Avoid setting universal thresholds for these. A regulated payments estate and an internal tooling estate will have very different baselines.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="how-to-build-an-api-security-governance-program">How to build an API security governance program<a href="https://choreolabs.com/api-management/learn/api-security-governance#how-to-build-an-api-security-governance-program" class="hash-link" aria-label="Direct link to How to build an API security governance program" title="Direct link to How to build an API security governance program" translate="no">​</a></h2>
<p>A program does not need to start large. It needs a sequence that turns each stage into the input for the next. If you are building a governance framework from scratch, run security governance as part of it rather than beside it.</p>
<ol>
<li class=""><strong>Inventory:</strong> Find every API, including those behind third-party gateways and older versions still taking traffic. Record an owner and a data classification for each.</li>
<li class=""><strong>Define policies:</strong> Run a threat modeling session on your highest-risk APIs. Turn the findings into a short list of rules. Agree on severities with security and platform teams together.</li>
<li class=""><strong>Automate enforcement:</strong> Put the rules in a linting ruleset that runs in CI and in your API platform. Configure runtime gateway policies for authentication and rate limiting. Start new rules as non-blocking.</li>
<li class=""><strong>Monitor and audit:</strong> Review the compliance dashboard on a regular cadence. Assign every open violation an owner. Promote stable rules to blocking.</li>
<li class=""><strong>Iterate:</strong> Revisit the threat model when the estate changes. Add rules when new risks appear and retire rules that no longer apply.</li>
</ol>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="who-owns-what-between-dev-and-security-teams">Who owns what between dev and security teams<a href="https://choreolabs.com/api-management/learn/api-security-governance#who-owns-what-between-dev-and-security-teams" class="hash-link" aria-label="Direct link to Who owns what between dev and security teams" title="Direct link to Who owns what between dev and security teams" translate="no">​</a></h3>
<p>Governance fails quietly when ownership is vague. A split that works in many organizations gives the security team ownership of the rules and their severities. The platform team owns the tooling that runs those rules in CI, in the API platform, and at the gateways. API teams own their violations and the fixes. Exceptions go through a short review with security, and each one is recorded against the API with an expiry date. This keeps security from becoming a ticket queue. It also stops API teams from quietly disabling a rule that slows them down, because any opt-out is visible in the same dashboard as every other finding.</p>
<p>The right time to move from manual review to automated enforcement depends on operational triggers, not API counts. Common triggers include running more than one gateway technology and moving regulated data through APIs. Others are a need to attribute access or AI spend to specific teams, and the arrival of agents or LLM-backed features. Any one of these makes manual review hard to sustain.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://choreolabs.com/api-management/learn/api-security-governance#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>API security governance only works when policies are enforced automatically and re-evaluated as the estate changes. Written standards alone do not stop shadow APIs, quiet opt-outs like <code>security: []</code>, or drift between teams. Start with a threat model and encode its findings as rules with clear severities. Enforce them at design time and runtime as separate controls. Then keep an audit trail that shows the rules are working.</p>
<p>This is not a one-time project. Rules, threats, and APIs all change, so the program has to keep checking. For the wider governance picture, the API governance hub linked earlier covers the full framework. To put rulesets, policies, and compliance dashboards to work across your gateways, explore <a href="https://wso2.com/api-platform/" target="_blank" rel="noopener noreferrer" class="">Choreo API Management.</a></p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="frequently-asked-questions">Frequently asked questions<a href="https://choreolabs.com/api-management/learn/api-security-governance#frequently-asked-questions" class="hash-link" aria-label="Direct link to Frequently asked questions" title="Direct link to Frequently asked questions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-is-api-governance">What is API governance?<a href="https://choreolabs.com/api-management/learn/api-security-governance#what-is-api-governance" class="hash-link" aria-label="Direct link to What is API governance?" title="Direct link to What is API governance?" translate="no">​</a></h3>
<p>API governance is the set of standards, processes, and automated checks that keep APIs consistent across an organization. It covers design, documentation, versioning, security, and retirement. API security governance is the part focused on security controls.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="what-are-the-three-pillars-of-api-security">What are the three pillars of API security?<a href="https://choreolabs.com/api-management/learn/api-security-governance#what-are-the-three-pillars-of-api-security" class="hash-link" aria-label="Direct link to What are the three pillars of API security?" title="Direct link to What are the three pillars of API security?" translate="no">​</a></h3>
<p>Secure-by-default design, policy enforcement at runtime, and continuous monitoring with audit. Design rules catch problems in the contract, while runtime policies protect live traffic. Monitoring confirms both keep working as the estate changes.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="why-is-api-security-governance-important">Why is API security governance important?<a href="https://choreolabs.com/api-management/learn/api-security-governance#why-is-api-security-governance-important" class="hash-link" aria-label="Direct link to Why is API security governance important?" title="Direct link to Why is API security governance important?" translate="no">​</a></h3>
<p>It makes security consistent across teams and gateways. Without it, controls depend on each team remembering every rule, and gaps such as unauthenticated endpoints or forgotten API versions reach production.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="is-api-security-governance-the-same-as-api-security-testing">Is API security governance the same as API security testing?<a href="https://choreolabs.com/api-management/learn/api-security-governance#is-api-security-governance-the-same-as-api-security-testing" class="hash-link" aria-label="Direct link to Is API security governance the same as API security testing?" title="Direct link to Is API security governance the same as API security testing?" translate="no">​</a></h3>
<p>No. Testing checks whether a specific API behaves securely at a point in time. Governance defines the required controls, checks every API against them automatically, and records the results. Testing is one input to a governance program.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="does-api-security-governance-apply-to-ai-and-mcp-traffic">Does API security governance apply to AI and MCP traffic?<a href="https://choreolabs.com/api-management/learn/api-security-governance#does-api-security-governance-apply-to-ai-and-mcp-traffic" class="hash-link" aria-label="Direct link to Does API security governance apply to AI and MCP traffic?" title="Direct link to Does API security governance apply to AI and MCP traffic?" translate="no">​</a></h3>
<p>Yes. Outbound calls to LLM providers and inbound agent calls to MCP servers need the same discipline: defined requirements, enforcement at a gateway, and an audit record. The controls differ, for example PII masking on prompts and per-tool authorization for MCP.</p>]]></content>
        <author>
            <name>Enakshi Ratwatte</name>
            <uri>https://wso2.com</uri>
        </author>
        <category label="API management" term="API management"/>
        <category label="API architecture" term="API architecture"/>
        <category label="API governance" term="API governance"/>
    </entry>
</feed>