WSO2 is now Choreo LabsLearn more

Trusted AI Governance
Build

Create agents and AI apps with the tools and languages your teams already use.

Connect

Link agents securely to your APIs, data, models and business systems.

Run

Deploy, observe and operate agents and AI apps reliably at any scale.

Govern

Control identity, access, cost and policy for every agent in production.

Agent Management

Developer Platform

  • API Management Cloud

    Design, secure and publish APIs with governance and analytics built in.

    Sign Up Now
  • Integration Cloud

    Connect apps, data and AI agents with low-code and pro-code integration.

    Sign Up Now
  • Identity & Access Management Cloud

    Secure sign-in and access for customers, partners, workforce and agents.

    Sign Up Now
  • Agent Management Cloud

    Build, run and govern AI agents with full visibility and control.

    Sign Up Now
  • Developer Platform Cloud

    Ship cloud-native apps faster on an internal developer platform.

    Sign Up Now

Self-Hosted

Full control over your stack. Deploy directly to your own servers, bare metal, cloud, or Kubernetes environment. Your data never leaves your perimeter.

Product Downloads
Choreo Labs
AI Governance
Platform
Solutions
Resources
Article

Static API throttling or elastic API scaling

Stephen posted a good API architecture question in a LinkedIn group,

So your API management initiative is a success. Now how do you cope with the unpredictable volumes? Throttle requests over static infrastructure or make your next initiative “elastic” infrastructure?

Static API throttling and elastic API scaling are complementary design techniques.

Static API Throttling

Static throttle limits ensure that you can limit aggregate maximum demand and rightsize back-end capacity for worst-case load. API gateways enforce a choke point that can shape traffic volume and ensure back-end target service infrastructure is not overwhelmed. Throttle limits may be defined per user and per API, and API traffic routing rules can allocate demand across multiple static service infrastructure environments.

DevOp teams rely on usage monitoring and alerts to help them proactively scale infrastructure components. With adequate lead time, teams can manually provision additional capacity and scale static environments.

Elastic API Scaling

Elastic infrastructure reduces the lead time required to scale infrastructure up and down, and meet demand. Elastic infrastructure increases IT resource efficiency and reducing capacity delivery timelines. When designing an elastic API platform, consider interactions between API consumers, API Delivery Networks, Elastic Load Balancers, API Gateways, Enterprise Service Bus mediators, and service hosts.

Static or Elastic?

Static and elastic are complimentary. Static throttle tiers enable teams to offer their API as a well-defined product offering with expected operating margins. Subscription based usage monitoring helps teams track API usage per customer and understand how to best monetize the API through SLA tiers and charges. Elastic scaling enables teams to open up new API consumer channels ‘at the speed of Now‘ and offer your business as a service while minimizing IT infrastructure delivery timeline constraints.