WSO2 is now Choreo LabsLearn more

Trusted AI Governance
Build

Create agents and AI apps with the tools and languages your teams already use.

Connect

Link agents securely to your APIs, data, models and business systems.

Run

Deploy, observe and operate agents and AI apps reliably at any scale.

Govern

Control identity, access, cost and policy for every agent in production.

Agent Management

Developer Platform

  • API Management Cloud

    Design, secure and publish APIs with governance and analytics built in.

    Sign Up Now
  • Integration Cloud

    Connect apps, data and AI agents with low-code and pro-code integration.

    Sign Up Now
  • Identity & Access Management Cloud

    Secure sign-in and access for customers, partners, workforce and agents.

    Sign Up Now
  • Agent Management Cloud

    Build, run and govern AI agents with full visibility and control.

    Sign Up Now
  • Developer Platform Cloud

    Ship cloud-native apps faster on an internal developer platform.

    Sign Up Now

Self-Hosted

Full control over your stack. Deploy directly to your own servers, bare metal, cloud, or Kubernetes environment. Your data never leaves your perimeter.

Product Downloads
Choreo Labs
AI Governance
Platform
Solutions
Resources
Article

Benefits of a multi-regional API management solution for a global enterprise

Introduction to Data Centers and Regions

A data center (DC) is a physical group of networked computer servers typically used by organizations for the remote storage, processing, or distribution of large amounts of data. A data center often requires extensive redundant or backup power supply systems, cooling systems, redundant networking connections, and policy-based security systems to run the enterprise's core applications.

A region is a geographical location with a collection of physical data centers in that region. Every region is physically isolated from and independent of every other region in terms of location, power, water supply, etc.

Multi regional deployment is a special case of multi data center deployment.

Why a Multi Data Center Deployment?

A multi data center deployment can provide valuable benefits to organizations and developers to deal with specific issues as follows:

  • Machine failures
  • Data center outages
  • Disaster recovery
  • Efforts to increase availability (more 9s)

When you have business-critical applications, their failures will naturally impact your day-to-day business. Failures happen in different ways. For instance, an application can fail due to a bug in the software, which can be addressed by applying fixes. Sometimes the machine that hosts the application can fail; for this, it’s recommended to run the application at least in two machines so it provides a minimum set of high availability (HA).

Typically, data centers have redundant or backup power supply systems, cooling systems, redundant networking connections, etc. But past experiences have shown that data centers can face outages due to various reasons.

Therefore, it’s important to have a disaster recovery (DR) site to minimize the impact from such outages.

If you are providing a set of services, your customers want to have availability guarantee of your services. To this end, a multi-data center deployment will help you to increase the availability of your services.

Why are Multi Data Center or Multi-regional Deployments Complex?

Many believe that replicating a database across the data center is sufficient when dealing with a multi-data center deployment. Even though it might hold true for simple websites, it’s not the case for middleware. Artifacts sometimes persist in file systems or databases, or even both. Some distributed components need to have coordination between them, while some components must maintain states to be able to make decisions. These types of applications, however, have some complexity when deployed in the single data center.

Given the complexity, configuring and operating a middleware stack consistently across multiple locations is a non-trivial (and expensive) task that potentially introduces more points of failure than a single data center deployment. Latency between data centers will introduce inconsistencies, and distributed decision making will also become more complex.

Maintaining the state across multi-data centers is challenging; therefore, depending on the use case, you would need to do stateful analytics on the collected data. If you carefully analyze all these factors, multi-data centers are not trivial and you need to weigh the benefits as well as the complexity of the deployment.

What About Active-Active Multi Datacenter Deployment?

An Active-Active multi datacenter deployment comes with the following dependencies:

  • Master-Master Database replication
  • Master-Master (both way) file sharing mechanism
  • Fast network access (low latency)
  • Reliable network

If you can achieve the above dependencies, it is possible to have an active-active multi datacenter deployment. However, given our past experiences, we have seen the following limitations with such a deployment:

  • Throttling will have to be approximated, and not precise. High latency in between Gateway node and TM nodes will decrease accuracy of execution of throttle out decision. When database and filesystem synchronization takes a significant amount time (due to high latency) , two datacenters will be have an inconsistency to a time period. This leads system to Inconsistent.
  • If we consider all these factors, and when weighing the complexities versus benefits, an active-active multi datacenter deployment is not recommended unless you can do traffic partitioning.

Use Cases of Multi Region Deployment

Multi regional deployment can provide valuable benefits to organizations. A few of them are as follows:

  • Regional data center outages
  • Disaster recovery
  • Increase availability (more 9s)
  • Reduce latency for regional access
  • Help to partition traffic among the regions
  • Align with regional regulatory enforcement. (application calls)
  • Avoid cloud vendor lock-in (with multi-cloud deployment)

The key difference between a multi-regional and multi-datacenter is that regional data centers can be fully isolated with large distances between them as they would generally be across different countries or continents. Therefore, traffic partitioning will provide tremendous benefits in a multi-regional deployment scenario. Regional traffic can be routed to the nearest region and it will give low latency and fast access to the APIs (given that backend services are also deployed in the region). In addition, it will help to get tottering decisions optimized to regional access of the APIs.

Sometimes businesses need to comply with regional regulatory enforcement; therefore, in such a scenario, you might need to terminate API traffic within the regions. One way of achieving this is by using multi regional deployment.

Many organizations use public clouds to get infrastructure services and a common major concern is vendor lock-in. Multi regions can also be considered as multi cloud providers. Therefore, having a multi regional deployment can be extended to a multi-cloud deployment, which in turn will help to avoid lock-in.

You can have multi-regional deployment by covering two key use cases as follows:

  • Use Case 1: Deploy all APIs in all regions. API traffic will be partitioned and be routed to the relevant region.
  • Use Case 2: Deploy selected APIs in selected regions. API traffic will be partitioned and routed to the relevant region.

The following section will explain these use case as separate deployments; however, it’s possible to merge these two into one as well.

Use Case 1: Deploy All APIs in All the Regions

In this scenario, we deploy all the APIs in all the regions. It is the same as discussed in the multi data center use cases, but API traffic will be partitioned region by region.

Deployment can be any sort of pattern, but we have used a fully distributed pattern. A distributed pattern will help to scale the deployment.

Figure 7

In the second region, we have used separate GW, KM, and TM, but you can combine them too. The only concern when doing so, however, is that you should only have API runtime components in the second DC, not the management components (publisher, store or admin apps). Publisher, Store and Admin traffic should be routed to one (master) region.

Having a separate KM in the second region will help to reduce latency for key validation. In the same line, having a TM in the second region will help to increase accuracy of the throttling. Regionalized backend services will increase performance, and we can use dynamic endpoints to auto select backend services in those regions.

Let us take a detailed look of synchronization across the regions.

A - All tables of APIM-DB (except IDN_OAUTH2_ACCESS_TOKEN table) of Region 1 should be replicated to Region 2 via primary-secondary database replication. Since we are having API traffic partitioning, one token will be only valid for one region. If someone switches regions, they need to create a new token to work in the new region.

B - Region 1 to Region 2 shared file system (for APIs) should be replicated using RSYNC (or any file system replication mechanism)

C - Region 1 to Region 2 shared file system (throttling Policies) should be replicated using RSYNC (or any file system replication mechanism)

D - API analytics traffic should be published to the master region. Analytics is stateful and, therefore, we can’t do local (regional) analyzing and mergers the results unless you have partition key in the API analytics data. Other reasons are:

  • Some analytics (fraud detection, anomaly detection etc) need to get all API analytics data to analyze and provide accurate decisions
  • Publisher, Store, and Admin app have the presentation layer of these analytics

Use Case 2: Deploy Selected APIs in Selected Regions

In some use cases, we need to push selected APIs to particular regions. API management can be done in a master region and access to these APIs can be regionalized. In such a scenario, the following multi regional deployment will help. In this scenario too we have the same assumption that API traffic will be partitioned according to region (Figure 8).

Figure 8

As discussed earlier, you can use any deployment pattern in Region 1 and depending on the optimization, you can select the Region 2 deployment pattern.

Let us take a detailed look of the synchronization across the regions. One important note is that since we have selectively pushed APIs to regional GW, there’s no need to sync all API files across the regions.

A - All tables of APIM-DB (except IDN_OAUTH2_ACCESS_TOKEN table) of Region 1 should be replicated to Region 2 via primary-secondary database replication. Since we have API traffic partitioning, one token will only be valid for one region. If someone switches regions, they need to create a new token to work in the new region.

B - Region 1 to Region 2 shared file system (throttling Policies) should be replicated using RSYNC (or any file system replication mechanism)

C - API analytics traffic should be published to the master region. Analytics is stateful and, therefore, you can’t do local (regional) analyzing and merge the results unless you have a partition key in the API analytics data. Other reasons are:

  • Some analytics (fraud detection, anomaly detection, etc.) need to get all API analytics data to analyze and provide accurate decisions
  • Publisher, Store, and Admin app have the presentation layer of this analytics

D - Selected API should be pushed to Region 1 (master) to Region 2. When you add an external GW (regional GW) to the publisher it will appear in the API publishing window. Then from the Publisher node (in master), initiate an API call to the Regional gateway (this can be via LB to avoid a single point of failure) to create the API files in the regional GW file system.

Conclusion

A regional multi-data center deployment can provide valuable benefits to organizations and developers to deal with specific issues that are common across many verticals. Among these are machine failures, data center outages, disaster recovery, and organizations’ efforts to increase availability. Multi-regional deployments also help organizations to meet regulatory requirements and avoid cloud vendor lock-in. While there are many patterns an organization can explore for a multi-data center deployment, we recommend an active-passive deployment as it helps to minimize complexities, and ensures stability and scalability. To this end, partitioning of traffic is the recommended way in a multi-region deployment. In this use case, publisher/store/admin traffic should only be routed to one selected region and application traffic should be partitioned based on regions to ensure stability.