WSO2 is now Choreo LabsLearn more

Trusted AI Governance
Build

Create agents and AI apps with the tools and languages your teams already use.

Connect

Link agents securely to your APIs, data, models and business systems.

Run

Deploy, observe and operate agents and AI apps reliably at any scale.

Govern

Control identity, access, cost and policy for every agent in production.

Agent Management

Developer Platform

  • API Management Cloud

    Design, secure and publish APIs with governance and analytics built in.

    Sign Up Now
  • Integration Cloud

    Connect apps, data and AI agents with low-code and pro-code integration.

    Sign Up Now
  • Identity & Access Management Cloud

    Secure sign-in and access for customers, partners, workforce and agents.

    Sign Up Now
  • Agent Management Cloud

    Build, run and govern AI agents with full visibility and control.

    Sign Up Now
  • Developer Platform Cloud

    Ship cloud-native apps faster on an internal developer platform.

    Sign Up Now

Self-Hosted

Full control over your stack. Deploy directly to your own servers, bare metal, cloud, or Kubernetes environment. Your data never leaves your perimeter.

Product Downloads
Choreo Labs
AI Governance
Platform
Solutions
Resources
Blog

WSO2 cloud incident report: jan 12, 2016

WSO2 Cloud experienced a serious service degradation on 12th January 2016: users were not able to login to the cloud for few hours.

Start time: 12th January 2016, 0833 PST
Recovery time: 12th January 2016, 1217 PST

Impact:

  • Users were not able to log into the cloud,
  • Sign-up was not working,
  • API Gateway was functioning throughout the incident serving API calls at normal performance level. There was only a 5 minute gateway downtime during database restart: https://uptime.cloud.wso2.com/

Root cause:

  • One of the housekeeping tasks running in our Identity Servers has failed due to a failure in acquiring a lock on a database table. This locked table is also responsible for storing the sessions and since it was locked, system was not able to complete the new user logins.
  • Since it was not possible to find out, which component was keeping the table locked, we had to restart the database server to get the system back on track.

Actions:

  • We have decreased the frequency of the aforementioned housekeeping tasks as advised by our Identity Server team.
  • We have also raised a support ticket with our internal support team to fix any possible future failures for this task.
  • We are investigating further to figure out which component had the table locked and to fix it.
  • We are looking into alerting and maintenance processes to ensure quicker resolution time in the future.