WSO2 is now Choreo LabsLearn more

Trusted AI Governance
Build

Create agents and AI apps with the tools and languages your teams already use.

Connect

Link agents securely to your APIs, data, models and business systems.

Run

Deploy, observe and operate agents and AI apps reliably at any scale.

Govern

Control identity, access, cost and policy for every agent in production.

Agent Management

Developer Platform

  • API Management Cloud

    Design, secure and publish APIs with governance and analytics built in.

    Sign Up Now
  • Integration Cloud

    Connect apps, data and AI agents with low-code and pro-code integration.

    Sign Up Now
  • Identity & Access Management Cloud

    Secure sign-in and access for customers, partners, workforce and agents.

    Sign Up Now
  • Agent Management Cloud

    Build, run and govern AI agents with full visibility and control.

    Sign Up Now
  • Developer Platform Cloud

    Ship cloud-native apps faster on an internal developer platform.

    Sign Up Now

Self-Hosted

Full control over your stack. Deploy directly to your own servers, bare metal, cloud, or Kubernetes environment. Your data never leaves your perimeter.

Product Downloads
Choreo Labs
AI Governance
Platform
Solutions
Resources
Article

[Blog post] CEP performance: Processing 100k to millions of events per second using WSO2 complex event processing (CEP) server

With WSO2 CEP, you can use SQL style queries to detect interesting patterns across many data streams. We call the standalone version of the CEP as Siddhi, and that is what you need to use if you need to embed a CEP engine within a java program. On the other hand, WSO2 CEP provides CEP query support running as a server, and you can send events using Thrift, Web Services, REST calls, JMS, and emails.

WSO2 CEP can handle a few 100k events over the network and a few million events within the JVM. We have done and published those numbers before. In this post, I will try to put this all together and give some context to the different numbers.

The following event includes multiple properties and queries that match those events against the given conditions.

Same JVM events performance

Setup: We used Intel(R) Xeon(R) X3440 @2.53GHz , 4 cores 8M cache 8GB RAM running Debian 2.6.32-5-amd64 Kernel. We genereted events from the same JVM.

Case 1: Simple filler (from StockTick[prize >6] return symbol, prize)

Case 2: Window (From StickTick[symbol=‘IBM’]#win.time(0.005) return symbol, avg(prize))

Case 3: Events patterns A->B (A followed by B).

From f=FraudWarningEvent ->

p=PINChangeEvent(accountNumber=f.accountNumber)

return accountNumber;

Performance over the network

Setup: We used Intel® Core™ i7-2630QM CPU @ 2.00GHz, 8 cores, 8GB RAM running Ubnthu 12.04, 3.2.0-32-generic Kernel, for running CEP and used Intel® Core™ i3-2350M CPU @ 2.30GHz, 4 cores, 4GB RAM running Ubnthu 12.04, 3.2.0-32-generic Kernel, for the three client nodes.

The following results are for a simple filter; we sent events over the network using thrift.

Performance of a complex scenario

Finally, the following is the performance for a DEBS grand challenge. The grand challenge was to detect the following scenarios from the event generated from a real football game.

Usecase 1: Running analysis

The first usecase measures each player’s running speeds and calculates how long he spent on different speed ranges. For example, results will show that the player "Martin" is running fast from the time 27 minutes and 01 second of the game to 27 minute and 35 second of the game.

Usecase 2 and 4: Ball possession and shots on goal

For the second usecase, we need to calculate the time each player controlled the ball (ball possession). A player controls the ball from the time he hit the ball until someone has hit the ball, the ball goes out of the ground, or the game has been stopped. We identify hits when a ball is within one meter of a player and its acceleration increases by more than 55ms-2.

The usecase 4 is to detect hits and emit events if the ball is going to the goal.

Usecase 3: Heatmap of activity

Usecase 3 divides the ground to a grid, and calculates the time a player spends on each cell. However, this usecase needs updates once every second. In the first part, we can solve it just like as in the first usecase, but to make sure we get an event once a second, we had to couple it with a timer.

You can find more information from https://www.orgs.ttu.edu/debs2013/index.php?goto=cfchallengedetails , and you can find the queries from this blog post.

Setup: VM with 4 cores (@2.8 GHz), 4 GB RAM, SSD HDD, and 1GB Ethernet, and we replayed events from the same JVM.

Source : https://srinathsview.blogspot.com/2013/08/cep-performance-processing-100k-to.html