Firms must have an accurate market view in order to drive actions and remain competitive in the face of changing conditions.

Be it positions, trades, risk exposure or profits, having the most recent, comprehensive information enables companies to move toward more automated decision making and choose the right course of action in a timely manner.


Accurate and Real-time Detection

Detection of previously unknown insights with minimal false positives

Simple Infrastructure

An iterative agile approach that abandons complex data pipelines and long engineering cycles

Rapid Development and Deployment

Quick development of new machine learning models from training to production

Case Study – Stock Exchange

A major stock exchange outgrew its market surveillance platform due to the increasing complexity of modern-day trading strategies. The exchange turned to iguazio for a platform that powers machine learning and continuous analytics to adjust in real-time to changing market behaviors.


The Challenge

The stock exchange’s original surveillance system was based on a rules engine, which was well adept at detecting known market manipulation schemes such as pump and dump. However, it wasn’t able to detect attempts to manipulate the market using new, unfamiliar methods, which cannot be readily translated to algorithmic rules. These attempted manipulations resulted in many false positives and did not provide enough accuracy. A new system was necessary, one that is based on anomaly detection generated by machine learning and data science.

The exchange then tried to deploy a Hadoop-based data lake, so that its data scientists could research and develop machine learning models. But that presented a significant challenge in deploying those models into the operational detection system, as it required significant engineering efforts, which did not allow for agile, iterative refinement of new models.


The Solution

iguazio provides a Unified Data Platform that handles not just data science functionality, but also the runtime detection environment, so that models are quickly deployed to production in an agile, effective manner.

Deploying machine learning code to production and having it execute against streaming data without extensive re-coding is a unique value provided by the iguazio Unified Data Platform, which supports both research and production. This is enabled by iguazio’s unique multi-model, highly performant database, which handles different data access models and APIs, including streams, files, dataframes and objects. The support for streams and dataframes in particular facilitates a quick turnaround of machine learning models into operational, event-driven decisioning modules – effectively bringing the agile paradigm into the realm of data science and data engineering.

Using iguazio, the exchange was able to build a system that could detect previously unknown market manipulation schemes with minimal false positives. The iguazio Unified Data Platform is used as a basis for data science R&D as well as for real time detection, in a way that enables quick iteration and deployment of new machine learning models, without requiring complex data pipelines or long data engineering cycles. Using this iterative agile approach allows the exchange to refine their models in reaction to changing market conditions, keeping them on top of market abuse and a step ahead of their competition.