AI & Big Data Expo: Maximising value from real-time data streams

Ryan Daws is a senior editor at TechForge Media with over a decade of experience in crafting compelling narratives and making complex topics accessible. His articles and interviews with industry leaders have earned him recognition as a key influencer by organisations like Onalytica. Under his leadership, publications have been praised by analyst firms such as Forrester for their excellence and performance. Connect with him on X (@gadget_ry) or Mastodon (

As digital transformation accelerates across industries, more and more companies are recognising the untapped value in their real-time data streams. Enterprise streaming analytics firm Streambased aims to help organisations extract impactful business insights from these continuous flows of operational event data.

In an interview at the recent AI & Big Data Expo, Streambased founder and CEO Tom Scott outlined the company’s approach to enabling advanced analytics on streaming data. At the foundation of Streambased’s offering is Apache Kafka, an open-source event streaming platform that has been widely adopted by Fortune 500 companies.

“Where [Kafka] falls down is in large-scale analytics,” explained Scott. While Kafka reliably transports high-volume data streams between applications and microservices, conducting complex analytical workloads directly on streaming data has historically been challenging. 

Streambased adds a proprietary acceleration technology layer on top of Kafka that makes the platform suitable for the type of demanding analytics use cases data scientists and other analysts want to perform.

Because these continuously flowing event streams power critical operational systems and core business functions, data quality must already meet high standards in terms of accuracy, timeliness, and structure. By leveraging these existing Kafka data pipelines, Streambased ensures its analytical capabilities have access to up-to-date, clean and well-organised data.

Use cases that showcase the power of Streambased’s approach include fraud detection in financial services. If an anomalous transaction occurs, analysts can quickly query similar or related transactions to investigate – which would be difficult and inefficient to accomplish with a pure streaming architecture. Streambased’s optimization for analytical interactivity enables users to rapidly gather contextual insights without disrupting their workflow.

The convergence of operational and analytical data platforms represents an impactful trend that Streambased calls the “streaming data lake” movement

“I think we are at the period of the streaming data lake movement. And by a streaming data lake, I mean a complete convergence between data systems that we use for analytical purposes and data systems that we use for operational purposes,” explains Scott.

Recent enhancements like infinite data retention in Kafka and native streaming analytics services lay the foundation for this new paradigm. For now, Streambased remains focused on empowering business analysts through frictionless self-service access to granular real-time data, without requiring changes to existing tools and processes.

You can watch our full interview with Tom Scott below:

(Photo by Robert Zunikoff on Unsplash)

See also: AI & Big Data Expo: Unlocking the potential of AI on edge devices

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with Cyber Security & Cloud Expo and Digital Transformation Week.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

Tags: , , , , , , , , ,

View Comments
Leave a comment

Leave a Reply