Country: The Netherlands
Bas is a technology leader in the AI and big data domain. His academic background is in Artificial Intelligence and Informatics. Trained as a software engineer and architect, he has 15 years experience in delivering succesful data-driven projects with a wide range of companies and technologies. He occasionally teaches programming courses and is a regular speaker on conferences and informal meetings, where he brings a mixture of market context, his own vision, business cases, architecture and source code in an enthusiastic way towards his audience.
Fast Data – an Overview of the Concepts, Architecture and Technology of Streaming Analytics
Streaming Analytics (or Fast Data processing) is becoming an increasingly popular subject in financial services, marketing, the internet of things, and healthcare. Organizations want to respond in real-time to events such as clickstreams, transactions, logs and sensory data. A typical streaming analytics solution follows a ‘pipes and filters’ pattern that consists of three main steps: detecting patterns on raw event data (Complex Event Processing), evaluating the outcomes with the aid of business rules and machine learning algorithms, and deciding on the next action. At the core of this architecture is the execution of predictive models that operate on enormous amounts of never-ending data streams.
But with opportunities comes complexity. When you’re switching from batch to streaming, suddenly time-related aspects of the data become important. Do you want to preseve the order of events, and have a guarantee that each event is only processed once? In this talk, Rob will present an architecture for streaming analytics solutions that covers many use cases, such as actionable insights in retail, fraud detection in finance, log parsing, traffic analysis, factory data, the IoT, and others. He will go through a few architecture challenges that will arise when dealing with streaming data, such as latency issues, event time vs server time, and exactly-once processing. Finally, Rob will discuss some technology options as possible implementations of the architecture.