The Future of Intelligent Middleware

The Future of Intelligent Middleware, IBM Research at Impact 2013 Conference

Contextual analytics
Continuous insight
Key technical problems: unstructured, unstructured, semi-structured; have been working on scale OR performance, not both, which will happen in a clustered environment

Why is contextual analytics important?
An urgent need to understand information, particularly asynchronous ones

Scenario: financial company with transactional credit card swipes, want to know as fraudulent or not
Fraud detection could be on scoring, but could have gang using card quickly over a short period of time
Scenario: Homeland security
Scenario: In-store video analytics
Scenario: Healthcare, sensors on patient

Want improved time to decision-making
Think of Continuous Insight as a Platform, with an engine that could scale up

This is an application server redux: not just three tier, now want transactions in memory not as a single JVM, but as in memory clustered
Challenge is analytics across JVMs
Would use MapReduce and Hadoop, except those are for scaling up, not in a cluster

Today, can have sales pipeline management, sales exec wants to see SmartSeller with Sales Challenge Alerts

Demo trains with hidden Markov Model

Started project on scale out computation, previously funded by DARPA, open source language called X10, for place-centric, asynchronous computing
Generates both C++ and Java
Will have programmer deal with place and asynchrony
Don’t have to code in X10, have a global programming model above that

Next steps to make this work: what is a good way to provide an analytics language at scale?