Thursday, December 21, 2006

Two Dot Oh No!

What's with all the 2.0-ishness that everybody jumps on as if their life depends on it? Maybe it's all Tim O'Reilly's fault, since he created this Web 2.0 buzz, although a 2.0 moniker has been around for years, such as in Business 2.0. More recently, companies on my radar also start to use the term and position themselves in the Business Intelligence 2.0 space. Oh, please.

I had to find out what the hype is all about.

According to DMReview,
...business intelligence (BI) today has not changed in concept since the invention of the relational database and the SQL query - until the advent of BI 2.0.
What? Under which rock have they been hiding? First of all, BI wasn't there yet when RDBMS and SQL were invented, and I know that because I participated in projects around System R in the early 80s. And if I think about BI's evolution with reporting, OLAP, data mining, dashboards, scorecards, predictive models, etc.... this DMReview quote is pure nonsense. If anything, we should be talking about at least BI 8.0, considering the conceptual changes in the world of BI since we began. The article continues
If the goal of BI 2.0 is to reduce latency - to cut the time between when an event occurs and when an action is taken - in order to improve business performance, existing BI architectures will struggle.

With BI 2.0, data isn't stored in a database or extracted for analysis; BI 2.0 uses event-stream processing. As the name implies, this approach processes streams of events in memory, either in parallel with actual business processes or as a process step itself.
While I agree that there is an ongoing trend to reduce latency, this is nothing new. Changed Data Capture (CDC) mechanisms have been picking up events and transactions in near-realtime for years. So the "realtime"-aspect does not describe a completely new paradigm. And what's this thing about data not being stored or extracted? OK, an event does not need to be stored first in a database or any other content store to be analyzed, but event data all by itself is rather useless. Only in combination with existing BI infrastructure such as a data warehouse, any analytical component (in a process stream or not) can generate insight.

For example, let's assume there is a fraudulent event happening. Of course, nobody knows that fraud is about to be committed. The data associated with the event sits in this process stream and is not stored in a database (yet). So, everything is according to the definition above. Then what? How does that transaction event trigger any action elsewhere? How can any application or user figure out anything? Certainly not by simply looking at the transaction. The event data could be sent to some data mining model that generates a fraud score and then it triggers an alert. Is that what is meant by "event stream processing"? If so, we've been seeing this kinda thing for years. In fact, the whole discussion about analytical applications describes the same scenario: Operational applications (executed in "real-time") call analytical components that figure out something, send an alert, call another program or otherwise influence what is happening next. Sounds the same as BI 2.0 to me. Yawn.

2 Comments:

Frank Buytendijk said...

Andy,

I have to disagree with you. BI is some shape actually exists longer than the relational database. The first versions of multidimensional analysis predate the relational database for a few years. There is a common misperception that BI is all about query and reporting (BTW, was there no Q&R in the days of hierarchal and network databases?), but it is not. Q&R is only the information consumption layer. A robust BI stack also contains an information production layer. You know, OLAP is not such a bad thought. It provides calculation types that requires tons of SQL programmers need to sweat on (e.g. ABC), and it provides an environment is both compliant (predefined data and structures) and flexible (slice and dice)...

December 22, 2006 11:52 AM  
dcunningham said...

Hi Andy, it's nice to see you blogging. It's also good to see some good banter with your former colleague, Mr. Buytendijk. I've also been talking with a lot of ISVs talking about the so-called 2.0 era of BI. I think there will be some major enhancements and disruptive changes in the BI market in 2007 and of course at Saleforce.com we're expecting this to be all about software-as-as-service (aka "on demand"). I've been writing about on-demand BI on my Saleforce blog: http://blogs.salesforce.com/analytics/. Happy New Year!

December 27, 2006 4:03 AM  

Post a Comment

Links to this post:

Create a Link

<< Home