Tuesday, October 30, 2007

Bringing relational data to the Web

Everyone has data locked away in relational databases and other data sources. There have been several approaches for getting at this data- EJB entity beans being the worst of all the possible approaches probably! All the mess with object-relational mapping and the tens of ORM tools out there have been part of the strategies people have used to bring relational data to the Web.

WSO2's Web Services Application Server, WSAS, now has a feature that allows one to easily map relational data to the Web and back. Basically, what you do is write a parametrized query (or stored procedure call), decide how to get the values for those parameters from the incoming request (from the URI and the body) and then describe how to map the resulting table to XML. There's no bringing the relational data to objects at all- its directly relational to/from XML/URIs. Its actually quite a bit more general than that- we can also go to JSON or other serializations and on the other side, we can handle not only relational data but also Microsoft Excel files, CSV files, JDNI data sources and more. Kyle Gabhart has written a nice article on this technology for xml.com and has a figure I really like:

The details of the data services language we came up with can be found here. We've also sorted out how to make these data services into properly RESTful data resources (details in that document) and will have that working soon as well. (If any REST fan wants to review that bit of the spec and help improve it and/or help implement it please do!)

This is somewhat similar to what Microsoft Project Astoria is doing, but that's a lot grander and grandiose than our approach. Ours is a bottom up, pragmatic approach that targets folks with databases and specific query patterns they'd like to make available on the Web. The result is that someone can now write a mashup that directly touches your relational database and all you had to do was fill in a few forms and write down the query to execute. Basically, if you know a query you want to execute against a database you have, in 10 minutes you'll be view your database contents using the Web browser. Don't believe it? We have customers who've done that with mainframe databases! Try it and see.

And our stuff is ready to use right now and we'll support it for real use.

More information about data services is available in the following articles and blogs witten by Sumedha Rubasinghe, the lead on that work:

You can download WSO2 WSAS v2.1 from here. The source for all of this is available too - everything under Apache License. See the open source project home for WSAS on WSO2 Oxygen Tank.


Ravi Pinto said...

Hi Sanjiva,

Nice post. I wanted to check what is the difference between WSAS and SDO based application framework such as BEA's AquaLogic Data Services Platform or IBM's Esperanto?


Sanjiva Weerawarana said...

Hi Ravi,

SDO is a *much* more complicated solution to a wider problem. (I was involved with some prior work to SDO while at IBM Research .. and also some prior work to SCA.) SDO basically is a normalization of the data. So yes, you can take relational data, normalize it and then bring it to XML (or to any other thing). However, in practice all such abstractions leak and to really do stuff you have to sniff the details and look at where the data came from.

We just cut the crap and say "I want to take relational data to XML- just give me the shortest path." Just compare the level of understanding difference a programmer needs to have in order to do the same task with WSAS vs. with SDO and you'll become a believer ;-).


James Snell said...

Hey Sanjiva. This looks interesting. My one major concern is that data access layer frameworks are not exactly new; neither is mapping relational data to other formats. The problems with such frameworks tends to boil down to the details of sql query construction, performance, scalability, leaky abstractions, etc.