Sunday, June 14, 2009

New LSF project: peer-to-peer distributed Web cache

The Lanka Software Foundation started a project at the beginning of this year to develop a P2P application framework (called Dalesa) and then a set of applications on top of that. Wathsala Withanage leads that work and has two other great team members working with him (Nuwan & Nishshanka).

Wathsala has recently blogged about the architecture of Dalaesa and the DCache application - a distributed, peer-to-peer Web cache which is an alternative to Squid-like centralized caches. The architecture figure Wathsala drew is the following:

There are lots of potential applications that can be built on top of a P2P application framework. The DCache was basically a POC .. to show that the lower layer works. Other more interesting apps include building autonomous security boxes that mesh together and provide a P2P monitoring layer. Have ideas? Contact Wathsala and join the project!


Aj said...

This is indeed interesting. I can think of many different applications that can benefit from this. Media caching for example - a popular video in youtube that has a viral expansion can benefit from having a distributed cache.

Wathsala Vithanage said...

@Aj Unfortunately youtube does not allow caching, they set no-store/no-cache in Cache-Control header. However we can still cache those by not honouring headers, but I would not like such hacks creeping in to the code. Instead we will provide an API for module writers so they will be able to write modules that will affect the final caching decision. By using this method one could write modules to rule out caching restrictions enforced by origin servers.

Kevin Clark said...

Vithanage, that would be really cool. Keep up the good work.