15 February 2007 by Ian Davis
I’m excited because I just got back from a great long weekend away walking the ancient Wiltshire landscape and found a message in my inbox confirming that my XTech proposal had been accepted. My talk and paper’s going to be on RSS Remixing – using RSS 1.0 to “augment” feeds and search results.
Here’s the abstract I submitted:
In this session Iâ€™ll demonstrate and explain a new ultra-simple protocol for augmenting search results with enrichments and related content. Similarly to OpenSearch the protocol is RSS based. But instead of sending the search terms to each registered search provider we send the results and ask the providers to inspect them and add anything else they know about the items. Iâ€™ll show how this protocol allows rich search applications to be built very simply by remixing results from several different data sources.
As Adam Bosworth pointed out, using RSS and its kin as a data transfer format makes a lot of sense when you consider the number of clients that can consume it. It is particularly useful for ordered sets of items such as the output of search engines. We used RSS to design a very simple protocol for combining results from different search engines into a single RSS feed that can be consumed by any feed client or application. In fact, because the final result is itself RSS it can in turn be used to augment results from other searches.
One untapped feature of RSS 1.0 is the potential of merging multiple RSS feeds together. The underlying RDF nature of RSS 1.0 provides this capability for free and with some work it could be adapted to work with other flavours of RSS and Atom. Using a range of techniques from simple pattern matching to Sparql querying of the full feed we can look up related items and with a few simple conventions mix the results into the original feed.
One example Iâ€™ll use will show how an RSS feed of book details can be used to build a simple book search application. Other search services can match on key information such as ISBN, authors and titles to augment the book information with jacket images, reviews, author biographies and relevant links, all drawn from different data stores.
There are, perhaps, some similarities with Yahoo’s pipes but the main distinction is the embarrassingly parallel nature of our approach plus the focus on augmenting results rather than filtering or aggregating. We used this technique to build Cenote at Talis, which is really a tiny PHP script and some XSLT leveraging the RSS remixing services we’ve implemented in the underlying platform.