Internet Alchemy

For a while now I’ve been experimenting with writing my own little PHP applications that run against the Talis Platform. Most of these have never been seen in public because they’re mainly just for scratching an itch I have at the time. I’ve also used a lot of them to validate my own thinking around the types of services that the platform needs to provide to build interesting applications. The core of most of those applications became Moriarty my PHP library for accessing the platform. I use Moriarty extensively now to kick start any development I do. I’m even using it to write PHP scripts for running at the command line. I’m not sure that PHP is going to usurp Perl from my toolbox, but it’s certainly becoming my language of choice for working with RDF.

I’ve been looking carefully at the core patterns that my PHP applications have been following to see if there’s anything else I could pull out. This is generally how I prefer to build new libraries: extracting them from several different projects. Assuming you know how your library is going to work before you’ve written any applications is almost always wrong. I like using libraries that have distilled the essence of repeated attempts at solving the same problem. That’s why I never think about modularization of a codebase until I need to.

I’ve been gravitating towards Konstrukt because it appears to be the least intrusive of the PHP web application frameworks out there and it keeps fairly true to REST principles. I used it to build Kniblet as part of a platform tutorial. However, there are some quirks that it has that I don’t like. For example, to return anything other than HTML requires you to throw an exception. That mechanism works quite well for most applications but doesn’t really suit data-rich applications that have multiple output formats.

It’s with this in mind that I’ve started a new PHP web application framework called Paget. Calling it a framework is somewhat of an overstatement. It’s a few classes that make it easy to publish RDF as linked data. It’s very primitive at the moment, but it’s quite versatile.

It uses a simple configuration array that is passed to a dispatcher that handles the request. The application’s default behaviour is specified using this configuration. One part sets up a series of regular expressions that match URI paths handled by the application and map them to the resources it provides. The data about each resource is obtained by using one or more “generators”. These are simply classes that generate RDF for the given resource. Paget runs each generator to gather the RDF data describing the resource and then handles the serving up of that data according to linked data principles. Right now that’s just enough behaviour to function as a generic linked data publishing framework.

I have three different deployments of Paget that are publishing three RDF data sets using different generators. Each of these was quite trivial to set up, being a few lines of confiiguration. For my own site’s data space I wrote a generator that fetched RDF directly from one of my platform stores (this one) and served it up as HTML and various flavours of RDF. See, for example, http://iandavis.com/id/me which is URI that identifies me.

My second deployment was for PlaceTime, a URI space that I have operated since 2003. It provides RDF data for timelike entities like instants and intervals and spacelike points. However, it hasn’t been fully linked data compliant (mainly because it pre-dated the decision on httpRange-14). I wrote a generator for each type of entity that creates trivial RDF for each valid URI in the space. Some examples:

• http://placetime.com/instant/gregorian/2003-05-19T08:00:00Z – an instant in time
• http://placetime.com/interval/gregorian/1970-06-15T19:31:00Z/P1D – an interval in time
• http://placetime.com/geopoint/wgs84/X-126.817Y46.183 – a point in space

Finally, I created a generator that reads a local RDF file. I then used it to serve up the whisky vocabulary that Tom, I and several others created at the recent VoCamp Oxford

Admittedly, all these datasets and spaces look pretty similar but this is still early days for Paget. I have some ideas for future development that will flesh out Paget into a fully-fledged RDF driven application framework. For example: as well as generators I plan to add filters, augmenters and transformers that alter the generated data in arbitrary fashions. These could be used to trim the data down, or to convert it to a more usable structure. I can imagine that it would be very useful to be able to pull in more RDF from arbitrary locations on the Web to supplement data in the initial set, e.g. with schema information or additional details. In my opinion that’s one of the significant differences between the web of data and the web of documents: the web of data is going to enable more information to be brought automatically together for the user rather than forcing them to seek it out.

Paget’s HTML rendering of RDF is very primitive at the moment, making only basic attempts to make it human readable. It’s still extremely tabular which is hardly a great use of structured information. One area that I’ve been interested in exploring is that of dynamic user interfaces that adapt to the underlying data automatically. RDF is particularly amenable to building these kinds of interfaces because of its uniform data model. A lot of work on this was done by the Fresnel project and it would be interesting to apply some of the learnings from that project to building dynamic web applications. My goal here is to code as little specific behaviour into the application as possible, instead making the application detect patterns in the data and provide suitable user interface behaviours at runtime. This is really the only way we’re going to be able to build true open world applications, i.e. those that are tolerant of missing data and can adapt to new and unanticipated data.

What I’m still experimenting with is whether these user interface additions should be server-side or passed on to the client. Some of the augmentations could make more sense when actioned by the client based on user activity.

There’s lots to research here and hopefully some of these ideas will make it into Paget very soon.

Update: see my follow up post that describes the major revisions I made to Paget after this was written

Last Friday I gave a presentation on the Talis Platform to the SWIG meeting, kindly hosted by HP Labs Bristol. I’ve posted the slides up at our n² developer community site. They’re not much to look at but I wrote them to be informative rather than suggestive. Although I love seeing beautifully spare presentations being given, they frustrate me when I want to go back and see what the speaker said and find a picture of a butterfly on a flower

Nad’s written a good summary over on his blog and managed to capture the questions I was asked at the end too, which is nice to see after the event. The whole day was brilliant with lots of chances to natter and catch up with everyone. I met lots of new people too and everyone seemed to be doing something interesting. I even managed to harangue Stuart Williams off of the W3C TAG on my recent Web architecture posts. Andy said about 50 people were attending despite there being no marketing of the event at all which is a good indicator of the rising popularity of the Semantic Web. Talis are planning to host a SWIG meeting like this in the middle of next year – hopefully we can get more people from the Midlands interested.

There were plenty of other cool presentations on the day too. Nad and Rob were blogging but couldn’t post them live for some reason so check out their sites and Nodalities too in the coming days. I particularly enjoyed Leigh’s talk on Facet, another templating framework for RDF, this time in bog-standard Java allowing the use of JSP and/or Velocity; Richard’s talk on Sindice (which I didn’t get to see at ISWC and for us ignorant Brits is pronounced “sin-dee-chee”); and Graham’s talk on image publication. All great stuff!

Next month will mark the end of my second year at Talis. I was inspired to join by my conversations with the CEO, Dave Errington and CTO, Justin Leavesley. They both shared this strong belief that something was changing, that there was a coming wave of disruption based on increased connectivity and that Talis could be well placed to catch it. Throughout the conversations ran a refreshing vein of humility and honesty, an admission that not everything was figured out and that anyone joining could help shape Talis strategy. For me, that presented an amazing challenge that I didn’t want to miss. As a backdrop, Talis had been in the middle of huge organisational changes, moving away from 36 years of being a co-operative not-for-profit, through to a limited company and, ultimately, to being majority owned by its employees. A tough transition, but one that was necessary if Talis were to catch this next wave.

Today’s Talis is vastly different even from the Talis I joined. Just about every employee is a shareholder and Talis is paying dividends on those shares. We’ve built a great team, developed some amazing core technology and have some very cool applications in the pipeline. Without the bureacracy that enveloped the company in its former incarnation it’s become a place you can get great things done.

Last month I was flattered and honoured to be invited to join Dave and Justin as an executive director on the board of Talis Group. Justin is moving across to become Chief Strategy Officer, which very much suits his economic strategy expertise, and I’m stepping up to become CTO, responsible primarily for our platform strategy and technology. What a huge challenge and responsibility: I never thought I’d be CTO of a company older than both me and the Internet! Even though the company has been through enormous changes over the past four years we still have a lot of work to do. With my appointment to the board comes a new focus for the company on becoming a global software business, active across every domain. We’ll continue to grow in our traditional library market, while at the same time expanding the platform strategy and technology we have been working on for the past two years.

If you follow this blog then you’ll have a good idea about the direction we’re taking with our Platform. Call it the Semantic Web (upper or lowercase), Web 3.0 or the Web of Data, it’s about connecting people, businesses, machines and devices together, making it more efficient to communicate and giving us access to even more information with which to make better decisions. The whole company is working toward this vision, so it permeates all of our products, and my particular focus is on the technology needed to get there.

Part of that is ensuring that Talis continues to excel in technology. But that doesn’t mean devising better processes or selecting tools or inventing architectures. Those kinds of activities are just fiddling around the edges. The single most important factor in a successful software business is having happy, talented people

Talented people, to me, are those that can make things happen, often against the odds, through perseverance, inspiration and sheer passion. They learn from every challenge and they get better. And the best way to help them do that is to get out of their way.

If you’re not convinced, if you think you can horizontally scale the development process, go and read some of Joel Spolsky’s writings on the subject. Then read Founders at Work and try to find a startup whose success wasn’t entirely predicated on the talent of its founders.

I’m lucky that Talis already has a fantastic team of talented developers, certainly the best I’ve ever worked with. But to achieve future success we need to increase in strength, not only in the dev group, but across the whole company. We’ve got a big vision, but we haven’t got all the answers yet and we’re looking for people to come and help us figure some stuff out.

So here’s the pitch from the new CTO: if you want to face the challenge of creating something world-changing; if you like the idea of a company that controls its own destiny and is small enough for you to change it; if you never want to stop learning; then I want to hear from you.

Come and find me at XTech this week, or grab one of the other 8 Talisians that are going and get the lowdown on what it’s really like to work here. If you can’t make XTech, then email careers@talis.com and tell them I sent you

Over on O’Reilly Radar there’s an interesting post about what it means to be an open platform (in the context of iPhone vs Treo). Some key points (which I think I agree with):

If you think about the original PC architecture, the openness was via the expansion ports and the commodity nature of the components used on the motherboard. It engendered whole new industries for adding functionality (e.g. video and sound cards) and scaring the platform provider (networking – who needs mainframes and minis?). But we’re all pretty much entrenched in that PC architecture. IBM of course lost control of the market to the clones and their attempt to close up the architecture failed miserably (MCA – micro channel architecture in the PS/2). But they did reap a lot of licence fees from the clone makers during that time (via patents on the technology) and I suspect they would still be doing so if they hadn’t tried to close the market up, thereby alienating the clone makers and forcing them to devise new alternate architectures.

Open platforms can be scary for the provider but if you hold your nerve, as we are with our open platform at Talis, then the rewards can be immense.