Weblogs in Research

As I write this, we're seeing a burst of useful, practical innovation in the engineering practices that underpin our community's Web tools -- the things like Radio Userland, Technorati, Tinderbox, and NetNewsWire were use to keep in touch. I'd like to step back and show the non-technical audience what's going on, to the limited extent that I see it -- and to show why the digital arts must learn how to do this.

This week, I'm in Maastricht at the wonderful Jan van Eyck Academie, an advanced school for the study of arts and theory, talking about Personal Publishing Pandemonium and participating in a roundtable on the nature of the book, where my talk is titled Not Chopped Trees. So, I'm even further from the center of things than usual; at least, this lends perspective.

The core question is a very technical one, about the best way tools like Tinderbox should talk to Technorati, a service that keeps track of who is linking to your weblog. Sifry, the guy behind Technorati, added a new communication method early this week. Winer, the guy behind Radio Userland, said "hooray", and arranged for Radio Userland to use this new communication channel. But, he observed, this was harder to do than he thought was necessary.

The difference of opinion was interesting to a bunch of people because it touches on two things we all suddenly need to do. Things that each of us has done, but that are still new enough that nobody is completely sure how to do them best.

1. How should we best talk to Web servers when we're asking questions, not just fetching pages? Web servers let us do two things; we can GET something or we can PUT something. PUT was really intended, way back when the Web was first designed, for something else that never worked out, but the street finds a use for things. So, which should we use, and how?
2. Nowadays, what we send to the Web server is often XML, and what we get back is bound to be XML. This has lots of advantages, but it also takes some work. One of the reasons Winer complained is that the new Technorati approach meant more work for him.

So, we have lots of discussion amongst technical people all over the world this week on these two interlinked questions. Some of it shows up in weblogs. Lots of it happens over email. Still more, I'm sure, happens around water coolers. But a few years back, it would have been water coolers all the way down, and you wouldn't even know it was happening.

Dave Winer's unique talents are another reason it's happening. His weblog has a lot of readers. He has strong opinions, and he speaks his mind. He doesn't mind upsetting people he thinks are wrong. So, voices have been raised, and that attracts a crowd.

The whole question is one of avoiding extra work, not making things harder than necessary. Because we all expect to be doing a lot of this in the future, we want to get it right as soon as we can. (One lesson from the Web that we all have learned is that making things very easy on programmers often means that lots of programs get written. Inconveniences and small obstacles add enough friction that progress moves much more slowly, or just stops. So, if your business or your spiritual calling depend on Web services of some kind, you fervently want to get everything easy.)

As a result, some of the top people in the field started working on the problem right away -- both expressing opinions and writing code. Aaron Swartz, for example, said, "The usual way we get information from XML files is too hard; here's a library that makes it easier." Email flurries followed. Is it really easier? What if I need to do X or Y? Revisions and improvements were added. (We're now up to the third release, and I don't think it's been 48 hours)

Swartz's idea is to read the XML file and make it all available in a data structure, with a nice syntax for getting at parts of it. So, for example, we might say

trackback.weblog[3].title

to get the title of the third weblog in our trackback list. This isn't brand-new -- most of us have something like this in our personal toolbox -- but it's nicely done. Some of Swartz's tools are better, for example, than what we've build for Tinderbox, so we'll add them.

It took me several emails (to Winer, to Swartz, to one of our customers who'd like some other XML changes) just to understand the disagreements and to see why Swartz's approach is better than what I'm doing now. The importance of this kind of collegial share cannot be underestimated.

Doing this will make Tinderbox just a little bit smaller and easier to improve. It's not a lot of work in any case; it looks like rain in Maastricht this morning, and I might be able to write the key part in a café should the clouds break.

Some of these questions are about rhetoric and style, about what kinds of writing are clear and persuasive. The writing we're talking about is computer code, but the underlying issues are the same as for prose. This particular community is willing to discuss style and rhetoric, to look closely at each other's work, and to quickly adopt better ways of doing things. This is a lesson that literary hypertext and the DAC community should take to heart. (The eXliterature way of doing this would require a year of planning, a foundation grant for a conference, a committee report, and then they'd decide that it was all obsolete, uncool, and complicit in late capitalism so let's talk about another Space Invaders satire instead)

Some of these questions jump high barriers. Swartz is writing libraries for Python, Winer is working in Manilla, I'm building products in C++. I can't actually use Swartz's code, but I can use the ideas. Winer's problems aren't mine -- but, if I can understand where Winer is getting stuck, maybe I can avoid the muddy place when I get there. But we're not wrangling (today) over whether Python is better or worse than C++ or Squeak or Manilla. We're building together. (Another contrast with the humanities, alas, where people can't bother to leap barriers -- like the fence between interactive fiction and literary hypertext -- that are so low as to be almost invisible to outsiders)