I’ve gotten some really excellent feedback about FETHR since I unveiled it at IPTPS last week. There’s been a steady hum of RTs and hosannahs on Twitter, a handful of thoughtful emails, and a few FriendFeed discussions (notably Chris Messina and Andy Baio1).
The FETHR slide deck has been particularly well-received. In it I compare microblogging in 2009 to email in 1983, an analogy which seems to resonate with people (at least, people of a certain age); I think it succinctly summarizes where we are today and what needs to happen in order for microblogging to become, in fact, a communication utility alongside email, IM, blogs, and so on.
FETHR, Laconica, and OpenMicroBlogging
I’ve gotten a lot of useful criticism and feedback as well. Several people asked whether I was aware of the open-source Laconica project and how what I’m doing differs. In short: yes, I’m aware of Laconica and OpenMicroBlogging (OMB). I started my work in April ’08, and when Laconica launched in July I was gratified to see someone else pursuing open microblogging.
I think FETHR and OMB are cousins. Evan Prodromou, the creator of Laconica and the owner of identi.ca (a Twitter-like multi-user microblogging service) and I both see the same need: for an ecosystem of microblogging systems that seamlessly2 interoperate. Each is a RESTful protocol designed to be used exclusively over HTTP; participants are uniquely identified by a canonical URL, which is used as a rendezvous point for API calls.
Substantial technical differences exist between the two protocols, however:3
Message distribution. A lesson I took away from my MS work on RSS feeds is that being popular is a curse: you have to satisfy all those hungry new readers. Unlike RSS, FETHR (and OMB) are push protocols, so those readers aren’t periodically making (redundant, useless) requests, but a popular microblogger (e.g. Heather Champ) might still have to make half a million HTTP requests every time she wants to post a message. FETHR addresses this problem by allowing a publisher to ask subscribers to assist with message dissemination by gossiping updates among one another.
Security. Each applies security techniques to prevent abuse, particularly spam (messages from sources the user isn’t subscribed to). OMB uses OAuth4 to secure individual connections, but this won’t fly in a true p2p environment like FETHR (in which you might receive a new message from someone other than its author). FETHR therefore secures data rather than channels; that is, individual messages are signed and hash chained in order to authenticate messages and a publisher’s timeline as a whole. (There are some other useful properties of this approach; see §3.2 of the IPTPS paper for details.)
Message content. Finally, as currently written, OMB is naturally focused on replicating the Twitter experience, including some maximum field lengths set at 140 characters (e.g. the user’s “location”) and specifying the size (96x96 pixels) of avatar images. It expands on Twitter in certain areas (including a sorely-needed “seealso” property to allow a message to refer to some external resource), but these too are fixed in the protocol. FETHR attempts to remain agnostic on these data, choosing instead to specify a bare minimum of necessary properties and allowing applications to superimpose arbitrary key-value data (read: a JSON object). Essentially, FETHR anticipates that application developers will create their own micro-formats5 that best describe the type of micropublishing service model that best suits them.
There’s lots of work to do on FETHR, and now that I’ve gotten another large blocking project out of the way, I can resume work on both the protocol and the Birdfeeder prototype. In rough priority order:
Protocol docs. Many (including Dave Winer in a comment on Messina’s FriendFeed) have asked for a public specification of the FETHR protocol. I absolutely intend to provide this, although the exact wire protocol (as spoken by Birdfeeder) is something of a moving target at the moment. If you’re desperate, you can suss it out from the source, which brings me to…
Refactoring. There’s a lot of work to do on the code; I grew it organically and unpredictably as my ideas about FETHR developed, and as a result, parts of it bear a strong resemblance to a pasta dinner. Aside from general cleanups (more comprehensive tests, documentation, etc.) I’d like to better separate the Birdfeeder front-end from the FETHR transport so that it’s easier to develop new applications without all the HTML.
Local Twitter support in Birdfeeder. Currently, Twitter connectivity is handled by a single FETHR node, twittergw, which acts as a gateway between the two networks. (I first described the gateway in a previous blog post.) Long-term, Twitter should support the FETHR API directly; twittergw was created to provide a crude replica of such functionality. I’m running up against the limitations of this approach, however; I now plan to move Twitter support into the Birdfeeder client itself, overcoming the limitations of twittergw as well as facilitate the out-of-box experience for people looking to try out the system.
More refactoring. Birdfeeder makes use of the web.py application framework, but it currently uses a number of work threads, which requires that web.py be used as a standalone server. I want Birdfeeder to operate as a stateless CGI, which means moving background tasks out of threads and into special URL handlers (suitable for tickling with a cron job). This would pave the way for…
Google App Engine. The above improvements will allow Birdfeeder to run on GAE, which is a big step toward a large multi-user system that non-technical users can sign up (similar to identi.ca). Note, however, that the point of this project is not to create a public microblogging service, but to lay the foundation for many public micropublishing services; still, it can be beneficial to have a public example as an “ambassador” for the technology.
This is just the “development” side of the R&D equation. I’ve got a stack of research topics to tackle as well (some of which can be found toward the end of my slide deck); I expect to pick these threads back up a little later in the summer.
Thoughts? Requests? I’ll leave the regular blog comments open so you can write longer notes than the Twitter watercooler will allow (although tweets are welcome as well).
Well, almost seamlessly. In particular, the user experience for subscribing to a user becomes much more complex: you can’t simply create a “Follow!” button, because the subscriber may be using a different service. This is an open problem, related to that of RSS subscription, with similar solutions (copy/pasting addresses, bookmarklets, or new URI schemes). ↩
This discussion is based on my reading of the current draft of the OpenMicroBlogging specification; there are portions of the document that are ambiguous and some important terms (such as “user”) are ill-defined. If you think I’ve misunderstood something, please leave a note below. ↩
OAuth was designed to allow a user to grant a limited amount of authority from one service (of which he is a user) to another (which he also uses). For example, Twitter might want to look through your Gmail contacts to see if any of them is already a Twitter user; it could do this by asking for your Gmail username and password, but this would give Twitter access to all your email as well (which is not good). Instead, Twitter makes an OAuth request (on your behalf) to Gmail, which then asks you (interactively) if this is in fact what you want before handing a special authorization token back to Twitter for this specific purpose only (think of it as a “valet key”). Because it’s essentially a capability system for Web resources, OAuth doesn’t have to be used this way, and OMB chooses to use it as a way to allow one user (the subscriber) to delegate a right (the ability to send him messages) to another user (the publisher). ↩