FETHR roadmap.
I’ve gotten some really excellent feedback about [FETHR](http://brdfdr.com) since I [unveiled it at IPTPS](http://www.usenix.org/events/iptps09/tech/) last week. There’s been a steady hum of RTs and hosannahs [on Twitter](http://search.twitter.com/search?q=fethr), a handful of thoughtful emails, and a few FriendFeed discussions (notably [Chris Messina](http://friendfeed.com/chrismessina/9015ec9b/brdfdr-birdfeeder) and [Andy Baio](http://friendfeed.com/waxpancake/a694b787/birdfeeder-prototype-client-for-open)[^BAIO]).
[^BAIO]: I owe particular thanks to Andy for giving the FETHR talk a huge bump by noting it on his widely-read [links](http://waxy.org/links/) page for April 21st.
The [FETHR slide deck](http://brdfdr.com/pres) has been particularly well-received. In it I compare microblogging in 2009 to email in 1983, an analogy which seems to resonate with people (at least, people of a certain age); I think it succinctly summarizes where we are today and what needs to happen in order for microblogging to become, in fact, a [communication utility](http://www.businessweek.com/technology/content/may2008/tc20080514_269697.htm) alongside email, IM, blogs, and so on.
### FETHR, Laconica, and OpenMicroBlogging
I’ve gotten a lot of useful criticism and feedback as well. Several people asked whether I was aware of the open-source [Laconica](http://laconi.ca) project and how what I’m doing differs. In short: yes, I’m aware of Laconica and OpenMicroBlogging (OMB). I started my work in April ’08, and when Laconica launched in July I was gratified to see someone else pursuing open microblogging.
I think FETHR and OMB are cousins. Evan Prodromou, the creator of Laconica and the owner of [identi.ca](http://identi.ca) (a Twitter-like multi-user microblogging service) and I both see the same need: for an *ecosystem* of microblogging systems that seamlessly[^SEAMLESS] interoperate. Each is a [RESTful](http://en.wikipedia.org/wiki/Representational_State_Transfer) protocol designed to be used exclusively over HTTP; participants are uniquely identified by a canonical URL, which is used as a rendezvous point for API calls.
[^SEAMLESS]: Well, _almost_ seamlessly. In particular, the user experience for subscribing to a user becomes much more complex: you can’t simply create a “Follow!??? button, because the subscriber may be using a different service. This is an open problem, related to that of RSS subscription, with similar solutions (copy/pasting addresses, [bookmarklets](http://googlereader.blogspot.com/2005/11/subscribing-to-feeds-via-little-google.html), or [new URI schemes](http://en.wikipedia.org/wiki/Feed_protocol)).
Substantial technical differences exist between the two protocols, however:[^OMB]
[^OMB]: This discussion is based on my reading of the current draft of [the OpenMicroBlogging specification](http://gitorious.org/projects/openmicroblogging/repos/mainline/blobs/master/openmicroblogging.txt); there are portions of the document that are ambiguous and some important terms (such as “user”) are ill-defined. If you think I’ve misunderstood something, please leave a note below.
* **Message distribution.** A lesson I took away from my MS work on RSS feeds is that being popular is a curse: you have to satisfy all those hungry new readers. Unlike RSS, FETHR (and OMB) are push protocols, so those readers aren’t periodically making (redundant, useless) requests, but a popular microblogger (e.g. Heather Champ) might still have to make half a million HTTP requests every time she wants to post a message. FETHR addresses this problem by allowing a publisher to ask subscribers to assist with message dissemination by gossiping updates among one another.
* **Security.** Each applies security techniques to prevent abuse, particularly spam (messages from sources the user isn’t subscribed to). OMB uses [OAuth](http://oauth.net)[^OAUTH] to secure individual connections, but this won’t fly in a true p2p environment like FETHR (in which you might receive a new message from someone other than its author). FETHR therefore secures _data_ rather than channels; that is, individual messages are signed and hash chained in order to authenticate messages and a publisher’s timeline as a whole. (There are some other useful properties of this approach; see [§3.2](http://brdfdr.com/doc/iptps-fethr/#tth_sEc3.4) of the [IPTPS paper](http://brdfdr.com/doc/iptps-fethr/#tth_sEc3.4) for details.)
* **Message content.** Finally, as currently written, OMB is naturally focused on replicating the Twitter experience, including some maximum field lengths set at 140 characters (e.g. the user’s “location”) and specifying the size (96×96 pixels) of avatar images. It expands on Twitter in certain areas (including a sorely-needed “seealso??? property to allow a message to refer to some external resource), but these too are fixed in the protocol. FETHR attempts to remain agnostic on these data, choosing instead to specify a bare minimum of necessary properties and allowing applications to superimpose arbitrary key-value data (read: a JSON object). Essentially, FETHR anticipates that application developers will create their own micro-formats[^MICRO] that best describe the type of _micropublishing_ service model that best suits them.
[^MICRO]: Not to be confused with [Microformats](http://microformats.org), which are (X)HTML-based metadata that allow a human-readable webpage to be seamlessly interpreted by “semantic Web” agents.
[^OAUTH]: OAuth was designed to allow a user to grant a limited amount of authority from one service (of which he is a user) to another (which he also uses). For example, Twitter might want to look through your Gmail contacts to see if any of them is already a Twitter user; it could do this by asking for your Gmail username and password, but this would give Twitter access to all your email as well (which is not good). Instead, Twitter makes an OAuth request (on your behalf) to Gmail, which then asks you (interactively) if this is in fact what you want before handing a special authorization token back to Twitter for this specific purpose only (think of it as a “valet key”). Because it’s essentially a [capability system](http://en.wikipedia.org/wiki/Capability_system) for Web resources, OAuth doesn’t have to be used this way, and OMB chooses to use it as a way to allow one user (the subscriber) to delegate a right (the ability to send him messages) to another user (the publisher).
### Birdfeeder roadmap
There’s lots of work to do on FETHR, and now that I’ve gotten another [large blocking project](http://dsandler.org/pub/thesis) out of the way, I can resume work on both the protocol and the Birdfeeder prototype. In rough priority order:
1. **Protocol docs.** Many (including Dave Winer in a [comment](http://friendfeed.com/chrismessina/9015ec9b/brdfdr-birdfeeder#c-f015cb0dd03a4ce9b1321d754b5eb438) on Messina’s FriendFeed) have asked for a public specification of the FETHR protocol. I absolutely intend to provide this, although the exact wire protocol (as spoken by Birdfeeder) is something of a moving target at the moment. If you’re desperate, you can suss it out from the [source](http://bitbucket.org/dsandler/brdfdr/), which brings me to…
2. **Refactoring.** There’s a lot of work to do on the code; I grew it organically and unpredictably as my ideas about FETHR developed, and as a result, parts of it bear a strong resemblance to a pasta dinner. Aside from general cleanups (more comprehensive tests, documentation, etc.) I’d like to better separate the Birdfeeder front-end from the FETHR transport so that it’s easier to develop new applications without all the HTML.
3. **Local Twitter support in Birdfeeder.** Currently, Twitter connectivity is handled by a single FETHR node, [twittergw](http://twittergw.brdfdr.com), which acts as a gateway between the two networks. (I first described the gateway in a [previous blog post](http://dsandler.org/wp/archives/2008/11/18/twitter-outage).) Long-term, Twitter should support the FETHR API directly; twittergw was created to provide a crude replica of such functionality. I’m running up against the limitations of this approach, however; I now plan to move Twitter support into the Birdfeeder client itself, overcoming the limitations of twittergw as well as facilitate the out-of-box experience for people looking to try out the system.
4. **More refactoring.** Birdfeeder makes use of the [web.py](http://webpy.org) application framework, but it currently uses a number of work threads, which requires that web.py be used as a standalone server. I want Birdfeeder to operate as a stateless CGI, which means moving background tasks out of threads and into special URL handlers (suitable for tickling with a cron job). This would pave the way for…
5. **Google App Engine.** The above improvements will allow Birdfeeder to run on GAE, which is a big step toward a large multi-user system that non-technical users can sign up (similar to identi.ca). Note, however, that the point of this project is _not_ to create a public microblogging service, but to lay the foundation for many public micropublishing services; still, it can be beneficial to have a public example as an “ambassador” for the technology.
### Research
This is just the “development” side of the R&D equation. I’ve got a stack of research topics to tackle as well (some of which can be found [toward the end](http://brdfdr.com/pres/slides/iptps.html#80) of my slide deck); I expect to pick these threads back up a little later in the summer.
Thoughts? Requests? I’ll leave the regular blog comments open so you can write longer notes than the [Twitter watercooler](http://dsandler.org/wp/archives/2009/02/26/twitter-comments) will allow (although tweets are welcome as well).