waving android

I am currently a software engineer at Google, where as a member of the Android platform team I build frameworks and user interfaces.

The blog here at is mostly historical; you can find more recent posts on .

Down for everyone, but not me.

November 18th, 2008

A quick note about my Twitter experience this morning:

I went to post a remark about this article from yesterday’s New York Times on the computer science gender gap (particularly how this graph shows how the tech boom utterly failed to inspire young women) and found that Twitter was down:

Actually, that’s a lie; at first, I didn’t notice anything, because this is what I see when I use Twitter:

This is a screenshot of Birdfeeder, a prototype implementation of a distributed and secure microblogging protocol called FETHR, which operates independently of (but connects to) Twitter. I posted my comment to a personal instance of Birdfeeder, which happily accepted it, digitally signed it, entangled it with other messages in my timeline, and forwarded it on to my FETHR subscribers—one of which is a Twitter gateway, which takes care of forwarding my messages on to my Twitterstream. It’s also responsible for sending me messages from people I follow, so (as you can see from the screenshot) I never have to leave my Birdfeeder interface—and therefore can occasionally miss minor Twitter glitches. (Or, as in this case, major hour-long outages. Nothing yet on the status blog, either, so it must really be all-hands-on-deck over at Twitter HQ. Update: Over an hour in, there’s a small note on the status blog.)

When Twitter eventually comes back up, the gateway will busy itself with the task of forwarding along my queued messages and fetching news from my Twitter subscriptions. I can continue to tweet in the meantime and even page back through my entire history and the archived messages of my friends. This is the fundamental benefit of decentralized micropublishing: independent providers may experience local failures without bringing the whole damn thing to a screeching halt.

So, in something of a twist on the usual, Twitter is down for everyone—but not me.

This is fun and exciting new research (currently under submission) and I hope to push out a technical report version soon so that others may begin to evaluate and improve the system design. Hit me at dsandler@ (or @dsandler, once Twitter wakes up again) if you can’t wait and would like to know more.

7 responses

  1. dsandler  

    Update: Today’s outage identified a new failure mode that I wasn’t properly catching in the twittergw. During the outage, Twitter responded 200 OK but with an error embedded in the JSON:

    {'error': 'Twitter is down for database maintenance. It will return in about an hour.'}

    It’s implied in the API docs that twitter.com should have returned something other than 200 in this case; as it turns out, the twittergw code looked no further than the 200 and did not (as I hoped it would) batch my tweets for re-posting once the service returned.

    Bug fixed.

    comment posted at 1:17 pm on 18 Nov 2008

  2. Dave Peck  

    Dan — is this built on top of the Pastry/Scribe work that FeedTree was originally built on top of?

    comment posted at 1:25 pm on 18 Nov 2008

  3. dsandler  

    @Dave No, it’s a totally new framework based on RESTful HTTP requests between servers. (FETHR := “Featherweight Entangled Timelines via HTTP Requests.”)

    A FETHR server can be run from a vanilla webhost, or at home, or anywhere you can be reached on port 80. Servers interact by gossiping updates via POST.

    I really need to get that TR out.

    comment posted at 1:55 pm on 18 Nov 2008

  4. Matt  

    Shouldn’t we be concerned that on some level we’re doomed to re-implement SMTP.

    comment posted at 4:18 pm on 24 Nov 2008

  5. dsandler  

    This design shares the spirit of SMTP: It is decentralized and federated by design. In fact, you could probably hack together a passable micropublishing implementation based entirely upon email. But email is easily abused; the default-accept nature of SMTP combined with its lack of mandatory security features is what makes spam possible. FETHR an opt-in subscription model and security features to make this work in a decentralized environment. The design is tuned for the µpublishing/µblogging paradigm, however; I’m not simply trying to design a “better email.” That’s a problem for another day. :)

    comment posted at 9:54 am on 25 Nov 2008

  6. Odin/Velmont  

    Huh. Didn’t OpenMicroBlogging and Laconica exist back then? :-)


    Why another federated microblogging-protocol? They are trying to make a 0.2, adressing problems with 0.1 — so please say if there is something to be improved upon.

    comment posted at 11:27 am on 22 Apr 2009

  7. FETHR roadmap. (dsandler.org)  

    [...] twittergw, which acts as a gateway between the two networks. (I first described the gateway in a previous blog post.) Long-term, Twitter should support the FETHR API directly; twittergw was created to provide a [...]

    comment posted at 8:15 am on 01 May 2009

Add a comment

html help (show)

newer: older: