dsandler.org » search http://dsandler.org/wp a beautiful blog by daniel sandler Tue, 07 Sep 2010 22:43:39 +0000 http://wordpress.org/?v=2.9.2 en hourly 1 Update on the Twitter comments experiment http://dsandler.org/wp/archives/2009/03/15/twitter-comments-results http://dsandler.org/wp/archives/2009/03/15/twitter-comments-results#comments Sun, 15 Mar 2009 19:00:57 +0000 Daniel Sandler http://dsandler.org/wp/?p=21341

A couple of weeks ago I explored the idea of using Twitter as the primary forum for blog discussion. Multiple-round conversation is hard to sustain on a personal blog, but seems to appear effortlessly on Twitter, so went the thinking. Many people responded on Twitter (and elsewhere) with their thoughts, both on the general idea and the particular implementation; they were captured and displayed along with the original post using some client-side JavaScript called watercooler.js that I built for the experiment.

Below, I’ll share some of the points raised by those responses, along with some observations and data of my own.

Discussion, I haz it

155 Twitter messages matched either the URL, TinyURL, or TinyURL path-component of the original post as of Friday the 6th of March, making it by far the most-discussed article ever on this site. For comparison, the closest competitor (using conventional blog comments) is my 2006 gripe about the iTunes 7 user interface, with 86 comments. Of those 86, 49 were actual comments (not trackbacks or pingbacks), of which 12 were mine. Five readers posted more than once—twice, in fact, in every case—and the remaining 43 comments were hand grenades (that is: fire and forget).

First: this is a tremendous level of interest for little old dsandler.org. I don’t have clear numbers about the readership of my blog—let’s put it at around 4001—but it’s on the same order of magnitude as my Twitter subscribership.2 So we can say that the post had roughly the same audience on Twitter that it would have had otherwise, and yet, it received twice as many responses as the reigning champ on the blog.3

I believe that the ease of responding via Twitter, rather than

  1. clicking through to the blog (if reading it via RSS)
  2. finding the comments form
  3. entering a bunch of orthogonal information (your email address, name, some sort of anti-spam challenge)

before finally being able to enter a comment, was indeed conducive to discussion. Additionaly, Twitter facilitates the second hop (that is, reaching readers of my readers) thanks to the ease with which messages may be forwarded along (retweeted in the lingo).

Digging into the data

One of the concerns raised by several commentators was that, while potentially valuable for increasing the reach of a blog post, these RTs would clog the discussion. Being an ABD doctoral student in computer science, my conditioned response is to disprove this hypothesis with a graph:


Fig. 1.   Classification of Twitter replies over time.

See that big spike (Figure 1, at T+24 hours)? That’s when John Gruber mentioned the experiment on Daring Fireball, which proliferated the idea much more widely than my own immediate circle of attentive contacts (the spike at T=0). In fact, the orange retweet bar makes its first (and nearly only) appearance at T+24; in all, retweets made up only 3% (51 messages) of the weeklong discussion. @replies, a robust indicator of discussion rather than simply undirected opinion, represented 39% of the traffic.

So who was doing all this talking? Well, I was certainly responsible for a lot of it (more than a quarter), particularly at the beginning, where I was responding individually to responses from my friends. As mentions of the post broadened, there were fewer questions, so I had fewer responses (see Fig. 2).


Fig. 2.   Author (28%) vs. audience (72%).

Much of this was multiple-round discussion; 13 users had two tweets, several had 3, and some offered as many as 6 or 7 responses.

Which way to the forum?

Beyond simply sparking conversation (easy enough to do with a single Twitter message), I wanted to be able to discover that conversation and point to it from the blog. It’s not a discussion if you can’t join in, and without a way to link all the relevant tweets together, there’s no way to collect them on the blog for easy perusal (which would be no better than having no comments facility at all).

In short, this technique needs a rendezvous point. The most obvious Twitter-compatible approach is to require respondents to embed in their messages a unique string that I could send to search.twitter.com to collect all the relevant messages at any time. I chose the TinyURL of the original post (later simplified to just the path part, d2nrsv) for two reasons:

  1. Anyone making mention of the blog post in a Twitter message but oblivious to the conversation protocol would still have his message included in the conversation.
  2. It’s unique across any site using the same technique.

So did I actually pick up any random messages (case #1 above)? Figure 3 tells the story:


Fig. 3.   TinyURL: fragments vs. full URLs.

Indeed, there were plenty of messages that included the full TinyURL. Those that did not either inferred the protocol from the messages of others, or used the “click here to comment” link at the end of the blog post (which used the Twitter ?status= query string in order to pre-populate the input box with the “d2nrsv” fragment, like so).

Guessing what the “d2nrsv” meant, however, wasn’t easy; in general, this approach suffers badly from a lack of self-documentation. It might have been better to satisfy goal #2 (globally unique) by generating a symbol of the form #dsandlerdotorg_twittercomments4 that has some hope of being deciphered. A related approach might be to create a Twitter account for the blog and require discussion of the form “@dsandlerdotorg twittercomments Nice idea!”—a two-part hierarchical conversation identifier.5

It’s also worth noting that relying on TinyURL introduces fragility; such an external dependency can go down or change its behavior at any time, with huge impact to this technique. (Twitter has the same fate-sharing problem with TinyURL, which is why I’ve advocated for some time that Twitter needs to run its own URL-shortening service.)

Perhaps the best approach, as in so many situations, is to be liberal in what is accepted: search for (1) the TinyURL, (2) some sort-of-unique keyword referring to the post, possibly (3) directed as an @reply to a Twitter account for the blog. Maybe the unadorned @replies to the blog’s Twitter account (i.e., those without other identifying tokens) could be automatically associated with the latest prior post at that time. I’ll probably try something along these lines in my next revision of the watercooler code.

Other important considerations

All this hassling over exactly which nonce to include in responses avoids the very real problem (again, pointed out by many) that to include a tag burns precious bytes, of which Twitter message text may only use 140. Of course, there’s plenty of metadata outside those 140 bytes that is nevertheless associated with each tweet (such as time, author, status ID of an antecedent message, and so forth), so we can imagine that an in_reply_to_url datum might not be an unreasonable addition to the overall structure. 6

A number of people mentioned that sharing fate with Twitter search is also potentially problematic. It might not survive as long as your blog, or its API might change (forcing you to upgrade your blog to recover the comments), or it might start returning inconsistent or undesirable data (for example, truncating result sets during peak load). More generally, there was a general sense (with which I agree) that while Twitter might be a really interesting way to discuss something, if you want to collect that discussion in one place, you really ought to be copying it there. This has the unfortunate side-effect of obviating the cleverness of the watercooler.js implementation, which is entirely client-side and therefore puts plenty of load on Twitter but none on the original author’s blog. A future implementation should create a permanent local copy of all relevant public tweets, so that they persist as long as the antecedent article does.7

Once more unto the data, my friends

Finally, if you’d like to see what you can make of all this, I’ll save you the trouble of fetching the data from the Twitter Search API: here is JSON for the tweets used to produce the graphs above. If you see any interesting trends in there that I didn’t catch, please share it with the class; as before, include in your message the TinyURL path, dc946u or full URL.

  1. Counting feed subscribers is known to be an inexact science (unless you outsource all your feeds). Yesterday there were 2940 individual requests to the various WordPress feed URLs, so assuming the conventional half-hourly refresh rate, that’s a little over 60 individual fetchers; one of those is Google, which tells me in my logs that I have about 120 subscribers there; similarly, Bloglines adds another 60, Yahoo another 160. So, all told, I figure I’ve got around 400 readers out in RSS-land. ↩

  2. Currently about 500 followers, which includes inactive accounts, spammers, and so on. ↩

  3. This isn’t apples-to-apples, of course: there are too many confounding factors here (RSS subscribership increase; selection bias of readers; size of excerpt seen by potential commenters) to consider the 2006 and 2009 posts as control and experiment. I’m merely pointing out, qualitatively, that this is the biggest response I’ve ever seen, by a factor of two. One environmental similarity between the two posts: they were each Fireballed (2006, 2009). ↩

  4. Many suggested creating a unique hashtag, which has the benefit of being presented in many Twitter frontends as a quick search for that term. I’m mixed on this approach, mostly because I think hashtags are nerds-only tools that won’t really take hold as Twitter continues to grow past the early-adopter phase. #provemewrongfolks ↩

  5. Several people even proposed creating a whole new Twitter account for each post, such that @dsandlerdotorg_twittercomments would be both unique (enforced by Twitter) and fully scoped to the post. This seems like an abuse of Twitter, but a more general micropublishing system that allowed third parties to arbitrarily create new endpoints might be able to handle this easily. Worth coming back to. ↩

  6. In fact, some proposed using the existing in_reply_to_status_id field to tie the conversation together, and indeed, this would be done automatically by many clients if I asked people to respond to a single tweet instead of including a token in their message text. Unfortunately, this wreaks havoc with messages that are in fact more directly tied to some other message—particularly a more recent tweet in the thread. I think that both pointers, ultimately, are required: one for the conversation to which a message belongs, and one for the messages (not necessarily just one) to which a message is a direct response. ↩

  7. I did this by hand on March 5, copying the generated HTML into the end of my original post, and in so doing I truncated the “official” record of the discussion at that time. ↩

]]> http://dsandler.org/wp/archives/2009/03/15/twitter-comments-results/feed 0 Experimental: Twitter for comments http://dsandler.org/wp/archives/2009/02/26/twitter-comments http://dsandler.org/wp/archives/2009/02/26/twitter-comments#comments Thu, 26 Feb 2009 19:58:39 +0000 Daniel Sandler http://dsandler.org/wp/?p=21304 Abstract

Blog comments don’t really encourage robust discussion. (The only people who look twice at the comments are the original author and readers with an axe to grind.) But ad hoc multi-party discussion does happen on Twitter. I’m experimenting with promoting Twitter to a first-class blog comments system.

No Comment

I’m not the first to notice this, but blog comments are awkward. Many excellent blogs have eschewed them for various reasons; Jason Kottke doesn’t want the babysitting chore of dealing with a large, vocal community. Alex Payne doesn’t think good conversations emerge from blog comments because they encourage “hit-and-run attacks, unintelligible ramblings, and truckloads of spam.” John Gruber simply prefers a streamlined reading environment.1 It’s not that they’re uninterested in discussion; rather, they’d prefer it happen elsewhere:

  1. On your own time and property (viz., on your blog), or
  2. In some neutral forum (Reddit, Twitter, etc.).

Option #1—the promise of a blogosphere criscrossed with scintillating conversation—is the reason TrackBack exists: without it, the author and readers alike have no way of knowing where on the web to find the next response. Unfortunately, because it allows untrusted parties to post links and content to your blog automatically, TrackBack is an enormous spam magnet and has therefore fallen into disuse.2 A number of ad-hoc techniques replace it, exemplified by the convention of promulgating choice responses (received out-of-band, e.g. by email or IM) by appending them, simply, under the heading UPDATE.

A few conscientious bloggers—even those that, nominally, want you to Get Your Comments Off My Lawn—carefully comb their referrer logs for links to well-considered responses from other bloggers, which they will then compile into a “Responses” post. Twitter hacker Alex Payne, for example, recently rounded up the rebuttals to his “ranty”3 article of a few weeks prior. It is his claim that, because each of those bloggers chose to respond on his own blog—taking full responsibility for his opinions under his own masthead and in front of his own readers, rather than leaving a comment “off the record” on a distant site—those perspectives were more thoughtful and more clearly articulated.

This leaves us with option #2: Comments in the Neutral Zone. Assuming you have a high-profile discussion-free blog4 and also read Reddit, this works pretty well: your adherents and detractors have a sandbox in which to have it out, and you can drop in occasionally to poke the discussion. Unfortunately, the stars must align for this technique to result in interesting conversation; the likelihood is that some essential piece of the conversation—either author or readers—is not looking at your shiny new Reddit thread.5 In fact, the only thing that ties the group together is the original blog post, which puts us right back where we started: blog discussion, it would seem, must occur—or at least appear—on the blog itself.

But this is still insufficient, because while the ideal discussion group for a given blog is defined as a blog’s readership, we have only solved the problem in space, not in time. How often do you check the same blog post to see if there are exciting new comments? As a reader, I hardly have the time to leave a single comment, let alone check back to see if there are interesting responses. I find this to be true of readers of dsandler.org as well: it’s fruitless for me to respond to your comment with one of my own, because you’re unlikely ever to visit this post again to read it.

This is what I think of when Alex Payne refers to “hit-and-run attacks”—a natural side-effect of the fact that most readers will look at a blog post (and the current state of the discussion) at most once. The only people who look twice at the same blog entry’s comments are the author (who has the luxury—or compulsion—to respond to each and every one) and anyone with an axe to grind. All others, if they comment at all, will leave a single comment and then never return. This is not conducive to discussion.

And yet, discussion happens

I’m increasingly getting feedback about blog posts on Twitter (with good reason: I mention the post on Twitter myself, explicitly inviting a conversation). For example, when I finally deployed the recent redesign of dsandler.org late last year, I mentioned it on Twitter (where a few people had already been offering complaints about my intentionally awful interim design). I received a few comments on the blog post itself, but just as many via Twitter, so I decided to (laboriously!) copy some of those messages into the text of the post (see UPDATE).

Wouldn’t it be nice if such a thing were done automatically? For bloggers whose controversial articles (picking on @al3x again here) spawn a lot of Twitter discussion, could we actually promote Twitter to a first-class discussion mechanism?

See below

Yesterday I threw together some code to go search Twitter for any messages including the URL of the current page and display them as if they were comments. I had considered creating a unique keyword for each post on the fly (possibly using a hashtag, like “#dsandler_post_5″) that would string the conversation together, but the URL seems more natural (because people will put that into responses anyway, and besides, the TinyURL version is usually every bit as short as a hashtag).

Blindly including the results of a search on your blog is an easy way to get spammed, of course,6 so we’ll have to see how this goes. Note, however, that because the tweets are fetched by the client, they never actually appear in the blog when it’s crawled by search engines, so you can’t mess with my PageRank (or boost your own) by abusing this system—all you can do is cause grief. (So. Don’t.)

So, let the experiment begin: follow the link below to pre-populate a Twitter post about this entry (or simply include the URL or TinyURL in your message). Try to include the (Tiny)URL in any tweets that have to do with this post, insofar as that’s practical.

Oh, and: regular blog comments are turned off.

Update

5 March: I’m working on a new post to document the results of this little experiment. For now, I’m turning off the live Twitter search results and ossifying the state of the conversation as it stands today by including the results statically below.

The Twitter watercooler

Messages referencing this page as of March 5th (newest at bottom):

  1. This is a nice way of saying “Daring Fireball is intended to be The John Gruber Show—Starring John Gruber—A John Gruber Production—Filmed In Grubervision,” a sentiment with which I think John would agree. ↩

  2. I did some work on this problem early in my PhD program, upping the ante somewhat for TrackBack spammers, but blogs have mostly moved on. ↩

  3. “Ranty” via Adam Rice via Alex’s recap. (Look, it’s all intertwingled!) ↩

  4. A webcomic, particularly one that inspires conversation, is isomorphic to a blog in this regard. ↩

  5. Sometimes an eager reader is so helpful as to [volunteer for the position of Comments Czar, though unless the original author sanctions and links to this outsourced comments system, any discussion that takes place there will be doomed to obscurity. ↩

  6. You may recall that election.twitter.com was very easy to abuse as it showed, live, any tweet that included election keywords. ↩

]]> http://dsandler.org/wp/archives/2009/02/26/twitter-comments/feed 9