dsandler.org/wp

Archive for November 14th, 2004

Hard RSS numbers from Boing Boing.

Sunday, November 14th, 2004

I had previously scraped my RSS numbers together by extrapolating from Bloglines subscriber data, but today’s Boing Boing article on RSS bandwidth includes a helpful link to Boing Boing’s own Web stats page.

File type	Hits	Percent	Bandwidth	Percent
xml1	2230472	11.1 %	46.36 GB	22 %
html	1924753	9.6 %	43.47 GB	20.6 %
rdf2	225715	1.1 %	4.11 GB	1.9 %

¹ from /rss.xml and /atom.xml

² from /index.rdf

That’s right—they’ve served 50 GB of RSS/Atom data so far this month (somewhere in the vicinity of 3 GB/day, since we’re but halfway through November). Straight HTML only accounts for 44 GB over the same period, which is surprising: RSS readers are pulling down more bits of text and markup than web browsers.

I had previously conjectured (based on a rather larger RSS feed size, 40 KB; the logs say the average is more like 20 KB) that Boing Boing serves 22 GB/day (40 K × 11,500 Bloglines subscribers × 48 requests per day), so my estimate was perhaps a little high.

In the most conservative case, this means that Boing Boing has (3.6 GB·day^-1) / (48 users·polls·day^-1) / (20 KB·poll^-1) = 3,750 unique RSS readers. Many clients don’t poll every half-hour or all day long, so there are probably quite a few more. What this tells me is that bandwidth problems are real, and we can expect them to get worse as more users discover RSS (to wit: yet more mainstream press, this time in Sunday’s TIME Magazine Europe).

Judging by these stats (and Glenn Fleishman’s figures), it’s clearly still important that we repair the distribution architecture of RSS. Even with the best-behaved clients, the growing user population spells DOOM for polled RSS.

[Aside: Glenn's graph shows a beautiful weekly heartbeat in RSS bandwidth; he attributes this to well-behaved readers cooling off over the weekend when his XML feed is completely static. I think that's part of it, but I'd also be willing to bet that some of this can be attributed to clients being switched off over the weekend. Rodrigo Rodrigues had some great p2p membership graphs in his IRIS workshop talk that showed many hosts disconnecting for about two days—Saturday and Sunday.]

[Note: This article was updated as of about 21:15 to reflect a closer reading of BB's stats page.]

Posted in Notebook | Comments Off

• Archives

  Select Month November 2008 October 2008 September 2008 June 2008 May 2008 April 2008 March 2008 February 2008 January 2008 December 2007 November 2007 October 2007 September 2007 August 2007 July 2007 June 2007 May 2007 April 2007 February 2007 January 2007 December 2006 November 2006 October 2006 September 2006 August 2006 July 2006 June 2006 May 2006 April 2006 March 2006 February 2006 January 2006 December 2005 November 2005 October 2005 September 2005 August 2005 July 2005 June 2005 May 2005 April 2005 March 2005 February 2005 January 2005 December 2004 November 2004 October 2004 September 2004 August 2004 July 2004 June 2004 May 2004 April 2004 March 2004 February 2004 January 2004 December 2003 November 2003 October 2003 September 2003 August 2003 July 2003 June 2003 May 2003 April 2003 March 2003 February 2003 January 2003 December 2002 November 2002 October 2002 September 2002 August 2002 July 2002 June 2002 May 2002 April 2002 March 2002 February 2002 January 2002 December 2001 November 2001 October 2001 September 2001 August 2001 July 2001

• twitter

  • dsandler: Why am I seeing Kabel everywhere all of a sudden? Is it 1986 and nobody told me? Dec 03 2008
  • dsandler: Why is @-reply spam not more prevalent on Twitter, given that people frequently check for replies from strangers? Dec 02 2008
  • dsandler: Baby formula with iron tastes like blood. Baby is currently trying to suck on my neck. COINCIDENCE? DOUBTFUL Dec 02 2008
  • dsandler: @gruber Much has been made of Google's CalDAV support, but until iCal syncs them r/w with iPhone it's of limited use. Dec 02 2008