BBC iPlayer RSS Feeds - 4 million / day
When the BBC decided to turn off its iPlayer RSS feeds with no up front warning, I was annoyed, really annoyed, which isn’t like me.
Why was I annoyed? I think it was the total arrogance of the “Yeah sorry you used to use them, we don’t do that anymore, try this new thing”, “but the new things not ready yet?”, “Yeah sorry about that”.
In short a fairly sizable group of people used these RSS feeds either directly or indirectly via 3rd party tools to access iPlayer content and it was the only reliable and open data method for getting information about iPlayer content.
I was interested, other than me, who else used these RSS feeds, so I made a freedom of information (FOI) request to the BBC and asked them. It turns out for the 30 days prior to them being shutdown they were accessed 4,000,000 (yes 4 MILLION) times a day!
The response from the BBC goes on to state that they suspect that a fair number of the downloads where from bots, yep probably, but those bots where doing so to deliver services to people and it could be that 1 bot download serviced many people.
The moral of the story is that you can’t expect services to be provided forever regardless of who is providing them and how many people use them or assume that there will be a replacement.
Below is my request and subsequent response from the BBC.
Subject: FOI Request - BBC iPlayer RSS Feeds Usage
From: “Martin O’Hanlon”
To: foi@bbc.co.uk
Dear Sir,
I am requesting information under the Freedom of Information act about the
usage of BBC iPlayer RSS feeds.
On (or around) the 29th October, access to the BBC iPlayer RSS Feeds was
removed (http://iplayerhelp.external.bbc.co.uk/tv/feeds).
Could you provide information relating to the use of the RSS Feeds prior to
them being removed.
By month, for the 12 months previous could you state the number of access
requests for the RSS feeds supplied under http://feeds.bbc.co.uk/iplayer/.
If you are unable to provide 12 months of data by month, could you provide
the information for the last 30 days.
Kind regards
Martin O’Hanlon
From: FOI Enquiries <FOIEnquiries@bbc.co.uk>
To: “‘Martin O’Hanlon’”
Subject: Response to your request for information - RFI20141782
Date: Fri, 28 Nov 2014 14:49:53 +0000
Dear Mr O’Hanlon
Please find attached the response to your request for information, referenc=
e RFI20141782.
Kind regards
BBC Information Policy and Compliance
BC2 B6, 201 Wood Lane, London W12 7TP
You can download the attached from here, or view the contents below:
Martin O’Hanlon
Via email: martin@ohanlonweb.com
28th November 2014
Dear Martin,
Freedom of Information Act 2000 – RFI20141782
Thank you for your request under the Freedom of Information Act (‘the Act’) of 30th
October 2014, seeking:
“I am requesting information under the Freedom of Information act about the usage of BBC iPlayer
RSS feeds.
On (or around) the 29th October, access to the BBC iPlayer RSS Feeds was removed
(http://iplayerhelp.external.bbc.co.uk/tv/feeds).
Could you provide information relating to the use of the RSS Feeds prior to them being removed.
By month, for the 12 months previous could you state the number of access requests for the RSS
feeds supplied under http://feeds.bbc.co.uk/iplayer/.
If you are unable to provide 12 months of data by month, could you provide the information for the
last 30 days.”
I estimate that to deal with your request for 12 months’ worth of data, given the amount of
data this involves, would take more than two and a half days; under section 12 of the Act, we
are allowed to refuse to handle the request if it would exceed the appropriate limit. The
appropriate limit has been set by the Regulations (SI 2004/3244) as being £450 (equivalent to
two and a half days work, at an hourly rate of £25).
With regard to the 30 days’ worth information, the following is the total number of requests
for urls under http://feeds.bbc.co.uk/iplayer which relates to iPlayer RSS feeds (date followed
by the total):
28/09/2014 - 4139102
29/09/2014 - 3969429
30/09/2014 - 3688324
01/10/2014 - 3884073
02/10/2014 - 3883081
03/10/2014 - 3925662
04/10/2014 - 4159130
05/10/2014 - 4222500
06/10/2014 - 4131642
07/10/2014 - 3985815
08/10/2014 - 4012385
09/10/2014 - 3915717
10/10/2014 - 3882014
11/10/2014 - 4048638
12/10/2014 - 4208782
13/10/2014 - 4116161
14/10/2014 - 3970894
15/10/2014 - 4140721
16/10/2014 - 3840215
17/10/2014 - 3929542
18/10/2014 - 4183741
19/10/2014 - 4298197
20/10/2014 - 4059930
21/10/2014 - 4046123
22/10/2014 - 3941945
23/10/2014 - 3922209
24/10/2014 - 3962244
25/10/2014 - 4070949
26/10/2014 - 4307425
27/10/2014 - 4074942
28/10/2014 - 4050387
Please note that the data above does not give any indication of what was making the request
i.e. whether human or bot (a software application which runs automated tasks over the
internet). The rather flat level probably indicates that there was a very large amount of
automated processing going on, so this should not be confused with actual users doing
something with the information in the feeds.
You may request an internal review of our decision that your request exceeds the
appropriate limit. Please contact us at the address above, explaining what you would like us to
review and including your reference number. If you are not satisfied with the internal review,
you can appeal to the Information Commissioner. The contact details are: Information
Commissioner’s Office, Wycliffe House, Water Lane, Wilmslow, Cheshire, SK9 5AF, Tel:
0303 123 1113 (local rate) or 01625 545 745 (national rate) or see http://www.ico.gov.uk/
Yours sincerely,
Kate Leece
Head of Legal and Business Affairs
BBC Future Media