It's never nice to hear about the demise of a piece of simply brilliant software. when I discovered that get_iplayer was being pulled by its developer I was, to use a cliche, gutted. The potential loss of a piece of software that did just what it said on the tin is bad enough but it was impeccably free and open. What's more, it was an example to the BBC about how things should be done. It was the work of one lone, unpaid developer, not the product of professional developers subsidised by the BBC licence. What happened exemplifies everything that is wrong with proprietary software.
For the benefit on non-British readers I should explain that the BBC has an excellent website and it includes the iPlayer which allows visitors to view BBC audo and video content in their browser. When it was launched it was, surprise, Windows only. As many licence fee payers were also GNU/Linux users, they were enraged that they had effectively been excluded from the experience. BBC FUD ensued. Eventually though, lobbying and petitioning paid off and the BBC enabled the iPlayer for platforms other than Windows. You needed (and still need) (Adobe) Flash to view the video content and the content was encumbered with DRM and was not yours for keeps. It was only a thirty-day visitor to your hard drive.
The iPlayer did stuff that left the BBC's iPlayer eating its dust
The first breakthrough, at least for me, came with the discovery of Paul Battley's Ruby script. I installed it and it worked fine. I liked it so much that I interviewed Paul for Freesoftware Magazine. The preamble to the interview will explain the background. Checking back on his webpage to update the software (as he was playing a cat and mouse game with the BBC who kept changing their specifications). I chanced upon a reference to another iPlayer downloader. Like Paul's Ruby script it was free and open source and allowed the user to download BBC audio and video content without DRM or time restrictions. It worked beautifully, it was not a memory or CPU hog and it all played back beautifully in VLC without stutter. It came with a shed load of features and a manpage longer than War and Peace. It did stuff that left the BBC eating its dust. It was called get_iplayer. It was the Crab Nebula of software. It had (has) one of everything.
Unlike the BBC's iPlayer it facilitated a huge number of features including streaming, PVR (scheduling va Cron), downloading and formats. It was impressive. Really impressive. I've written everything in the past tense but although the developer (Paul, I don't know his second name) has pulled both the download link and the documentation for the software it is still functional. As I write this article, get_iplayer is purring away in the background downloading an episode of Shaun the Sheep (just for test purposes, honestly). Of course, the software is no longer available so if the BBC tweak the iPlayer and lock out get_iplayer, intentionally or otherwise, then a superlative piece of software will have been lost. If the experience of Paul Battley's Ruby script is anything to go by, its a constant game of lockout and update, except that updates are off the menu. That's the story so far. This article is now concerned with two things: why get_iplayer was pulled and the behaviour of the BBC.
They just pulled this trick.....
First, a caveat. I have nothing but the greatest admiration for developers like Paul. They plough a lone furrow, at their own time and expense, to write great software and release it under a GPL licence for others to use and extend. The world is a better place for such people being in it. They give, we take. Too often. More of us need to become active contributors as well as passive consumers. Including me. With that out of the way, why was the software taken down?
The wording used struck me as a little strange. He refers to get_iplayer being "dropped in response to the BBC's lack of support for open source". This sounded odd to me. It seems to imply that there was a time when the BBC actively supported open source access to the iPlayer. As far as I know, they haven't. If the BBC had been enthusiastic supporters of open source access to the iPlayer they would have enabled from day one of the launch. Clearly, neither he or Paul Battley waited around for any kind of imprimatur from the BBC. They had an itch to scratch and they wrote software without a permission slip from Broadcasting House, by exploiting the BBC's desire to make the iPlayer available on the iPhone platform.
I believe, without doubt, that the BBC did try to deliberately disable get_iplayer
The comments above are taken from the main page but if you click on this link there is more explanation and some interesting links. I hope and think I am summing up his position fairly and accurately when I say that his view is that the BBC iPlayer has benefited from open source, that the BBC has failed to abide by the spirit of FOSS and since they believe software like get_iplayer is "detrimental to their service" he felt constrained to cease development as he had no desire to act against the BBC's wishes. The nearest he gets to a detailed account of events is this:
I was the sole get_iplayer developer until I dropped it last week and, as you say, the BBC didn't send in lawyers -- but then there would be no point in 'sending in the lawyers' for something entirely legal albeit 'unauthorised' as Ian Hunter puts it above (no open source client is authorised AFAIK). The only publicly reported 'Cease and desist' letter sent from the BBC for an unauthorised iplayer client was, AFAIK, for an upcoming app that was going to commercially exploit the BBCs content (even though those developers initially tried to contact the BBC to no avail by all accounts).
No, the BBC didn't ask anything of me - they never even attempted to communicate - they just pulled a trick like this one without simply contacting me, the developer, to discuss even though my contact details had always been on the get_iplayer web site...
However, I believe, without doubt, that the BBC did try to "deliberately disable it" knowing full well that get_iplayer has steered very clearly away from breaking the law and that SWF verification would potentially cause an issue for get_iplayer. The quote from Ian Hunter sums it up quite well IMHO:
"We know that a number of applications have been making unauthorised use of some media types and we have tightened security accordingly - this was done for several of the formats and content delivery types, not just for Flash. The result was that some applications that 'deep link' to our content may no longer work."
As the sole developer of get_iplayer Paul has the absolute right to cease development and take the download links off site but as he will also of course appreciate that, in the nature of FOSS and the GPL, anyone can modify and redistribute the software by hosting it on another website. It's just a matter of programming skills, time and the necessary finances to host it all on another website--and I'm sure he wouldn't object to that. But that's a separate issue. The thing that intrigues me is what exactly the BBC said and or did to cause Paul to feel constrained to close up shop. There are no direct clues on the get_iplayer site--except one link.
Are the Daleks in charge?
The BBC is prone to saying it is under pressure from the content producers ("rights holders"), the kind of management consultant spin that would make Lord Reith turn in his grave. The BBC has been taken over by the Daleks in suits--as the late Dennis Potter said. The BBC feign to imply that rights holders have the whip hand but the corporation has rights too.
Even though the open source get_iplayer with flvstreamer is slightly inconvenienced by this move (it resumes the stream every minute automatically), if you use it with rtmpdump instead of flvstreamer it works perfectly. (you have to add the rtmpdump swf verify option to the command using --rtmptvopts in get_iplayer). rtmpdump is the open-source tool that had the Adobe DMCA takedown notice last year but has since restarted development outside of the US.
This whole episode flags up the importance of open web standards like HTML5. If that is universally adopted what future would there be for Adobe's Flash and if the BBC used that video tag in their HTML canvas why would they want to use a processor and memory hog like Flash? (to say nothing of the fact that for internet users in areas of the UK where broadband is either slow or not available at all the iPlayer is not avialable option, but get_iplayer was). The irony is twofold: the BBC decided to make the iPlayer available to the iPhone and in doing so gave hackers their opening--but the iPhone doesn't use Flash and the BBC is playing a constant game of cat and mouse with developers to block third party open source software from accessing the iPlayer by circumventing Adobe Flash's SWF authentification (by using flvstreamer or rtmdump--and development of rtmdump continues outside the USA, even after their DCMA take down notice. The DCMA doesn't apply in the UK but the European Copyright Directive (ECD) does).
The BBC will argue that the implementation of SWF verification is not DRM per se but just ensuring the rights of their rights holders but it really is a form of content protection mechanism, not matter what they say. They are also being less than candid when they invoke the headlock of their rights holders. Many of their programs are made by independent production companies but many others are in-house and in theses cases the BBC is the actual rights holder. It has also been said (and I don't know if this is true or not) that the BBC is obliged under its own rules to have a controlling financial interest in any company involved in co-branding with it, and this puts claims about the primacy of rights holders into a different perspective. Either way, the BBC has publically committed itself to supporting the UK Digital Economy Bill. It is supremely ironic that the BBC's effective support for DRM, however motivated, will have the effect of pushing some users to downloading BBC content via the very software that the DEB bill criminalises and compel ISPs to cut them off. And promoting piracy into the bargain.
(Slightly off topic, the DEB not only allows ISPs to disconnect persistent download "offenders" but, amazingly, also allows anyone--anyone--under Clause 43, to use so-called orphan works (including photographs and other artwork) without permission or direct payment. Welcome to the world of Extended Collective Licensing, a Creative Commons Licence with menace? You don't ever get to choose a licence, even CC by SA. And don't even ask about the Henry V111 Clause 46. If you want to understand the motives behind this, follow the money.)
The fact that the BBC Trust has refused to investigate this surreptitious switcheroo speaks volumes. SWF is Adobe and the BBC use its Flash player for streaming iPlayer content. If you are looking for an explanation for the BBC's behaviour it probably lies in the direction of its overseas sales. This is a real revenue raiser and it would be more difficult to shift product if the BBC was unable to demonstrate tight "copyright" protection to national and international buyers. All that the BBC's fait accompli has done is, as one BBC employee has described it, to "push developers towards the dark net". "Legal" users will gain nothing from the changes but BitTorrent, Rapidshare and Usenet will see an an upsurge in traffic--and given the nature of these sites piracy will flourish. In contrast, the average user of get_iplayer and its ilk would only be downloading overwhelmingly for purely personal use.
This is not the only irony. The BBC and the iPlayer have both benefited heavily from free software. Their own the BBC lock out website lists the projects using it and it is worth quoting it:
For the BBC, open source software development is an extension of our Public Service remit. Releasing open source software helps our audience get additional value from the work they've funded, and also get tools for free that they couldn't get any other way. It also allows people outside the BBC to extend projects in such a way that may in future be used in the BBC
Now, that's ironic. The BBC can use free software to extend projects but the end users, many of whom fund them via the licence fee, will be frozen out if they use free software to build (superior) client apps to record iPlayer content. It is a further irony that the BBC, in collaboration with Canonical, developed a BBC plugin for the Totem, a well known piece of free software. That will also run foul of the SWF lockout.
Who's making policy: the IT crowd or the management consultants?
The BBC don't seem to understand the difference between open source and open standards and they don't really clarify "platform". Is it a simple OS, an OS plus third party software, plugins? They also seem to be confused about SWF too. They conflate it with DRM but some people take the view that it is just a protocol. All that third party open source clients were/are doing is to access iPlayer content without Flash and therefore without the need for SWF verification. That said, Adobe think SWF is content protection and describe it as a key tool that can be used to monetize premium video online. The BBC seem to think along the same lines. In an address at the IEA Future of Broadcasting Conference in 2007, Ashley Highfield (Director, Future Media and Technology) said:
BBC iPlayer is a service for everyone in the UK. The BBC earns over £500m from the international exploitation of its content, every penny of which goes back into making better programmes for the British public. We therefore needed to ensure, through the use of digital rights management, DRM, that our programmes could be viewed, in high quality, in the UK, for free, but not get instantly distributed around the world and undermine our international licensing and syndication deals.
DRM is not popular, but having it means the difference between being able to afford to make Blue Planet or not. Protecting our content is not optional anyway. A third of our content is made by independent producers who insist we protect our content as their future depends on exploiting that content themselves outside our rights window.
They also seem to have a peculiar idea of end user rights too. The BBC refer to "unauthorised" access. How exactly do they define "authorised"? It seems that their definition is bounded by their use of Adobe SWF, itself third party software and what they lock out, the BBC lock out too. Were they legally obliged to do this and should they ever have decided to use Flash in the first place?
In my preamble to the interview with Paul Battley about his Ruby script hack, I pointed out that the BBC's stance was both illogical and pointless because of course no such restrictions apply to recording content on DVD recorder and saving it off to the hard drive on your computer. In fact, this allows you to freely record an imported American series like Heroes, a series which the iPlayer explicitly does not make available for obvious reasons. Go figure. However, all native/in house content is usually available--on both platforms.
However, just when you thought it was safe to record regardless, there are some nasty and ominous rumblings that the BBC is trying to close that "loophole" too. Incredibly, they have applied to the UK telecoms regulator for permission to put DRM on digital TV signals and to encrypt the Electronic Programme Guide (EPG). Once again, the BBC has invoked the bogeyman of unspecified "rights holders", the same rights holders whose remit has allegedly hobbled the iPlayer (a tactic tried in the USA by rights holders too but their bluff was called). Like the term "platform" this is conveniently vague but the effects would not be. If successful, the BBC would force any receiver manufacturers to sign up to highly (non dislosure-agreed)restrictive DRM courtesy of an offshore consortium called DTLA and this would affect free and open source software like MythTV and laptop TV cards to mention a few. A conflict would arise between the terms of the MythTV licence and any BBC-enforced licence agreement used by hardware manufacturers. It has implications for software developers and programmers too as, increasingly, receiving/broadcasting moves from hardware to software. It would also create a Pacific Gyre of e-waste too and a digital underclass of people who cannot afford to "upgrade" their hardware (or get into debt to do so).
Once new technologies are created the beneficiaries proceed to pull up the drawbridge and deny access except on their exclusive terms, aided and abetted by legal eagles, patent trolls and deep pockets
The Open Rights Group (ORG) has submitted a credible and detailed response and Cory Doctrow has summed up why all of this is utterly incompatible and inimical to free software in an article for The Guardian. It is consoling to note though that technological history is littered with the corpses of previous attempts to hold back the tide of progress. You name it, they've opposed it: railways, records, tape recorders, VCRs, MP3 players and yet these inventions have gone on to create whole new businesses -- but once they are created the beneficiaries proceed to pull up the drawbridge and deny access except on their exclusive terms, aided and abetted by legal eagles, patent trolls and deep pockets.
This time things are additionally complicated by the fact that the BBC is a public broadcasting organisation which is funded entirely by the licence fee. So, if you own a television set (or radio) you are required by law to pay for a licence--despite the fact that your television can receive many other channels, all of which are funded by advertising revenues. So, even if you signed an oath in your own blood that you would never watch a single BBC programme, you would still need to pay the licence fee. Like Skynet, they have you by the sphericals. Checkmate. Therefore, the threats to open source are real and present. Can this really be the same BBC that announced proudly in 2004 the birth of the Creative Archive project which had the blessing of Lawrence Lessig? (and the presence at the launch of no less a free software luminary than Cory Doctorow. The irony of it. Again.):
The announcement by the BBC of its intent to develop a Creative Archive has been the single most important event in getting people to understand the potential for digital creativity, and to see how such potential actually supports artists and artistic creativity. If the vision proves a reality, Britain will become a centre for digital creativity, and will drive the many markets – in broadband deployment and technology – that digital creativity will support
Then, the iPlayer wasn't even a twinkle in the Director General's eye. How the mighty are fallen to the level of corporate pimps touting business for Microsoft and Adobe. The Creative Archive was a test case to see if trust could be superior to the digital misanthropy of DRM. It's almost as if the BBC has forgotten the lessons of the browser wars, when businesses were reluctant to support anything other than Internet Explorer. Firefox is now rampant and the market share of IE declines by the day. If the BBC spent less time in an endless battle to deny access to third-party open source iPlayer clients and more time improving the iPlayer (which is inferior to the clients it seeks to block) and enlisting the support of the free software community it might be pleasantly surprised at how this would improve their software (and ultimately their revenue stream).
From adversity, opportunity
What are the lessons to be learned from this debacle? First, that the BBC either simply doesn't understand the difference between open standards and open protocols or that it is abusing the terms of its licence and public remit in an act of shameless, cynical bad faith in order to advantage BBC commercial interests. It will gradually dawn on them, as it is beginning to dawn on other in the multimedia domain, that DRM is simply unworkable. The technically proficient will hack on regardless, the pirates will circumvent too while the mere end user with no programming skills with suffer all the inconvenience. DRM is only a tax on the BBC's own average licence payer. And a betrayal.
in the best traditions of free software and open source, get_iplayer will not perish from the face of the Earth
At the other end of the log two main conclusions emerge: First, that while it is a frequently expressed complaint that GNU/Linux is too fragmented, in this case its very diversity is its best protection. Judging by the response of users on the getiplayer website this piece of software was/is held in very high regard--and rightly so. So, in the best traditions of free software and open source, get_iplayer will not perish from the face of the Earth. It _is still available. So, where can you get it? Well, if you have a binary for it on your computer make it available to your friends, or upload it to any site you own or have access to. Already, others have made it available, in various formats:
-
Debian Package(debian binary for get_iplayer version 2.68-1)
-
A Perl script of get_iplayer (Version 2,76) is also available on Mediafire
If you've installed the beta of Lucid Lynx (Ubuntu) you should find an older verion of getiplayer in the repositories. Your mileage may vary. For the really desperate, there is iDownloader, a Java based cross platform implementation iPlayer client. I tried it. It works fine except that it runs in a browser, not a desktop client and that the free version's feature set is badly hobbled and riddled with annoying and frequent pop ups nagging you to but a licence. The only way to escape the pester power is to buy a licence but that licence is for _a single machine. It is not transferable. I experienced a distinct sense of deja vu and felt like I had regressed back ten years to using Windows. This is a pity as I quite liked iDowloader. It looked good, worked and the full features would have been impressive but that licence restriction is a deal breaker. GNU/Linux users simply won't tolerate than kind of proprietary recidivism.
This brings me to the second point: there is a third option, well known to lovers of free software. Forking. The nature of licences like the GPL makes this possible and get_iplayer has legions of admirers. One of them has done just that. He has forked it and made it available on Github. The developer's reasons for forking get_iplayer are well articulated and an inspirational clarion call:
Prometheus is the next generation of get_iplayer. It won't steal fire from the Gods, but it may enable you to use get_iplayer on a non-BBC-supported platform.
The BBC have a well documented platform neutrality policy which they seem to be reluctant to keep themselves to and after repeated digs at open source, I thought we'd show them what open source can do.
So get hacking!
Wonderful. It makes you proud to be a user of free software. As you will gather from that, the developer is re-writing the code and if he improves on get_iplayer it can only receive greater plaudits.
Github is an alternative and successor to other versioning control software (used by the Kernel developers) so it is ideal for collaborative projects. However, you won't find a pre-compiled binary there, just the source code but it is only a matter of time before someone tinkers with the code and creates a binary and makes it available.
This is great for programmers, so if you want to make a contribution it's all there, waiting for your efforts. Most of us, alas, are not skilled or experienced programmers. I am only taking baby steps with Python so I won't be doing anything any time soon. Besides, get_iplayer is written in Perl. This flags up one of the most important things everyone can do: learn to program, whether it be mere Bash scripting, Perl, Ruby, Java or Python. Software won't code itself. Only when you can code can you ever hope to be truly free from the proprietary trap.
In the meantime, post bug reports, proselytise, document--and start learning to program! Programming gives you the keys to the kingdom. Without it, like the command line, we see through a glass darkly. Oh, and Phil, please reconsider your decision to take down get_iplayer.