get_iplayer - generate an RSS / podcast

Anyway, Ive been improving the BBC iPlayer custom podcast solution I created for the Raspberry Pi by making it more robust and re-factoring the code to introduce some 're-usability' and future proofing.

A big area of change is the python program I originally created which creates an RSS (podcast) feed; this program recursed a directory path and created the xml file based on some hard-coded constants in the code such as "rssURL".

So I created a re-usable component for creating RSS feeds from get_iplayer which I have dubbed "get_iplayer_genrss" which reads the get_iplayer download_history file and by using parameters passed in such as URL where downloads are e.g. "http://server/path/to/downloads" creates an RSS file based on its contents.

It supports functions such as:

• Days in the past - only include in the RSS, downloads from the past [so many] days
• Media type - only include downloads with a specific media type (tv, radio, etc)
• Alternative directories - search alternative directories for the download, useful if the download has been copied to a different location

I have 'open sourced' the project and the code is being managed github https://github.com/martinohanlon/get_iplayer_genrss.git and would welcome any contributions, comments or feedback.

I am currently using get_iplayer_genrss to create rss feeds for radio shows which I download to my mobile phone and for tv shows which I use in XBMC.

For instructions on installation and usage see this page.

Python - Create RSS / Podcast of MP3 files

Anyway, a nice guy called Dan Goff left a comment on my post about creating a RSS / podcast file using Python letting me know that he had taken my code template and modified it to use the Mutagen library to pick up certain values from the ID3 tags inside MP3 files.

Dan says:

• I'm populating the title for each entry and the description directly from the ID3 tags provided using the Mutagen library.
• I ran into an issue with my podcatching software, because the files are only tagged with the year and not a full date, so part of the code is to take a fully-formatted date string and write it back to the file so that it is listed correctly when my software "downloads" the file from the feed.
• Also, I ran into an issue with some Unicode characters in the description (mostly an ellipsis), so I took the quick-and-dirty (and probably not exactly correct) route and just converted everything in the description field to ASCII before writing it out to the feed.

Dan has also kindly agreed to let me share his code:

 

# update environment to handle Unicode
PYTHONIOENCODING="utf-8"

# import libraries
import os
import sys
import datetime
import time


# import constants from stat library
from stat import * # ST_SIZE ST_MTIME

# import ID3 tag reader
from mutagen.id3 import ID3, ID3TimeStamp, TDRC
from time import strptime, strftime

# format date method
def formatDate(dt):
    return dt.strftime("%a, %d %b %Y %H:%M:%S +0000")


# get the item/@type based on file extension
def getItemType(fileExtension):
    if fileExtension == "aac":
         mediaType = "audio/mpeg"
    elif fileExtension == "mp4":
         mediaType = "video/mpeg"
    else:
         mediaType = "audio/mpeg"
    return mediaType


# constants
# the podcast name
rssTitle = "The podcast title"
# the podcast description
rssDescription = "The podcast description"
# the url where the podcast items will be hosted
rssSiteURL = http://xxx.xxx.xxx.xxx
# the url of the folder where the items will be stored
rssItemURL = rssSiteURL + "/Podcasts"
# the url to the podcast html file
rssLink = rssSiteURL #+ ""
# url to the podcast image
rssImageUrl = rssSiteURL #+ "/logo.jpg"
# the time to live (in minutes)
rssTtl = "60"
# contact details of the web master
rssWebMaster = "me@me.com"


#record datetime started
now = datetime.datetime.now()


# command line options
#    - python createRSFeed.py /path/to/podcast/files /path/to/output/rss
# directory passed in
rootdir = sys.argv[1]
# output RSS filename
outputFilename = sys.argv[2]


# Main program

# open rss file
outputFile = open(outputFilename, "w")


# write rss header
outputFile.write("<?xml version=\"1.0\" encoding=\"UTF-8\" ?>\n")
outputFile.write("<rss version=\"2.0\">\n")
outputFile.write("<channel>\n")
outputFile.write("<title>" + rssTitle + "</title>\n")
outputFile.write("<description>" + rssDescription + "</description>\n")
outputFile.write("<link>" + rssLink + "</link>\n")
outputFile.write("<ttl>" + rssTtl + "</ttl>\n")
outputFile.write("<image><url>" + rssImageUrl + "</url><title>" + rssTitle + "</title><link>" + rssLink + "</link></image>\n")
outputFile.write("<copyright>mart 2012</copyright>\n")
outputFile.write("<lastBuildDate>" + formatDate(now) + "</lastBuildDate>\n")
outputFile.write("<pubDate>" + formatDate(now) + "</pubDate>\n")
outputFile.write("<webMaster>" + rssWebMaster + "</webMaster>\n")


# walk through all files and subfolders
for path, subFolders, files in os.walk(rootdir):
   
    for file in files:
 
        # split the file based on "." we use the first part as the title and the extension to work out the media type
        fileNameBits = file.split(".")
        # get the full path of the file
        fullPath = os.path.join(path, file)
        # get the stats for the file
        fileStat = os.stat(fullPath)
        # find the path relative to the starting folder, e.g. /subFolder/file
        relativePath = fullPath[len(rootdir):]

      # Extract ID3 info
      #title
      audio = ID3(fullPath)
      fileTitle = audio["TIT2"].text[0]
      #date
      datePos = fileTitle.find(":")
      fileDate = fileTitle[(datePos+2):]
      fileDate = time.strptime(fileDate, "%B %d, %Y")
      #correct date format in the file's ID3 tag
      fileTS = ID3TimeStamp(time.strftime("%Y-%m-%d", fileDate))
      audio['TDRC'] = TDRC(0, [fileTS])
      #description
      fileDesc = audio["COMM::'eng'"].text[0]
      fileDesc = fileDesc.encode('ascii', 'ignore') #converts everything to ASCII prior to writing out
      audio.save()

        # write rss item
        outputFile.write("<item>\n")
        #outputFile.write("<title>" + fileNameBits[0].replace("_", " ") + "</title>\n")
      outputFile.write("<title>" + fileTitle + "</title>\n")
        #outputFile.write("<description>A description</description>\n")
        outputFile.write("<description>" + fileDesc + "</description>\n")
        outputFile.write("<link>" + rssItemURL + relativePath + "</link>\n")
        outputFile.write("<guid>" + rssItemURL + relativePath + "</guid>\n")
        #outputFile.write("<pubDate>" + formatDate(datetime.datetime.fromtimestamp(fileStat[ST_MTIME])) + "</pubDate>\n")
      outputFile.write("<pubDate>" + time.strftime("%Y-%m-%d", fileDate) + "</pubDate>\n")
        outputFile.write("<enclosure url=\"" + rssItemURL + relativePath + "\" length=\"" + str(fileStat[ST_SIZE]) + "\" type=\"" + getItemType(fileNameBits[len(fileNameBits)-1]) + "\" />\n")
        outputFile.write("</item>\n")

       
# write rss footer
outputFile.write("</channel>\n")
outputFile.write("</rss>")
outputFile.close()
print "complete"
 

Thanks again to Dan.

Raspberry Pi - Python - create podcast / RSS

Anyway as part of my iPlayer personal podcast solution I needed a way of creating an RSS feed.  Python seemed like a logical choice given its a flexible and powerful scripting language and it comes ready on the Pi Debian distro.  Even though I had never written a python script before!

Create a podcast from a directory
I chose a really simple implementation, a python program that re-cursed the directory where the media files are stored and created an RSS xml file based on the content.

It only implements the base RSS specification, there are a number of tags for providing additional meta data, particularly for use with iTunes, but for my purposes this wasn't required.  For more information about the RSS standard and podcasts see http://www.podcast411.com/howto_1.html.

I also chose to output the xml as strings rather than using an XML parser, simply because it seemed like a significant overhead (mainly in terms of learning how) just to output a simple structure.

This script has only been tested for my requirements and its not really designed to have a tremendous amount of re-use, but feel free to adapt it to your needs.

Python script - createRSSFeed.py

# import libraries
import os
import sys
import datetime
import time

# import constants from stat library
from stat import * # ST_SIZE ST_MTIME

# format date method
def formatDate(dt):
    return dt.strftime("%a, %d %b %Y %H:%M:%S +0000")

# get the item/@type based on file extension
def getItemType(fileExtension):
    if fileExtension == "aac":
         mediaType = "audio/mpeg"
    elif fileExtension == "mp4":
         mediaType = "video/mpeg"
    else:
         mediaType = "audio/mpeg"
    return mediaType

# constants
# the podcast name
rssTitle = "Podcast title"
# the podcast description
rssDescription = "Podcast description"
# the url where the podcast items will be hosted
rssSiteURL = "http://www.myurl.com/mypodcast"
# the url of the folder where the items will be stored
rssItemURL = rssSiteURL + "/iPlayerRadioDownloads"
# the url to the podcast html file
rssLink = rssSiteURL + "/index.html"
# url to the podcast image
rssImageUrl = rssSiteURL + "/logo.jpg"
# the time to live (in minutes)
rssTtl = "60"
# contact details of the web master
rssWebMaster = "me@me.com"

#record datetime started

now = datetime.datetime.now()

# command line options

#    - python createRSFeed.py /path/to/podcast/files /path/to/output/rss

# directory passed in

rootdir = sys.argv[1]

# output RSS filename

outputFilename = sys.argv[2]

# Main program

# open rss file

outputFile = open(outputFilename, "w")

# write rss header

outputFile.write("<?xml version=\"1.0\" encoding=\"UTF-8\" ?>\n")
outputFile.write("<rss version=\"2.0\">\n")
outputFile.write("<channel>\n")
outputFile.write("<title>" + rssTitle + "</title>\n")
outputFile.write("<description>" + rssDescription + "</description>\n")
outputFile.write("<link>" + rssLink + "</link>\n")
outputFile.write("<ttl>" + rssTtl + "</ttl>\n")
outputFile.write("<image><url>" + rssImageUrl + "</url><title>" + rssTitle + "</title><link>" + rssLink + "</link></image>\n")
outputFile.write("<copyright>mart 2012</copyright>\n")
outputFile.write("<lastBuildDate>" + formatDate(now) + "</lastBuildDate>\n")
outputFile.write("<pubDate>" + formatDate(now) + "</pubDate>\n")
outputFile.write("<webMaster>" + rssWebMaster + "</webMaster>\n")

# walk through all files and subfolders 

for path, subFolders, files in os.walk(rootdir):
    
    for file in files:

# split the file based on "." we use the first part as the title and the extension to work out the media type
        fileNameBits = file.split(".")
        # get the full path of the file
        fullPath = os.path.join(path, file)
        # get the stats for the file
        fileStat = os.stat(fullPath)
        # find the path relative to the starting folder, e.g. /subFolder/file
        relativePath = fullPath[len(rootdir):]

        # write rss item
        outputFile.write("<item>\n")
        outputFile.write("<title>" + fileNameBits[0].replace("_", " ") + "</title>\n")
        outputFile.write("<description>A description</description>\n")
        outputFile.write("<link>" + rssItemURL + relativePath + "</link>\n")
        outputFile.write("<guid>" + rssItemURL + relativePath + "</guid>\n")
        outputFile.write("<pubDate>" + formatDate(datetime.datetime.fromtimestamp(fileStat[ST_MTIME])) + "</pubDate>\n")
        outputFile.write("<enclosure url=\"" + rssItemURL + relativePath + "\" length=\"" + str(fileStat[ST_SIZE]) + "\" type=\"" + getItemType(fileNameBits[len(fileNameBits)-1]) + "\" />\n")
        outputFile.write("</item>\n")

       

# write rss footer

outputFile.write("</channel>\n")
outputFile.write("</rss>")
outputFile.close()
print "complete"

Running the script
The script expects 2 parameters:

• The path where the media files are stored
• The path of the output rss file

python createRSSFeed.py /path/to/media/files /path/to/output/RSSFile.rss

Update - I came across some problems when there was escape characters in the xml, so had to write a function to encode text to make it xml safe.

Update - Dan Goff sent me on a modified version of this program which uses the mutagen library to include data from ID3 tags in mp3 files

Raspberry Pi - BBC iPlayer Personal Podcast

Anyway... I like radio, love it, I like music and I love radio comedies, particularly those on BBC Radio 4, but, I never get to hear them, I'm just not around at the right time and when I do have time to listen (in the car, on the train, etc) they aren't on and streaming them is a right pain.

So along came the Raspberry Pi and I thought maybe here is an opportunity for me to have an always on, low power device, which could download stuff from BBC iPlayer, using the open source project get_iplayer (see my post about how to install get_iplayer on the Pi), then I could put it on my phone / ipod and listen to it when I do have time.

Then I thought how about making my own personal custom podcast of the things I have downloaded from iPlayer, then I can subscribe to it and hey presto, all that stuff I wanted to listen too, but never got the time will automatically end up on my phone!  Not only brilliant for listening to radio, but it would also work for TV shows as well!

This is how I did it:

I used the DVR capabilities of get_iplayer to download TV & Radio streams from BBC iPlayer, I also set up the WebUI component which allowed me to tell get_iplayer to download specific programmes or a complete series as and when a new episode becomes available from the comfort of my laptop.

Every hour a scheduled job (CRON) runs a series of commands:

• The DVR function in get_iplayer in called which downloads and encodes any new programmes which have become available since it last ran.  See post re get_iplayer setup.
• A script then moves the programmes from the SD onto my NAS drive using Samba.  See post re connecting to a NAS.
• A small python program, get_iplayer_genrss, then runs which uses the get_iplayer download history to write out a RSS file which contains the latest programmes which have been downloaded. See here for installation and usage.

Lighttpd (lighty) is then used to provide http access to the RSS file and the programmes via a symbolic link to the NAS drive.

I can then subscribe to the RSS using a podcast app (I use google listen), it will then download the new programmes allowing me to use them at my leisure.

I am SO impressive with this, not with me for putting it together, but because I have achieved this using a £25 peice of hardware, some free readily available software and 1 small python program.  This for me shows the power of the Raspberry Pi, not its computational power, but the inspirational it can bring to people to try a create something.