Sunday, 28 July 2013

Raspberry Pi Car Cam overlaid with OBD data

I like motorsport and love driving my car around race tracks, the only problem is when you capture this on film, using a dashcam or similar, it looks rubbish!  I wanted to add some data about what was going on.

I recently blogged about how to use a raspberry pi to read car diagnostics data, I modified this program to log the data to a file, I then went for a drive around Castle Combe racetrack using the raspberry pi camera board to capture HD video alongside the OBD-II diagnostics data from the car

I then created a program which converted the OBD-II data into a subtitle file and used mencoder to overlay the data (RPM, MPH, Temperature & Throttle position) as subtitles on top of the video.


A couple of laps around Castle Combe in a Lotus Elise

Capture Video
I used the standard raspivid program to capture the video from the camera board I had mounted on the dash:

filename=$(date -u +"%Y%m%d_%H%M%S").h264
raspivid -o /home/pi/$filename -hf -vf -t 1200000 > /home/pi/camera.log 2>&1 &

The line filename=$(date -u +"%Y%m%d_%H%M%S").h264 creates a variable for the filename which is the date + the time + a .h264 at the end.  e.g.  20131715_214854.h264

raspivid:
  • -o /home/pi/$filename - sets the output filename to the variable just created
  • -hf - flips the video horizontally, otherwise the perspective when looking forward from the car is wrong
  • -vt - flips the video vertically, because I had the camera mounted upside down
  • -t 1200000 - the number of milliseonds to run the video (20 mins * 60 seconds * 1000 milliseconds)
  • > /home/pi/camera.log - logs the standard output to a file
  • 2>&1 - logs the standard error to a file
  • & - runs the program 'in the background'

Capture OBD data
The program I wrote reads the car diagnostics data using a ELM237 usb reader and creates a simple file in a CSV (comma seperated variable) file format of [Time, Sensor, Value, Unit]

22:51:18.316714,       Throttle Position,9.41176470588,%
22:51:18.442497,     Coolant Temperature,81,C
22:51:18.598279,              Engine RPM,2664,
22:51:18.728214,           Vehicle Speed,14.9160969546,MPH
22:51:18.858959,       Throttle Position,9.41176470588,%
22:51:18.990455,     Coolant Temperature,81,C
22:51:19.147226,              Engine RPM,2767,
22:51:19.277207,           Vehicle Speed,14.9160969546,MPH
22:51:19.405847,       Throttle Position,9.01960784314,%
22:51:19.536875,     Coolant Temperature,81,C
22:51:19.700181,              Engine RPM,2837,


Create Subtitle File
I created a program to read the OBD data file and convert this into a SubRip subtitle file.  The SubRip files are simple text files which allow you to define subtitles which can be imported into or played alongside movies.

The subtitle file looks like this:

675
00:02:28,098 --> 00:02:28,224
           Vehicle Speed 14.9 MPH
     Coolant Temperature 81 C
       Throttle Position 5.0 %
              Engine RPM 2774

676
00:02:28,225 --> 00:02:28,353
           Vehicle Speed 14.9 MPH
     Coolant Temperature 81 C
       Throttle Position 5.0 %
              Engine RPM 2774

677
00:02:28,354 --> 00:02:28,513
           Vehicle Speed 14.9 MPH
     Coolant Temperature 81 C
       Throttle Position 5.0 %
              Engine RPM 2774

The first number (675) is the number of the subtitle in the sequence, the 2 times are the times that the subtitle should appear and then disappear within the movie, the text after this is the subtitle, the subtitle is then terminated with a carriage return.

The code:
#www.stuffaboutcode.com
#Create SRT file from odb data file

#import modules
import datetime
import time

#Class to manage creating SRT File
class CreateSRT:
    def __init__(self, srtFileName):
        #Open subtitle file
        self.srtFile = open(srtFileName, "w")
        self.subTitleCount = 0

    def addSubTitle(self, startTime, endTime, text):
        #Add subtitle to file
        # increment count
        self.subTitleCount += 1
        # write count
        self.srtFile.write(str(self.subTitleCount) + "\n")
        # write from and to time
        self.srtFile.write("{0:02d}".format(startTime.seconds / 3600) + ":" +
                           "{0:02d}".format((startTime.seconds % 3600) / 60) + ":" +
                           "{0:02d}".format((startTime.seconds % 3600) % 60) + "," +
                           "{0:03d}".format(startTime.microseconds / 1000) +
                           " --> " +
                           "{0:02d}".format(endTime.seconds / 3600) + ":" +
                           "{0:02d}".format((endTime.seconds % 3600) / 60) + ":" +
                           "{0:02d}".format((endTime.seconds % 3600) % 60) + "," +
                           "{0:03d}".format(endTime.microseconds / 1000) + "\n")
        # write subtitle text
        self.srtFile.write(text + "\n")

    def close(self):
        self.srtFile.close()

def stripOBDData(obdDataLine):
    obdDataItems = obdDataLine.split(",")
    return obdDataItems[0], obdDataItems[1], obdDataItems[2], obdDataItems[3]

def getSubTitleText(subTitleOutput):
    subTitleText = ""
    for subTitle in subTitleOutput:
        subTitleText += subTitle + " " + subTitleOutput[subTitle]
    return subTitleText    

#data to drive program 
obdDataFileName = "22_49_0_obd.log"
srtFileName = "22_49_0.srt"
#the time the video started, using 1/1/1900, as only have time for obd data
videoStartTime = datetime.datetime(1900, 1, 1, 22,48,55,312014)
subTitleOutput = {"           Vehicle Speed": "\n",
                  "              Engine RPM": "\n",
                  "       Throttle Position": "\n",
                  "     Coolant Temperature": "\n"}

if __name__ == "__main__":
    #Open OBD data file
    obdDataFile = open(obdDataFileName)
    # read in obd data
    obdData = obdDataFile.readlines()
    obdDataFile.close()
    #Create SRT file object
    srtFile = CreateSRT(srtFileName)
    lastSRTTime = datetime.timedelta(0,0,0,1)
    # loop through obd data
    for obdDataLine in obdData:
        # get data items about of the obd data string
        sensorTime, sensorName, sensorValue, SensorUnit = stripOBDData(obdDataLine)
        # is the sensor in the subtitle output?
        if (sensorName in subTitleOutput.keys()):
            # get sensor time as object
            sensorTime = datetime.datetime.strptime(sensorTime, "%H:%M:%S.%f")
            # get the difference between the start of the video and sensor time
            timeDiff = sensorTime - videoStartTime
            # occasionally the sensors are not written in cronological order!  
            # no idea way, but i only want ones which are after the last one I saw
            if timeDiff > lastSRTTime:
                #Found a new sensor so the subtitle has changed
                # create the previous subtitle 
                srtFile.addSubTitle(lastSRTTime, timeDiff, getSubTitleText(subTitleOutput))
                # update the subtitle output and store the last time
                # if the sensor has a "." take only the first 1 digits after the point
                if sensorValue.find(".") != -1: sensorValue = sensorValue[:sensorValue.find(".") + 2]
                subTitleOutput[sensorName] = sensorValue + " " + SensorUnit
                # add one millisecond to the last subtitle time so the next one doesn't appear exactly as the last one disappeared!
                lastSRTTime = timeDiff + datetime.timedelta(0,0,0,1)

    #close srt file class
    srtFile.close()

Overlay Subtitles onto Video
I used MP4Box to put the h264 video stream created by raspivid into an mp4 file:

MP4Box -add 20130715_214854.h264 -fps 30 20130715_214854.mp4

The -fps 30 option is important, without it MP4Box will assume the video is taken in 25fps and the resulting output will be playback slower than the original.  It took me a while to work this out!  I couldn't work out why my video and my data wouldn't sync.  If your short on space on your SD card you can output to a USB or NAS drive but you will also have to use the -tmp /folder option as MP4Box uses the sd card for temporary storage while creating the video.

I then used mencoder to encode the mp4 video and the subtitles together into one mp4 video:

mencoder -ovc lavc -sub 22_49_0.srt -subfont-text-scale 2.0 -subpos 95 -sub-bg-alpha 50 -utf8 -o 20130715_214854_sub.mp4 20130715_214854.mp4

Be prepared to wait though, encoding a 13 minute full HD video took 9 hours on the Pi!

This is very much in the prototype stage at the moment, when I have tided up the code, automated it and made it more resilient, Ill provide all the code and more instructions on how to use it.  It was also painfully difficult to get the data to sync with the video, so I need to think of a better solution for this.

22 comments:

  1. Hi Martin,

    a great post - is your real name the Stig?

    Seriously, isn't there a way of using the graphics chip in the Raspberry Pi to do overlays? Then you might be able to get a real-time stream for the spectators.

    ReplyDelete
    Replies
    1. Maybe! Probably! One step at a time though Victor.

      Delete
  2. Very Cool. Maybe for some further inspiration, look at the addons to Torque for Android. It also reads the ELM327 and one of the addons does guage overlays onto the recorded video from the android device.

    If the camera module is available in the android install for Pi, might even be able to just reuse the current hardware config and just change out the software.

    ReplyDelete
  3. I've been thinking about OBDII computer project for a while now, 1st a 8051 and serial port based one, but was disappointed about how little information was available about the protocol. I could not find enough to write my own driver. Now you can buy a $10 device that will communicate with your PC or smartphone over USB or BT and use a free software for that. I like this project and I want to duplicate your path by doing my own car computer with video recording capabilities. Can you tell what kind of camera module did you use in your project? I see the ribbon cable going to camera and I wonder if a regular USB web cam can be used with Raspberry PI?

    ReplyDelete
  4. Good post, nice car, do you know of the kickstarter project for carputers "Raspberry Pi Car Power Supply Ignition Switch" http://www.kickstarter.com/projects/1312527055/raspberry-pi-car-power-supply-ignition-switch might be of interest.

    ReplyDelete
  5. It's quite a bit heavier but you might want to look at the OpenXC project as well. :) http://openxcplatform.com Right now Ford is the only company supporting it but I'm going to do a bit of work to try to reverse engineer some of the signals on my Nissan 370Z when I get time.

    ReplyDelete
  6. Very cool project. I can't wait to see further iterations as you develop it.

    I have a question too. It's been my experience that OBD data is fairly laggy and doesn't have a very high sample rate. Has this been your experience or does the Lotus spit data at the OBD bus as fast as it drives?

    ReplyDelete
    Replies
    1. In my project the end-to-end time to send a command to OBD, pull back the data, parse it and save it to a file was about 0.2 seconds. This is adequate if your only logging 1 peice of data, but if you want 5, you are making 5 separate requests, so suddenly my request time was 1 second, not great. I've not looked at baud rate tho, I suspect im using the default and slowest so there is probably improvement to be made.

      Delete
    2. Cozy up with a Lotus service tech and find out if Lotus uses other buses like CAN or a proprietary bus that is available on the OBD II port or elsewhere in the vehicle. While all manufacturers support the mandated OBD II functions and standards, many find it inadequate to their needs and actually use another bus topology along side the mandated ones. These often are not only faster and more responsive, but offer more detailed data and control options.
      That said, you're right to try exploring connection speed settings with your existing setup first.

      Delete
    3. A little Googling led me here http://wiki.seloc.org/a/On_board_diagnostics
      which states that on some models Lotus provides CAN access right on the OBD port so I'm thinking this may be worth exploring.

      Delete
  7. Yes, it's me again. Strangely enough I came across this today and it sounds like exactly what is needed for this project.
    OpenXC
    http://openxcplatform.com/overview/faq.html

    ReplyDelete
  8. Hello Martin, I am an avid reader of your blog and, like Nicholas, I am a young programmer. I have been working on a program that creates a menu GUI that can take you to other GUIs that are full of minecraft games/tools, including your snake game. Sorry if that's bad, I'll take it out. But, I did credit you in it and put your site in it. It would absolutely make my day for you to just look at it.
    But, I have no way of sending you the file. Pleas help

    ReplyDelete
    Replies
    1. Hi Jakey,
      It sounds like a good project. Anyone and everyone is welcome to use my code, Im not precious about it, a mention is always nice tho, so thank you.
      Would love to look at your program. In terms of publishing it so people can access it, I would encourage you to look at github, although it can be a little daunting to start with, or , how about putting your program on dropbox or google drive and sharing it - they are really easy to use.
      Mart

      Delete
  9. Great thanks so much for replying! I will put the program up on my google drive as soon as I finish changing it a little to make the menu itself look better.

    ReplyDelete
  10. Hello! Thank you for sharing your project and great integration using the power of Raspberry Pi! Out of curiosity, which camera did you choose for video capture?

    Over the last couple of years I've developed the hardware and firmware for a rugged, high performance Open Source Hardware and Software data acquisition system for motorsports called RaceCapture/Pro http://autosportlabs.net/RaceCapturePro

    We had a successful indiegogo campaign that completed at the beginning of this year: http://indiegogo.com/RaceCapture

    The benefit of OBD2 is that while it's super convenient to interface, the sample rate is very low when many channels are enabled. Our goal with the RaceCapture/Pro project was to enable up to 100Hz logging across all channels and then build in support for an OBD2 interface for additional convenience when desired.



    ReplyDelete
    Replies
    1. The camera is the standard raspberry pi camera board. It delivers 1080p at 30fps and plugs directly into the raspberry pi.

      I had seen your product before, it looks really good, I had always wanted to make my own and for me data gather through OBD is good enough and doesnt involve the installation of sensors through-out the car.

      My aim was always to overlay data on top of video as well, Im not using it to get detailed info about the car's performance, I just want to get some nice video for little money.

      Delete
  11. You could definitely productize this as a car version of the plane black box. Would be good for insurance, and monitoring delivery and service fleets and young drivers.

    ReplyDelete
  12. This comment has been removed by the author.

    ReplyDelete
  13. Great tutorial! How did you convert the output to csv? I'm using tee to do a capture but can't really get a hang of converting it to csv...Could you post the example?

    ReplyDelete
  14. Hi can you send me the link to your google drive, i would love to set this up on my car

    ReplyDelete
    Replies
    1. Why do u need a link to my google drive?

      Delete
  15. sorry i jsut wanted to grab the code thats all so ic an have a go, i have got my pi running, got it recording video via the terminal, i have my elm and need to get the data writing to csv for the subtitle bit, then the rest i think i can follow from your guide, so just wondered if you had a project i could download or if you could help me out a bit as i want to learn to develop more in python as i do c sharp / asp normally

    ReplyDelete