Thursday, 22 August 2013

Shared Memory C & Python IPC

I had a problem whereby I needed to use a C program to capture video, in this case RaspiVid (the raspberry pi camera capture program), but I wanted to sync the video with data being capture by a Python program; in order to get the sync right I need to grab data about the video capture as it was running.

To do this I had to find a method of doing Inter Process Communication (IPC), very quickly, with a very low performance impact on the C program.  I explored several IPC options between C and Python (stdin/stdout, named pipes, tcp, shared memory) and found that using Shared Memory was the only way to deliver the performance I needed.

I pulled together a quick proof of concept to learn the basics.

C - Writing to Shared Memory
I created a C program which writes data into a shared memory segment and then waits (to allow me time to run the python program to read it out).  See for a description of how to use shared memory and this video for a tutorial.


#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>

int main(int argc, const char **argv)
   int shmid;
   // give your shared memory an id, anything will do
   key_t key = 123456;
   char *shared_memory;

   // Setup shared memory, 11 is the size
   if ((shmid = shmget(key, 11, IPC_CREAT | 0666)) < 0)
      printf("Error getting shared memory id");
   // Attached shared memory
   if ((shared_memory = shmat(shmid, NULL, 0)) == (char *) -1)
      printf("Error attaching shared memory id");
   // copy "hello world" to shared memory
   memcpy(shared_memory, "Hello World", sizeof("Hello World"));
   // sleep so there is enough time to run the reader!
   // Detach and remove shared memory
   shmctl(shmid, IPC_RMID, NULL);

Compile using:

gcc -o shmwriter shmwriter.c

Python - Reading from Shared Memory
I found a great module for python, sysv_ipc, which greatly simplifies the interaction with shared memory, see for more information, download and install instructions.

import sysv_ipc

# Create shared memory object
memory = sysv_ipc.SharedMemory(123456)

# Read value from shared memory
memory_value =

# Find the 'end' of the string and strip
i = memory_value.find('\0')
if i != -1:
    memory_value = memory_value[:i]

print memory_value

Run using:


You can download the code from github,


  1. Great little write up, and *very* helpful. I've never used Shared Mem or semaphores. Learn something new and useful everyday.

    A couple of quick notes:

    1) Looks like there is a cast issue in the line: shmdt(shmid); Not a C guy, but I changed it to shmdt(&shmid); and it compiled fine.

    2) I am also not a Python guy. I cut my teeth on Perl, so I played around a bit and got the reader working under Perl.

    #!/usr/bin/perl -w
    use strict;

    use IPC::SharedMem;

    my $shm = IPC::SharedMem->new(12345678, 11, S_IRUSR);
    my $val = $shm->read(0, 11);

    print "$val\n";

    Looking forward to going home tonight, and trying to figure out how to get the Perl semaphore code to work with Shared mem, so I can try and emulate what you did with your python example over on the Data Synching post:

    Already pulled and complied your userland fork for RASPIVID.



    1. whoops:

      Need to change the Key up above to 123456 (not 12345678) to make it work with your C example.

      my $shm = IPC::SharedMem->new(123456, 11, S_IRUSR);