Monday, 4 March 2013

Raspberry Pi - Google Drive & Grive

I use google drive a lot and I wanted to use my raspberry pi to automatically backup some of my more important files (i.e. documents and pictures) to google's cloud.

Now google hasn't released a google drive client for linux, as yet, but there is a really good open source linux client called grive which allows you to sync with google drive.

Install grive
I got most of this info from, http://www.lbreda.com/grive/installation.

Install dependencies
Use apt-get to install grive's dependencies, I already had some of these installed, but it hurt to re-run if your not sure.

sudo apt-get install git cmake build-essential libgcrypt11-dev libjson0-dev libcurl4-openssl-dev libexpat1-dev libboost-filesystem-dev libboost-program-options-dev binutils-dev

Get the grive source code

cd ~
git clone git://github.com/Grive/grive.git

Compile grive

cd grive
cmake .
make

Wait a while for it to compile (10-20 mins)

Setup your google drive folder and authenticate 
You need to create a directory which grive will sync Google Drive with; anything in Google Drive will be synced into this folder and anything in this folder will be synced to Google Drive.

mkdir ~/googleDrive

In order to make grive work it needs to be in your google drive copy the grive executable to the google drive directory, so copy it there:

cp ~/grive/grive/grive ~/googleDrive

Authenticate grive to work with your Google Drive account.

cd ~/googleDrive
./grive -a

Grive will now give you a URL and ask you to provide an access code.  Cut and paste the URL into a browser, if your not logged in to google, login, and it will present you with an access code, cut and paste this access code into grive.

NOTE - In order to run grive, you must be in the google drive directory, for some reason if you are not, grive doesn't work and moans about having to authenticate even if you have!

Test
Grive can be run in a test mode, which will do everything other than uploading / downloading.

./grive --dry-run

Run

./grive

Once completed, your ~/googleDrive directory should now hold a copy of everything which is in your google drive.  Also any files you put in your googleDrive directory will also get synced to google drive.

Backup other files and folders
I wanted to backup other files and folders and didn't want to put them in my ~/googleDrive folder so I did this by using symbolic links.  Using symbolic links we can make it appear that a file over folder from a different location is in the google drive folder and as such will get synced to google drive.  I used this technique to create a link to my Pictures folder which is stored on my NAS drive, that way all my pictures are backed up to google drive.

ln -s /path/to/my/pictures ~/googleDrive/pictures

For more information on ln command.

Restricting bandwidth using trickle
One of the challenges I found in using backing up my files to google drive was a simple one of bandwidth, or more accurately a lack of bandwidth.  When grive first ran, it uploaded a LOT of data to google drive and totally saturated by broadband (particularly as upload speeds are generally a lot less than download).

I came across a great utility called "trickle" which allows you to specify a maximum upload and download speed for any command, see http://www.tuxradar.com/content/control-your-bandwidth-trickle for more info.

Install trickle

sudo apt-get install trickle

Run grive using trickle
The following command will specify a maximum download (-d) and upload (-u) limit for grive of 50k.

cd ~/googleDrive
trickle -d 50 -u 50 ./grive

Schedule
I then chose to schedule grive to run once a week, using cron, thereby automating my file backup to google drive.


14 comments:

  1. Great idea, just stumbled across you site and this was one of the things on my "to do" list. Many thanks for putting this together.

    One correction though, libcurlost-filesystem-dev should be libboost-filesystem-dev.

    Others mileage may vary too, I had to include another couple of libraries that I didn't already have, specifically libexpat1-dev and libcurl4-openssl-dev.

    ReplyDelete
    Replies
    1. Thanks for the info, are you running Raspbian?

      Delete
  2. FYI and for all other noobs like me

    there is a typo in your dependencies, after looking for libcurlost-filesystem-dev package I realised that you should replace this with

    - libcurl4-openssl-dev
    - libexpat1-dev
    - libboost-filesystem-dev

    packages (I spent a half hour on this, yes i'm a tux noob)

    thanks for the work

    regards,
    Cem

    ReplyDelete
  3. Martin - thanks for this; have recreated this using the latest raspbian build (as of 22/03/2013) without trouble.

    One concern though - I wondered if you had given this thought - my gdrive is pretty big (>5GB now) and I don't want my poor pi to have to sync to the whole thing. I would like a 'pi' subdirectory, say, on GDrive that will get synchronised with the pi's local filesystem (or part of it). Have you seen anyone achieve this ? I don't think grive supports it although there was some discussion about this idea.

    Without this I think I will have to resort to googlecl, which has been a bit flakey for the last nine months or so (largely because it ought to be obsolete really). Any bright ideas ?



    ReplyDelete
  4. I get an error when I run this command

    sudo apt-get install libcurl4-openssl-dev

    ReplyDelete
  5. I also had to manually install libboost test as well as qt4, 10 mins of googling told me how

    ReplyDelete
  6. same Problem with qt4. plz help me

    ReplyDelete
    Replies
    1. sudo aptitude install libqt4-dev

      Delete
    2. CMake Error at /usr/share/cmake-2.8/Modules/FindBoost.cmake:1194 (message):
      Unable to find the requested Boost libraries.

      Boost version: 1.49.0

      Boost include path: /usr/include

      The following Boost libraries could not be found:

      boost_unit_test_framework

      Some (but not all) of the required Boost libraries were found. You may
      need to install these additional Boost libraries. Alternatively, set
      BOOST_LIBRARYDIR to the directory containing Boost libraries or BOOST_ROOT
      to the location of Boost.



      the old version is working


      Delete
    3. This comment has been removed by the author.

      Delete
  7. I'm running raspbian and I keep getting these errors.

    /home/pi/grive/libgrive/src/json/JsonWriter.cc:24:27: fatal error: yajl/yajl_gen.h: No such file or directory
    compilation terminated.
    make[2]: *** [libgrive/CMakeFiles/grive.dir/src/json/JsonWriter.cc.o] Error 1
    make[1]: *** [libgrive/CMakeFiles/grive.dir/all] Error 2

    ReplyDelete
    Replies
    1. Try:

      sudo apt-get install libyajl-dev

      ... and then go back to building grive.

      I get a linking error at this point, but the solution to that can be found here:

      https://github.com/Grive/grive/issues/168

      Making the edits suggested there allows grive to compile completely.

      Delete
    2. That did the trick. Thanks.
      I followed the suggestions at that link and now it's compiled.

      Delete