Marrying Google Docs and WordPress (or really any CMS)

At the BDN, we love WordPress, but we didn’t feel it was the right place to have our reporters working. Even with distraction-free writing, there’s too much cruft that would confuse reporters or that we don’t want reporters touching. We wanted something lightweight, fast, simple to grasp and that would make editing easier.

Enter Google Docs. It’s easy to use, lightweight and a large number of people are already familiar with it. And, it’s got beautiful tools that enable real-time collaboration with reporters and editors.

That’s where we decided to start. Reporters write stories in Google Docs, and the docs are sorted in a series of collections, which are shared out to everybody. We have collections for each desk, such as metro, state, sports, etc.; collections for workflow, such as Needs Copy Editing, On Hold, etc.; and collections for actions, such as Send to Publish and Published.

The action collections are important — they’re how docs actually get from Google to WordPress. When we first started using Docs with our sports department in August or September of last year, Docs was much different, more neanderthal. But, that was nice in some ways. The docs came through with inline markup, and Docs supported XML-RPC, which made it easy to connect the two systems.

Then came an upgrade, which was on the whole a good one. It combined what they’d learned building Wave to bring real-time collaboration to a new level. But it also eliminated XML-RPC support, and docs aren’t marked up as nicely as they used to be.

So, we delved into the API, and I think what we have now is an even nicer system than XML-RPC could provide. You can find a version of what we built in the WordPress Plugin Repository.

On the whole, the process is fairly simple. When we’re all done copy editing an article, the doc goes into a collection called Send to Publish. We have a script running every two minutes that will grab docs from that collection, process them, and move them to WordPress. Then the script takes the doc out of Send to Publish and puts the doc in a collection called Published.

Here are a few features the plugin provides:

  • If the doc is a new post, it will go into WordPress as a draft. If the doc has already been put into WordPress, it will update the previous post.
  • Usernames in Docs must correspond with usernames in WordPress.
  • If a doc is in a collection and there is a corresponding category in WordPress, it will automatically put that the post in that collection. Else it will put it in the default category.

There are a few filters and actions in the plugin that allow you to extend it, and the plugin actually currently comes with one extender, which will strip out comments into a custom field and normalize the content. It’s a slightly stripped-down version of the extender we use.

We actually go one step further by fielding data using delimiters. We name the doc with a slug instead of a headline, and then the first line of the doc becomes a headline, followed by a pipe (|). After that comes the body copy, and at the very end we can add another pipe. Anything after that last pipe acts as a comment. We do this because we also use the API to put things, such as AP stories, into Google Docs, and we couldn’t figure out a way to add comments via the API.

To install the plugin, you’ll need to upload it to your /wp-content/plugins/ folder. Right now there isn’t an extender for wp-cron (there will be soon), so you’ll have to put the action on a page and point a cron job to it. I do this like so:

[php]
<?php
include(‘./wp-load.php’);
$docs_to_wp = new Docs_To_WP();
$gdClient = $docs_to_wp->docs_to_wp_init( ‘me@example.com’, ‘mypassword’ );
//We’re just going to call one function:
$docs_to_wp->retrieve_docs_for_web( $gdClient, ID of origin folder, ID of destination folder );
[/php]

As always, the code is on Github.

81 thoughts on “Marrying Google Docs and WordPress (or really any CMS)”

  1. I’m really impressed that you’ve created this plugin. I’ve been thinking about this for a long time. I feel like this could be a perfect solution for small dailies and weekly newspapers that can’t afford a huge Web/Print CMS.

    I’m sure sooner or later someone will port this to django and people running the Armstrong CMS will be able to use that as well.

    1. Thanks Chip. We also have an XML-RPC version of this, which might be useful for people using other CMSes, that I plan on open-sourcing soon.

  2. (How) do you add hyperlinks? If you add them in Google Docs, do they carry over to wordpress?

    I’m having visions of using Markdown along with Brett Terpstra’s ‘Markdown Quicktags’ plugin.

    This is way cool. Thanks for sharing! Seriously.

    1. Rick,
      We don’t strip out links when going between Google Docs and WordPress. We do strip out images (because we field those), comments and other extraneous formatting.

      1. Related to “We do strip out images (because we field those)”, is that a function of the extender or the core plugin? I can see plenty of situations where having the images brought over as well would be very beneficial.

        1. Please note, however, that the images will still be hosted on Google’s servers, and there may be performance implications for that (or not, depending on the speed of your server or if you’re using a CDN). My main point is that the images won’t be imported into the WordPress library, though a plugin to accomplish that might be forthcoming.

          1. Understandable, I guess until a plugin is built for such a purpose that having to manually re-add the images isn’t “too bad”. Thankfully at the moment this is all just play testing for me with no real purpose 🙂

  3. Hey William,
    This is really something amazing.

    When you say:
    “you’ll have to put the action on a page and point a cron job to it”

    In what template file do you suggest dropping the code? Can I just add it to the theme’s functions.php?

    Thanks for open-sourcing this.

    Rich

    1. You don’t want to put it in any of your template files or it will fire every time someone visits your website, which will add significant overhead. I’d suggest creating a file in your home directory (where wp-login.php is) called cron.php or something similar. Then you’ll need to add a line to cron that will go something like this:

      */2 * * * * curl http://mysite.com/cron.php

      That will hit the script every other minute.

  4. Great job on this plugin.

    We’re looking for this functionality, with a few additions. Our need is for a tool that will allow translators, reviewers and site owners to communicate easily.

    Google Docs, with the comment notifications and collaborative editing is perfect for us.

    Our workflow is:
    1) Translators use their CAT tools to translate offline.
    2) They upload to WordPress.
    3) The content is auto-exported to Google Docs and reviewers receive notification emails.
    4) Review and QA is done in Google Docs.
    5) The corrected document is auto-imported into WordPress.

    We’ll need to add the back-and-forth communication, as well as revision management to your plugin.

    I think that our additions would be very useful for your editorial process as well. We need it for translators and reviewers, but it’s basically the same for writers and editors.

    Would you like to collaborate on this?

    1. Amir,
      This seems a bit out of our scope, but I’d be glad to provide tactical support on integrating WordPress and Google Docs. For starters, I’d recommend importing the doc into Docs first instead of WordPress.

  5. This is a great idea. However, I need some help understanding how to get the Google Docs document into WordPress. I enabled the cron.php with the code you suggested earlier, put the command in crontab. I created a text file at Google Docs, but the article never appears in WordPress. (I enabled the Docs to WordPress plugin, too.)

    How do I configure both sides? I couldn’t find a configuration for the plugin. Is there a particular collection in Google Docs that is needed? Any help is appreciated.

  6. Hi,
    This is a great, pioneering open-source way of running a newspaper – thanks so much for sharing.

    I’ll be thinking about using a similar editorial workflow myself, but may wait until the cron job step is a little simpler (I’m more at the stage of ‘check the box’, click and go with WordPress 😉

      1. That was quick! 😉

        I’ll be suggesting this to the scientific journals I work for. One publisher is in the beginning stages of setting everything up. Unfortunatley, I don’t think I was successful in even getting them to adopt wordpress… now things are 100x more a) difficult; 2) clunky; 3) expensive than it has to be.

        Thanks again for sharing,
        Neil

  7. This whole setup looks immensely cool, and I’m hoping to try it with a site I’ve been working on.
    Quick question –
    Which Google Docs username and password do you use?
    And can you give some more info on the Google Docs setup – sharing, permissions, folders, users, etc?
    Thanks a lot for sharing all your hard work, and looking forward to trying it!

    1. At the BDN, we have one username and password that all docs are shared to that we use for authentication into Google Docs. Basically, that user owns the folder that we put docs in to send them to WordPress, and that folder is shared out to users so they can put docs in that folder. Don’t know if this helps at all.

    1. At the BDN, everyone has the same username in Docs as in WordPress — for example, stevenweathers@example.com corresponds to the username stevenweathers in WordPress. I think if the username is not the same the post will be attributed to the admin in WordPress, but I’ve not tested this.

          1. My username (minus @gmail.com) is the same as the username in WP, and I’m still not posting. I’ve rechecked the GDocs folder ID, as well as my own username and password, and I’m still getting nothing. Is there somewhere else I can check to troubleshoot? Thanks in advance for your reply, and thanks so much for making this plugin available. This will solve a lot of issues for me if I can get it to work.

      1. First off, killer concept. Thanks for sharing it with us all.

        Regarding the requirement to have Google Docs username be the same as WP username, this appears to be a showstopper for us, as we have pre-existing Google Docs accounts, some with different usernames than in WP.

        Seems like an easy enough fix for us, however, to associate someone’s Google Docs username with a custom field added to their WP profile, but in order to do this without modifying the core code of your plugin (so as not to break its upgradeability) we’d have to use a hook provided in the code. Any chance such a hook is included already, or planned for a future version?

        Again, great concept. Thanks!

        1. Jeremy,
          It doesn’t have a hook that you could effectively use right now, but the next version, slated for the next few weeks, will. You’d want to use the existing pre_docs_to_wp_insert filter, which currently passes $post_array right before the post is inserted into the database and will soon also pass $args, which will include the username from Google Docs.

          Will

  8. First off, this is fantastic. This gives me a great option for running the journalism class at the high school I teach at. I think I got everything to work except one thing. It’s using my slug as the title, not the headline in the body separated by the |.

    For example: I named the document ‘today622’ then put on the first line of the gDoc ‘Today is June 22nd|’ then I have the body of my text. When it pulls it into WordPress it takes the ‘today622’ as my title. Am I doing something wrong or should I just use the title in gDocs as my title?

    1. Chris,
      Separating the headline with a pipe is something we do at the BDN and isn’t included in the plugin, so I’d recommend using the gDoc title as your headline. However, if you’d like, I can throw the code we use to separate out the headline on github.

      1. That’s ok. Using the gDoc title will work fine for our operation. I just thought I was doing something wrong. This plugin is great! Thanks for sharing your talent and awesomeness with all.

      2. I’d like to get the code for separating the headline with the pipe. Using a slug that’s separated from the headline makes sense. The slug can stay the same through all versions, while the headline might change many times.

  9. Hi,

    This is great job you have done!Like this kind of “experimental” workflow.I set everything, work well except separating title with pipe.How to do that?Can you post that code on github?
    One more question: What do you mean by using slug as the title?Sorry for begginer question, but I never pay much attention to post slug feature of WP.

    Best regards

  10. First off, this is totally awesome! Thank you so much for sharing.

    Having come from an ‘open access background working with the publication First Monday in the past, this is great. Currently I’m moving a city library website to WordPress, and this is about the best thing I’ve found for helping folks comfortable working in Docs to create, edit and post items…..

    I’m not sure if you’re still reading the posts here or if you’re providing any sort of support, but this question might come up with more than one person out here in the digital scrubbrush – we’ve got a setup where we have 3 separate users/gmail logins creating posts for blogs on a multisite. I’ve defined one account in the wp-config, but would like to know if there’s some way to push the doc to a different blog on the multisite or if it will just push to the main site?

    Thanks again for a great great service – you’ve helped a lot of organizations come one step closer to making a CMS solution for non-techies a reality! 🙂

    1. AJ,
      Sorry for not replying sooner. It seems I shut off my email alerts by accident.

      This isn’t really a use case we’d planned for, but I don’t see why it couldn’t be done. You’d want to use the WP Cron extender. In your wp-config, you could have a conditional to detect what blog you’re on and therefore what collection to draw from. You’d want to have a different collection for each site.

      if might look something like this:
      if( $_SERVER['SERVER_NAME] == 'blog1.mysite.com' ) {
      define( 'DOCSTOWP_ORIGIN', 'origin1' );
      } elseif( $_SERVER['SERVER_NAME] == 'blog2.mysite.com' ) {
      define( 'DOCSTOWP_ORIGI, 'origin2' );
      } etc...

  11. Great great plugin. I am using this in one of my websites.

    The thing is that I am trying to use your plugin with wordpress multisite but still could not handle to make it work.

    The way I am trying to implement this includes these steps….
    1. Activate the plugin to a single blog in multisite
    2. Create a file named: update-from-gdocs.php under the current bplogs themes directory
    3. Paste in to the file the cron job
    4. Use Utopia cron to run the update-from-gdocs.php

    5…… Nothing happens!!!!!

    Any help will be much appreciated

    1. I’m not familiar with Utopia cron, but I’ll try to help.

      What are you putting in the update-from-gdocs.php file?

      1. update-from-gdocs.php

        docs_to_wp_init( ‘me@example.com’, ‘mypassword’ );
        //We’re just going to call one function:
        $docs_to_wp->retrieve_docs_for_web( $gdClient, ID of origin folder, ID of destination folder );

        1. You must require wp-load.php, which is located in your home directory. If the script is in /wp-content/themes/my-theme/, then you’d need either the file at ../../../wp-load.php. If the script is located in your home directory (the same directory as wp-config.php) then you will have to require the file at ./wp-load.php.

          Will

  12. Would there be any technical hurdle around a non-cron-based implementation of this? So that, e.g., when I create a new Google Doc and want to add it to WordPress, I can go to mysite.com/put_it_in_wordpress.php (or, better, a protected link inside wp-admin) and it will run the import?

  13. I have this working pretty good. It’s really cool. I work with community websites I’m not sure how best to use this yet. It’s a tool in the box waiting to be used. One thing I seem to be missing is this – how do we know who just submitted the story. For me it comes in anonymously (‘cept I know it’s really me logged on in another browser but you see what I mean)

    1. What do you mean by knowing who submitted the post?

      For some background information, the plugin assumes that usernames are the same in Google Docs and in WordPress. For people using Google Apps, the username of the email me@example.com would be example. For someone using regular docs, the username for the email me@gmail.com would be me@gmail.com.

      Will

  14. This is a great find. I’m thinking of implementing it for an online based news source. Have writers submit their pieces in one space with the ability to comment would be GREAT. For curiosity’s sake, do you have the entire company on google apps? Or are they under individual accounts, still?

    1. The entire company is using Google Apps. We’ve found a lot of success using Google Docs — the ease of use and number of features, including commenting and real-time editing, is fantastic.

      1. That’s a great idea – when I came across it a light bulb lit right up.
        Just the ability to comment on the stories, not within the actual text and a place to organize the stories seem like 2 reasons enough to use it.
        Have you found any apps that are particularly useful for organization within the industry?

  15. William,
    This is a great plugin, thank you for making it public.
    May I ask the best way to contact you via email, or if you would send an email to me?
    bryantjones(at)gmail.com
    Thanks

  16. We are very impressed with your ingenuity and interested in implementing – do you offer assistance to other newspapers. We are 18 weeklies in CT.

    Thanks and great work!
    Tracey Iaizzi
    IT Director

  17. Hi,
    thanks a lot for your plugins, they work very well !

    Quick question : I run a public Google Doc, modified each day. I would love to publish it on the website, but it’ll create a new post each time the cron run.

    Is there a way to re-publish a modified Google Doc to the same WP post ID ?

    Thanks a lot,

    Cedric

    1. Cedric,
      Sorry for the slow response. If you’re using the same Google Doc and just modifying it each day, it should update the same WordPress post. If that isn’t the case let me know here and I’ll look into what’s not working correctly in the plugin.

      Will

  18. Hi Will,
    thanks for your feedback – no matter the times it took 🙂

    You’re right it works perfectly !

    The solution was : don’t assign a folder “out” on Google Docs.

    Thanks again,

    Cedric

    1. Nick,
      You can put it anywhere, really. I recommend at the top, near the database declarations.

      Will

  19. Hi William,
    Cédric again.

    Images uploaded into Gdocs aren’t displayed on the blog – you can have a look here on WP (click on the big red button “Editez cette page” to go on Google Docs)

    Am I missing something to configure with the plugin ?

    Thanks a lot (again !)

    1. Cédric,
      Currently the plugin strips out images, though the next version of the plugin will include an extender to grab those images.

  20. Hi,

    very useful plugin, I needed it so much!

    but I saw that it do not copy the images from Google Docs, and I saw that it is a function of the extended plugin.. but I didn’t find that function there

    but I ask you, if I want to take a image from docs and automatically add to WordPress’s media, what should I do, just remove your function from ‘extended’ and what…?

    I really appreciate the work that you are doing,

    best,

    1. Cleber,
      The newest version of the plugin, which will be out in a few days, will include an extender to bring in images.

    1. Vicente,
      I haven’t experienced any problems using the plugin with Google Drive. Was the plugin working previously and then stop after you transitioned to Google Drive?

  21. Firstly, fantastic stuff you’re doing.

    I have implemented in a sandbox your plugin and it all works great.

    We often have stories/press releases that have their genesis in an email. I have added into gmail the labs extension to convert an email to a Google Doc, it can then be edited and placed into the Send to Publish collection.

    My question is this; are you aware or would you have a suggestion how I might automate the creation of a google doc from an email. I’m thinking a filter that has as it’s action create a document. How do you manage email->document process.

    Thanks

    Charles

    1. Glad to hear you’re seeing success with the plugin, Charles. Unfortunately I don’t have any suggestions for further automating sending emails to docs except for the Google Labs extension you’re already using. It’s quite possible you might be able to use Gmail’s API to automatically read the emails and then use the Google Docs API to send the docs there, but I’ve not tried it.

  22. Hi, William,

    We’ve been using Google Drive with Docs to WP Successfully for some time now. We have a folder called “Send to Web,” which sends the files to WordPress. Then the Gdoc files get moved to “Ready for Print” folder. Suddenly, though, the stories are moving to the “Ready for Print folder,” and also hanging up in the “Send to Web” folder. This seems to be causing some erratic behavior on the WordPress. Edits made in WordPress, such as style changes or slideshow additions, are disappearing, as though Docs to Drive is sending the file repeatedly. (And I’m sure it is because the file is stuck in the folder.)

    Do you have any clue as to why this is happening and how to remedy it?

    Thank you. Thank you. This is the best workflow we’ve had in years, despite the current difficulty.

    1. The Google Docs to WP plugin definitely still works with Google Drive and WordPress 3.6. Media Credit doesn’t work with 3.6 but I’m hoping to release a version soon that does.

  23. Mr. Davis,

    My name is Josh Slowiczek and I’m an aspiring journalist and current EIC for The Connection, a community college campus newspaper in northern CA. We’ve been looking into consolidating and streamlining our production process with a desire to turn ourselves into an online first publication.

    I’ve been reading over your blog and the content of your WordPress plugin page and am fascinated by the tools you have created to aid this process. I think it could truly help us in our goals.

    Unfortunately, of the group, I’m one of the more technologically inclined and what you’ve produced still goes WAY over my head. If you have some time, I’d appreciate any help or advice you could give me.

    I downloaded the most recent version Docs to WP with all of the attatchments. I activated the plugins and then contacted our site support site to embed the following code into our wp-config page.

    define( ‘DOCSTOWP_USER’, ‘beatreport.connect@gmail.com’ );
    define( ‘DOCSTOWP_PASS’, ‘XXXXXXXXXXX’ );
    define( ‘DOCSTOWP_ORIGIN’, ‘0B0Y-q7S-xMIpZVZyZGVfUHlNN2M’ );
    define( ‘DOCSTOWP_DESTINATION’, ‘0B0Y-q7S-xMIpd3JrNDZzdzNYY1U’ );

    I have a user account in our wordpress page that is ‘beatreport.connect’ and has permission to post as an ‘editor’

    The email account has access to a shared folder on drive which I have placed test documents into. Unfortunately, nothing showed up on wordpress.

    A few things I think I might have done wrong but don’t know for sure:
    1) Installed/Activated plugin before adding code
    2) beatreport.connect did not create the folder which it accesses

    I sent you an email though figured I should reach out on here as well. Any help would be really appreciated.

    Thanks!

Leave a Reply

Your email address will not be published. Required fields are marked *