Reference management software is an incredibly useful tool researcher and academic. It solves two problems that are otherwise very frustrating: (1) organizing and searching PDFs of journal articles, and (2) properly formatting a bibliography when writing a paper with citations.

Last week, I tried out several reference managers. (I had been using a combination of Zotero and Bookends, but both had some annoyances.) Here are my thoughts on each, and what I think you should consider if you want to start using a reference manager or are thinking about switching.

Update: July 10, 2015

I’ve found two new reference management options since this post was originally published.

The first is Paperpile. This is Chrome extension that integrates reference management with Google Docs. I haven’t had a chance to try it in-depth, but if it works as advertised, it may be the holy grail of collaborative reference management.

The second is Cite This For Me. This is a tool for generating references section entries automatically from an article title or a DOI. If you end up having to make a references section manually, this is a very easy way to do it.

Update: September 10, 2015

I have used Paperpile now, and it works pretty well. They have a beta Google Docs plugin that I use (rather than their Chrome extension; the Google Docs plugin is compatible with Safari, and I don’t like the security implications of browser extensions).

I’ve completely given up on organizing and searching my references in the same software I use to do citations. I now use DEVONthink for organizing and searching. DEVONthink has very advanced search capabilities, which solves my biggest reference management problem: finding papers that I remember reading, but do not remember the title or authors.

So I can’t speak to the reference organization and searching features of Paperpile, but inserting citations and building a list of references in a Google Docs file works just fine.

Paperpile’s killer feature is that it is built around Google Docs, which makes collaborative writing so much easier: everyone has the most current version of the document, and there is no manual merging of changes from multiple authors. The "suggesting mode" feature in Google Docs is now fairly comparable to track changes in Word, the bread and butter of collaborative academic writing.

Regardless of how you like to organize your references, I definitely think Parepile is worth a look just so you can use Google Docs. (Here’s how I use Google Docs with collaborators to avoid any hiccups with people who don’t regularly use Google accounts.)

Update: September 14, 2015

I just heard about Flow, from the Refworks people (thanks Peyton!). Like Paperpile, it is web-based and integrates with Google Docs. I have not tried it yet myself, so if you use it let me know what you think.

Zotero

I’ve been using Zotero to organize and search my references for more than a year, so this is the application I have the most to say about.

Zotero's interface is fairly bare-bones, and not Mac native. Most of the heavy lifting is done by the web browser plugin or in an external PDF viewer.

Pros:

• Free, cross-platform. Additional cloud storage is very cheap, and you probably won’t need it.
• Open source, maintained by a group at George Mason in a sustainable manner.
• Really good cloud sync (important for backups even if you don’t use Zotero on multiple computers).
• Good multi-user sharing features, including sharing of PDFs.
• Integration with Microsoft Word seems to work well (I haven’t used it a ton though).
• The PaperShip iOS and Mac app.
• The browser plugin will import citations and their accompanying PDFs from PubMed and many journal websites.
• Really fast full search of all PDFs in your library.

Cons:

• No good way I’ve found to track which articles need to be read, or mark articles for follow-up.
• Somewhat confusing reference organization capabilities (definitely read the documentation page on this before you try Zotero).
• Annotations in PDFs are kept separately from notes on a reference written in Zotero: you have to open the PDF to see them.
• Desktop application is based off of Firefox, so it’s more clunky than it would be as a fully native application. (Long-term, I think this reliance on XULRunner is a big liability. Users will increasingly expect native-feeling applications, which I doubt will be possible without a complete rewrite.)
• Web interface is mediocre.
• No internal PubMed search, ~~or way to add references without using the browser extension~~. Update: references can be quickly added by entering an ISBN, PMID, or DOI. This feature works great.
• Initial setup process can be very confusing for new users.
• No built-in PDF viewer/annotation. This works fine for me because I prefer to use the great PDF support built into OS X, but would definitely be a downside on Windows or for Mac users unfamiliar with Preview.app’s annotation support.

Bottom line: I think Zotero would be great if your reference management needs are lightweight, or you’re willing to put up with a few annoyances to use open source software. If you use notes and annotations heavily, or want read/unread/flagged status for references, look elsewhere.

Mendeley

Mendeley is probably the most widely-used modern reference manager. "Modern" in this context basically means that they have modern interfaces, take the web seriously, and have social features.

Mendeley has a nice multi-line display for each reference, which is much better than Zotero's single line. Unfortunately, the font is tiny and un-adjustable.

Pros:

• Best web interface of any reference manager out there.
• Cross-platform.
• Free account tier (limited storage).
• Widely used, especially among younger scientists.
• Sync seems to work well in my limited testing.
• Desktop interface is generally good.
• PDF notes displayed alongside notes taken in Mendeley, but viewing annotations still requires opening the PDF.
• Built-in PDF viewer has good annotation functions.
• First-time setup is easy.
• The PaperShip iOS and Mac app also works with Mendeley.
• Social features may be useful.

The notes feature in Mendeley is pretty good. It includes notes made in PDFs, but there is no way to display highlighted text from a PDF in this interface.

Cons:

• Owned by Elsevier, which has a terrible reputation for supporting open reserach. This is a dealbreaker for me.
• No easy way to import PDFs (i.e. no feature like Zotero’s browser plugin that will automatically import PDFs along with the citation). This is also a dealbreaker for me, because importing PDFs quickly saves a huge amount of time.
• Additional storage is relatively expensive ($5/month for just 5gb of space; for comparison, Apple provides 20gb for $1/month).
• Mac app feels non-native in the same way Zotero does. It does not follow standard Mac application best practices. For example, the font size in the main view is smaller than the system font and can’t be adjusted.
• Social features may not be what you want in your reference managers.

Bottom line: If you can look past Elsevier’s reputation and find a PDF import workflow that works for you, this is a good option.

EndNote

EndNote is probably the most widely used mature reference manager. By "mature," I mean that it’s been around forever. I used to use EndNote back in college, but it’s changed substantially since then.

Pros:

• Cross platform.
• Widely used, so there are citation formats for essentially every journal.
• Automatic PDF downloading. This feature is amazing when it works. You can select one or more references in EndNote, click a button, and it will automatically download the PDF into your library if you have access.

Cons:

• Mac application interface is atrocious in recent versions. Watch the first 30 seconds of this video.
• Costs $220 with education pricing (!!!).
• Web sync is very difficult to set up.
• Web interface is also atrocious.
• Software updates only go one version at a time, so I had to run six upgrades to get from version 7.0 to 7.2. I’m sure there’s a work-around, but this is inexcusable default behavior in modern software.

Bottom line: Zotero and Mendeley both have much better interfaces for much less money. I would try one of these first before trying EndNote. The only stand-out feature in EndNote is automatic PDF downloads, but Zotero’s browser plugin is as good as this in my experience.

Papers

Papers used to be Mac-only, but recently released new comparable versions for both Mac and Windows. They also have an iOS app and a beta web interface. I used to use Papers prior to the latest major release and liked it, but had a ton of trouble upgrading to the new version so I switched to Zotero.

Papers has a gorgeous interface for Mac (screenshot from papersapp.com).

Unfortunately, I still hit numerous stability and performance issues even just using Papers for a few hours. Considering this is their flagship app on their original platform, I can’t recommend any of their apps.

Pros:

• Hands down best desktop interface for both Windows and Mac.
• The Mac app is native and follows good Mac application design practices.
• Good built-in PubMed search interface, which can easily import PDFs.

Cons:

• Still lots of bugs, performance issues, and crashes with v3 of the Mac app.

The Windows interface for Papers is also great and is modern. Papers is the only application that seems to really care about keeping up with the design language of the newest versions of Mac OS X and Windows. (Screenshot from papersapp.com.)

RefWorks

This is a web application site-licensed by my institution.

Pros:

• LOL

Cons:

• Web-only, and the website feels dated by a decade or more.

Bottom line: Any application in this post is a better choice.

Mac-only applications

Bookends and Sente are also options if you use a Mac.

Bookends is a native Mac application, but I didn't find it to be that intuitive to use.

Bookends has a ton of customization options, I found that I needed to do a fair amount of customization to use the bibliography builder, and that the process for doing this was not very intuitive. (I used it because my word processor integrates tightly with it.) It also is clunky in a number of other ways, including a lack of cloud sync. I don’t think there are any big advantages to using it over Zotero or Mendeley.

Sente is also a native Mac application. I found the interface superficially fairly intuitive, but had to read the manual to understand more advanced features like tagging.

I really like Sente. It has my favorite sync system of any of the applications I tried and it is the only application I found that combines notes and PDF annotations in the main view. The UI is not as modern as Papers, but comparable in terms of usability. It has a steep learning curve for tagging and was difficult to set up with my library’s proxy server, but it has great documentation. If you can live without sync with Windows users, I think Sente is a great choice.

Final thoughts

I think Zotero is the best general purpose choice. But if you can live with Mac and iOS only, check out Sente.

If you can ignore Elsevier’s bad reputation and don’t mind slower PDF import, Mendeley is also a good option.

Note that Mendeley has an option to stay in continuous sync with Zotero. You can use this feature to try out both applications without having to import references twice.