C-Command Software Forum

Q. for users about EF and research papers

Given the recent question in the forums about Bookends, I’m curious how many people are using EF to help organize research papers and/or dissertations. (I’m a journalist, but our needs would be similar.)

I’m finding EF unparalelled at keeping track of lots of information, and searching it. I chose it over the much-hyped DevonThink. (Especially once I discovered Skim and so could handle PDF’s better.) I’m not yet sure whether the note-taking function is ideal for reviewing and organizing notes, though. I use Word’s Outline view for final organization, but that means cutting and pasting out of EF’s notes, which is less than ideal.

I notice that users of Zotero, designed specifically for academia (and with bibliographic functions I don’t care so much about), have the same concern (organizing notes). Oddly, with all these great programs, there are still things for which index cards are superior (organizing things chronologically, moving them around into an outline).

Anyway, I’m curious whether and how people are using EF for writing papers that draw on massive amounts of notes.

I plan to, as soon as EF can index the contents of folders external to its database. At that point I will switch completely from DEVONThink. (Another feature I would like to see implemented, soon, in EF is rankings of search results.) My workflow is to create notes in an RTF file which is attached to a reference in Bookends. That RTF note, as well as the reference’s PDF (also “stored” in Bookends) are currently indexed in DEVONThink.

EF and research papers
Thanks – I don’t use bibliography software, so that’s not an issue for me. I haven’t yet figured out the best way for creating and sorting notes on EF, on the index card model. My workflow remains a work in progress.

I love the organizational aspects of EF, though. I may end up mainly using it as a reference manager. then cutting and pasting into an outliner.

workflow for research papers

If you’ve got a minute, I’d love to hear what your workflow is – bibliography excluded. As I said, EF works great for me as a searchable repository for my pdfs, web clippings, notes, and emails. But turning that into something akin to the old 3x5 notecard system is stumping me. I suppose I could create lots of rich text notes that are short, containing one thought or a pithy quote, and tag them. But I find myself getting bogged down as I construct this system and then, as a deadline looms, retreating to a paper system. (Printing out longer notes, searching them for quotes and facts, then putting them into a Word outliner. I’m even just typing them in.)

Anyway, just looking for tips, as my shift toward some degree of paperlessness has hit some snags.

I doubt this is an Eaglefiler problem, as I see similar queries in the Zotero forums.


Have you had a look at Scrivener?

Not to replace EF by any means, but as a companion. I find it a nearly ideal work environment. And you’ll find index cards there.

Maybe I should give it another shot. I tried using Scrivener as a combination document-manager and word processor, and the document management came up short. But I could use EF as the repository, and Scrivener as the organizer.

I don’t want to pile up too many programs though – ultimately I need to use Word, and EF > Scrivener > Word seems like overkill. (But maybe I’m wrong.)

To each her own, of course, but after much wandering I’ve come to the conclusion that ‘more is better’—not in principle but in this case.
Scrivener has a very generous trial policy and is not expensive. In the end, I think these programs work together well. I use EF to manage my Scrivener projects. Yes, you need a word processor, too.

For simple writing tasks Scrivener is unnecessary, but for writing projects that call for many notes, sources, etc. it is brilliant. And your mention of index cards made me think of suggesting it.

I certainly haven’t ruled it out – thanks.

I have yet to design an ideal workflow. I have a very large number of texts (PDFs, text files, notes and clips from the web) that I keep in several DEVONthink Pro databases. These are dumped in with little or no organization. These databases also index text in files located in the Finder, including a large repository of PDFs managed by Bookends. The notes are small text-only or RTF files, and include “metadata” such as the source of the material and a date stamp.

DT’s strong points are its flexible data capture options, and unparalleled search engine. Its main deficiency, for me, is that it is not an intuitive environment to map and store relationships (“links”) between data. I also find it difficult to write within the database, as I can in Scrivener (see below). Others disagree (see, for example, this post, or this one .) My criticism is somewhat unfair, though: DT is primarily a data repository and not a writing environment.

However, as soon as EF can index external files, and adds the ability to sort search results by relevance, I will migrate all these data to EF. I am much more comfortable with EF’s storage system, which is, after all, what a database is all about.

I write (research articles and grant applications) in Scrivener. If I am working on a relatively small, one-time writing project, I migrate relevant notes to the Scrivener file I am working on. I data-mine in DT, and export relevant data to Scrivener’s Research area.

If the project is large, or I predict that I might revisit its topic, I arrange data in another application. This is where I am still uncommitted to a workflow. Tinderbox is looking more and more appealing, and I have several ongoing projects as TB files. However, it is not ideal. Importantly, TB cannot search in or link between several files. Since large files are cumbersome to work with in TB, I need to use several, smaller projects file. The inability to associate among them is a major deficiently in TB.

Interesting – thanks.

This is only partly related, but I recently wrote a blog post in how I review research papers, mainly using Skim and EagleFiler. It’s here: http://alan.petitepomme.net/blog/2009/04/paperlesser-reviewing-papers.html

This is interesting. Thanks.

I’m wondering one thing.
You write—and EF Manual seems to confirm—that EF “lets me easily create a PDF bundle out of a PDF, it is able to display the annotations in its viewer pane, and it can search both in the annotations as well as in the PDF text.”

I don’t think EF can display more than the first page of a PDF (of PDFD). At least I don’t know how to make it do so.

Are you able to view notes attached to pages other than the first?

(EF is quite able to search these notes, of course.)

EagleFiler can display the entire PDF. You can change pages using:

  • Page Up/Page Down
  • Up/Down
  • The PDF commands in the View menu
  • Control-click on the page and use Next/Previous
  • Control-click on the page and set it to Continuous, then scroll using the scroll bar

Thanks, Michael.

I said this quite badly.

What’s awkward is finding—so as to view—a note on any page other than the first.

EF indexes these just fine and reports that the searched term is in the PDF somewhere, but if it’s in a note on page 17, you’ll spend a lot of time finding it. Or so it seems to me.

Now that I reread the post—" it is able to display the annotations in its viewer pane, and it can search both in the annotations as well as in the PDF text"—I see that brab doesn’t actually say he can search AND view the notes as a coordinated act. Eventually one can view the notes, yes, but not in a manner efficient enough to make it useful.

Or am I missing something? I’d love to find that it’s easier than I thought. (Of course one can open the PDF in Skim and search there.)

I don’t think there’s currently a way to search within a PDF for Skim annotations, except using Skim. However, if you double-click the PDF to open it in Skim, the query should transfer from EagleFiler to Skim.

(ok, this turned into quite an epistle, but here it goes anyway…)

CTS, if you’re still keeping an eye on this thread, could you elucidate on how you use (physical) index cards? Perhaps a short description of why this organizational model appeals to you can help us think of an app that best approximates what you want to do.

My own paper-writing workflow has been in flux for quite some time. Before I came across EagleFiler, I would just have a filesystem folder with the source material, an OmniOutliner document (with separate sections for notetaking and paper structure), and ultimately a Word document when the time came to compose. During my senior year I experimented with LaTeX/BibTeX and the (partial) integration offered by Scrivener in that regard. However, I didn’t really have the time to get the process streamlined, so I just ended up frustrated. Now that I will be heading to grad school and have apps such as EagleFiler at my disposal, I have a renewed interest in developing a system that works. Here are my initial thoughts on the components of such a system:

  • EagleFiler will play the role of long-form document storgage. Any complete document (a blog post, an article, a pdf report from a government agency) will get stored in EF. The fact that I can full-text search through a 10,000-entry library near-instantaneously makes EF the natural pick for this repository-type use.

  • DevonThink for storing one-off bits of information (facts, data points, quotes, etc.). From what I understand, DevonThink’s matching algorithms work best when the individual entries in its database are small in size, so I think this will best let DT do what’s designed to do. I’ve considered storing the factoids in EagleFiler, but there is just something semantically offputting about having tiny factoids clash with hundred-page documents in the same database. That said, this is just a personal preference and perhaps with a meaningful effort to tag and organize stuff intelligently, one could avoid this problem.

  • OmniOutliner for note-taking and document organization. This has been the one consistent part of my workflow since 2005 (when I started paying attention to the writing process), so I guess it works for me :).

  • (Insert app here) for the actual document writing. This will probably be Pages (with .doc export), but I still don’t know whether to focus on using a normal word processor or LaTeX. I envision that I will continue to use a combination of Word, Pages, LaTeX, and maybe even Scrivener depending on the context of any given project. As such, I have come to expect my actual writing app to be fully separate from my notetaking/document organization app(s).

So, that’s how I’m thinking about approaching it at least. CTS, I hope that this description has helped you think about your workflow at least somewhat, and I look forward to hearing about your notecard system! :slight_smile:

notecards, etc.
Thanks for replying. I was using notecards mostly as a metaphor – I sometimes use them when I’m really stuck with organization, but usually have a more rough-and-ready system.

My archaic system as of last year: I had printed copies of just about everything: Notes from books (on legal pads), pdfs, transcripts of interviews. I’d number the pages of those and give each document an identifying letter. Then, again on paper, I’d do a rough outline, using a code "A32: quote on effects of plan: ‘they were incred.’] I sometimes would literally use scissors when I had second thoughts about organization. I was overwhelmed with paper.

Now I basically do what you do, but just with Eaglefiler and Word. I use EF both for documents and for smaller snippets. (Aside from tags, I use internal tags like “@goodgeneralquote” or “@07041776”) I then pull the stuff I want to use into a Word outline. I don’t use index-card sized notes in EF but tend to have, say, one longish RTF note per interviewee, or per academic article. I use Skim for pdfs.

I still print some stuff out, if I get sick of staring at a screen, and just for variety, and I will do multiple iterations of the outline. (And also do some outlining by hand.)

I’m sure OmniOutliner is better than Word’s outliner, but I like the idea of minimizing software (and cost). Feel free to tell me it’s absolutely worth the $80 or whatever.

By the way, check out this picture (scroll down): http://www.danbaum.com/Nine_Lives/Margaret.html

What I’m looking for his a program that would do that for me. (Or an assistant to do that for me!) I suspect the task is difficult regardless of software – or even whether you use a computer.

Thanks for the insight!

As for OmniOutliner, check your Applications folder; the standard version has been bundled with new Macs for a long time (I’m assuming they still do? My latest Mac is from early 2006). Regardless, I’d give it a try… for me at least, it’s one my all-time favorite Mac apps.

Closest thing I can think of is software-wise is Curio, but if you value being able to see everything at once, I think we’re still stuck with paper for achieving that ‘big board’ look.

index cards
Check out this forum in Curio – right on point. (My problem with Scrivener’s notecards is that they are more like section headings – too big. I like the idea of manipulable info-nuggets, which would force the writer to boil down thoughts.)


No Omnioutliner on my '07 aluminum iMac …

Question about metadata
I’m experimenting with EF in my research workflow, which is mainly based on pdfs and pdf images. I’ve been storing content with Papers for some time, but I don’t like its closed model for tagging, and based on Michael’s commentary over at the OpenMeta community site I can see why they wouldn’t necessarily see OpenMeta as a viable way to get to open tagging, since in some real sense ‘open’ tagging doesn’t exist yet.

So as an ex-Papers user, I’m looking for research journal sorts of simple metadata management. To this end, I’m using the Title field for the paper title, and the From field for a minimal bibliographical reference (“The Historical Journal, Vol. 10, No. 1. (1967), pp. 89-105”). Tagging is fabulous (the reason I’m considering switching from Papers) and search is also excellent. But I’m also using Bookends in order to in effect extent EF’s bibliographical metadata—as I crank a paper into EF I move it into Bookends also, only for bibliography, and paste a Bookends entry URL for the work back into the EF Notes field.

What I’m curious about is whether there is a more EF-centric way to do this sort of workflow. I’m not against using two tools—multiple tools is kind of a standard condition of the universe, as far as I’m concerned—but I’m new to EF and want to make sure i’m exploiting it fully.