I have a large email database in an EagleFiler library. When I search it, sometimes, when I click on a search result (an email in the results list), EagleFiler SWOD’s for long periods of time, 30-60 seconds. Any action in the app will cause it to SWOD again so long as that email is selected in the results list. If I send a few mouse clicks to a different email while it’s SWODing, it will eventually un-SWOD, and the other email will get selected, and then the app is responsive again. I took a sample of the EagleFiler process in Activity Monitor. Can I send it to you? (assuming MIchael Tsai is reading this)
Yes, please send the sample log to eaglefiler@c-command.com.
Thanks for the sample. It’s unfortunately not clear what’s causing this. It doesn’t look like EagleFiler is busy doing anything. Did you happen to have View ‣ Message ‣ Use Quick Look selected? Does it help if you instead select View ‣ Message ‣ Rich Text?
If not, please send an error report, and perhaps that will shed more light on what’s happening here.
Attached is a slightly sanitized EagleFiler Diagnostic report (I deleted a few of the files you collected in there, as it’s a bit invasive how much data is gathered), and another process sample.
View > Message > Rich Text was selected. I also tried > Plain Text. No difference. Same SWOD behavior.
Does enabling View ‣ Message ‣ Use Quick Look help?
Are the e-mails stored as mailbox files or as individual .eml files? Is it always the same ones that cause the hang? If so, are you able to export one of them?
Yes, ok, using Quicklook avoids the SWOD. Now I can open the email.
In EagleFiler, with QuickLook enabled, the header is shown when I click on it in the search results list, but none of the body content.
When I double click the email in the search results list, it opens in Mail.app, and I see that the email contains an attachment which displays as a (very) long string of base64 encoded text.
And now, as I write this, I see that if I leave the email selected for about 20-30 seconds in EagleFiler, the body content eventually gets displayed in EagleFiler’s preview window using QuickLook (bottom right, below the search results)
But if I switch from EagleFiler to my browser, and then back to EagleFiler, the body content disappears, and I have to wait 20-30 seconds again for it to display again.
If I touch-scroll the body content (using a Magic Trackpad), it scrolls and I can see the email contents, but if I actually click on the vertical scroll bar and drag it, the content disappears, and I have to wait for it to be displayed again.
The email at issue is in an mbox. The mbox is about 560 MB. I’m not sure how big the email is. I can’t share it with you (it’s a company email with proprietary client data in it).
There may be other emails where I’ve encountered this issue.
Perhaps the thing to test is an email containing attachments, within an mbox. The attachment in this case was a ZIP file.
This EagleFiler Library contains about 9 years worth of company emails.
I exported the emails from old MS Outlook 2011 because the company switched from MS mail server to Google enterprise email.
I had the 2rd largest email box at the company, second only to the CEO (or so they say), and the IT staff blew me off and wouldn’t prioritize migrating all my data into Google because there was so much they had problems.
So…I made it your problem 
by buying EagleFiler and importing it all into a Library. 
For the most part it works, but I’m obviously stressing it. When I select the “Records” node at the top of the Library, and then select all items, after a long delay, EagleFiler reports about 383,000 records.
I just needed an intuitive way to get all this stuff into a database that I could easily search on occasion. My email archive is a knowledge base.
I messed around with some other app, I think it was called Mail Steward, which could export all the email into a SQL database, and it worked, but it also had several issues that were showstoppers.
EagleFiler is powerful, but it lacks some functionality to filter vast search results. You know how in MySQL Workbench you can click the popup menu in a table query and select a filter for the # of search results that get returned (ie 10, 100, 1000, etc), to limit the time/bandwidth on queries? Something like that would be very handy when I’m browsing this database/library…
It sounds like the message is malformed if you’re seeing Base64 data and it’s not displaying normally and quickly with either EagleFiler’s engine, Quick Look, or Mail. If you encounter this with a message that you can share I’d be happy to take a look. I’m also in the process of rewriting EagleFiler’s message engine so that it will be more robust, faster, and easier to diagnose problems. That may “automatically” fix the problem.
Normally, that should be fine. There must be something special about this message or the way Outlook exported it.
It may take a while to load them at first, but thereafter I would expect interacting with them to be pretty fast.
What kind of queries are you doing? I just did an Anywhere search on 465K messages, and it took about 3 seconds to return 12K results, so I don’t see how pagination would be helpful here.