Trying to debug EagleFiler failure to import mail attachments from Mail.app

[jimratliff]

I’d like to offload my years of legacy email into EagleFiler. To test the process, I used the F1 capture key to import one folder (and its many subfolders) of messages from Mail.app. The Errors log showed many entries that looked like:

Missing Apple Mail Message Attachment
100610.partial.emlx

Question 1: Is there a way for me to use the info in the error message (“100610”) to figure out which email message triggered the error message?

To investigate these error messages, I looked at some messages in EagleFiler that had attachments (using the “#” column), and tried to open those attachments by double-clicking that record. In some cases that worked (i.e., Mail.app opened EagleFiler’s copy of the message and then I could open the attachment).

However, in other cases when Mail.app opened EagleFiler’s copy of the message, the attachment showed up as an icon with a downward-pointing arrow, indicating I needed to download the attachment from the server.

• When I clicked on the downward-pointing arrow, the icon changed to have a progress bar, and blue text “Downloading…”.
• The “Downloading…” indicator never went away: EF never successfully downloaded the attachment. This despite the fact that:
• When I located the original message in Mail.app, it DID have a fully downloaded and available and openable attachment.

Question 2: What can I do about these attachments that (a) aren’t being correctly imported by EF even though (b) they are fully downloaded and openable from my original emails?

I note that my problem sounds like it might be related to a thread “Mail macOSX Sierra Import messages script problem?” from September 2016. That problem was supposed to have been fixed by EagleFiler 1.7.2. (I’m running EF 1.76.)

I’m running macOS Sierra 10.12.5.

Thanks!
Jim

[Michael_Tsai]

Question 1: Is there a way for me to use the info in the error message (“100610”) to figure out which email message triggered the error message?

Please see the Importing Attachments section of the manual. In brief, what you probably want to do is Option-double-click the Reveal button in the Errors window to open Mail’s copy of the messages. This should get it to download the attachments. Then if you capture the messages again, the attachments will be there for EagleFiler to import.

To investigate these error messages, I looked at some messages in EagleFiler that had attachments (using the “#” column), and tried to open those attachments by double-clicking that record. In some cases that worked (i.e., Mail.app opened EagleFiler’s copy of the message and then I could open the attachment).

I think that just means that you happened to pick messages that didn’t have errors in the first place.

However, in other cases when Mail.app opened EagleFiler’s copy of the message, the attachment showed up as an icon with a downward-pointing arrow, indicating I needed to download the attachment from the server.

• When I clicked on the downward-pointing arrow, the icon changed to have a progress bar, and blue text “Downloading…”.
• The “Downloading…” indicator never went away: EF never successfully downloaded the attachment. This despite the fact that:
• When I located the original message in Mail.app, it DID have a fully downloaded and available and openable attachment.

EagleFiler does not connect to your mail server or download any messages. Mail does not know how to download attachments into EagleFiler’s copy of a message.

Question 2: What can I do about these attachments that (a) aren’t being correctly imported by EF even though (b) they are fully downloaded and openable from my original emails?

You need to make sure the messages are fully downloaded by Mail before importing them into EagleFiler.

I note that my problem sounds like it might be related to a thread “Mail macOSX Sierra Import messages script problem?” from September 2016. That problem was supposed to have been fixed by EagleFiler 1.7.2.

That thread is about EagleFiler not being able to find attachments that already were downloaded. I think you are in the normal situation where EagleFiler can find the attachments if they are there, but some messages have them missing either because Mail purged them to save space or it never downloaded them in the first place.

[jimratliff]

Please see the Importing Attachments section of the manual. In brief, what you probably want to do is Option-double-click the Reveal button in the Errors window to open Mail’s copy of the messages. This should get it to download the attachments. Then if you capture the messages again, the attachments will be there for EagleFiler to import.

Michael, thanks for the quick reply!

In your instructions above, you say “Option-double-click the Reveal button in the Errors window to open Mail’s copy of the messages.” In the documentation you cite, it says “The easiest way to do this is to select the error(s) in EagleFiler and Option-click the Reveal button.” (Note that in your response, you say Option-double-click, but in the documentation it says “Option-click”, the Reveal button.)

In either case, selecting the “Missing Apple Mail Message Attachment” entry in the Errors log and either (a) clicking on the Reveal button in the toolbar, (b) Option-clicking, or (b) Option-double-clicking on the Reveal button in the toolbar isn’t doing anything productive. (When I click on Reveal, it shifts me to Finder, but doesn’t open a window. When I option-click or option-double-click, it leaves me in EagleFiler but doesn’t open any window.)

I’m sure this isn’t the intended behavior, so I’m puzzled.

Thanks in advance!
Jim

[Michael_Tsai]

In your instructions above, you say “Option-double-click the Reveal button in the Errors window to open Mail’s copy of the messages.” In the documentation you cite, it says “The easiest way to do this is to select the error(s) in EagleFiler and Option-click the Reveal button.” (Note that in your response, you say Option-double-click, but in the documentation it says “Option-click”, the Reveal button.)

Option-click is correct. Sorry for the confusion.

In either case, selecting the “Missing Apple Mail Message Attachment” entry in the Errors log and either (a) clicking on the Reveal button in the toolbar, (b) Option-clicking, or (b) Option-double-clicking on the Reveal button in the toolbar isn’t doing anything productive. (When I click on Reveal, it shifts me to Finder, but doesn’t open a window. When I option-click or option-double-click, it leaves me in EagleFiler but doesn’t open any window.)

Reveal should show you the file in the Finder, which you could then double-click to open in Mail. Option-click should directly open it in Mail, although perhaps it won’t if the Mac has .emlx files mapped to another app. Does it open in Mail if you double-click from the Finder?

[jimratliff]

Reveal should show you the file in the Finder, which you could then double-click to open in Mail. Option-click should directly open it in Mail, although perhaps it won’t if the Mac has .emlx files mapped to another app. Does it open in Mail if you double-click from the Finder?

When I click on Reveal, it switches to Finder and displays the directory:
~/Library/Mail/V4/MailData

In other words, it doesn’t display a specific file.

In case it’s helpful, here are the contents at the root level of that directory:

• BackupTOC.plist
• DefaultCounts
• Envelope Index
• Envelope Index-shm
• Envelope Index-wal
• ExternalUpdates.storedata
• ExternalUpdates.storedata-shm
• ExternalUpdates.storedata-wal
• FlagMailboxes.plist
• LSMMap2
• OpenedAttachmentsV2.plist
• RulesActiveState.plist
• Signatures
• SmartMailboxesLocalProperties.plist
• SyncedFilesInfo.plist
• SyncedRules.plist
• SyncedSmartMailboxes.plist
• UnsyncedRules.plist
• VIPSenders.plistBackupTOC.plist
• DefaultCounts
• Envelope Index
• Envelope Index-shm
• Envelope Index-wal
• ExternalUpdates.storedata
• ExternalUpdates.storedata-shm
• ExternalUpdates.storedata-wal
• FlagMailboxes.plist
• LSMMap2
• OpenedAttachmentsV2.plist
• RulesActiveState.plist
• Signatures
• SmartMailboxesLocalProperties.plist
• SyncedFilesInfo.plist
• SyncedRules.plist
• SyncedSmartMailboxes.plist
• UnsyncedRules.plist
• VIPSenders.plist

I don’t see anything here that would have the actual messages in them.

[Michael_Tsai]

When I click on Reveal, it switches to Finder and displays the directory:
~/Library/Mail/V4/MailData

In other words, it doesn’t display a specific file.

I haven’t seen that behavior before. Perhaps the Finder is encountering an error showing it. It should be showing you the actual file, which definitely exists because that’s how EagleFiler found the filename displayed in the error. If you hover your mouse over the error, there should be a tooltip showing the exact path of the file in question. Are you able to find the file manually?

[jimratliff]

I haven’t seen that behavior before. Perhaps the Finder is encountering an error showing it. It should be showing you the actual file, which definitely exists because that’s how EagleFiler found the filename displayed in the error. If you hover your mouse over the error, there should be a tooltip showing the exact path of the file in question. Are you able to find the file manually?

There is a tooltip, which I was able to capture in a screengrab. When I then trace out the path in the Finder (showing package contents when necessary), I get as far as:

{path partly redacted} Archives/EagleFiler_archives_libraries/Email_library_EagleFiler/Email_library_EagleFiler.eflibrary/Temporary Items.nobackup/EagleFiler-2017-06-29-sk23UdrxRc6pMsUG0usDuQ/8

And that directory is empty. The rest of the tooltip says that within the “8” directory there should be an additional hierarchy (e.g., “Legacy folders/Voice Mail.mbox/DE4962A3-6930-463A-95EE-D9E75A9DB72B/Data/0/0/1/Messages/100687.partial.emlx”), however there’s nothing there.

Since this was just an exploratory exercise by me, and I still have all the original emails still in Mail.app, I think I’ll just start again from scratch, but biting off smaller pieces for testing. Perhaps I’ll have entirely better luck just rebooting and starting over.

[Michael_Tsai]

When I then trace out the path in the Finder (showing package contents when necessary), I get as far as:

{path partly redacted} Archives/EagleFiler_archives_libraries/Email_library_EagleFiler/Email_library_EagleFiler.eflibrary/Temporary Items.nobackup/EagleFiler-2017-06-29-sk23UdrxRc6pMsUG0usDuQ/8

And that directory is empty.

I think I see what’s happening here. Thanks for pointing the way to a bug. There are some cases where instead of importing directly from Mail’s data store, EagleFiler will create a copy in a temporary folder and import from that. After the error occurs, it cleans up the temporary folder, so that’s probably why you don’t see the file.

I think if you temporarily rearrange your mailboxes so that you are only importing from mailboxes that have no submailboxes, you can avoid this code path and it will work.

[Michael_Tsai]

Thanks for pointing the way to a bug. There are some cases where instead of importing directly from Mail’s data store, EagleFiler will create a copy in a temporary folder and import from that. After the error occurs, it cleans up the temporary folder, so that’s probably why you don’t see the file.

I think this is fixed in the latest public beta (1.8b5).

[Michael_Tsai]

I think this is fixed in the latest public beta (1.8b5).

The fix has now shipped in EagleFiler 1.8.