C-Command Software Forum

Can EF import a 65Gig .mbox via Drag and Drop?

EagleFiler
, , ,
#1

I know this is crazy but … I have a client whose Trash Mailbox is about 1,000,000 messages and about 65Gigs. She is buying a new machine (because she has to send her current 2017 MBP in for keyboard nonsense, grrr) and I will be tasked with the migration.

I probably won’t use Migration Assistant, for a few reasons, and plan to set up the new machine and the Mail account and then Import Mailboxes from the old machine–she doesn’t have many.

I want to avoid migrating/importing this mailbox to the new machine. I’d like to drag and drop this “Deleted Messages.mbox” to EagleFiler and then delete it in the Finder (or just not import it to the new machine.

Do you think this is possible? There may be tens of thousands that have pdfs attached to them.

I read in the User Manual: “For better data fidelity it is recommended that you import the mailbox back into Mail and then capture the messages from Mail to EagleFiler, rather than importing archive mailbox to EagleFiler directly”. Not sure if that applies to Archived mboxes or Live mboxes.

The Trash Mailbox, within the Mail.app is simply un-navigable. We cannot even select a few hundred messages to empty/get rid of. Her Time Machine backups fail frequently. It’s a bomb waiting to go off (has been for years!)

Bottom Line:

  • She wants access to search that Trash Box for a message she might have received years ago. I can’t just throw it away.
  • I want that Mailbox out of her Mail.app

Can EagleFiler come to my rescue?

#2

I don’t think I’ve tried a single mailbox quite that large, but it should work. Make sure that you have lots of free disk space because EagleFiler will be copying all of the message data at least once.

That only applies to archived mailboxes that you’ve exported from Mail. Live mailboxes use a different format that supports full data fidelity.

That’s OK. For a large number of messages, importing by dragging and dropping the mailbox file from the Finder will be much faster than using the capture key from within Mail, anyway.

#3

This is great news.

It’s not clear to me if she will need to keep EF running and use it to search that Trash Can. Or is it the case that once EF has saved the mailbox that a Spotlight search would find things. I’m assuming that once we get the mailbox into EF, that searching just within Mail won’t find the messages we’ve eliminated from Mail.

I ask these things because if she does me the favor of letting me get it out of her mail, if I have to introduce a new workflow to her she might hesitate.

Clients, right? lol

Thank you for your time

#4

By default, EagleFiler stores the entire mailbox in one file, as this is much more efficient. The downside is that the messages are not accessible to Spotlight so you would need to view/search them from within EagleFiler. You can drag/drop the messages out of the mailbox to have EagleFiler extract them as individual files (which Spotlight will index, like it does for Mail), but I don’t recommend that due to the overhead of a million separate files.

Right, once you delete the mailbox from Mail you would not be able to access the messages there anymore.

#5

Thank you for the responses. I hope to persuade my client to give EF a try.

As far as the licensing goes, will I be able to use it on the old machine to import the mbox into EF and then transfer the app and the EF library to the new machine?

Is there an easy way to move the Library from one machine to another?

#6

Yes.

Yes, please see this page.

#7

Great! We’re going to do it. I’ll let you know how well it handles a 65Gig mailbox!

#8

Greetings Michael,

I’ve hit a hurdle that might or might not be surmountable:

I successfully (probably, see below**) imported two mailboxes (9Gig and 55Gigs, 1,183,420 messages)
Performance was really slow. I noticed it was “indexing messages”
I moved the Library to the new machine (I’m sure before it was finished indexing)
It opened fine on the new machine with poor performance (takes about 20-30 seconds to open, starting a search accepts/stops on the first typed letter but eventually completes a successful search(?) amid blue beachballs) in about 15 seconds.
I let it try to finish indexing over night and woke up to “EagleFiler has unexpectedly quit”
Now, when I try to let it index messages it will get about halfway through and the program quits.
If I cancel the indexing the program appears to be stable, and searches appear to work, albeit slowly, but without a full indexing I’m unsure if they are complete and accurate

**The original “Deleted.mbox” was 99Gig in size. Inside it were two Folders (roughly 50Gigs apiece) named with those long alpha-numerical strings. But one of them ended in “.noindex”. I think that might be a remnant of when I tried to “Archive” the Trash mailbox from within the Mail.app. I’m not really concerned about that part of this but mention it in case it matters.

We’re very close to success. I’m hoping that if we can get the indexing to complete the performance will improve?

Your thoughts?

#9

It should be faster once the indexing has completed. Until then, exact searches will search everything but Anywhere searches will only search what has been indexed.

Could you please send in the crash log files so that I can see why it’s crashing?

#10

Thanks for sending the crash log. It looks like the crash was caused by a macOS bug (in the XML processor), but you may be able to work around it by clicking this link, which will tell EagleFiler to avoid that code path.

#11

I ran that link on the old machine and am letting it index. It’s got a few hours to go but I’m curious: if it successfully indexes it on this old machine can I then move this indexed EF library to the new machine as already indexed? Or will it need to be run and indexed on the new machine?

I figured I’d run it on this old machine to see if it works first. Which makes me wonder if this is a significant data point: I moved the library from this old machine to the new machine (Sierra to High Sierra) before it ever finished an indexing.

#12

Aw shucks. It crashed. Have we reached the end of the line?

#13

Yes. Also, if you restore a library from backup you won’t have to let it index again.

No, please send in the new crash log so that I can see what caused the crash.

#14

I’m sorry, it looks like I sent you the wrong link before. It should be this link to avoid the crash.

#15

I think this is going to work. It’s been running for about 5 hours and is about 3/4 of the way through. I don’t think it’s ever gotten past half way before. It says about 2 hours remaining, but it’s been saying that for an hour or so. I think it will be several more hours.

#16

It worked!

The performance is about what’s to be expected of opening and searching a 65Gig text file. As mentioned, when you type a word for search, it only registers the first letter and then will show the results for that letter, beachball for a moment, and then show results for the word. It might be a hard sell but I’m not aware of her experience searching when it was all in the mail.app. I’m happy to have it out of her mail program, so we’ll see.

Michael, I really appreciate you taking the time to bring this to fruition.

#17

You can avoid that by typing faster or by using the SendsWholeSearchString esoteric preference to make it wait until you press Return before it starts searching.

#18

Pretty cool. I turned off calculate folder sizes and backup metadata in addition to that. Feels a little better.

#19

Man I was on the edge of my seat on this thread. Brilliant. That is certainly a large file of any type.

Good job