Hi,
I have huge mailboxes that I need to clean up.
For example over 37K emails close to 1Gb - I run the script Remove Duplicate Messages (Remove Duplicate Messages - EagleFiler AppleScripts), however, it always times out.
MacBook Pro M2 w 96GB Ram and 4TB SSD - doubt that machine is a problem
Any help is greatly appreciated
Thank you so much!
How long does it run before timing out? Itâs set to run for up to 24 hours, and for 37K messages I would expect it to finish within just a few minutes. So I wonder if itâs timing out in an unexpected place. If you run the script from within Script Editor, you can use View ⣠Show Log, and perhaps then it will show you which command timed out and give us a clue as to whatâs happening.
It runs for about 30 min or so
thast all it shows
error âAppleEvent timed out.â number -1712
Now i am trying to extract 10k messages and run the duplicate remover in a separate folder. i am not sure if that will even work
Takes a long time, but maybe it will work
You may need to click on Events at the bottom of the window before you click Run.
What do you mean by âextractâ?
Events are empty -
Extract - created a separate folder and moving the messages over to it. then will run a duplicate in a new folder
â
Have a better idea or maybe how to do it faster?
Oh maybe a novel idea - run the script only on âselectedâ emails? is that even possible?
Events should not be empty if you open that part of the window before running the script. I just tested this on my Mac. You can also click the clock icon at the right to see the events and replies in the separate Log History window.
Separating the mailboxes into per-message files will be slower. Also, the script in question operates on mailboxes rather than on individual messages. Thereâs a separate script for removing duplicate files.
No, it operates on the whole mailbox file. But it should normally be plenty fast and not time out, so I think we just need to figure out whatâs causing the unexpected timeout.
When I started the process at 9:50 AM this is what I
tell application âEagleFilerâ
get current records of browser window 1
get universal type identifier of library record id 1680 of library document âUntitled.eflibraryâ
get file of library record id 1680 of library document âUntitled.eflibraryâ
get filename of library record id 1680 of library document âUntitled.eflibraryâ
end tell
tell current application
do shell script âmktemp -d -t âEFRemoveDuplicateMessagesââ
end tell
tell application âEagleFilerâ
path to current application
end tell
tell current application
do shell script âcat â/Users/dimka/Downloads/Archive_Documents/Eagle_Filer/Untitled/Files/9A1C2037-93A2-4BEF-BDE8-ED663B8F25EE-1/[Gmail]/All Mailâ | perl -p -e âs/\r\n/\n/gâ | perl -p -e âs/\r/\n/gâ | /Applications/EagleFiler.app/Contents/Frameworks/WashFramework.framework/Versions/A/formail -b -e -q- -Y -D 104857600 â/var/folders/_1/_kqj_6gd251068rs1pg5f88r0000gn/T/EFRemoveDuplicateMessages.FwNy4Hlq/idcacheâ -s > â/var/folders/_1/_kqj_6gd251068rs1pg5f88r0000gn/T/EFRemoveDuplicateMessages.FwNy4Hlq/NewMailbox.mboxâ 2> â/var/folders/_1/_kqj_6gd251068rs1pg5f88r0000gn/T/EFRemoveDuplicateMessages.FwNy4Hlq/Log.logââ
@Michael_Tsai
at 10:18 AM
Hmmm⌠now it came back and told me no duplicates were found.
I stopped everything else on the machine including usb-c backup drive (literally turned it off)
Is it possible that because its Google mail - itâs not working?
Well, this looks good that it didnât time out. Perhaps there really were no duplicates? It looks by Message-ID so messages that are similar do not count as duplicates unless they are actually two copies of the same message.
you donât think those are duplicates?
because they have different # eml?
I only see 5 unique messages
@Michael_Tsai Still timeout for some reason. here is the event log
tell application âEagleFilerâ
get current records of browser window 1
get universal type identifier of library record id 12811 of library document âUntitled.eflibraryâ
get file of library record id 12811 of library document âUntitled.eflibraryâ
get filename of library record id 12811 of library document âUntitled.eflibraryâ
end tell
tell current application
do shell script âmktemp -d -t âEFRemoveDuplicateMessagesââ
end tell
tell application âEagleFilerâ
path to current application
end tell
tell current application
do shell script âcat â/Users/dimka/Downloads/Archive_Documents/Eagle_Filer/Untitled/Files/mboxâ | perl -p -e âs/\r\n/\n/gâ | perl -p -e âs/\r/\n/gâ | /Applications/EagleFiler.app/Contents/Frameworks/WashFramework.framework/Versions/A/formail -b -e -q- -Y -D 104857600 â/var/folders/_1/_kqj_6gd251068rs1pg5f88r0000gn/T/EFRemoveDuplicateMessages.mug4GBee/idcacheâ -s > â/var/folders/_1/_kqj_6gd251068rs1pg5f88r0000gn/T/EFRemoveDuplicateMessages.mug4GBee/NewMailbox.mboxâ 2> â/var/folders/_1/_kqj_6gd251068rs1pg5f88r0000gn/T/EFRemoveDuplicateMessages.mug4GBee/Log.logââ
do shell script âgrep -c "^formail: Duplicate key found:" â/var/folders/_1/_kqj_6gd251068rs1pg5f88r0000gn/T/EFRemoveDuplicateMessages.mug4GBee/Log.logââ
do shell script âgrep -vEc "^(\s*<|formail: Duplicate key found:)" â/var/folders/_1/_kqj_6gd251068rs1pg5f88r0000gn/T/EFRemoveDuplicateMessages.mug4GBee/Log.logââ
end tell
tell application âScript Editorâ
display alert âNo Duplicates Foundâ message âThere were no duplicate messages in âmboxâ.â buttons {âCancelâ, âOKâ} cancel button 1
Result:
error âAppleEvent timed out.â number -1712
Those are .eml message files. The script that you are using is for removing duplicates within a single mailbox file. To remove duplicates among a group of selected .eml files you would need this script.
@Michael_Tsai
what about in here? i am running the RDS on that mbox and yet it still shows me -No duplicates.
@Michael_Tsai
I dug a little deeper inside the RAW and it looks like the MessageIDâs of the above are Different for each line (have no clue how), but when I look inside the rest of the message they are the same.
So I figure the script needs to disregard MessageID and then just compare From, Subject, and inside the text to be the same.
Unfortunately, I donât know the script or perl to comment out the MessageID check and add inside text comparison. OR. check for MessageID AND but after that, in addition, check if the text is the same
I donât think thereâs a way to modify the script to work that way. The Message-ID is core to how the formail tool works. Perhaps I can add this sort of duplicates removal as an option in a future version of EagleFiler.
Tha would be awesome. Thank you!
