Major delay in filtering

Hi, Michael.

Since I moved to a new M4 PowerBook, I’ve noticed that SpamSieve is taking longer and longer to filter messages—sometimes up to 10 minutes or more. Is it still a problem of Mail passing messages on too slowly? Are you still suggesting the workaround of setting up a Mail rule that filters incoming messages with SpamSieve via AppleScript?

Currently on 3.3.1b1.

Thanks,
Steve

I’ve not heard any reports like this recently, so it’s not clear what might be happening. If you send in a diagnostic report I can take a look.

No.

Here it is.

SpamSieve Diagnostic Report.tbz

I think the problem may be that you have multiple mail accounts with the same name (“Info @ KA”). You are probably seeing errors to this effect in the Log window. Please try renaming or disabling one of the accounts.

I can try, but these duplicate accounts have existed for years and are inactive accounts for if I want to switch to IMAP from POP3 (or vice versa). And they didn’t have this delayed response until now.

Maybe the accounts somehow got re-enabled?

Not from what I’m seeing:

Looking at the code, I think there’s a bug where it’s incorrectly warning about accounts that are not active. Duplicate names could still be the problem—if they affect other accounts that are active—because it only reports an error for the first duplicate name that it finds.

If that’s not the source of the problem, I would suggest clicking this link to enable some extra debug logging (can turn off here). Then, the next time the delay occurs, please save a new diagnostic report.

Okay, I’ve deleted the bulk of the IMAP accounts (I realized that I don’t remember the last time I switched to IMAP from POP) which (I think) means all of the duplicates are gone. (Apple doesn’t make that process very easy.) I quit and restarted Mail afterward, and the problem is still there.

I’ve turned on the extra logging and will save a diagnostic report if the problem persists. I’ll probably restart the computer and see if that helps…

This time about 4-5 minutes to process about 40 spam messages (31 shown).

Diagnostics report attached.

Oh, how I hope this is helpful to someone else, not just me!}

SpamSieve Diagnostic Report.tbz

Thanks. It looks like the problem is that SpamSieve doesn’t have full disk access. There is probably a warning to this effect in the Settings ‣ Apple Mail ‣ Setup window.

Yep. That sped things up a lot. Still a bit slow at 30 seconds, but so much better.

Thanks, Michael.

30 seconds still sounds like too much. How many messages does the Log window say were filtered in that time? Also, please see this page.

In my latest batch, starting at 11:33 today, the logs showed 537 log entries. Of those, approximately 65 showed up in the “All Junk” folder, and 3 of those were false negatives.
I didn’t notice any SpamSieve moving messages from the inboxes at all, as if the spam was being filtered as it came in. So that’s a plus. (It missed about 15 spam messages that I would have expected it to catch since it’s been trained on similar messages.)


I wrote that yesterday, wanting to gather more information before posting. Just now there were 318 log entries, I didn’t see if there were any messages that were filtered immediately, but a bunch of them were filtered after about a minute and then some more after another 45 seconds. 46 emails were filtered as spam. SpamSieve failed to filter 39 messages that I would have expected it to catch.

The messages will be filtered right away (before showing in the inbox) if Apple’s mail extension system is working. Often, it’s not, and then SpamSieve will find them later when it scans the inbox. In the Filtering settings you can set how often it does this. Also, the Filter Now button will change to indicate when SpamSieve is currently scanning one of the mailboxes. So, if it seems like filtering is delayed, you can check to see whether SpamSieve is in the middle of a scan and, if so, which mailbox it’s busy with.

The uncaught spams are a separate issue. Please see this page and, if necessary, make a new thread or send me an e-mail.

I’m not ghosting you. I did so many of the tips you’ve suggested in your help file, but I’m still seeing a delay. Green flags are helpful, though.
In the meantime, my bombardment of spam seems to have stopped. On Friday, I had 798 spam messages, but on Saturday that had dropped off to about 120 and on Sunday less that 70. I’ve only got 7 so far today—and all filtered by SpamSieve.

I just now did an experiment watching the Filter Now button. It took about 20–22 seconds to begin filtering and 32–34 seconds to complete filtering on 51 messages (2 filtered as spam).

How often do you have it set to check for new unread messages?

The next time you’re about to start filtering a batch of messages like this, please use Activity Monitor to start recording a sample from SpamSieve. This will hopefully show what’s causing the slowness.

60 seconds.

Sample of SpamSieve sent.

That explains the 20-second delay before filtering starts. You could reduce it to, say, 5 seconds to make filtering start sooner.

Unfortunately, it looks like the sample was not recorded during a time when SpamSieve was actively filtering. Could you try to capture the period of time when the Filter Now button indicates that it’s busy?