Yes, this is a problem. It looks like it is related to this thread.
I at least need to know where SpamSieve might have put all my non-spam mail from the last 24 hours. I looked in the SpamSieve Log.log to find:
=====================================================================
Trained: Good (Manual)
Subject: Your certificate is ready for collection!
Identifier: vOAyvz5+9Wljty9MUQuQlQ==
Actions: added rule <From (address) Is Equal to "secureemail@comodogroup.com"> to SpamSieve whitelist, added rule <From (name) Is Equal to “Certificate Customer Services”> to SpamSieve whitelist, disabled rule <Body (any text part) Contains "src=“cid:”> in SpamSieve blocklist, added to Good corpus (348)
Date: 2014-10-14 20:30:47 +0000
Trained: Good (Manual)
Subject: Your certificate is ready for collection!
Identifier: HfZXDjfrwSnothz7D6Zt/w==
Actions: added to Good corpus (349)
Date: 2014-10-14 20:30:47 +0000
First, not sure why it is trained “Good (Manual)” as I did not ever touch or see this email. But maybe “Manual” in this context means something different than I assume.
What I really need is an option for expanded logging that includes a report of the action taken by SpamSieve so that I can find out where my non-spam emails have been sent.
After doing the manual rebuild as recommended I can no longer open Apple Mail. It crashes while rebuilding the database.
However, I’m still glad to have found this post, because it does give me a plan of action: disable all my email accounts, enable only the ones which have been disabled, and see if all of my emails from today are in one of those other accounts. I HOPE that the database rebuild will work if it is only trying other accounts. That still sort of scares me. Where would SpamSieve put the mail for an IMAP account that isn’t connected because it’s been disabled?
Also: really? You don’t limit the accounts you’ll save mail to to only enabled ones? And SpamSieve has no way of knowing where it found the email in the first place in order to just put it back wherever it came from? I’m sorry, that sounds pretty thick.
So far, not really providing good motivation for me to give you my $30.
Are you accessing your mail account from multiple computers or devices? Also, have you seen this page?
“Trained: Good (Manual)” means that you selected the message in Apple Mail and invoked the “SpamSieve - Train as Good” command. Or that you have set up some sort of other script or rule that trained the message.
All the primary SpamSieve actions are already logged. You can also click this link to have more information logged (to the All Messages section of Console) about what the “Train as Good” command is doing.
That would seem to indicate a more general problem with either your Mac or Mail. Which version of Mac OS X are you using?
I’m not exactly sure what you’re asking here. SpamSieve doesn’t touch mail in disabled accounts. And I would expect that you couldn’t manually train it with those messages either because they would be hidden.
The “Train as Good” command specifically does not move messages to accounts that are disabled in Mail. It either finds a matching account or uses the first account that’s enabled.
I have been accessing my accounts from multiple computers, though only one at a time right now because SpamSieve is a new variable in my setup. SpamSieve itself is only on one computer. I have spam filtering disabled on the server and have disabled it in all client applications on all devices. My iPad does also poll for mail periodically. I haven’t disabled that.
Apple Mail on my desktop system was the only app open this morning. My desktop system is the one with SpamSieve.
I absolutely, specifically, categorically, without any shred of doubt (unless I was hypnotized and instructed to forget) did not manually invoke “SpamSieve - Train as Good” on the one important mail I’ve been using as an example/test case (because I’m still hoping I’ll find it.) But if that rule from the setup is what you’re talking about, OK. That would leave a mystery of how anything ever gets marked “Auto” but I really don’t much care about that.
From what I’ve seen, none of the logs show where SpamSieve moves the messages when they are done. You say (below) that “SpamSieve tries hard to figure out the proper inbox by matching the e-mail addresses”. It would be great to know what SpamSieve determined was the proper inbox. I did click the above link, so I’ll check later to see if the destination mailboxes are now getting reported.
I’ll look in to that.
Mavericks, 10.9.5.
And yes, I did have to rebuild the database, as I mentioned. That really really ought not delete messages though. The email in question was missing before the database rebuild and is still gone.
And I mean really really gone. I’ve grepped for it in ~/Library/Mail/V2 . The SpamSieve log is the only evidence I have that it actually arrived. I haven’t searched the SpamSieve log for other missing emails yet.
I ought to grep for it on the drive on the mailserver, see what that turns up.
One of the other threads about this/a similar issue said that SpamSieve puts the messages back in the inbox but (as already discussed) it gets the mailbox wrong under some circumstances (as you’ve described). Could have sworn the person solved the problem by enabling a disabled account to retrieve the mail which had been placed there by SpamSieve. Maybe I misunderstood or maybe that person had disabled the mailbox after SpamSieve had moved the mails there. No matter.
OK, c’est la guerre. Sorry I called the software’s behavior “thick”.
The alias for the address to which the missing emails were sent is set up as directed (and have been for years—otherwise I couldn’t send mail “from” that address.)
And hey, the title here was meant to be funny inside the threaded post. It looks a bit more angry as the title of a whole topic. Feel free to remove “is not what I signed up for!” And change “sending my good mail to dev/null” to something more accurate like “good mail disappears.” I tried to make the edit myself but it’s not possible. Hopefully an admin can.
No. The standard SpamSieve rule is what asks SpamSieve to examine the incoming messages to determine whether they’re spam. It generates “Predicted: Good” and “Predicted: Spam” log entries (as well as the “Auto” ones).
Is it possible that you accidentally pressed the keyboard shortcut, Command-Control-G? There really is no mechanism in SpamSieve to do a manual training without user initiation. And in 12 years I’ve never heard anyone else suggest that this happened.
That is correct.
The only possibilities are the inboxes of the accounts that you have set up in Mail. I suppose it’s possible that Mail or the server lost the message instead of moving it. However, I don’t think I’ve ever heard of that happening. In all the cases I’ve seen of messages actually disappearing, the problem ended up being that Mail moved the message to the inbox and then another computer’s mail rules saw it as new and moved it off the server and into a local mailbox on that computer.
Yes, there used to be a problem where the closest matching account was a disabled one and Mail would incorrectly report to SpamSieve that the account was enabled. So, in that case, the message could end up in a hidden inbox. You could get to it by enabling the account in Mail’s preferences. However, that was years ago. Also, if the message were in the inbox of a disabled account, you would have found it by grepping the V2 folder.
Extremely unlikely. In the case of this specific email, I asked submitted a form online and basically sat and waited for the mail to come in. I did some CMD-Tab between Mail and Console to watch the log, but I wasn’t sitting here speed-typing where I might have had an accidental finger-mashup on the keyboard. I did CMD-G when I was in Console searching, but that’s about the closest thing I can think of.
Well, that’s the additional information I said would be helpful to have. Consider it a feature suggestion. Not helpful unless things go wrong, but that’s why logging exists in the first place.
I do not use local mailboxes at all, ever. However, I have closed down mail on this machine and started Mail on my laptop to see if the email was somewhere in there. Also with both copies of Mail closed (quit, I mean, not just the windows closed) I have opened up AirMail to look for missing messages.
OK. Heh. Usually when looking through support forums I have the problem of not looking deeply enough. Looks like maybe I dug too far back. At least it wasn’t just my imagination.
So… here’s more info, and no less disturbing. Why would SpamSieve be looking at my “Sent” folder or my “Drafts” folder? I was starting a reply in a conversation with my dad a few minutes ago, and could not find the mail I sent to him yesterday. As it is another strange disappearance, I looked in the SpamSieve log. Again, here I would not have thought it in any way likely that SpamSieve would have touched it, but I thought I may as well run a search. I found a couple dozen instances of the mail from me. Not sure whether it was in “Sent” or “Drafts” but considering the multiple timestamps it looks like it must have been looking in “Drafts”. The sent mail did reappear (it took a while for Mail to index? I don’t know) and the only place it exists now is in “Sent”. Considering how many of them there are and how many different timestamps, they must have been from yesterday morning.
=====================================================================
Predicted: Good (0)
Subject: Re: Logic
From: MYADDRESS@MYDOMAIN.net
Identifier: /XrCzLWkoQl4GRxchOqvFQ==
Reason: sender <MYADDRESS@MYDOMAIN.net> in address book
Date: 2014-10-14 17:49:01 +0000
Trained: Good (Auto)
Subject: Re: Logic
Identifier: /XrCzLWkoQl4GRxchOqvFQ==
Actions: added to Good corpus (168)
Date: 2014-10-14 17:49:03 +0000
Predicted: Good (0)
Subject: Re: Logic
From: MYADDRESS@MYDOMAIN.net
Identifier: VaHkphallYbYr24dvhl6wQ==
Reason: sender <MYADDRESS@MYDOMAIN.net> in address book
Date: 2014-10-14 17:49:30 +0000
Trained: Good (Auto)
Subject: Re: Logic
Identifier: VaHkphallYbYr24dvhl6wQ==
Actions: added to Good corpus (169)
Date: 2014-10-14 17:49:30 +0000
…23 nearly identical sets of entries…
Predicted: Good (0)
Subject: Re: Logic
From: MYADDRESS@MYDOMAIN.net
Identifier: fHyVYqZwboClnLXIQcV2AQ==
Reason: sender <MYADDRESS@MYDOMAIN.net> in address book
Date: 2014-10-14 18:02:00 +0000
Trained: Good (Auto)
Subject: Re: Logic
Identifier: fHyVYqZwboClnLXIQcV2AQ==
Actions: added to Good corpus (192)
Date: 2014-10-14 18:02:00 +0000
Predicted: Good (0)
Subject: Re: Logic
From: MYADDRESS@MYDOMAIN.net
Identifier: 0XOqseklnmY5NaQtzzB1qw==
Reason: sender <MYADDRESS@MYDOMAIN.net> in address book
Date: 2014-10-14 18:02:30 +0000
Trained: Good (Auto)
Subject: Re: Logic
Identifier: 0XOqseklnmY5NaQtzzB1qw==
Actions: added to Good corpus (193)
Date: 2014-10-14 18:02:30 +0000
The log also shows that SpamSieve is examining mails that were sent and received months ago, and are sitting in Archive folders.
My inbox is full of spams with the nice multicolor coding that SpamSieve assigns. facepalm …though that might be because i let my computer go to sleep last night, which the instructions tell me I shouldn’t have done. Sorry.
And I can’t even close SpamSieve—it is reopening itself every few minutes after I tell it to quit.
PS your forum logs me out faster than I can write a reply, even if I check “remember me” when I log in. That’s a great way to keep people from leaving feedback. Presumably not what you want.
Noted, thanks. I’ve been adding a lot of logging recently, and the only reason this isn’t in there yet is that it hasn’t been an issue for anyone. Another idea I have is for SpamSieve to temporarily store its own copy of messages that you’ve recently trained so that you can recover if the client or server loses the message or remote computer “steals” it.
It doesn’t. SpamSieve only looks at new messages that arrive, unread in the inbox.
However, Mail will apply SpamSieve to old messages if you select them and choose Apply Rules or if you edit a rule and click Apply when it asks if you want to apply the rules to the current mailbox.
SpamSieve doesn’t open itself, but Mail will launch it whenever it applies the rules or you train a message.
Sorry about that. I’m not sure what’s happening there since it always remembers me and no one else has mentioned this. Which browser are you using? Do you see the same problem with other vBulletin 4 forums?
Well, the more info the better. But you’ll probably want “verbose” option that could be enabled/disabled so that you don’t just dump gobs of (usually) unnecessary info. What you’re describing would be great for me, but most people aren’t having this much trouble, I presume! It might not be worth getting my $30 for you to go crazy.
Theory: since I deleted and recreated one of my accounts (part of restoring that database which was somehow corrupted—if we knew what happened there maybe that would shed some light. I could pull those files out from Time Machine but I don’t have the knowledge or documentation to try to diagnose or test those files) perhaps they’re getting recognized as “new” as they are being downloaded from the server. These are things in my archive mailboxes, not my inbox, but Courier seems to report everything as being a sub-folder of Inbox. Not sure how gmail handles things (I have both). Is that in any way an avenue worth my time investigating?
I’ll try disabling GPGMail, but I’ve got some other things to take care of first. But if there’s a known conflict there I’ll jump on that sooner rather than later.
One thing that occurs to me. That page specifies the home directory as /Users/<username>/Library/Mail/Bundles I have my user directory on another hard drive, so the path for ~/Library/Mail/Bundles is actually /Volumes/User Data/Users/<username>/Library/Mail/Bundles . So if you have /Users/<username>/etc somehow hardcoded instead of using ~/ or $HOME and you’re copying anything directly to the filesystem, that could be a problem. That seems really unlikely (you’d also have to be failing to test that the copy succeeded) but at this point I’m throwing out bad ideas because I don’t have any other kind.
Hm. That plug-in must be a bit more sophisticated than I thought. I’d assumed it wouldn’t see something until it had made it into the local Spam mailbox created for SpamSieve. I’ve disabled the rule for the moment (and am drowning in spam) until I/we can think of something to try next.
Not the end of the world. Autosave does a good job; I haven’t lost more than a half a sentence, and now I’m copying my message before hitting submit just in case. It does scare the heck out of me though. I’m using Firefox 33.0. Haven’t tried other browsers and don’t spend a whole lot of time on forums. I have encountered quick autologoff before but usually not so fast. And I complained before checking my profile settings so I’ll look there.
Didn’t see anything in the forum settings but I do see that the “bb_lastactivity”, “bb_lastvisit”, “bb_password”, and “bb_userid” cookies are set to expire almost 40 minutes ago instead of ‘Session’ like the others.
Anyway, not a priority and it could be just me. Simply giving you what info I can.
Of course. It’s just that once I’ve seen something weird happen once, I like to be prepared in case it happens again.
I’m not sure there’s anything to investigate there. I think this is an Apple Mail issue, not a server issue.
There is a known and reportedbug in the GPG plug-in that causes it to apply Mail rules to old messages outside the inbox. So that probably explains at least one of these mysteries.
SpamSieve does use $HOME. The documentation is written that way because most users find ~/ really confusing.
You have it backwards. SpamSieve doesn’t move anything out of the Spam mailbox except when you train a message as good. Instead, all the mail comes into the inbox, and the plug-in determines which messages the rule is allowed to move to the Spam mailbox.
It looks like the default session timeout is 15 minutes. I don’t think that interacts with the “Remember me” setting at all. In any case, I’ve increased the timeout so hopefully it will work better for you now.
Well, there we go. So sooner rather than later my action plan here is:
Close Apple Mail
Move GPG plug-in
Delete my IMAP account in Internet Accounts (so that it gets totally rebuilt)
Re-open Apple Mail and “rebuild database”
Close Apple Mail
Recreate my IMAP account
Open Apple Mail and wait for all the syncing activity to abate
Turn the SpamSieve rule back on
Follow the threads about the GPG incompatibility
Use GPG only on the laptop for the time being.
In other words, try to eliminate every possible variable. Sometimes you gotta nuke the site from orbit. The only way to be sure.
One other question: Any known issues with using both AirMail and Mail with the SpamSieve options on both? Not at the same time, of course.
Understood. As I said, I was throwing out the bad ideas since I didn’t have good ones to go by.
Thanks! I’ve certainly spent more than 15 minutes on a couple of these. I’m trying to give you as accurate and complete information as possible, and there are a lot of angles to cover.
Good to know. I think you’ll agree at this point I should be proceeding with caution and adding variables only one at a time. I’ll of course report back how it goes. Thank you!
I’ve added some additional logging in SpamSieve 2.9.17 that will report which account’s inbox each message is moved to when trained as good. You can enable it with the AppleMailScriptDebug link.
At least it wasn’t just my imagination.