Spam Filtering Service

ISC Networking and Telecommunications offers a spam filtering service for all of its users. This email server serves users in the following domains:

ac.upenn.edu	ben.dev.upenn.edu	dental.upenn.edu	design.upenn.edu
dolphin.upenn.edu	ehrs.upenn.edu	gse.upenn.edu	isc.upenn.edu
mail.med.upenn.edu	magpi.net	pobox.upenn.edu	sp2.upenn.edu

Unsolicited e-mail that pitches products, goods, services, etc., can be considered spam. These messages can be quite annoying and many users would prefer not to see these messages in their Inbox. Users with accounts on the ISC Networking and Telecommuncations e-mail servers can opt to filter all incoming messages for spam. The spam filtering service uses SpamAssassin to filter incoming messages. Messages that are marked as spam are stored in a separate mail folder and are automatically deleted after 28 days by a nightly process. Messages stored in the spam folder should be regularly monitored as real mail may be incorrectly identified as spam. Once a message is deleted from the spam folder, it cannot be recovered.

• Opting in
• Opting out
• Maintenance of the spam filter folder
• Forwarding mail
• SpamAssassin mail headers
• Interaction with existing .procmailrc file

If you wish to filter your mail for spam, you must opt into the service. Spam filtering is not automatically turned on. You can opt-in from the Account Services menu, authenticating yourself with your PennKey/password.

From the Spam Filtering menu, choose Opt-in/Opt-out and you will be presented with the following form option:

Choose your level of spam filtering: Low Medium High None

To opt-in to the spam filtering service, you must pick the level of filtering that will be applied to all incoming mail. Three levels of filtering are available: low, medium, high. The levels of filtering refer to the amount of filtering rules that will be applied. Low filtering means that a message must match more of the spam rules and less spam will be identified but less real mail will be incorrectly identified as spam. High filtering means that a message must match less of the spam rules and more messages may be identified as spam but this also increases your changes of real mail being identified as spam. We suggest that you start by using "low filtering". If you find that low filtering does not catch the kind of spam messages that you receive, switch to medium filtering. You can change your filtering level at any time.

See below for more information on spam filtering levels or consult with your local support provider for help in determining which filter level is best for you and the type of mail that you receive.

If you have opted for spam filtering but find that you'd rather not have your mail filtered, choose No filtering from Opt-in/Opt-out form as described in the Opting In section above and that will opt you out of the service.

Once you have opted out of the service, there will be no more automatic maintenance of mail that has already been identified as spam. You must manually delete any unwanted messages from the spam filter folder as these messages do count towards your disk quota.

After opting into the service, messages that have been identified as spam will be stored in a separate folder in your Mail directory.

Example:
/home/j/o/joeuser/Mail/caught-spam

It is important that you monitor this folder since it is possible that real mail may be identified as spam if the real message has characteristics that match the spam filtering rules. If a message is older than 28 days and the nightly process deletes that message, it cannot be recovered.

Since the spam filter folder does count towards your disk quota, you may want to delete messages sooner than 28 days. You can use your mail client to manually delete messages from the spam filter folder or you can use the Spam Filtering menu option to Maintain spam filter folder.

To delete messages using your mail client, you'll need to change to the "caught-spam" mail folder first. If you are using Elm, please see full instructions for changing folders. If you are using Webmail, you are automatically subscribed to the "caught-spam" folder when you opt in and should be able to change to the "caught-spam" folder. Other IMAP clients will handle this "caught-spam" folder just like any other mail folder outside of the main inbox. How you access the "caught-spam" folder will depend on your IMAP mail client. POP mail clients cannot change mail folders. If you are using a POP mail client, you may want to consider using Webmail to view your "caught-spam" folder. A quick guide to managing your caught-spam folder through Webmail is available.

To delete messages using the Spam Filtering menu, choose Maintain spam filter folder and if you have a caught-spam folder, you will be presented with the following form option:

Delete messages older than   0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 day(s)

If you wish to delete all messages that are older than 21 days, choose "21" from the pull-down menu. Once the form is submitted, all messages older than 21 days will be deleted from the spam filter folder. Choosing to delete messages from this form will not affect the nightly process of deleting messages older than 28 days from the spam folder. The nightly process will continue to delete only messages that are older than 28 days.

Choosing "0" from the pull-down menu will delete all messages from the spam filter folder. Once again we must stress that once messages are deleted from the spam filter folder, they cannot be recovered. It is strongly recommended that you scan messages in the spam filter folder before opting to delete messages.

If you have setup a .forward in your home directory or have opted to forward your e-mail through Account Services and you are forwarding all of your mail to some other account, you will not be able to use spam filtering.

If you are forwarding a copy of your mail to another e-mail address and still want to keep a copy of your e-mail on the local server, you will be able to use spam filtering but you must set up your .forward with the proper syntax.

Example:
Correct - joeuser, another-user@sas.upenn.edu

Incorrect - joeuser@pobox.upenn.edu, another-user@sas.upenn.edu

Your .forward file should include just your username without any hostname, then the list of address(es) to which you wish to send a copy of all of your e-mail. All addresses should be separated with commas. In the above "Correct" example, a copy of the mail will be delivered locally to joeuser's mailbox where it will then be filtered for spam and a copy of the mail will be forwarded unfiltered to another-user@sas.upenn.edu.

If you have chosen to filter your mail and you look at the full mail headers of your filtered messages, you will notice some additional mail headers will have been added to your messages.

Example of a message that is not considered spam:

X-Spam-Status: No, hits=0.0 required=5.0
        tests=none
        version=2.55
X-Spam-Level: 
X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp)

Example of a message that is considered spam:

X-Spam-Flag: YES
X-Spam-Status: Yes, hits=8.8 required=5.0
        tests=DATE_IN_PAST_12_24,DRASTIC_REDUCED,HOME_EMPLOYMENT,
              INVALID_DATE,INVALID_MSGID,MIME_HEADER_CTYPE_ONLY,
              NO_REAL_NAME,ONCE_IN_LIFETIME,REMOVE_SUBJ,UNDISC_RECIPS
        version=2.55
X-Spam-Level: ********
X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp)

Messages that are multi-part may also include a report from SpamAssassin that lists in detail the spam tests for which the message matches:

---- Start SpamAssassin results
6.90 points, 5 required;
*  0.8 -- From: does not include a real name
*  0.6 -- Invalid Date: header (not RFC 2822)
*  0.1 -- RAW: Message includes Microsoft executable program
*  0.3 -- Date: is 3 to 6 hours before Received: date
*  0.7 -- 'Message-Id' was added by a relay (3)
*  3.5 -- Forged mail pretending to be from MS Outlook
*  0.5 -- Message has X-MSMail-Priority, but no X-MimeOLE
*  0.4 -- Spam tool pattern in MIME boundary

---- End of SpamAssassin results

Following is an explanation of some of the tags in the spam mail headers.

hits=	The spam ranking for the message. When a message is filtered, it is examined for certain spam characteristics and if it matches any of those characteristics, it is given a numerical spam ranking.
required=	Messages must have at least a 5.0 ranking to be considered spam. This is a system-wide configuration. Because a message meets the minimum requirements to be considered spam does not mean that the message will be stored in your spam filter folder. Whether a spam message is stored in your Inbox or your spam folder will depend on the filtering level that you have chosen for your spam filtering and the X-SpamLevel of the message.
tests=	The list of spam tests for which the message has matched. A full list of tests performed by SpamAssassin is available.
X-Spam-Level:	The overall whole integer representation of the spam level of the message that displays as a series of asterisks. If a message scores a ranking of hits=7.5, the X-Spam-Level for that message will be X-Spam-Level: *******. It is this value that determines whether a message will be stored in your spam folder. For example, if you have chosen low filtering, the message must have at least 7 asterisks for the messages to be stored in your spam filter folder. Even if a message is considered spam because it has a 5.0 spam ranking, it will only be stored in your spam filter folder if it has 7 asterisks or more. low filtering - 7 asterisks or more medium filtering - 6 asterisks or more high filtering - 5 asterisks or more

To implement spam filtering with Spam Assassin, the following statement is written to the .procmailrc file in your home directory

INCLUDERC=/usr/local/etc/filtering/spamfilter.level

If you already have a .procmailrc file with existing Procmail recipes, the system will write the above statement to the top of the file so that spam filtering will be applied to all mail before any other filtering rule is used. If you decide to edit your .procmailrc file to change this behavior, please be aware that anytime you use the Spam Filtering menu, the system will again put this INCLUDERC= statement back to the top of your .procmailrc file to again force spam filtering as the first Procmail action.