DRAFT - Email 2002 Test Script - DRAFT

Name:
E-Mail Address:
Date:
Please choose the product you are testing and the operating system of the machine on which you are testing. If your environment is not included in the following select lists, please do not proceed but notify the group about the discrepancy.
Product and Version Being Evaluated:			Select a client Mulberry 2.1.2 Messenger 6.2.1 Messenger 4.79 Eudora 5.1 Outlook 2000 Outlook XP OE 5.5x Entourage
OS and Version used for Testing:			Select an OS Mac OS 8.6 Mac OS 9.2.2 Mac OS X 10.1 Windows 98/ME Windows NT 4.0 Windows 2000 Windows XP Linux/Solaris
Installation
Ranking:   0=Client doesn't have this functionality     1=Not very   2=Moderately   3=Very
	1.	How intuitive was the installation process?	Untested     0     1     2     3
	2.	Are you asked where to store user data on multiple-user machines?	Untested     No     Yes
	3.	Will the client function for a user with restricted access to the system?	Untested     No     Yes
	4.	What options are allowed (e.g., user's profile area)?
Additional comments on installation:
Configuration
Ranking:   0=Client doesn't have this functionality     1=Not very   2=Moderately   3=Very
	5.	How easily can you configure the client?	Untested     0     1     2     3
	6.	Does the client have "wizards" to help with the configuration?	Untested     No     Yes
	7.	Can you configure the client to use an external editor when composing or reading messages?	Untested     No     Yes
Additional comments on configuration:
Message Creation
Ranking:   0=Client doesn't have this functionality     1=Not very   2=Moderately   3=Very
Create a message using the following functions and indicate how easily or how well the client performs these functions.
	8.	Copy/Paste	Untested     0     1     2     3
	9.	Cut/Paste	Untested     0     1     2     3
	10.	Word wrapping	Untested     0     1     2     3
	11.	Realignment or paragraphs	Untested     0     1     2     3
	12.	Spell-checking	Untested     0     1     2     3
	13.	Is there a default dictionary for the spell-checker?	Untested     No     Yes
	14.	Can you create a personal dictionary for the spell-checker	Untested     No     Yes
	15.	Search for and replace a word	Untested     0     1     2     3
	16.	Search forward for a word	Untested     0     1     2     3
	17.	Search backward for a word	Untested     0     1     2     3
	18.	Create a signature file	Untested     0     1     2     3
	19.	Can you create multiple signature files?	Untested     No     Yes
	20.	Create a stationery file	Untested     0     1     2     3
	21.	Can you create multiple stationery files?	Untested     No     Yes
	22.	Are there options for how/whether to send web formatted messages?	Untested     No     Yes
	23.	Can you create a message and specify a delivery receipt?	Untested     No     Yes
	24.	Can you create a message and specify a precedence header?	Untested     No     Yes
	25.	Can you create a message and specify an expiration date?	Untested     No     Yes
	26.	Can you create a message with a blank "From:" header?	Untested     No     Yes
	27.	Can you blind copy ("Bcc:" mail header) on all sent messages?	Untested     No     Yes
Additional comments on message creation:
Incoming Messages
Ranking:   0=Client doesn't have this functionality     1=Not very   2=Moderately   3=Very
	28.	How easily can you read messages?	Untested     0     1     2     3
	29.	How easily can you open a message using your desktop word processing package?	Untested     0     1     2     3
	30.	Can you preview a message before you download it to your desktop?	Untested     No     Yes
	31.	In the case where the message line is longer than the reading window, how well does the client handle text wrapping?	Untested     0     1     2     3
	32.	How well does the client handle printing of messages?	Untested     0     1     2     3
	33.	Did you receive a delivery receipt for a message that had one?	Untested     No     Yes
	34.	Could you tell the precedence for a message that had one?	Untested     No     Yes
	35.	Could you tell the expiration date for a message that had one?	Untested     No     Yes
Try to delete one of your received messages.
	36.	Which of the following best describes what happens to a message when you delete it?	Untested actually deleted marked for deletion moved to a deleted folder none of the above
	37.	Is there an "undelete" option?	Untested     No     Yes
Try to reply to one of your received messages.
	38.	How easily can you include the text of the original message in your reply?	Untested     0     1     2     3
	39.	How easily can you edit the text of the original message in your reply?	Untested     0     1     2     3
	40.	How easily can you exclude the text of the original message in your reply?	Untested     0     1     2     3
	41.	When replying to a message and including the text of the original message, try to run the spell-checker on your reply. Does the spell-checker also check the quoted text from the original message?	Untested     No     Yes
Try to forward one of your received messages.
	42.	How easily can you add a preface to the body of the message?	Untested     0     1     2     3
	43.	How easily can you change the subject of the message?	Untested     0     1     2     3
	44.	How easily can you edit the forwarded message?	Untested     0     1     2     3
	45.	How easily can you bounce/redirect a message?	Untested     0     1     2     3
	46.	How easily can you resend a message that you already sent?	Untested     0     1     2     3
	47.	Did you have to create a new message in order to resend a message that you had already sent?	Untested     No     Yes
Additional comments on incoming messages:
Attachments
Ranking:   0=Client doesn't have this functionality     1=Not very   2=Moderately   3=Very
Try to send yourself a message with the following types of attachments and indicate whether you could successfully send them.
	48.	ASCII text	Untested     No     Yes
	49.	Images	Untested     No     Yes
	50.	Sound files	Untested     No     Yes
	51.	Spreadsheet	Untested     No     Yes
	52.	Word processor file	Untested     No     Yes
	53.	Database file	Untested     No     Yes
	54.	MIME file	Untested     No     Yes
Once you've sent the above messages, how easily can you read them with this client.
	55.	ASCII text	Untested     0     1     2     3
	56.	Images	Untested     0     1     2     3
	57.	Sound files	Untested     0     1     2     3
	58.	Spreadsheet	Untested     0     1     2     3
	59.	Word processor file	Untested     0     1     2     3
	60.	Database file	Untested     0     1     2     3
	61.	MIME file	Untested     0     1     2     3
	62.	When you forward a message with an attachment, is the attachment correctly preserved?	Untested     No     Yes
	63.	When you resend a message with an attachment, is the attachment correctly preserved?	Untested     No     Yes
Additional comments on attachments:
Filtering
Ranking:   0=Client doesn't have this functionality     1=Not very   2=Moderately   3=Very
	64.	How easily can you create a filter that is triggered by the sender's identity?	Untested     0     1     2     3
	65.	How easily can you create a filter that is triggered by the text of the message?	Untested     0     1     2     3
	66.	Can you test multiple conditions when filtering?	Untested     No     Yes
	67.	When testing multiple conditions, must all conditions be met for filtering to work? (e.g., the sender = "joeuser@server.upenn.edu" AND the message contains "testing")	Untested     No     Yes
	68.	When testing multiple conditions, can you opt to only have some condition met? (e.g., the sender = "joeuser@server.upenn.edu" OR the message contains "testing")	Untested     No     Yes
	69.	Can you have multiple actions when filtering?	Untested     No     Yes
	70.	If you don't store local copies of your messages, does filtering still work?	Untested     No     Yes
	71.	How easily can you import/export filters to/from other email applications?	Untested     0     1     2     3
Additional comments on filtering:
Addressbooks
Ranking:   0=Client doesn't have this functionality     1=Not very   2=Moderately   3=Very
	72.	How easily can you lookup an email address in the Public Directory?	Untested     0     1     2     3
	73.	When you do a search that results in multiple hits, how easily can you tell which email address is the one that you want?	Untested     0     1     2     3
	74.	When you do a search that results in multiple hits, did you get all possible hits or was there a system limit on how many addresses display?	Untested     No     Yes
	75.	How easily can you add an email address to your addressbook?	Untested     0     1     2     3
	76.	Could you add the address without having to retype or cut/paste?	Untested     No     Yes
	77.	Is a history of past addresses maintained for auto-completion?	Untested     No     Yes
	78.	Can you edit the history list?	Untested     No     Yes
	79.	How easily can you maintain your own contact list?	Untested     0     1     2     3
	80.	Can you share your contact list with others?	Untested     No     Yes
Additional comments on addressbooks:
Folders/Mail Storage
Ranking:   0=Client doesn't have this functionality     1=Not very   2=Moderately   3=Very
Are the following mailboxes automatically created by the client?
	81.	Inbox	Untested     No     Yes
	82.	Sent	Untested     No     Yes
	83.	Trash	Untested     No     Yes
	84.	What other folders are automatically created by the client?
Can you change the default location for the following mailboxes?
	85.	Inbox	Untested     No     Yes
	86.	Sent	Untested     No     Yes
	87.	Trash	Untested     No     Yes
Create some folders on both the server and desktop.
	88.	Are you able to copy a message from one folder on the server to another folder on the server using the "copy" command.	Untested     No     Yes
	89.	Using click-drag-drop, drag an e-mail message from one folder on the server to another folder on the server. Did the message actually move rather than being copied?	Untested     No     Yes
	90.	Using click-drag-drop, drag an e-mail message from one folder on the server to another folder on the desktop. Did the message actually move rather than being copied?	Untested     No     Yes
	91.	How easily can you search a folder on the server and a folder on the desktop for a particular word?	Untested     0     1     2     3
	92.	Are you able to specify a search order?	Untested     No     Yes
	93.	How easily can you reorganize how messages are listed in a folder?	Untested     0     1     2     3
	94.	What format is used for locally stored mail (e.g., Berkeley mail format)?
	95.	How easily can you import/export messages to/from other email applications?	Untested     0     1     2     3
	96.	How easily can you resync your folders?	Untested     0     1     2     3
	97.	Is resynchronization automatic?	Untested     No     Yes
	98.	Can resynchronization be forced?	Untested     No     Yes
Additional comments on mail storage:
Multiple Accounts
Ranking:   0=Client doesn't have this functionality     1=Not very   2=Moderately   3=Very
	99.	How easily can you check mail for multiple accounts?	Untested     0     1     2     3
	100.	How easily can you send mail for multiple accounts?	Untested     0     1     2     3
Additional comments on multiple accounts:
Security
Ranking:   0=Client doesn't have this functionality     1=Not very   2=Moderately   3=Very
	101.	Can you change your password from within the client?	Untested     No     Yes
Does the client support the following protocols?
	102.	Kerberos with POP	Untested     No     Yes
	103.	Kerberos with IMAP	Untested     No     Yes
	104.	Kerberos with SMTP	Untested     No     Yes
	105.	Kerberos with LDAP	Untested     No     Yes
	106.	Kerberos with NNTP	Untested     No     Yes
	107.	SSL with POP	Untested     No     Yes
	108.	SSL with IMAP	Untested     No     Yes
	109.	SSL with SMTP	Untested     No     Yes
	110.	SSL with LDAP	Untested     No     Yes
	111.	SSL with NNTP	Untested     No     Yes
	112.	How easily can the client be used with PGP?	Untested     0     1     2     3
Describe how client responds if SSL is used with a hostname that does not match the hostname of the security certificate on server (e.g., home.seas instead of imap.seas):
Additional comments on security:
Technical Support
Ranking:   0=Client doesn't have this functionality     1=Not very   2=Moderately   3=Very
	113.	Based on review of vendor web site, how well supported does the product seem to be?	Untested     0     1     2     3
	114.	How good is the online help system?	Untested     0     1     2     3
Additional comments on technical support:
Palm-OS Interoperability
Ranking:   0=Client doesn't have this functionality     1=Not very   2=Moderately   3=Very
	115.	How easily can the client synch with Palm-OS products?	Untested     0     1     2     3
Describe synch options in more detail (e.g., can both mail and addresses be synchronized?):
Additional comments on palm interoperability:
Overall Evaluation
How easy and intuitive is the installation and configuration process? Are there tools for creating a custom installer?
Is it easy to mess up the configuration.
Do you think the application is sturdy and robust enough to handle configuration problems? Did it crash a lot? Did it auto-recover when the system crashed?
Is the application resource intensive? Did you notice slowdown of your machine during the testing process?
How easy to use is the client? How Intuitive? How much can you change/personalize the user interface?
How end-user friendly do you consider it to be?
How difficult will the learning curve be for a beginning user?
Did the application have the ability to multi-task?
Will cost be a factor for this product?
Other comments