| Installation | |||
| Ranking: 0=Client doesn't have this functionality 1=Not very 2=Moderately 3=Very | |||
| 1. | How intuitive was the installation process? | ||
| 2. | Are you asked where to store user data on multiple-user machines? | ||
| 3. | Will the client function for a user with restricted access to the system? | ||
| 4. |
What options are allowed (e.g., user's profile area)? |
||
|
Additional comments on installation: |
|||
| Configuration | |||
| Ranking: 0=Client doesn't have this functionality 1=Not very 2=Moderately 3=Very | |||
| 5. | How easily can you configure the client? | ||
| 6. | Does the client have "wizards" to help with the configuration? | ||
| 7. | Can you configure the client to use an external editor when composing or reading messages? | ||
|
Additional comments on configuration: |
|||
| Message Creation | |||
| Ranking: 0=Client doesn't have this functionality 1=Not very 2=Moderately 3=Very | |||
| To: Fields: Rate how easily or how well the client performs these functions. | |||
| 8. | Paste a column of text into the To: field (e.g., an Excel column of e-mail addresses) | ||
| 9. | Type a known recipient in the To: field and check evaluate autocompletion of address | ||
| Message body: create a message using the following functions and indicate how easily or how well the client performs these functions. | |||
| 10. | Copy/Paste | ||
| 11. | Cut/Paste | ||
| 12. | Word wrapping | ||
| 13. | Realignment or paragraphs | ||
| 14. | Spell-checking | ||
| 15. | Is there a default dictionary for the spell-checker? | ||
| 16. | Can you create a personal dictionary for the spell-checker | ||
| 17. | Search for and replace a word | ||
| 18. | Search forward for a word | ||
| 19. | Search backward for a word | ||
| 20. | Create a signature file | ||
| 21. | Can you create multiple signature files? | ||
| 22. | Create a stationery file | ||
| 23. | Can you create multiple stationery files? | ||
| 24. | Are there options for how/whether to send web formatted messages? | ||
| 25. | Can you create a message and specify a delivery receipt? | ||
| 26. | Can you create a message and specify a read receipt? | ||
| 27. | Can you create a message and specify a precedence header? | ||
| 28. | Can you create a message and specify an expiration date? | ||
| 29. | Can you create a message with a blank "From:" header? | ||
| 30. | Can you blind copy ("Bcc:" mail header) on all sent messages? | ||
|
Additional comments on message creation: |
|||
| Incoming Messages | |||
| Ranking: 0=Client doesn't have this functionality 1=Not very 2=Moderately 3=Very | |||
| 31. | How easily can you read messages? | ||
| 32. | How easily can you display the full headers of a message? | ||
| 33. | How easily can you open a message using your desktop word processing package? | ||
| 34. | Can you preview a message before you download it to your desktop? | ||
| 35. | In the case where the message line is longer than the reading window, how well does the client handle text wrapping? | ||
| 36. | How well does the client handle printing of messages? | ||
| 37. | Did you receive a delivery receipt for a message that had one? | ||
| 38. | Did you receive a read receipt for a message that had one? | ||
| 39. | Could you tell the precedence for a message that had one? | ||
| Try to delete one of your received messages. | |||
| 40. | How well does it handle message deletion with POP, Leave Mail on Server configuration? | ||
| 41. | How well does it handle message deletion with POP, download mail locally configuration? | ||
| 42. | How well does it handle message deletion with IMAP configuration? | ||
| 43. | Is there an "undelete" option? | ||
| Try to reply to one of your received messages. | |||
| 44. | How easily can you include the text of the original message in your reply? | ||
| 45. | How easily can you edit the text of the original message in your reply? | ||
| 46. | How easily can you exclude the text of the original message in your reply? | ||
| 47. | How easily can you unquote text (remove one level of quotation character)? | ||
| 48. | How easily can you re-quote text (paste back a passage of text with quotation character)? | ||
| 49. | How easily can you include the full headers of the message in your reply? | ||
| 50. | How easily can you exclude headers of the message in your reply? | ||
| 51. | How easily can you include an attachment of the message in your reply? | ||
| 52. | When replying to a message and including the text of the original message, try to run the spell-checker on your reply. Does the spell-checker also check the quoted text from the original message? | ||
| Try to forward one of your received messages. | |||
| 53. | How easily can you add a preface to the body of the message? | ||
| 54. | How easily can you change the subject of the message? | ||
| 55. | How easily can you edit the forwarded message? | ||
| 56. | How easily can you bounce/redirect a message? | ||
| 57. | How easily can you resend a message that you already sent? | ||
| 58. | Did you have to create a new message in order to resend a message that you had already sent? | ||
| 59. | How easily can you include the full headers of the message that you forward? | ||
| 60. | How easily can you exclude headers of the message that you forward? | ||
| 61. | How easily does the client include the original attachment in the message that you forward? | ||
|
Additional comments on incoming messages: |
|||
| Attachments | |||
| Ranking: 0=Client doesn't have this functionality 1=Not very 2=Moderately 3=Very | |||
| Try to send yourself a message with the following types of attachments and indicate whether you could successfully send them. | |||
| 62. | HTML file | ||
| 63. | Image | ||
| Once you've sent the above messages, how easily can you read them with this client. | |||
| 64. | HTML file - how easily is the HTML file sent as a file, rather than being displayed in-line? | ||
| 65. | Image - how easily is the image sent as a file, rather than being displayed in-line? | ||
| 66. | When you forward a message with an attachment, is the attachment correctly preserved? | ||
| 67. | When you resend a message with an attachment, is the attachment correctly preserved? | ||
|
Additional comments on attachments. Describe how the client
treats HTML files or image files only, touching on whether the client tries
to be "helpful" and display HTML or images inline when it shouldn't. |
|||
| Filtering | |||
| Ranking: 0=Client doesn't have this functionality 1=Not very 2=Moderately 3=Very | |||
| 68. | How easily can you create a filter that is triggered by the sender's identity? | ||
| 69. | How easily can you create a filter that is triggered by the text of the message? | ||
| 70. | Can you test multiple conditions when filtering? | ||
| 71. | When testing multiple conditions, must all conditions be met for filtering to work? (e.g., the sender = "joeuser@server.upenn.edu" AND the message contains "testing") | ||
| 72. | When testing multiple conditions, can you opt to only have some condition met? (e.g., the sender = "joeuser@server.upenn.edu" OR the message contains "testing") | ||
| 73. | Can you have multiple actions when filtering? | ||
| 74. | If you don't store local copies of your messages, does filtering still work? | ||
| 75. | How easily can you import/export filters to/from other email applications? | ||
|
Additional comments on filtering: |
|||
| Address Books | |||
| Ranking: 0=Client doesn't have this functionality 1=Not very 2=Moderately 3=Very | |||
| 72. | How easily can you lookup an email address in the Public Directory? | ||
| 73. | When you do a search that results in multiple hits, how easily can you tell which email address is the one that you want? | ||
| 74. | When you do a search that results in multiple hits, did you get all possible hits or was there a system limit on how many addresses display? | ||
| 75. | How easily can you add an email address to your addressbook? | ||
| 76. | Could you add the address without having to retype or cut/paste? | ||
| 77. | Is a history of past addresses maintained for auto-completion? | ||
| 78. | Can you edit the history list? | ||
| 79. | How easily can you maintain your own contact list? | ||
| 80. | Can you share your contact list with others? | ||
| 81. | How easily does your client's address book integrate with other application or OS address books (importing/exporting)? | ||
|
Additional comments on address books: |
|||
| Folders/Mail Storage | |||
| Ranking: 0=Client doesn't have this functionality 1=Not very 2=Moderately 3=Very | |||
| Are the following mailboxes automatically created by the client? | |||
| 82. | Inbox | ||
| 83. | Sent (local on hard drive, relevant to POP) | ||
| 84. | Sent (on remote server, relevant to IMAP) | ||
| 85. | Trash | ||
| 86. | What other folders are automatically created by the client? | ||
| Can you change the default location for the following mailboxes? | |||
| 87. | Inbox | ||
| 88. | Sent | ||
| 89. | Trash | ||
| Create some folders on both the server and desktop. | |||
| 90. | Are you able to copy a message from one folder on the server to another folder on the server using the "copy" command. | ||
| 91. | Using click-drag-drop, drag an e-mail message from one folder on the server to another folder on the server. Did the message actually move rather than being copied? | ||
| 92. | Using click-drag-drop, drag an e-mail message from one folder on the server to another folder on the desktop. Did the message actually move rather than being copied? | ||
| 93. | How easily can you search a folder on the server and a folder on the desktop for a particular word? | ||
| 94. | Are you able to specify a search order? | ||
| 95. | How easily can you reorganize how messages are listed in a folder? | ||
| 96. |
What format is used for locally stored mail (e.g., Berkeley mail format)? |
97. | How easily can you import/export messages to/from other email applications? |
| 98. | How easily can you resync your folders? | ||
| 99. | Is resynchronization automatic? | ||
| 100. | Can resynchronization be forced? | ||
| 101. | How easily can you refresh the IMAP cache (locally kept list of server mailboxes)? | ||
| 102. | How easily does the client read/process stored, archived mail (i.e., mailbox data archived on CD)? | ||
|
Additional comments on mail storage: |
|||
| Multiple Accounts | |||
| Ranking: 0=Client doesn't have this functionality 1=Not very 2=Moderately 3=Very | |||
| 103. | How easily can you check mail for multiple accounts? | ||
| 104. | How easily can you send mail for multiple accounts? | ||
|
Additional comments on multiple accounts: |
|||
| Security | |||
| Ranking: 0=Client doesn't have this functionality 1=Not very 2=Moderately 3=Very | |||
| Does the client support the following protocols? | |||
| 105. | Kerberos with POP | ||
| 106. | Kerberos with IMAP | ||
| 107. | Kerberos with SMTP | ||
| 108. | Kerberos with LDAP | ||
| 109. | Kerberos with NNTP | ||
| 110. | SSL with POP | ||
| 111. | SSL with IMAP | ||
| 112. | SSL with SMTP | ||
| 113. | SSL with LDAP | ||
| 114. | SSL with NNTP | ||
| 115. | How easily can the client be used with PGP? | ||
| 116. | How easily can the client be patched with SPs/bug fixes to keep secure? | ||
|
Describe how client responds if SSL is used with a hostname
that does not match the hostname of the security certificate
on server (e.g., home.seas instead of imap.seas): |
|||
|
Additional comments on security, i.e. comment on frequency of security patches
released: |
|||
| Technical Support | |||
| Ranking: 0=Client doesn't have this functionality 1=Not very 2=Moderately 3=Very | |||
| 117. | Based on review of vendor web site, how well supported does the product seem to be? | ||
| 118. | How good is the online help system? | ||
|
Additional comments on technical support: |
|||
| Palm-OS Interoperability | |||
| Ranking: 0=Client doesn't have this functionality 1=Not very 2=Moderately 3=Very | |||
| 119. | How easily can the client synch with Palm-OS products? | ||
| 120. | How easily can the client synch e-mail addresses? | ||
| 121. | How easily can the client synch address book data? | ||
| Provide more details about synch options
if necessary. |
|||
|
Additional comments on palm interoperability: |
|||
| Uninstallation | |||
| Ranking: 0=Client doesn't have this functionality 1=Not very 2=Moderately 3=Very | |||
| 122. | How easily does this client handle uninstallation and preserve (not destroy!) user data folders or files? | ||
| 123. | How easily does this client handle uninstallation and preserve user configuration? | ||
| Additional comments on uninstallation: |
|||
| Overall Evaluation | |||
|
How easy and intuitive is the installation and configuration
process? Are there tools for creating a custom installer? |
|||
| How easily does the client handle importing
mail during set-up, and reading archived mail formats (e.g., CD)? |
|||
|
Is it easy to "mess up" the configuration? |
|||
|
Do you think the application is sturdy and robust enough to handle configuration
problems? Did it crash a lot? Did it auto-recover when the system crashed? |
|||
|
Is the application resource intensive? Did you notice slowdown of your machine
during the testing process? |
|||
|
How easy to use is the client? How intuitive? How much can you
change/personalize the user interface? |
|||
|
How end-user friendly do you consider it to be? |
|||
|
How difficult will the learning curve be for a beginning user? |
|||
|
Did the application have the ability to multi-task? |
|||
|
Will cost be a factor for this product? |
|||
|
Other comments |
|||