Penn Computing
Computing Menu Computing A-Z
Computing Home Information Systems & Computing Penn

Microsoft's Incident Report for PennO365

Following is a data extract from the PennO365 administrative console of incident reports sent by Microsoft within the last 30 days. We pull out incidents that are relevant to the current PennO365 service offerings and we refresh this data every 15 minutes. The last extract was taken on October 16, 2017 23:45.

--------------------------------------------------------------------------------------------------
Title EX123218
ServiceAffected Exchange Online
Status Service restored
Time 10/10/2017 9:30:00 AM UTC
LastUpdated 10/11/2017 11:32:55 AM UTC
Message Title: Emails attachment issue

EX123218 Details

==========
User Impact: Users are unable to receive emails containing PDF attachments.

More info: Emails without PDF attachments are getting delivered.

Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes., Title: Emails attachment issue

==========
User Impact: Users are unable to receive emails containing PDF attachments.

More info: Emails without PDF attachments are getting delivered.

Current status: We're analyzing data provided by your organization to determine next troubleshooting steps.

Scope of impact: Your organization is affected by this event, and impact is specific to a subset of your users.

Start time: Tuesday, October 10, 2017, at 9:30 AM UTC

Next update by: Wednesday, October 11, 2017, at 11:30 AM UTC, Title: Emails attachment issue

==========
User Impact: Users were unable to receive emails containing PDF attachments.

More info: This issue only affected PDF attachments, all other attachments were delivered without error.

Final status: We've analyzed the message trace logs and determined that emails containing PDF attachments were being blacklisted. We've added an exclusion to whitelist the PDF attachments, which resolved the issue.

Scope of impact: Your organization was affected by this event, and impact was specific to a subset of your users.

Start time: Tuesday, October 10, 2017, at 9:30 AM UTC

End time: Wednesday, October 11, 2017, at 10:23 AM UTC

Preliminary root cause: The root cause of this incident is currently under investigation.

Next steps: - We'll provide the root cause, if identified, within five business days.}
--------------------------------------------------------------------------------------------------
Title MO121590
ServiceAffected Office 365 Portal
Status Service restored
Time 9/25/2017 12:00:00 AM UTC
LastUpdated 10/3/2017 9:29:27 PM UTC
Message Title: Admin usage reports processing delays

MO121590 Details

==========
User Impact: Admins may notice that usage reports do not contain data from the last few days.

Current status: We discovered an issue with a newly-added report that was impacting the ability of our systems to process usage reporting data. We've resolved the issue that was causing impact, and our systems are now processing the backlog of data. We expect that reporting data will be current by October 3, 2017.

Scope of impact: Impact is specific to service usage data after September 25, 2017.

Start time: Monday, September 25, 2017, at 12:00 AM UTC

Estimated time to resolve: Based on current rate of processing, we expect this problem to be resolved by October 3, 2017.

Preliminary root cause: A configuration problem related to a newly-added report caused impact to the systems that process usage report data, leading to a delay in data freshness.

Next update by: Tuesday, October 3, 2017, at 10:00 PM UTC, Title: Admin usage reports processing delays

==========
User Impact: Admins may have noticed that usage reports didn't contain data from the last week.

Final status: The backlog of data has been processed and reporting is now current.

Scope of impact: Impact was specific to service usage data after September 25, 2017.

Start time: Monday, September 25, 2017, at 12:00 AM UTC

End time: Tuesday, October 3, 2017, at 9:00 PM UTC

Preliminary root cause: A configuration problem related to a newly-added report caused impact to the systems that process usage report data, leading to a delay in data freshness.

Next steps: - We're reviewing our procedures for deploying new reports to better identify similar issues during our development and testing cycles.

This is the final update for the event.}
top

Information Systems and Computing
University of Pennsylvania
Comments & Questions


University of Pennsylvania Penn Computing University of Pennsylvania Information Systems & Computing (ISC)
Information Systems and Computing, University of Pennsylvania