DAT Solutions Internal

Back
May 14, 2020 1:35 PM PDT

Product Outage

Updates

  • May 14, 2020 2:16 PM PDT

    This incident has been resolved.

    Resolved
  • May 14, 2020 2:12 PM PDT (4 minutes earlier)

    The production issues appear to be resolved. In roughly ten minutes we will communicate to customers that this issue is resolved unless we get further reports of customers unable to access services.

    Monitoring
  • May 14, 2020 1:55 PM PDT (17 minutes earlier)

    Products appear to be back online, but engineering is still in the process of verifying the stability of all systems.

    Monitoring
  • May 14, 2020 1:49 PM PDT (5 minutes earlier)

    We have verified the root cause was the restarting of the pdxoradb01 server. The CSB team has confirmed that this means there are many postings and searches that were lost. Please be advised that customers may believe they have created postings, searches, or alarms that were not actually created.

    Identified
  • May 14, 2020 1:35 PM PDT (14 minutes earlier)

    We're investigating a mass outage at the moment. Root cause initially appears to be a failed drive causing a reboot. DBAs and CSB are investigating now.

    Investigating