DAT Solutions Internal
FMIC AWS FSx File Share Outage – \pawswfmcalx01\share
Updates
-
May 4, 2026 1:55 PM PDT
FMIC’s AWS FSx file share \pawswfmcalx01\share is down due to a failed FSx–Active Directory connection, leaving the share inaccessible since roughly 12 a.m. PT on 5/2 and impacting IQ dashboard processing for 80+ customers. On-call has verified DC/DNS reachability; multiple FSx recovery attempts have failed, and further recovery vs. restore actions are pending with the Platform team. Resolution: The FSx security group sg-0cff90e1c8704fb64 was updated to allow TCP/UDP 464 to the Domain Controllers, enabling the misconfigured state recovery process. After connectivity was verified by Eddie Boss, the incident was mitigated and marked resolved at 4:05 PM UTC.
Resolved -
May 4, 2026 10:09 AM PDT (3 hours and 46 minutes earlier)
FMIC’s AWS FSx file share \pawswfmcalx01\share is down due to a failed FSx–Active Directory connection, leaving the share inaccessible since roughly 12 a.m. PT on 5/2 and impacting IQ dashboard processing for 80+ customers. On-call has verified DC/DNS reachability; multiple FSx recovery attempts have failed, and further recovery vs. restore actions are pending with the Platform team. - The mitigation was performed by updating the FSx security group (sg-0cff90e1c8704fb64) to allow TCP/UDP 464 to the Domain Controllers, enabling the misconfigured state recovery process.
- After verification that connectivity was restored (Eddie’s confirmation), the incident was marked mitigated.Monitoring