DAT Solutions Internal

Back
Jun 3, 2026 5:48 PM PDT

Legacy postings missing ownershipV2 following account API outage

Updates

  • Jun 3, 2026 7:10 PM PDT

    We are noticing an unusual volume of 500 responses from the Load Posting API's PATCH endpoint which is driving down the service's availability. The impacted postings are missing their ownershipV2, resulting in a TypeError. The impacted postings appear to have originated in the legacy systems during the time window that the Account API was down in INC-1288. Due to the outage, the freight-posting-service was unable to backfill critical information from the Account API onto the postings. Resolved by identifying the 201 impacted postings missing ownershipV2 and manually deleting them to stop the 500 errors on the PATCH API. The team also planned cluster upgrades and a post-incident review to address the Account API outage root cause and prevent recurrence.

    Resolved
  • Jun 3, 2026 5:48 PM PDT (1 hour and 21 minutes earlier)

    We are noticing an unusual volume of 500 responses from the Load Posting API's PATCH endpoint which is driving down the service's availability. The impacted postings are missing their ownershipV2, resulting in a TypeError. The impacted postings appear to have originated in the legacy systems during the time window that the Account API was down in INC-1288. Due to the outage, the freight-posting-service was unable to backfill critical information from the Account API onto the postings. The team mitigated by identifying the 201 impacted postings and manually deleting them to remove entries missing ownershipV2 and stop the 500 errors on the PATCH API. They also planned cluster upgrades and a post-incident review to address the root cause (Account API outage) and prevent recurrence.

    Monitoring