DAT Solutions Internal

DAT directory is missing company info pulled from the FMCSA for multiple large Enterprise accounts. As tested by Karlie, there are no issues with those 4 accounts in the new directory. Additionally, there have been no new reports regarding this issue. Hence resolving this.

Lots of intermittent 504s from requests to geo-v3, seemingly not ever making it to the service Fix has been implemented and deployed

(This is a test)
Lanemakers taking long time to load, getting timed out The incident was resolved as indicated by the status change and Grant Turner's confirmation that \"everything is much better now,\" along with the note that customers were no longer experiencing issues with loading. Additionally, the mitigation message reflects a successful resolution of the loading problem.

FMIC team is unable to download the files from files.dat.com however the site is up and reachable with no recent DNS change. The IP block imposed by files.com has been lifted, thereby resolving the issue.

It looks like geo API is throwing below errors for few records while processing through the lambda.

Failed to resolve geography after 5 retries reason {"name":"RequestError","message":"Error: ESOCKETTIMEDOUT","cause":{"code":"ESOCKETTIMEDOUT","connect":false},"error":{"code":"ESOCKETTIMEDOUT","connect":false},"options":{"uri":"https://analytics.api.dat.com/geo/v3/areas/minimalPoints" As there have been no new errors, proceeding to resolve the issue.

While trying to upload a file to IQ Bulk Uploads, customer receives an error that says Bulk Uploads are Not Available

GH Actions stuck with a lot of actions in queue and not a lot of actions running.

Two affected repos:
https://github.com/dat-freight-and-analytics/search-system/actions

https://github.com/dat-freight-and-analytics/freight-search-module-v2/actions

The incident was resolved by identifying and updating the deprecated runner version, which allowed jobs to run again. Confirmation from team members indicated that actions were successfully executing after the mitigation.

The incident was cancelled.

Have had two customers so far that are able to login to DAT one web on mobile devices (one on android and one on an Iphone). Both are stuck in a spinning wheel after they complete a search. Tried every troubleshooting step I could with no resolution. As per mixpanel, both have the error :

Distinct ID Before Identity
auth0|616db6eb6465a20071ed2fe2
Failure Description
too many distinctIDs mapped together, Max limit: 500
Failure Reason
errTooManyIds

I believe this might be related to change to SV4 Fix was deployed

NOC Testing JIRA ticket automation workflows The Test is completed.

Assuresign is broken for a subset of Assuresign Contracts since 7/2
Known affected customers today:
4d Logistics Enterprises
Rowdy Logistics
RDS Logistics Group
CSD Express
DAL USA
Greatway Transportation Inc
HC SCHMIEDING LLC
LYNK LOGISTICS The increase in file size mitigated the issue

health check job of the BA Shipper failed (DAT_IQ_PROD.SHIPPER_BA_ANALYTICS) which will cause benchmark scores seen by customers may not update as scheduled, meaning the platform could serve stale, outdated, or even incomplete results. The scheduled job runs have been completed, hereby resolving the incident.

A fix was deployed, and the missed data during the outage was backfilled.

We have received few customers are still reporting issues with TMS postings. Paged CSB on call to check and update. The incident was resolved by restarting the FTP host, which resulted in no errors or latency, and customers confirmed that their postings via TMS began working fine thereafter. The status of the incident is marked as resolved.

Testing PIR automation

It's a test incident created manually by the Serenity Now team.
https://transcore.jira.com/browse/TMS-8182 Rootly auto resolved this incident since the alert ([Serenity Now] Te...) attached to the incident was resolved.

Please ignore the test incident. Test Completed.

This incident has been resolved.

This incident has been resolved.

This incident has been resolved.

This incident has been resolved.

This incident has been resolved.

This incident has been resolved.

This incident has been resolved.

This test is resolved.

This incident has been resolved.

The rollback was completed and users are able to post loads.