DAT Solutions Internal

Back
Jun 30, 2026 3:18 AM PDT

Non-pay cancel reactivation records missing from daily list

Updates

  • Jun 30, 2026 9:57 AM PDT

    The reactivation team is not seeing non-pay / credit-card-decline cancels on the daily list. Initial review indicates the records may be stuck in progress, and the attached Salesforce report filtered to Credit Card Decline for this week shows no new opportunities for the last two days, which is unusual.
    This is likely customer impacting for customers attempting self-reactivation after a non-pay cancel; if the underlying Salesforce/reactivation flow is failing, customers may be unable to complete self-reactivation and may instead be directed to contact support. Resolution summary

    • Root cause: A Salesforce email template update on 6/24 caused failed payment transactions to become stuck in an In Progress state, preventing notification emails and disrupting the reactivation flow.

    • Mitigation and resolution: Reverted the Salesforce email template back to the original version, enabling failed-payment emails to resume and allowing transactions to move out of In Progress. This mitigated the issue and halted the customer impact caused by the stuck records.

    • Scope and impact: Approximately 669 accounts were impacted (accounts receiving free services) due to the stalled transactions. The incident status is MITIGATED as of 2026-06-26 14:43 MDT.

    • Next steps / remediation plan:

      • Review the 669 impacted accounts to identify which have already paid and exclude them from further action.
      • For remaining accounts, choose a remediation path:
      • Option A: Retry payment and, if it fails, cancel the service.
      • Option B: Immediate cancellation for all remaining impacted accounts.

    • Follow-ups and actions created:

      • Schedule Post-Incident Review meeting
      • Ensure PIR (Post-Incident Report) document is complete and up to date
      • Capture current impact to customers

    Current status: mitigated (mitigated_at 2026-06-26 14:43 MDT).

    Resolved
  • Jun 30, 2026 3:18 AM PDT (6 hours and 39 minutes earlier)

    The reactivation team is not seeing non-pay / credit-card-decline cancels on the daily list. Initial review indicates the records may be stuck in progress, and the attached Salesforce report filtered to Credit Card Decline for this week shows no new opportunities for the last two days, which is unusual.
    This is likely customer impacting for customers attempting self-reactivation after a non-pay cancel; if the underlying Salesforce/reactivation flow is failing, customers may be unable to complete self-reactivation and may instead be directed to contact support. Mitigation summary (as described in the incident report):

    • Stop the issue by reverting the Salesforce email template change from 6/24, so failed payment emails resume and transactions no longer remain stuck in Progress. This re-enables the failure-notification flow and unblocks accounts.
    • Account review: Review the 669 impacted accounts to determine if any have paid their outstanding balance; exclude accounts with successful payments from further action.
    • Remediation options for remaining accounts:
      • Option A: Retry payment (attempt to charge the card again; if payment fails, cancel the service).
      • Option B: Immediate cancellation for all remaining impacted accounts.
    • Additional long-term control (noted in the discussion): Restrict or delay production edits to email templates in Salesforce to prevent similar issues.
    • Action items to support mitigation and learning:
      • Schedule Post-Incident Review meeting.
      • Ensure PIR (Post-Incident Report) document is complete and up to date.
      • Capture current impact to customers.
    Monitoring