starslingdev - Jobs are delayed from starting for specific customers. – Incident details

All systems operational

Jobs are delayed from starting for specific customers.

Resolved
Degraded performance
Started about 1 month agoLasted about 8 hours

Affected

GitHub Runners

Degraded performance from 1:38 PM to 6:07 PM, Operational from 6:07 PM to 6:51 PM, Degraded performance from 6:51 PM to 9:58 PM

Updates
  • Resolved
    UTC
    Resolved

    This incident has been resolved. Job queue times are nominal. We pinpointed the secondary factor as an issue with a specific cloud provider and shifted load.

  • Investigating
    UTC
    Investigating

    We have observed a reoccurrence and are currently investigating this incident.

  • Resolved
    UTC
    Resolved

    This incident has been resolved. Zero recurrence in the last 2 hours.

  • Monitoring
    UTC
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Identified
    UTC
    Identified

    We've identified an issue with Github API rate limiting that causes jobs to be delayed up to 25mins for some customers. We are continuing to work on a fix for this incident.