Jobs taking longer than usual to process

Resolved
Operational
Started over 4 years ago Lasted about 1 hour

Affected

Website
API
Voices
Updates
  • Resolved
    Resolved

    We just resolved the issue! We flushed everything on our Redis server and it wiped all of our background jobs. We'll monitor things over the next few days but things seem to be back to normal.

  • Monitoring
    Monitoring

    We have stopped Redis, the service we use to run all of our background jobs. We are going to monitor CPU usage on the servers until they are back at a reasonable level, then we will attempt to flush all Redis jobs on our server.

  • Investigating
    Update

    All of our background jobs seem to be broken at the moment. Our server has decided to reprocess every single job that's ever been processed. We're working to clear the queue to try and resolve the issue.

  • Investigating
    Investigating

    We are currently investigating this incident.