Unplanned downtime – Aug 18

Home Forums General Unplanned downtime – Aug 18

New Topic
Viewing 1 post (of 1 total)
  • Author
    Posts
  • #136609
    Christina
    Moderator

    Starting around 10:15pm GMT, a database corruption has rendered most of our services inoperable. The chain of events that lead to the malfunction has ultimately been caused by human error.

    Issue has been identified immediately, however the extent of the data inconsistency was massive. Efforts to piece it back together from backups and logs took more than 5 hours.

    Currently the service and platform are fully operational, after having been fully or partially down for about 6.5 hours. Events started before issue arose have continued to operate normally.

    We deeply regret the situation and already enforced stricter handling rules into the CICD pipeline to ensure this never happens again. Thank you for your patience and understanding.

Viewing 1 post (of 1 total)
  • You must be logged in to reply to this topic.