On Monday morning, a two hour outage was planned to transition our Decidim instance from using a private image over to using the Mautic GitHub repository and DockerHub image, with the end goal being able to have automated pipelines for deployment when a community PR is merged.
During the process, one of the encryption keys which are essential for Decidim’s functioning - which was supposed to be backed up - unfortunately was overwritten and could not be recovered.
We reached out to the hosting provider, Infomaniak, to request a rollback to backup, but it was after hours which meant we had to wait for the next morning.
The next day, we called Infomaniak again to request the rollback and they assured us it would happen in the early evening.
Upon calling to get a status update, we were informed that they did not have any backups from which to restore from. We therefore had to set about the process of removing all encrypted data and re-configuring it with new keys, which happened on Wednesday morning.