Lucky backup might save 100 days of data for InfluxData's GCP Belgium users
AWS Sydney users less fortunate – the backup was deleted
Updated Luck, rather than judgement, has given InfluxDB users some hope of restoring 100 days of data after the vendor decided to shut down its Belgium Google Cloud region. Customers depending on the database in Australia are not so fortunate.
Earlier this week, InfluxData, the company behind the open source time series database, apologized for discontinuing two cloud regions without sufficiently preparing customers.
Although it had written emails to those affected – in AWS Sydney and GCP Belgium – users were still caught out. The company later admitted it should have conducted a "scream test," where a service provider turns off access to a service. Those who are locked out "scream" and the company tells them the services will be turned off for good at a later date, giving customers time to back up their data and migrate applications.
InfluxData lost the data of customers using its services in Australia while users in Belgium are struggling to figure out if they can restore the last 100 days as a result of good fortune, CTO and co-founder Paul Dix said in a blog.
He was later quizzed on Twitter about why GCP data might be recoverable, but AWS data was not, and whether it was anything to do with the cloud services themselves.
- InfluxData apologizes for deleting cloud regions without performing 'scream test'
- MariaDB sent dollar dip warning from NYSE
- Databricks puts cards on the table format as Snowflake looks for more players
- CockroachDB hits Azure at last after five-year mission
"This is the result of not one, but multiple process failures on our part for shutting these systems down. The potential 100 days of data is from a separate backup system we have that pulls data from Kafka and writes into different buckets," he said on Twitter.
"GCP doesn't have a role in this at all, the responsibility is ours alone to bear. And we'll be changing these processes significantly going forward. Including scream tests in the future for anything that is getting closed down or shut off."
He later explained that while the backup system was deployed in AWS Sydney, "the bucket in which that backup was stored was deleted."
The decision to delete customer data and end services with sub-optimal communication continues to provoke fallout.
One user complained they had paid $3,000 for the AWS Sydney service less than a month before it was turned off. "There is nothing left now," they said on a company forum.
An InfluxData employee said they would let "the team know" about the significant expenditure.
The Register has contacted InfluxData for comment. ®
Updated to add
Late on Friday, InfluxData issued an update on its blog. CTO Paul Dix said the company had recovered the time series data for the GCP Belgium users and was in the process of getting that data back to them. "We have also recovered the user-defined functions and Tasks. I want to express my sincere gratitude to the Google Cloud Platform team for their incredible partnership in helping us recover the data," he said. The blog said the company would respond directly to users with a support ticket to tell them how to download their data. Others affected were asked to open a ticket. Users of the AWS/Sydney service have not been so lucky and data remains lost to users caught unaware when the service ended.