Duration: 2 days, 13 hours and 30 minutes

Affected Components: S3 Gateway

May 5, 16:44:36 GMT+0
Monitoring - We implemented a fix and are currently monitoring the result..

May 6, 16:00:00 GMT+0
Resolved - This incident has been resolved..

May 7, 13:36:29 GMT+0
Postmortem - The underlying cause was a DNS resolution incident over Scaleway's cluster, affecting our gateway instances: <https://status.scaleway.com/incidents/clnwm841161t>.

May 5, 00:00:00 GMT+0
Investigating - During the night, the team investigated the incident and implemented mitigations that have partially reduced the impact on the public gateway’s performance. We are working with our partners to mitigate the underlying infrastructure issue and are closely monitoring the situation. Further updates will be provided as soon as they become available..

Duration: 6 hours and 17 minutes

Affected Components: S3 Gateway

Jan 26, 15:34:24 GMT+0
Monitoring - We implemented a fix and are currently monitoring the result..

Jan 26, 16:16:55 GMT+0
Resolved - This incident has been resolved..

Jan 26, 10:00:00 GMT+0
Investigating - We are currently investigating this incident. You may experience some failures in the downloads..

Jan 26, 15:07:41 GMT+0
Identified - We are continuing to work on a fix for this incident..

Duration: 4 hours and 14 minutes

Affected Components: Composer, , S3 Gateway, Coordinator [Europe] →

Dec 16, 10:59:13 GMT+0
Resolved - This incident has been resolved..

Dec 16, 09:50:00 GMT+0
Investigating - We are experiencing some issues with the collection of agent errors. This has affected the detection of their status, as well as the download of objects stored in the impact agents. The operation on the swarm resources may have been affected by a slowdown..

Duration: 18 minutes

Affected Components: Metadata

Dec 1, 09:00:00 GMT+0
Identified - Perceived impact: brief (<15min) interruption of writes, during the switch of the coordinator's data services to new hardware. Interruptions may occur at different times for different clients during the maintenance window.

Dec 1, 09:00:01 GMT+0
Identified - Maintenance is now in progress.

Dec 1, 09:18:29 GMT+0
Completed - The maintenance scheduled for today has been postponed. We will communicate the new maintenance window as soon as it is confirmed..

Duration: 6 hours

Affected Components: Metadata

Nov 24, 09:00:00 GMT+0
Identified - Perceived impact: brief (<15min) interruption of writes, during the switch of the coordinator's data services to new hardware. Interruptions may occur at different times for different clients during the maintenance window.

Nov 24, 09:00:01 GMT+0
Identified - Maintenance is now in progress.

Nov 24, 15:00:00 GMT+0
Completed - Maintenance has completed successfully.

Duration: 14 days, 23 hours and 54 minutes

Affected Components: Report and Quota API

Dec 2, 07:24:17 GMT+0
Resolved - This incident has been resolved, you will find all quota updated. If you find any discrepancy or misalignment, please contact us and we will perform a detailed check..

Nov 17, 07:30:00 GMT+0
Identified - We are currently investigating this incident. For the moment, you will see the quota of your buckets, projects, and tenants stuck at November 17th. We are working to restore the situation and realign the data to their real value. We apologize for the inconvenience..

Nov 25, 16:34:41 GMT+0
Monitoring - We implemented a fix and are currently monitoring the result. You will find the amount progressively recalculated over the next few hours..

Duration: 14 days, 15 hours and 51 minutes

Affected Components: Metadata, S3 Gateway

Nov 12, 19:30:00 GMT+0
Investigating - We are currently investigating this incident..

Nov 21, 17:13:50 GMT+0
Monitoring - We implemented a fix and are currently monitoring the result..

Nov 27, 11:20:46 GMT+0
Resolved - This incident has been resolved..

Duration: 1 hour and 14 minutes

Affected Components: Composer, Metadata, Report and Quota API, S3 Gateway, Identity and Access Management (IAM), Console

Sep 23, 13:00:00 GMT+0
Identified - We are continuing to work on a fix for this incident..

Sep 23, 14:14:23 GMT+0
Resolved - This incident has been resolved..

Duration: 11 hours and 37 minutes

Affected Components: S3 Gateway

Sep 13, 07:11:47 GMT+0
Identified - The issue has been identified and we are implementing a solution for it..

Sep 12, 22:00:00 GMT+0
Investigating - We are currently investigating this incident..

Sep 13, 07:49:36 GMT+0
Identified - All the uploads should now be operational. Downloads may still experience failures..

Sep 13, 09:36:58 GMT+0
Resolved - This incident has been resolved..

Duration: 2 days, 21 hours and 45 minutes

Affected Components: S3 Gateway

Sep 8, 14:57:31 GMT+0
Monitoring - We implemented a fix and are currently monitoring the result..

Sep 11, 14:14:00 GMT+0
Resolved - This incident has been resolved..

Sep 2, 17:30:00 GMT+0
Investigating - We are currently experiencing partial system slowness that is currently under investigation. We apologize for the inconvenience and appreciate your patience.

Duration: 8 hours and 43 minutes

Affected Components: Console

Aug 2, 07:00:00 GMT+0
Investigating - We are currently investigating this incident. It may be related to a slowdown in infrastructure..

Aug 2, 15:43:00 GMT+0
Resolved - This incident has been resolved..

Aug 2, 13:30:00 GMT+0
Monitoring - We have applied a fix and we are monitoring the status of the system.

Duration: 5 days, 4 hours and 26 minutes

Affected Components: Metadata, S3 Gateway

Jun 25, 11:30:00 GMT+0
Investigating - We are currently experiencing partial system slowness. We apologize for the inconvenience and appreciate your patience..

Jun 30, 15:56:15 GMT+0
Resolved - This incident has been resolved..

Duration: 4 days, 12 hours and 35 minutes

Affected Components: Composer, Metadata, Console, Identity and Access Management (IAM), Report and Quota API, S3 Gateway

Jun 18, 22:30:00 GMT+0
Investigating - We are currently investigating this issue. .

Jun 20, 16:06:25 GMT+0
Monitoring - We implemented a fix and are currently monitoring the result..

Jun 23, 11:04:52 GMT+0
Resolved - This incident has been resolved..

Duration: 5 hours and 59 minutes

Affected Components: Composer, Identity and Access Management (IAM), Console, S3 Gateway

Jun 12, 05:30:00 GMT+0
Investigating - Authentication requests are currently failing. The authentication micro-service is unresponsive and the root cause is under investigation ### .

Jun 12, 08:30:00 GMT+0
Identified - The outage has been traced to a failed upstream load balancer that is no longer routing traffic to the authentication cluster. Traffic is being redirected ### .

Jun 12, 10:05:00 GMT+0
Monitoring - Traffic flow has been restored and authentication success rates are normal. Service health metrics are being watched for continued stability.

Jun 12, 11:29:03 GMT+0
Resolved - This incident has been resolved..

Duration: 1 day, 15 hours and 16 minutes

Affected Components: S3 Gateway

Jun 10, 07:30:00 GMT+0
Identified - We are currently investigating this incident. Our gateway provider has an open incident, which is affecting our system..

Jun 10, 09:41:34 GMT+0
Monitoring - Our gateway provider declared the issue as resolved. We are currently monitoring our system..

Jun 11, 12:45:00 GMT+0
Resolved - This incident has been resolved..

Duration: 2 hours and 5 minutes

Affected Components: Console

Jun 10, 08:05:00 GMT+0
Resolved - This incident has been resolved..

Jun 10, 06:00:00 GMT+0
Investigating - We are currently investigating this incident..

Duration: 10 minutes

Affected Components: Console

May 21, 21:33:17 GMT+0
Investigating - Console is down. We're investigating the root cause..

May 21, 21:43:16 GMT+0
Resolved - The incident has been resolved..

Duration: 10 minutes

Affected Components: Console

May 21, 09:13:47 GMT+0
Investigating - Console is down. We're investigating the root cause..

May 21, 09:23:17 GMT+0
Resolved - The incident has been resolved..

Duration: 24 minutes

Affected Components: Console

May 19, 17:02:38 GMT+0
Investigating - Console is down. We're investigating the root cause..

May 19, 17:26:40 GMT+0
Resolved - The incident has been resolved..

Duration: 5 hours and 57 minutes

Affected Components: Composer, Identity and Access Management (IAM), , Console, Coordinator [Europe] →

May 15, 13:27:04 GMT+0
Resolved - The database connectivity issue has been fully resolved. All systems are now operating normally. We have performed necessary measures and will continue to monitor performance closely..

May 15, 07:30:00 GMT+0
Investigating - We are currently investigating this incident..

May 15, 08:45:12 GMT+0
Monitoring - The connectivity issue with our database has been stabilized. Our systems are currently functioning as expected, and we are continuing to monitor performance closely to ensure everything remains stable..

Duration: 4 minutes

Affected Components: Console

May 6, 09:30:41 GMT+0
Resolved - The incident has been resolved..

May 6, 09:27:10 GMT+0
Investigating - Console is down. We're investigating the root cause..

Duration: 3 days, 3 hours and 46 minutes

Affected Components: S3 Gateway

May 7, 08:15:41 GMT+0
Resolved - This incident has been resolved..

May 4, 04:30:00 GMT+0
Investigating - We are currently experiencing a **partial degradation** affecting the `PutObject` API operation. A subset of users may encounter elevated error rates or increased latency when attempting to upload objects. Our engineering team is actively investigating the issue and working to restore full service. We will provide an update as soon as more information becomes available. Thank you for your patience..

May 6, 10:56:39 GMT+0
Monitoring - We implemented a fix and are currently monitoring the result..

Duration: 22 minutes

Affected Components: Composer, , Coordinator [Europe] →

Apr 8, 15:07:19 GMT+0
Completed - The migration has been completed. A new UI is now available for the Swarm section of the Composer.

Apr 8, 14:45:00 GMT+0
Identified - We are planning for a scheduled maintenance during that time. Tenant/Swarm/Project creation will be limited during the period..

Duration: 24 days, 13 hours and 11 minutes

Affected Components: S3 Gateway

Mar 29, 18:13:21 GMT+0
Investigating - We are currently investigating this incident..

Mar 29, 21:23:47 GMT+0
Monitoring - We’ve identified the issue, and our team is actively monitoring the situation..

Mar 30, 16:13:00 GMT+0
Monitoring - Yesterday, our provider experienced a major infrastructure failure, leading to a prolonged service disruption. This issue impacted multiple components, resulting in degraded performance and unavailability for some users. The provider has since resolved the underlying problem, and our services have been fully restored. While all systems are currently operational, we are closely monitoring performance to ensure continued stability. We appreciate your patience and understanding. If you encounter any lingering issues, please reach out to our support team..

Apr 23, 07:24:08 GMT+0
Resolved - After careful monitoring, performance appears to have improved, and the system is functioning properly..

Duration: 5 days, 19 hours and 12 minutes

Affected Components: Composer, , S3 Gateway, Identity and Access Management (IAM), Console, Report and Quota API, Coordinator [Europe] →

Mar 25, 11:45:11 GMT+0
Resolved - All systems should now be fully operational..

Mar 25, 09:25:59 GMT+0
Investigating - We're currently experiencing some infrastructure issues that may affect certain users. We're actively investigating the situation..

Duration: 9 days, 23 hours and 2 minutes

Affected Components: Report and Quota API

Mar 24, 14:38:56 GMT+0
Resolved - The incident has been resolved, and the aggregated quotas are now aligned with the expected standards, with a current delay of 12 hours compared to the current time. We appreciate your patience and cooperation during the resolution process. Please let us know if you need any further assistance..

Mar 14, 15:37:08 GMT+0
Identified - We are experiencing a slowdown in the quote aggregation process, resulting in higher-than-normal processing times. To maintain system stability, we are temporarily increasing the latency threshold. Users may notice delays in aggregated data compared to standard response times. Our team is actively investigating and working to resolve the issue. We appreciate your patience and apologize for any inconvenience. Further updates will be provided as we make progress..

Mar 21, 10:21:15 GMT+0
Monitoring - We implemented a fix and are currently monitoring the result..

Duration: 12 hours and 15 minutes

Affected Components: Composer

Mar 14, 12:15:00 GMT+0
Resolved - This incident has been resolved..

Mar 14, 09:00:00 GMT+0
Investigating - The buckets list and bulk node creation are unavailable due to some issue in the resolution of the https addresses.We are investigating the issue.Thanks for your patience.

Duration: 4 minutes

Affected Components: Identity and Access Management (IAM)

Mar 13, 13:37:03 GMT+0
Investigating - IAM API is down. We're investigating the root cause..

Mar 13, 13:40:34 GMT+0
Resolved - The incident has been resolved..

Duration: 16 days, 8 hours and 14 minutes

Affected Components: S3 Gateway

Feb 17, 07:00:00 GMT+0
Identified - We will be performing a long-term maintenance operation from **February 17 through the end of July** to migrate all clients’ S3 buckets to our new infrastructure. ### **What to Expect** * **Bucket-by-Bucket Migration:** Each S3 bucket will be migrated in its own maintenance window. During these individual windows, the affected bucket will be **read-only**. * **Client Notifications:** We will provide specific maintenance windows for each bucket to the respective clients in advance. * **Read-Only Impact:** While the bucket is read-only, you can still access and download existing objects. However, **write operations—including changing ACLs—will not be possible** during the migration window. * **IAM Policies Unaffected:** **Changing IAM policies** will still be **allowed** throughout the process. ### **Next Steps** Please watch for additional emails or notifications from us with the specific schedule for your buckets. If you have any questions or concerns, please reach out to our Support Team. We appreciate your patience as we work to enhance our infrastructure and improve overall performance. Thank you for your understanding and support..

Nov 3, 15:14:06 GMT+0
Completed - Maintenance has completed successfully..

Duration: 2 hours and 42 minutes

Affected Components: Composer, , S3 Gateway, Identity and Access Management (IAM), Console, Coordinator [Europe] →

Oct 16, 14:00:01 GMT+0
Identified - Maintenance is now in progress.

Oct 16, 14:00:00 GMT+0
Identified - We have scheduled a maintenance window to perform important updates to our systems. During this time, all our services will be temporarily offline and you will not be able to access our services..

Oct 16, 16:41:32 GMT+0
Completed - We are pleased to inform you that the scheduled maintenance has been successfully completed. All systems are now back online and fully operational. Thank you for your patience during this time!.

Duration: 4 hours

Affected Components: , S3 Gateway, Identity and Access Management (IAM), Coordinator [Europe] →

Jul 24, 12:30:00 GMT+0
Identified - We are planning for a scheduled maintenance. Our services will be unavailable for short periods of time during the scheduled window..

Jul 24, 12:30:01 GMT+0
Identified - Maintenance is now in progress.

Jul 24, 16:30:00 GMT+0
Completed - Maintenance has completed successfully.

Duration: 2 hours

Affected Components: Console

May 15, 14:30:00 GMT+0
Completed - Maintenance has completed successfully.

May 15, 12:30:00 GMT+0
Identified - Our Console will be put on maintenance. Console will be unavailable for short periods of time during the scheduled window..

May 15, 12:30:01 GMT+0
Identified - Maintenance is now in progress.

Duration: 3 hours

Affected Components: Composer, , Console, Coordinator [Europe] →

Apr 17, 15:30:00 GMT+0
Completed - Maintenance has completed successfully.

Apr 17, 12:30:00 GMT+0
Identified - We will be conducting essential infrastructure upgrades to enhance the scalability of our system. As part of this maintenance we need to temporarily make READ ONLY all APIs about: \- Projects \- Tenants \- Distributors Creating, updating, and deleting Projects, Tenants, and Distributors will therefore be impossible during this maintenance. We apologize for any inconvenience this may cause and appreciate your patience and understanding as we work to improve our services..

Apr 17, 12:30:01 GMT+0
Identified - Maintenance is now in progress.

Duration: 2 hours

Affected Components: S3 Gateway

Mar 27, 13:30:01 GMT+0
Identified - Maintenance is now in progress.

Mar 27, 15:30:00 GMT+0
Completed - Maintenance has completed successfully.

Mar 27, 13:30:00 GMT+0
Identified - We are planning for a scheduled maintenance during that time..

Duration: 3 days, 21 hours and 35 minutes

Affected Components: Composer

Mar 1, 18:00:00 GMT+0
Identified - We will be conducting essential infrastructure upgrades to enhance the reliability of our system. As part of this maintenance, we need to temporarily disable the automatic tenant creation feature. During this time, new tenant creation will be unavailable. However, if you require it during this period, please reach out to our support team and they will assist you promptly. **Existing tenants will not be affected, and all other system functionalities will remain operational.** .

Mar 5, 15:35:14 GMT+0
Completed - Maintenance has completed successfully. The automatic tenant creation is back online. Thank you for your patience..

Duration: 1 hour

Affected Components: , S3 Gateway, Identity and Access Management (IAM), Console, Coordinator [Europe] →

Jan 3, 13:00:00 GMT+0
Identified - We are planning for a scheduled maintenance during that time..

Jan 3, 13:00:01 GMT+0
Identified - Maintenance is now in progress.

Jan 3, 14:00:00 GMT+0
Completed - Maintenance has completed successfully.

Duration: 3 hours and 15 minutes

Affected Components: , S3 Gateway, Identity and Access Management (IAM), Console, Coordinator [Europe] →

Aug 2, 12:00:00 GMT+0
Identified - We are planning for a scheduled maintenance during that time..

Aug 2, 15:07:28 GMT+0
Identified - Maintenance is still in progress..

Aug 2, 15:14:51 GMT+0
Completed - Maintenance has completed successfully..

Aug 2, 12:00:01 GMT+0
Identified - Maintenance is now in progress.

Duration: 2 hours

Affected Components: S3 Gateway

Jul 6, 12:00:00 GMT+0
Identified - S3 database maintenance will impact S3 API usage.

Jul 6, 14:00:00 GMT+0
Completed - Maintenance has completed successfully.

Jul 6, 12:00:01 GMT+0
Identified - Maintenance is now in progress.