Majority of customer servers are now online and are responsive.
Teams are continuing to work on the remaining servers showing offline.
Dec 04, 2023 - 11:18 UTC
A number of customer servers are now beginning to come online.
Our teams are working as fast as possible to resolve the underlying issues and restore full working services.
If your site or service is not yet online, do be assured we are working on it.
Dec 04, 2023 - 02:40 UTC
Our network and systems teams have made good progress and we are pleased to report the majority of the compute pod known as P1 is operating customer workload normally.
We will continue to provide status updates regarding our other PODs as soon as possible.
Dec 04, 2023 - 02:13 UTC
Our teams continue to work on the unprecedented challenges.
We apologise for the prolonged maintenance and are doing everything possible to resolve.
We will continue to provide further updates.
Dec 03, 2023 - 22:16 UTC
Again we wish to apologise for the additional time taken in this maintenance.
Our teams have come against some further challenges and are working to address these as quickly as they are able.
Timelines:
While we are not able to provide specific times our engineers are working to address the issues.
We anticipate more services to come online shortly, within the next couple of hours.
Dec 03, 2023 - 19:13 UTC
We want to provide a new update on progress.
POD 4:
Some customers hosted in compute pod STL P4 will be starting to see services coming back online.
Other PODs:
Our teams are working diligently to bring services back online.
While we cannot provide a definitive time, we anticipate this will be in the next couple of hours.
Dec 03, 2023 - 16:40 UTC
Challenges:
Our teams are have been addressing some challenges as certain aspects of the migration to the new facility have proved to be more time consuming than initially anticipated.
We understand the importance of a swift resolution, and our technicians are working diligently to overcome these challenges.
Core Network Status:
On a positive note, the core network is now online, marking a significant point in the maintenance process.
Management Services:
We have initiated the process of bringing up management services, this will be done gradually in a controlled manner.
Customer Services:
Servers hosting our customer workloads will be brought online as soon as possible.
We currently anticipate this to be within the next two hours.
Dec 03, 2023 - 14:41 UTC
As of now, the maintenance is running slightly behind schedule.
Revised Schedule:
We anticipate the first servers to begin coming online between 8 and 9am CST. Subsequent servers will follow shortly after.
We understand the importance of your services and apologize for any inconvenience this delay may cause. Our team is actively working to expedite the process and ensure a smooth transition.
Further Updates:
We will continue to monitor the maintenance progress closely and provide further updates as necessary. Your patience during this time is greatly appreciated.
Dec 03, 2023 - 13:04 UTC
Revised Schedule:
We anticipate the first servers to begin coming online between 6 and 7am CST. Subsequent servers will follow shortly after.
We understand the importance of your services and apologize for any inconvenience this delay may cause. Our team is actively working to expedite the process and ensure a smooth transition.
Further Updates:
We will continue to monitor the maintenance progress closely and provide further updates as necessary. Your patience during this time is greatly appreciated.
Thank you for your understanding.
Dec 03, 2023 - 11:40 UTC
Dec 03, 2023 - 06:00 UTC
Scheduled - Maintenance Window:
From 00:01 CST on December 3rd
Duration:
Approximately 6 hours
Nature of Maintenance:
We are implementing essential changes to our infrastructure to enhance the quality and reliability of our services. This includes a significant upgrade to our colocation facility in the St. Louis location.
Facility Upgrade:
Miss Group, representing Hostek, has invested over $1 million in new hardware and networking infrastructure. The new facility will feature full SSD storage backed by NVME, providing improved performance, stability, and future growth opportunities.
Facility Migration:
Following the facility upgrade, we will be migrating our operations to the newly improved facility. This migration is a crucial step in ensuring that you receive the benefits of the upgraded infrastructure. We anticipate the entire facility will be migrated within the 6-hour window.
Sunday, December 3, 2023