Scheduled Maintenance of Khabarovsk Region
Incident Report for Misaka Network, Inc.
Resolved
All done.
Posted Nov 19, 2020 - 22:32 UTC
Update
Migration is still in progress, remaining 23% and ETA 50 minutes.
Posted Nov 19, 2020 - 21:48 UTC
Update
Migration is still in progress, ETA 1.5 hrs
Posted Nov 19, 2020 - 19:48 UTC
Update
Server is back though IO performance is not fixed.

We'll continue the migration and let you know once it's done.
Posted Nov 19, 2020 - 16:09 UTC
Update
Updates from provider: We have checked and see that your issue is demanded more time to be resolved. Our engineers from HDWR are working on it!
Posted Nov 19, 2020 - 06:18 UTC
Update
Still waiting for our provider to change the RAID controller. Service is down for 30hrs+
Posted Nov 18, 2020 - 14:19 UTC
Update
Passing the RUKHV01C08 node to our provider's hardware team, this node will be temporarily offline for diagnosis.
Posted Nov 16, 2020 - 09:36 UTC
Update
All VMs on C08 are now migrated to new system.
Due to the I/O performance issue, we will pause migration of other nodes before it's solved.
Posted Nov 14, 2020 - 06:50 UTC
Update
Migration still in progress (18/50)
Posted Nov 14, 2020 - 05:03 UTC
Update
Data migration is still in progress, now 20%.
Also, We noticed that the RAID controller installed is not working well.
Unfortunately it's weekend and we'll ask our provider to replace it next week. Before it's replaced, we'll limit IOPS and throughput of each VM to reduce the load.
Posted Nov 14, 2020 - 03:58 UTC
Update
Data migration is still in progress.
We except that it can be done in 12 hours.
Posted Nov 14, 2020 - 00:50 UTC
Update
We're still waiting for the data migration. Current speed is only around 80MB/s.
We'll contact our provider to check the SSD speed issue next week.
Posted Nov 13, 2020 - 23:26 UTC
Identified
Maintenance took longer than excepted due to slow I/O performance.

Approximately 50 servers are still down.
Posted Nov 13, 2020 - 22:20 UTC
This incident affected: Cloud Servers and Datacenters (Khabarovsk, Russia - (KHV01)).