[Completed][Maintenance][Urgent] Win6

[Steven Byrd]

we have picked up a predicted failure of the RAID system within the WIN6 shared hosting server.  The issue is non service impacting right now and all services are operating properly.  However, considering the nature of this sort of alert, when this component fails, it could disrupt the RAID array on this server and lead to an extended outage.

*Plan to Action*
- we have racked a new server and are preparing to bare metal restore WIN6 to this new server over our LAN and using a private IP so as not to conflict with the public web server
- we are taking an immediate backup of the current live WIN6 server right now so as to minimize any data loss that may occur with our backup data (current backup of win6 is from last night, but, we wanted a newer restore point)
- once the new backup finishes (running now), we will immediately commence the Bare Metal Restore to the new server (we expect this process to take 1-3 hours)
- As soon as the Bare Metal Restore to the new server is completed, we will be performing a full defrag on the new server (which may take several hours), and then take the old WIN6 server down and bring up the new WIN6 server.  The defrag is required as a failing RAID card can cause some fragmentation issues.
- We will maintain the old WIN6 server on the internal private LAN in case any changed data is required by any clients.  We do ask that clients try not to make any changes to their data on this server until this process is complete (wherever possible)

*Expected Impact*
- we only anticipate a service interuption of a few minutes while we flip the switch on the new WIN6 server to go live and take down the old one
- any changed data between the backup that is being taken now and the time we get the new server up and running will be lost.  However, customers can request us to recover whatever files they may require from the original server, which will have up to the minute data available.  Based on our differential data metrics, we expect very nominal impact in this capacity, especially if customers can hold off on some front end website updates until tomorrow

[Steven Byrd]

Bare metal is at good progress at this point.

[Steven Byrd]

The defrag on the new server is continuing, though it appears will take a while to do.  Once that is done, we'll flip the network ports so that the new is active and the old is not.  A minute, if not less, outage during traffic rerouting is expected.

[Steven Byrd]

This is complete and the new win6 is running smoothly.  No downtime happened.