November 16, 2020 4:10pm – This post comes after the resolution of the issue as the server failure also affected this web site. At approximately 3:20pm, one of our web servers experienced a hardware issue. The equipment was pulled from the rack and repaired. However, when the server was brought back online, the database server did not restart resulting in many users seeing messages like “Unable to establish database connection”. Initially, we thought that there had been some corruption of the database during the failure. However, we found that a repair previously made on a database on the server was not properly cleaned up by the technician. The server was trying to load the corrupted backup of the database which was repaired (months ago) and was failing. We managed to locate the bad database and remove it and service was restored at approximately 4:05pm.
This server will be monitored on our test bench for a few days before being returned to the production rack during a scheduled maintenance window.
If you are still experiencing problems which might be related to this issue, please email us at support@internetpro.net or call 256-547-6817. We do apologize for any inconvenience this may have caused.