Forums » Help & Troubleshooting



db53-58

The hardware hosting these 6 database servers is having a serious issue with one of its drives we believe. There appears to be some corruption on db58, the others appear clean, but once this is sorted out db58 will likely need to be taken offline again for some repairs.

Traffic processing has been halted on all of them to prevent any further potential corruption while we sort this out.

I'm headed to the database center now to try to resolve.

I'll try to keep this updated but check Twitter for the most up-to-date info

http://twitter.com/clicky

Posted Thu Aug 14 2014 2:16p by Your Friendly Clicky Admin


We're still investigating this trying to figure out the exact issue. One problem we're having is that this is the only hardware where we are using hardware RAID, which we had to because of the motherboard, the problem being that this masks our ability to check the SMART status of the individual drives. So we're trying to see if there's something else we can do in the meantime without having to shut down the server entirely and move each drive one at a time to another server to check its SMART status. This would likely take close to an hour, meantime all 6 db servers would be offline so that's not ideal.

Posted Thu Aug 14 2014 4:39p by Your Friendly Clicky Admin


After some mucking around and a few servers reboots yesterday, things are stable right now. db58 lost a few hours of data because it was accidentally rebooted again after it had resumed processing, but the rest of the dbs are all good. We're keeping our eyes on this.

Posted Fri Aug 15 2014 9:10a by Your Friendly Clicky Admin


You must be logged in to your account to post!