Database maintenance Fri/Sat/SunWe've been hard at work the last 4 months on an entirely new infrastructure that will be considerably faster than what we have now. In prep for the migration, we need to do our standard database maintenance to shrink down our data pool, which we normally do once or twice a year as needed. This time, we need to do it to *all* databases though, which is a first. This procedure deletes old data from sites that are no longer using the service, deletes data older than 30 days for non-paying sites, and also trims down the visitor and actions logs to the last 6 months for all sites (which is all we guarantee for those particular types of data).
During maintenance on any specific server, reports are still available, but real time traffic processing is paused. Once maintenance is finished - typically 4-12 hours, depending on the server - the backlog of traffic will be processed and the server will catch back up with real time, usually within a few hours.
Normally we reserve this exclusively for the weekend but in order to get it all done in one weekend without feeling too overwhelmed, we need to start on Friday. The list of servers on Friday is relatively small though. That list is below.
You can see which server any of your sites are assigned to, either from your user homepage or from the main site preferences page for any site.
All databases will be affected by this sometime Friday-Sunday, so keep that in mind when you see reports lagging.
We have schedules for all 3 days already but are only posting Friday for now, as the other days may need to be tweaked in case there are any issues on Friday.
Database servers affected on Friday:
1, 2, 8, 53, 54, 63, 69, 70, 71, 76
Posted Thu Apr 28 2016 8:47pm by Your Friendly Clicky Admin
We started Friday @ 10:30am PST.
Posted Fri Apr 29 2016 10:49am by Your Friendly Clicky Admin
Is there any easy way to download all visitor and action logs for future reference?
Posted Fri Apr 29 2016 11:07am by harpcodave
db1, 53, 54, and 63 are still under maintenance and will be for 2-4 more hours, except for db1 which is almost done.
The other 6 are all done and caught up with real time.
Posted Fri Apr 29 2016 4:44pm by Your Friendly Clicky Admin
db1 is done and caught up.
db63 will be done in about 1 hour.
db54 will be done in about 4 hours.
db53 will be take a while longer, I'm guessing 8-10 hours. As certain as the sun rising in the east, there's always *one* database that takes forever... this is that one today.
Posted Fri Apr 29 2016 7:21pm by Your Friendly Clicky Admin
63 just finished.
Posted Fri Apr 29 2016 8:23pm by Your Friendly Clicky Admin
Saturday's maintenance started at 10am PST. The following database servers are affected.
4 9 11 14 16 18 21 22 24 26 29 30 33 34 37 39 41 42 43 44 45 46 50 51 56 58 60 62 65 67 73 75 78 80 82
Posted Sat Apr 30 2016 10:22am by Your Friendly Clicky Admin
Posted Sat Apr 30 2016 11:07am by helenejbannerman
"trims down the visitor and actions logs to the last 6 months for all sites (which is all we guarantee for those particular types of data)."
That's a big bummer. I use your service for the historical data. This can't be at least a year of data?
Posted Sat Apr 30 2016 1:13pm by aspkin
About half are done, a quarter will be done within a few hours... and the rest will be quite a while longer.
Posted Sat Apr 30 2016 3:41pm by Your Friendly Clicky Admin
Status as of now.
These servers are almost done (2 hours or less) - 9, 14, 26, 30, 37, 43, 46, 60, 62
These servers have 4-8 hours left - 16, 18, 42, 56, 58, 65, 67
All other servers are done.
Posted Sat Apr 30 2016 6:29pm by Your Friendly Clicky Admin
67 is going to be the slow one tonight. Everything should be done within 4 hours, except 67, which will definitely be at least a few hours longer.
Posted Sat Apr 30 2016 7:40pm by Your Friendly Clicky Admin
All db servers for Saturday maintenance are done except:
db18, ~1 hour left
db58, ~1 hour left
db67, ~2 hours left
Posted Sun May 1 2016 midnight by Your Friendly Clicky Admin
Sunday's maintenance started at 9:30am.
Posted Sun May 1 2016 9:40am by Your Friendly Clicky Admin
db16 and 18 (which were done yesterday) are lagging because of serious i/o on the other dbs on the same hardware that are being done today... the reverse didn't happen yesterday from what i saw but anyways, the "bad" table in question is almost done so once tha thappens, they should catch back up relatively quickly.
Posted Sun May 1 2016 11:28am by Your Friendly Clicky Admin
Is 66 under maintenance?
Posted Sun May 1 2016 1pm by normdogg
What about server 79?
Posted Sun May 1 2016 1:17pm by ivandabomb
how i can uses my subscriptions
Posted Sun May 1 2016 2:07pm by AMANEBRO
Something is wrong most of my DB are down and not listed for updates.
Posted Sun May 1 2016 5:18pm by redgsr
Most servers are done and either caught up with real time or will be caught up within an hour.
The following servers are almost done (2 hours or less) - 6, 15, 17, 25, 35, 64, 66
These servers have 3-4 hours - 20, 28, 55
And db5 is our designated "slow server" today apparently. It's looking like it has about 6 hours left.
Posted Sun May 1 2016 6:04pm by Your Friendly Clicky Admin
All servers are done with maintenance, although 7 of them are still processing their backlog so will be another few hours until they're caught up.
Thanks everyone for your patience. This is by far the biggest purge we've ever done at once, it's a bit overwhelming but it will make things a lot easier as we start moving data to new servers.
Posted Sun May 1 2016 10:11pm by Your Friendly Clicky Admin