RetiredDBA.com

Blog covers all things for Microsoft SQL Server

All Database Administrators Man Your Battle Stations

So awesome SQL Server Database Teams always have meticulous battle plans that are used to implement projects, upgrades, SQL Server service packs, SQL server cumulative updates, etc. into production.
Right?

Well let’s just say it’s a long, involved, process that will continually change for each new project. The entire IT staff including the database administrator team must contribute to flawlessly execute an entire project battle plan. Hey, I’m talking multiple iterations of the battle plan before you get even close to comfortable. Plus, you will need hundreds of iterations in developer testing, QA testing, production simulation testing and finally production implementation.

The project battle plans need as much detail as your disaster recovery plans. Of course you have a disaster recovery plan.
Right?

What’s in the SQL Server Database Team battle plan?
Specific associates assigned to lead the battle plan for each team, each associate name and number on-call to answer and resolve issues, associates for testing and finally product owners to sign off. This includes all DBAs and the times during the process they are on-call.

  • Expected times each team will need to validate and test their stuff in each phase
  • Specific break points to verify a phase has been completed or rollback the implementation
  • Plan to roll back the implementation at each critical phase (hopefully never used)
  • Responsible teams on-call and ready to confirm a critical phase has been completed.
  • Project owners that are required to sign off before letting production users back onto systems.
  • Name and phone numbers of all team contacts for after sign-off support.
  • Plans for fixing any issues that arise via a hot fix or patch depending on the severity of errors.
  • And on and on

These items are just the tip of the iceberg behind all the details.

SQL Server database administrators need to be ready for the worst but plan for the fastest implementation possible with zero major errors.
While you can’t control the entire outcome, make sure your all database changes are good to go.

A real-life example:

  • Before: It used to take from Friday night till Monday morning to implement any system upgrade. And then 1-2 weeks to fix all bugs while the production users, developers and support staffs were swamped trying to identify and fix all the issues.
  • After: Implementing any project with a battle plan changed that to a week night automated upgrade in less than 2 hours  (no major impacts or disruptions.)  to  4 hours (worst case scenario with rolling back changes) .
  • From 10 days to 2 hours not bad for starters.  This was achieved with better automated QA testing, database update automation and code deployment automation. And of course all IT teams, product owners and end users involved from start to finish to make it all happen.