DBA is nothing without a recent backup

If you are starting-up your career as a DBA then you should be aware of the job responsibilities of a Database Administrator. Data is the heart of the enterprise, it’s crucial for you to protect it. For a DBA, database is everything, like a patient to a doctor. You always have to be proactive and be prepared for the worst situation. You never know anything can happen at anytime. It’s your job as an administrator to make sure that backups are performed and that backup tapes are stored in a secure location. Never trust tomorrow because emergency does not knock the door before entering.

Couple of months ago on a Sunday morning I got a call from one of my old friend, he asked me to come online immediately in a low voice.. Follow the conversation to know what happened?

Friend: I am in a very critical situation, don’t know what to do.. Can you please help me?
Myself: What happened, is everything all right?
Friend: I don’t know what exactly happened this morning to the production database, I got a call from the operations manager that the application is not authenticating and giving some connection error. He usually generates some analytic reports on holidays from home. Initially I thought this is some service problem, but later I found that the database is not connecting at all and showing ‘Suspect’ with exclamation mark in SSMS. 
Myself: Oh God! the database is corrupted!! Take the database to Emergency mode and try to see which page is corrupted by applying DBCC CHECKDB command then recover the database.
Friend: The database is not coming up in emergency mode, showing error “cannot be opened due to inaccessible files or insufficient memory or disk space.”
Myself: OK, now since we don’t have any options left other than recycling the SQL Server service. Let’s recycle the service.
Friend: Recycled but no luck! Still showing the same error. 
Myself: Did you try any third party application to check your datafile or any online solutions?
Friend: Yes, I did.. tried all the options and solutions from the internet but no luck so far..
<<We both tried all the possibilities to recover the database without any luck>>
Myself: Dude, its showtime.. call up your manager, take permission and restore the database to the most recent possible.
Friend: Called up, he said fine, most recent would be great but make sure we have all the transaction data available till Saturday midnight.
Myself: Great! Go ahead and restore the database.. 
Friend: Prasad.. there is another problem now.. we had some storage problem the whole week and due to that the backup job failed on Friday and I didn’t take any backup yesterday, thought the storage guys will fix the issue over the weekend and the backup job will start automatically. Did not know, like this will be happened?
Myself: What?? Don’t you have a recent backup of your production system? how can you be so careless? Do you thing SQL Server will ping you before going down?
Friend: I am sorry.. Now please give me some advise to resolve this issue.. I don’t know what to do now?
Myself: Never mind, I don’t have any solution for you neither Microsoft has.. update your resume and start applying from today..
>>Somehow he convinced his manager and the higher authorities with one day business data loss.

This was my friend’s story and a lesson for us.

Data backup is an insurance plan. Important files are accidentally deleted all the time. Mission-critical data can become corrupt. Natural disasters can leave your office in ruin. With a solid backup and recovery plan, you can recover from any of these. Without one, you’re left with nothing to fall back on. If a system have been running smoothly so far then that does not mean it will run that smooth for the rest of the year or so.. Always keep a backup as recent as possible. If you have automated the backup task then do verify your backup files regularly. Do not rely on the automation job,

Prasad Sahoo

Prasad Sahoo, is an enthusiast of Microsoft technologies in general and a passionate database professional, mainly focusing on SQL Server performance tuning and business intelligence. He has contributed a long time of his career working on SQL Server and other RDBMS. If he is not busy with SQL stuff then he must be watching some Hollywood movies.

Leave a Reply

%d bloggers like this: