Tipsandtricks

Template structure

Issue description
How to troubleshoot it
How to fix it

Template Example

Cannot connect to database
Check if mysqld is running and if the ports are available on the localhost and from the host you try to connect from. See if the service is running and if you can connect locally. This should usually provide some info to start troubleshooting from.
Actions
Log the start of the incident and indicators
Check monitoring
Check the logging less /var/log/mysql.log
Check mysqld service service mysqld status
Check load on database server using top
Check the ports using nmap -Pn localhost
Check the ports from remote using nmap -Pn databasehost
Try to login locally on the database server mysql -u root -p
Try to login from the remote host mysql -h databasehost -u dbusername -p
Fixing (you might kill the webserver/monitoring first)
Restart mysql service service mysqld restart
Follow the logging tail -f /var/log/mysql.log
Check mysql service service mysqld status
Last resort reboot perhaps check most recent backup first
Restore most recent backup, dump the database first
Post actions
Reactivate monitoring
Log actions and problem resolution
Keep an eye on monitoring and logfiles
Log rootcause, lessons learned and pending actions