No edit summary |
No edit summary |
||
| (6 intermediate revisions by the same user not shown) | |||
| Line 18: | Line 18: | ||
! Actions | ! Actions | ||
|- | |- | ||
|Log the start of the incident and indicators | |||
|- | |- | ||
| Check load on database server using '''''top''''' | |Check monitoring | ||
|- | |||
|Check the logging '''''less /var/log/mysql.log''''' | |||
|- | |||
|Check mysqld service '''''service mysqld status''''' | |||
|- | |||
|Check load on database server using '''''top''''' | |||
|- | |- | ||
|Check the ports using '''''nmap -Pn localhost''''' | |Check the ports using '''''nmap -Pn localhost''''' | ||
| Line 25: | Line 32: | ||
|Check the ports from remote using '''''nmap -Pn databasehost''''' | |Check the ports from remote using '''''nmap -Pn databasehost''''' | ||
|- | |- | ||
| | |Try to login locally on the database server '''''mysql -u root -p''''' | ||
|- | |- | ||
| Try to login | |Try to login from the remote host '''''mysql -h databasehost -u dbusername -p''''' | ||
|- | |- | ||
! Fixing (you might kill | ! Fixing (you might kill the webserver/monitoring first) | ||
|- | |- | ||
| Restart mysql service '''''service mysqld restart''''' | |Restart mysql service '''''service mysqld restart''''' | ||
|- | |- | ||
|Follow the logging '''''tail -f /var/log/mysql.log''''' | |Follow the logging '''''tail -f /var/log/mysql.log''''' | ||
| Line 39: | Line 44: | ||
|Check mysql service '''''service mysqld status''''' | |Check mysql service '''''service mysqld status''''' | ||
|- | |- | ||
|Last resort '''''reboot''''' perhaps check most recent backup first | |||
|- | |||
|Restore most recent backup, dump the database first | |||
|- | |||
! Post actions | |||
|- | |||
|Reactivate monitoring | |||
|- | |||
|Log actions and problem resolution | |||
|- | |||
|Keep an eye on monitoring and logfiles | |||
|- | |||
|Log rootcause, lessons learned and pending actions | |||
|} | |} | ||
Latest revision as of 20:49, 16 April 2020
Template structure
| Issue description |
|---|
| How to troubleshoot it |
| How to fix it |
Template Example
| Cannot connect to database |
|---|
| Check if mysqld is running and if the ports are available on the localhost and from the host you try to connect from. See if the service is running and if you can connect locally. This should usually provide some info to start troubleshooting from. |
| Actions |
| Log the start of the incident and indicators |
| Check monitoring |
| Check the logging less /var/log/mysql.log |
| Check mysqld service service mysqld status |
| Check load on database server using top |
| Check the ports using nmap -Pn localhost |
| Check the ports from remote using nmap -Pn databasehost |
| Try to login locally on the database server mysql -u root -p |
| Try to login from the remote host mysql -h databasehost -u dbusername -p |
| Fixing (you might kill the webserver/monitoring first) |
| Restart mysql service service mysqld restart |
| Follow the logging tail -f /var/log/mysql.log |
| Check mysql service service mysqld status |
| Last resort reboot perhaps check most recent backup first |
| Restore most recent backup, dump the database first |
| Post actions |
| Reactivate monitoring |
| Log actions and problem resolution |
| Keep an eye on monitoring and logfiles |
| Log rootcause, lessons learned and pending actions |