No edit summary |
No edit summary |
||
| (3 intermediate revisions by the same user not shown) | |||
| Line 18: | Line 18: | ||
! Actions | ! Actions | ||
|- | |- | ||
|Check monitoring | |Log the start of the incident and indicators | ||
|- | |||
|Check monitoring | |||
|- | |||
|Check the logging '''''less /var/log/mysql.log''''' | |||
|- | |- | ||
|Check mysqld service '''''service mysqld status''''' | |Check mysqld service '''''service mysqld status''''' | ||
| Line 27: | Line 31: | ||
|- | |- | ||
|Check the ports from remote using '''''nmap -Pn databasehost''''' | |Check the ports from remote using '''''nmap -Pn databasehost''''' | ||
|- | |- | ||
|Try to login locally on the database server '''''mysql -u root -p''''' | |Try to login locally on the database server '''''mysql -u root -p''''' | ||
| Line 43: | Line 45: | ||
|- | |- | ||
|Last resort '''''reboot''''' perhaps check most recent backup first | |Last resort '''''reboot''''' perhaps check most recent backup first | ||
|- | |||
|Restore most recent backup, dump the database first | |||
|- | |||
! Post actions | |||
|- | |||
|Reactivate monitoring | |||
|- | |||
|Log actions and problem resolution | |||
|- | |||
|Keep an eye on monitoring and logfiles | |||
|- | |||
|Log rootcause, lessons learned and pending actions | |||
|} | |} | ||
Latest revision as of 20:49, 16 April 2020
Template structure
| Issue description |
|---|
| How to troubleshoot it |
| How to fix it |
Template Example
| Cannot connect to database |
|---|
| Check if mysqld is running and if the ports are available on the localhost and from the host you try to connect from. See if the service is running and if you can connect locally. This should usually provide some info to start troubleshooting from. |
| Actions |
| Log the start of the incident and indicators |
| Check monitoring |
| Check the logging less /var/log/mysql.log |
| Check mysqld service service mysqld status |
| Check load on database server using top |
| Check the ports using nmap -Pn localhost |
| Check the ports from remote using nmap -Pn databasehost |
| Try to login locally on the database server mysql -u root -p |
| Try to login from the remote host mysql -h databasehost -u dbusername -p |
| Fixing (you might kill the webserver/monitoring first) |
| Restart mysql service service mysqld restart |
| Follow the logging tail -f /var/log/mysql.log |
| Check mysql service service mysqld status |
| Last resort reboot perhaps check most recent backup first |
| Restore most recent backup, dump the database first |
| Post actions |
| Reactivate monitoring |
| Log actions and problem resolution |
| Keep an eye on monitoring and logfiles |
| Log rootcause, lessons learned and pending actions |