Trouble Shooting Library Website
carolyncole edited this page Dec 15, 2021
·
2 revisions
There are three main reasons the library site will go down
- Apache Errors: See Apache Debugging
- Postgres Errors: See MariaDB Debugging
- Solr: See Solr Debugging (note this is unlikely to bring the entire site down)
Viewing the Load Balancer can also help to diagnose some issues.
Please log into the library servers library-prod1, library-prod3, and library-prod4 to check the apache server
ssh pulsys@library-prod1
ps -ef |grep apache
If there are more than ten apache processes the disk has likely dropped. Reboot the server
sudo /sbin/reboot
Otherwise check the connection to mariadb
ssh pulsys@mariadb-prod2.princeton.edu
sudo su root
mysqladmin status
That should respond quickly and not have a ton of slow queries. For example:
> mysqladmin status
Uptime: 8295460 Threads: 11 Questions: 1519382769 Slow queries: 0 Opens: 5741634 Flush tables: 1 Open tables: 2000 Queries per second avg: 183.158
If you do not get a quick response or things look weird reboot the server
sudo /sbin/reboot
Otherwise check the connection to solr
cd <local pul_solr directory>
bundle exec cap library-prod solr:console
If the graphs to the right are full, or there are lots of errors in the log. Stop the solr service (It may take a minute or two to stop)
sudo service solr stop
sudo service solr start