API Logging and Troubleshooting

Log Files

Logs are split into multiple files, which can be customized via environment variables in setup_environment.sh as described in the Deployment Guide. By default, these files rotate every 5 MB and keep 5 backups.

Default File	Contents
auth.log	Authorization logs; particularly SAML2 authentication requests
access.log	All website access is logged here
permissions.log	Any changes to user permissions are logged here
db.log	All database interactions are stored here when debug is set to True
sync.log	All logs relating to data synchronization via SOAP are logged to this file
talentmap.log	All logs that fall outside of the other logs are logged to this file

Troubleshooting

Listed here are some issues one may encounter during deployment and maintenance of the software, along with solutions to try.

Problem: API responds 500 on all routes

Solution: Generally this is due to some error with the Apache service itself. Try the following:

• Restart the httpd service
• Check /var/log/httpd/error_log

Problem: error_log reports permission issues on log files

Solution: Ensure all log files specified in setup_environment.sh have permissions such that the apache user can edit and create files in the directory

Problem: SOAP synchronization hangs

Solution: This can stem from a few issues

• Ensure that the application's IP is whitelisted on the SOAP server
• Check the response objects for the specific service; sometimes the data is too large for a single payload. Decrease the maximum output rows for the service in question (talentmap_api/integrations/synchronization_helpers.py)

Problem: The API doesn't appear to be listening

Solution: Check http.d configuration to ensure the port you're visiting is the one being listened on. If it is, ensure you are requesting https if SSL is enabled.

Problem: API reports missing columns

Solution: Run python manage.py migrate to ensure the database is migrated to the current expected version, then restart httpd

Problem: Running management commands give log permission errors

Solution: Ensure the currently active user has sufficient permissions in the log directory and files

Problem: SOAP synchronization isn't running

Solution: Ensure there are synchronization jobs manage.py schedule_synchronization_job --list; if there are none, schedule them according to the SOAP Synchronization guide