Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
CPU spikes: Lack of Indexing on OWTF db? #444
OWTF seems to struggle in huge assessments, below is after running Dirbuster, with just 2 workers, with almost NO urls found, this CPU spikes suggest that OWTF would probably benefit from indexing the DB.
The scope in question was a FULL web scan (i.e. ALL web plugins) of 35 URLs, towards the end.
Confirmed as lack of indexing: adding the following index eliminated ALL CPU spikes "idle in transaction", "SELECT", etc.
TODO: The db schema creation scripts, perhaps even the installer (which will be re-run after upgrading, maybe), should fix the DB schema, maybe OWTF should have backwards checks like this on start
Can somebody please add this to the DB init scripts? the index is still missing on develop and the spikes are therefore happening.
Simply create a btree index on the transaction_id column of the transaction_grep_association table, as above, and the spike is gone, as I troubleshooted and fixed 2 months ago above! :)
NOTE: This takes forever using the postgress DB UI, if using phppgadmin, you need to do this on an empty database to have a chance to see the full command generated.