Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLI utility to validate database schema #1263

Closed
cigamit opened this issue Jan 24, 2018 · 10 comments
Closed

CLI utility to validate database schema #1263

cigamit opened this issue Jan 24, 2018 · 10 comments
Labels
enhancement General tag for an enhancement

Comments

@cigamit
Copy link
Member

cigamit commented Jan 24, 2018

Over the years, Cacti upgrades can result in inconsistencies between the published Cacti schema and the actual user schema. Create a utility that audits and/or repairs schemas that are not upto specification.

@cigamit cigamit added the enhancement General tag for an enhancement label Jan 24, 2018
@netniV
Copy link
Member

netniV commented Jan 24, 2018

One issue you will have with this, because plugins have direct access to the database via the functions db_execute, etc, they can add their own fields to the core tables. Best practice would be that shouldn't be allowed because it modifies the core schema. One example is thold which although a cacti-owned plugin is not present on every database; yet this plugin actively creates fields in the base host table.

It could be useful to have the plugins supply details of the tables/fields that they create in order to properly check against these too.

@cigamit
Copy link
Member Author

cigamit commented Jan 24, 2018

There is an API for that (which includes a table too), but not all plugins use it. So, we would have to work around things that are 'new' and simply identify them.

cigamit added a commit that referenced this issue Jan 25, 2018
Allow Auditiong of the Database
@cigamit
Copy link
Member Author

cigamit commented Jan 25, 2018

I dropped in an audit tool. It does not currently verify plugin tables or columns. It checks for sequencing, and I know some plugins did not respect sequencing. So, may have to pull that out. There is a lot of work required for the indexes and make sure little things don't cause false positives.

cigamit added a commit that referenced this issue Jan 25, 2018
Removing some attributes that have little impact.  There are still
issues in the index parsing, and indexes that are missing are not caught
presently.
@cigamit
Copy link
Member Author

cigamit commented Jan 25, 2018

More updates, makes doing the check easy, but still some things are not caught, like the engine, and missing indexes, and missing columns. Also, right now no repair is performed.

@netniV
Copy link
Member

netniV commented Jan 25, 2018

Looks good so far. Will have to remember to update this on any DB change now too ;-)

Just wondering if the primer should be done during upgrade so it's only done once?

@cigamit
Copy link
Member Author

cigamit commented Jan 25, 2018

I was thinking about that last night too. My schema was a little wrecked too. Most of it inconsequential, but still not right.

cigamit added a commit that referenced this issue Jan 27, 2018
The utility includes the options:
--report, --alters, --report, --repair, and --load (for developers)

The load will create the default tables from the current schema.  The
developer who runs with the --load option is still responsible for
loading the 'revised' table contents into the two string variables
though.  The --repair option will repair all tables and would be
suitable for the installers purposes.
@cigamit
Copy link
Member Author

cigamit commented Jan 27, 2018

Now that I cleaned up most of my schema and can see the forest from the trees, I've found a few indexes missing from the default schema and a few columns that were supposed to have been dropped.

@cigamit
Copy link
Member Author

cigamit commented Jan 28, 2018

This utility is implemented and not working pretty well. In addition, it was used to remediate all the issues when upgrading from earlier version of Cacti. Going to close this.

@cigamit cigamit closed this as completed Jan 28, 2018
@ronytomen ronytomen changed the title Create utility to audit the Cacti schema for issues CLI utility to validate database schema Feb 5, 2018
@netniV
Copy link
Member

netniV commented Feb 13, 2018

Not working well? Think that is a typo there ;-)

Additionally, I just had a thought that if this generated a warning for a mismatch of the database, TravIs CI would be able to pick that up which could help ensure we have the cacti.sql file updated?

@cigamit
Copy link
Member Author

cigamit commented Feb 13, 2018

Reach out to Rony via traditional email. He currently owns the Travis process. If it's doable in a pull request, all the better.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 30, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement General tag for an enhancement
Projects
None yet
Development

No branches or pull requests

2 participants