Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BASE assumes that a set $use_referential_integrity conf var indicates presence of RI DB functionality.. #215

Open
NathanGibbs3 opened this issue Jun 14, 2023 · 7 comments
Open

BASE assumes that a set $use_referential_integrity conf var indicates presence of RI DB functionality.. #215

NathanGibbs3 opened this issue Jun 14, 2023 · 7 comments
Assignees
@NathanGibbs3
Labels
bug Something isn't working DB Database Issues invalid This doesn't seem right LCB-TechDebt Issue exists in Legacy Code Base. We inherited it. Prod Observed in Production Environment. Stability Issue impacts or is related to App Stability. UI User Interface Issues.
Projects
Code Quality - Implementation Standar...
Milestone
1.4.6

Comments

@NathanGibbs3
Copy link
Owner

NathanGibbs3 commented Jun 14, 2023 •
edited
Loading

Item Description
File(s):
Class:
Function:
Similar Issues: #103 #212
Depends on Issue(s): #214
Dependency Type: Hard
Misc. Info.:

Expected Behavior:
BASE ONLY uses RI functionality when it is in the DB.
Current Behavior:
BASE assumes that a set $use_referential_integrity conf var indicates presence of RI DB functionality..
@NathanGibbs3 NathanGibbs3 added bug Something isn't working invalid This doesn't seem right Prod Observed in Production Environment. LCB-TechDebt Issue exists in Legacy Code Base. We inherited it. UI User Interface Issues. DB Database Issues Stability Issue impacts or is related to App Stability. labels Jun 14, 2023
@NathanGibbs3 NathanGibbs3 added this to the 1.4.6 milestone Jun 14, 2023
This was referenced Jun 14, 2023
Open
Open
@NathanGibbs3 NathanGibbs3 self-assigned this Jun 20, 2023
@NathanGibbs3 NathanGibbs3 mentioned this issue Jun 12, 2023
Open
NathanGibbs3 added a commit that referenced this issue Jun 21, 2023
@NathanGibbs3
20230621 Closes #212 Closes #215 Work om #103 & #214
b1f0544 
         Code Cleanup.

     File(s): base_conf.php.dist
            : base_maintenance.php
            : includes/base_action.inc.php
            : setup/base_conf_contents.php
    Issue(s): #103 #212 #214 #215
              Code Cleanup.
     File(s): includes/base_auth.inc.php
              Code Cleanup.
     File(s): includes/base_db.inc.php
    Issue(s): #103 #212 #214 #215
              Added DB connection awarenes to multiple functions.
              This allows us to log transient DB connection errors.
              Code Cleanup.
 Function(s): baseisDBUp( LogError )
              Returns true if DB connection is up, false otherwise.
              Optionally logs the connection error.
            : baseSetRI( State )
              DB Referential Integrity (RI) Control. Enables / Disables
              RI on DB server software that supports it. Adding /
              removing the necessary RI structure to the DB. Respects
              the $use_referential_integrity conf var.
            : baseGetRI()
              Returns the RI status of the connected DB.
            : baseTSE( table )
              Returns the MySQL/MariaDB storage engine of the
              requested table, empty string otherwise.
     File(s): includes/base_krnl.php
              Bumped Kernel Version to 0.0.7
              Code Cleanup.
 Function(s): function VS2SV( VS )
              Returns Semantic Version Array determined from
              supplied Version string, false if unsucessful.
     File(s): includes/base_rtl.php
              Bumped Kernel Version to 0.0.11
              Code Cleanup.
 Function(s): function VS2SV( VS )
              Returns Semantic Version Array determined from
              supplied Version string, false if unsucessful.
     File(s): sql/create_snort_tbls_*.sql
              Added DB creation scripts from the SNORT project.
              Schema 107
              Now it is possible to manually initialize the entire DB
              from a BASE snapshot.
     File(s): sql/disable_RI*.sql
              Scripts to manually disable RI in the DB.
Unit Test(s): Covers the following in the baseCon Class baseisDBUp()
              baseSetRI() baseGetRI() baseTSE()
@mesteele
Copy link

I need some clarification on this added and updated feature: On an existing or new DB if I set BASE to $use_referential_integrity = 1 will BASE add the RI tables to the DB?

@NathanGibbs3
Copy link
Owner Author

That's the idea, if the DB server can support it, it enables RI when set.
It's also supposed to follow the $use_referential_integrity setting and remove the RI constraints from the DB when set to zero.

However, I WOULD NOT deploy what is currently in devel into the wild yet.
What is currently in devel is a steaming pile of 💩 that only works as intended on MariaDB.

Thank God I built Unit Tests around this, when it hit travis-CI, it went off the rails and is a train wreck.

We are currently having issues with the RI structure verification code on PostgreSQL and RI structure teardown on MySQL which does things just a "little bit" differently than MariaDB.

The PostgreSQL issue is the one I'm working on right now, because on PostgreSQL, enabling RI will crash BASE with this feature. 💩 🔥 🧯 🚒 😄

@mesteele
Copy link

mesteele commented Jun 23, 2023 •
edited
Loading

I'm confused, but that doesn't take much :) Would you feel comfortable putting out the one listed in the "Master" release to the wild or should the last version released by secureideas be used?

I'm assuming that the Master is the last PHP7 compatible version?

NathanGibbs3 added a commit that referenced this issue Jun 24, 2023
@NathanGibbs3
20230623 Better Closes for #212 & #215
8b638e3 
         Code Cleanup.

     File(s): includes/base_db.inc.php
 Function(s): baseFKeyExists( $key )
              Returns true if RI is enabled and the RI Foreign Key
              exists in the DB structure, false otherwise.
Unit Test(s): Covers the following in the baseCon Class baseFKeyExists()
@NathanGibbs3
Copy link
Owner Author

I'm confused, but that doesn't take much :) Would you feel comfortable putting out the one listed in the "Master" release to the wild or should the last version released by secureideas be used?

The code in master is OK.

The code in devel is all the work around getting RI working on MySQL/ MariaDB, and making sure that even if the end user sets $use_referential_integrity to 1 on a DB setup that can't support it, the chance of DB corruption is minimized.

I'm assuming that the Master is the last PHP7 compatible version?

There are no plans to drop PHP 7x compatibility.
That is part of why I build Unit Tests around what I am working on, to ensure where possible that BASE works on older PHP's as well.

@NathanGibbs3
Copy link
Owner Author

Good news, the last commit to devel passed CI checks.
Use it in the wild if you want.

I'll go over the RI stuff again, sorry about the confusion earlier.

Classic BASE 'secureideas' behavior.

  • $use_referential_integrity = 0
    • BASE deletes alerts from event table.
    • BASE deletes alerts from other tables too.
  • $use_referential_integrity = 1
    • BASE deletes alerts from event table.
    • THE DB server deletes alerts from other tables, if RI has been setup and the DB server even supports RI.
    • BASE assumes that the DB server took care of it.

New BASE behavior.

  • $use_referential_integrity = 0 ( Same as classic at the moment. )
    • BASE deletes alerts from event table.
    • BASE deletes alerts from other tables too.
  • $use_referential_integrity = 1
    • BASE deletes alerts from event table.
    • THE DB server deletes alerts from other tables, if RI has been setup.
    • When BASE connects to the DB it does the following checks.
      • Server Check: Can DB Server ( Version / Features ) support RI?
        • If Yes, DB Check: ( Is RI already setup in the DB? Is RI structure valid? )
          • If No,
            • Remove parts of invalid RI structure if they exists.
            • Create new RI structure.
          • If Yes, Carry on, nothing more to do concerning RI.
        • If No, DB Server cannot support RI, act like $use_referential_integrity = 0

Hope that clarifies things for you. 😄
BTW, great feature request. 🎉

@mesteele
Copy link

mesteele commented Jun 24, 2023 •
edited
Loading

There are no plans to drop PHP 7x compatibility.

Sorry, the Master has no PHP 8 code included, or does it?

Is the Master what we call stable?

@NathanGibbs3
Copy link
Owner Author

There are no plans to drop PHP 7x compatibility.

Sorry, the Master has no PHP 8 code included, or does it?

As far as I know master will work with PHP 5x or earlier through PHP 8x.
For instance, the bugs that you found relating to issue #149 have been fixed and those fixes merged into master and tested successfully in travis-CI on PHP 5.2x - 8.1x.

When a PHP forward compatibility bug, ( PHP 7x or 8x breakage ), gets fixed, I try to find a way of doing it that does not break backwards compatibility ( PHP 5x or earlier ).

Is the Master what we call stable?

It is more stable than devel, although at times mistakes get merged into master, eg. #194

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@NathanGibbs3 NathanGibbs3

Labels
bug Something isn't working DB Database Issues invalid This doesn't seem right LCB-TechDebt Issue exists in Legacy Code Base. We inherited it. Prod Observed in Production Environment. Stability Issue impacts or is related to App Stability. UI User Interface Issues.
Projects
Code Quality - Implementation Standar...
  
Needs triage
Milestone
1.4.6
Development

No branches or pull requests
2 participants
@NathanGibbs3 @mesteele