Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bareos Python Plugins: fix handling of extra pluginoptions #1177

Merged
merged 7 commits into from
May 18, 2022

Conversation

joergsteffens
Copy link
Member

@joergsteffens joergsteffens commented May 6, 2022

This PR fixes a problem with handling additional pluginoptions, provided during a restore. When these are set, the plugin "forgets" about the Python module path.

Thank you for contributing to the Bareos Project!

Please check

  • Short description and the purpose of this PR is present above this paragraph
  • Your name is present in the AUTHORS file (optional)

If you have any questions or problems, please give a comment in the PR.

Helpful documentation and best practices

Checklist for the reviewer of the PR (will be processed by the Bareos team)

General
  • PR name is meaningful
  • Purpose of the PR is understood
  • Separate commit for this PR in the CHANGELOG.md, PR number referenced is same
  • Commit descriptions are understandable and well formatted
  • If backport: add original PR number and target branch at top of this file: Backport of PR#000 to bareos-2x
Source code quality
  • Source code changes are understandable
  • Variable and function names are meaningful
  • Code comments are correct (logically and spelling)
  • Required documentation changes are present and part of the PR
  • bareos-check-sources --since-merge does not report any problems
  • git status should not report modifications in the source tree after building and testing
Tests
  • Decision taken that a test is required (if not, then remove this paragraph)
  • The choice of the type of test (unit test or systemtest) is reasonable
  • Testname matches exactly what is being tested
  • On a fail, output of the test leads quickly to the origin of the fault

@joergsteffens joergsteffens force-pushed the dev/joergs/master/pluginoptions branch 2 times, most recently from 98a87ba to ddec5cf Compare May 9, 2022 14:22
@joergsteffens joergsteffens requested a review from pstorz May 9, 2022 14:26
@joergsteffens joergsteffens marked this pull request as ready for review May 9, 2022 14:26
@joergsteffens joergsteffens force-pushed the dev/joergs/master/pluginoptions branch from ddec5cf to 015a4b1 Compare May 9, 2022 14:35
Copy link
Member

@pstorz pstorz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have the following change requests:

  • The problem that is fixed with this PR is not described
  • Variable name enhancements
  • changelog.md entry is missing
  • bareos-check-sources is not happy about systemtests/python-modules/bareos_unittest/json.py, needs to be formatted
  • system:python-bareos:python_fd_plugins does not pass:
361: === python-bareos/test_python_fd_plugins: starting at 12:05:14 ===
361: =
361: =
361: /home/pstorz/git/PR-REVIEW/python-bareos/bareos/bsock/lowlevel.py:57: UserWarning: Connection encryption via TLS-PSK is not available, as the module sslpsk is not installed.
361:   warnings.warn(
361: test_additional_pluginoptions (test_python_fd_plugins.BareosFdPythonPluginOptions) ... FAIL
361: test_overwrite_pluginoptions (test_python_fd_plugins.BareosFdPythonPluginOptions)
361: Overwrites a plugin option ... expected failure
361: test_without_extra_pluginoptions (test_python_fd_plugins.BareosFdPythonPluginOptions) ... FAIL
361: 
361: ======================================================================
361: FAIL: test_additional_pluginoptions (test_python_fd_plugins.BareosFdPythonPluginOptions)
361: ----------------------------------------------------------------------
361: Traceback (most recent call last):
361:   File "/home/pstorz/git/PR-REVIEW/b/systemtests/tests/python-bareos/test_python_fd_plugins.py", line 89, in test_additional_pluginoptions
361:     jobid = self.run_job(
361:   File "/home/pstorz/git/PR-REVIEW/systemtests/python-modules/bareos_unittest/json.py", line 103, in run_job
361:     self.wait_job(director, jobId)
361:   File "/home/pstorz/git/PR-REVIEW/systemtests/python-modules/bareos_unittest/json.py", line 91, in wait_job
361:     self.assertEqual(result["job"]["jobstatuslong"], expected_status)
361: AssertionError: 'Fatal Error' != 'OK'
361: - Fatal Error
361: + OK
361: 
361: 
361: ======================================================================
361: FAIL: test_without_extra_pluginoptions (test_python_fd_plugins.BareosFdPythonPluginOptions)
361: ----------------------------------------------------------------------
361: Traceback (most recent call last):
361:   File "/home/pstorz/git/PR-REVIEW/b/systemtests/tests/python-bareos/test_python_fd_plugins.py", line 55, in test_without_extra_pluginoptions
361:     jobid = self.run_job(director, jobname, level="Full", wait=True)
361:   File "/home/pstorz/git/PR-REVIEW/systemtests/python-modules/bareos_unittest/json.py", line 103, in run_job
361:     self.wait_job(director, jobId)
361:   File "/home/pstorz/git/PR-REVIEW/systemtests/python-modules/bareos_unittest/json.py", line 91, in wait_job
361:     self.assertEqual(result["job"]["jobstatuslong"], expected_status)
361: AssertionError: 'Fatal Error' != 'OK'
361: - Fatal Error
361: + OK
361: 
361: 
361: ----------------------------------------------------------------------
361: Ran 3 tests in 6.685s
361: 
361: FAILED (failures=2, expected failures=1)
361: ERROR: 
361: ERROR: 
361:  
361:   !!!!! python-bareos/test_python_fd_plugins failed!!! 12:05:21 !!!!! 
361:    Status: estat=9 zombie=0 backup=0 restore=0 diff=0
361:  
2/3 Test #361: system:python-bareos:python_fd_plugins ...***Failed    6.94 sec

core/src/plugins/dird/python/plugin_private_context.h Outdated Show resolved Hide resolved
core/src/plugins/dird/python/plugin_private_context.h Outdated Show resolved Hide resolved
core/src/plugins/filed/python/plugin_private_context.h Outdated Show resolved Hide resolved
core/src/plugins/filed/python/plugin_private_context.h Outdated Show resolved Hide resolved
@joergsteffens joergsteffens force-pushed the dev/joergs/master/pluginoptions branch from 05842a0 to 870ad84 Compare May 16, 2022 15:21
@joergsteffens joergsteffens requested a review from pstorz May 16, 2022 17:37
Copy link
Member

@pstorz pstorz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have the following remarks:

  • The Pluginoptions are in one line in the docs, while the multiline is much better readable.
  • Last build failed, now waiting for next build.

":vcuser=bakadm@vsphere.local"
":vcpass=Bak.Adm-1234"
":localvmdk=yes"
"python:module_name=bareos-fd-vmware:dc=dass5:folder=/:vmname=stephand-test02:vcserver=virtualcenter5.dass-it:vcuser=bakadm@vsphere.local:vcpass=Bak.Adm-1234:localvmdk=yes"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we want to break the plugin options at the ":" as before? That makes things much better readable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a string that has to be typed during the restore command. I assume, splitting it over multiple lines is not an option, especially not in Bareos <= 15.2.3, as stated above.
But now that you point to it, I think, also the quoted around this string have to be removed.

The test was marked broken when the plugin was using psycopg2, but now
we use pg8000 so we want to enable it again if all requirements are met.

Also we allow python >= 3.10 to be used as we now avoid the problems
arising from the fact that the stat time filds are floats instead of
ints:

> mystatp.st_atime = statp.st_atime
> TypeError: 'float' object cannot be interpreted as an integer
@pstorz pstorz merged commit db1af02 into bareos:master May 18, 2022
@joergsteffens joergsteffens deleted the dev/joergs/master/pluginoptions branch June 22, 2022 14:47
bruno-at-bareos pushed a commit to tuxmaster5000/bareos that referenced this pull request Jul 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants