ResourceWarnings in py3 during test #86

KSchopmeyer · 2016-03-12T23:39:16Z

While probably not critical, there is no way I could see to disconnect a connection when finished.

Andy: Generalized the issue from ResourceWarnings about open sockets / WBEMConnection has no close() to cover all ResourceWarnings during test.

andy-maier · 2016-03-13T03:07:48Z

Because the connection is only conceptual, it does not need to be closed. The WBEMConnection object will be garbage collected once unused, but that does not matter for that.

KSchopmeyer · 2016-03-14T15:27:04Z

Note that with python 3, the run_cim_operations.py test repeatedly reports:

timeout=self.timeout)
/home/kschopmeyer/pywbem/githubpywbem/pywbem/pywbem/cim_operations.py:596: ResourceWarning: unclosed <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.3.138', 59717), raddr=('192.168.3.138', 5988)>
timeout=self.timeout)

Which is where I figured out that there was no timeout.

KSchopmeyer · 2016-03-14T15:59:34Z

In retrospect this may be a more serious problem than I originally thought but not sure yet.

Note that we are running tests with timeout = 0 at this point

andy-maier · 2016-03-14T18:41:59Z

Ok. The discussion in this issue of the requests package: https://github.com/kennethreitz/requests/issues/1882 seems to say that the warning is not an issue, but it remains unclear why it is being raised.

I don't think we want to explicitly destroy the socket after each CIM operation; that would be an efficiency problem.

Maybe you are right and an explicit close() on the WBEMConnection object would be a solution to avoid the warning. But we need to investigate what we would do in such a close() method.

KSchopmeyer · 2016-03-15T01:43:57Z

Most other platforms have a close and that seems logical. I agree. This is really a 0.9.0 issue and should not be 0.8.0

andy-maier · 2016-03-15T07:01:55Z

Moving to 0.9.0.

andy-maier · 2016-09-06T18:49:00Z

Discussion point: The requests package is able to maintain a socket pool and handle socket reuse automatically. Because we want to use that package, I don't think that we need a close() method on WBEMConnection.

KSchopmeyer · 2016-09-07T14:43:43Z

What we do here depends on the package we use in the future. If we go to requests, Andy's point above is correct. Left open until we pick next ssl/ http package to work with

andy-maier · 2017-09-28T08:37:28Z

This issue waits for the decision to move or not to move to using the requests package (see issue #280).

andy-maier · 2020-02-01T16:19:22Z

This issue needs to be re-evaluated with the current master version (which is based on the requests package).

andy-maier · 2020-07-17T07:26:00Z

The current master branch (close to 1.0.0 final) shows the following ResourceWarnings during test, on Python 3.8 with latest package levels.

They are all about unclosed files; the earlier open sockets warnings are gone since we moved to use 'requests'.

$ py.test --color=yes --cov pywbem --cov pywbem_mock  --cov-config coveragerc -W default -W ignore::PendingDeprecationWarning  tests/unittest tests/functiontest -s

. . . (reduced to just ResourceWarning):

tests/unittest/pywbem/test_cim_obj.py: 1431 tests with warnings
  /Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/utils/validate.py:186:
   ResourceWarning: unclosed file <_io.BufferedReader name=6>
    validate_cim_xml(obj_xml_str, root_elem_name)

tests/unittest/pywbem/test_cim_xml.py: 183 tests with warnings
  /Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_cim_xml.py:4261:
   ResourceWarning: unclosed file <_io.BufferedReader name=6>
    validate_cim_xml(act_xml_str)

tests/unittest/pywbem/test_logging.py: 12 tests with warnings
  /Users/maiera/PycharmProjects/pywbem/pywbem/pywbem/_cim_operations.py:1442:
   ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log' mode='a' encoding='UTF-8'>
    cls._activate_logger(logger_name, simple_name, detail_level, handler,

tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file-blah-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-api]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file-blah-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-http]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file-blah-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-all]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file--9-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-api]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file--9-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-http]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file--9-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-all]
  /Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.py:151:
   ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log' mode='a' encoding='UTF-8'>
    configure_logger(log_name, log_dest=log_dest,

tests/unittest/pywbem/test_mof_compiler.py::TestRemove::test_remove
  /Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_mof_compiler.py:331:
   ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/moflog.txt' mode='w' encoding='UTF-8'>
    self.logfile = open(moflog_file, 'w')

tests/unittest/pywbem/test_mof_compiler.py::TestSchemaSearch::test_search_None
  /Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_mof_compiler.py:655:
   ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/moflog.txt' mode='w' encoding='UTF-8'>
    open(moflog_file, 'w')

tests/unittest/pywbem/test_mof_compiler.py::TestSchemaSearch::test_search_single_string
  /Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_mof_compiler.py:627:
   ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/moflog.txt' mode='w' encoding='UTF-8'>
    open(moflog_file, 'w')

tests/unittest/pywbem/test_mof_compiler.py::TestPartialSchema::test_build_from_schema_string
  /Users/maiera/virtualenvs/pywbem38/lib/python3.8/site-packages/ply/lex.py:325:
   ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/moflog2.txt' mode='w' encoding='UTF-8'>
    tok = LexToken()

andy-maier · 2020-10-10T05:20:24Z

PR #2488 fixed a ResourceWarning for not closing a MOF log file.

With the current master, we still have the following ResourceWarnings:

$ py.test --color=yes --cov pywbem --cov pywbem_mock  --cov-config coveragerc -W default -W ignore::PendingDeprecationWarning tests/unittest tests/functiontest -s

tests/unittest/pywbem/test_logging.py: 12 tests with warnings
  /Users/maiera/PycharmProjects/pywbem/pywbem/pywbem/_cim_operations.py:1431: ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log' mode='a' encoding='UTF-8'>
    cls._activate_logger(logger_name, simple_name, detail_level, handler,

tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file-blah-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-api]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file-blah-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-http]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file-blah-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-all]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file--9-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-api]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file--9-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-http]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file--9-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-all]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file-detail_level14-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-api]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file-detail_level14-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-http]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file-detail_level14-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-all]
  /Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.py:154: ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log' mode='a' encoding='UTF-8'>
    configure_logger(log_name, log_dest=log_dest,

tests/unittest/pywbem/test_mof_compiler.py::TestRemove::test_remove
  /Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_mof_compiler.py:351: ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/moflog.txt' mode='w' encoding='UTF-8'>
    self.logfile = open(moflog_file, 'w')

tests/unittest/pywbem/test_mof_compiler.py::TestSchemaSearch::test_search_None
  /Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_mof_compiler.py:675: ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/moflog.txt' mode='w' encoding='UTF-8'>
    open(moflog_file, 'w')

tests/unittest/pywbem/test_mof_compiler.py::TestSchemaSearch::test_search_single_string
  /Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_mof_compiler.py:647: ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/moflog.txt' mode='w' encoding='UTF-8'>
    open(moflog_file, 'w')

andy-maier · 2020-10-10T10:52:04Z

PR #2496 fixes the ResourceWarning for the test_mof_compiler.py cases.

That leaves the following cases, all related to logging:

$ py.test --color=yes --cov pywbem --cov pywbem_mock  --cov-config coveragerc -W default -W ignore::PendingDeprecationWarning tests/unittest tests/functiontest -s

tests/unittest/pywbem/test_logging.py: 12 tests with warnings
  /Users/maiera/PycharmProjects/pywbem/pywbem/pywbem/_cim_operations.py:1431: ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log' mode='a' encoding='UTF-8'>
    cls._activate_logger(logger_name, simple_name, detail_level, handler,

tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file-blah-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-api]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file-blah-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-http]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file-blah-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-all]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file--9-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-api]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file--9-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-http]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file--9-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-all]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file-detail_level14-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-api]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file-detail_level14-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-http]
tests/unittest/pywbem/test_logging.py::TestLoggingConfigure::test_configure_logger[file-detail_level14-/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log-ValueError-all]
  /Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.py:154: ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/maiera/PycharmProjects/pywbem/pywbem/tests/unittest/pywbem/test_logging.log' mode='a' encoding='UTF-8'>
    configure_logger(log_name, log_dest=log_dest,

Fixing the cases related to logging requires some thought:

The logger object is global, but the control about logging can be global and specific to each connection. So if we stay with this approach, it requires an awareness of that.
Should we stay with one global logger or change to add logger objects per connection?

andy-maier added type: enhancement resolution: no issue labels Mar 13, 2016

andy-maier added this to the 0.8.0 milestone Mar 13, 2016

andy-maier self-assigned this Mar 13, 2016

andy-maier removed the resolution: no issue label Mar 14, 2016

andy-maier modified the milestones: 0.9.0, 0.8.0 Mar 15, 2016

andy-maier added the discussion needed label Mar 16, 2016

andy-maier removed their assignment Mar 21, 2016

KSchopmeyer self-assigned this Mar 24, 2016

KSchopmeyer removed this from the 0.9.0 milestone Apr 6, 2016

andy-maier removed the discussion needed label Apr 6, 2016

andy-maier added the discussion needed label Sep 6, 2016

andy-maier added this to the 0.10.0 milestone Sep 6, 2016

andy-maier removed the discussion needed label Sep 7, 2016

KSchopmeyer removed this from the 0.10.0 milestone Sep 7, 2016

andy-maier added area: code waiting labels Sep 28, 2017

andy-maier added this to the 0.13.0 milestone Apr 10, 2018

andy-maier mentioned this issue Apr 11, 2018

Directions and Priorities for 0.13.0 #1170

Closed

KSchopmeyer added the investigation needed label Aug 31, 2018

andy-maier added release: mandatory and removed waiting labels Oct 10, 2018

andy-maier unassigned KSchopmeyer Oct 10, 2018

andy-maier mentioned this issue Oct 10, 2018

Directions and Priorities for pywbem 1.0.0 #1412

Closed

andy-maier mentioned this issue Dec 20, 2019

Fixes #280: Migrated HTTP communication to use requests #2023

Merged

andy-maier added the investigation needed label Feb 3, 2020

andy-maier assigned KSchopmeyer Feb 3, 2020

andy-maier removed the investigation needed label Mar 25, 2020

andy-maier modified the milestones: 1.0.0, 1.1.0 Mar 25, 2020

andy-maier mentioned this issue Jul 17, 2020

Directions and Priorities for pywbem 1.1.0 #2352

Closed

andy-maier changed the title ~~ResourceWarning in py3 about open sockets / WBEMConnection has no close()~~ ResourceWarnings in py3 Jul 17, 2020

andy-maier changed the title ~~ResourceWarnings in py3~~ ResourceWarnings in py3 during test Jul 17, 2020

andy-maier removed this from the 1.1.0 milestone Aug 19, 2020

andy-maier mentioned this issue Oct 3, 2020

Directions and Priorities for pywbem 1.2.0 #2478

Closed

andy-maier added this to the 1.2.0 milestone Oct 7, 2020

andy-maier removed the release: mandatory label Oct 7, 2020

andy-maier assigned andy-maier and unassigned KSchopmeyer Oct 10, 2020

andy-maier added the under work label Oct 10, 2020

andy-maier linked a pull request Oct 10, 2020 that will close this issue

Added ability to disable logging; Fixed ResourceWarnings in test_logging.py #2497

Merged

andy-maier added resolution: fixed and removed under work labels Oct 10, 2020

This was linked to pull requests Oct 10, 2020

Test: Fixed ResourceWarning for not closing MOF compiler log file #2488

Merged

Partly fixes #86: Test: Fixed ResourceWarning issued in test_mof_compiler.py #2496

Merged

andy-maier closed this as completed in #2496 Oct 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ResourceWarnings in py3 during test #86

ResourceWarnings in py3 during test #86

KSchopmeyer commented Mar 12, 2016 •

edited by andy-maier

andy-maier commented Mar 13, 2016

KSchopmeyer commented Mar 14, 2016

KSchopmeyer commented Mar 14, 2016

andy-maier commented Mar 14, 2016

KSchopmeyer commented Mar 15, 2016

andy-maier commented Mar 15, 2016

andy-maier commented Sep 6, 2016

KSchopmeyer commented Sep 7, 2016

andy-maier commented Sep 28, 2017

andy-maier commented Feb 1, 2020

andy-maier commented Jul 17, 2020 •

edited

andy-maier commented Oct 10, 2020

andy-maier commented Oct 10, 2020 •

edited

ResourceWarnings in py3 during test #86

ResourceWarnings in py3 during test #86

Comments

KSchopmeyer commented Mar 12, 2016 • edited by andy-maier

andy-maier commented Mar 13, 2016

KSchopmeyer commented Mar 14, 2016

KSchopmeyer commented Mar 14, 2016

andy-maier commented Mar 14, 2016

KSchopmeyer commented Mar 15, 2016

andy-maier commented Mar 15, 2016

andy-maier commented Sep 6, 2016

KSchopmeyer commented Sep 7, 2016

andy-maier commented Sep 28, 2017

andy-maier commented Feb 1, 2020

andy-maier commented Jul 17, 2020 • edited

andy-maier commented Oct 10, 2020

andy-maier commented Oct 10, 2020 • edited

KSchopmeyer commented Mar 12, 2016 •

edited by andy-maier

andy-maier commented Jul 17, 2020 •

edited

andy-maier commented Oct 10, 2020 •

edited