Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests with errors #328

Open
Nilsonfsilva opened this issue Jan 24, 2023 · 11 comments
Open

tests with errors #328

Nilsonfsilva opened this issue Jan 24, 2023 · 11 comments
Assignees
Labels

Comments

@Nilsonfsilva
Copy link

Hey !
First congratulations on the project.

I would like to inform you that we are packaging your project to be part of our repositories.

In the construction of the package, some tests presented errors. it would be possible to help us to discover the reason.

Here are the errors:

FAIL: test_is_valid (checker.syntax.test_ip.TestIPSyntaxChecker.test_is_valid)
Tests the method which let us check if the given subject is valid

Traceback (most recent call last):
File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ip.py", line 267, in test_is_valid
self.assertEqual(expected, actual, subject)
AssertionError: True != False : 255.45.65.0/24

======================================================================
FAIL: test_is_valid_range_v4 (checker.syntax.test_ip.TestIPSyntaxChecker.test_is_valid_range_v4)
Tests the method which let us check if the given subject is valid range.

Traceback (most recent call last):
File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ip.py", line 142, in test_is_valid_range_v4
self.assertEqual(expected, actual, subject)
AssertionError: True != False : 255.45.65.0/24

======================================================================
FAIL: test_is_valid_range_v6 (checker.syntax.test_ip.TestIPSyntaxChecker.test_is_valid_range_v6)
Tests the method which let us check if the given subject is valid range.

Traceback (most recent call last):
File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ip.py", line 157, in test_is_valid_range_v6
self.assertEqual(expected, actual, subject)
AssertionError: True != False : 2001:db8::/128

======================================================================
FAIL: test_is_valid_v4 (checker.syntax.test_ip.TestIPSyntaxChecker.test_is_valid_v4)
Tests the method which let us check if the given subject is valid.

Traceback (most recent call last):
File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ip.py", line 80, in test_is_valid_v4
self.assertEqual(expected, actual, subject)
AssertionError: True != False : 255.45.65.0/24

======================================================================
FAIL: test_is_valid_v6 (checker.syntax.test_ip.TestIPSyntaxChecker.test_is_valid_v6)
Tests the method which let us check if the given subject is valid.

Traceback (most recent call last):
File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ip.py", line 95, in test_is_valid_v6
self.assertEqual(expected, actual, subject)
AssertionError: True != False : 2001:db8:1234::/48

======================================================================
FAIL: test_is_valid (checker.syntax.test_ipv4.TestIPv4SyntaxChecker.test_is_valid)
Tests the method which let us check if the given subject is valid.

Traceback (most recent call last):
File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ipv4.py", line 80, in test_is_valid
self.assertEqual(expected, actual, subject)
AssertionError: True != False : 255.45.65.0/24

======================================================================
FAIL: test_is_valid_range (checker.syntax.test_ipv4.TestIPv4SyntaxChecker.test_is_valid_range)
Tests the method which let us check if the given subject is valid range.

Traceback (most recent call last):
File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ipv4.py", line 110, in test_is_valid_range
self.assertEqual(expected, actual, subject)
AssertionError: True != False : 255.45.65.0/24

======================================================================
FAIL: test_is_valid (checker.syntax.test_ipv6.TestIPv6SyntaxChecker.test_is_valid)
Tests the method which let us check if the given subject is valid.

Traceback (most recent call last):
File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ipv6.py", line 80, in test_is_valid
self.assertEqual(expected, actual, subject)
AssertionError: True != False : 2001:db8:1234::/48

======================================================================
FAIL: test_is_valid_range (checker.syntax.test_ipv6.TestIPv6SyntaxChecker.test_is_valid_range)
Tests the method which let us check if the given subject is valid range.

Traceback (most recent call last):
File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/checker/syntax/test_ipv6.py", line 111, in test_is_valid_range
self.assertEqual(expected, actual, subject)
AssertionError: True != False : 2001:db8::/128

======================================================================
FAIL: test_get_converted (converter.test_cidr2subject.TestCIDR2Subject.test_get_converted)
Tests the method which let us get the converted data.

Traceback (most recent call last):
File "/pkgs/trabalhando/PACOTES-PRONTOS/pkg-pyfunceble/pyfunceble-4.1.3/tests/converter/test_cidr2subject.py", line 113, in test_get_converted
self.assertTrue(Counter(expected) == Counter(actual))
AssertionError: False is not true

@funilrys
Copy link
Owner

Hi @Nilsonfsilva,

thank you! Let me know when the packaging is done so that I can link it to the documentation or the landing page!

I can't reproduce this error on my machine + 2 VMs. May I ask you which version of Python has been used? That will help a lot!

I wish you all the best in this New Year!

@Nilsonfsilva
Copy link
Author

Nilsonfsilva commented Jan 28, 2023

Hi!

I wish you all the best in this New Year!
Thank you for the congratulations and I wish you the same.

I ask for a little patience in advance. Because according to the packaging policy, debian cannot use any external repositories. To package pyfunceble it is necessary that all dependencies are already in the distribution's repositories.
I suspect these errors are due to missing some dependency:
answering your question about the python version is Python 3.9.13
pyfunceble is complaining about this dependency:
pyf_test_dataset
E ModuleNotFoundError: No module named 'pyf_test_dataset'

  • Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
    =========================== short test summary info ============================
    ERROR tests/checker/reputation/test_base.py
    ERROR tests/checker/syntax/test_domain.py
    ERROR tests/checker/syntax/test_ip.py
    ERROR tests/checker/syntax/test_ipv4.py
    ERROR tests/checker/syntax/test_ipv6.py
    ERROR tests/checker/syntax/test_second_lvl_domain.py
    ERROR tests/checker/syntax/test_subdomain.py
    ERROR tests/checker/syntax/test_url.py
    ERROR tests/query/dnss/test_nameserver.py
    !!!!!!!!!!!!!!!!!!! Interrupted: 9 errors during collection !!!!!!!!!!!!!!!!!!!!
    ======================== 2 warnings, 9 errors in 3.98s =========================

tests/checker/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/availability/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/availability/test_domain.py
ERROR tests/checker/availability/test_ip.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/availability/test_url.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/reputation/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/reputation/test_domain.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/reputation/test_domain_and_ip.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/reputation/test_ip.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/reputation/test_url.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/syntax/test_domain.py
ERROR tests/checker/syntax/test_ip.py
ERROR tests/checker/syntax/test_ipv4.py
ERROR tests/checker/syntax/test_ipv6.py
ERROR tests/checker/syntax/test_second_lvl_domain.py
ERROR tests/checker/syntax/test_subdomain.py
ERROR tests/checker/syntax/test_url.py
ERROR tests/checker/utils/test_whois.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/config/test_loader.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/converter/test_internal_url.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/dataset/test_ipv4_reputation.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/helpers/test_download.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/query/test_http_status_code.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/query/dnss/test_nameserver.py
ERROR tests/query/dnss/test_resolver.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/query/netinfo/test_address.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/query/whois/test_query_tool.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/query/whois/converter/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers

all these files have errors due to this absence
Can you measure which repository this module is in?
Thanks!

@funilrys
Copy link
Owner

Oh, you discovered a problem in the way I process tests ...

I'm testing with:

 python -m unittest discover -s tests

If I run with pytest I can reproduce an error:

python -m pytests tests
==================================================================== short test summary info ====================================================================
ERROR tests/checker/reputation/test_base.py
ERROR tests/checker/syntax/test_domain.py
ERROR tests/checker/syntax/test_ip.py
ERROR tests/checker/syntax/test_ipv4.py
ERROR tests/checker/syntax/test_ipv6.py
ERROR tests/checker/syntax/test_second_lvl_domain.py
ERROR tests/checker/syntax/test_subdomain.py
ERROR tests/checker/syntax/test_url.py
ERROR tests/query/dnss/test_nameserver.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 9 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
======================================================================= 9 errors in 0.77s =======================================================================

Let me try to fix those issue ASAP.

Used version: Python 3.10.9

funilrys added a commit that referenced this issue Jan 28, 2023
This patch fixes #328.

Indeed, before this patch, as I'm not using pytest in this
project, an undiscovered issue was sitting there since the
begining of the project.
This patch fixes the issue by trying to correct the import path.

Contributors:
  * @Nilsonfsilva
@funilrys
Copy link
Owner

@Nilsonfsilva can you test against the origin/issue-328 branch ?

I know that you are trying to package the stable version, but if that dev-based branch deliver the certainty that the issue is fixed, I will release it into the dev and later the stable version of PyFunceble.

@Nilsonfsilva
Copy link
Author

Hi, thanks in advance for your attention. It will be a great pleasure to bring your project to Debian.

take it easy, try to fix the bugs, and when you're ready and want to release a new version, we appreciate it. Since the debian robot will fetch the latest tag.

If you want to send me before, the .tar.gz. with corrections to check, feel free. Thank you and I'm at your disposal! a hug!

@Nilsonfsilva
Copy link
Author

@Nilsonfsilvavocê pode testar na origin/issue-328 ? ramificação

Eu sei que você está tentando empacotar o stableversão, mas se isso devramo baseado em entregar a certeza de que o problema está resolvido, vou liberá-lo para o deve mais tarde o stableversão do PyFunceble.

I believe we have evolved. But there are still other mistakes.
I'm simulating in two different environments:
Work environment, presents errors in 10 functions:
As for that, it's quiet. Because I can escape these functions through patch

=========================== short test summary info ============================
FAILED tests/checker/syntax/test_ip.py::TestIPSyntaxChecker::test_is_valid - ...
FAILED tests/checker/syntax/test_ip.py::TestIPSyntaxChecker::test_is_valid_range_v4
FAILED tests/checker/syntax/test_ip.py::TestIPSyntaxChecker::test_is_valid_range_v6
FAILED tests/checker/syntax/test_ip.py::TestIPSyntaxChecker::test_is_valid_v4
FAILED tests/checker/syntax/test_ip.py::TestIPSyntaxChecker::test_is_valid_v6
FAILED tests/checker/syntax/test_ipv4.py::TestIPv4SyntaxChecker::test_is_valid
FAILED tests/checker/syntax/test_ipv4.py::TestIPv4SyntaxChecker::test_is_valid_range
FAILED tests/checker/syntax/test_ipv6.py::TestIPv6SyntaxChecker::test_is_valid
FAILED tests/checker/syntax/test_ipv6.py::TestIPv6SyntaxChecker::test_is_valid_range
FAILED tests/converter/test_cidr2subject.py::TestCIDR2Subject::test_get_converted

In the second environment, I simulate as if pyfunceble were inside the debian repositories, being built. That is, isolated without connections to external repositories. Because, for security reasons, deboian does not allow any packages to be communicated with anything external. Such errors occur:

=================== ERRORS ===========================================================================================
________________________________________________________________________ ERROR collecting tests/checker/test_base.py _________________________________________________________________________
tests/checker/test_base.py:55: in
from PyFunceble.checker.base import CheckerBase, CheckerStatusBase
PyFunceble/init.py:56: in
from PyFunceble.checker.availability.domain import DomainAvailabilityChecker
PyFunceble/checker/availability/domain.py:55: in
import PyFunceble.factory
PyFunceble/factory.py:55: in
Requester = requester()
PyFunceble/query/requests/requester.py:169: in init
self.dns_query_tool = DNSQueryTool()
PyFunceble/query/dns/query_tool.py:126: in init
self.nameservers.guess_and_set_nameservers()
PyFunceble/query/dns/nameserver.py:254: in guess_and_set_nameservers
self.set_nameservers(dns.resolver.get_default_resolver().nameservers)
/usr/lib/python3/dist-packages/dns/resolver.py:1166: in get_default_resolver
reset_default_resolver()
/usr/lib/python3/dist-packages/dns/resolver.py:1178: in reset_default_resolver
default_resolver = Resolver()
/usr/lib/python3/dist-packages/dns/resolver.py:758: in init
self.read_resolv_conf(filename)
/usr/lib/python3/dist-packages/dns/resolver.py:850: in read_resolv_conf
raise NoResolverConfiguration('no nameservers')
E dns.resolver.NoResolverConfiguration: no nameservers

ERROR tests/checker/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/availability/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/availability/test_domain.py
ERROR tests/checker/availability/test_ip.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/availability/test_url.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/reputation/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/reputation/test_domain.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/reputation/test_domain_and_ip.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/reputation/test_ip.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/reputation/test_url.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/checker/utils/test_whois.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/config/test_loader.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/converter/test_internal_url.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/dataset/test_ipv4_reputation.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/helpers/test_download.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/query/test_http_status_code.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/query/dnss/test_resolver.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/query/netinfo/test_address.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/query/whois/test_query_tool.py - dns.resolver.NoResolverConfiguration: no nameservers
ERROR tests/query/whois/converter/test_base.py - dns.resolver.NoResolverConfiguration: no nameservers

@funilrys
Copy link
Owner

That's an interesting output. I never thought it was possible to have a server without any DNS nameservers.

Let's me write the patch :-)

@funilrys
Copy link
Owner

One question for you: Are you planing to deploy both package or only pyfunceble ?

I deploy and maintain 2 packages:

  • pyfunceble (master branch)
    • -> This is one is a stable an slow package that only get a new version when pyfunceble-dev is stable enough (running several weeks on several environments with any huge bugs)
  • pyfunceble-dev (devbranch)
    • -> This is one is also stable package but it gets updated quicker. All new features and bug fixes are always available in this package first. This let us get a huge pool of feedback giver before the change are published in pyfunceble. Please also note that this package only gets updated when all tests (unit + integration + system) are passed.

funilrys added a commit that referenced this issue Apr 29, 2023
This patch touches #328.

Indeed, in some rare case, no system-wide resolver/nameservers
may not be configured; causing our tests to fail.
This patch fixes the issue by testing and handling the case that
no resolver is given.

From now on, if no nameserver is configured, the default one will
be set instead to:
  - 1.1.1.1
  - 8.8.8.8
  - 9.9.9.9

Contributors:
  * @Nilsonfsilva
@funilrys
Copy link
Owner

@Nilsonfsilva can you please test https://github.com/funilrys/PyFunceble/archive/refs/heads/issue-328.zip ?

Or is there a way I can see the automated tests?

I appreciate your patience.

funilrys added a commit that referenced this issue Apr 29, 2023
Indeed, since @Nilsonfsilva discovered a fatal error as described
in #328 and fixed by 3c3405f
he should be ackowledged as one of the contributor of the project.
@funilrys
Copy link
Owner

funilrys commented May 1, 2023

Reopening as other issues may come.

@funilrys funilrys reopened this May 1, 2023
@funilrys funilrys modified the milestone: __future__ May 1, 2023
@funilrys
Copy link
Owner

@Nilsonfsilva Can you please test again when you get some time?
Thank a lot!

funilrys added a commit that referenced this issue May 29, 2023
Fixed:
  * Security / Dependency Management: cryptography
    Mitigation of CVE-2023-0286 & CVE-2023-23931 through
    version bump.
  * Fatal Error: When no nameservers are configured or provided by the
    hosting system. (#328)
  * Semantic: git.io (#341)
    URLs with git.io were replaced with other one.
  * New linting issues.

Improved:
  * SPECIAL Rules: weebly.com
    We now take down subdomains that return the 406 status code.
  * SPECIAL Rules: wordpress.com (#321)
    We now recognize subjects that were took down by Wordpress.
  * SPECIAL Rules: internal
    Uniformiization of the method for better and quicker development.
  * Dependency Management: sqlalchemy
    We upgraded to SQLAlchemy v2.x+.
  * Converters: internal
    Convertion can now be performed directely without initialization of
    subjects through the convet method.

Removed:
  * Python Support: <=3.7
    We do not test or support any usage of PyFunceble with python<=3.7.

New:
  * Python Support: ~=3.11
    We now test (CI/CD) and support python~=3.11.
  * Testing: pytest (#328)
    pytest can now be used by packager to tests pyfunceble before
    deployment.
  * Database: PostgreSQL
    We now support PostgreSQL as database type.
  * Filesystem: IPs as first-class citizens in plain text outputs (#268)
    From now on, IPs will be stored into the `ips` subdirectory when the
    plain text format is active.
  * SPECIAL Rules: subject-switch (#185 | #185#issuecomment-1290866362)
    We now support the subject switch from any domains.
    For example:

    - m.example.com -> example.com
      Outcome: m.example.com as INACTIVE
    - m.example.com -> example.org
      Outcome: NO status switch.
    - m.example.com -> a.example.com -> example.com
      Outcome: m.example.com as INACTIVE.
    - m.example.com/hello/world -> example.com/hello/world
      Outcome: m.example.com/hello/world as INACTIVE.
  * SPECIAL Rules: changeip (#311)
    When one of the known changeip domains provides `abuse.change.com`
    in the SOA record, the subject will be flagged as INACTIVE.
  * SPECIAL Rules: imgur.com (#319)
    We now flag removed images.
  * SPECIAL Rules: eToxic (#334)
    When a blog from the eToxic infrastructure (known domains) doesn't
    exists anymore, we flag them as INACTIVE.

Contributors:
  * @Nilsonfsilva
  * @smed79
  * @spirilln
  * @T145
funilrys added a commit that referenced this issue Sep 8, 2023
Indeed, before this patch, we were installing serveral tools that are
only meant for developers. This patch fixes the issue by making them
available through some environment variables.

Therefore, from now on, the following console scripts will only be
available when the PYFUNCEBLE_INSTALL_DEVTOOLS or PYFUNCEBLE_DEVTOOLS
environment variables are found when installing:

  - production-pyfunceble

The following console scripts will from now on only be available when
then PYFUNCEBLE_INSTALL_HELPERS or PYFUNCEBLE_HELPTOOLS environment
variables are found when installing:

  - public-suffic-pyfunceble - the tool to regenerate the public suffix
    index file.

  - iana-pyfunceble - the tool the regenerate the iana index file.

That means that from now on, only the following console scripts will
be available:

  - pyfunceble - the pyfunceble tool.
  - PyFunceble - the pyfunceble tool - for retro-compatiblity.
  - clean-pyfunceble - the tool to cleanup pyfunceble's directories.

This patch touches #328.

Contributors:
  * @Nilsonfsilva
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: 🏗 In progress
Development

When branches are created from issues, their pull requests are automatically linked.

2 participants