Optional dependencies not listed with their corresponding extras

[diazona]

When running the test tests/distribution/test_distribution_packages.py::TestExternalProject::test_optional_dependencies[pytest==7.3.0], setuptools-pyproject-migration does not list the corresponding extras as part of the Requirement objects.

I'm not sure what the correct approach should be; we can figure that out while solving this issue.

self = <test_distribution_packages.TestExternalProject object at 0x7f32b0ca7090>
expected = StandardMetadata(name='pytest', version=<Version('7.3.0')>, description='pytest: simple powerful testing with Python',...are Development :: Testing', 'Topic :: Utilities'], keywords=['test,unittest'], scripts={}, gui_scripts={}, dynamic=[])
actual = StandardMetadata(name='pytest', version=<Version('7.3.0')>, description='pytest: simple powerful testing with Python',..., 'unittest'], scripts={'pytest': 'pytest:console_main', 'py.test': 'pytest:console_main'}, gui_scripts={}, dynamic=[])

    def test_optional_dependencies(self, expected: StandardMetadata, actual: StandardMetadata):
        # Check that the set of extras is the same
        assert expected.optional_dependencies.keys() == actual.optional_dependencies.keys()
    
        # Check that the dependencies associated with each extra are the same
        for extra in expected.optional_dependencies:
>           assert sorted(expected.optional_dependencies[extra], key=str) == sorted(
                actual.optional_dependencies[extra], key=str
            )
E           assert [<Requirement('argcomplete; extra == "testing"')>, <Requirement('attrs>=19.2.0; extra == "testing"')>, <Requirement('hypothesis>=3.56; extra == "testing"')>, <Requirement('mock; extra == "testing"')>, <Requirement('nose; extra == "testing"')>, <Requirement('pygments>=2.7.2; extra == "testing"')>, <Requirement('requests; extra == "testing"')>, <Requirement('xmlschema; extra == "testing"')>] == [<Requirement('argcomplete')>, <Requirement('attrs>=19.2.0')>, <Requirement('hypothesis>=3.56')>, <Requirement('mock')>, <Requirement('nose')>, <Requirement('pygments>=2.7.2')>, <Requirement('requests')>, <Requirement('xmlschema')>]
E             At index 0 diff: <Requirement('argcomplete; extra == "testing"')> != <Requirement('argcomplete')>
E             Full diff:
E               [
E             -  <Requirement('argcomplete')>,
E             +  <Requirement('argcomplete; extra == "testing"')>,
E             ?                           ++++++++++++++++++++
E             -  <Requirement('attrs>=19.2.0')>,
E             +  <Requirement('attrs>=19.2.0; extra == "testing"')>,
E             ?                             ++++++++++++++++++++
E             -  <Requirement('hypothesis>=3.56')>,
E             +  <Requirement('hypothesis>=3.56; extra == "testing"')>,
E             ?                                ++++++++++++++++++++
E             -  <Requirement('mock')>,
E             -  <Requirement('nose')>,
E             +  <Requirement('mock; extra == "testing"')>,
E             +  <Requirement('nose; extra == "testing"')>,
E             -  <Requirement('pygments>=2.7.2')>,
E             +  <Requirement('pygments>=2.7.2; extra == "testing"')>,
E             ?                               ++++++++++++++++++++
E             -  <Requirement('requests')>,
E             -  <Requirement('xmlschema')>,
E             +  <Requirement('requests; extra == "testing"')>,
E             +  <Requirement('xmlschema; extra == "testing"')>,
E               ]

/home/diazona/programming/setuptools-pyproject-migration/tests/distribution/test_distribution_packages.py:146: AssertionError

[diazona]

After investigating, I think the root of this problem is that our code which parses the core metadata keeps the extra == ... markers when it creates Requirement objects for optional dependencies, but pyproject-metadata's StandardMetadata.from_pyproject() creates Requirement objects without extra == ... markers. It adds the marker when writing the StandardMetadata object out to RFC822 headers. So we either need to add the extra marker to the Requirement objects we create in TestExternalProject.actual() (it could be done there, or later, while running the test for optional dependencies), or remove the extra marker from the Requirements created in PyPiPackagePreparation._core_metadata_*(). Either way I think to do this robustly would require some parsing of the markers, but perhaps we could take advantage of the fact that pyproject-metadata always adds an extra marker at the end.

I'll continue looking into this.