Conflicting dependencies running scripts/eachdist.py develop

[droppedbars]

Describe your environment
FWIW, on Mint 20.2 using Python 3.8.10

Steps to reproduce
Brand new clone and attempt to install the dependencies:
git clone https://github.com/open-telemetry/opentelemetry-python-contrib.git
cd ./opentelementry-python-contrib
python3 -m venv .venv
source ./.venv/bin/activate
python3 -m pip install --upgrade pip
python3 scripts/eachdist.py develop

What is the expected behavior?
Successful installation of dependencies

What is the actual behavior?
ERROR: Cannot install opentelemetry-exporter-datadog[test]==0.30b0, opentelemetry-instrumentation-aiohttp-client[test]==0.33b0, opentelemetry-instrumentation-aiopg[test]==0.33b0, opentelemetry-instrumentation-asgi==0.33b0, opentelemetry-instrumentation-asgi[test]==0.33b0, opentelemetry-instrumentation-asyncpg[test]==0.33b0, opentelemetry-instrumentation-aws-lambda[test]==0.33b0, opentelemetry-instrumentation-boto3sqs[test]==0.33b0, opentelemetry-instrumentation-boto[test]==0.33b0, opentelemetry-instrumentation-botocore[test]==0.33b0, opentelemetry-instrumentation-celery[test]==0.33b0, opentelemetry-instrumentation-dbapi==0.33b0, opentelemetry-instrumentation-dbapi[test]==0.33b0, opentelemetry-instrumentation-django[test]==0.33b0, opentelemetry-instrumentation-elasticsearch[test]==0.33b0, opentelemetry-instrumentation-falcon[test]==0.33b0, opentelemetry-instrumentation-fastapi[test]==0.33b0, opentelemetry-instrumentation-flask[test]==0.33b0, opentelemetry-instrumentation-grpc[test]==0.33b0, opentelemetry-instrumentation-httpx[test]==0.33b0, opentelemetry-instrumentation-kafka-python[test]==0.33b0, opentelemetry-instrumentation-pymemcache[test]==0.33b0, opentelemetry-instrumentation-pymongo[test]==0.33b0, opentelemetry-instrumentation-pyramid[test]==0.33b0, opentelemetry-instrumentation-redis[test]==0.33b0, opentelemetry-instrumentation-remoulade[test]==0.33b0, opentelemetry-instrumentation-requests[test]==0.33b0, opentelemetry-instrumentation-sqlalchemy[test]==0.33b0, opentelemetry-instrumentation-starlette[test]==0.33b0, opentelemetry-instrumentation-tornado[test]==0.33b0, opentelemetry-instrumentation-urllib3[test]==0.33b0, opentelemetry-instrumentation-urllib[test]==0.33b0, opentelemetry-instrumentation-wsgi==0.33b0, opentelemetry-instrumentation-wsgi[test]==0.33b0 and opentelemetry-test-utils because these package versions have conflicting dependencies.

The conflict is caused by:
opentelemetry-instrumentation-wsgi[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-sdk 1.12.0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-dbapi[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-asgi[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-botocore[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-django[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-wsgi 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-kafka-python[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-aws-lambda[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-tornado[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-pymemcache[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-urllib3[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-aiohttp-client[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-falcon[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-fastapi[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-asgi 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-boto[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-pyramid[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-redis[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-requests[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-boto3sqs[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-flask[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-pymongo[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-grpc[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-starlette[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-sqlalchemy[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-asyncpg[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-aiopg[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-dbapi 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-httpx[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-celery[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-remoulade[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-elasticsearch[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-instrumentation-urllib[test] 0.33b0 depends on opentelemetry-semantic-conventions==0.33b0
opentelemetry-exporter-datadog[test] 0.30b0 depends on opentelemetry-semantic-conventions==0.30b0

To fix this you could try to:

1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

[droppedbars]

Updated opentelemetry-python-contrib/exporter/opentelemetry-exporter-datadog/setup.cfg:
45: opentelemetry-semantic-conventions == 0.33b0

allowed the script to run, but then running python3 scripts/eachdist.py test failed with:
ImportError: cannot import name 'escape' from 'jinja2'

[droppedbars]

Updated instrumentation/opentelemetry-instrumentation-flask/src/opentelemetry/instrumentation/flask/package.py
16: _instruments = ("flask >= 2.0, < 3.0",)

now rerunning eachdist.py develop fails with:

ERROR: Cannot install opentelemetry-instrumentation-flask and opentelemetry-instrumentation-pyramid[test]==0.33b0 because these package versions have conflicting dependencies.

The conflict is caused by:
opentelemetry-instrumentation-pyramid[test] 0.33b0 depends on werkzeug==0.16.1
flask 2.2.2 depends on Werkzeug>=2.2.2
opentelemetry-instrumentation-pyramid[test] 0.33b0 depends on werkzeug==0.16.1
flask 2.2.1 depends on Werkzeug>=2.2.0
opentelemetry-instrumentation-pyramid[test] 0.33b0 depends on werkzeug==0.16.1
flask 2.2.0 depends on Werkzeug>=2.2.0
opentelemetry-instrumentation-pyramid[test] 0.33b0 depends on werkzeug==0.16.1
flask 2.1.3 depends on Werkzeug>=2.0
opentelemetry-instrumentation-pyramid[test] 0.33b0 depends on werkzeug==0.16.1
flask 2.1.2 depends on Werkzeug>=2.0
opentelemetry-instrumentation-pyramid[test] 0.33b0 depends on werkzeug==0.16.1
flask 2.1.1 depends on Werkzeug>=2.0
opentelemetry-instrumentation-pyramid[test] 0.33b0 depends on werkzeug==0.16.1
flask 2.1.0 depends on Werkzeug>=2.0
opentelemetry-instrumentation-pyramid[test] 0.33b0 depends on werkzeug==0.16.1
flask 2.0.3 depends on Werkzeug>=2.0
opentelemetry-instrumentation-pyramid[test] 0.33b0 depends on werkzeug==0.16.1
flask 2.0.2 depends on Werkzeug>=2.0
opentelemetry-instrumentation-pyramid[test] 0.33b0 depends on werkzeug==0.16.1
flask 2.0.1 depends on Werkzeug>=2.0
opentelemetry-instrumentation-pyramid[test] 0.33b0 depends on werkzeug==0.16.1
flask 2.0.0 depends on Werkzeug>=2.0

To fix this you could try to:

1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

[droppedbars]

Updated instrumentation/opentelemetry-instrumentation-pyramid/setup.cfg
51: werkzeug == 2.0.3

now can run eachdist.py develop, but eachdist.py test fails with:

====================================================================== FAILURES ======================================================================
_____________________________________________ TestSqlalchemyInstrumentation.test_async_trace_integration _____________________________________________

self = <tests.test_sqlalchemy.TestSqlalchemyInstrumentation testMethod=test_async_trace_integration>

@pytest.mark.skipif(
    not sqlalchemy.__version__.startswith("1.4"),
    reason="only run async tests for 1.4",
)
def test_async_trace_integration(self):
    async def run():
        from sqlalchemy.ext.asyncio import (  # pylint: disable-all
            create_async_engine,
        )

        engine = create_async_engine("sqlite+aiosqlite:///:memory:")
        SQLAlchemyInstrumentor().instrument(
            engine=engine.sync_engine, tracer_provider=self.tracer_provider
        )
        async with engine.connect() as cnx:
            await cnx.execute(sqlalchemy.text("SELECT       1 + 1;"))
        spans = self.memory_exporter.get_finished_spans()
        self.assertEqual(len(spans), 2)
        # first span - the connection to the db
        self.assertEqual(spans[0].name, "connect")
        self.assertEqual(spans[0].kind, trace.SpanKind.CLIENT)
        # second span - the query
        self.assertEqual(spans[1].name, "SELECT :memory:")
        self.assertEqual(spans[1].kind, trace.SpanKind.CLIENT)
        self.assertEqual(
            spans[1].instrumentation_scope.name,
            "opentelemetry.instrumentation.sqlalchemy",
        )

  asyncio.get_event_loop().run_until_complete(run())

instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlalchemy.py:114:

---

/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlalchemy.py:95: in run
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
.venv/lib/python3.8/site-packages/sqlalchemy/ext/asyncio/engine.py:43: in create_async_engine
sync_engine = _create_engine(*arg, **kw)
:2: in create_engine
???
.venv/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py:309: in warned
return fn(*args, **kwargs)
.venv/lib/python3.8/site-packages/sqlalchemy/engine/create.py:548: in create_engine
dbapi = dialect_cls.dbapi(**dbapi_args)

---

cls = <class 'sqlalchemy.dialects.sqlite.aiosqlite.SQLiteDialect_aiosqlite'>

@classmethod
def dbapi(cls):
    return AsyncAdapt_aiosqlite_dbapi(

      __import__("aiosqlite"), __import__("sqlite3")

    )

E ModuleNotFoundError: No module named 'aiosqlite'

.venv/lib/python3.8/site-packages/sqlalchemy/dialects/sqlite/aiosqlite.py:313: ModuleNotFoundError
----------------------------------------------------------------- Captured log call ------------------------------------------------------------------
WARNING opentelemetry.instrumentation.instrumentor:instrumentor.py:127 Attempting to uninstrument while already uninstrumented
___________________________________________ TestSqlalchemyInstrumentation.test_create_async_engine_wrapper ___________________________________________

self = <tests.test_sqlalchemy.TestSqlalchemyInstrumentation testMethod=test_create_async_engine_wrapper>

@pytest.mark.skipif(
    not sqlalchemy.__version__.startswith("1.4"),
    reason="only run async tests for 1.4",
)
def test_create_async_engine_wrapper(self):
    async def run():
        SQLAlchemyInstrumentor().instrument()
        from sqlalchemy.ext.asyncio import (  # pylint: disable-all
            create_async_engine,
        )

        engine = create_async_engine("sqlite+aiosqlite:///:memory:")
        async with engine.connect() as cnx:
            await cnx.execute(sqlalchemy.text("SELECT       1 + 1;"))
        spans = self.memory_exporter.get_finished_spans()
        self.assertEqual(len(spans), 2)
        # first span - the connection to the db
        self.assertEqual(spans[0].name, "connect")
        self.assertEqual(spans[0].kind, trace.SpanKind.CLIENT)
        # second span - the query
        self.assertEqual(spans[1].name, "SELECT :memory:")
        self.assertEqual(spans[1].kind, trace.SpanKind.CLIENT)
        self.assertEqual(
            spans[1].instrumentation_scope.name,
            "opentelemetry.instrumentation.sqlalchemy",
        )

  asyncio.get_event_loop().run_until_complete(run())

instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlalchemy.py:217:

---

/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
return future.result()
instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlalchemy.py:201: in run
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
instrumentation/opentelemetry-instrumentation-sqlalchemy/src/opentelemetry/instrumentation/sqlalchemy/engine.py:59: in _wrap_create_async_engine_internal
engine = func(*args, **kwargs)
.venv/lib/python3.8/site-packages/sqlalchemy/ext/asyncio/engine.py:43: in create_async_engine
sync_engine = _create_engine(*arg, **kw)
:2: in create_engine
???
.venv/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py:309: in warned
return fn(*args, **kwargs)
.venv/lib/python3.8/site-packages/sqlalchemy/engine/create.py:548: in create_engine
dbapi = dialect_cls.dbapi(**dbapi_args)

---

cls = <class 'sqlalchemy.dialects.sqlite.aiosqlite.SQLiteDialect_aiosqlite'>

@classmethod
def dbapi(cls):
    return AsyncAdapt_aiosqlite_dbapi(

      __import__("aiosqlite"), __import__("sqlite3")

    )

E ModuleNotFoundError: No module named 'aiosqlite'

.venv/lib/python3.8/site-packages/sqlalchemy/dialects/sqlite/aiosqlite.py:313: ModuleNotFoundError
================================================================== warnings summary ==================================================================
instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlalchemy.py: 12 warnings
instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlcommenter.py: 4 warnings
/home/patrick/dev/opentelemetry-python-contrib/.venv/lib/python3.8/site-packages/opentelemetry/sdk/trace/init.py:1162: DeprecationWarning: Call to deprecated method init. (You should use InstrumentationScope) -- Deprecated since version 1.11.1.
InstrumentationInfo(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
====================================================== 2 failed, 8 passed, 16 warnings in 0.65s ======================================================

[droppedbars]

While I expect the required change to the datadog exporter was simply an oversight as all others appear to be on 0.33b0, the other two changes I'm less certain of their impacts (though tests generally look ok) since they change the versions of flask and werkzeug.

The last round of errors I haven't looked too deeply at.

If this is all from some mistake on my part, my apologies, this is my first time attempting to clone and work with the project.

[droppedbars]

Next step did the following two changes:
instrumentation/opentelemetry-instrumentation-sqlalchemy/setup.cfg
51: added "aiosqlite" to the test requirements

instrumentation/opentelemetry-instrumentation-httpx/setup.cfg
50: added "respx" to the test requirements

now it seems to have failures in test_elasticsearch.py and appears to freeze at:
instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py::TestElasticsearchIntegration::test_dsl_create FAILED [ 7%]
instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py::TestElasticsearchIntegration::test_dsl_index FAILED [ 14%]
instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py::TestElasticsearchIntegration::test_dsl_search FAILED [ 21%]
instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py::TestElasticsearchIntegration::test_instrumentor FAILED [ 28%]
instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py::TestElasticsearchIntegration::test_multithread

[droppedbars]

Just saw the first issue was captured here #1083 and it appears I'm beating down a no longer supported path.

[ocelotl]

@droppedbars yes, please use tox ✌️