Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

vendor: fix debtcollector imports #1152

Merged
merged 2 commits into from Nov 27, 2019
Merged

vendor: fix debtcollector imports #1152

merged 2 commits into from Nov 27, 2019

Conversation

majorgreys
Copy link
Collaborator

@majorgreys majorgreys commented Nov 26, 2019 •
edited

This adjustment had already been applied elsewhere in the vendored code but I had missed ddtrace/vendor/debtcollector/_utils.py. When I attempted to run it locally I got:

ModuleNotFoundError: No module named 'six'

I am not sure why this did not cause problems in CircleCI.

@majorgreys majorgreys requested a review from a team as a code owner November 26, 2019 21:05
brettlangdon
brettlangdon previously approved these changes Nov 26, 2019
View reviewed changes
jd
jd previously approved these changes Nov 27, 2019
View reviewed changes
Copy link
Contributor

@jd jd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't see that in CircleCI because it's likely something else pulls the dep.

@majorgreys majorgreys dismissed stale reviews from jd and brettlangdon via a119b49 November 27, 2019 18:07
zacharycmontoya
zacharycmontoya approved these changes Nov 27, 2019
View reviewed changes
Copy link

@zacharycmontoya zacharycmontoya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The one commit after the previous approval looks good 👍

@majorgreys majorgreys merged commit 12e48eb into master Nov 27, 2019
@majorgreys majorgreys deleted the majorgreys/fix-debtcollector-vendor branch November 27, 2019 19:58
@majorgreys majorgreys added this to the 0.32.0 milestone Dec 18, 2019
codeboten pushed a commit to lightstep/ls-trace-py that referenced this pull request Mar 4, 2020
@brettlangdon @jd
@majorgreys @JBKahn @adamchainz @jeremy-lq @thieman @Kyle-Verhoog
Updating code to 0.34.0 (#8)
03fb2ca 
* test: remove unneeded deps listing

* Remove quote enforcement

The current quote enforcement is a restrictive and annoying.

Furthermore, it arbitrary enforces single quotes whereas tools like black —
which are now standard in Python ecosystem — sticks to the Python widely used
double quotes.

* Add flake8-blind-except

Checks that no except statement is used without specifying an exception type.

* flake8: ignore no error

We only ignore W503 in favor of W504 to define where we put the line break
before binary operators.

* Add flake8-builtins

This makes sure we pick proper variable names.

* Add flake8-logging-format

This makes sure that we use a correct logging format in our log strings.

* Enable flake8-rst-docstrings

This fixes various RST directives that we got wrong and log format

* flake8: enable flake8-docstrings

* internal: use histogram and counters for metrics (DataDog#1134)

* internal: move health metrics enabled to config (DataDog#1135)

* internal: move health metrics enabled to config

* remove unused import

* ci: add black step to ci (DataDog#1137)

* ci: add black step to ci

* also ignore docs/

* Fixing sqlalchemy test failures (DataDog#1138)

Version 8.0.18 is causing segfaults in sqlalchemy tests.

Signed-off-by: Alex Boten <aboten@lightstep.com>

* internal: use debtcollector for keyword argument warnings (DataDog#1147)

* core: deprecate unused app type (DataDog#1144)

* core: remove unused app type

* deprecate app_type

* pin multidict dependency for aiobotocore02 tests (DataDog#1145)

Signed-off-by: Alex Boten <aboten@lightstep.com>

* vendor: fix debtcollector imports (DataDog#1152)

* vendor: fix debtcollector imports

* use relative path for all other vendored libraries

* core: common span types and revise language tag (DataDog#1150)

* core: common span types

* include enum backport in install_requires

* attempt to fix tox issues in ci for celery

* Start black formatting some files (DataDog#1141)

* Start black formatting some files

* fix flake8

* fix black too

* Add support for dogpile.cache (DataDog#1123)

* Add dogpile support

* Docs

* Tests

* Add to CI

* Apply suggestions from code review

Co-Authored-By: Tahir H. Butt <tahir@tahirbutt.com>

* Fix tests

* core: Change DatadogSampler defaults (DataDog#1151)

* bottle: fix status code for responses (DataDog#1158)

* bottle: handle 400 responses

* fix tests

* cleaner conditionals

* core: Deprecate ddtrace.ext.AppTypes (DataDog#1162)

* core: Deprecate ddtrace.ext.AppTypes

* fix flake8

* Remove unused helper

* core: safely deprecate ext type constants (DataDog#1165)

* core: fix new sampler defaults and add env variables to configure (DataDog#1166)

* core: fix new sampler defaults and add env variables to configure

* fix tests

* internal: Set numeric tags on Span.metrics instead of Span.meta

* core: default to DatadogSampler
As well, ensure that unless explicitly configured continue to use RateByServiceSampler for DatadogSampler

* removed unused import

* encode python in the install_requires string

This is breaking my build right now when I try to upgrade versions.

* not install enum34 for Python 3.4

* add funcsigs backport to install

* copy from debtcollector's requirements

* tests: Fix botocore py34 tests
PyYAML 5.3 dropped support for Python 3.4, we need 5.2 or lower

* default rate limit of 100, always set span.sampled = True

* Apply suggestions from code review

* fix flake8

* Improve tracer.trace docs

* Dual License

* Dual License

* core: ensure http.status_code is always set as str on meta

* fix flake8

* update missed assertions

* fix flask test

* Prefer random.getrandbits on Python 3+, fall back to urandom implementation on 2 (DataDog#1183)

* tests: fix botocore tests on py3.4 (DataDog#1189)

* black: ddtrace/bootstrap (DataDog#1187)

* black: ddtrace/utils/ (DataDog#1188)

* black: ddtrace/internal/ (DataDog#1190)

* black: ddtrace/internal/

* fix flake8

* internal: Refactor setup.py c-extension building (DataDog#1191)

* internal: Vendor psutil dependency

* fix flake8/black issues

* update comment

* move vendor's setup code into ddtrace/vendor/*/setup.py

* catch any exceptions loading vendor setup

* add back required dep

* remove unnecessary caching step

* re-run setup.py develop before every test run

* simplify setup.py

* fix black formatting

* refactor setup.py extension building

* chord(flake8,black): ignore W504 and E231 not respected by black (DataDog#1185)

black does not respect those error codes and therefore they should be ignored.

It does support W503 so that one should be included.
This patch therefore black-ifies the files that do not respect W503.

* fix(writer): start thread as late as possible (DataDog#1193)

Fixes DataDog#1192

* internal: Vendor psutil dependency (DataDog#1160)

* internal: Vendor psutil dependency

* fix flake8/black issues

* update comment

* move vendor's setup code into ddtrace/vendor/*/setup.py

* catch any exceptions loading vendor setup

* add back required dep

* remove unnecessary caching step

* re-run setup.py develop before every test run

* simplify setup.py

* fix black formatting

* fix bad merge

* tests: Run django_contrib and django_drf_contrib tests separately (DataDog#1197)

This should help with concurrency in CircleCI and help speed up the slow Django builds

* [django] New Django Integration  (DataDog#1161)

* [django] Setup pytest-django

* fix djangorestframework tests

* Move existing django tests (DataDog#997)

*  [django] Remove autopatching (DataDog#998)

* Move existing django tests

* [django] Remove autopatching

* add back missing 'apply_django_patches' changes

* fix merge conflict

* [django] new patching approach

* [django] trace all middleware __call__, process_request, process_response

* [django] add documentation

* [django] add other middleware hooks

* [django] updated request data

* [django] add tests

* [django] update tox.ini

* [django] add instrumentation for rendering and views

* [django] add db and cache instrumentation back

* [django] add idempotency checks, tests

* [django] add django v1 tests

* [django] add patch test case

* [django] fix view instrumentation

* [django] pull duplicate app code into shared modules

* [django] clarify middleware patching

* [django] finally get working db test

* [django] use pytest test cases

* [django] get caches working

* [django] add cache client tests

* [django] begin adding legacy configuration support

* [django] add migration guide, integration documentation

* [django] check if DRF installed before attempting to patch

* [django] add service name config tests

* [django] add back database service name prefix

* [django] enable by default

* [django] add template tests

* [django] remove old code

* [django] add analytics config tests

* [django] error handling tests

* [django] port over rest of middleware tests

* [django] fix trace_query_string

* [django] delete old tests

* [django] python2 compatibility

* [django] get DRF working and tested

* [django] update django, drf matrix support

* [django] housekeeping

* [django] lint

* [django] update tox matrix

* [django] actually use vendored wrapt

* [django] add tests for view methods; change log levels

* [django] linting

* [django] convert to SpanTypes, use debtcollector

* [django] run black

* [django] rm more old code

* [django] add black-compatibility flake ignore

* [django] docstring tweaks; cleanup

* [django] ignore rst whitespace flake rule

* [django] add config to disable caches and databases

* [django] add urlpattern tests

* [django] remove static.static instrumentation

* Add back missing TraceMiddleware

* manually call patch() when using old settings without calling patch

* add migration from settings

* add migration docs

* fix service_name

* fix django db

* fix unpatching

* black format

* fix black/flake8

* fix black

* black check django contrib

* update docs

* skip unpatch in test

* remove unused import

* allow wider range of env names

* just use _SERVICE_NAME for now

* fix migration tests

* fix db patching

* fix drf tests

* avoid info logs

Co-authored-by: Brett Langdon <me@brett.is>
Co-authored-by: Tahir H. Butt <tahir@tahirbutt.com>

* fix(ci): limit Werkzeug to <1 for old Flask version

The API changed in Werkzeug >= 1 and is not compatible with older Flask
version. Capping Werkzeug makes tests work again.

* fix(tox): flake8 does not need to install (DataDog#1200)

This makes running flake8 in tox faster.

* core: unvendor msgpack (DataDog#1199)

* core: unvendor msgpack

* fix deps

* remove msgpack version check for use_bin_type

* fix flake8

* remove checks for msgpack c-extension

* remove old comment

Co-authored-by: Brett Langdon <me@brett.is>

* chord(encoding): remove unnused global variable

* refactor(encoding): simplify msgpack/json encoders

This refactor the code to make it simpler:
- we only define the useful class at load time based on msgpack availability
- we declare methods are static when they are
- we use class attributes rather than useless per-instance attribute

* refactor(encoding): msgpack is a dependency, don't "try" to import it

We just import msgpack for sure since ddtrace depends on it, it has to be
there.

* fix(setup): set minimum msgpack version we support

We test with 0.5.0 and above in the CI.

* ci(tox): do not install ddtrace for running black (DataDog#1204)

This is a follow-up of DataDog#1200, doing the same thing for black.

* ci(wait): do not install ddtrace in the wait target

We don't use ddtrace at all, we just run a Python script.

* ci(docs): modernize documentation building

This streamlines the doc building process by:
- Using the latest shiny Python 3.8 in the CI
- Leveraging tox to create a venv to build the doc
- Upgrade to the latest polished Sphinx version to build the doc
- Remove the bulky Makefile
- Remove the esoteric Ruby build code, brrrr

* ci(update): use standard Python images for black and flake8 jobs

There's no need to pull our custom images there.

* ci(docs): fix deploy_dev target for releasing docs (DataDog#1212)

This currently fails since the docs are not built by rake anymore (I presume).

- Remove unneeded `sudo` usage in docs target
- Upgrade deploy_dev to Python 3.8

* utils: Update get_env to support any number of parts (DataDog#1208)

* utils: Update get_env to support any number of parts
Changes get_env(integration, key) to get_env(*parts) to make it easier to get
an env built from 1 part or more than 2 parts

the major change here to pay attention to is the third positional argument used
to be the default, not the default *must* be provided as a keyword argument

* keyword argument after *args is not supported in Python2

* remove *

* tests: fix ordering of PYTHONPATH (DataDog#1210)

* tests: fix ordering of PYTHONPATH
If you run the sitecustomize test cases on their own they fail because
they import the ddtrace/bootstrap/sitecustomize.py module and not
the tests/commands/bootstrap/sitecustomize.py module.

This is because this helper code would put the test bootstrap module
on the end of the PYTHONPATH, which means it gets looked at last.

I have no idea how this seems to work ok when you run the entire
test suite together, must be an odd dependency on a previous test
case?

* fix flake8

* ci(update) use standard Python image for test_build job (DataDog#1209)

* ci(update) use standard Python image for test_build job

* no longer use virtualenv

* Update .circleci/config.yml

* ci(tox): remove useless basepython specifiers (DataDog#1214)

* ci: rework .circleci/config.yml (DataDog#1213)

* ci: rework .circleci/config.yml

* add labels to docker images

* fix test_build job

* don't forget to checkout code

* run test build on all versions

* restructure test builds

* fix docker definition

* run flask python tests in parallel

* organize more

* fix pip install for python34

* use sudo with py34 to install pip deps

* use python:3.4 image instead

* reorganize the build flow

* break out more test suites

* fix job keys

* rename wait_all_tests

* we don't need a placeholder for pre_test

* remove build_docs from pre_test requirement

* remove python specific integration jobs

Co-authored-by: Tahir H. Butt <tahir@tahirbutt.com>

* core: Set _dd.measured tag on integration spans (DataDog#1196)

* internal: Add ability to easily mark a span to be measured

* contrib: start to update integrations to indicate measured spans

* fix flake8

* tests: start work on _dd.measured test updates

* finish up tests for existing modified integrations

* fix flake8 issues

* fix s/assert_span_is_measured/assert_is_measured/g

* fix issue

* ensure we measure old patching method

* measure more integrations

* fix black formatting

* more integrations and tests

* add missing import

* finish up integration changes

* fix flake8 issues

* fix failed tests

* fix double dot

* fix failing test

* use span.set_tag(SPAN_MEASURED_KEY) instead

* move _dd.measured to metrics

* more consistently set _dd.measured value

Co-authored-by: Tahir H. Butt <tahir@tahirbutt.com>

* Support msgpack 1.0.0 (DataDog#1216)

* fix(tests): remove encoding argument to msgpack.encode

This has been removed in msgpack 1.0.0.
Just use the decoder decode method.

* style(black): black test_integrations

* fix(encoding): always return bytes when decoding (DataDog#1220)

msgpack 1.0.0 has a different default behaviour which returns string rather
than bytes. Keep it to bytes by default for all versions.

Note that this is only used in testing anyway.

* fix(tests): use encoder decode in tests (DataDog#1221)

* fix(tox): do not skip install for docs

* fix: treat spinx warnings as errors

* fix: fix sphinx errors

* fix: typo

* lint/black fixes

* fix docs and django builds

* ci: requires virtualenv < 20

There's a regression in virtualenv >= 20 that makes one of the ddtrace-run +
Celery test to fails.

virtualenv now adds a site.py which modifies sys.path and breaks the PYTHONPATH
that is set by ddtrace-run.

Co-authored-by: Brett Langdon <me@brett.is>
Co-authored-by: Julien Danjou <julien@danjou.info>
Co-authored-by: Tahir H. Butt <tahir@tahirbutt.com>
Co-authored-by: Sam Park <spark@goodrx.com>
Co-authored-by: Joseph Kahn <josephbkahn@gmail.com>
Co-authored-by: Adam Johnson <me@adamj.eu>
Co-authored-by: Jeremy <jeremy-lq@users.noreply.github.com>
Co-authored-by: Travis Thieman <travis.thieman@gmail.com>
Co-authored-by: Kyle Verhoog <kyle@verhoog.ca>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@palazzem palazzem palazzem approved these changes

@zacharycmontoya zacharycmontoya zacharycmontoya approved these changes

@jd jd jd left review comments

@brettlangdon brettlangdon brettlangdon left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
0.32.0
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@majorgreys @jd @brettlangdon @palazzem @zacharycmontoya