Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(seer grouping): Add seer.similarity.supported_platforms option #72729

Conversation

lobsterkatie
Copy link
Member

@lobsterkatie lobsterkatie commented Jun 13, 2024

This changes our list of Seer similarity supported platforms to be configurable via the options automator. As we canonically support new platforms, we should still add them to the default value here (rather than adding them in the automator), but this will let us selectively enable or disable platforms on the fly as we're testing things out.

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Jun 13, 2024
Copy link

codecov bot commented Jun 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.05%. Comparing base (473b4dd) to head (8b324b9).
Report is 6 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master   #72729       +/-   ##
===========================================
+ Coverage   51.46%   78.05%   +26.59%     
===========================================
  Files        6574     6600       +26     
  Lines      293173   294329     +1156     
  Branches    50560    50756      +196     
===========================================
+ Hits       150878   229740    +78862     
+ Misses     140724    58349    -82375     
- Partials     1571     6240     +4669     
Files Coverage Δ
src/sentry/options/defaults.py 100.00% <100.00%> (ø)
src/sentry/seer/similarity/utils.py 100.00% <100.00%> (+78.57%) ⬆️

... and 2121 files with indirect coverage changes

@lobsterkatie lobsterkatie marked this pull request as ready for review June 13, 2024 19:56
@lobsterkatie lobsterkatie merged commit 93862df into master Jun 13, 2024
51 checks passed
@lobsterkatie lobsterkatie deleted the kmclb-make-seer-similarity-supported-platforms-configurable branch June 13, 2024 19:58
from sentry.constants import PLACEHOLDER_EVENT_TITLES
from sentry.eventstore.models import Event
from sentry.utils.safe import get_path

logger = logging.getLogger(__name__)

MAX_FRAME_COUNT = 50
SEER_ELIGIBLE_PLATFORMS = frozenset(["python", "javascript", "node"])
SEER_ELIGIBLE_PLATFORMS = options.get("seer.similarity.supported_platforms")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this doesn't work for a couple reasons:

  • this attempts to access the database at import time when it is not available
  • the value will not change as options are changed because it is only set at import time

@asottile-sentry
Copy link
Member

going to revert because this is breaking mypy locally:

Error constructing plugin instance of NewSemanalDjangoPlugin

Traceback (most recent call last):
  File "/Users/asottile/workspace/sentry/.venv/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
             ^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "mypy/main.py", line 100, in main
  File "mypy/main.py", line 182, in run_build
  File "mypy/build.py", line 192, in build
  File "mypy/build.py", line 237, in _build
  File "mypy/build.py", line 501, in load_plugins
  File "mypy/build.py", line 482, in load_plugins_from_config
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/mypy_django_plugin/main.py", line 65, in __init__
    self.django_context = DjangoContext(self.plugin_config.django_settings_module)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/mypy_django_plugin/django/context.py", line 98, in __init__
    apps, settings = initialize_django(self.django_settings_module)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/mypy_django_plugin/django/context.py", line 81, in initialize_django
    settings._setup()  # type: ignore[misc]
    ^^^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/django/conf/__init__.py", line 76, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/django/conf/__init__.py", line 190, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/.local/share/sentry-devenv/pythons/3.11.8/python/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/asottile/workspace/sentry/src/sentry/conf/server_mypy.py", line 3, in <module>
    configure(skip_service_validation=True)
  File "/Users/asottile/workspace/sentry/src/sentry/runner/__init__.py", line 33, in configure
    _configure(ctx, py, yaml, skip_service_validation)
  File "/Users/asottile/workspace/sentry/src/sentry/runner/settings.py", line 121, in configure
    initialize_app(
  File "/Users/asottile/workspace/sentry/src/sentry/runner/initializer.py", line 366, in initialize_app
    django.setup()
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/django/apps/registry.py", line 116, in populate
    app_config.import_models()
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/django/apps/config.py", line 269, in import_models
    self.models_module = import_module(models_module_name)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/.local/share/sentry-devenv/pythons/3.11.8/python/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/asottile/workspace/sentry/src/sentry/plugins/sentry_interface_types/models.py", line 2, in <module>
    from sentry.plugins.bases.tag import TagPlugin
  File "/Users/asottile/workspace/sentry/src/sentry/plugins/bases/__init__.py", line 2, in <module>
    from .issue2 import IssueTrackingPlugin2  # NOQA
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/src/sentry/plugins/bases/issue2.py", line 19, in <module>
    from sentry.plugins.endpoints import PluginGroupEndpoint
  File "/Users/asottile/workspace/sentry/src/sentry/plugins/endpoints.py", line 9, in <module>
    from sentry.api.bases.group import GroupEndpoint
  File "/Users/asottile/workspace/sentry/src/sentry/api/bases/__init__.py", line 7, in <module>
    from .sentryapps import *  # NOQA
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/src/sentry/api/bases/sentryapps.py", line 20, in <module>
    from sentry.coreapi import APIError
  File "/Users/asottile/workspace/sentry/src/sentry/coreapi.py", line 8, in <module>
    from sentry.ingest.consumer.processors import CACHE_TIMEOUT
  File "/Users/asottile/workspace/sentry/src/sentry/ingest/consumer/processors.py", line 15, in <module>
    from sentry.event_manager import save_attachment
  File "/Users/asottile/workspace/sentry/src/sentry/event_manager.py", line 72, in <module>
    from sentry.grouping.ingest.seer import get_seer_similar_issues, should_call_seer_for_grouping
  File "/Users/asottile/workspace/sentry/src/sentry/grouping/ingest/seer.py", line 13, in <module>
    from sentry.seer.similarity.utils import (
  File "/Users/asottile/workspace/sentry/src/sentry/seer/similarity/utils.py", line 12, in <module>
    SEER_ELIGIBLE_PLATFORMS = options.get("seer.similarity.supported_platforms")
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/src/sentry/options/manager.py", line 299, in get
    result = self.store.get(opt, silent=silent)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/src/sentry/options/store.py", line 97, in get
    result = self.get_store(key, silent=silent)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/src/sentry/options/store.py", line 193, in get_store
    value = self.model.objects.get(key=key.name).value
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/django/db/models/query.py", line 645, in get
    num = len(clone)
          ^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/django/db/models/query.py", line 382, in __len__
    self._fetch_all()
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/django/db/models/query.py", line 1928, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/django/db/models/query.py", line 91, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
    cursor.execute(sql, params)
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 98, in _execute
    warnings.warn(self.APPS_NOT_READY_WARNING_MSG, category=RuntimeWarning)
RuntimeWarning: Accessing the database during app initialization is discouraged. To fix this warning, avoid executing queries in AppConfig.ready() or when your app modules are imported.

@asottile-sentry asottile-sentry added the Trigger: Revert add to a merged PR to revert it (skips CI) label Jun 14, 2024
@getsentry-bot
Copy link
Contributor

PR reverted: 2b3dc95

getsentry-bot added a commit that referenced this pull request Jun 14, 2024
…s` option (#72729)"

This reverts commit 93862df.

Co-authored-by: asottile-sentry <103459774+asottile-sentry@users.noreply.github.com>
@github-actions github-actions bot locked and limited conversation to collaborators Jun 30, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Scope: Backend Automatically applied to PRs that change backend components Trigger: Revert add to a merged PR to revert it (skips CI)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants