Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make some dependency changes, support Python 3.10 and 3.11 #5611

Merged
merged 28 commits into from
Dec 30, 2022

Conversation

Jackenmen
Copy link
Member

@Jackenmen Jackenmen commented Mar 10, 2022

Description of the changes

This PR makes a bunch of dependency changes (not just updates) and enables support for Python 3.10.

Added primary dependencies:

  • typing_extensions (we don't use it but we probably will in the future)
  • yarl (we use it in our code so it shouldn't be transitive)
  • aiodns, Brotli, orjson (some of the speedup dependencies for aiohttp and discord.py)

Replaced dependencies:

  • apsw-wheels with apsw
  • appdirs replaced by platformdirs
    • I'm not entirely sure this should happen, a lot of popular projects (pip, virtualenv, pipenv, black, isort) have switched to it which suggests it's going to be preferred in the future but I have seen a few worrying things about how the maintainer approaches breaking changes.
      Not that I don't go through changelogs of all our dependencies but I'm not a fan of this.
  • chardet replaced by charset-normalizer (transitive dependency of aiohttp)
  • fuzzywuzzy replaced by rapidfuzz

Removed dependencies:

  • aiosqlite (this library is not used by Red anymore)
  • PyNaCl (transitive dependency of discord.py[voice] which we don't actually use)
  • python-Levenshtein-wheels (transitive dependency of fuzzywuzzy which we replaced with rapidfuzz)

The PyNaCl removal is still up for consideration, I'm not entirely sure why we have it added if we're using LL rather than d.py's own voice functionality.

Test run of updated install instructions:

Have the changes in this PR been tested?

Yes

@Jackenmen Jackenmen added Breaking Change Will potentially break some cogs. Blocked By: Dependency Blocked by a package listed as a dependency. Type: Dependency Update Adding/updating/removing some of Red’s dependencies. labels Mar 10, 2022
@Jackenmen Jackenmen added this to the 3.5.0 milestone Mar 10, 2022
@github-actions github-actions bot added Category: Cogs - Audio This is related to the Audio cog. Category: Bot Core Category: CI This is related to repository's CI configuration. Category: Cogs - CustomCommands This is related to the CustomCommands cog. Category: Docs - Other This is related to documentation that doesn't have its dedicated label. Category: Meta This is related to the repository maintenance. Category: Core - API - Utils Package This is related to stuff in `redbot.core.utils` labels Mar 10, 2022
@Jackenmen Jackenmen force-pushed the dep_changes branch 3 times, most recently from 9beecbd to 33cdb36 Compare March 12, 2022 07:08
@Jackenmen Jackenmen removed the Blocked By: Dependency Blocked by a package listed as a dependency. label Apr 3, 2022
@Jackenmen Jackenmen force-pushed the dep_changes branch 6 times, most recently from c2f69e9 to 8874238 Compare April 10, 2022 01:54
@aikaterna
Copy link
Member

I'm using a fork for testing some things that includes this PR, and I received this error with a brand new Python 3.10 install (installed through Brew) with a brand new venv. This is also a new Red install inside that venv. I had existing instances already configured on this device with alternate python installs and alternate venvs.

(hybrid) aikaterna@Mac-mini ~ % redbot-setup
Traceback (most recent call last):
  File "/Users/aikaterna/hybrid/bin/redbot-setup", line 5, in <module>
    from redbot.setup import run_cli
  File "/Users/aikaterna/hybrid/lib/python3.10/site-packages/redbot/setup.py", line 17, in <module>
    from redbot.core.cli import confirm
  File "/Users/aikaterna/hybrid/lib/python3.10/site-packages/redbot/core/__init__.py", line 4, in <module>
    from .config import Config
  File "/Users/aikaterna/hybrid/lib/python3.10/site-packages/redbot/core/config.py", line 22, in <module>
    from .drivers import IdentifierData, get_driver, ConfigCategory, BaseDriver
  File "/Users/aikaterna/hybrid/lib/python3.10/site-packages/redbot/core/drivers/__init__.py", line 4, in <module>
    from .. import data_manager
  File "/Users/aikaterna/hybrid/lib/python3.10/site-packages/redbot/core/data_manager.py", line 61, in <module>
    _old_config_location.rename(config_file)
  File "/opt/homebrew/Cellar/python@3.10/3.10.4/Frameworks/Python.framework/Versions/3.10/lib/python3.10/pathlib.py", line 1232, in rename
    self._accessor.rename(self, target)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/aikaterna/Library/Application Support/Red-DiscordBot/config.json' -> '/Users/aikaterna/Library/Preferences/Red-DiscordBot/config.json'

I copied my config.json from the Application Support directory to the Preferences directory and I was able to use redbot-setup.

@Jackenmen
Copy link
Member Author

@aikaterna thanks for testing! This should be resolved now.

@aikaterna
Copy link
Member

I encountered the same issue on another Mac that had a Python 3.9 venv, while running redbot and not redbot-setup but using the changes in 7e74062 has resolved it.
Posting the traceback for information only, in case you were interested.

(redenv39) username@iMac ~ % redbot margaret --dev
Traceback (most recent call last):
  File "/Users/username/redenv39/bin/redbot", line 5, in <module>
    from redbot.__main__ import main
  File "/Users/username/redenv39/lib/python3.9/site-packages/redbot/__main__.py", line 28, in <module>
    from redbot.core.bot import Red, ExitCodes, _NoOwnerSet
  File "/Users/username/redenv39/lib/python3.9/site-packages/redbot/core/__init__.py", line 4, in <module>
    from .config import Config
  File "/Users/username/redenv39/lib/python3.9/site-packages/redbot/core/config.py", line 22, in <module>
    from .drivers import IdentifierData, get_driver, ConfigCategory, BaseDriver
  File "/Users/username/redenv39/lib/python3.9/site-packages/redbot/core/drivers/__init__.py", line 4, in <module>
    from .. import data_manager
  File "/Users/username/redenv39/lib/python3.9/site-packages/redbot/core/data_manager.py", line 62, in <module>
    _old_config_location.rename(config_file)
  File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pathlib.py", line 1382, in rename
    self._accessor.rename(self, target)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/username/Library/Application Support/Red-DiscordBot/config.json' -> '/Users/username/Library/Preferences/Red-DiscordBot/config.json'

@Zephyrkul
Copy link
Contributor

Getting this traceback in my console whenever I get a CommandNotFound when I try this PR out:

Ignoring exception in on_command_error
Traceback (most recent call last):
  File "C:\Users\...\Desktop\Github\Red-DiscordBot\.venv\lib\site-packages\discord\client.py", line 375, in _run_event
    await coro(*args, **kwargs)
  File "C:\Users\...\Desktop\GitHub\Red-DiscordBot\redbot\core\events.py", line 260, in on_command_error
    fuzzy_commands = await fuzzy_command_search(
  File "C:\Users\...\Desktop\GitHub\Red-DiscordBot\redbot\core\utils\_internal_utils.py", line 159, in fuzzy_command_search
    extracted = process.extract(term, choices, limit=5, scorer=fuzz.QRatio)
  File "cpp_process.pyx", line 945, in rapidfuzz.cpp_process.extract
  File "cpp_process.pyx", line 803, in rapidfuzz.cpp_process.extract_list
  File "cpp_process.pyx", line 745, in rapidfuzz.cpp_process.extract_list_f64
  File "cpp_process.pyx", line 166, in rapidfuzz.cpp_process.preprocess_list
  File "cpp_utils.pyx", line 44, in rapidfuzz.cpp_utils.default_process_capi
  File "cpp_common.pxd", line 201, in cpp_common.conv_sequence
  File "cpp_common.pxd", line 171, in cpp_common.hash_sequence
TypeError: object of type 'Command' has no len()

Copy link
Contributor

@Drapersniper Drapersniper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't run bumpdeps for the extras but I assume some of them are out of date already.

setup.cfg Outdated Show resolved Hide resolved
setup.cfg Outdated Show resolved Hide resolved
setup.cfg Outdated Show resolved Hide resolved
setup.cfg Outdated Show resolved Hide resolved
setup.cfg Outdated Show resolved Hide resolved
setup.cfg Outdated Show resolved Hide resolved
setup.cfg Outdated Show resolved Hide resolved
setup.cfg Outdated Show resolved Hide resolved
setup.cfg Outdated Show resolved Hide resolved
setup.cfg Outdated Show resolved Hide resolved
Jackenmen added a commit to Jackenmen/Red-Install-Tests that referenced this pull request Jun 1, 2022
leetfin added a commit to leetfin/Red-DiscordBot that referenced this pull request Jun 3, 2022
@red-githubbot red-githubbot bot added the Changelog Entry: Pending Changelog entry for this PR hasn't been added by repo maintainers yet. label Dec 30, 2022
@Jackenmen Jackenmen deleted the dep_changes branch December 30, 2022 02:38
Jackenmen added a commit to Jackenmen/Red-DiscordBot that referenced this pull request Dec 31, 2022
…ors#5611)

Co-authored-by: jack1142 <6032823+jack1142@users.noreply.github.com>
Jackenmen added a commit to Jackenmen/Red-DiscordBot that referenced this pull request Dec 31, 2022
…ors#5611)

Co-authored-by: jack1142 <6032823+jack1142@users.noreply.github.com>
Jackenmen added a commit to Jackenmen/Red-DiscordBot that referenced this pull request Dec 31, 2022
…ors#5611)

Co-authored-by: jack1142 <6032823+jack1142@users.noreply.github.com>
@Flame442 Flame442 added Changelog Entry: Added Changelog entry for this PR has already been added to changelog PR. and removed Changelog Entry: Pending Changelog entry for this PR hasn't been added by repo maintainers yet. labels Apr 30, 2023
@github-actions github-actions bot added Category: Cogs - Mod This is related to the Mod cog. Category: Cogs - Downloader This is related to the Downloader cog. Category: Cogs - Alias This is related to the Alias cog. Category: Cogs - Economy This is related to the Economy cog. Category: Core - API - Config This is related to the `redbot.core.config` module and `redbot.core.drivers` package. Category: Core - API - Other This is related to core APIs that don't have a dedicated label. Category: Core - Bot Class This is related to the `redbot.core.bot.Red` class. Category: Core - Command-line Interfaces This is related to Red's CLIs (redbot, redbot-launcher, redbot-setup). Category: Core - Other Internals This is related to core internals that don't have a dedicated label. Category: Docs - Install Guides This is related to install guides. labels Apr 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Breaking Change Will potentially break some cogs. Category: CI This is related to repository's CI configuration. Category: Cogs - Alias This is related to the Alias cog. Category: Cogs - Audio This is related to the Audio cog. Category: Cogs - CustomCommands This is related to the CustomCommands cog. Category: Cogs - Downloader This is related to the Downloader cog. Category: Cogs - Economy This is related to the Economy cog. Category: Cogs - Mod This is related to the Mod cog. Category: Core - API - Config This is related to the `redbot.core.config` module and `redbot.core.drivers` package. Category: Core - API - Other This is related to core APIs that don't have a dedicated label. Category: Core - API - Utils Package This is related to stuff in `redbot.core.utils` Category: Core - Bot Class This is related to the `redbot.core.bot.Red` class. Category: Core - Command-line Interfaces This is related to Red's CLIs (redbot, redbot-launcher, redbot-setup). Category: Core - Other Internals This is related to core internals that don't have a dedicated label. Category: Core - RPC/ZMQ Related to provided (provisional) API for implementing RPC in cogs Category: Docs - Install Guides This is related to install guides. Category: Docs - Other This is related to documentation that doesn't have its dedicated label. Category: Meta This is related to the repository maintenance. Changelog Entry: Added Changelog entry for this PR has already been added to changelog PR. Type: Dependency Update Adding/updating/removing some of Red’s dependencies.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants