-
Notifications
You must be signed in to change notification settings - Fork 590
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: fix CI, use macOS-11 and remove pydantic #2339
Merged
Merged
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
8572c62
ci: use macOS-11
maartenbreddels d04dafd
fix: observed is not exported since traitlets 5.8
maartenbreddels 1fe217f
chore: trigger ci
maartenbreddels 01e2190
fix: use joblib < 1.3 for python 3.6
maartenbreddels e43ac6e
fix: make compatible with pydantic 2
maartenbreddels 6dd788f
fix: comparison with None
maartenbreddels b3137c4
ci: add pytz to conda yaml
maartenbreddels 4257de2
ci: let conda solve more
maartenbreddels 7afbb39
ci: try unpinning pip
maartenbreddels 29aa6cd
chore: pin pip again, and pin pydantic < 2
maartenbreddels 78fef18
ci: install pydantic 2 using conda
maartenbreddels 21ebf7f
ci: install more via conda-forge
maartenbreddels ff84b2e
feat: do not use pydantic for settings
maartenbreddels 4d94ac0
fix: no more pydantic
maartenbreddels eba84c3
fix: clean up code and remove breakpoint
maartenbreddels 3b79d90
ci: zzip 3.6.0 breaks semver/py3.7 support
maartenbreddels f97a3fb
fix: typing and conversion for fastapi/pydantic
maartenbreddels File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,16 +1,7 @@ | ||
| import os | ||
| from pydantic import BaseSettings | ||
| import vaex.utils | ||
| import pydantic | ||
|
|
||
|
|
||
| # in seperate module to avoid circular imports | ||
|
|
||
| class ConfigDefault: | ||
| env_file = ".env" | ||
| @classmethod | ||
| def customise_sources(cls, init_settings, env_settings, file_secret_settings): | ||
| return ( | ||
| env_settings, | ||
| file_secret_settings, | ||
| init_settings, # constructor argument last, since they come from global yaml file | ||
| ) | ||
| pass |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,130 @@ | ||
| import os | ||
| from pathlib import Path | ||
| from typing import Any, Optional | ||
|
|
||
| # similar API to pydantic/pydantic-settings but we prefer not to have a dependency on pydantic | ||
| # since we cannot be compatible with pydantic1 and 2 | ||
| # NOTE: not a public api | ||
|
|
||
|
|
||
| def _get_type(annotation): | ||
| check_optional_types = [str, int, float, bool, dict, list] | ||
| for check_type in check_optional_types: | ||
| if annotation == Optional[check_type]: | ||
| return check_type | ||
| if hasattr(annotation, "__origin__"): | ||
| if annotation.__origin__ == dict: | ||
| return dict | ||
| return annotation | ||
|
|
||
|
|
||
| class _Field: | ||
| def __init__(self, default=None, env=None, title=None, default_factory=None, gt=None, alias=None) -> None: | ||
| self.default = default | ||
| self.env = env | ||
| self.fullenv = None | ||
| self.title = title | ||
| self.annotation = None | ||
| self.default_factory = default_factory | ||
| self.gt = gt | ||
| self.alias = alias | ||
| self.field_info = self | ||
| self.extra = {"env_names": [env] if env else []} | ||
|
|
||
| def __set_name__(self, owner, name): | ||
| prefix = "SOLARA_" | ||
| config = getattr(owner, "Config") | ||
| if config: | ||
| prefix = getattr(config, "env_prefix", prefix).upper() | ||
| if hasattr(config, "fields"): | ||
| fields = config.fields | ||
| if name in fields: | ||
| self.alias = fields[name] | ||
| self.name = name | ||
| self.alias = self.alias or self.name | ||
| self.title = self.title or self.name | ||
| if self.env is None: | ||
| self.env = f"{prefix}{self.name.upper()}" | ||
| else: | ||
| self.env = self.env | ||
| self.annotation = owner.__annotations__.get(self.name) | ||
| assert self.annotation is not None, f"Field {self.name} must have a type annotation" | ||
| self.type_ = _get_type(self.annotation) | ||
|
|
||
| def __get__(self, instance, owner): | ||
| if instance is None: | ||
| return self | ||
| return instance._values[self.name] | ||
|
|
||
|
|
||
| def convert(annotation, value: str) -> Any: | ||
| check_optional_types = [str, int, float, bool, Path] | ||
| for check_type in check_optional_types: | ||
| if annotation == Optional[check_type]: | ||
| annotation = check_type | ||
| return convert(annotation, value) | ||
| if annotation == str: | ||
| return value | ||
| elif annotation == int: | ||
| return int(value) | ||
| elif annotation == float: | ||
| return float(value) | ||
| elif annotation == bool: | ||
| if value in ("True", "true", "1"): | ||
| return True | ||
| elif value in ("False", "false", "0"): | ||
| return False | ||
| else: | ||
| raise ValueError(f"Invalid boolean value {value}") | ||
| else: | ||
| # raise TypeError(f"Unsupported type {annotation}") | ||
| return annotation(value) | ||
|
|
||
|
|
||
| def Field(*args, **kwargs) -> Any: | ||
| return _Field(*args, **kwargs) | ||
|
|
||
|
|
||
| class BaseSettings: | ||
| __fields__: dict | ||
|
|
||
| def __init__(self, **kwargs) -> None: | ||
| cls = type(self) | ||
| self._values = {**kwargs} | ||
| keys = set([k.upper() for k in os.environ.keys()]) | ||
| for key, field in cls.__dict__.items(): | ||
| if key in kwargs: | ||
| continue | ||
| if isinstance(field, _Field): | ||
| value = field.default | ||
| if field.default_factory: | ||
| value = field.default_factory() | ||
|
|
||
| if field.env: | ||
| env_key = field.env.upper() | ||
| if env_key in keys: | ||
| # do a case-insensitive lookup | ||
| for env_var_cased in os.environ.keys(): | ||
| if env_key.upper() == env_var_cased.upper(): | ||
| value = convert(field.annotation, os.environ[env_var_cased]) | ||
| self._values[key] = value | ||
|
|
||
| def __init_subclass__(cls) -> None: | ||
| cls.__fields__ = {} | ||
| for key, field in cls.__dict__.items(): | ||
| if key.startswith("_"): | ||
| continue | ||
| if key == "Config": | ||
| continue | ||
| if not isinstance(field, _Field): | ||
| field = Field(field) | ||
| setattr(cls, key, field) | ||
| field.__set_name__(cls, key) | ||
| cls.__fields__[key] = field | ||
|
|
||
| def dict(self, by_alias=True): | ||
| values = self._values.copy() | ||
| for key, value in values.items(): | ||
| if isinstance(value, BaseSettings): | ||
| values[key] = value.dict(by_alias=by_alias) | ||
| return values |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
dunno if its valid syntax in the conda world, but without the marker, people will most likely start running into dephell in their more modern envs