Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redo package download #234

Merged
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
82085fe
Added proper DeprecationWarning
teutoburg Jun 14, 2023
41fbf0b
Catch HTTPErrors from both urllib and urllib3
teutoburg Jun 15, 2023
40d86a9
Migrate from os.path to pathlib.Path
teutoburg Jun 15, 2023
5c52124
Formatting
teutoburg Jun 15, 2023
b6311c0
Variable names were still from deprecated function, now undefined
teutoburg Jun 15, 2023
7fca2f6
Lazy formatting for logging
teutoburg Jun 15, 2023
4a23f18
Properly do except clauses
teutoburg Jun 15, 2023
21a1786
Refactor to check and raise before, include release info in error msg
teutoburg Jun 15, 2023
abad453
Add type hints on the go
teutoburg Jun 15, 2023
7a79439
Improve function structure
teutoburg Jun 15, 2023
52f953b
Add functions get_all_packages_on_server & get_server_folder_package_…
teutoburg Jun 15, 2023
14d9aa8
Make better use of beautifulsoup api to reduce need for filtering
teutoburg Jun 18, 2023
c561fa2
Raise Error when no packages found in given dir
teutoburg Jun 18, 2023
06b9728
docstring
teutoburg Jun 18, 2023
a40f5ea
Refactor function for less complex structure
teutoburg Jun 19, 2023
2b1f215
Fix messed up string tuple
teutoburg Jun 19, 2023
d6bd60a
Add new functions, deprecate old API
teutoburg Jun 19, 2023
fac4e15
Implementing new download with retry incl cooldown and improved cache
teutoburg Jun 20, 2023
fcdce5e
Don't raise ValueError from KeyboardInterrupt
teutoburg Jun 20, 2023
2956070
Refactoring
teutoburg Jun 20, 2023
18ef1e5
Progress on the progress bar, and better server error handling
teutoburg Jun 21, 2023
858e89b
Refactoring for better error handling
teutoburg Jun 21, 2023
aad0941
Add deprecation warning to obsolete function
teutoburg Jun 21, 2023
e6986aa
Move github-related functions to separate submodule for better structure
teutoburg Jun 21, 2023
de258f5
Move example package functions to separate submodule for better struc…
teutoburg Jun 21, 2023
1ba2027
Update dependencies in toml file
teutoburg Jun 21, 2023
3495c32
Forgot a dependency
teutoburg Jun 21, 2023
20a0da7
Fix imports
teutoburg Jun 21, 2023
c7a7025
Attempt to fix imports finally
teutoburg Jun 21, 2023
9b0ee24
Add support for Python 3.8, because we have to :(
teutoburg Jun 21, 2023
8dd9e5e
Fix failing tests caused by API change
teutoburg Jun 21, 2023
44f1dab
Add more test to appease the allmighty lords of the holy Codecov
teutoburg Jun 21, 2023
722ac22
Actually call all tests test, add another one
teutoburg Jun 21, 2023
44759cf
Actually catch 404 from server. This is why we listen to Codecov
teutoburg Jun 21, 2023
c135649
Added @pytest.mark.webtest to all tests using internet connection
teutoburg Jun 21, 2023
80d8ba8
Make custom composite type private
teutoburg Jun 22, 2023
f93898f
Isolate tqdm kwargs function
teutoburg Jun 22, 2023
e8e434f
Use proper (double) with block instead of try/finally
teutoburg Jun 22, 2023
ded0cc3
Add messages informing the user about the download status
teutoburg Jun 22, 2023
ff5f664
That was too fancy for Python 3.8 ...
teutoburg Jun 22, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ dependencies = [
"beautifulsoup4>=4.4",
"lxml",
"pyyaml>5.1",
"more-itertools>=9.0",
"tqdm>=4.64",
"requests-cache>1.0",

"synphot>=0.1.3",
"skycalc_ipy>=0.1.3",
Expand Down
4 changes: 3 additions & 1 deletion scopesim/server/OLD_database.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

from scopesim import rc

from warnings import warn

Check warning on line 16 in scopesim/server/OLD_database.py

View check run for this annotation

Codecov / codecov/patch

scopesim/server/OLD_database.py#L16

Added line #L16 was not covered by tests

def get_local_packages(path):
"""
Expand All @@ -29,6 +30,8 @@
Names of packages on the local disk

"""
warn("Function Depreciated --> please use scopesim.download_package-s-()",

Check warning on line 33 in scopesim/server/OLD_database.py

View check run for this annotation

Codecov / codecov/patch

scopesim/server/OLD_database.py#L33

Added line #L33 was not covered by tests
DeprecationWarning, stacklevel=2)
dirnames = os.listdir(path)
pkgs = []

Expand Down Expand Up @@ -166,4 +169,3 @@
save_path = os.path.abspath(save_path)

return save_path

4 changes: 2 additions & 2 deletions scopesim/server/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from .database import (download_packages,
list_packages,
download_example_data,
list_example_data)
get_all_packages_on_server)
from .example_data_utils import download_example_data, list_example_data
Loading