Skip to content

fix: fix ftp downloader in HpGFZ#51

Merged
sahiljhawar merged 1 commit intoGFZ:mainfrom
sahiljhawar:hp-requests
Feb 12, 2026
Merged

fix: fix ftp downloader in HpGFZ#51
sahiljhawar merged 1 commit intoGFZ:mainfrom
sahiljhawar:hp-requests

Conversation

@sahiljhawar
Copy link
Collaborator

No description provided.

Copilot AI review requested due to automatic review settings February 12, 2026 16:27
@sahiljhawar sahiljhawar merged commit cf2be31 into GFZ:main Feb 12, 2026
3 of 4 checks passed
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes HpGFZ data downloading by switching from an HTTP-based downloader to FTP, aligning the implementation with GFZ’s FTP endpoint.

Changes:

  • Replace requests.get(...) download logic with an ftplib.FTP-based implementation in HpGFZ._download.
  • Update HpGFZ download unit test to mock FTP interactions instead of HTTP requests.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
swvo/io/hp/gfz.py Implements FTP download logic for HpGFZ files (replacing the previous HTTP approach).
tests/io/hp/test_hp.py Updates the download/process test to mock FTP calls and assert expected FTP operations.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mock_ftp.retrbinary = mocker.Mock()
mocker.patch("swvo.io.hp.gfz.FTP", return_value=mock_ftp)

mocker.patch("shutil.rmtree")
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This patch won’t intercept the cleanup call: gfz.py imports rmtree as from shutil import rmtree and calls it as rmtree(...). Patch swvo.io.hp.gfz.rmtree instead of shutil.rmtree so the test reliably avoids deleting the temp directory.

Suggested change
mocker.patch("shutil.rmtree")
mocker.patch("swvo.io.hp.gfz.rmtree")

Copilot uses AI. Check for mistakes.
Comment on lines +154 to +161
ftp = FTP("ftp.gfz-potsdam.de")
ftp.login()
ftp.cwd("/pub/home/obs/Hpo/")

with open(local_path, "wb") as f:
ftp.retrbinary(f"RETR {filename}", f.write)

ftp.quit()
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ftp.quit() is only called on the success path. If login(), cwd(), or retrbinary() raises, the FTP connection can remain open. Use a context manager (with FTP(...) as ftp:) or a finally block that calls ftp.close()/ftp.quit() when the connection was created.

Suggested change
ftp = FTP("ftp.gfz-potsdam.de")
ftp.login()
ftp.cwd("/pub/home/obs/Hpo/")
with open(local_path, "wb") as f:
ftp.retrbinary(f"RETR {filename}", f.write)
ftp.quit()
with FTP("ftp.gfz-potsdam.de") as ftp:
ftp.login()
ftp.cwd("/pub/home/obs/Hpo/")
with open(local_path, "wb") as f:
ftp.retrbinary(f"RETR {filename}", f.write)

Copilot uses AI. Check for mistakes.
Comment on lines +162 to +164
except Exception as e:
logger.error(f"FTP download failed: {e}")
raise
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The except Exception as e: logger.error(...) pattern drops the traceback, which makes diagnosing download failures harder. Prefer logger.exception("FTP download failed") (or pass exc_info=True) and consider catching ftplib.all_errors instead of a bare Exception.

Copilot uses AI. Check for mistakes.
Comment on lines +154 to +157
ftp = FTP("ftp.gfz-potsdam.de")
ftp.login()
ftp.cwd("/pub/home/obs/Hpo/")

Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The FTP host and base directory are hard-coded even though URL = "ftp://ftp.gfz-potsdam.de/pub/home/obs/Hpo/" already exists on the class. Consider deriving the host/path from self.URL (or splitting it into FTP_HOST/FTP_BASEDIR constants) to avoid future drift between the constant and the actual download logic.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant