-
-
Notifications
You must be signed in to change notification settings - Fork 34.2k
Add digest auth option to downloader integration #147161
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: dev
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @mossymaker
It seems you haven't yet signed a CLA. Please do so here.
Once you do that we will be able to review and accept this pull request.
Thanks!
Hey there @erwindouna, mind taking a look at this pull request as it has been labeled with an integration ( Code owner commandsCode owners of
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your first contribution, @mossymaker! I have left a few initial feedback points. Also, do you think it's possible to extend the test for the new services? 😃
username = service.data.get(ATTR_USERNAME) or "" | ||
|
||
password = service.data.get(ATTR_PASSWORD) or "" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are the or operators really needed here? :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not strictly? Otherwise, the linter complains.
@erwindouna thanks for reviewing!
I'm looking at this presently, but I'm unsure what the best pattern is here. There are a few examples of integrations tests that mock HTTP requests, but they're challenging for me to understand. I'm pretty new to python and Home Assistant dev. 😅 I think I want to mock a request to # tests/components/downloader/test_action_digest_auth.py
"""Test downloader action."""
# ...
async def test_digest_auth(hass: HomeAssistant) -> None:
"""Test digest authentication in downloader action."""
config_entry = MockConfigEntry(
domain=DOMAIN,
data={
CONF_DOWNLOAD_DIR: "/test_dir",
},
)
config_entry.add_to_hass(hass)
with patch("os.path.isdir", return_value=True):
assert await hass.config_entries.async_setup(config_entry.entry_id)
with requests_mock.Mocker() as mock:
mock.get(
"http://example.com/protected-resource",
status_code=HTTPStatus.OK,
)
response = await hass.services.async_call(
DOMAIN,
SERVICE_DOWNLOAD_FILE,
{
ATTR_URL: "http://example.com/protected-resource",
ATTR_DIGEST_AUTH: True,
ATTR_USERNAME: "test",
ATTR_PASSWORD: "test"
},
blocking=True,
return_response=True,
)
assert mock.called
assert (
"Authorization" in mock.last_request.headers
and mock.last_request.headers["Authorization"] == "Digest..."
) |
req = requests.get(url, stream=True, timeout=10) | ||
auth = HTTPDigestAuth(username, password) if digest_auth else None | ||
|
||
req = requests.get( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just wondering... is there a reason why we still use requests
and not asynchronous aiohttp
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't have that context, sorry! aiohttp appears to be widely used in the project, though. While it would be a nice upgrade, I want to keep the PR focused on the digest auth.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes - for sure - definitely don't put it in this PR
But it would be nice to understand as digest auth was also recently added to aiohttp
fd956f4
to
a7e794f
Compare
Updated to add tests. The second commit makes authentication type explicit. |
22d0a9f
to
b1af5ba
Compare
b1af5ba
to
10c78ae
Compare
Updated to apply suggestions. @epenet does this exception look right? It seems like the translation is maybe not being found:
|
I've looked a bit more at this, and the issue is that the threading "swallows" the errors, so I think it would make sense for as much validation as possible to occur BEFORE starting the thread. def download_file(service: ServiceCall) -> None:
"""Start thread to download file specified in the URL."""
entry = service.hass.config_entries.async_loaded_entries(DOMAIN)[0]
download_path = entry.data[CONF_DOWNLOAD_DIR]
# Validate
url: str = service.data[ATTR_URL]
filename: str | None = service.data.get(ATTR_FILENAME)
subdir = service.data.get(ATTR_SUBDIR)
auth_type: str | None = service.data.get(ATTR_AUTH_TYPE)
password: str | None = service.data.get(ATTR_AUTH_PASSWORD)
username: str | None = service.data.get(ATTR_AUTH_USERNAME)
auth: AuthBase | None = None
if auth_type:
if not username or not password:
raise ServiceValidationError(
translation_domain=DOMAIN, translation_key="missing_credentials"
)
if auth_type == HTTP_BASIC_AUTHENTICATION:
auth = HTTPBasicAuth(username, password)
elif auth_type == HTTP_DIGEST_AUTHENTICATION:
auth = HTTPDigestAuth(username, password)
if subdir:
# Check the subdir
try:
raise_if_invalid_path(subdir)
except ValueError:
raise ServiceValidationError(
translation_domain=DOMAIN, translation_key="invalid_subdir"
)
def do_download() -> None: |
cae0c79
to
92f48d7
Compare
username: str | None = service.data.get(ATTR_AUTH_USERNAME) | ||
password: str | None = service.data.get(ATTR_AUTH_PASSWORD) | ||
subdir = service.data.get(ATTR_SUBDIR) | ||
filename = service.data.get(ATTR_FILENAME) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
filename = service.data.get(ATTR_FILENAME) | |
target_filename = service.data.get(ATTR_FILENAME) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧐 filename
is succinct and unambiguous.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, but it would nice to avoid nonlocal
, and to differentiate between the "original" filename from the service call, and the final filename which has been adjusted
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! I'm still learning how python scope and closures work.
if subdir: | ||
# Check the path | ||
raise_if_invalid_path(subdir) | ||
nonlocal filename |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nonlocal filename | |
filename = target_filename |
@@ -0,0 +1,80 @@ | |||
"""Test downloader action.""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please rename as test_services.py
to match the module being tested
config_entry = MockConfigEntry( | ||
domain=DOMAIN, | ||
data={ | ||
CONF_DOWNLOAD_DIR: "/test_dir", | ||
}, | ||
) | ||
config_entry.add_to_hass(hass) | ||
with patch("os.path.isdir", return_value=True): | ||
await hass.config_entries.async_setup(config_entry.entry_id) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest to move all this to a fixture in conftest.py
, so it can be reused in other/future tests
@pytest.fixture
async def loaded_config_entry(hass: HomeAssistant) -> MockConfigEntry:
"""Loaded config entry."""
config_entry = MockConfigEntry(
domain=DOMAIN,
data={
CONF_DOWNLOAD_DIR: "/test_dir",
},
)
config_entry.add_to_hass(hass)
with patch("os.path.isdir", return_value=True):
await hass.config_entries.async_setup(config_entry.entry_id)
return config_entry
return "Authorization" in request.headers | ||
|
||
|
||
async def test_digest_auth(hass: HomeAssistant) -> None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
async def test_digest_auth(hass: HomeAssistant) -> None: | |
@pytest.mark.usefixtures("loaded_config_entry") | |
async def test_digest_auth(hass: HomeAssistant) -> None: |
92f48d7
to
5f620ce
Compare
|
||
assert hass.services.has_service(DOMAIN, SERVICE_DOWNLOAD_FILE) | ||
assert config_entry.state is ConfigEntryState.LOADED | ||
assert loaded_config_entry.state is ConfigEntryState.LOADED |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
assert hass.services.has_service(DOMAIN, SERVICE_DOWNLOAD_FILE)
and assert loaded_config_entry.state is ConfigEntryState.LOADED
implies the previous assert await hass.config_entries.async_setup(config_entry.entry_id)
, correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Explicit is better than implicit.
I think it makes sense to keep has_service
assertion.
"description": "Username to use with authentication." | ||
}, | ||
"auth_password": { | ||
"name": "Authentication Password", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"name": "Authentication Password", | |
"name": "Authentication password", |
"description": "Authentication type to use." | ||
}, | ||
"auth_username": { | ||
"name": "Authentication Username", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"name": "Authentication Username", | |
"name": "Authentication username", |
@@ -21,6 +21,18 @@ | |||
"name": "[%key:common::config_flow::data::url%]", | |||
"description": "The URL of the file to download." | |||
}, | |||
"auth_type": { | |||
"name": "Authentication Type", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"name": "Authentication Type", | |
"name": "Authentication type", |
@pytest.mark.usefixtures("loaded_config_entry") | ||
async def test_initialization(hass: HomeAssistant, loaded_config_entry) -> None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You only need to use pytest.mark.usefixtures
if it's not in the arguments.
@pytest.mark.usefixtures("loaded_config_entry") | |
async def test_initialization(hass: HomeAssistant, loaded_config_entry) -> None: | |
from tests.common import MockConfigEntry | |
async def test_initialization(hass: HomeAssistant, loaded_config_entry: MockConfigEntry) -> None: |
|
||
|
||
@pytest.mark.usefixtures("loaded_config_entry") | ||
async def test_digest_auth(hass: HomeAssistant) -> None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would be nicer with the fixture, then you can un-indent most of the code below:
async def test_digest_auth(hass: HomeAssistant) -> None: | |
async def test_digest_auth(hass: HomeAssistant, requests_mock: Mocker) -> None: |
from unittest.mock import mock_open, patch | ||
|
||
import pytest | ||
import requests_mock |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
import requests_mock | |
from requests_mock import Mocker |
with requests_mock.Mocker() as mock: | ||
mock.get( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
with requests_mock.Mocker() as mock: | |
mock.get( | |
requests_mock.get( |
status_code=HTTPStatus.UNAUTHORIZED, | ||
) | ||
|
||
mock.get( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mock.get( | |
requests_mock.get( |
|
||
|
||
@pytest.mark.usefixtures("loaded_config_entry") | ||
async def test_digest_auth(hass: HomeAssistant) -> None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a few extra tests?
- One for "invalid subdir"
- One for "invalid credentials"
- One for "no-auth"
- One for "basic-auth"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the main code looks good...
Just a few more tweaks on the tests needed I think
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Adds support for HTTP digest authentication (and basic auth) to the downloader integration by introducing new service parameters and handling.
- Implemented
auth_type
,auth_username
, andauth_password
in the service handler and schema. - Extended constants, translation strings, and selectors for authentication options.
- Refactored tests with a shared
loaded_config_entry
fixture and added a digest auth test.
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
File | Description |
---|---|
tests/components/downloader/test_services.py | Added test for HTTP digest authentication |
tests/components/downloader/test_init.py | Refactored initialization test to use shared fixture |
tests/components/downloader/conftest.py | Introduced loaded_config_entry fixture |
homeassistant/components/downloader/strings.json | Added translation strings for auth fields and exceptions |
homeassistant/components/downloader/services.yaml | Added selectors for auth_type , auth_username , and auth_password |
homeassistant/components/downloader/services.py | Implemented authentication handling and validation |
homeassistant/components/downloader/const.py | Added new ATTR_AUTH_* constants |
Comments suppressed due to low confidence (4)
tests/components/downloader/test_services.py:69
- Add a similar test for HTTP basic authentication (using
HTTP_BASIC_AUTHENTICATION
) to verify that theAuthorization
header is correctly formed for basic auth.
)
homeassistant/components/downloader/services.py:54
- Add a test to ensure that omitting
auth_username
orauth_password
whenauth_type
is provided raises the expectedServiceValidationError
with themissing_credentials
key.
if not username or not password:
homeassistant/components/downloader/services.py:66
- Add a test for invalid
subdir
values to confirm that aServiceValidationError
with keyinvalid_subdir
is raised whensubdir
fails validation.
raise_if_invalid_path(subdir)
tests/components/downloader/test_init.py:10
- The
usefixtures
decorator is redundant whenloaded_config_entry
is already declared as an argument; you can remove it to simplify the test signature.
@pytest.mark.usefixtures("loaded_config_entry")
Also moves some validation out of the thread to avoid swallowing exceptions.
9e77221
to
1ac9819
Compare
Updated to apply suggestions and add more tests. |
Please do not squash commits... it makes code reviews harder as we need to review the whole code each time instead of just the latest changes. |
subdir = service.data.get(ATTR_SUBDIR) | ||
target_filename = service.data.get(ATTR_FILENAME) | ||
overwrite = service.data.get(ATTR_OVERWRITE) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add type hints. Also, overwrite is optional for the user, but not in the resulting dict as it has a default value:
subdir = service.data.get(ATTR_SUBDIR) | |
target_filename = service.data.get(ATTR_FILENAME) | |
overwrite = service.data.get(ATTR_OVERWRITE) | |
subdir: str | None = service.data.get(ATTR_SUBDIR) | |
target_filename: str | None = service.data.get(ATTR_FILENAME) | |
overwrite: bool = service.data[ATTR_OVERWRITE] |
@pytest.fixture | ||
def auth_requests_mock(requests_mock: Mocker) -> Mocker: | ||
"""Set up request mock.""" | ||
return requests_mock |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should not be needed... just pass the global requests_mock
when needed.
@pytest.fixture | |
def auth_requests_mock(requests_mock: Mocker) -> Mocker: | |
"""Set up request mock.""" | |
return requests_mock |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't believe requests_mock
is a global; do you find an example of the project using it that way?
So anywhere I want to use requests_mock
, I'll need to create an instance of Mocker. Isn't avoiding that the point of using a fixture?
with requests_mock.Mocker() as auth_requests_mock:
|
||
assert hass.services.has_service(DOMAIN, SERVICE_DOWNLOAD_FILE) | ||
assert config_entry.state is ConfigEntryState.LOADED | ||
assert loaded_config_entry.state is ConfigEntryState.LOADED |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Explicit is better than implicit.
I think it makes sense to keep has_service
assertion.
from homeassistant.exceptions import ServiceValidationError | ||
|
||
|
||
def digest_challenge_matcher(request): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we have type hints, and maybe rename now it is being reused?
def digest_challenge_matcher(request): | |
def no_authorization_request_matcher(request: XYZ) -> bool: |
return "Authorization" not in request.headers | ||
|
||
|
||
def digest_response_matcher(request): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we have type hints here also, and maybe rename now it is being reused?
def digest_response_matcher(request): | |
def authorization_request_matcher(request: XYZ) -> bool: |
requests_mock.get( | ||
TEST_URL, | ||
status_code=HTTPStatus.OK, | ||
) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
requests_mock.get( | |
TEST_URL, | |
status_code=HTTPStatus.OK, | |
) |
with ( | ||
pytest.raises( | ||
ServiceValidationError, | ||
match="Subdirectory must be valid", | ||
), | ||
patch("builtins.open", new=mock_open(), create=True), | ||
): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
with ( | |
pytest.raises( | |
ServiceValidationError, | |
match="Subdirectory must be valid", | |
), | |
patch("builtins.open", new=mock_open(), create=True), | |
): | |
with pytest.raises(ServiceValidationError) as exc_info: |
await hass.services.async_call( | ||
DOMAIN, | ||
SERVICE_DOWNLOAD_FILE, | ||
{ATTR_URL: TEST_URL, ATTR_SUBDIR: "~"}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
{ATTR_URL: TEST_URL, ATTR_SUBDIR: "~"}, | |
{ATTR_URL: TEST_URL, **extra_args}, |
blocking=True, | ||
) | ||
|
||
assert not requests_mock.called |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
assert not requests_mock.called | |
assert exc_info.value.translation_domain == DOMAIN | |
assert exc_info.value.translation_key == error_code |
@pytest.mark.usefixtures("loaded_config_entry") | ||
async def test_invalid_credentials(hass: HomeAssistant, requests_mock: Mocker) -> None: | ||
"""Test invalid credentials in downloader action.""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you use parametrize above, this test is no longer necessary
Proposed change
Adds a digest authentication option to the downloader integration. This resolves a feature request and unblocks users downloading URLs requiring digest auth.
Type of change
Additional information
Checklist
ruff format homeassistant tests
)If user exposed functionality or configuration variables are added/changed:
If the code communicates with devices, web services, or third-party tools:
Updated and included derived files by running:
python3 -m script.hassfest
.requirements_all.txt
.Updated by running
python3 -m script.gen_requirements_all
.To help with the load of incoming pull requests: