Skip to content

Add digest auth option to downloader integration #147161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: dev
Choose a base branch
from

Conversation

mossymaker
Copy link

@mossymaker mossymaker commented Jun 19, 2025

Downloader with auth options

Proposed change

Adds a digest authentication option to the downloader integration. This resolves a feature request and unblocks users downloading URLs requiring digest auth.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.

To help with the load of incoming pull requests:

Copy link

@home-assistant home-assistant bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mossymaker

It seems you haven't yet signed a CLA. Please do so here.

Once you do that we will be able to review and accept this pull request.

Thanks!

@home-assistant
Copy link

Hey there @erwindouna, mind taking a look at this pull request as it has been labeled with an integration (downloader) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of downloader can trigger bot actions by commenting:

  • @home-assistant close Closes the pull request.
  • @home-assistant rename Awesome new title Renames the pull request.
  • @home-assistant reopen Reopen the pull request.
  • @home-assistant unassign downloader Removes the current integration label and assignees on the pull request, add the integration domain after the command.
  • @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
  • @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

Copy link
Contributor

@erwindouna erwindouna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your first contribution, @mossymaker! I have left a few initial feedback points. Also, do you think it's possible to extend the test for the new services? 😃

Comment on lines 49 to 51
username = service.data.get(ATTR_USERNAME) or ""

password = service.data.get(ATTR_PASSWORD) or ""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the or operators really needed here? :)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not strictly? Otherwise, the linter complains.

@mossymaker
Copy link
Author

@erwindouna thanks for reviewing!

do you think it's possible to extend the test for the new services?

I'm looking at this presently, but I'm unsure what the best pattern is here. There are a few examples of integrations tests that mock HTTP requests, but they're challenging for me to understand. I'm pretty new to python and Home Assistant dev. 😅

I think I want to mock a request to http://example.com/protected-resource, assert that the request has the expected auth headers, and respond with a 200. Maybe something like this:

# tests/components/downloader/test_action_digest_auth.py
"""Test downloader action."""

# ...

async def test_digest_auth(hass: HomeAssistant) -> None:
    """Test digest authentication in downloader action."""
    config_entry = MockConfigEntry(
        domain=DOMAIN,
        data={
            CONF_DOWNLOAD_DIR: "/test_dir",
        },
    )
    config_entry.add_to_hass(hass)
    with patch("os.path.isdir", return_value=True):
        assert await hass.config_entries.async_setup(config_entry.entry_id)

    with requests_mock.Mocker() as mock:
        mock.get(
            "http://example.com/protected-resource",
            status_code=HTTPStatus.OK,
        )

    response = await hass.services.async_call(
        DOMAIN,
        SERVICE_DOWNLOAD_FILE,
        {
            ATTR_URL: "http://example.com/protected-resource",
            ATTR_DIGEST_AUTH: True,
            ATTR_USERNAME: "test",
            ATTR_PASSWORD: "test"
        },
        blocking=True,
        return_response=True,
    )

    assert mock.called
    assert (
        "Authorization" in mock.last_request.headers
        and mock.last_request.headers["Authorization"] == "Digest..."
    )

req = requests.get(url, stream=True, timeout=10)
auth = HTTPDigestAuth(username, password) if digest_auth else None

req = requests.get(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wondering... is there a reason why we still use requests and not asynchronous aiohttp ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have that context, sorry! aiohttp appears to be widely used in the project, though. While it would be a nice upgrade, I want to keep the PR focused on the digest auth.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - for sure - definitely don't put it in this PR
But it would be nice to understand as digest auth was also recently added to aiohttp

@mossymaker mossymaker force-pushed the downloader-digest-auth branch from fd956f4 to a7e794f Compare June 20, 2025 18:09
@mossymaker
Copy link
Author

Updated to add tests. The second commit makes authentication type explicit.

@mossymaker mossymaker marked this pull request as ready for review June 20, 2025 19:46
@mossymaker mossymaker requested review from epenet and erwindouna June 20, 2025 19:47
@mossymaker mossymaker force-pushed the downloader-digest-auth branch from 22d0a9f to b1af5ba Compare June 23, 2025 15:40
@mossymaker mossymaker requested a review from epenet June 23, 2025 15:43
@mossymaker mossymaker force-pushed the downloader-digest-auth branch from b1af5ba to 10c78ae Compare June 24, 2025 06:06
@mossymaker
Copy link
Author

Updated to apply suggestions. @epenet does this exception look right? It seems like the translation is maybe not being found:

Traceback (most recent call last):
  File "/usr/local/lib/python3.13/threading.py", line 1041, in _bootstrap_inner
    self.run()
    ~~~~~~~~^^
  File "/usr/local/lib/python3.13/threading.py", line 992, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/homeassistant-core/homeassistant/components/downloader/services.py", line 71, in do_download
    raise ServiceValidationError(
        translation_domain=DOMAIN, translation_key="missing_credentials"
    )
homeassistant.exceptions.ServiceValidationError: <exception str() failed>

@epenet
Copy link
Contributor

epenet commented Jun 24, 2025

Updated to apply suggestions. @epenet does this exception look right? It seems like the translation is maybe not being found:

I've looked a bit more at this, and the issue is that the threading "swallows" the errors, so I think it would make sense for as much validation as possible to occur BEFORE starting the thread.

def download_file(service: ServiceCall) -> None:
    """Start thread to download file specified in the URL."""

    entry = service.hass.config_entries.async_loaded_entries(DOMAIN)[0]
    download_path = entry.data[CONF_DOWNLOAD_DIR]

    # Validate
    url: str = service.data[ATTR_URL]
    filename: str | None = service.data.get(ATTR_FILENAME)
    subdir = service.data.get(ATTR_SUBDIR)
    auth_type: str | None = service.data.get(ATTR_AUTH_TYPE)
    password: str | None = service.data.get(ATTR_AUTH_PASSWORD)
    username: str | None = service.data.get(ATTR_AUTH_USERNAME)

    auth: AuthBase | None = None
    if auth_type:
        if not username or not password:
            raise ServiceValidationError(
                translation_domain=DOMAIN, translation_key="missing_credentials"
            )
        if auth_type == HTTP_BASIC_AUTHENTICATION:
            auth = HTTPBasicAuth(username, password)
        elif auth_type == HTTP_DIGEST_AUTHENTICATION:
            auth = HTTPDigestAuth(username, password)

    if subdir:
        # Check the subdir
        try:
            raise_if_invalid_path(subdir)
        except ValueError:
            raise ServiceValidationError(
                translation_domain=DOMAIN, translation_key="invalid_subdir"
            )

    def do_download() -> None:

@mossymaker mossymaker force-pushed the downloader-digest-auth branch 2 times, most recently from cae0c79 to 92f48d7 Compare June 26, 2025 05:58
username: str | None = service.data.get(ATTR_AUTH_USERNAME)
password: str | None = service.data.get(ATTR_AUTH_PASSWORD)
subdir = service.data.get(ATTR_SUBDIR)
filename = service.data.get(ATTR_FILENAME)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
filename = service.data.get(ATTR_FILENAME)
target_filename = service.data.get(ATTR_FILENAME)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧐 filename is succinct and unambiguous.

Copy link
Contributor

@epenet epenet Jun 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but it would nice to avoid nonlocal, and to differentiate between the "original" filename from the service call, and the final filename which has been adjusted

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I'm still learning how python scope and closures work.

if subdir:
# Check the path
raise_if_invalid_path(subdir)
nonlocal filename
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
nonlocal filename
filename = target_filename

@@ -0,0 +1,80 @@
"""Test downloader action."""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rename as test_services.py to match the module being tested

Comment on lines 38 to 46
config_entry = MockConfigEntry(
domain=DOMAIN,
data={
CONF_DOWNLOAD_DIR: "/test_dir",
},
)
config_entry.add_to_hass(hass)
with patch("os.path.isdir", return_value=True):
await hass.config_entries.async_setup(config_entry.entry_id)
Copy link
Contributor

@epenet epenet Jun 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to move all this to a fixture in conftest.py, so it can be reused in other/future tests

@pytest.fixture
async def loaded_config_entry(hass: HomeAssistant) -> MockConfigEntry:
    """Loaded config entry."""
    config_entry = MockConfigEntry(
        domain=DOMAIN,
        data={
            CONF_DOWNLOAD_DIR: "/test_dir",
        },
    )
    config_entry.add_to_hass(hass)

    with patch("os.path.isdir", return_value=True):
        await hass.config_entries.async_setup(config_entry.entry_id)

    return config_entry

return "Authorization" in request.headers


async def test_digest_auth(hass: HomeAssistant) -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
async def test_digest_auth(hass: HomeAssistant) -> None:
@pytest.mark.usefixtures("loaded_config_entry")
async def test_digest_auth(hass: HomeAssistant) -> None:

@mossymaker mossymaker force-pushed the downloader-digest-auth branch from 92f48d7 to 5f620ce Compare June 26, 2025 16:25

assert hass.services.has_service(DOMAIN, SERVICE_DOWNLOAD_FILE)
assert config_entry.state is ConfigEntryState.LOADED
assert loaded_config_entry.state is ConfigEntryState.LOADED
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assert hass.services.has_service(DOMAIN, SERVICE_DOWNLOAD_FILE) and assert loaded_config_entry.state is ConfigEntryState.LOADED implies the previous assert await hass.config_entries.async_setup(config_entry.entry_id), correct?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explicit is better than implicit.
I think it makes sense to keep has_service assertion.

@mossymaker mossymaker requested a review from epenet June 26, 2025 16:37
"description": "Username to use with authentication."
},
"auth_password": {
"name": "Authentication Password",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"name": "Authentication Password",
"name": "Authentication password",

"description": "Authentication type to use."
},
"auth_username": {
"name": "Authentication Username",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"name": "Authentication Username",
"name": "Authentication username",

@@ -21,6 +21,18 @@
"name": "[%key:common::config_flow::data::url%]",
"description": "The URL of the file to download."
},
"auth_type": {
"name": "Authentication Type",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"name": "Authentication Type",
"name": "Authentication type",

Comment on lines 10 to 11
@pytest.mark.usefixtures("loaded_config_entry")
async def test_initialization(hass: HomeAssistant, loaded_config_entry) -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You only need to use pytest.mark.usefixtures if it's not in the arguments.

Suggested change
@pytest.mark.usefixtures("loaded_config_entry")
async def test_initialization(hass: HomeAssistant, loaded_config_entry) -> None:
from tests.common import MockConfigEntry
async def test_initialization(hass: HomeAssistant, loaded_config_entry: MockConfigEntry) -> None:



@pytest.mark.usefixtures("loaded_config_entry")
async def test_digest_auth(hass: HomeAssistant) -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be nicer with the fixture, then you can un-indent most of the code below:

Suggested change
async def test_digest_auth(hass: HomeAssistant) -> None:
async def test_digest_auth(hass: HomeAssistant, requests_mock: Mocker) -> None:

from unittest.mock import mock_open, patch

import pytest
import requests_mock
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
import requests_mock
from requests_mock import Mocker

Comment on lines 37 to 38
with requests_mock.Mocker() as mock:
mock.get(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
with requests_mock.Mocker() as mock:
mock.get(
requests_mock.get(

status_code=HTTPStatus.UNAUTHORIZED,
)

mock.get(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
mock.get(
requests_mock.get(



@pytest.mark.usefixtures("loaded_config_entry")
async def test_digest_auth(hass: HomeAssistant) -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a few extra tests?

  • One for "invalid subdir"
  • One for "invalid credentials"
  • One for "no-auth"
  • One for "basic-auth"

Copy link
Contributor

@epenet epenet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the main code looks good...
Just a few more tweaks on the tests needed I think

@frenck frenck requested review from Copilot and removed request for erwindouna June 27, 2025 06:38
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds support for HTTP digest authentication (and basic auth) to the downloader integration by introducing new service parameters and handling.

  • Implemented auth_type, auth_username, and auth_password in the service handler and schema.
  • Extended constants, translation strings, and selectors for authentication options.
  • Refactored tests with a shared loaded_config_entry fixture and added a digest auth test.

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/components/downloader/test_services.py Added test for HTTP digest authentication
tests/components/downloader/test_init.py Refactored initialization test to use shared fixture
tests/components/downloader/conftest.py Introduced loaded_config_entry fixture
homeassistant/components/downloader/strings.json Added translation strings for auth fields and exceptions
homeassistant/components/downloader/services.yaml Added selectors for auth_type, auth_username, and auth_password
homeassistant/components/downloader/services.py Implemented authentication handling and validation
homeassistant/components/downloader/const.py Added new ATTR_AUTH_* constants
Comments suppressed due to low confidence (4)

tests/components/downloader/test_services.py:69

  • Add a similar test for HTTP basic authentication (using HTTP_BASIC_AUTHENTICATION) to verify that the Authorization header is correctly formed for basic auth.
        )

homeassistant/components/downloader/services.py:54

  • Add a test to ensure that omitting auth_username or auth_password when auth_type is provided raises the expected ServiceValidationError with the missing_credentials key.
        if not username or not password:

homeassistant/components/downloader/services.py:66

  • Add a test for invalid subdir values to confirm that a ServiceValidationError with key invalid_subdir is raised when subdir fails validation.
            raise_if_invalid_path(subdir)

tests/components/downloader/test_init.py:10

  • The usefixtures decorator is redundant when loaded_config_entry is already declared as an argument; you can remove it to simplify the test signature.
@pytest.mark.usefixtures("loaded_config_entry")

Also moves some validation out of the thread to avoid swallowing exceptions.
@mossymaker mossymaker force-pushed the downloader-digest-auth branch from 9e77221 to 1ac9819 Compare June 29, 2025 18:05
@mossymaker mossymaker requested a review from epenet June 29, 2025 18:07
@mossymaker
Copy link
Author

Updated to apply suggestions and add more tests.

@epenet
Copy link
Contributor

epenet commented Jun 30, 2025

Updated to apply suggestions and add more tests.

Please do not squash commits... it makes code reviews harder as we need to review the whole code each time instead of just the latest changes.

Comment on lines 48 to 50
subdir = service.data.get(ATTR_SUBDIR)
target_filename = service.data.get(ATTR_FILENAME)
overwrite = service.data.get(ATTR_OVERWRITE)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add type hints. Also, overwrite is optional for the user, but not in the resulting dict as it has a default value:

Suggested change
subdir = service.data.get(ATTR_SUBDIR)
target_filename = service.data.get(ATTR_FILENAME)
overwrite = service.data.get(ATTR_OVERWRITE)
subdir: str | None = service.data.get(ATTR_SUBDIR)
target_filename: str | None = service.data.get(ATTR_FILENAME)
overwrite: bool = service.data[ATTR_OVERWRITE]

Comment on lines 31 to 34
@pytest.fixture
def auth_requests_mock(requests_mock: Mocker) -> Mocker:
"""Set up request mock."""
return requests_mock
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should not be needed... just pass the global requests_mock when needed.

Suggested change
@pytest.fixture
def auth_requests_mock(requests_mock: Mocker) -> Mocker:
"""Set up request mock."""
return requests_mock

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe requests_mock is a global; do you find an example of the project using it that way?

So anywhere I want to use requests_mock, I'll need to create an instance of Mocker. Isn't avoiding that the point of using a fixture?

with requests_mock.Mocker() as auth_requests_mock:


assert hass.services.has_service(DOMAIN, SERVICE_DOWNLOAD_FILE)
assert config_entry.state is ConfigEntryState.LOADED
assert loaded_config_entry.state is ConfigEntryState.LOADED
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explicit is better than implicit.
I think it makes sense to keep has_service assertion.

from homeassistant.exceptions import ServiceValidationError


def digest_challenge_matcher(request):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we have type hints, and maybe rename now it is being reused?

Suggested change
def digest_challenge_matcher(request):
def no_authorization_request_matcher(request: XYZ) -> bool:

return "Authorization" not in request.headers


def digest_response_matcher(request):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we have type hints here also, and maybe rename now it is being reused?

Suggested change
def digest_response_matcher(request):
def authorization_request_matcher(request: XYZ) -> bool:

Comment on lines 67 to 71
requests_mock.get(
TEST_URL,
status_code=HTTPStatus.OK,
)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
requests_mock.get(
TEST_URL,
status_code=HTTPStatus.OK,
)

Comment on lines 72 to 78
with (
pytest.raises(
ServiceValidationError,
match="Subdirectory must be valid",
),
patch("builtins.open", new=mock_open(), create=True),
):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
with (
pytest.raises(
ServiceValidationError,
match="Subdirectory must be valid",
),
patch("builtins.open", new=mock_open(), create=True),
):
with pytest.raises(ServiceValidationError) as exc_info:

await hass.services.async_call(
DOMAIN,
SERVICE_DOWNLOAD_FILE,
{ATTR_URL: TEST_URL, ATTR_SUBDIR: "~"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{ATTR_URL: TEST_URL, ATTR_SUBDIR: "~"},
{ATTR_URL: TEST_URL, **extra_args},

blocking=True,
)

assert not requests_mock.called
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
assert not requests_mock.called
assert exc_info.value.translation_domain == DOMAIN
assert exc_info.value.translation_key == error_code

Comment on lines 89 to 91
@pytest.mark.usefixtures("loaded_config_entry")
async def test_invalid_credentials(hass: HomeAssistant, requests_mock: Mocker) -> None:
"""Test invalid credentials in downloader action."""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you use parametrize above, this test is no longer necessary

@mossymaker mossymaker requested a review from epenet July 4, 2025 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants