New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AnyUrl adds trailing slash #7186
Comments
somewhat related: pydantic/pydantic-settings#104 At this point, to my understanding, it seems its intended to add the slash. Is it? In that case, I am interested in whats the optimal way to remove this trailing slash, without forgoing the other validation (as seems to be the proposed solution in the related issue) |
This problem also becomes apparent in the url builder: from pydantic import AnyHttpUrl as AnyHttpUrl
AnyHttpUrl.build(
scheme="https",
host="google.com",
path="/search",
) produces: Is that the desired outcome? If yes, then does it mean that all Url builds we have in our codebase should now have a path that starts without a slash? |
The second case is unrelated. This is done by the URL rust crate, so it probably mandated by a rfc somewhere. |
Sorry, the second case is related. |
Another weird situation from pydantic import AnyHttpUrl as AnyHttpUrl
AnyHttpUrl.build(
scheme="https",
host="google.com/",
path="search",
)
|
That's technically valid and works, not sure we need to do anything special there. I think we might come up with a better solution for this long term but for now if you don't need a from typing import Annotated, Any, List
from pydantic_core import CoreSchema, core_schema
from pydantic import AfterValidator, BeforeValidator, GetCoreSchemaHandler, TypeAdapter
from pydantic.networks import AnyUrl
class Chain:
def __init__(self, validations: List[Any]) -> None:
self.validations = validations
def __get_pydantic_core_schema__(self, source_type: Any, handler: GetCoreSchemaHandler) -> CoreSchema:
return core_schema.chain_schema(
[
*(handler.generate_schema(v) for v in self.validations),
handler(source_type)
]
)
def strip_slash(value: str) -> str:
return value.rstrip('/')
MyAnyUrl = Annotated[str, AfterValidator(strip_slash), BeforeValidator(str), Chain([AnyUrl])]
ta = TypeAdapter(MyAnyUrl)
print(ta.validate_python('https://www.google.com'))
# https://www.google.com |
Thank you for posting this workaround, I am not 100% sure whats going on there, but to my understanding it seems that it creates a type However, I am still a bit baffled that it seems to be seen as not a problem that AnyUrl adds trailing For now, we have decided to roll back to pydantic v1. Will there be any way to keep using the pydantic v1 Url behaviour (i.e. not adding Lastly, just wanted to mention, I am a very happpy user of pydantic, and I love the work you all do, just this change of behaviour of Url is giving us a bit of a headache. |
Also finding this new behavior very annoying. In pydantic v1 urls inherited from str, so it can be easily passed to library codes, where they usually expects string representation of urls. But now we need to I'm not sure which variant right (old or new url), but this little slash causes very strange and hours of uncatchable bug fixing. FYI, I come up with this solution: from typing import Annotated
from pydantic import AfterValidator, AnyUrl, BaseModel
URL = Annotated[AnyUrl, AfterValidator(lambda x: str(x).rstrip("/"))]
class Model(BaseModel):
url: URL
print(Model(url="https://google.com").url)
# > https://google.com But sadly we cannot use this solution everywhere, because sometimes we need the last slash. Seems, that v2 url validator adding slashes only to from pydantic import AnyUrl, BaseModel
class Model(BaseModel):
url: AnyUrl
print(Model(url="https://google.com").url)
# > https://google.com/
print(Model(url="https://google.com/").url)
# > https://google.com/
print(Model(url="https://google.com/api").url)
# > https://google.com/api
print(Model(url="https://google.com/api/").url)
# > https://google.com/api/ Would be nice, if old behavior preserves, or like author suggested, we can choose behavior type. Thanks in advance! |
+1 this is a bug. In addition to a broken behavior when config URLs used to be used to append a path, like: There is also a violated principle of a least surprise when original value modified by a validation framework as when an URL is configured without slash is returned or saved in other components: DB, external services, they receive a new value which silently violated previous protocol until those actually start to execute it in a request. |
Stumbled upon this, as well. It's not possible to take an instance of I consider this a breaking change. Can we please add it to the migration docs at least? To my knowledge, the double slash will cause webservers to respond with a 404 as they expect a single slash as a path separator. def test_v2():
from pydantic import TypeAdapter, AnyHttpUrl
redirect_uri = TypeAdapter(AnyHttpUrl).validate_python(
"http://localhost:3000/api/v1"
)
assert (
str(
AnyHttpUrl.build(
scheme=redirect_uri.scheme,
host=redirect_uri.host,
port=redirect_uri.port,
path=redirect_uri.path,
)
)
== "http://localhost:3000/api/v1"
)
def test_v1():
from pydantic.v1 import parse_obj_as, AnyHttpUrl
redirect_uri = parse_obj_as(AnyHttpUrl, "http://localhost:3000/api/v1")
assert (
str(
AnyHttpUrl.build(
scheme=redirect_uri.scheme,
host=redirect_uri.host,
port=redirect_uri.port,
path=redirect_uri.path,
)
)
== "http://localhost:3000/api/v1"
) |
@adriangb The use case I have in mind is sending a redirect URI to a client during an OAuth2 dance. |
new behavior adds trailing `/` to end of links without a path: pydantic/pydantic#7186 (comment)
I had some validation checks because of this issue: the validator was testing for
|
Another less-than-ideal workaround (if it's acceptable to strip all trailing slashes): from typing import Annotated
from pydantic import AnyHttpUrl, AfterValidator, BaseModel, PlainValidator, TypeAdapter
AnyHttpUrlAdapter = TypeAdapter(AnyHttpUrl)
HttpUrlStr = Annotated[
str,
PlainValidator(lambda x: AnyHttpUrlAdapter.validate_strings(x)),
AfterValidator(lambda x: str(x).rstrip("/")),
]
class Model(BaseModel):
url: HttpUrlStr
url = Model(url="https://google.com").url
print(url)
# > https://google.com This also solves another somewhat related issue: #6395 URL = f"{BASE_URL}/api/v1" than this: URL = f"{BASE_URL}api/v1" |
I tried: service_url: HttpUrl
@field_validator("service_url", mode="after")
@classmethod
def service_url_remove_trailing_slash(cls, service_url: HttpUrl) -> str:
return str(service_url).rstrip("/") But then I got warnings:
How to keep the field as Edit:
And if I workaround this by using custom type based on |
This inconsistency is my biggest issue, especially when trying to add on paths or sub-paths as others have mentioned. If you can't be certain if a url has a trailing slash or not out of the box then we almost need an For now I guess I'll just do this: new_url = f"{str(URL).rstrip('/')}/path/here" |
We got this problem when we migrated from pydantic 1 as well. In addition to that, a lot of code was broken since AnyUrl is not accepted as str any more. This is our solution if it will be helpful to anyone: AnyUrlTypeAdapter = TypeAdapter(AnyUrl)
AnyUrlStr = Annotated[
str,
BeforeValidator(lambda value: AnyUrlTypeAdapter.validate_python(value) and value),
] So we currently just use our "wrapper" AnyUrlStr everywhere for compatibility. It also keeps the original string without any modification if it passes the validation. |
Wouldn't it be better to enhance the |
It's definitely a BUG, not "documentation issue".
|
Thanks everyone for commenting. I'd be happy to accept a PR to allow an opt-in change of behaviour. While i know many people here think the current behaviour is a bug, there will be others who are relying on it, so we can't change the default behavior in a minor release. Here's my proposal for what needs to change:
If anyone's interested, feel free to submit a PR. Might be a nice start for anyone interested in getting their hands dirty with rust as (AFAIK) it shouldn't be a too complicated change, but also not trivial. |
If no one is against it, I want to try! |
@tworedz, go for it.
@JensHeinrich, interesting idea - PR welcome. |
@samuelcolvin that's the problem with this ticket - it puts the problem wrong. The problem is not with trailing slashes, it's about leading path slash.
There are TWO problems:
|
For what it's worth, the outcome of pydantic/pydantic-core#1173 was that we think it's best to get the following upstream feature added for us to do Based on the upvotes on that issue, anyone who is interested in taking that on would make a lot of people happy! |
I keep running into this issue in a project I work on. The real problem isn't omission of slashes, it's the fact that hostnames get slashes added to them even if the user or developer did not specify them. So
But
I feel like I've been on a discussion about this somewhere already, but I keep running into this issue. If we had a way to make it so that we could use the type validation features but not add a slash to hostname (like it was in v1) that would be fantastic. @samuelcolvin would |
also having this issue with CORS and need to work around |
Initial Checks
Description
AnyURL now adds a trailing slash in validation of URLs that dont have a trailing slash. This caused CORS issues for us, because CORS doesnt allow trailing slash (I think, seems to be written here: https://www.w3.org/TR/2008/WD-access-control-20080214/)
In my opinion i see this as a bug because I dont see the option toggle this addition of trailing slash. Maybe its expected behaviour, please let me know then how I can turn it off.
Example Code
Python, Pydantic & OS Version
Selected Assignee: @dmontagu
The text was updated successfully, but these errors were encountered: