-
Notifications
You must be signed in to change notification settings - Fork 351
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ImportCache of CanonicalizeResult is broken for relative imports handled by any importer that is not a baseImporter #2208
Comments
After reviewing the code, I think the issue is in the API design that we are expecting a single dart-sass/lib/src/import_cache.dart Lines 156 to 174 in 783c248
In these lines, the I can see two issues here:
In my opinion, I think the idea of |
Relative paths are resolved using the importer that loaded the current module (this is what the baseImporter is, which is why there is only one). When going through other importers, URLs are not meant to be considered relative ones. To me, the test you added in sass-spec does not respect the rules than were defined in the spec for importers (and I think it is impossible to define the same importer in the Dart API). @nex3 reading the changelog in https://github.com/sass/dart-sass/releases/tag/1.68.0, it looks like filesystem importers have a special behavior in the JS API which has no equivalent in the Dart API. Is it expected ? It looks like this is the root cause of this caching issue because it allows JS filesystem importers to resolve the same URL to different canonical URLs. |
This is exactly where the bug is. We are passing relative urls to The fundamental issue I see is the concept of resolving relative url and canonicalizing a url is not necessarily the same: a resolved url from base url is only meant to be an absolute url, and not necessarily a canonical url. This makes it utterly confusing to ”canonicalize” a relative url with three different patterns in the current API:
In my experience, these mixed patterns make it extremely difficult to write generic custom importer or importers that correctly handles all the cases (and it’s not well documented). As a user of the API, I end up finding out that not relying on the base importer which may or may not resolve the url, and only using importers to handle both url resolution and canonization, is much more flexible, easier and consistent.
From thread I’ve read, it appears that the initial design is exactly as you described, that the base importer is meant to take a resolved url, and user can further canonicalize a resolved url, and then importers are meant to handle any unresolved url, which is “assumed” to be absolute, but not enforced. Unfortunately, the community missed the initial RFC and when the API is already out for a long time, webpack team comes and say they need the ability to customize the “resolving” of the url, rather than just given an already resolved url to canonicalize. This was accepted and that’s why we have containingUrl. This change means the previous restrictions in spec, if any, should have been lifted, that what I’m doing (customizing the resolution of relative url) is completely legal as long as my implementation is deterministic. One of the major design objectives of new module system and new API is deterministic resolution and canonization. The cache in question is based on the assumption of that importers are deterministic and always returns the canonicalized absolute url with the same input. For any single importer it should be deterministic given the same |
Sorry to chime in a bit late here, I was out sick for a chunk of last week. I do think there are bugs here, but I don't think the fix in #2209 is exactly correct. Let me give some background that may help explain the way things work today. History LessonThe Old DaysWhen we first added support for importers to Sass, we didn't have a rigorous understanding or specification of exactly what the strings after an Given that, all In this system, we still wanted to enforce the idea that relative loads always take precedence over loads from the load path. But because we weren't officially working with URLs, the only way we were able to do so was by fiat. We just had a separate Dawn of Dart SassWhen I was implementing Dart Sass, especially knowing that But we still had to deal with the fact that all the existing importers (and users!) thought in load-path terms, where they just used relative URLs to load from their importers. We couldn't just say "all URLs going into importers are absolute from the start". And we also needed to ensure that relative loads would take precedence over these load-path-style loads. I think the solution was rather clever: because all loads were now represented as URLs anyway, we'd represent relative paths as URLs that were absolute but not canonical, based on taking the relative URL that appeared in the file and resolving it based on that file's canonical URL. Then we could safely pass a truly relative URL to Contextual ComplicationsThings get more complex from here. In order to support Node-style package resolution, we had to add a means for even absolute loads to see what file they're being loaded from. While it's true that the WebKit loader folks were the first to ask for this, it's also strictly necessary for the Node package importer, so we'd have wanted to add it sooner or later anyway. Node.js's package resolution algorithm is unavoidably sensitive to where the load occurs, because it needs to use the most local To accommodate this without going fully back to the old Ruby Sass days of fully relying on importer implementors to correctly follow all the rules, we added a way for importers to "opt in" to getting the contextual information: they would indicate which absolute URL schemes they would only accept as non-canonical absolute URLs and never return from This is where the second bug snuck in, though. Because in the old system canonicalization was expected to be purely dependent on its input arguments, we had started caching it to avoid extra filesystem hits for commonly-loaded stylesheets. But now, in some circumstances, canonicalization could depend on the containing URL, and we didn't include that in the cache. This is the specific bug that's highlighted by sass/sass-spec#1969. How Should We Fix It?I think the critical element of the fix here is pretty simple: don't cache The more complex answer would be to stop caching for a given importer once that importer accesses In the short term, since this does produce visible behavior bugs, I'm going to fix those using the broader disable. Then we can look into adding caching back with appropriate cache-busting later on. |
When I use compileString API with no "url" defined for string input, a "relative" import from the root stylesheet would go into pattern 2, with a custom base importer. For example: sass.compileString('@import "a";', {
importer: {
canonicalize (url, _context) {
console.log(`// canonical(${url})`)
return new URL('u:a')
},
load (_url) {
return {
contents: 'a {b: c}',
syntax: 'scss'
}
}
},
}) |
My initial solution was to effectively have a per importer canonicalization cache whenever baseUrl (containingUrl) is passed, regardless of whether it is used, but only for relative url that has no scheme. Performance wise it is going to be slower but better than no cache. I understand this is not necessarily correct. For example, if somehow absolute urls are canonicalized differently depends on different baseUrl, this can very well still be broken (IMO if this really happens it's the importer's fault, that's why I did not consider this case). However, if we expand this idea to both relative and absolute urls to always use a cache key of @nex3 What do you think? I updated the PR to implement this. |
This is an invalid invocation per the TypeScript types: That said, it looks like the Dart API and the embedded protocol both do allow this pattern. Per spec, this would end up invoking the WHATWG URL parser with a relative URL string and an undefined base URL, which would return a failure. We don't do that currently—we just treat this as a load-path-style canonicalization for the base importer, which doesn't really make sense. We should issue a deprecation for this behavior and clarify that the
This is intentional and desirable behavior: for example, in the Node.js package importer, absolute
An important goal of caching canonicalization is to avoid the repeated process of querying each importer in the chain for URLs we already know how to load. You're right that we could avoid the issue by just building all the possible information into the cache key, but then we'll only get cache hits if we're reloading the same URL from the same source file—something that's only going to happen in I have an in-progress PR which should address these issues. |
Having spent the day working through all the consequences of deprecating Pattern 2, I'm not quite sure that it's correct to remove it. We do currently use it to represent situations like parsing stdin on the CLI and allowing the entrypoint to load files from the working directory without making that available as a load path everywhere. Using a load-path-style importer for this is a bit of a hack, but any alternative—like creating a fake URL for the file—is also a bit of a hack, and has knock-on consequences like not being able to use All that is separate from the caching issue, which is still definitely a bug and essentially independent of the question of how the base importer is handled (since the base importer never has access to the containing URL anyway). |
I agree that adding an While this do require additional infrastructure, it should be pretty straight forward to implement. As we already have @nex3 Let me know if you want to take it, or I'm happy to contribute some time on it. I took a brief look and it is indeed a bit complicated, because in the native dart code containingUrl is implemented as |
Yeah, after sleeping on this, I think leaving the existing behavior as-is for Pattern 2 probably makes the most sense. The most accurate representation of the underlying logic here would be to have a mandatory base URL and a flag indicating whether that's also the canonical URL, but I think that's more complicated than it's worth for what is ultimately an edge case—as is going through a full deprecation process to move to a solution that's also not ideal. @ntkme I agree that we don't need to cache-bust the entire importer on a |
See sass/dart-sass#2208 Co-authored-by: Carlos (Goodwine) <2022649+Goodwine@users.noreply.github.com>
One note we might want to add to the change log is that the
In real world, the first case is likely pretty rare. However, the second case can be somewhat common as a performance optimization to 1) avoid compute the same result twice in two different importers; and 2) avoid reading large raw files on embedded host and then send via protobuf to compiler. - In fact, this optimization can also be applied to the legacy importer in embedded-host-node. We will never know if a user's importer is stateful or stateless. Most of them are likely stateless, but there might be some stateful Importers getting impacted by this change, so I think it's worth to document how caching for canonicalization works and how could it impact a stateful importer. |
Importers are really expected not to be stateful, so I'm not concerned about subtle behavioral changes for them. It's fine to have a few exceptions for backwards-compatibility purposes with old bad importer APIs, but non-tooling authors should never do that. |
Closing this as we decided to abandon the deprecation and cache optimizations has been delivered in |
----- It is inappropriate to include political and offensive content in public code repositories. Public code repositories should be neutral spaces for collaboration and community, free from personal or political views that could alienate or discriminate against others. Political content, especially that which targets or disparages minority groups, can be harmful and divisive. It can make people feel unwelcome and unsafe, and it can create a hostile work environment. Please refrain from adding such content to public code repositories. --- sass#2000 sass#2001 sass#2002 sass#2003 sass#2004 sass#2005 sass#2006 sass#2007 sass#2008 sass#2009 sass#2010 sass#2011 sass#2012 sass#2013 sass#2014 sass#2015 sass#2016 sass#2017 sass#2018 sass#2019 sass#2020 sass#2021 sass#2022 sass#2023 sass#2024 sass#2025 sass#2026 sass#2027 sass#2028 sass#2029 sass#2030 sass#2031 sass#2032 sass#2033 sass#2034 sass#2035 sass#2036 sass#2037 sass#2038 sass#2039 sass#2040 sass#2041 sass#2042 sass#2043 sass#2044 sass#2045 sass#2046 sass#2047 sass#2048 sass#2049 sass#2050 sass#2051 sass#2052 sass#2053 sass#2054 sass#2055 sass#2056 sass#2057 sass#2058 sass#2059 sass#2060 sass#2061 sass#2062 sass#2063 sass#2064 sass#2065 sass#2066 sass#2067 sass#2068 sass#2069 sass#2070 sass#2071 sass#2072 sass#2073 sass#2074 sass#2075 sass#2076 sass#2077 sass#2078 sass#2079 sass#2080 sass#2081 sass#2082 sass#2083 sass#2084 sass#2085 sass#2086 sass#2087 sass#2088 sass#2089 sass#2090 sass#2091 sass#2092 sass#2093 sass#2094 sass#2095 sass#2096 sass#2097 sass#2098 sass#2099 sass#2100 sass#2101 sass#2102 sass#2103 sass#2104 sass#2105 sass#2106 sass#2107 sass#2108 sass#2109 sass#2110 sass#2111 sass#2112 sass#2113 sass#2114 sass#2115 sass#2116 sass#2117 sass#2118 sass#2119 sass#2120 sass#2121 sass#2122 sass#2123 sass#2124 sass#2125 sass#2126 sass#2127 sass#2128 sass#2129 sass#2130 sass#2131 sass#2132 sass#2133 sass#2134 sass#2135 sass#2136 sass#2137 sass#2138 sass#2139 #2140 sass#2141 sass#2142 sass#2143 sass#2144 sass#2145 sass#2146 sass#2147 sass#2148 sass#2149 sass#2150 sass#2151 sass#2152 sass#2153 sass#2154 sass#2155 sass#2156 sass#2157 sass#2158 sass#2159 sass#2160 sass#2161 sass#2162 sass#2163 sass#2164 sass#2165 sass#2166 sass#2167 sass#2168 sass#2169 sass#2170 sass#2171 sass#2172 sass#2173 sass#2174 sass#2175 sass#2176 sass#2177 sass#2178 sass#2179 sass#2180 sass#2181 sass#2182 sass#2183 sass#2184 sass#2185 sass#2186 sass#2187 sass#2188 sass#2189 sass#2190 sass#2191 sass#2192 sass#2193 sass#2194 sass#2195 sass#2196 sass#2197 sass#2198 sass#2199 sass#2200 sass#2201 sass#2202 sass#2203 sass#2204 sass#2205 sass#2206 sass#2207 sass#2208 sass#2209 sass#2210 sass#2211 sass#2212 sass#2213 sass#2214 sass#2215 #2216 sass#2217 sass#2218 sass#2219 sass#2220 sass#2221 sass#2222 sass#2223 sass#2224 sass#2225 sass#2226 sass#2227 sass#2228 sass#2229 sass#2230 sass#2231 sass#2232 sass#2233 sass#2234 sass#2235 sass#2236 sass#2237 sass#2238 sass#2239 sass#2240 sass#2241 sass#2242 sass#2243 sass#2244 sass#2245 sass#2246 sass#2247 sass#2248 sass#2249 sass#2250 sass#2251 sass#2252 sass#2253 sass#2254 sass#2255 sass#2256 sass#2257 sass#2258 sass#2259 sass#2260 sass#2261 sass#2262 sass#2263 sass#2264 sass#2265 sass#2266 sass#2267 sass#2268 sass#2269 sass#2270
----- It is inappropriate to include political and offensive content in public code repositories. Public code repositories should be neutral spaces for collaboration and community, free from personal or political views that could alienate or discriminate against others. Political content, especially that which targets or disparages minority groups, can be harmful and divisive. It can make people feel unwelcome and unsafe, and it can create a hostile work environment. Please refrain from adding such content to public code repositories. --- sass#2000 sass#2001 sass#2002 sass#2003 sass#2004 sass#2005 sass#2006 sass#2007 sass#2008 sass#2009 sass#2010 sass#2011 sass#2012 sass#2013 sass#2014 sass#2015 sass#2016 sass#2017 sass#2018 sass#2019 sass#2020 sass#2021 sass#2022 sass#2023 sass#2024 sass#2025 sass#2026 sass#2027 sass#2028 sass#2029 sass#2030 sass#2031 sass#2032 sass#2033 sass#2034 sass#2035 sass#2036 sass#2037 sass#2038 sass#2039 sass#2040 sass#2041 sass#2042 sass#2043 sass#2044 sass#2045 sass#2046 sass#2047 sass#2048 sass#2049 sass#2050 sass#2051 sass#2052 sass#2053 sass#2054 sass#2055 sass#2056 sass#2057 sass#2058 sass#2059 sass#2060 sass#2061 sass#2062 sass#2063 sass#2064 sass#2065 sass#2066 sass#2067 sass#2068 sass#2069 sass#2070 sass#2071 sass#2072 sass#2073 sass#2074 sass#2075 sass#2076 sass#2077 sass#2078 sass#2079 sass#2080 sass#2081 sass#2082 sass#2083 sass#2084 sass#2085 sass#2086 sass#2087 sass#2088 sass#2089 sass#2090 sass#2091 sass#2092 sass#2093 sass#2094 sass#2095 sass#2096 sass#2097 sass#2098 sass#2099 sass#2100 sass#2101 sass#2102 sass#2103 sass#2104 sass#2105 sass#2106 sass#2107 sass#2108 sass#2109 sass#2110 sass#2111 sass#2112 sass#2113 sass#2114 sass#2115 sass#2116 sass#2117 sass#2118 sass#2119 sass#2120 sass#2121 sass#2122 sass#2123 sass#2124 sass#2125 sass#2126 sass#2127 sass#2128 sass#2129 sass#2130 sass#2131 sass#2132 sass#2133 sass#2134 sass#2135 sass#2136 sass#2137 sass#2138 sass#2139 #2140 sass#2141 sass#2142 sass#2143 sass#2144 sass#2145 sass#2146 sass#2147 sass#2148 sass#2149 sass#2150 sass#2151 sass#2152 sass#2153 sass#2154 sass#2155 sass#2156 sass#2157 sass#2158 sass#2159 sass#2160 sass#2161 sass#2162 sass#2163 sass#2164 sass#2165 sass#2166 sass#2167 sass#2168 sass#2169 sass#2170 sass#2171 sass#2172 sass#2173 sass#2174 sass#2175 sass#2176 sass#2177 sass#2178 sass#2179 sass#2180 sass#2181 sass#2182 sass#2183 sass#2184 sass#2185 sass#2186 sass#2187 sass#2188 sass#2189 sass#2190 sass#2191 sass#2192 sass#2193 sass#2194 sass#2195 sass#2196 sass#2197 sass#2198 sass#2199 sass#2200 sass#2201 sass#2202 sass#2203 sass#2204 sass#2205 sass#2206 sass#2207 sass#2208 sass#2209 sass#2210 sass#2211 sass#2212 sass#2213 sass#2214 sass#2215 #2216 sass#2217 sass#2218 sass#2219 sass#2220 sass#2221 sass#2222 sass#2223 sass#2224 sass#2225 sass#2226 sass#2227 sass#2228 sass#2229 sass#2230 sass#2231 sass#2232 sass#2233 sass#2234 sass#2235 sass#2236 sass#2237 sass#2238 sass#2239 sass#2240 sass#2241 sass#2242 sass#2243 sass#2244 sass#2245 sass#2246 sass#2247 sass#2248 sass#2249 sass#2250 sass#2251 sass#2252 sass#2253 sass#2254 sass#2255 sass#2256 sass#2257 sass#2258 sass#2259 sass#2260 sass#2261 sass#2262 sass#2263 sass#2264 sass#2265 sass#2266 sass#2267 sass#2268 sass#2269 sass#2270 sass#2271
Deprecate importer without base URL (*ImportCache.canonicalize: Deprecate base importer without URL #2213)containingUrl
(Explicitly allow a base importer without a base URL sass#3831)containingUrl
and cache the rest (Implement access tracking for containingUrl #2220) (Add a per-importer cache for loads that aren't cacheable en masse #2219)Drop support for importer without base URLSee https://github.com/sass/sass-spec/pull/1969/files for reproduction of the issue.
The text was updated successfully, but these errors were encountered: