Skip to content

Extracting URLs from CSS @import rule - #339

Merged
NGTmeaty merged 3 commits into
mainfrom
css
Jun 16, 2025
Merged

NGTmeaty merged 3 commits into
mainfrom
css

Conversation

@yzqzss

@yzqzss yzqzss commented Jun 14, 2025 •

Copy link
Copy Markdown
Collaborator

#323
close: #296

This PR adds the ability to extract @import urls at the css extractor level.


Crawling and archiving these @import urls will be implemented in the next PR.

@yzqzss yzqzss added enhancement New feature or request extractor GSoC labels Jun 14, 2025
@yzqzss yzqzss mentioned this pull request Jun 2, 2025
3 tasks done
@yzqzss
yzqzss requested a review from Copilot June 14, 2025 16:35

This comment was marked as outdated.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for extracting @import URLs from CSS content at the extractor level. Key changes include updating the HTML asset extractor to call the new CSS extraction functions, expanding test coverage for various @import rule scenarios, and refactoring the CSS parser to differentiate between regular URLs and @import URLs.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
internal/pkg/postprocessor/extractor/html.go Updated to use the new ExtracFromStringCSS function for inline styles and HTML style blocks with appropriate warning log messages.
internal/pkg/postprocessor/extractor/css_test.go Extended tests to cover multiple @import rule cases and validate error conditions.
internal/pkg/postprocessor/extractor/css.go Refactored CSS parsing to handle @import rules and introduced helper functions with panic calls for unexpected token types.
Comments suppressed due to low confidence (1)

internal/pkg/postprocessor/extractor/css.go:334

  • The function name 'ExtracFromStringCSS' appears to contain a typo. Consider renaming it to 'ExtractFromStringCSS' for clarity and consistency.
func ExtracFromStringCSS(cssBody string, inline bool) (links []string, atImportLinks []string, err error) {

Comment thread internal/pkg/postprocessor/extractor/css.go
Comment thread internal/pkg/postprocessor/extractor/css.go

@NGTmeaty NGTmeaty left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Thanks!

@NGTmeaty
NGTmeaty merged commit c4914ab into main Jun 16, 2025
@NGTmeaty
NGTmeaty deleted the css branch June 16, 2025 23:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request extractor GSoC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Can't extract URL from CSS @import rule

3 participants