Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is it possible to modify the response content through Scrapy Selector? #36

Closed
seaguest opened this issue Mar 7, 2016 · 1 comment
Closed

Comments

@seaguest
Copy link

seaguest commented Mar 7, 2016

I am using Scrapy to deep copy some content on one page, to crawl the content and download the images in that content and update the image original value accordingly.

For example I have:

<div class="A">
    <img original="example1.com/1/1.png"></img>
</div>

I need to download the image and update the new image original value(for example to mysite.com/1/1.png), then save the content.

what I will have finally is:

<div class="A">
    <img original="mysite.com/1/1.png"></img>
</div>

and image on my disk.

Is it possible to modify the value through Selector?

Or must I download the image first and update the "original" value separately? any better solution?

Another use case:
In some pages, some content are hidden by some CSS setting ( style="display:none"), or we want to do some preprocessing for some content, we will need to inspect the content and update it.

@Digenis
Copy link
Member

Digenis commented Mar 7, 2016

No.

Selectors are meant to address parts of a document, not to transform it.
Although some elementary things are possible,
like striping namespaces and running a regex over a string result,
transformations are out of the scope of this project for now
(and will remain so in the near future for practical reasons).

You should look into xslt or some similar technology.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants