You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ScriptCreatorGraph is really exiting, but it lacks some features for my use case.
I want to create scraping script for an arbitrary website containing several pages with incomplete data.
I want to be able to run this script repeatedly over time to check for updates (without hitting the LLM for cost minimisation).
Since ScriptCreatorGraph is a one page a time solution (one page generates one script) and pages I want to generate script for are incomplete, I will not be able to use the script for the complete website.
What I would like to be able to do is:
run a separate step before code generation that collects context from all pages that then can be used to run the ScriptCreatorGraph on that collected context to create a script that can handle all pages.
run ScriptCreatorGraph over a list of pages updating the script until it stops improving (and without it declining in quality during the run). This would need some kind of refining of the current script.
The later method would also make it possible to rerun the script generation on a later stage, refining the script. (or maybe just starting over is a better approach in that case)
Any thought on this, is there a better way to do this with Scrapgraph?
Would this be possible to do?
Any plans for this type of solution?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
ScriptCreatorGraph is really exiting, but it lacks some features for my use case.
I want to create scraping script for an arbitrary website containing several pages with incomplete data.
I want to be able to run this script repeatedly over time to check for updates (without hitting the LLM for cost minimisation).
Since ScriptCreatorGraph is a one page a time solution (one page generates one script) and pages I want to generate script for are incomplete, I will not be able to use the script for the complete website.
What I would like to be able to do is:
The later method would also make it possible to rerun the script generation on a later stage, refining the script. (or maybe just starting over is a better approach in that case)
Any thought on this, is there a better way to do this with Scrapgraph?
Would this be possible to do?
Any plans for this type of solution?
Beta Was this translation helpful? Give feedback.
All reactions