Skip to content

Conversation

@epage480
Copy link
Contributor

Created a fix that allows examples/openai/script_generator_openai.py to run successfully.

I made 3 major changes:

  1. Added a new parameter "parse_html" to the node_config for ParseNode. If set to False it will give the unparsed html, which is useful for creating parsing scripts. Default behavior is identical to before.
  2. Removed the RAG node from the ScriptCreatorGraph, RAG for HTML is a non-trivial task and was not being done well.
  3. script_creator_graph.py contains a lot of dead or non-functional code, it appears someone attempted to use RAG unsuccessfully.

While ScriptCreatorGraph will fail for websites with a high token count, these changes will allow at least some functionality and a starting point for expansion.

@epage480 epage480 marked this pull request as ready for review May 11, 2024 00:00
@VinciGit00 VinciGit00 merged commit 1fa77e5 into ScrapeGraphAI:pre/beta May 11, 2024
@github-actions
Copy link

🎉 This PR is included in version 0.11.0-beta.3 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions
Copy link

🎉 This PR is included in version 0.11.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants