Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
2ae19ae
update function
VinciGit00 Jul 22, 2024
0c4b290
feat: add generate_answer node paralellization
VinciGit00 Jul 22, 2024
cf27343
removed unused function
VinciGit00 Jul 23, 2024
df1ecc0
Update generate_answer_node.py
VinciGit00 Jul 23, 2024
7080a0a
ci(release): 1.11.0-beta.1 [skip ci]
semantic-release-bot Jul 23, 2024
ddbfbe1
Merge branch 'pre/beta' into generate_answer_parallel
f-aguzzi Jul 23, 2024
2edad66
chore: rebuild requirements
f-aguzzi Jul 23, 2024
377d679
feat: pdate models_tokens.py
VinciGit00 Jul 24, 2024
bf6d487
ci(release): 1.11.0-beta.2 [skip ci]
semantic-release-bot Jul 24, 2024
f336c95
fix: add llama 3.1
VinciGit00 Jul 25, 2024
66f9421
ci(release): 1.11.0-beta.3 [skip ci]
semantic-release-bot Jul 25, 2024
d745564
Merge pull request #485 from ScrapeGraphAI/generate_answer_parallel
f-aguzzi Jul 25, 2024
51db43a
ci(release): 1.11.0-beta.4 [skip ci]
semantic-release-bot Jul 25, 2024
5137b8a
Update models_tokens.py
VinciGit00 Jul 26, 2024
05cf9ad
Ollama: Use no json format when creating the search query
AmosDinh Jul 26, 2024
ecc5e35
Merge pull request #490 from AmosDinh/pre/beta
VinciGit00 Jul 26, 2024
d177afb
refactor(Ollama): integrate new LangChain chat init
f-aguzzi Jul 29, 2024
5e3eb6e
refactor(OpenAI): integrate new LangChain chat init
f-aguzzi Jul 29, 2024
9e795f4
removed unused init files
VinciGit00 Jul 29, 2024
2c5f934
refactor: remove LangChain wrappers
f-aguzzi Jul 29, 2024
25066b2
refactor: remove LangChain wrappers for Ollama
f-aguzzi Jul 29, 2024
f6b7c6a
refactoring
VinciGit00 Jul 29, 2024
5007167
removed unused models
VinciGit00 Jul 29, 2024
9275486
refactor: remove redundant LangChain wrappers
f-aguzzi Jul 29, 2024
bc2c996
refactor: remove redundant wrappers for Ernie and Nvidia
f-aguzzi Jul 30, 2024
07ef383
add rye packages
VinciGit00 Jul 30, 2024
88710f1
chore: remove unused import
f-aguzzi Jul 30, 2024
1db164e
feat: fix tests
VinciGit00 Jul 30, 2024
c8e150d
Merge pull request #494 from ScrapeGraphAI/langchain-common-chat
VinciGit00 Jul 30, 2024
b15fd9f
ci(release): 1.11.0-beta.5 [skip ci]
semantic-release-bot Jul 30, 2024
b17756d
style: enforce pylint styling
f-aguzzi Jul 31, 2024
bb73d91
refactor: reuse code for common interface models
f-aguzzi Jul 31, 2024
4caed54
feat: intregration of firebase
VinciGit00 Jul 31, 2024
74ed8d0
ci(release): 1.11.0-beta.6 [skip ci]
semantic-release-bot Jul 31, 2024
fb87d01
Create browser_base.py
VinciGit00 Aug 1, 2024
7076ab1
allignment
VinciGit00 Aug 1, 2024
a94ebcd
refactor: move embeddings code from AbstractGraph to RAGNode
f-aguzzi Aug 1, 2024
5ecdbe7
feat: add integration in the abstract grapgh
VinciGit00 Aug 1, 2024
4cd5986
Merge branch 'pre/beta' into browserbase_integration
VinciGit00 Aug 1, 2024
65f9e3a
Delete browser_base.py
VinciGit00 Aug 1, 2024
d03eedc
Update chromium.py
VinciGit00 Aug 1, 2024
e21d461
push
VinciGit00 Aug 1, 2024
968c69e
fix: fixed bug on fetch_node
VinciGit00 Aug 1, 2024
6d8e02c
Update browser_base.py
VinciGit00 Aug 1, 2024
be870a4
Update fetch_node.py
VinciGit00 Aug 1, 2024
0b4cfd6
fix: abstract_graph and removed unused embeddings
VinciGit00 Aug 1, 2024
9ac74de
Update speech_graph.py
VinciGit00 Aug 1, 2024
53dc06a
Update speech_graph.py
VinciGit00 Aug 1, 2024
abd0ea1
Merge pull request #500 from ScrapeGraphAI/497-abstract-graph-refactor
lurenss Aug 1, 2024
55f706f
ci(release): 1.11.0-beta.7 [skip ci]
semantic-release-bot Aug 1, 2024
fe099f9
Merge branch 'pre/beta' into browserbase_integration
VinciGit00 Aug 1, 2024
78343e5
Merge pull request #499 from ScrapeGraphAI/browserbase_integration
DiTo97 Aug 1, 2024
3e07f62
ci(release): 1.11.0-beta.8 [skip ci]
semantic-release-bot Aug 1, 2024
9355507
feat: refactoring of the code
VinciGit00 Aug 2, 2024
4440790
ci(release): 1.11.0-beta.9 [skip ci]
semantic-release-bot Aug 2, 2024
ade28fc
fix(AbstractGraph): instantiation of Azure GPT models
f-aguzzi Aug 2, 2024
ddb3135
Merge pull request #503 from ScrapeGraphAI/497-abstract-graph-refactor
VinciGit00 Aug 2, 2024
ee30a83
ci(release): 1.11.0-beta.10 [skip ci]
semantic-release-bot Aug 2, 2024
a3d7f1b
Update csv_scraper_graph.py
VinciGit00 Aug 5, 2024
1fb711a
Merge branch 'pre/beta' of https://github.com/ScrapeGraphAI/Scrapegra…
VinciGit00 Aug 5, 2024
66a29bc
refactoring of the openai examples
VinciGit00 Aug 5, 2024
886bab7
Merge branch 'main' into temp-1
VinciGit00 Aug 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,90 @@
## [1.11.0-beta.10](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.11.0-beta.9...v1.11.0-beta.10) (2024-08-02)


### Bug Fixes

* **AbstractGraph:** instantiation of Azure GPT models ([ade28fc](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/ade28fca2c3fdf40f28a80854e3b8435a52a6930)), closes [#498](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/498)

## [1.11.0-beta.9](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.11.0-beta.8...v1.11.0-beta.9) (2024-08-02)


### Features

* refactoring of the code ([9355507](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/9355507a2dc73342f325b6649e871df48ae13567))

## [1.11.0-beta.8](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.11.0-beta.7...v1.11.0-beta.8) (2024-08-01)


### Features

* add integration in the abstract grapgh ([5ecdbe7](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/5ecdbe715f4bb223fa1be834fda07ccea2a51cb9))


### Bug Fixes

* fixed bug on fetch_node ([968c69e](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/968c69e217d9c180b9b8c2aa52ca59b9a1733525))

## [1.11.0-beta.7](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.11.0-beta.6...v1.11.0-beta.7) (2024-08-01)
## [1.10.0-beta.7](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.10.0-beta.6...v1.10.0-beta.7) (2024-07-23)

## [1.11.3](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.11.2...v1.11.3) (2024-07-25)



### Bug Fixes


* abstract_graph and removed unused embeddings ([0b4cfd6](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/0b4cfd6522dcad0eb418f0badd0f7824a1efd534))


### Refactor

* move embeddings code from AbstractGraph to RAGNode ([a94ebcd](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/a94ebcde0078d66d33e67f7e0a87850efb92d408))
* reuse code for common interface models ([bb73d91](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/bb73d916a1a7b378438038ec928eeda6d8f6ac9d))

## [1.11.0-beta.6](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.11.0-beta.5...v1.11.0-beta.6) (2024-07-31)


### Features

* intregration of firebase ([4caed54](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/4caed545e5030460b2d5e46f9ad90546ce36f0ee))

## [1.11.0-beta.5](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.11.0-beta.4...v1.11.0-beta.5) (2024-07-30)


### Features

* fix tests ([1db164e](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/1db164e9e682eefbc1414989a043fefa2e9009c2))


### chore

* remove unused import ([88710f1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/88710f1a7c7d50f57108456112da30d1a12a1ba1))


### Refactor

* **Ollama:** integrate new LangChain chat init ([d177afb](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/d177afb68be036465ede1f567d2562b145d77d36))
* **OpenAI:** integrate new LangChain chat init ([5e3eb6e](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/5e3eb6e43df4bd4c452d34b49f254235e9ff1b22))
* remove LangChain wrappers ([2c5f934](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/2c5f934f101e319ec4e61009d4c464ca4626c1ff))
* remove LangChain wrappers for Ollama ([25066b2](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/25066b2bc51517e50058231664230b8edef365b9))
* remove redundant LangChain wrappers ([9275486](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/927548624034b3c30eca60011d216720102d1815))
* remove redundant wrappers for Ernie and Nvidia ([bc2c996](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/bc2c9967d2f13ade6eeb7b23e9b423f6e79aa890))

## [1.11.0-beta.4](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.11.0-beta.3...v1.11.0-beta.4) (2024-07-25)


### Features

* add generate_answer node paralellization ([0c4b290](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/0c4b2908d98efbb2b0a6faf68618a801d726bb5f))


### chore

* rebuild requirements ([2edad66](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/2edad66788cbd92f197e3b37db13c44bfa39e36a))

## [1.11.0-beta.3](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.11.0-beta.2...v1.11.0-beta.3) (2024-07-25)
=======
* add llama 3.1 ([f872bdd](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/f872bdd24f9874660eea04f9ade570c96b6e7e93))


Expand All @@ -18,6 +98,16 @@

### Bug Fixes

* add llama 3.1 ([f336c95](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/f336c95c2d1833d1f829d61ae7fa415ac2caf250))

## [1.11.0-beta.2](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.11.0-beta.1...v1.11.0-beta.2) (2024-07-24)


### Features

* pdate models_tokens.py ([377d679](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/377d679eecd62611c0c9a3cba8202c6f0719ed31))

## [1.11.0-beta.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.10.4...v1.11.0-beta.1) (2024-07-23)
* md conversion ([1d41f6e](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/1d41f6eafe8ed0e191bb6a258d54c6388ff283c6))

## [1.11.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.11.0...v1.11.1) (2024-07-23)
Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/csv_scraper_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,6 @@
"client": "client_name",
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}
# ************************************************
Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/csv_scraper_graph_multi_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,6 @@
"client": "client_name",
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}

Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/custom_graph_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,6 @@
"client": "client_name",
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}

Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/json_scraper_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,6 @@
"client": "client_name",
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}

Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/json_scraper_multi_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,6 @@
"client": "client_name",
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}
FILE_NAME = "inputs/example.json"
Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/pdf_scraper_graph_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,6 @@
"client": "client_name",
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}

Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/pdf_scraper_graph_multi_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,6 @@
"client": "client_name",
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}
# ***************
Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/scrape_plain_text_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,6 @@
"client": "client_name",
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}

Expand Down
5 changes: 1 addition & 4 deletions examples/bedrock/script_generator_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,7 @@
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
},
"library": "beautifulsoup"
"library": "beautifulsoup"
}

# ************************************************
Expand Down
5 changes: 1 addition & 4 deletions examples/bedrock/script_multi_generator_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,7 @@
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
},
"library": "beautifulsoup"
"library": "beautifulsoup"
}

# ************************************************
Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/search_graph_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,6 @@
"client": "client_name",
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}
# ************************************************
Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/search_graph_schema_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,6 @@ class Dishes(BaseModel):
"client": "client_name",
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}

Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/search_link_graph_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,6 @@
"client": "client_name",
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}

Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/smart_scraper_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,6 @@
"client": "client_name",
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}

Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/smart_scraper_multi_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,6 @@
"client": "client_name",
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}

Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/smart_scraper_schema_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,6 @@ class Projects(BaseModel):
"client": "client_name",
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}

Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/xml_scraper_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,6 @@
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}

# ************************************************
Expand Down
3 changes: 0 additions & 3 deletions examples/bedrock/xml_scraper_graph_multi_bedrock.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,6 @@
"model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"temperature": 0.0
},
"embeddings": {
"model": "bedrock/cohere.embed-multilingual-v3"
}
}

# ************************************************
Expand Down
5 changes: 0 additions & 5 deletions examples/deepseek/csv_scraper_deepseek.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,6 @@
"model": "deepseek-chat",
"openai_api_key": deepseek_key,
"openai_api_base": 'https://api.deepseek.com/v1',
},
"embeddings": {
"model": "ollama/nomic-embed-text",
"temperature": 0,
# "base_url": "http://localhost:11434", # set ollama URL arbitrarily
},
"verbose": True,
}
Expand Down
5 changes: 0 additions & 5 deletions examples/deepseek/csv_scraper_graph_multi_deepseek.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,6 @@
"model": "deepseek-chat",
"openai_api_key": deepseek_key,
"openai_api_base": 'https://api.deepseek.com/v1',
},
"embeddings": {
"model": "ollama/nomic-embed-text",
"temperature": 0,
# "base_url": "http://localhost:11434", # set ollama URL arbitrarily
},
"verbose": True,
}
Expand Down
Loading