Skip to content

Conversation

@rbrugaro
Copy link
Collaborator

Validated readme commands work as expected:

# download pdf file
wget https://raw.githubusercontent.com/opea-project/GenAIComps/v1.1/comps/retrievers/redis/data/nke-10k-2023.pdf
# upload pdf file with dataprep
curl -X POST "http://${host_ip}:11103/v1/dataprep/ingest" \
    -H "Content-Type: multipart/form-data" \
    -F "files=@./nke-10k-2023.pdf"
curl http://${host_ip}:8888/v1/graphrag \
    -H "Content-Type: application/json"  \
    -d '{"messages": [{"role": "user","content": "where do Nike subsidiaries operate?
    "}]}'

Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>
@rbrugaro rbrugaro requested a review from ashahba as a code owner February 19, 2025 01:00
@github-actions
Copy link

github-actions bot commented Feb 19, 2025

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

@xiguiw
Copy link
Collaborator

xiguiw commented Feb 25, 2025

@rbrugaro
The CI error message is:
Error response from daemon: No such container: neo4j-apoc

Is the container name typo? Would you please check it?

  • [[ neo4j-apoc == \e\x\t\r\a\c\t_\g\r\a\p\h_\n\e\o\4\j ]]
  • [[ neo4j-apoc == \n\e\o\4\j-\a\p\o\c ]]
    Mega service start duration is 503 s
    ++ curl --silent --write-out 'HTTPSTATUS:%{http_code}' 100.83.111.229:7474
  • HTTP_RESPONSE=HTTPSTATUS:000
    ++ echo HTTPSTATUS:000
    ++ tr -d '\n'
    ++ sed -e 's/.*HTTPSTATUS://'
  • HTTP_STATUS=000
    ++ echo HTTPSTATUS:000
    ++ sed -e 's/HTTPSTATUS:.*//g'
  • RESPONSE_BODY=
  • docker logs neo4j-apoc
    Error response from daemon: No such container: neo4j-apoc
  • '[' 000 -ne 200 ']'
  • echo '[ neo4j-apoc ] HTTP status is not 200. Received status was 000'
  • exit 1
    [ neo4j-apoc ] HTTP status is not 200. Received status was 000

Signed-off-by: yuanwu <rita.brugarolas.brufau@intel.com>
…A_PATH

Signed-off-by: yuanwu <rita.brugarolas.brufau@intel.com>
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolas.brufau@intel.com>
rbrugaro and others added 2 commits February 27, 2025 05:10
Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>
@rbrugaro
Copy link
Collaborator Author

#1592

@Spycsh
Copy link
Collaborator

Spycsh commented Mar 4, 2025

From the log it seems there are some remaining issues on frontend tests. @rbrugaro have you checked the UI locally? Is it working successfully? Maybe there are some proxies not set correctly that lead to a 124 timeout on frontend tests?
image

rbrugaro added 2 commits March 4, 2025 05:24
Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>
Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>
@rbrugaro rbrugaro merged commit 8dac9d1 into opea-project:main Mar 4, 2025
14 checks passed
chyundunovDatamonsters pushed a commit to chyundunovDatamonsters/OPEA-GenAIExamples that referenced this pull request Mar 4, 2025
… post refactor (opea-project#1567)

Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolas.brufau@intel.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
Co-authored-by: WenjiaoYue <wenjiao.yue@intel.com>
Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>
@rbrugaro
Copy link
Collaborator Author

rbrugaro commented Mar 5, 2025

Fixes #1509

jedwards-habana pushed a commit to jedwards-habana/GenAIExamples that referenced this pull request Mar 11, 2025
… post refactor (opea-project#1567)

Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolas.brufau@intel.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
Co-authored-by: WenjiaoYue <wenjiao.yue@intel.com>
Signed-off-by: Edwards, James A <jaedwards@habana.ai>
chyundunovDatamonsters pushed a commit to chyundunovDatamonsters/OPEA-GenAIExamples that referenced this pull request Mar 21, 2025
… post refactor (opea-project#1567)

Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolas.brufau@intel.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
Co-authored-by: WenjiaoYue <wenjiao.yue@intel.com>
Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>
chyundunovDatamonsters pushed a commit to chyundunovDatamonsters/OPEA-GenAIExamples that referenced this pull request May 16, 2025
… post refactor (opea-project#1567)

Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolas.brufau@intel.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
Co-authored-by: WenjiaoYue <wenjiao.yue@intel.com>
Signed-off-by: Chingis Yundunov <c.yundunov@datamonsters.com>
letonghan pushed a commit that referenced this pull request Sep 17, 2025
* Enlarge DocSum prompt buffer
Follow PR opea-project/GenAIComps#1471

Signed-off-by: XinyaoWa <xinyao.wang@intel.com>
letonghan added a commit that referenced this pull request Sep 17, 2025
* Fix image build issue (#1553)

Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Unified default port number for the same service in text2graph and text2sql (#1554)

Signed-off-by: Yao, Qing <qing.yao@intel.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* new: `OpeaArangoDataprep` (#2)

* new: `third_parties/arangodb`

* new: `OpeaArangoDataprep`

* cleanup

* fix: `vllm` instead of `tgi`

* fix: dataprep compsoe

* cleanup

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* new: `OpeaArangoRetriever` (#3)

* new: `OpeaArangoRetriever`

* cleanup

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* new: deps

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* fix typo: `test_retrievers_arango.sh`

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* updated retriever-arango compose file

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* correction

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* add json-repair to dataprep-arango requirements

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Fix network error, change WORKPATH

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* extra time for health check retriever

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* extended retriever healthcheck 90secs

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* correction

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Update arangodb.py

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Removing hugging face token requirement from test file

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Update test_dataprep_arango with network tests and additional logs

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Running CI after docker rate limit

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Base case remove HF_token, no additional tests

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Adding VLLM check and logs, currently VLLM not working in CI/CD

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* cleanup: compose.yaml

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* update: arangodb healthcheck

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* cleanup

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* cleanup: retriever test

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* fix: typo

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* rem: unused vars

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* fix: indent

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* temp: swap vllm healthcheck with sleep

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* fix: typo

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* fix: component name typo

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* fix: support `EmbedDoc` for retriever

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* fix: `getattr`

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* fix: CURL command

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* revert 60614841

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Update xtune file and change DDP paramter (#1552)

Signed-off-by: jilongwa <jilong.wang@intel.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* add N/A option (#1561)

Signed-off-by: ZhangJianyu <zhang.jianyu@outlook.com>
Co-authored-by: ZhangJianyu <zhang.jianyu@outlook.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Test latest gaudi docker container (#1477)

Update base gaudi container into the latest version, docker pull vault.habana.ai/gaudi-docker/1.20.1/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest, https://docs.habana.ai/en/latest/Installation_Guide/Additional_Installation/Docker_Installation.html#use-intel-gaudi-containers

Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* fix audioqna male voice setting (#1559)

Co-authored-by: Letong Han <106566639+letonghan@users.noreply.github.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* added error handling for lvm (#1556)

Signed-off-by: okhleif-IL <omar.khleif@intel.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* enable mysql db for sql agent (#1431)

Signed-off-by: cheehook <chee.hoo.kok@intel.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Enlarge DocSum prompt buffer (#1567)

* Enlarge DocSum prompt buffer
Follow PR opea-project/GenAIComps#1471

Signed-off-by: XinyaoWa <xinyao.wang@intel.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Update vLLM parameter max-seq-len-to-capture (#1565)

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* fix: lint

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* fix: missing import

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* new: healtcheck for dataprep-arangodb

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* update: arangodb readmes

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* cleanup: test_dataprep_arango.sh

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* cleanup: test_dataprep_arango.sh (PT2)

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* cleanup: test_dataprep_arango.sh (PT3)

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* update: test_dataprep_arango.sh

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* fix: whitespace

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Remove Transformers versions from requirements.txt file (#1547)

* Remove Transformers versions from requirements.txt file

Signed-off-by: Abolfazl Shahbazi <12436063+ashahba@users.noreply.github.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Remove index_names from files for dataprep-get request  (#1569)

* remove index_names from files fot get request

Signed-off-by: Mustafa <mustafa.cetin@intel.com>

* update the tests

Signed-off-by: Mustafa <mustafa.cetin@intel.com>

* update the tests

Signed-off-by: Mustafa <mustafa.cetin@intel.com>

* update the tests

Signed-off-by: Mustafa <mustafa.cetin@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add validation check for 'all' as an index_name

Signed-off-by: Mustafa <mustafa.cetin@intel.com>

* fix for readme file

Signed-off-by: Mustafa <mustafa.cetin@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Mustafa <mustafa.cetin@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Abolfazl Shahbazi <12436063+ashahba@users.noreply.github.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Upgrade Optimum Habana version to fix security check issue (#1571)

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Make llamaguard compatible with both TGI and vLLM (#1581)

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Fix Dockerfile error and add CI test for IPEX (#1585)

* Fix Dockerfile error and add CI teat

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Reduce multilang tts docker image size (#1587)

* fix audioqna male voice setting

* reduce multilang tts docker image size

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* unset OPENAI_KEY in CI test (#1586)

Signed-off-by: Rita Brugarolas <rita.brugarolas.brufau@intel.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Add AWS Credentials for CD test (#1588)

* Fix CD test issue

Signed-off-by: ZePan110 <ze.pan@intel.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* update: shorten ingest_dataprep.txt

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* revert: a4d943e7

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* new: `DataprepRequest` model (#1525)

* new: `DataprepRequest`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: docstrings

* rem: `ingest_from_graphDB`

* new: dep injection

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: verbose `input` processing

* attempt: replace `kwargs` with params

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* rem: `db_type`

ref: opea-project/GenAIComps#1525 (comment)

* attempt: require `base`

* Revert "attempt: require `base`"

This reverts commit 620ca6bc1d0b1a23f81f8d86168d65088712cfdf.

* new: `DataprepRequest`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: docstrings

* rem: `ingest_from_graphDB`

* new: dep injection

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: verbose `input` processing

* attempt: replace `kwargs` with params

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* rem: `db_type`

ref: opea-project/GenAIComps#1525 (comment)

* attempt: require `base`

* Revert "attempt: require `base`"

This reverts commit 620ca6bc1d0b1a23f81f8d86168d65088712cfdf.

* Fix dataprep request class issue of Redis (#1)

* new: `DataprepRequest`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: docstrings

* rem: `ingest_from_graphDB`

* new: dep injection

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: verbose `input` processing

* attempt: replace `kwargs` with params

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* rem: `db_type`

ref: opea-project/GenAIComps#1525 (comment)

* attempt: require `base`

* Revert "attempt: require `base`"

This reverts commit 620ca6bc1d0b1a23f81f8d86168d65088712cfdf.

* fix dataprep request class of redis

Signed-off-by: letonghan <letong.han@intel.com>

* revert change in redis.py

Signed-off-by: letonghan <letong.han@intel.com>

---------

Signed-off-by: letonghan <letong.han@intel.com>
Co-authored-by: Anthony Mahanna <anthony.mahanna@arangodb.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Anthony Mahanna <43019056+aMahanna@users.noreply.github.com>
Co-authored-by: Liang Lv <liang1.lv@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* revert: `DataprepRequest` for multimodal

* revert: `DataprepRequest` for multimodal (PT2)

* fix: conditionally fetch unique `DataprepRequest` attributes

* fix bugs in dataprep util script

Signed-off-by: letonghan <letong.han@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* revert change of pgvector

Signed-off-by: letonghan <letong.han@intel.com>

* fix indices bug for redis

Signed-off-by: letonghan <letong.han@intel.com>

* minor fix for redis

Signed-off-by: letonghan <letong.han@intel.com>

* ingest file into rag_redis_test

Signed-off-by: letonghan <letong.han@intel.com>

* update indice name

Signed-off-by: letonghan <letong.han@intel.com>

---------

Signed-off-by: letonghan <letong.han@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Liang Lv <liang1.lv@intel.com>
Co-authored-by: Letong Han <106566639+letonghan@users.noreply.github.com>
Co-authored-by: letonghan <letong.han@intel.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* revert: bc4445cb

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* revert: d17f6aac

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* Revert "new: `DataprepRequest` model (#1525)" (#1592)

This reverts commit 88947ab.

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* add hyperlinks

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* revert: 4eb9ec4f

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* new: ArangoDBDataprepRequest

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* fix: lint

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* cleanup: delete_files

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* remove: env mutation

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

* fix: move openai key env var to top of file

Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>

---------

Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: Anthony Mahanna <anthony.mahanna@arangodb.com>
Signed-off-by: Yao, Qing <qing.yao@intel.com>
Signed-off-by: jilongwa <jilong.wang@intel.com>
Signed-off-by: ZhangJianyu <zhang.jianyu@outlook.com>
Signed-off-by: okhleif-IL <omar.khleif@intel.com>
Signed-off-by: cheehook <chee.hoo.kok@intel.com>
Signed-off-by: XinyaoWa <xinyao.wang@intel.com>
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Signed-off-by: Abolfazl Shahbazi <12436063+ashahba@users.noreply.github.com>
Signed-off-by: Mustafa <mustafa.cetin@intel.com>
Signed-off-by: Rita Brugarolas <rita.brugarolas.brufau@intel.com>
Signed-off-by: ZePan110 <ze.pan@intel.com>
Signed-off-by: letonghan <letong.han@intel.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
Co-authored-by: Yao Qing <Qing.Yao@intel.com>
Co-authored-by: lasyasn <lasyan640@gmail.com>
Co-authored-by: Ajay Kallepalli <ajay.r.kallepalli@gmail.com>
Co-authored-by: jilongW <109333127+jilongW@users.noreply.github.com>
Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
Co-authored-by: ZhangJianyu <zhang.jianyu@outlook.com>
Co-authored-by: Spycsh <39623753+Spycsh@users.noreply.github.com>
Co-authored-by: Letong Han <106566639+letonghan@users.noreply.github.com>
Co-authored-by: Omar Khleif <omar.khleif@intel.com>
Co-authored-by: cheehook <chee.hoo.kok@intel.com>
Co-authored-by: XinyaoWa <xinyao.wang@intel.com>
Co-authored-by: Liang Lv <liang1.lv@intel.com>
Co-authored-by: Abolfazl Shahbazi <12436063+ashahba@users.noreply.github.com>
Co-authored-by: Mustafa <109312699+MSCetin37@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: rbrugaro <rita.brugarolas.brufau@intel.com>
Co-authored-by: ZePan110 <ze.pan@intel.com>
Co-authored-by: letonghan <letong.han@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants