Skip to content

Commit

Permalink
[Reorg Folder] reorg transformers and langchain folders (#1360)
Browse files Browse the repository at this point in the history 
Co-authored-by: VincyZhang <wenxin.zhang@intel.com>
  • Loading branch information
@zhentaoyu @VincyZhang
zhentaoyu and VincyZhang committed Mar 19, 2024
1 parent 22536a4 commit ae54f69
Show file tree
Hide file tree
Showing 1,147 changed files with 813 additions and 1,986 deletions.
16 changes: 8 additions & 8 deletions .github/CODEOWNERS
View file
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -25,16 +25,16 @@
# *.txt @octo-org/octocats


/intel_extension_for_transformers/llm/runtime/deprecated/Cmake* yu.luo@intel.com
/intel_extension_for_transformers/llm/runtime/deprecated/cmake/ yu.luo@intel.com
/intel_extension_for_transformers/llm/runtime/deprecated bo1.dong@intel.com
/intel_extension_for_transformers/llm/operator bo1.dong@intel.com
/intel_extension_for_transformers/llm/evaluation chang1.wang@intel.com
/intel_extension_for_transformers/llm/finetuning liang1.lv@intel.com
/intel_extension_for_transformers/llm/inference liang1.lv@intel.com
/intel_extension_for_transformers/transformers/runtime/Cmake* yu.luo@intel.com
/intel_extension_for_transformers/transformers/runtime/cmake/ yu.luo@intel.com
/intel_extension_for_transformers/transformers/runtime bo1.dong@intel.com
/intel_extension_for_transformers/transformers/llm/operator bo1.dong@intel.com
/intel_extension_for_transformers/transformers/llm/evaluation chang1.wang@intel.com
/intel_extension_for_transformers/transformers/llm/finetuning liang1.lv@intel.com
/intel_extension_for_transformers/transformers/llm/inference liang1.lv@intel.com
/intel_extension_for_transformers/neural_chat liang1.lv@intel.com
/workflows/chatbot liang1.lv@intel.com
/intel_extension_for_transformers/llm/quantization penghui.cheng@intel.com
/intel_extension_for_transformers/transformers/llm/quantization penghui.cheng@intel.com
/intel_extension_for_transformers/transformers penghui.cheng@intel.com
/intel_extension_for_transformers/utils penghui.cheng@intel.com
/docs wenxin.zhang@intel.com
Expand Down
61 changes: 23 additions & 38 deletions .github/checkgroup.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ subprojects:
checks:
- "format-scan (pylint)"
- "format-scan (bandit)"
- "format-scan (clangformat)"
- "format-scan (cloc)"
- "format-scan (cpplint)"

Expand All @@ -20,27 +19,21 @@ subprojects:
- intel_extension_for_transformers/transformers/**
- requirements.txt
- setup.py
- intel_extension_for_transformers/utils/**
- intel_extension_for_transformers/llm/evaluation/**
- intel_extension_for_transformers/llm/quantization/**
- intel_extension_for_transformers/llm/runtime/graph/**
- intel_extension_for_transformers/llm/operator/**
- intel_extension_for_transformers/transformers/llm/evaluation/**
- intel_extension_for_transformers/transformers/llm/quantization/**
- intel_extension_for_transformers/transformers/llm/operator/**
- tests/**
- .github/workflows/unit-test-optimize.yml
- ".github/workflows/script/unitTest/run_unit_test_optimize.sh"
- "!intel_extension_for_transformers/llm/runtime/graph/*.md"
checks:
- "optimize-unit-test-baseline"
- "optimize-unit-test-PR-test"
- "Genreate-OptimizeUT-Report"

- id: "Neural Speed Unit Test workflow"
paths:
- intel_extension_for_transformers/llm/runtime/neural_speed/**
- .github/workflows/unit-test-neuralspeed.yml
- ".github/workflows/script/unitTest/run_unit_test_neuraspeed.sh"
- "!intel_extension_for_transformers/llm/runtime/neural_speed/docs/**"
- "!intel_extension_for_transformers/llm/runtime/neural_speed/*.md"
checks:
- "neural-speed-unit-test"

Expand All @@ -51,16 +44,14 @@ subprojects:
- "intel_extension_for_transformers/neural_chat/**"
- "requirements.txt"
- "setup.py"
- "intel_extension_for_transformers/llm/finetuning/**"
- "intel_extension_for_transformers/llm/quantization/**"
- "intel_extension_for_transformers/llm/runtime/graph/**"
- "intel_extension_for_transformers/transformers/llm/finetuning/**"
- "intel_extension_for_transformers/transformers/llm/quantization/**"
- "intel_extension_for_transformers/transformers/**"
- "intel_extension_for_transformers/langchain/**"
- "!intel_extension_for_transformers/neural_chat/docs/**"
- "!intel_extension_for_transformers/neural_chat/examples/**"
- "!intel_extension_for_transformers/neural_chat/assets/**"
- "!intel_extension_for_transformers/neural_chat/README.md"
- "!intel_extension_for_transformers/llm/runtime/graph/*.md"
checks:
- "neuralchat-unit-test-baseline"
- "neuralchat-unit-test-PR-test"
Expand All @@ -71,11 +62,11 @@ subprojects:
- ".github/workflows/unit-test-engine.yml"
- "requirements.txt"
- "setup.py"
- "intel_extension_for_transformers/llm/runtime/deprecated/**"
- "!intel_extension_for_transformers/llm/runtime/deprecated/kernels/**"
- "!intel_extension_for_transformers/llm/runtime/graph/**"
- "!intel_extension_for_transformers/llm/runtime/deprecated/third_party/**"
- "!intel_extension_for_transformers/llm/runtime/deprecated/docs/**"
- intel_extension_for_transformers/transformers/**
- "intel_extension_for_transformers/transformers/runtime/**"
- "!intel_extension_for_transformers/transformers/runtime/kernels/**"
- "!intel_extension_for_transformers/transformers/runtime/third_party/**"
- "!intel_extension_for_transformers/transformers/runtime/docs/**"
checks:
- "engine-unit-test-baseline"
- "engine-unit-test-PR-test"
Expand All @@ -86,22 +77,17 @@ subprojects:
- ".github/workflows/windows-test.yml"
- "requirements.txt"
- "setup.py"
- "intel_extension_for_transformers/llm/runtime/deprecated/**"
- "intel_extension_for_transformers/llm/runtime/graph/**"
- "!intel_extension_for_transformers/llm/library/jblas/**"
- "!intel_extension_for_transformers/llm/runtime/graph/*.md"
- "!intel_extension_for_transformers/llm/runtime/graph/docs/**"
- "!intel_extension_for_transformers/llm/runtime/graph/scripts/**"
- "!intel_extension_for_transformers/llm/runtime/graph/tests/**"
- "!intel_extension_for_transformers/llm/runtime/deprecated/third_party/**"
- "!intel_extension_for_transformers/llm/runtime/deprecated/docs/**"
- "!intel_extension_for_transformers/llm/runtime/deprecated/test/**"
- "intel_extension_for_transformers/transformers/runtime/**"
- "intel_extension_for_transformers/transformers/llm/operator/**"
- "!intel_extension_for_transformers/transformers/runtime/third_party/**"
- "!intel_extension_for_transformers/transformers/runtime/docs/**"
- "!intel_extension_for_transformers/transformers/runtime/test/**"
checks:
- "Windows-Binary-Test"

- id: "SparseLib Tests workflow"
paths:
- "intel_extension_for_transformers/llm/runtime/deprecated/kernels/**"
- "intel_extension_for_transformers/transformers/runtime/kernels/**"
- ".github/workflows/sparse_lib_CI.yml"
- ".github/workflows/script/SparseLibCI"
checks:
Expand All @@ -111,12 +97,11 @@ subprojects:
paths:
- ".github/workflows/llm-test.yml"
- ".github/workflows/script/models/run_llm.sh"
- "intel_extension_for_transformers/llm/runtime/deprecated/**"
- "!intel_extension_for_transformers/llm/runtime/deprecated/kernels/**"
- "!intel_extension_for_transformers/llm/runtime/graph/**"
- "!intel_extension_for_transformers/llm/runtime/deprecated/test/**"
- "!intel_extension_for_transformers/llm/runtime/deprecated/third_party/**"
- "!intel_extension_for_transformers/llm/runtime/deprecated/docs/**"
- "intel_extension_for_transformers/transformers/runtime/**"
- "!intel_extension_for_transformers/transformers/runtime/kernels/**"
- "!intel_extension_for_transformers/transformers/runtime/test/**"
- "!intel_extension_for_transformers/transformers/runtime/third_party/**"
- "!intel_extension_for_transformers/transformers/runtime/docs/**"
checks:
- "LLM-Workflow (gpt-j-6b, engine, latency, bf16,int8,fp8)"
- "Generate-LLM-Report"
Expand All @@ -133,8 +118,8 @@ subprojects:
- ".github/workflows/script/chatbot/**"
- ".github/workflows/sample_data/**"
- "intel_extension_for_transformers/neural_chat/**"
- "intel_extension_for_transformers/llm/finetuning/**"
- "intel_extension_for_transformers/llm/quantization/**"
- "intel_extension_for_transformers/transformers/llm/finetuning/**"
- "intel_extension_for_transformers/transformers/llm/quantization/**"
- "intel_extension_for_transformers/transformers/**"
- "workflows/chatbot/inference/**"
- "workflows/chatbot/fine_tuning/**"
Expand Down
16 changes: 8 additions & 8 deletions .github/workflows/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,15 @@ Intel Extension for Transformers use Github Action (GHA) for CI test, if you are
| Spell Check | ALL |
| Copyright Check | intel_extension_for_transformers/** setup.py .github/workflows/format_scan.yml |
| DCO | ALL (use commit -s -m"...") |
| UT + Coverage (Optimize) | intel_extension_for_transformers/transformers/** intel_extension_for_transformers/utils/** intel_extension_for_transformers/llm/evaluation/** intel_extension_for_transformers/llm/quantization/** tests/** .github/workflows/unit-test-optimize.yml .github/workflows/script/unitTest/** |
| UT + Coverage (Backend) | ".github/workflows/unit-test-engine.yml" .github/workflows/script/unitTest/** intel_extension_for_transformers/llm/runtime/deprecated/** !intel_extension_for_transformers/llm/runtime/deprecated/kernels/** !intel_extension_for_transformers/llm/runtime/graph/** !intel_extension_for_transformers/llm/runtime/deprecated/third_party/** !intel_extension_for_transformers/llm/runtime/deprecated/docs/** |
| UT + Coverage (Optimize) | intel_extension_for_transformers/transformers/** intel_extension_for_transformers/utils/** intel_extension_for_transformers/transformers/llm/evaluation/** intel_extension_for_transformers/transformers/llm/quantization/** tests/** .github/workflows/unit-test-optimize.yml .github/workflows/script/unitTest/** |
| UT + Coverage (Backend) | ".github/workflows/unit-test-engine.yml" .github/workflows/script/unitTest/** intel_extension_for_transformers/transformers/runtime/** !intel_extension_for_transformers/transformers/runtime/kernels/** !intel_extension_for_transformers/transformers/llm/runtime/neural_speed/** !intel_extension_for_transformers/transformers/runtime/third_party/** !intel_extension_for_transformers/transformers/runtime/docs/** |
| UT + Coverage (NeuralChat) | intel_extension_for_transformers/neural_chat/tests/** .github/workflows/unit-test-neuralchat.yml |
| Kernel gtest | intel_extension_for_transformers/llm/library/kernels/** .github/workflows/unit-test-kernel.yml .github/workflows/script/unitTest/** !intel_extension_for_transformers/llm/library/kernels/docs/** !intel_extension_for_transformers/llm/runtime/graph/** !intel_extension_for_transformers/llm/library/kernels/README.md |
| Kernel Benchmark | intel_extension_for_transformers/llm/library/kernels/** intel_extension_for_transformers/llm/runtime/deprecated/test/kernels/benchmark/** .github/workflows/sparse_lib_CI.yml' '.github/workflows/script/SparseLibCI !intel_extension_for_transformers/llm/library/kernels/docs/** !intel_extension_for_transformers/llm/library/kernels/README.md |
| BERT Model Test (Pytorch/Tensorflow) | intel_extension_for_transformers/transformers/** intel_extension_for_transformers/llm/evaluation/** intel_extension_for_transformers/utils/** intel_extension_for_transformers/transformers/** .github/workflows/optimize-test.yml .github/workflows/script/models/run_optimize.sh |
| BERT Model Test (LLM Runtime) | .github/workflows/deploy-test.yml .github/workflows/script/models/run_deploy.sh intel_extension_for_transformers/llm/runtime/deprecated/** !intel_extension_for_transformers/llm/runtime/deprecated/kernels/** !intel_extension_for_transformers/llm/runtime/deprecated/test/** !intel_extension_for_transformers/llm/runtime/graph/** !intel_extension_for_transformers/llm/runtime/deprecated/third_party/** !intel_extension_for_transformers/llm/runtime/deprecated/docs/** |
| LLM Model Test (LLM Runtime) | .github/workflows/llm-test.yml .github/workflows/script/models/run_llm.sh intel_extension_for_transformers/llm/runtime/deprecated/** !intel_extension_for_transformers/llm/runtime/deprecated/kernels/** !intel_extension_for_transformers/llm/runtime/deprecated/graph/** !intel_extension_for_transformers/llm/runtime/deprecated/test/** !intel_extension_for_transformers/llm/runtime/deprecated/third_party/** !intel_extension_for_transformers/llm/runtime/deprecated/docs/** |
| LLM Runtime CPP Graph Test | .github/workflows/cpp-graph-test.yml .github/workflows/script/models/cpp_graph_inference.sh intel_extension_for_transformers/llm/runtime/graph/** !intel_extension_for_transformers/llm/runtime/graph/README.md |
| Kernel gtest | intel_extension_for_transformers/transformers/llm/library/kernels/** .github/workflows/unit-test-kernel.yml .github/workflows/script/unitTest/** !intel_extension_for_transformers/transformers/llm/library/kernels/docs/** !intel_extension_for_transformers/transformers/llm/runtime/neural_speed/** !intel_extension_for_transformers/transformers/llm/library/kernels/README.md |
| Kernel Benchmark | intel_extension_for_transformers/transformers/llm/library/kernels/** intel_extension_for_transformers/transformers/runtime/test/kernels/benchmark/** .github/workflows/sparse_lib_CI.yml' '.github/workflows/script/SparseLibCI !intel_extension_for_transformers/transformers/llm/library/kernels/docs/** !intel_extension_for_transformers/transformers/llm/library/kernels/README.md |
| BERT Model Test (Pytorch/Tensorflow) | intel_extension_for_transformers/transformers/** intel_extension_for_transformers/transformers/llm/evaluation/** intel_extension_for_transformers/utils/** intel_extension_for_transformers/transformers/** .github/workflows/optimize-test.yml .github/workflows/script/models/run_optimize.sh |
| BERT Model Test (LLM Runtime) | .github/workflows/deploy-test.yml .github/workflows/script/models/run_deploy.sh intel_extension_for_transformers/transformers/runtime/** !intel_extension_for_transformers/transformers/runtime/kernels/** !intel_extension_for_transformers/transformers/runtime/test/** !intel_extension_for_transformers/transformers/llm/runtime/neural_speed/** !intel_extension_for_transformers/transformers/runtime/third_party/** !intel_extension_for_transformers/transformers/runtime/docs/** |
| LLM Model Test (LLM Runtime) | .github/workflows/llm-test.yml .github/workflows/script/models/run_llm.sh intel_extension_for_transformers/transformers/runtime/** !intel_extension_for_transformers/transformers/runtime/kernels/** !intel_extension_for_transformers/transformers/runtime/graph/** !intel_extension_for_transformers/transformers/runtime/test/** !intel_extension_for_transformers/transformers/runtime/third_party/** !intel_extension_for_transformers/transformers/runtime/docs/** |
| LLM Runtime CPP Graph Test | .github/workflows/cpp-graph-test.yml .github/workflows/script/models/cpp_graph_inference.sh intel_extension_for_transformers/transformers/llm/runtime/neural_speed/** !intel_extension_for_transformers/transformers/llm/runtime/neural_speed/README.md |
| NeuralChat Workflow Test | .github/workflows/script/chatbot/** .github/workflows/sample_data/** intel_extension_for_transformers/** workflows/chatbot/inference/** .github/workflows/chatbot-test.yml |

## Test Scripts
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/chatbot-test.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ on:
- '.github/workflows/script/chatbot/**'
- '.github/workflows/sample_data/**'
- 'intel_extension_for_transformers/neural_chat/**'
- 'intel_extension_for_transformers/llm/finetuning/**'
- 'intel_extension_for_transformers/llm/quantization/**'
- 'intel_extension_for_transformers/transformers/llm/finetuning/**'
- 'intel_extension_for_transformers/transformers/llm/quantization/**'
- 'intel_extension_for_transformers/transformers/**'
- 'workflows/chatbot/inference/**'
- 'workflows/chatbot/fine_tuning/**'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/cpp-graph-test.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ jobs:
${{ env.INPUT_COMPILER_VERSION }}
env:
WORKSPACE: ${{ env.WORKING_DIR }}
GRAPH_DIR: ${{ env.WORKING_DIR }}/intel_extension_for_transformers/llm/runtime/graph
GRAPH_DIR: ${{ env.WORKING_DIR }}/intel_extension_for_transformers/transformers/llm/runtime/neural_speed

- name: Rename summary
run: |
Expand Down

0 comments on commit ae54f69

Please sign in to comment.