Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge Upstream Changes to Main Branch #16

Open
wants to merge 56 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
665c26c
feat: Add redirect to document page (#29)
leehuwuj Mar 29, 2024
2739714
feat: Add database data source (MySQL and PostgreSQL) (#28)
leehuwuj Apr 1, 2024
c06d4af
feat: Update FastAPI endpoint to support nodeSources (#30)
leehuwuj Apr 1, 2024
29b17ee
Allow tools without datasource and clean up (#33)
marcusschiesser Apr 4, 2024
5512a9e
add example request (#32)
leehuwuj Apr 4, 2024
625ed4d
feat: support astra vectordb (#34)
thucpn Apr 5, 2024
0742d3b
fix: don't create devcontainer if it exists (#35)
marcusschiesser Apr 5, 2024
8201e2f
refactor: replace mjs vectordb files by ts files (#36)
thucpn Apr 5, 2024
58388b0
fix: remove non-working vercel streaming with source nodes
marcusschiesser Apr 5, 2024
83e7b65
RELEASING: Releasing 1 package(s)
marcusschiesser Apr 5, 2024
ee2cc66
feat: observability for Python with OpenTelemetry (#39)
nirga Apr 8, 2024
1d78202
docs(changeset): Add observability for Python
marcusschiesser Apr 8, 2024
2aaf99e
chore: add changeset release script
marcusschiesser Apr 8, 2024
a6e76e1
fix: release script
marcusschiesser Apr 8, 2024
0cf6386
feat: use setting config (#38)
thucpn Apr 8, 2024
9efcffe
docs(changeset): Use Settings object for LlamaIndex configuration
marcusschiesser Apr 8, 2024
753229d
feat: optionally ask to select AI models and use default models (#40)
thucpn Apr 9, 2024
6acccd2
feat: use poetry run generate (#41)
thucpn Apr 9, 2024
1be69a5
feat: Qdrant support (#42)
Anush008 Apr 10, 2024
a0dec80
docs(changeset): Use `gpt-4-turbo` model as default. Upgrade Python l…
marcusschiesser Apr 10, 2024
1c3d0b1
feat: Use `gpt-4-turbo` model as default. Upgrade Python llama-index …
marcusschiesser Apr 10, 2024
66b5f38
docs: don't autocommit changesets
marcusschiesser Apr 11, 2024
897feb9
ci: add github releases action
marcusschiesser Apr 11, 2024
7225e91
fix: use new ToolsFactory
marcusschiesser Apr 12, 2024
5463d3b
fix: nextjs type checks
marcusschiesser Apr 12, 2024
e4665b6
add npmrc file for express to fix long path name issue with node 20 o…
thucpn Apr 15, 2024
5fb64b7
fix: aws4 warning (#51)
marcusschiesser Apr 16, 2024
ae7b301
feat: display sources in chat messages (#45)
thucpn Apr 17, 2024
e5d20b6
ci: add npm publish to release script
marcusschiesser Apr 19, 2024
418bf9b
refactor: use tsx instead of ts-node (#54)
thucpn Apr 22, 2024
3bb94da
feat: show alert when getting chat error (#55)
thucpn Apr 22, 2024
089916a
chore: fix format and update to pnpm 9.0.5
marcusschiesser Apr 22, 2024
f1c3e8d
Adding support for Llama 3 and Phi3 (via Ollama) (#53)
marcusschiesser Apr 24, 2024
8b81b17
fix: only render sources with metadata
marcusschiesser Apr 24, 2024
046ff06
Release 0.1.0 (#44)
github-actions[bot] Apr 24, 2024
cfb5257
feat: display chat events (#52)
thucpn Apr 24, 2024
1ad2545
fix: allow onnxruntime in nextjs server side (#59)
thucpn Apr 24, 2024
6bdd4ac
ci: name changeset PRs with version
marcusschiesser Apr 25, 2024
e29ef92
ci: fix pnpm
marcusschiesser Apr 25, 2024
94a2809
fix: changeset status
marcusschiesser Apr 25, 2024
a0b04be
fix: hide events per default and optimize python messaging (#64)
marcusschiesser Apr 25, 2024
5d8d752
fix: filter none events in Python (#66)
marcusschiesser Apr 26, 2024
e2567ff
feat: use TOP_K env variable also for TS (#67)
marcusschiesser Apr 26, 2024
c094b0c
Use ingestion pipeline in Python code (#61)
leehuwuj Apr 26, 2024
c981eb1
Fix escaping double quotes in vercel_response.py (#68)
leehuwuj Apr 26, 2024
7bd3ed5
feat: support anthropic and gemini model providers and update to LITS…
thucpn May 2, 2024
06823da
fix: stream type
marcusschiesser May 2, 2024
a1feb52
Revert "Use ingestion pipeline in Python code (#61)"
marcusschiesser May 3, 2024
2e80ef4
Fix typo in settings.py (#73)
leehuwuj May 3, 2024
63bb050
Release 0.1.1 (#60)
github-actions[bot] May 3, 2024
8a5ece1
chores: update wrong example system prompt and fix missing switch bre…
leehuwuj May 8, 2024
819cccb
feat: use 3.5 as default model (#77)
thucpn May 9, 2024
056e376
feat: add weather widget and weather tool (#72)
thucpn May 10, 2024
988bfc2
Release 0.1.2 (#79)
github-actions[bot] May 10, 2024
fa77cd3
Set our merge strategy for specific files.
May 13, 2024
d5dc558
Merge remote-tracking branch 'upstream/main'
May 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
README.md merge=ours
LICENSES/* merge=ours
workflows/* merge=ours
LICENSE.md merge=ours
4 changes: 0 additions & 4 deletions .github/ISSUE_TEMPLATE/bugfix.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
# SPDX-FileCopyrightText: 2024 Deutsche Telekom AG, LlamaIndex, Vercel, Inc.
#
# SPDX-License-Identifier: MIT

name: Bugfix
title: ""
description: Write something like "We notice ... behavior when ... happens instead of ..."
Expand Down
4 changes: 0 additions & 4 deletions .github/ISSUE_TEMPLATE/feature.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
# SPDX-FileCopyrightText: 2024 Deutsche Telekom AG, LlamaIndex, Vercel, Inc.
#
# SPDX-License-Identifier: MIT

name: Feature Request
title: ""
description: Write something like "Write an api endpoint that does "..." in the "..." file".
Expand Down
4 changes: 0 additions & 4 deletions .github/ISSUE_TEMPLATE/refactor.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
# SPDX-FileCopyrightText: 2024 Deutsche Telekom AG, LlamaIndex, Vercel, Inc.
#
# SPDX-License-Identifier: MIT

name: Refactor
title: ""
description: Write something like "Modify the ... api endpoint to use ... version and ... framework"
Expand Down
18 changes: 12 additions & 6 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
# SPDX-FileCopyrightText: 2024 Deutsche Telekom AG, LlamaIndex, Vercel, Inc.
#
# SPDX-License-Identifier: MIT

name: E2E Tests
on:
push:
Expand All @@ -28,36 +24,46 @@ jobs:
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4

- name: Set up python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: ${{ env.POETRY_VERSION }}
- uses: pnpm/action-setup@v2

- uses: pnpm/action-setup@v3

- name: Setup Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: "pnpm"

- name: Install dependencies
run: pnpm install

- name: Install Playwright Browsers
run: pnpm exec playwright install --with-deps
working-directory: .

- name: Build create-llama
run: pnpm run build
working-directory: .

- name: Install
run: pnpm run pack-install
working-directory: .

- name: Run Playwright tests
run: pnpm run e2e
env:
TSI_API_KEY: ${{ secrets.TSI_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
working-directory: .

- uses: actions/upload-artifact@v3
if: always()
with:
Expand Down
13 changes: 6 additions & 7 deletions .github/workflows/lint_on_push_or_pull.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
# SPDX-FileCopyrightText: 2024 Deutsche Telekom AG, LlamaIndex, Vercel, Inc.
#
# SPDX-License-Identifier: MIT

name: Lint on push or pull request

on:
Expand All @@ -17,17 +13,20 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v2
with:
version: latest

- uses: pnpm/action-setup@v3

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version-file: ".nvmrc"
cache: "pnpm"

- name: Install dependencies
run: pnpm install

- name: Run lint
run: pnpm run lint

- name: Run Prettier
run: pnpm run format
36 changes: 36 additions & 0 deletions .github/workflows/publish_github.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Publish to GitHub Releases

on:
push:
tags:
- "v*"

jobs:
build-and-publish:
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout@v4

- uses: pnpm/action-setup@v3

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version-file: ".nvmrc"
cache: "pnpm"

- name: Install dependencies
run: pnpm install

- name: Build tarball
run: |
pnpm pack

- name: Create release
uses: ncipollo/release-action@v1
with:
artifacts: "create-llama-*.tgz"
name: Release ${{ github.ref }}
bodyFile: "CHANGELOG.md"
token: ${{ secrets.GITHUB_TOKEN }}
55 changes: 55 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
name: Release

on:
push:
branches:
- main

concurrency: ${{ github.workflow }}-${{ github.ref }}

jobs:
release:
name: Release
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout@v4

- uses: pnpm/action-setup@v3

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version-file: ".nvmrc"
cache: "pnpm"

- name: Install dependencies
run: pnpm install

- name: Add auth token to .npmrc file
run: |
cat << EOF >> ".npmrc"
//registry.npmjs.org/:_authToken=$NPM_TOKEN
EOF
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

- name: Get changeset status
id: get-changeset-status
run: |
pnpm changeset status --output .changeset/status.json
new_version=$(jq -r '.releases[0].newVersion' < .changeset/status.json)
rm -v .changeset/status.json
echo "new-version=${new_version}" >> "$GITHUB_OUTPUT"

- name: Create Release Pull Request or Publish to npm
id: changesets
uses: changesets/action@v1
with:
commit: Release ${{ steps.get-changeset-status.outputs.new-version }}
title: Release ${{ steps.get-changeset-status.outputs.new-version }}
# build package and call changeset publish
publish: pnpm release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
4 changes: 0 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
# SPDX-FileCopyrightText: 2024 Deutsche Telekom AG, LlamaIndex, Vercel, Inc.
#
# SPDX-License-Identifier: MIT

# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.

# dependencies
Expand Down
4 changes: 0 additions & 4 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
# SPDX-FileCopyrightText: 2024 Deutsche Telekom AG, LlamaIndex, Vercel, Inc.
#
# SPDX-License-Identifier: MIT

apps/docs/i18n
apps/docs/docs/api
pnpm-lock.yaml
Expand Down
42 changes: 30 additions & 12 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,50 @@
<!--
# create-llama

SPDX-FileCopyrightText: 2024 Deutsche Telekom AG, LlamaIndex, Vercel, Inc.
## 0.1.2

SPDX-License-Identifier: MIT
-->
### Patch Changes

# create-tsi
- 056e376: Add support for displaying tool outputs (including weather widget as example)

## 0.0.18
## 0.1.1

### Patch Changes

- ddbc80f: Fix missing brave and duckduckgo search tools
- 7bd3ed5: Support Anthropic and Gemini as model providers
- 7bd3ed5: Support new agents from LITS 0.3
- cfb5257: Display events (e.g. retrieving nodes) per chat message

## 0.0.17
## 0.1.0

### Minor Changes

- f1c3e8d: Add Llama3 and Phi3 support using Ollama

### Patch Changes

- 06fdd7f: Add support for LLM Hub v2
- a0dec80: Use `gpt-4-turbo` model as default. Upgrade Python llama-index to 0.10.28
- 753229d: Remove asking for AI models and use defaults instead (OpenAIs GPT-4 Vision Preview and Embeddings v3). Use `--ask-models` CLI parameter to select models.
- 1d78202: Add observability for Python
- 6acccd2: Use poetry run generate to generate embeddings for FastAPI
- 9efcffe: Use Settings object for LlamaIndex configuration
- 418bf9b: refactor: use tsx instead of ts-node
- 1be69a5: Add Qdrant support

## 0.0.16
## 0.0.32

### Patch Changes

- ce2f24d: Update loaders and tools config to yaml format
- 625ed4d: Support Astra VectorDB
- 922e0ce: Remove UI question (use shadcn as default). Use `html` UI by calling create-llama with --ui html parameter
- ce2f24d: Update loaders and tools config to yaml format (for Python)
- e8db041: Let user select multiple datasources (URLs, files and folders)
- 78ded9e: Add Dockerfile template
- c06d4af: Add nodes to the response (Python)
- 29b17ee: Allow using agents without any data source
- 665c26c: Add redirect to documentation page when accessing the base URL (FastAPI)
- 78ded9e: Add Dockerfile templates for Typescript and Python
- 99e758f: Merge non-streaming and streaming template to one
- b3f2685: Add support for agent generation for Typescript
- 2739714: Use a database (MySQL or PostgreSQL) as a data source

## 0.0.31

Expand Down
8 changes: 1 addition & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,3 @@
<!--
SPDX-FileCopyrightText: 2024 Deutsche Telekom AG, LlamaIndex, Vercel, Inc.

SPDX-License-Identifier: MIT
-->

# Contributing

## Getting Started
Expand All @@ -13,7 +7,7 @@ Install NodeJS. Preferably v18 using nvm or n.
Inside the `create-llama` directory:

```
npm i -g pnpm ts-node
npm i -g pnpm
pnpm install
```

Expand Down
9 changes: 9 additions & 0 deletions LICENSE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
The MIT License (MIT)

Copyright (c) 2023 LlamaIndex, Vercel, Inc.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Loading