Skip to content

Commit

Permalink
✨ [Tasks] JSON Schema spec for Inference types + TS type generation (#…
Browse files Browse the repository at this point in the history
…449)

# TL;DR

- Introduces specification for inference input/output types for each
task, in JSON Schema format
- Add a "script" to generate typescript code from that JSON spec using
[`quicktype-core`](https://github.com/glideapps/quicktype/tree/master/packages/quicktype-core)
- Introduces `quicktype-core` as a dev dependency (from our fork of
quicktype
https://github.com/huggingface/quicktype/releases/tag/pack-18.0.15)
    - License: Apache-2.0


# TODO

- [x] Add a `text2text-generation` task to serve as a "canonical
reference" for `summarization` & `translation`
- [x] Add a `text-to-audio` task to server as a "canonical reference"
for `text-to-speech`
- [ ] Code generation tweaks
   - [x] Change `any` types to `unknown`
   - [x] Generate output types for `sentence-similarity`
- [ ] Generate output types for `feature-extraction` -> Let's do that
later?
- [x] add placeholder specs
  • Loading branch information
SBrandeis committed Jan 26, 2024
1 parent c51dea8 commit b70ac6c
Show file tree
Hide file tree
Showing 96 changed files with 4,355 additions and 3 deletions.
5 changes: 5 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,10 @@
},
"[svelte]": {
"editor.defaultFormatter": "esbenp.prettier-vscode"
},
"prettier.configPath": ".prettierrc",
"json.format.enable": false,
"[json]": {
"editor.defaultFormatter": "esbenp.prettier-vscode"
}
}
10 changes: 7 additions & 3 deletions packages/tasks/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,10 @@
"format": "prettier --write .",
"format:check": "prettier --check .",
"prepublishOnly": "pnpm run build",
"build": "tsup src/index.ts --format cjs,esm --clean --dts",
"build": "tsup src/index.ts src/scripts/**.ts --format cjs,esm --clean --dts",
"prepare": "pnpm run build",
"check": "tsc"
"check": "tsc",
"inference-codegen": "pnpm run build && node dist/scripts/inference-codegen.js"
},
"files": [
"dist",
Expand All @@ -40,5 +41,8 @@
],
"author": "Hugging Face",
"license": "MIT",
"devDependencies": {}
"devDependencies": {
"@types/node": "^20.11.5",
"quicktype-core": "https://github.com/huggingface/quicktype/raw/pack-18.0.15/packages/quicktype-core/quicktype-core-18.0.15.tgz"
}
}
209 changes: 209 additions & 0 deletions packages/tasks/pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit b70ac6c

Please sign in to comment.