Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(nodejs-polars): bindings for nodejs #1703

Merged
merged 90 commits into from
Dec 20, 2021
Merged
Show file tree
Hide file tree
Changes from 80 commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
d2904d5
feat(js-polars): POC for js-polars bindings
woodroach Nov 8, 2021
19bc114
add basic series methods and ts wrapper for implemented methods
universalmind303 Nov 11, 2021
5dd61e6
resolve merge conflicts
universalmind303 Nov 11, 2021
cc36e29
series: working on mapping all of the functions from py-polars.series…
universalmind303 Nov 12, 2021
a8e8ca6
adding more "series" methods
universalmind303 Nov 13, 2021
cd35ba3
add object conversion for 'series'
universalmind303 Nov 14, 2021
e60f76d
nsplitting conversions into a pkg
universalmind303 Nov 15, 2021
8a1e905
refactor: split conversion into multiple files
universalmind303 Nov 15, 2021
7339029
add more "series" methods
universalmind303 Nov 18, 2021
9ca77f4
finish implementing "series"
universalmind303 Nov 21, 2021
7364f89
refactor series for easy doc generating
universalmind303 Nov 22, 2021
afb1075
Merge branch 'master' of https://github.com/pola-rs/polars
universalmind303 Nov 22, 2021
62c2fc2
wip on dataframes
universalmind303 Nov 24, 2021
1e063b4
groupby method
universalmind303 Nov 28, 2021
8569733
refactor "series" to object for better extensibility
universalmind303 Nov 28, 2021
07c4a60
add a bunch more tests
universalmind303 Nov 29, 2021
4a2e41d
Merge branch 'master' of https://github.com/pola-rs/polars
universalmind303 Nov 29, 2021
616866d
revert accidental change to polars-core
universalmind303 Nov 29, 2021
24ff71a
chore: finish up tests for series, and start on expr
universalmind303 Nov 30, 2021
c2c3dd8
Merge branch 'master' of https://github.com/pola-rs/polars
universalmind303 Nov 30, 2021
a36ac7a
still adding more functionality
universalmind303 Dec 2, 2021
a653782
merge from main
universalmind303 Dec 2, 2021
7293ebe
a lot more tests
universalmind303 Dec 4, 2021
067fdc5
Merge branch 'master' of https://github.com/pola-rs/polars
universalmind303 Dec 4, 2021
3d20f2c
add tests for groupby methods
universalmind303 Dec 5, 2021
946e87c
Merge branch 'master' of https://github.com/pola-rs/polars
universalmind303 Dec 5, 2021
8650e54
Merge branch 'master' of https://github.com/pola-rs/polars
universalmind303 Dec 8, 2021
d51f959
finish tests for dataframe
universalmind303 Dec 8, 2021
86eaf49
chore: add more tests
universalmind303 Dec 9, 2021
c7ba81f
Merge branch 'master' of https://github.com/pola-rs/polars
universalmind303 Dec 9, 2021
abb3dbb
feat: add whenthen
universalmind303 Dec 11, 2021
f4cd175
add clone to when/then
universalmind303 Dec 11, 2021
abe38a1
Merge branch 'master' of https://github.com/pola-rs/polars
universalmind303 Dec 11, 2021
a481d29
chore: add more tests
universalmind303 Dec 11, 2021
94f84ad
chore: add a ton more tests
universalmind303 Dec 14, 2021
05143c3
Merge branch 'master' of https://github.com/pola-rs/polars
universalmind303 Dec 14, 2021
9bc62be
add cfg and update fork
universalmind303 Dec 14, 2021
cbc0513
gh actions wip
universalmind303 Dec 15, 2021
df699fe
gh actions wip
universalmind303 Dec 15, 2021
05156d8
gh actions wip
universalmind303 Dec 15, 2021
94ff171
gh actions wip
universalmind303 Dec 15, 2021
39f45be
gh actions wip
universalmind303 Dec 15, 2021
0f93659
gh actions wip
universalmind303 Dec 15, 2021
e727153
gh actions wip
universalmind303 Dec 15, 2021
9fc74eb
wip: ci
universalmind303 Dec 15, 2021
9cd412d
wip: ci
universalmind303 Dec 15, 2021
64f4efc
wip: ci
universalmind303 Dec 15, 2021
3f872c6
wip: ci
universalmind303 Dec 15, 2021
13dccc7
wip: ci
universalmind303 Dec 15, 2021
6dc5faf
wip: ci
universalmind303 Dec 15, 2021
91db515
wip: ci
universalmind303 Dec 15, 2021
fe60ec1
wip: ci
universalmind303 Dec 15, 2021
2841168
wip: ci
universalmind303 Dec 15, 2021
c24791d
debugging gh actions
universalmind303 Dec 15, 2021
63c55d5
debugging gh actions
universalmind303 Dec 15, 2021
7a2d367
debugging workflow
universalmind303 Dec 15, 2021
1cc01a2
debugging workflow
universalmind303 Dec 15, 2021
25640f1
debugging workflow
universalmind303 Dec 15, 2021
f6bfcef
debuggint workflows
universalmind303 Dec 15, 2021
ff551dc
debuggint workflows
universalmind303 Dec 15, 2021
bba4bb1
debuggint workflows
universalmind303 Dec 15, 2021
f83ed9f
debugging still
universalmind303 Dec 15, 2021
d09a867
debugging still
universalmind303 Dec 15, 2021
c34610b
fix workflow
universalmind303 Dec 15, 2021
ca32e0f
fix workflow
universalmind303 Dec 15, 2021
75b0cb8
fix workflow
universalmind303 Dec 15, 2021
df7db54
fix workflow
universalmind303 Dec 15, 2021
efd283c
fix workflow
universalmind303 Dec 15, 2021
7ae4ac9
fix workflow
universalmind303 Dec 15, 2021
adc772f
test npm publish
universalmind303 Dec 15, 2021
e2ec908
test npm publish
universalmind303 Dec 15, 2021
2aa5896
test npm publish
universalmind303 Dec 15, 2021
778d76d
test npm publish
universalmind303 Dec 15, 2021
e0f16d9
test publish again
universalmind303 Dec 15, 2021
da9d4a0
finish gh workflow and code cleaunp
universalmind303 Dec 16, 2021
76d14e7
Merge branch 'master' of https://github.com/pola-rs/polars
universalmind303 Dec 16, 2021
e8a633f
refactor: wrapper objects for consistency
universalmind303 Dec 16, 2021
6643430
docs: update readme
universalmind303 Dec 16, 2021
abb3544
docs: update readme
universalmind303 Dec 16, 2021
33c459c
chore: add original "js-polars" back in
universalmind303 Dec 17, 2021
0dfd411
wip: df and series proxies
universalmind303 Dec 19, 2021
74ecc64
feat: df "get" proxy"
universalmind303 Dec 19, 2021
9d9ff41
cargo fmt
universalmind303 Dec 19, 2021
2e9c3eb
Merge branch 'master' of https://github.com/pola-rs/polars
universalmind303 Dec 19, 2021
eaacf06
chore: code cleanup
universalmind303 Dec 19, 2021
14829ed
chore: code cleanup
universalmind303 Dec 19, 2021
28d1f05
chore: code cleanup
universalmind303 Dec 19, 2021
7ca1d8c
chore: run "make fmt_toml"
universalmind303 Dec 19, 2021
de30ab6
chore: remove "date_buckets" expr
universalmind303 Dec 19, 2021
f2b7e88
use nanos for dates
universalmind303 Dec 19, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
164 changes: 164 additions & 0 deletions .github/workflows/create-js-release.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
name: nodejs-polars

on:
push:
# Sequence of patterns matched against refs/tags
tags:
- 'nodejs-polars-v*' # Push events to matching nodejs-polars-v*, i.e. nodejs-polars-v1.0, nodejs-polars-v20.15.10
jobs:
build-js:
defaults:
run:
working-directory: nodejs-polars
strategy:
fail-fast: false
matrix:
settings:
# MAC
######

- host: macos-latest
target: 'x86_64-apple-darwin'
build: |
npm run build

- host: macos-latest
target: "aarch64-apple-darwin"
build: |
sudo rm -Rf /Library/Developer/CommandLineTools/SDKs/*;
export CC=$(xcrun -f clang);
export CXX=$(xcrun -f clang++);
SYSROOT=$(xcrun --sdk macosx --show-sdk-path);
export CFLAGS="-isysroot $SYSROOT -isystem $SYSROOT";
npm run build -- --target aarch64-apple-darwin

# Windows
##########

- host: windows-latest
target: "x86_64-pc-windows-msvc"
build: npm run build

- host: windows-latest
target: "i686-pc-windows-msvc"
setup: |
choco install nodejs-lts --x86 -y --force
echo "C:\\Program Files (x86)\\nodejs" >> $GITHUB_PATH
build: |
export CARGO_PROFILE_RELEASE_CODEGEN_UNITS=32;
export CARGO_PROFILE_RELEASE_LTO=false
npm run build -- --target i686-pc-windows-msvc

- host: windows-latest
target: "aarch64-pc-windows-msvc"
build: npm run build -- --target aarch64-pc-windows-msvc

# Linux
########

- host: ubuntu-latest
target: "x86_64-unknown-linux-gnu"
build: npm run build

- host: ubuntu-latest
target: "aarch64-linux-android"
build: |
export CARGO_TARGET_AARCH64_LINUX_ANDROID_LINKER="${ANDROID_NDK_HOME}/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android24-clang"
export PATH="${ANDROID_NDK_HOME}/toolchains/llvm/prebuilt/linux-x86_64/bin:${PATH}"
npm run build -- --target aarch64-linux-android

name: ${{ matrix.settings.target }}
runs-on: ${{ matrix.settings.host }}
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: 16
- name: Install latest Rust nightly
uses: actions-rs/toolchain@v1
with:
toolchain: nightly-2021-12-02
override: true
target: ${{ matrix.settings.target }}

- name: Generate Cargo.lock
uses: actions-rs/cargo@v1
with:
command: generate-lockfile

- name: Cache cargo registry
uses: actions/cache@v2
with:
path: ~/.cargo/registry
key: ${{ matrix.settings.target }}-node@16-cargo-registry-trimmed-${{ hashFiles('**/Cargo.lock') }}

- name: Cache cargo index
uses: actions/cache@v2
with:
path: ~/.cargo/git
key: ${{ matrix.settings.target }}-node@16-cargo-index-trimmed-${{ hashFiles('**/Cargo.lock') }}

- name: Pull latest image
run: ${{ matrix.settings.docker }}
env:
DOCKER_REGISTRY_URL: ghcr.io
DOCKER_USERNAME: ${{ github.actor }}
DOCKER_PASSWORD: ${{ secrets.GITHUB_TOKEN }}

if: ${{ matrix.settings.docker }}

- name: "Setup toolchain"
if: ${{ matrix.settings.setup }}
run: ${{ matrix.settings.setup }}

- name: "Install dependencies"
run: npm install --ignore-scripts

- name: "Build"
run: ${{ matrix.settings.build }}
shell: bash
- name: Upload artifact
uses: actions/upload-artifact@v2
with:
path: nodejs-polars/artifacts
- name: List artifacts
run: ls -R artifacts
shell: bash

publish:
name: Publish
needs:
- build-js
defaults:
run:
working-directory: nodejs-polars
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: "16"
registry-url: "https://registry.npmjs.org"

- name: "Install dependencies"
run: npm install --ignore-scripts

- name: Download all artifacts
uses: actions/download-artifact@v2
with:
path: nodejs-polars/artifacts

- name: List artifacts
run: ls -R artifacts
shell: bash

- name: Move artifacts
run: npm run artifacts

- name: Build TypeScript
run: npm run build:ts

- name: Publish to npm
run: npm publish
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,5 @@ polars/vendor
*.so
.ENV
.env
AUTO_CHANGELOG.md
AUTO_CHANGELOG.md
node_modules/
10 changes: 10 additions & 0 deletions nodejs-polars/.editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
root = true

[*]
end_of_line = lf
insert_final_newline = true

[*.{js,json,yml}]
charset = utf-8
indent_style = space
indent_size = 2
83 changes: 83 additions & 0 deletions nodejs-polars/.eslintrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
{
"env": {
"browser": true,
"es2021": true,
"jest/globals": true
},
"globals": {
"Buffer": "readonly",
"process": "writable"
},
"extends": [
"eslint:recommended"
],
"parser": "@typescript-eslint/parser",
"parserOptions": {
"ecmaVersion": 2020,
"sourceType": "module"
},
"plugins": [
"@typescript-eslint",
"jest"
],
"rules": {
"no-inner-declarations": "off",
"func-call-spacing": "off",
"@typescript-eslint/func-call-spacing": [
"error"
],
"no-duplicate-imports": "off",
"@typescript-eslint/no-duplicate-imports": [
"error"
],
"comma-spacing": "off",
"@typescript-eslint/comma-spacing": [
"error"
],
"quotes": [
"error",
"double",
{
"allowTemplateLiterals": true,
"avoidEscape": true
}
],
"no-redeclare": "off",
"no-trailing-spaces": "error",
"semi": [
"error",
"always"
],
"indent": [
"error",
2,
{
"MemberExpression": 1
}
],
"no-dupe-class-members": "off",
"no-unused-vars": "off",
"newline-per-chained-call": [
"error",
{
"ignoreChainWithDepth": 2
}
],
"@typescript-eslint/member-ordering": "error",
"no-multiple-empty-lines": [
"error",
{
"max": 2,
"maxEOF": 0
}
],
"padding-line-between-statements": [
"error",
{
"blankLine": "always",
"prev": "*",
"next": "return"
}
]
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you run a formatter that adds a newline char to all the files?

14 changes: 14 additions & 0 deletions nodejs-polars/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
/target
**/*.rs.bk
Cargo.lock
node_modules/
.yarn/cache
.yarn/install-state*
.pnp.*

coverage/
bin/
.vscode
.cargo
*.node
docs/
9 changes: 9 additions & 0 deletions nodejs-polars/.npmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
target
Cargo.lock
.cargo
.github
npm
# .eslintrc
.prettierignore
rustfmt.toml
yarn.lock
1 change: 1 addition & 0 deletions nodejs-polars/.nvmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
16
768 changes: 768 additions & 0 deletions nodejs-polars/.yarn/releases/yarn-3.1.1.cjs

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions nodejs-polars/.yarnrc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
nodeLinker: node-modules
yarnPath: .yarn/releases/yarn-3.1.1.cjs
26 changes: 26 additions & 0 deletions nodejs-polars/@types/jest.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import {DataFrame} from "@polars/dataframe";
import {Series} from "@polars/series";

declare global {
namespace jest {
interface Matchers<R> {
toSeriesEqual(b: Series<any>): R;
toSeriesStrictEqual(b: Series<any>): R;
toFrameEqual(b: DataFrame): R;
/**
* Compares two DataFrames, including the dtypes
*
* @example
* ```
* >>> df = pl.Dataframe([pl.Series("int32": [1,2], pl.Int32)])
* >>> other = pl.Dataframe([pl.Series("int32": [1,2], pl.UInt32)])
*
* >>> expect(df).toFrameEqual(other) // passes
* >>> expect(df).toFrameStrictEqual(other) // fails
* ```
*/
toFrameStrictEqual(b: DataFrame): R;
toFrameEqualIgnoringOrder(b: DataFrame): R;
}
}
}