Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project organization #58

Closed
bkniffler opened this issue Jun 20, 2018 · 7 comments
Closed

Project organization #58

bkniffler opened this issue Jun 20, 2018 · 7 comments

Comments

@bkniffler
Copy link
Contributor

@bkniffler bkniffler commented Jun 20, 2018

Hey,
I really like nano-sql! Maybe it would make sense to make nano-sql a mono-repository, this way all packages (sqlite etc) could be maintained in a single place and changes to core, plugins, adapters could always be tested agains the whole repository. This could be coupled with scoped packages (@nano-sql/sqlite).

Also Currently the whole feature is hard coded with english language into the core. Wouldn't it make more sense to make it all a plugin that gets initiated with the tokenizer? This way we could add more languages.

@only-cliches
Copy link
Owner

@only-cliches only-cliches commented Jun 21, 2018

Hello Benjamin,

Bringing all the adapters into the same git repo is a good idea, I didn't know about .npmignore until recently so it should be very doable!

I haven't documented the search feature yet, custom tokenizers can be easily passed into nanoSQL to be used instead of the built in english tokenizer. The english tokenizer is hard coded because english speaking countries are by far the majority on GitHub/NPM and it contributes minimally to nanoSQL's bundle size.

You can easily pass in your own tokenizer with the config object:

nSLQ().config({
    tokenizer: (table, column, args, value) => {
        /**
         * Arguments:
         * table: the table that the row is in being tokenized
         * column: the current column being tokenized
         * args: array of arguments passed into the data model.  For example with props: ["search(english, 3)"] you would get ["english", "3"].
         * value: the string to tokenize, will be the entire unmodified column value.
         *
         * Expects:
         * Array of objects, each object contains three properties:
         * o: original word
         * w: tokenized word
         * i: zero based index of this word's location in the provided string
         *
         * You can also optionally return false to let the default tokenizer take over.
         */
        return value.split(" ").map((w, i) => ({w: w, o: w, i: i}));
   }
}).connect()..

@bkniffler
Copy link
Contributor Author

@bkniffler bkniffler commented Jun 21, 2018

Ah, thanks for the info. I'd still take them out (+ the whole fuzzysearch feature as plugin) since bundle size matters to many people a lot, but its just a recommendation!

Something like:

import nano, { DatabaseEvent, NanoSQLInstance } from "@nano/sql";
import fuzzy, { en as tokenizer } from "@nano/plugin-fuzzy"; // taking advantage of tree shaking and esmodules
import adapter from "@nano/adapter-sqlite";

nano().config({
    tokenizer,
    adapter
}).connect()

About the structure, I thought about a mono repo (using https://github.com/lerna/lerna) approach like https://github.com/gatsbyjs/gatsby, https://github.com/babel/babel, https://github.com/facebook/react etc (look into their respective 'packages' folders). I'd recommend putting all related projects in there, not only adapters. Its root would house the tsconfig, linting, tooling into the root so it can be shared by all projects and to reduce duplication.

So:

.gitignore
package.json
tsconfig.json
tslint.json
examples
- react
- ...
packages
- adapter-sqlite
  - lib
  - src
  - .npmignore
  - package.json
- sql or core
- react
- vue
- plugin-fuzzysearch
- plugin-sync
- ....

Here is a simple example and explanation of the advantages of lerna:
https://github.com/reggi/lerna-tutorial

Also you could consider using yarn and their workspaces feature
https://yarnpkg.com/blog/2017/08/02/introducing-workspaces/

@only-cliches
Copy link
Owner

@only-cliches only-cliches commented Jul 3, 2018

1.7.0 has centralized all the "stem" projects into a single git repo (this one). I'll delete the other repos and publish new packages to NPM with updated details soon.

@heri16
Copy link
Contributor

@heri16 heri16 commented Jul 12, 2018

Monorepos are great

@only-cliches
Copy link
Owner

@only-cliches only-cliches commented Jul 20, 2018

The transition is almost complete. New versions of all the sub projects have been published to NPM with updated github links, I've archived all the old git repos and copied over any open issues.

I'll probably wait another week then delete the git repos, we can close this issue once that happens.

@only-cliches
Copy link
Owner

@only-cliches only-cliches commented Jul 23, 2018

All of the old git repos have been removed, closing this issue now since we've fully migrated to the new monorepo setup.

@codebling
Copy link

@codebling codebling commented Apr 11, 2021

Can someone give some hints for building? lerna complains:

lerna ERR! ENOPKG `package.json` does not exist, have you run `lerna init`?

I'm able to get lerna somewhat working using these steps:

lerna init
lerna bootstrap
npm i -D typescript webpack ts-node

I'm still not able to build (lerna run build) or run tests (lerna run test) successfully. I'm running into issues with

> nano-sql-vue@1.3.0 build /tmp/nano2/packages/Nano-SQL-Vue
> tsc && webpack -p

../../node_modules/@types/node/assert.d.ts(8,68): error TS1144: '{' or ';' expected.
../../node_modules/@types/node/base.d.ts(13,1): error TS1084: Invalid 'reference' directive syntax.
../../node_modules/@types/node/events.d.ts(43,56): error TS1005: '=' expected.
../../node_modules/@types/node/ts3.6/base.d.ts(10,1): error TS1084: Invalid 'reference' directive syntax.

Parts of the log have been clipped for brevity. These are all type errors, most of them for the node types, so I suspect there is a Typescript version mismatch, but maybe I'm doing something else wrong.

Any suggestions would be appreciated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants