Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

JugglingDB Support #153

Open
wants to merge 4 commits into
from

Conversation

Projects
None yet
2 participants

I added support for the ORM framework JugglingDB which creates a multi-database compatibility layer. The api is very similar to mongoose making it easier to transition to. IMO this will add even more flexibility to calipso.

This PR isn't completely ready yet but I did want to create it to get some feedback and possibly help with testing.

I've ran the tests and they all pass on mongodb and the in-place memory adapter. The redis adapter does not work when assigning roles because it won't store an array which I'll hopefully fix in the juggling adapter. I haven't been able to test the rest of the adapters though it shouldn't be to hard to do.

Let me know what you think.

Collaborator

richtera commented Jul 21, 2012

Could you rebase this? I am sorry but I merged a large commit of mine. I would like to quickly try this to see if it makes sense to merge or put it on another branch.

Just rebased it. I wasn't able to test it because I'm pretty busy with other projects currently. If I can do anything to help let me know and I should be able to work some time in.

Collaborator

richtera commented Jul 21, 2012

Ok I'll let you know. I might pull it to a new branch and test it.
Andy

On Jul 21, 2012, at 7:11 PM, Cameron Bytheway wrote:

Just rebased it. I wasn't able to test it because I'm pretty busy with other projects currently. If I can do anything to help let me know and I should be able to work some time in.


Reply to this email directly or view it on GitHub:
#153 (comment)

Collaborator

richtera commented Aug 29, 2012

Sorry, I did not get around to testing this. Are there other projects using jugglingdb yet? I actually have some other projects this might be interesting for as well.

Yeah railway.js uses it as its ORM.

Collaborator

richtera commented Oct 21, 2012

This is still pending, sorry. I kind of got caught up in windows specific things and wanted support for google, twitter and facebook auth.

Collaborator

richtera commented Oct 29, 2012

I am planning to do this next; getting windows support and everyauth took longer than expected. If you can rebase this it would be greatly appreciated, otherwise I am pretty sure I can figure it out.

No problem. I'll do that later today.

Collaborator

richtera commented Nov 6, 2012

I created a branch "CamShaft-jugglingdb" with the merge. I am having some problems with it and some comments.

  1. Somehow I can't login using it but it's also not throwing any errors. I'll need to debug but maybe you can check it out.
  2. Would it be possible to assume {where:{...}} if you only get {...} so that existing findOne or find doesn't need to change? This is important because other modules would need to be rewritten even through almost 100% of them are just doing a plain where.
Collaborator

richtera commented Nov 6, 2012

FYI the active branch in calipso is "master". The "devel" branch is a little behind currently.

Collaborator

richtera commented Nov 6, 2012

I am having a problem at https://gist.github.com/4021763 on line 22.
Basically it seems that the Schema can't be used as a constructor but tries to create a new schema.
This tries to decode a value as a type. Any suggestions?

GOOD NEWS! This redis adapter version is deprecated, use redis2 instead. A lot of improvements, and new indexes incompatible with old (sorry about that): now we only store id and not ModelName:id in indexes. Also dates format in indexes changed to unix timestamp for better sorting and filtering performance
5 Nov 19:58:25 - info: Installing module content

welcome, message, default

I added a console.log(self.__data[attr]) at node_modules/jugglingdb/lib/abstract-class.js:82

SyntaxError: Unexpected token w
    at Object.parse (native)
    at AbstractClass._initProperties.def (/Users/andy/calipso/node_modules/jugglingdb/lib/abstract-class.js:82:46)
    at Array.forEach (native)
    at Function.Schema.define.NewClass.forEachProperty (/Users/andy/calipso/node_modules/jugglingdb/lib/schema.js:193:33)
    at ModelConstructor.AbstractClass._initProperties (/Users/andy/calipso/node_modules/jugglingdb/lib/abstract-class.js:74:10)
    at ModelConstructor.AbstractClass (/Users/andy/calipso/node_modules/jugglingdb/lib/abstract-class.js:30:10)
    at new ModelConstructor (/Users/andy/calipso/node_modules/jugglingdb/lib/schema.js:153:23)
    at Object.install (/Users/andy/calipso/modules/core/content/content.js:989:12)
    at /Users/andy/calipso/modules/core/admin/admin.js:540:40
    at Array.forEach (native)
5 Nov 19:58:25 - error: Module 'content' was unable to be installed... Reason: Object welcome, message, default has no method 'forEach'
Collaborator

richtera commented Nov 6, 2012

FYI same thing happens with memory database type. Just so you don't specifically check redis:

5 Nov 20:30:23 - info: Installing module content
SyntaxError: Unexpected token w
    at Object.parse (native)
    at AbstractClass._initProperties.def (/Users/andy/calipso/node_modules/jugglingdb/lib/abstract-class.js:81:46)
    at Array.forEach (native)
    at Function.Schema.define.NewClass.forEachProperty (/Users/andy/calipso/node_modules/jugglingdb/lib/schema.js:193:33)
    at ModelConstructor.AbstractClass._initProperties (/Users/andy/calipso/node_modules/jugglingdb/lib/abstract-class.js:74:10)
    at ModelConstructor.AbstractClass (/Users/andy/calipso/node_modules/jugglingdb/lib/abstract-class.js:30:10)
    at new ModelConstructor (/Users/andy/calipso/node_modules/jugglingdb/lib/schema.js:153:23)
    at Object.install (/Users/andy/calipso/modules/core/content/content.js:989:12)
    at /Users/andy/calipso/modules/core/admin/admin.js:540:40
    at Array.forEach (native)
5 Nov 20:30:23 - error: Module 'content' was unable to be installed... Reason: Object welcome, message, default has no method 'forEach'
5 Nov 20:30:23 - info: Installing module contentTypes
Collaborator

richtera commented Nov 6, 2012

Figured out this particular problem. The defaults for the initial contents records didn't have an Array as the initial value for the tags property. Your lib was trying to parse it as Json, I suppose mongoose tries to split a string by "," and uses that.

Collaborator

richtera commented Nov 6, 2012

Having problems with strings. When I try to serialize the user object into the cookie it looks like this:

{"username":"andy","isAdmin":false,"id":1,"language":"en","roles":[{"0":"A","1":"d","2":"m","3":"i","4":"n","5":"i","6":"s","7":"t","8":"r","9":"a","10":"t","11":"o","12":"r","bold":"\u001b[1mAdministrator\u001b[22m","underline":"\u001b[4mAdministrator\u001b[24m","italic":"\u001b[3mAdministrator\u001b[23m","inverse":"\u001b[7mAdministrator\u001b[27m","grey":"\u001b[90mAdministrator\u001b[39m","black":"\u001b[30mAdministrator\u001b[39m","yellow":"\u001b[33mAdministrator\u001b[39m","red":"\u001b[31mAdministrator\u001b[39m","green":"\u001b[32mAdministrator\u001b[39m","blue":"\u001b[34mAdministrator\u001b[39m","white":"\u001b[37mAdministrator\u001b[39m","cyan":"\u001b[36mAdministrator\u001b[39m","magenta":"\u001b[35mAdministrator\u001b[39m","rainbow":"\u001b[31mA\u001b[39m\u001b[33md\u001b[39m\u001b[32mm\u001b[39m\u001b[34mi\u001b[39m\u001b[35mn\u001b[39m\u001b[31mi\u001b[39m\u001b[33ms\u001b[39m\u001b[32mt\u001b[39m\u001b[34mr\u001b[39m\u001b[35ma\u001b[39m\u001b[31mt\u001b[39m\u001b[33mo\u001b[39m\u001b[32mr\u001b[39m","zebra":"A\u001b[7md\u001b[27mm\u001b[7mi\u001b[27mn\u001b[7mi\u001b[27ms\u001b[7mt\u001b[27mr\u001b[7ma\u001b[27mt\u001b[7mo\u001b[27mr","zalgo":"A̸͕̖̲̲̝̙̱̝̯̯͖̱̰͍̍ͪ̇̅ͥ̑̄̾̈̇̌̌ͤ̽͋̑d̦͓͈̠̤͚̬͚̦̯̮͎̘͇̲̣̱̩̻̬̬͖̯̺̮̼̗̗͍̠͖͇̣͚̱͎͍̤̝̞̻̗͉͚̣̤̮͔͚̯̹̱͚̽ͫ͋ͬͨͧ͜m̷̗̺̲̰̙̩̯̟̹͔̙̼̭̫̰͙̜̻͍̮̮͚͍͚͍̙̬̦̻̖̻̹͕̦̊̉̎̔ͭ͊̈̈͊̍͋́ͦ͋̍ͅi̛̬͙̦̫̲̳̣͈͇̬̮̤̣̲̙͇̪̲̦̜̩͕̯̼̖̟̓ͧͩ̓n͉̲̯̯͇͔̄ͧ̌ͯͤ͊̂̎̇̅ͭ̈̓͗̽ͭ͞ͅi̬͖͎̯͚͓͕͈̬̮̜̠͓̩̻̲̙̲̠̳̻̥̥̞̳̩̜͚͚̪̯̬̪̤͙͙̲͔̪̝͖̓̿͌̓͂̅̃̈̎̀s̶̠͎̣̫̱̮̭̩̪̬͇̙̤̣͕̟̱̻͉̻̣̥̬̞̠̯͕̫ͤ́̉̌͋̾ͅt̸̟̪̱̬͎̟̟̳̹̯̲̝̦̼͎͎͙̑ͩ̏ͦͣͥͤ̄̓̎ͨ̏ͩ̇͆̀̔ͅȓ̡͍͎̼̭͎̠͓̪́̏̿͗͊ͥͮ̋ͫ̂̅͊á̫̯̰̼͕̤̻͎̝͔̖͓̲̠̜̔̾ͮͪ̽͊̈̄́̀̃̈͠ͅt̜̬̭̥̺̦̩̗̞̭̲͇̤͖̳̳̰̬̺̫͈̏́ͣ͋̋̋̽̓ͧ̓ͤ͡o̞̲̙͚̹̦͈̺̬̱̝̳̞͕͈̞̪̮̖̤̭͔̤̭̬̼̫̘̦̲̖̪̬̜͖̠̗̯͇̙͈̾̌̂͗̾̓ͬ̍ͣͨ̎ͩ̍̀̾͠ȓ̟̪͇̻͕̰̪̦̯̤͇ͭͤ̄́ͅ","stripColors":"Administrator","id":1}]}

Which seems to be due to the fact the "new String('something')" in node is not a string but an object of type String. I tried to find out on the node chat to see if this is recent but I doubt it.

Collaborator

richtera commented Nov 6, 2012

I think there are problems with the List class. Adding an itemType property to the roles definition makes it work a tiny bit better but it's still broken. The redis2 adapter doesn't work unless indexes are defined and the redis adapter says it's deprecated. Boolean values are converted to strings of "true" and "false" and therefore "false" will show up as true. Do you have any suggestions on how to proceed? Or should we just wait?

camshaft commented Nov 6, 2012

ATM I don't have much time to look at it. I'm not using calipso anymore so it isn't a high priority for me.

That being said, if you feel like this is something that it's something you guys want, I can set aside some time to look at it.

Collaborator

richtera commented Nov 6, 2012

Do any of the serialization problems seem familiar or already fixed in jugglingdb?
For example saving
{ username:"Andy", locked:false, roles:["Administrator"] }
to redis or memory comes back as
{ username:"Andy", locked: "false", roles:[{"0":"A","1":"d",....}]}
I do have a little time available but I just want to make sure I am not fighting an uphill battle; I will still request help if I need it.
Thanks
Andy

On Nov 6, 2012, at 12:11 PM, Cameron Bytheway notifications@github.com wrote:

ATM I don't have much time to look at it. I'm not using calipso anymore so it isn't a high priority for me.

That being said, if you feel like this is something that it's something you guys want, I can set aside some time to look at it.


Reply to this email directly or view it on GitHub.

camshaft commented Nov 6, 2012

I have seen that issue with it before and it probably should be fixed upstream. It's an issue with those drivers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment