New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Couchdb support #109

Closed
BitPuffin opened this Issue Oct 12, 2012 · 22 comments

Comments

Projects
None yet
3 participants
@BitPuffin

BitPuffin commented Oct 12, 2012

Hey, it would be really awesome if you added couchdb support, it's really one of the only things keeping me away from vibe.d!

I could try and hack on it myself I guess, could be fun to take on :)

@s-ludwig

This comment has been minimized.

Show comment
Hide comment
@s-ludwig

s-ludwig Oct 12, 2012

Member

That would be great, of course, if you would give it a shot ;)

I'm far too busy with the existing functions and other stuff to tackle this within the coming months. I guess there is no existing D implementation of a couchdb driver? In that case a port would probably be pretty straight forward - but it seems like the protocol is quite simple to implement, maybe even utilizing the vibe.http.rest module... grr now it sounds like a fun project but things like a full win32 back end are waiting in line and then epoll and winrt and DoS protection and...

Member

s-ludwig commented Oct 12, 2012

That would be great, of course, if you would give it a shot ;)

I'm far too busy with the existing functions and other stuff to tackle this within the coming months. I guess there is no existing D implementation of a couchdb driver? In that case a port would probably be pretty straight forward - but it seems like the protocol is quite simple to implement, maybe even utilizing the vibe.http.rest module... grr now it sounds like a fun project but things like a full win32 back end are waiting in line and then epoll and winrt and DoS protection and...

@BitPuffin

This comment has been minimized.

Show comment
Hide comment
@BitPuffin

BitPuffin Oct 12, 2012

Sounds like I am going to have to take it on then if I want to use Vibe (it would be pretty bad ass). And the Couch API is really just a bunch of HTTP stuff and I need to learn Couch better so why not :)

BitPuffin commented Oct 12, 2012

Sounds like I am going to have to take it on then if I want to use Vibe (it would be pretty bad ass). And the Couch API is really just a bunch of HTTP stuff and I need to learn Couch better so why not :)

@BitPuffin

This comment has been minimized.

Show comment
Hide comment
@BitPuffin

BitPuffin Feb 24, 2013

I'm trying to use the rest module from vibe to generate an interface, however it's a bit tricky.

CouchDB's REST API goes kind of deep sometimes, and sometimes the url's require the use of underscores.

For example:
GET /_config/section/key
or
POST /db/_design/design-doc/_list/list-name/other-design-doc/view-name

Is there any way that I can match those patterns with interfaces for RestInterfaceClient?

I'm getting some ideas to perhaps wrap things in classes, it would be nice if you could point me in the right direction!

BitPuffin commented Feb 24, 2013

I'm trying to use the rest module from vibe to generate an interface, however it's a bit tricky.

CouchDB's REST API goes kind of deep sometimes, and sometimes the url's require the use of underscores.

For example:
GET /_config/section/key
or
POST /db/_design/design-doc/_list/list-name/other-design-doc/view-name

Is there any way that I can match those patterns with interfaces for RestInterfaceClient?

I'm getting some ideas to perhaps wrap things in classes, it would be nice if you could point me in the right direction!

@s-ludwig

This comment has been minimized.

Show comment
Hide comment
@s-ludwig

s-ludwig Feb 24, 2013

Member

Right now that would unfortunately require a class for each level, somehow like this, with MethodStyle.Unaltered:

class CDB { @property DB db() { return _db; } }
class DB { List get_list(string id) { return _databases[id].list; } }
class List { Doc get(string id) { return _docs[id]; } }
class Doc { Entry get(string id) { return _entries[id]; } }

Currently the magical id parameter is the only way to dynamically match parts of the URL and it is also not possible to a whole path dynamically (e.g. to match any sub path and get back the path as a string[]). But now with user defined attributes it could be made much more flexible, e.g.:

class DB { @restName!"_list" List getList(string id) { return _databases[id].list; } }
class List { Entry get(@pathMatch!0 string list, @pathMatch!1 string entry) { return _docs[list][entry]; }

But I didn't have the time to implement or even think about a design for this, yet. If you have any good ideas for a clean way to annotate all such things (or even would like to implement it), that would be good. But I hope to have some time for this within the next month if nothing gets in-between.

Member

s-ludwig commented Feb 24, 2013

Right now that would unfortunately require a class for each level, somehow like this, with MethodStyle.Unaltered:

class CDB { @property DB db() { return _db; } }
class DB { List get_list(string id) { return _databases[id].list; } }
class List { Doc get(string id) { return _docs[id]; } }
class Doc { Entry get(string id) { return _entries[id]; } }

Currently the magical id parameter is the only way to dynamically match parts of the URL and it is also not possible to a whole path dynamically (e.g. to match any sub path and get back the path as a string[]). But now with user defined attributes it could be made much more flexible, e.g.:

class DB { @restName!"_list" List getList(string id) { return _databases[id].list; } }
class List { Entry get(@pathMatch!0 string list, @pathMatch!1 string entry) { return _docs[list][entry]; }

But I didn't have the time to implement or even think about a design for this, yet. If you have any good ideas for a clean way to annotate all such things (or even would like to implement it), that would be good. But I hope to have some time for this within the next month if nothing gets in-between.

@s-ludwig

This comment has been minimized.

Show comment
Hide comment
@s-ludwig

s-ludwig Feb 24, 2013

Member

Hmm, and with class I meant interface, since this is for a client...

Member

s-ludwig commented Feb 24, 2013

Hmm, and with class I meant interface, since this is for a client...

@BitPuffin

This comment has been minimized.

Show comment
Hide comment
@BitPuffin

BitPuffin Feb 24, 2013

Well what initally seemed natural to me would be something like this:

interface ClientAPI {
    @property ConfigAPI _config(); 
    Json get(); // GET /
    DbAPI getDb(string db);
}

interface ConfigAPI() {
    Json get(); // GET /_config
    Json get(string section); // GET /_config/section
    Json get(string section, string key); // GET /_config/section/key  -- not sure if it's a good way
    void put(string section, string key, Json value); // PUT /_config/section/key with value as the body and content-type application/json
    Json delete(string section, string key); // DELETE /_config/section/key
}

interface DbAPI {
    @property Json _changen();
    // Here is where it falls apart: POST /db/_compact whick doesn't have a body, and the _ makes it ideal for a property
    // And how would I cleanly implement GET/PUT /db/_revs_limit
}

But as you see it eventually dies on me.

I guess what could be useful would be if there was a way to annotate what http method to use.

And the real problem is that afaic there is no quick and easy way to wrap a REST api manually. With my current WIP implementation (https://github.com/BitPuffin/DouchVibe) I use HttpClient which is sort of "low level" for something as simple as this and right now the result is pretty messy.

What would be pretty sweet would be if there was a kind of UrlRouter like utility for manually mapping a REST API to specific functions. That way we wouldn't have any trouble knowing if the intention is to use GET or POST or PUT or.....

Thoughts?

BitPuffin commented Feb 24, 2013

Well what initally seemed natural to me would be something like this:

interface ClientAPI {
    @property ConfigAPI _config(); 
    Json get(); // GET /
    DbAPI getDb(string db);
}

interface ConfigAPI() {
    Json get(); // GET /_config
    Json get(string section); // GET /_config/section
    Json get(string section, string key); // GET /_config/section/key  -- not sure if it's a good way
    void put(string section, string key, Json value); // PUT /_config/section/key with value as the body and content-type application/json
    Json delete(string section, string key); // DELETE /_config/section/key
}

interface DbAPI {
    @property Json _changen();
    // Here is where it falls apart: POST /db/_compact whick doesn't have a body, and the _ makes it ideal for a property
    // And how would I cleanly implement GET/PUT /db/_revs_limit
}

But as you see it eventually dies on me.

I guess what could be useful would be if there was a way to annotate what http method to use.

And the real problem is that afaic there is no quick and easy way to wrap a REST api manually. With my current WIP implementation (https://github.com/BitPuffin/DouchVibe) I use HttpClient which is sort of "low level" for something as simple as this and right now the result is pretty messy.

What would be pretty sweet would be if there was a kind of UrlRouter like utility for manually mapping a REST API to specific functions. That way we wouldn't have any trouble knowing if the intention is to use GET or POST or PUT or.....

Thoughts?

@mihails-strasuns

This comment has been minimized.

Show comment
Hide comment
@mihails-strasuns

mihails-strasuns Feb 24, 2013

Contributor

AFAIK,

Json get(string section);

is supposed to be either

Json getSection();

if it is meant to be exactly "section"string or

Json get(string id);

if it is meant to be GET /:section-name . But if multiple variables are nested, things get nasty.

@s-ludwig , if you have some time to write a spec for more advanced variable handling in REST urls, I can implement it for you, right in the middle of choosing "next important thing to code".

Contributor

mihails-strasuns commented Feb 24, 2013

AFAIK,

Json get(string section);

is supposed to be either

Json getSection();

if it is meant to be exactly "section"string or

Json get(string id);

if it is meant to be GET /:section-name . But if multiple variables are nested, things get nasty.

@s-ludwig , if you have some time to write a spec for more advanced variable handling in REST urls, I can implement it for you, right in the middle of choosing "next important thing to code".

@BitPuffin

This comment has been minimized.

Show comment
Hide comment
@BitPuffin

BitPuffin Feb 24, 2013

@Dicebot Well section is actually not in the url, it's like GET /_config/:section

Also I've ran into another issue, CouchDB uses an almost REST thing where it really only uses the rest it likes. So couch requires support for the HTTP method COPY which isn't in the HttpMethod enum. How should I work my way around this?

BitPuffin commented Feb 24, 2013

@Dicebot Well section is actually not in the url, it's like GET /_config/:section

Also I've ran into another issue, CouchDB uses an almost REST thing where it really only uses the rest it likes. So couch requires support for the HTTP method COPY which isn't in the HttpMethod enum. How should I work my way around this?

@BitPuffin

This comment has been minimized.

Show comment
Hide comment
@BitPuffin

BitPuffin Feb 24, 2013

Maybe the HttpMethod needs to be updated. HTTP is ment to be extendable as far as I know, so maybe changing it so that it's more generic somehow is a good idea?

BitPuffin commented Feb 24, 2013

Maybe the HttpMethod needs to be updated. HTTP is ment to be extendable as far as I know, so maybe changing it so that it's more generic somehow is a good idea?

@mihails-strasuns

This comment has been minimized.

Show comment
Hide comment
@mihails-strasuns

mihails-strasuns Feb 24, 2013

Contributor

@BitPuffin I'll try to address your issues once Sonke's opinion on how final design should look like is read, as he have said he himself is a bit busy now. Let's do some open-source friendship magic here :)

Contributor

mihails-strasuns commented Feb 24, 2013

@BitPuffin I'll try to address your issues once Sonke's opinion on how final design should look like is read, as he have said he himself is a bit busy now. Let's do some open-source friendship magic here :)

@BitPuffin

This comment has been minimized.

Show comment
Hide comment
@BitPuffin

BitPuffin Feb 24, 2013

@Dicebot Nice! I'll continue finishing the CouchDB plugin when this is addressed. But I might have to (temporarily at least) write the app I'm working on in a different language and then port it to Vibe later on.

BitPuffin commented Feb 24, 2013

@Dicebot Nice! I'll continue finishing the CouchDB plugin when this is addressed. But I might have to (temporarily at least) write the app I'm working on in a different language and then port it to Vibe later on.

s-ludwig added a commit that referenced this issue Feb 25, 2013

@s-ludwig

This comment has been minimized.

Show comment
Hide comment
@s-ludwig

s-ludwig Feb 25, 2013

Member

Regarding HTTP methods, I would like to try and see how a pure enum approach suffices. Originally, the method was just represented as a string, but for performance reasons it was replaced by an enum. A hybrid approach could be used to only trade performance in case of unknown methods, but lets see first if it is enough to just add all well defined methods to the enum.

Regarding the full customization of certain interface methods, I've thought that maybe the following two annotations are a cleaner approach than the per-parameter annotation and should be more flexible:

  1. @path("/_config/:section/:key") void getConfigKey(string _section, string _key);
  2. @method(HttpMethod.COPY) void copyEntry(...);
  3. If the UrlRouter would also store the "_" match as a param, something like this would also work: @path("/some/path/_") void method(string _tail);or possibly@path("/some/path/*") void method(Path _tail);

The @path would get the path of the parent interface prepended (if any) and then passed to the UrlRouter as-is. Since the UrlRouter adds a param to the request for each match, the rest should already work fine. @method would just manually set a HTTP method and use the full function name (no verb stripped) as the base for the generated URL (if no override using @path is given). [The whole _pname thing could of course also replaced now by something along the lines of void method(@param("pname") my_param);]

For sure it would be great if you would get this started, @Dicebot. I'm currently feeling like a tornado of tasks is whirling around me :) Any comments/improvements are of course very appreciated. Also, the REST interface generator is growing quite impressively, a bit more and we need a range of tutorials to cover all functionality ;)

BTW, the initial intention for the REST interface generator was more to have a tool to quickly define a REST interface rather than implementing an existing definition. That's the main reason why the current functionality doesn't feel quite comfortable for this kind of application

Member

s-ludwig commented Feb 25, 2013

Regarding HTTP methods, I would like to try and see how a pure enum approach suffices. Originally, the method was just represented as a string, but for performance reasons it was replaced by an enum. A hybrid approach could be used to only trade performance in case of unknown methods, but lets see first if it is enough to just add all well defined methods to the enum.

Regarding the full customization of certain interface methods, I've thought that maybe the following two annotations are a cleaner approach than the per-parameter annotation and should be more flexible:

  1. @path("/_config/:section/:key") void getConfigKey(string _section, string _key);
  2. @method(HttpMethod.COPY) void copyEntry(...);
  3. If the UrlRouter would also store the "_" match as a param, something like this would also work: @path("/some/path/_") void method(string _tail);or possibly@path("/some/path/*") void method(Path _tail);

The @path would get the path of the parent interface prepended (if any) and then passed to the UrlRouter as-is. Since the UrlRouter adds a param to the request for each match, the rest should already work fine. @method would just manually set a HTTP method and use the full function name (no verb stripped) as the base for the generated URL (if no override using @path is given). [The whole _pname thing could of course also replaced now by something along the lines of void method(@param("pname") my_param);]

For sure it would be great if you would get this started, @Dicebot. I'm currently feeling like a tornado of tasks is whirling around me :) Any comments/improvements are of course very appreciated. Also, the REST interface generator is growing quite impressively, a bit more and we need a range of tutorials to cover all functionality ;)

BTW, the initial intention for the REST interface generator was more to have a tool to quickly define a REST interface rather than implementing an existing definition. That's the main reason why the current functionality doesn't feel quite comfortable for this kind of application

@BitPuffin

This comment has been minimized.

Show comment
Hide comment
@BitPuffin

BitPuffin Feb 25, 2013

@s-ludwig In this implementation, would it be possible to annotate with both @path and @method?

BitPuffin commented Feb 25, 2013

@s-ludwig In this implementation, would it be possible to annotate with both @path and @method?

@s-ludwig

This comment has been minimized.

Show comment
Hide comment
@s-ludwig

s-ludwig Feb 25, 2013

Member

Yes, @path would just override any effect on the path, but the changed HTTP method would still apply.

Member

s-ludwig commented Feb 25, 2013

Yes, @path would just override any effect on the path, but the changed HTTP method would still apply.

@BitPuffin

This comment has been minimized.

Show comment
Hide comment
@BitPuffin

BitPuffin Feb 25, 2013

Well I mean at the same time

BitPuffin commented Feb 25, 2013

Well I mean at the same time

@BitPuffin

This comment has been minimized.

Show comment
Hide comment
@BitPuffin

BitPuffin Feb 25, 2013

If not I think there shold be a combo annotation! (The reason I'm asking is because I haven't learned about D function tagging yet)

BitPuffin commented Feb 25, 2013

If not I think there shold be a combo annotation! (The reason I'm asking is because I haven't learned about D function tagging yet)

@s-ludwig

This comment has been minimized.

Show comment
Hide comment
@s-ludwig

s-ludwig Feb 25, 2013

Member

Yes, at the same time:
@method(HttpMethod.POST) @path("/my/path") void getItOn();
shoud work, as well as:
@(method(HttpMethod.Post), path("/my/path")) void getItOn();

Both would mean that getItOn() will me available as POST /my/path instead of GET /it_on.

Member

s-ludwig commented Feb 25, 2013

Yes, at the same time:
@method(HttpMethod.POST) @path("/my/path") void getItOn();
shoud work, as well as:
@(method(HttpMethod.Post), path("/my/path")) void getItOn();

Both would mean that getItOn() will me available as POST /my/path instead of GET /it_on.

@s-ludwig

This comment has been minimized.

Show comment
Hide comment
@s-ludwig

s-ludwig Feb 25, 2013

Member

Documentation for UDAs: http://dlang.org/attribute.html#uda
(that language reference is really not good for finding specific things... I had to use the search engine to actually find the right spot)

Member

s-ludwig commented Feb 25, 2013

Documentation for UDAs: http://dlang.org/attribute.html#uda
(that language reference is really not good for finding specific things... I had to use the search engine to actually find the right spot)

@BitPuffin

This comment has been minimized.

Show comment
Hide comment
@BitPuffin

BitPuffin Feb 25, 2013

Okay! Perfect then, I hope this gets implemented soon enough so I can use D for my project with the deadline of thursday next week.

In the mean time I'll have a look at making a scrypt dub package.

BitPuffin commented Feb 25, 2013

Okay! Perfect then, I hope this gets implemented soon enough so I can use D for my project with the deadline of thursday next week.

In the mean time I'll have a look at making a scrypt dub package.

@mihails-strasuns

This comment has been minimized.

Show comment
Hide comment
@mihails-strasuns

mihails-strasuns Feb 25, 2013

Contributor

@s-ludwig Sounds solid, will start hacking this tomorrow. I have a few concerns about proposed implementation but I guess need to check current one first.

@BitPuffin No deadline promises, sorry, it will go as it goes.

Contributor

mihails-strasuns commented Feb 25, 2013

@s-ludwig Sounds solid, will start hacking this tomorrow. I have a few concerns about proposed implementation but I guess need to check current one first.

@BitPuffin No deadline promises, sorry, it will go as it goes.

@BitPuffin

This comment has been minimized.

Show comment
Hide comment
@BitPuffin

BitPuffin Feb 25, 2013

@Dicebot Well I didn't mean to imply any pressure on you. I was really just thinking in general, that if this gets implemented quickly I'll finish the project in D, if not I'll just do it in ruby or something!

BitPuffin commented Feb 25, 2013

@Dicebot Well I didn't mean to imply any pressure on you. I was really just thinking in general, that if this gets implemented quickly I'll finish the project in D, if not I'll just do it in ruby or something!

@mihails-strasuns

This comment has been minimized.

Show comment
Hide comment
@mihails-strasuns

mihails-strasuns Mar 4, 2013

Contributor

@BitPuffin Please follow #189

Contributor

mihails-strasuns commented Mar 4, 2013

@BitPuffin Please follow #189

@s-ludwig s-ludwig closed this Jan 27, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment