Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Provide comprehensive documentation with examples. #91

Open
nrk opened this Issue · 24 comments

7 participants

@nrk
Owner
nrk commented

Unfortunately Predis still lacks in the documentation department. This is a shame but I can't really find the time to write decent entries in the wiki to cover this aspect, let alone the fact I'm not that good at writing documentation in the first place, so I guess this task will still take some time without some external contributions.

It would be cool to have a generic introduction on Predis with its supported connection parameters and options, an overview of certain features like the abstraction for Redis transactions or Lua scripting (and other obscure and rarely used things like iterable multibulk replies) and some entries about the extensibility of the library and some more complex cases.

@tyler-king

What kind of generic documentation are you looking for? Are you looking for something along the lines of document examples of how to use each command with Predis?

@dominics

Have you got an idea of what you want to use for documentation? Something like Sphinx (which would eventually let you use rtd.org), or something more Github orientated?

I'd be happy to have a go at a couple of pages. But with the current state of things it's hard to know where to start.

@tyler-king
@nrk
Owner

@tyler-king I think that providing an extended documentation for commands with explanations and examples would be redundant with the official Redis documentation since Predis follows pretty closely their signature by default, but providing a quick reference (mostly to document certain features such as using arrays for variadic commands or named-arrays to specify modifiers for SORT and the such) and point users to the official Redis documentation for a more in-depth description of each command could be useful.

@dominics I wasn't even considering alternatives to hosting everything on GitHub but Read The Docs seems indeed interesting. Do you have any past experience with it? As far as I understand it uses reStructuredText, I'm not really used to it but well that's an unimportant issue I can easily overcome :-)

Now aside from what we could use to generate and publish the docs, I'd start with a descriptive user guide covering the following macro-topics:

  • how to initialize Predis\Client (connection parameters, client options, various connection backends).
  • how to use abstractions such as command pipelines, transactions, pubsub, monitor, etc.
  • client-side sharding with Predis
  • replication with Predis
  • how to extend the library (e.g. add support for new commands, Lua-based commands, creation of new connection backends, etc).

In my opinion this should have priority over a detailed reference of the whole library, which will follow at a later stage and will be referred to from the user guide for a more in-depth explanation of certain topics.

Thoughts?

@dominics

The outline sounds great, and I agree regarding the command documentation. Redis' documentation is more than sufficient in practice; I've yet to hit any major abstraction leaks between commands as described on redis.io and the command interface on a Predis\Client (which is a fantastic feature).

I've done a fair bit of reStructuredText. It is more cumbersome than something like Markdown, but also more flexible. It's strengths are in handling indexing of terms; cross-linking between different types, like referring to a class versus a method versus a glossary term. There's a fairly good Sphinx "phpdomain", but that's more for formal API documentation, or auto-generation from code, whereas we're probably looking at long-form prose with examples (which I guess makes the choice of markup language even more a bikeshed).

Another advantage of Sphinx is that you can make it locally; c.f. documentation on a Github wiki or gh-pages. Also, the interface for editing .rst files on Github isn't too bad (although, it doesn't render Sphinx-specific stuff)

@nrk
Owner

@dominics alright after reading your comment and these slides I'm almost sold on using sphinx as it sounds already great enough to me :-) I just want to give it a quick try before deciding, will do that tomorrow!

@nrk
Owner

Hey @dominics,

I just played a bit with Sphinx and, as expected, I'm really liking it. reStructuredText is indeed more cumbersome than Markdown (not sure how I can get used to how inline external links are defined), but not by a great degree. Cross-referencing is awesome. I also like the fact that automatic versioning seems quite easy. All in all it seems a very flexible solution, so yes now I definitely want to use Sphinx to generate the documentation of Predis :-)

I just need to get familiar with it now. By the way, is you offer to help with writing a few pages still valid? It would be a great help, but honestly I'm already plenty grateful for getting this great hint about using Sphinx :-)

@dominics

not sure how I can get used to how inline external links are defined

Exactly the bit of markup I had in mind when I said that.

is your offer to help with writing a few pages still valid?

Absolutely. Point me at a directory. I've been using clustering, replication and pipelining at vendhq.com; Predis has been rock solid for us.

@nrk
Owner

not sure how I can get used to how inline external links are defined

Exactly the bit of markup I had in mind when I said that.

Heh that was easy, I nurtured hatred against them at first sight ;-)

Absolutely. Point me at a directory. I've been using clustering, replication and pipelining at vendhq.com; Predis has been rock solid for us.

Great! I guess we can use doc as the root directory for the documentation since it's basically the de-facto standard. Also I wouldn't bother with the style of the HTML output, we'll just keep the default theme for now unless you know of a cool one.

Would you prefer if I create a documentation branch so that you can initially open pull requests against it and I can add contents myself before merging everything into v0.8 and master? Though I'm not sure which is the most efficient workflow at VCS-level to handle changes in the docs while developing and maintaining different versions of the same library...

EDIT: v0.8 and master are pretty much the same right now, they will diverge in the future when I'll start working on Predis 0.9 in master.

@ronnylt

I am also willing to help.

I can start writing about how to extend the library (adding support for new commands, Lua-based commands, etc...).

@nrk
Owner

@ronnylt the more the merrier ;-) Just wait until we have the basic Sphinx structure in the repository and we'll coordinate from there.

Thanks!

@nrk
Owner

I applied the documentation label on closed issues containing stuff that might be worth documenting sooner or later.

A few of them like this one or this one could be added to a different section named troubleshooting weird behaviors (or anything along these lines).

@dominics

Would you prefer if I create a documentation branch so that you can initially open pull requests against it and I can add contents myself before merging everything into v0.8 and master?

Sounds like a good approach.

Though I'm not sure which is the most efficient workflow at VCS-level to handle changes in the docs while developing and maintaining different versions of the same library...

I think for most stuff you just use the versionadded, versionchanged and deprecated markup liberally.

If you host on readthedocs, they'll build documentation for all your branches and tags and make it all available, so old versions of the documentation can still be reached. But I don't think, for instance, you have to backport documentation fixes to old branches; I think you just aim for your master-branch documentation to explore all the currently supported versions and their differences.

@dominics

@nrk Some interesting conf.py tricks I've found:

  • Highlight PHP code without <?php:

    import sphinx.highlighting, pygments.lexers
    
    sphinx.highlighting.lexers['php'] = pygments.lexers.PhpLexer(startinline = True)
  • Load version and release from /VERSION

    from __future__ import with_statement
    
    # ...
    
    version = 'dev'
    release = version
    
    with open('../VERSION') as f:
        release = f.readline().strip()
        version = release[0:3]

Mine so far: https://github.com/dominics/predis/blob/1aed6e094627544cb4b82cef2d86957a3c6bc73b/doc/conf.py#L20

@nrk
Owner

@dominics OK I pushed a documentation branch on GitHub.

Now about versioning. First of all, we don't care about old or ancient versions of Predis (which is everything below v0.8 right now). To sum things up, in my development workflow for the library we always have a stable branch (v0.8 at this point) and a development branch (it's always master) that eventually gets promoted to stable with the creation of a new branch when a new major release happens (will be v0.9 in the future). Pretty much standard I'd say.

I'd like to maintain a separate documentation for master and the stable branch because, for example, master may contain major (and not yet finalized) breaking changes compared to the stable branch for months before reaching a new major release (this happened to a certain extent with v0.8 while it was in development), therefore the main docs available for Predis should just focus on the stable branch and accept fixes backported, if necessary, from master. The documentation from master will eventually become the main one when promoting a new major release. That said, when reaching new major releases all the documentation efforts should just focus on the current stable and the master branch with no fixes backported to older releases.

Does it sound a reasonable plan? On a related note I do plan to host the documentation on readthedocs, it seems a natural fit to do that now that we settled on using Sphinx.

Nice tricks by the way, Sphinx looks indeed awesome!

@dominics

Sounds exactly right. I'll focus on just v0.8 for now.

@nrk
Owner

Yup, right now master is really just a fast forward of v0.8 anyway which makes things easier.

@gescript

An example to non versed in php namespaces and reflection (me).
Predis/Examples/ServerSideScripting.php
Is written under same namespace. To work with another namespace:

namespace Another\Example;

use \Predis\Command\ScriptedCommand;

class IncrementExistingKeysBy extends \Predis\Command\ScriptedCommand { ... }
...

$client = new \Predis\Client($single_server);
// And the "trick" due Reflection
$client->getProfile()->defineCommand('increxby', '\\Another\\Example\\IncrementExistingKeysBy');

@nrk
Owner

@gescript we might put an explicit note in the documentation since we usually refer to fully-qualified names (that is, the full name of a class comprising its namespace) when describing such cases. We could also leverage the cross-referencing feature of Sphinx to link the fully-qualified name string to a glossary entry that illustrates what it means. Would it work for you?

@gescript

Probably the best way would be begin from error caught:
'ReflectionException' ... 'Class xxxxx does not exist'
My first search was:
'Predis ReflectionException Class does not exist'
If you find "fully-qualified name" under, sure you are going to read.

Thanks @nrk for your help and this excelent tool.

@rookie7799

I know you guys have been working hard on this, but is it possible to give a simple example on how to connect/use redis cluster (3.0.0)? When I try requesting for a key that exists on another node it just gives me:
PHP Fatal error: Uncaught exception 'Predis\ServerException' with message 'MOVED 6918 10.5.7.202:7003'

THank you!

@nrk
Owner

@rookie7799 you are right, we are in desperate need of docs and I hope to be able to continue working on this as soon as Predis v1.0.0 is ready.

As for using redis-cluster with Predis, you have to initialize the client by providing the whole list of nodes of your cluster in the same exact order as they were provided to redis-trib when creating the cluster, and then set the cluster client option to redis. This is an example:

$nodes = ["tcp://127.0.0.1:6381", "tcp://127.0.0.1:6382", "tcp://127.0.0.1:6383"];
$options = ["cluster" => "redis"];

$client = new Predis\Client($nodes, $options);

The client will compute the distribution using the same algorithm of redis-cluster. Starting with Predis v0.8.5 you are also be able to pass only a subset of the nodes, Predis will automatically ask for the complete cluster configuration by issuing a CLUSTER NODES command to one of those nodes.

@integrii

Does Predis identify slots being rebalanced to other nodes? Lets say I have a PHP process that never shuts off so the Redis connection is held for months at a time. When I go run a reshard and move slots to a new set of nodes - will Predis figure that out when it gets a MOVED command or will it error?

@nrk
Owner

@integrii sure, Predis handles -MOVED responses returned by Redis by persisting the change in the slot association, but you can also configure the aggregate connection in order to fetch the whole updated slots map automatically on such changes, as you can see in this example:

$nodes = ["tcp://127.0.0.1:6381", "tcp://127.0.0.1:6382", "tcp://127.0.0.1:6383"];

$options = [
    "cluster" => function ($options, $option) {
        $cluster = new Predis\Connection\RedisCluster($options->connections);
        $cluster->enableClusterNodes(true);

        return $cluster;
    },
];

$client = new Predis\Client($nodes, $options);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.