Document "common commands"

[cdeil]

I realise writing full documentation for astropy-helpers will take time and it not very urgent.

Would it be possible to start with a very short document of "common commands" to work with astropy-helpers? Here's some things I'd like to know:

• If I want to make a change to astropy-helpers, should I do that in the git submodule of astropy or check out a separate copy of the repo? In the past I've used the git submodule, but then after making my commit in a branch I didn't know the git commands to get back to a "clean state" locally.
• At my institute git via the HTTPS or GIT protocol doesn't work, only via SSH. If I clone astropy via SSH, the astropy-helpers submodule still has a HTTPS URL. Which command should I use to change this to an SSH URL (in the astropy or astropy-helpers repo?)
• Which git commands should be used to update the git submodule pointer to astropy-helpers after some fix has been made?

cc @embray @eteq

[embray]

Yes this sounds worthwhile.

I will say now that for myself I've found it best, when doing work on astropy-helpers, to not mess with the submodule (really it's not intended to be messed with). But instead I have a separate clone of the astropy-helpers repository and I just install it into my virtualenv with ./setup.py develop. That way use of the submodule should be bypassed entirely I think. I see it as being there for working on astropy itself, but not for working on astropy-helpers.

That said, I could see an advantage to coming up with and documenting an easier workflow for working on astropy-helpers directly within a submodule as well. It's just a little trickier since, as you said, that affects the repo in which it is a submodule as well.

[cdeil]

In my clone of the package-template I see this:

$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   astropy_helpers (new commits)

no changes added to commit (use "git add" and/or "git commit -a")

I don't know why astropy_helpers is modified, but I'd like to get rid of these modifications and have a clean status. What's the git invocation I should use?

[cdeil]

Concerning my last comment, I ran

$ python setup.py test

which printed

Updating submodule u'astropy_helpers'

and now I do have a clean git status.
Would it be possible for setup.py to print what git commands it is running for me?

[embray]

Would it be possible for setup.py to print what git commands it is running for me?

I thought it did, but it looks like it doesn't after all. That could be fine, though I wouldn't want it to display that information by default. Perhaps when the --verbose flag is enabled or something.

[eteq]

I've been using a slightly different workflow from what @embray suggested above with the virtualenv. I use astropy_helpers as just a regular repo, and make a new branch inside it when I want to change something. Then I build astropy with python setup.py --no-git build/test/whatever to see if it works. Once it's working, I sent it up to my github repo, and then just do python setup.py build in the astropy directory, which resets it to whatever it's supposed to be (I could also use git submodule commands, but building is usually what I want, anyway).

Does that sounds like a workflow worth documenting, @embray and @cdeil? I think it's simpler than the virutalenv approach, so people are more likely to pay attention to it... (although we can certainly offer both).

[cdeil]

I haven't used either of these workflows, so I don't want to comment which one is worth documenting.

Personally I find the fact that the build command executes a git reset astropy-helpers command (or whatever the command actually is) and doesn't even clearly print a warning or info to the console what it did disturbing ... it would really be great if someone that know what is going on would take the time to document this briefly.

@eteq You mention a --no-git option. There should be a one-line description in the python setup.py --help output, no?

[eteq]

@cdeil - If I execute python setup.py build with a "dirty" astropy_helpers directory, the first message I see is Updating submodule u'astropy_helpers'. Are you not seeing that message? If so, that's probably a bug...

@cdeil : I 100% agree those options should show up in --help. #31 is a reminder to address that... I had hoped this would make 0.4 (see #27), but we had to triage it because we didn't want to hold up the release for it. Hopefully it will get into astropy_helpers ASAP, though.

[eteq]

Oh, one other comment, re:

I realise writing full documentation for astropy-helpers will take time and it not very urgent.

I think this actually is urgent and should be a top priority, because right now it's very difficult for anyone not in the core dev team to contribute to the helpers at all. That said, @embray apparent plans some major changes that might affect this anyway, so I understand that we don't want to write docs that are immediately not relevant. But anything that's likely to stay stable we should start working on now. (along those lines, if @embray is ok with me documenting the workflow I described above, I'm happy to do a PR along those lines now)

[cdeil]

@eteq No, I do see the Updating submodule u'astropy_helpers'. message. But what does this mean? How did it change my astropy-helpers git sub-module? I'd prefer to simply see the git command it ran. But OK ... this is a matter of taste and a detail ... the main point about this issue is that there's no astropy-helpers docs at all.

[eteq]

@cdeil - I see your point about showing the git commands - see #62.

Note, though, that Updating submodule u'astropy_helpers'. just means it's executing the command git submodule update astropy_helpers, so the meaning is fairly clear if you know about submodules. But I can see how someone unfamiliar with submodules (i.e., nearly everyone) wouldn't necessarily know there's actually a command called submodule update.

[eteq]

@embray - this ties in somewhat with our #60 and #31 discussions: do you have an opinion where this should go? Are you thinking of adding sphinx documentation to the helpers, or should it all be in CONTRIBUTING.md?

[embray]

I don't really have an opinion on it. Per my comment in #7 I definitely want to have more complete docs for this, but I'd rather focus my own efforts on some larger structural changes that would invalidate a lot of documentation anyone would write now (except maybe docs for the Sphinx extensions, but those are already documented in the Astropy docs themselves).

[eteq]

Alright, in that case probably the thing to do is use CONTRIBUTING.md for now, and then if it seems like it is getting too big we can add docs then.

[cdeil]

How about changing CONTRIBUGING.md to CONTRIBUTING.rst now?

This will make it easy to change to Sphinx docs later if we want things like cross-linked pages and full-text search.

[eteq]

Yeah, that makes sense, @cdeil. So whatever PR adds those directions can also do the rename.

[cdeil]

I'd like to make a Gammapy 0.1 release later this week and would like to use the latest stable astropy-helpers version. Which git command moves the submodule pointer for astropy_helpers in Gammapy there?

[astrofrog]

@cdeil - if you go inside the submodule and do git fetch origin then git checkout v0.4.1 I think it will set the submodule to the latest 'stable' commit (you then need to git add the helpers directory and commit it).

[cdeil]

As an affiliated package maintainer, I find it pretty frustrating that astropy-helpers is introduced, but no documentation is written. Any chance to get some minimal docs (say a page or two)?

(and thanks to @astrofrog for starting to write up affiliated package release instructions at astropy/package-template#103 !)

[cdeil]

@astrofrog Thanks for the instructions, I can confirm that this works:

cd astropy_helpers
git fetch origin
git checkout v0.4.1
cd ..
git add astropy_helpers
git commit -m 'Update astropy_helpers sub-module pointer to v0.4.1'

Can you please add this to the affiliated package release instructions at astropy/package-template#103 ?

[astrofrog]

@cdeil - I don't think that should be part of the release instructions, since it should really be updated when available, independent of releases. I think we should instead have a section on updating affiliated packages to the latest template and astropy-helpers, in the astropy dev docs for affiliated packages. Do you want to add it there?

[astrofrog]

Now that we finally have proper docs (http://astropy-helpers.readthedocs.io) I think we can close this. But @cdeil, feel free to open a new issue if any parts of the docs are unclear!

[cdeil]

@astrofrog - wonderful. Thank you!