Filesystem path strategy

[lazka]

I've written a strategy that creates Python filesystem paths (everything which open() takes or which os.listdir() can return): https://github.com/quodlibet/senf/blob/master/tests/hypothesis_fspaths.py#L40

I was wondering if there is any interest in including such a strategy.

[mulkieran]

Hi!

There is definitely interest in strategies for useful things like this.

One thing to do is to submit a PR to add yourself to the list of external strategies. See https://github.com/HypothesisWorks/hypothesis-python/blob/master/CONTRIBUTING.rst for some instructions and https://github.com/HypothesisWorks/hypothesis-python/blob/master/docs/strategies.rst for the current list. A good thing to do in this case would be to promote this strategy to its own library.

The next step could be to prepare a PR to merge it into the Hypothesis distribution. One of the external strategies is in that process even now: #708.

[lazka]

Thanks for the info. I'll extract it into its own package and submit a PR for adding it to the docs when I have something.

[Zac-HD]

If you intend to contribute it to Hypothesis (ie this repo), there's no need to publish it as an external package first - and in fact it's probably better to just provide it as a pull request and avoid having multiple versions out there. See eg #536 for a simple example.

You can also add Senf to the list of projects using Hypothesis.

If you'd like to make the pull request I'd be happy to help you through the process with API design and code review; if you don't feel like it I could also do the merge for you (you'd end up on the contributors list either way, of course).

[mulkieran]

@Zac-HD Perhaps you had better outline what the policy is about strategies. I thought I had a quite good one, AND that it agreed with yours.

[DRMacIver]

@mulkieran @Zac-HD

I don't think that we have a formal policy about strategies. I think roughly the following are the guidelines I've been using:

1. Either an external package or adding it to Hypothesis is fine for most strategies.
2. If the strategy is likely to be of widespread use, we should encourage adding it to Hypothesis core (and offer to help with the work, as this isn't totally straightforward).
3. If the strategy is likely to be of quite niche use, there's still no problem adding it to Hypothesis core, but we don't need to be quite so enthusiastic about doing so.

(Plus when adding something to Hypothesis core usual rules about things needing an external dependency going in as hypothesis.extra packages)

Roughly the trade-off is that it's much easier to get things working outside of Hypothesis because there's more freedom to experiment and fewer requirements. but being inside Hypothesis gets you access to somewhat cooler implementation features and better long-term guarantees about maintenance.

[Zac-HD]

I don't think we have a policy specifically about strategies! guides/review.rst covers considerations for the public API, which are basically 'is this a good idea at all' and 'try to get it right the first time because breaking compatibility is hard'. So anything goes, subject to the usual standards and final approval specifically from @DRMacIver!

My personal guidelines (which I might write up as a proposal for loose policy):

• If it's trivial to construct (eg reducible to builds(something, primitive_strategy())), probably not.
• If it's hard to construct or there are sneaky corner cases that a naive version would not test, it might be a good idea. I think of this as encouraging good downstream APIs via tests - for example I really like the path-like objects provided by this one 😄
• Is the API elegant? ie options have orthogonal effects, take appropriate types, no obvious options missing, reasonably intutive... (my major review item; I'd ditch pathname_only as users can .map(os.path.basename) and add platform='native' (native/unix/windows/any) as a new first argument) This kind of issue is almost always fixable in review, so I'd accept pulls subject to revision.
• Will people use this? If someone is contributing a strategy, this is an automatic pass for anything short of support for personal projects. (eg no for a senf-specific strategy; yes please for the general fspath strategy - which can then be used in builds(senf.fsnative, fspaths()) as well as other projects)
• What changes might we want to make in future? Can we see a way to make them backwards compatible (in which case defer implementation / YAGNI), or do we need to make them now to keep a nice API?
• The small stuff should be entirely handled in review with the assurance that we will merge this and can help out as required. Passing tests and code standards all nice, but literally an implementation detail!

Aaand DRMacIver jumps in with a more concise version 😄

[mulkieran]

@DRMacIver That certainly all makes sense. I really like freedom, and tend to interact w/ others on the assumption that they share my enthusiasm.

However, I foresee a distant future (that may never come to pass), in which Hypothesis core strategies in the repo are so encumbered by backwards compatibility issues that externally provided strategies that provide the same values are actually preferred. I realize that this is optimism dressed up as pessimism, but I see nothing wrong with being grimly optimistic.

[alexwlchan]

@DRMacIver A brief observation, based on some of our recent conversations:

Hypothesis : strategies for Hypothesis :: Python : the Python standard library

Given that we bump the Hypothesis version fairly regularly, it’s less of a problem that improvements to strategies would be blocked on a new version (a la stdlib) – but it does mean that you have to upgrade to a new version of Hypothesis for new strategies or features. It also blocks changes on the Hypothesis core team having time to review patches.

It’s not a problem right now, but with more radical changes on the horizon (the settings cull, changes to the core engine, maybe some C or Rust), might strategies that don't involve core types would be better off outside Hypothesis core? Maybe separate repos in the Hypothesis org?

I don’t have a strong opinion on this – but that was the natural analogy that came to mind.

[lazka]

I've created #748

@Zac-HD

I'd ditch pathname_only as users can .map(os.path.basename)

Good idea, done.

and add platform='native' (native/unix/windows/any) as a new first argument)

What is that supposed to do?

[Zac-HD]

However, I foresee a distant future [where strategies] are so encumbered by backwards compatibility issues that externally provided strategies are preferred.

I think at this point we just bump the major version and break compatibility! After all, people can always pip install hypothesis<4 if they need a particular compatibility level.

It also blocks changes on the Hypothesis core team having time to review patches.

This hasn't been a problem for a while - for example none of our open pulls are waiting for review, and haven't been for more than 48 hours in some months (with the reasonable exception of my monster pull for type inference).

Might strategies that don't involve core types would be better off outside Hypothesis core?

For the foreseeable future, I'd keep things in the core - the monolithic approach gives much better discoverability and maintainability (in eg using and updating internal APIs), and the disadvantages of too-large scale are IMO a long way off with good basic design.

What is [platform='native' etc] supposed to do?

What if I want my app to support Unix paths, even when running under Windows? Or vice-versa? Or all paths at all times? The current native-paths behaviour makes sense as a default, but I think it's worth making the others possible and an argument of this kind was the first thing that sprang to mind.