Language volatility

[asemic-horizon]

This is meant as feedback: it's written from my point of view. I know there are other, more qualified points of view and that the project has the direction it has.

I know that feature stability is not in scope until 1.0. But @allison-casey said (I think I quote their intent correctly) in an issue I can't find anymore that continually breaking code has the effect of a highway billboard saying "don't use this language!".

The breakneck speed of change in Hy might be a suicidal tactic. There's only so many times you can convince people to rewrite projects like Adderall and Hydiomatic. Less advanced users like me can always pin a version of Hy (PyPI statistics suggest the most popular version is 0.11) to keep their stuff working, but can I recommend the language?

I could also fork something right before toolz functions are removed from the standard prelude and call it "Messy Hy". But I'm not at that level as a programmer (I mean, to maintain a fork, contribute back upstream, move forward with Hy in most non-aesthetic respects). Maybe Hy could do something like it --- it's what Python did at the 2.7 -> 3.0 discontinuity. I mean, I get that project leaders find much to improve in Hy, but I find that Messy Hy is great.

As it stands now, I think Hy will just keep losing users. This isn't a ragequit from a community in which I have literally no standing; I'm just an anecdote.

[scauligi]

(My personal opinion, and relatively uninformed as I only started using Hy a month or so ago) I think this should be less of an issue now, assuming we're timely with the 1.0 release.
But until then, I think trying to preserve backwards compatibility with pre-1.0 versions stifles our ability to implement things like #1975 and #1971 -- and since we're committing to backwards compatibility starting at 1.0, we should get as strong a start as we can before we lock things down.

Also, theoretically one could write a "hycompat" package, that would work something like this, and your code would otherwise need minimal changes:

(import [hycompat.from-0.11 [*]])
(require [hycompat.from-0.11 [*]])

;; ... your existing hy 0.11 code ...

Probably best to wait until 1.0 is released to start on any compatibility layers though...

[allison-casey]

I agree with scauligi with the fact that we shouldn't be looking to preserve backwards compatibility until 1.0. I think the bigger issue is that the Hy code base changes so frequently and we're pushing releases directly to main pypi line that it's hard for people to really settle in to hy before things break again. I was thinking that we move to a more python like release cycle. Leave Hy at 0.20.0 (or maybe 0.21.0 for the fn and builtins changes) except for bug fixes/contrib and the like and move to 1.0.0 pre-release so as not to pollute pypi with versions that continue to break things every 10 minutes. 0.20.0 is pretty good I think, and feels like a good place to introduce people to Hy knowing that it won't change for a while and then let them opt in to the pre-release line if they'd like. What exactly would go into a 1.0.0 feature lock is something for another discussion though

[asemic-horizon]

Hi. Thanks everyone who answered!

I'm slightly surprised that other mere-users haven't weighed on this. I think the solution that freezes 0.x and moves continued development to 1.0 pre-release is very good. I'd plead you to keep 0.20.0 as the final 0.x version -- there are still some Hy libraries like Hymn that have been keeping up, and advancing to the builtins changes might break them.

I'd think I'm completely alone on this, but googling to see if there were others with similar concerns as me, I discovered not one [1] but two [2] (e-)books on Hy. There's also blog posts kvetching hard about constant breaking and having multiple projects in multiple versions of Hy, but I don't think these make a point I haven't made, they just do it more forcefully.

Maybe there is (and maybe there is not) a disconnect between "product owners" (and I have a profound sense of "hierarchy" towards them; I'm just a mathematician in data scientist drag) who see Hy as a work in progress, full of flaws and less-than-ideal features, and "product users" -- I previously thought I might be one in a handful of these, but apparently there's more -- who see this as a programming language, and a very useful one.

This being open source, users don't have that much leverage over product owners. I have to resort to pleading:

• Please let us have 0.20.0 (before the removal of functionality that right now is built-in) as a "final 0.x" version. I know I can pin my own projects to whatever, but I'm invested in the language and would like to see an ecosystem flourishing around it -- which seems to "want to happen" on its own all the time, since the idea itself is very exciting, but also break again and again.
• Can we have some clear communication on this? Like a message in the docs / homepage/ etc. explaining a 0.x version is finalized and projects can target it, or wait for 1.x, which will break them like Python 3 breaks Python 2?

[1] https://leanpub.com/hy-lisp-python
[2] https://hy-files.readthedocs.io/en/latest/intro.html

[allison-casey]

pinning 0.20.0 is my intention. @peaceamongworlds and I continued this conversation in #2008 and we both agree that pinning Hy there and focusing on 1.0 would be best for Hy and it's community. Once 1.0 gets released we'll likely move to a longer release cycle with pre-releases, deprecation warnings, etc (the exact language of that policy needs work, but that's the idea anyway) so the community can lean on it without fear it'll break. Clear communication in the readme and docs is a good idea and I'll see what I can do about making that happen.

[rcarmo]

As one of the 0.0.x users, I'd like to say I keep coming back expecting to see a stable release that will let me pick up https://github.com/rcarmo/sushy again and re-spin it in "modern" Hy because I quite liked its conciseness. I left Hy when let was broken and all of a sudden all my nice LISPy code had to be refactored, and I have repeatedly tried 0.2x versions for smaller things and gone back to vanilla Python 3 because, well, I have enough issues with JavaScript library mutations.

This is just my $0.02, obviously, but having stable releases with good docs would greatly help.

[atisharma]

I'm another simple user - I use Hy as a numpy/scipy wrapper and for trading software - and I have had to pin to 0.20.0.
Pushing alpha releases to pypi did rather complicate my life and I await a stable release eagerly. It also makes collaborative work rather difficult.

Another problem is that there is no obvious way to access the docs for 0.20.0. Here, a message saying breaking changes: NEWS" on the front page of the docs (under source, discussions, pypi) would really help people understand the language status and avoid losing some of those casual users who are experiences the breakages. It would have saved me an hour or two and I follow Hy quite closely.

For me, Hy is a very nice tool, but still a tool. I really hope that breaking changes will be minimal after 1.0 as moving my code to 1.0 will be quite a large project. As rcarmo says, stable releases with good docs (and the docs are pretty good) will help greatly.

Using a moving target for trading software was in hindsight rather risky, but a lispy semantic structure makes reasoning about mathematical code so much easier.

[Kodiologist]

Pushing alpha releases to pypi did rather complicate my life

Yeah, it complicated our lives, too. It was a mistake, and we're back on 0.* releases.

I await a stable release eagerly

You're in luck. 0.24.0 was released in June.

Another problem is that there is no obvious way to access the docs for 0.20.0.

It's an obsolete release, so keeping the documentation online would lead to yet more cases of people reading it when they're using a different version and reporting a "bug" when the inevitable problems arise. You can still use help or generate the documentation locally.

Here, a message saying breaking changes: NEWS on the front page of the docs

We've had a giant banner and people ignored that, too. If people don't consider the possibility of version differences, there's really no way we can convince them to do that. My impression at this point is that people being flummoxed by version differences is part of the cost of doing business, and the most we can do is document things well.

It would have saved me an hour or two and I follow Hy quite closely.

I don't understand how you could follow Hy quite closely and not notice that it has breaking changes.

I really hope that breaking changes will be minimal after 1.0

Yeah, the point of such a release number, in my mind, is to indicate that no breaking changes are imminent. Presumably they'll happen eventually and there will be a Hy 2.0 and 3.0 someday, but if they were going to happen soon, I wouldn't call the latest release 1.0.

moving my code to 1.0 will be quite a large project

I think you'll have an easier time of it if you update your code for each release, instead of waiting till 1.0 and trying to cope with a massive number of changes simultaneously. That will also allow you to get new features and bugfixes (including compatibility with newer Pythons) and help us test our latest code.

[atisharma]

Clearly I did notice the breaking changes, and was aware of the intent to push for 1.0; my point was that it is not obvious where those breaking changes are documented (in NEWS.rst) unless you already knew. Finding that list is what took the time.

So, to put one's self in the shoes of a naive user, you upgrade with pip, your code breaks, you want to fix your code, you go to the docs or look for a file called "changelog" or similar. Finding NEWS.rst is not immediately obvious.

It wasn't obvious to me that 0.24.0 is considered stable, since it's described as "a direct successor to 1.0a4" and that the return to 0.xx version numbers was primarily to work around limitations of pip. If it's stable and likely to be not too far from 1.0, then I'll move to it, thanks.

In the end, I really care about Hy. There's nothing that fills the same niche.

[Kodiologist]

look for a file called "changelog" or similar. Finding NEWS.rst is not immediately obvious.

I think NEWS files are pretty similar to ChangeLogs, even if they don't necessarily follow the GNU Coding Standards for a changelog or whatever. If we called it "ChangeLog", somebody else could just as easily say it should be called "NEWS" or (or "CHANGES", or "HISTORY").

It wasn't obvious to me that 0.24.0 is considered stable

It's no more or less stable than previous releases. I try not to make promises about when (or if, for that matter) 1.0 will be released because I'd rather pleasantly surprise people than miss a deadline because there turned out to be more work to do than I guessed.