Why is lime not 'go get'-able? And other questions...

[EdVanDance]

Hi,

I would like to get some answers here and summarize them for the wiki.

1. I would like to know why lime is not 'go get'-able? Why are the imports just "lime/..." and not "github.com/limetext/lime/..." as usual? Wouldn't it be possible to create something like a import.go file which only imports the required packages.
2. Is it necessary to generate the parsers everytime? Why not generating them once and put them under version control? I guess there won't be many changes to parsers right?
3. Same question as 2. for python API?
4. Why do we use submodules instead of own forks where we manage our own go1 branch?
5. Why are we not using pkg-config for cflags and ldflags for rubex and pygo?

I'm currently trying to make lime 'go get'-able in my forks' go1-branch (please don't base anything on that, there will be rebases for sure).

Hopefully we can clear things up and document the decisions made.

Greetings,
Ed

[quarnster]

1, Just legacy from the pre github days, not an active decision, but until there is a solution for 5 I prefer to have it this way to make it very clear that it isn't go get-able.

2, 3, They aren't generated every time, just when the source file is changed. By not having the generated code checked in it makes it IMO clearer that they are in fact a) generated and b) there's a different file that should be changed if its functionality needs to change.

4, What do you mean? rubex, gopy, termbox are all both submodules and forks.

5, a) Windows, b) Python must be special compiled, you can't just use any python3 lib that happens to be installed on a users system.

[EdVanDance]

2,3: Sure as long as we have other reasons for not being go get-able that is totally fine.

4: I meant using import "github.com/limetext/rubex" instead of submodules. I don't see the the benefit of using submodules. Seems to add unneeded complexity. For me it's not a big deal, but I thought maybe for others ...

5: Too bad, yeah I remember I read about the signal stuff ... I read something about pkg-config for windows today, I think it is possible. But the special python config sucks ...

Thanks for clarifying.

[EdVanDance]

Just had a look at the cmakelists
Is it is ok to use a prebuild python on windows?

[quarnster]

2,3: Thinking about it I'd be very ok with checking in the generated source files if they are named *_generated.go, but a moot point for now. They should probably by named that way no matter whether they are checkin in or not (pull request welcome).

4, I think it's just pre github legacy as well. Feel free to cleanup.

5, I don't think windows needs the sigaltstack tweak (no sigaltstack on windows afaik), but there's a bug in Go preventing it from properly use dlls so it's still broken. Linking statically doesn't trigger the bug though, however IIRC building python statically was a no-no.

Side note: some days I dream about a C/C++ to Go source code convertor using libclang or a llvm bitcode to Go convertor just to get rid of the nastiness of C/C++ and to make cross compilation (yeah, I want it on arm too) trivial.

[EdVanDance]

2,3, One more question regarding the generated files. The generated parsers have a comment which tells the used
command line arguments, shall I remove them before checking in? Or maybe just remove the absoule paths?

4, We would need a correct go1 tag or branch on the repos right?

5, Is there a test that fails when sigaltstack is used??

I will write the summary for the wiki probably on monday. That is the only day I have a bit time for myself 😄

[quarnster]

2,3, I don't care about that myself, but if you think removing the comment and/or abs path is important I'd rather have it be done automatically instead of as a manual step.

4, Do we? The absense of it makes it use master IIRC but it was a while since I read what it does. I think some of the submodules use a specific lime branch that would either need to go into master or go1 or whatever makes sense. Go 1.1 features are used (method values and the relaxed return requirements spring to mind) so Go 1.0 is out.

5, I think go1.2 usually panics nowadays rather than just silently deadlocks but afaik there's no specific test written for this, and I don't know offhand what exactly in a default python 3 installation triggers sigaltstacks use.

[EdVanDance]

2,3 Yeah otherwise it would generate noise. Ok I'll try to do it via cmake.

4, Yup it uses master by default, but we need python3 branch of pygo IIRC. Go 1.1 also fetches from go1, or do you mean something else? I think it would be nice to have development and upstream in master and go1 as a stable branch.

5, The callchain is Py_InitializeEx() -> _Py_InitializeEx_Private -> _PyFaulthandler_Init() -> sigaltstack(). Ok, I'll have a look.

[quarnster]

2,3 Or you could just change the tool to accept a flag-provided header rather than the default generated one.

4, Just meant that using "go1" suggests that the code works with go1 which it doesn't (or maybe those particular repos do). go1.1 was added as a tag in (surprise!) go 1.1, so it would be preferable if we are going for a versioned tag. Stable in goxxx and development in master makes sense.

[EdVanDance]

Ok alright, going to do that as soon as I got some time 😄

[quarnster]

FYI, took care of 2,3 (I think)

[quarnster]

5, Perhaps we should just disable python by default via build tags. Nothing in the core really depends on it, and the initialization of the python/sublime bits can be put in a separate file where it is used with a build constraint to optionally enable it. Or, it could be a completely separate executable like ST3's "plugin_host".

Obviously this removes a lot of the functionality currently provided by the Vintageous python plugin, but I see no reason why the same functionality couldn't be implement in pure Go code.

Rubex, with the underlying Oniguruma regexp library isn't strictly needed neither, but there are currently no alternative syntax highlighting code implemented. As its source code doesn't really need to be tweaked it's easier to deal with than python though.

[EdVanDance]

I wouldn't like to reimplement Vintageous in Go, so much effort just for replication??.
I thought the ultimate goal is to have a full sublime text clone, that could run all the available plugins. Not having the python API by default would be a show stopper for me.

The separate "plugin_host" is probably there because of stability, so the host could be restarted on crashes instead
of crashing everything.

We shouldn't drop important features for having an idiomatic (go-get-able) package right?

[quarnster]

Bear in mind the context of this specific issue. It would absolutely not be be dropped, just not be part of what's built when you do a simple "go get".

The goal of having a full sublime text clone, able to run all the plugins people care about is not something I intend to drop, and at the same time as much as possible should be designed with it being optional. This is already the case, removing the dependency on the lime/backend/sublime and gopy packages is just a couple of lines to comment in the frontends. Ideally it'd be changed to be a single unnamed import (ie import _ "lime/backend/sublime"), and that import could then be put in a file with a build constraint selectively enabling or disabling it in whatever configuration users care about.

Personally I'd like to be able to write scripts in lua, other might prefer javascript or some other extension mechanism. The point is, the backend doesn't require any of these and any specific scripting method does not exclude another.

Similarly to the optionality of using Python, the current commands implemented in Go are actually optional too. While I would probably not even try to reimplement Vintageous in Go myself, I wouldn't stop the effort if someone wanted to. It would have to go into it's own package (lime/backend/commands/vi suggested) for an again optional unnamed import though.

(@mtahmed You might want to put some of this in that Architecture documentation of yours in case you are listening)

[EdVanDance]

Ahh now I do understand what you mean. Yeah I like that idea.

[EdVanDance]

@quarnster Do you think the wiki entry is ok so far?

[quarnster]

Sure, LGTM

[EdVanDance]

All right, so I'm closing this.