Automatically synthesizing LinuxMain.swift

[felix91gr]

This would help us run tests on projects that don't have the LinuxMain.swift file. How to do it, is up to us. We know that:

• SPM on Darwin can generate it.
• Sourcery could probably do it as well.

And here are some ideas on how to do it:

• Make a python crawler that generates the LinuxMain file, with extensions and all (I've already tried this partially and it works; it can be done).
• Sourcery can do it, but it's still not working on Linux.
• Synthesize it on Darwin (with SPM) and copy it to Linux through Travis (I don't know if this is possible, the VMs are probably sandboxed).

Other use cases:

• This could help Linux users to port a project they like to Linux. It could even be integrated into something like swiftenv, s.t. you can generate the LinuxMain.swift file automatically.

[vknabel]

Ideally generating the LinuxMain.swift shall be possible on Linux directly as it makes a huge difference during development. Though I’d prefer Swift, especially as installing pip on macOS is a pain.

Generating the LinuxMain on macOS on CI and copying it over to the Linux build is not possible AFAIK. Some other ideas on this:

• macOS Build commits new LinuxMain when changed and pushes to the repo: highly CI specific and kind of hacky
• Test Linux on macOS using https://github.com/iainsmith/swift-docker. Wait. What? 😅

Regarding integrating this into different tools: swiftenv is probably the wrong tool for this. Maybe this fits into jakeheis/ice or https://github.com/iainsmith/swift-docker.

[felix91gr]

Ideally generating the LinuxMain.swift shall be possible on Linux directly as it makes a huge difference during development

Yes! According to my digging through SPM's source, the one and only thing that's stopping it from discovering tests automatically on Linux (and thus, the ability to generate something like the LinuxMain.swift file) is unimplemented functionality in Foundation.FileHandle and Foundation.FileManager.

Generating the LinuxMain on macOS on CI and copying it over to the Linux build is not possible AFAIK

Dang it, I was hoping there was a way, but if you say it can't be done through CI then you're probably right.

swiftenv is probably the wrong tool for this

Sorry, I should've explained my idea before just posting it like that 😅
I was thinking on maybe making the crawler in a bash script and then integrating it into swiftenv, how does that sound?

Edit: I would also first try to make it work in Python, because bash programming is a pain in the ass 😆 (I use fish because of that reason, among others 🙂. Speaking of which, you use zsh right? How is it?)

[vknabel]

I was thinking on maybe making the crawler in a bash script and then integrating it into swiftenv, how does that sound?

That's what I meant. I am not sure if I want swiftenv to change my projects. It can do just one thing, but it does it incredibly well.

I think the best solutions are:

• ruby/python/swift script that genereates SPM compatible output on linux
• Fixing Foundation.FileHandle and Foundation.FileManager

OT: I don't like bash programming for more advanced tasks either. That's why I usually write some Swift scripts using marathon and beak. I have used fish, too. But I switched to zsh as it is more compatible with bash. And mostly because of oh-my-zsh.

[felix91gr]

I am not sure if I want swiftenv to change my projects. It can do just one thing, but it does it incredibly well.

That's a fair point. I think it's better to keep the functionalities separate, specially given how important is the work that swiftenv already does (in Linux it has simplified my workflow by a huge amount).

ruby/python/swift script that genereates SPM compatible output on linux

I think I'm gonna finish my python script then and we'll see where do we go from there. I plan on having a working prototype tomorrow.

Fixing Foundation.FileHandle and Foundation.FileManager

Yeah, that'd be the best option long-term. I've sent a DM to Tony Parker from the corelibs team asking about how can I help. Specifically, what does one need in order to make an effective PR.

Since that part of Foundation provides OS level functionality, and the corelibs-foundation isn't a SPM project, I've kept myself from trying to make a PR... it seems to be very non-trivial to me. Specially after seeing Ryan struggle with the one he's tried to make.

My fingers itch. I want to help but I don't know how.

I guess not seeing a roadmap only makes it worse 😅

But I switched to zsh as it is more compatible with bash. And mostly because of oh-my-zsh.

Oh, nice. I'll take a look :)

[felix91gr]

I think I'm gonna finish my python script then and we'll see where do we go from there. I plan on having a working prototype tomorrow.

In the end, I'm going to try something more sophisticated. This is what happened:

• I was making the regex-based python crawler.
• Decided that SwiftSyntax was a better alternative
• Asked Harlan (Haskins) about it
• He convinced me that anything syntax-based (like regexes or SwiftSyntax) was too fragile, because of inheritance and inter-file interactions like extensions.

In the end, we need something that knows about semantics. So, I've changed my focus and now I'm learning how to use SourceKitten 😅
And indeed, you were right about Sourcery. A binary of Sourcery is all that we'd need, since they already have a stencil template that synthesizes the LinuxMain file. So an alternative is making Sourcery compile under Linux, even if that means restricting its runtime features, as mentioned here. As long as it can parse code and run Stencil templates, that's good enough.
I'm seeing how far can I go with SourceKitten; if it proves too complex I'll switch gears and try making Sourcery compile.

[felix91gr]

I've also posted a new digging through SPM on the forums, asking for a community push on this issue.

I was wrong, I think. It's not corelibs-foundation that's missing functionality, but something else. I don't know what it is, but I've pointed out what lines seem to be behind it.

My gut feeling is that those lines are using ObjC runtime tools or something like that. But I've also given arguments about why today you can add test discovery functionality to SPM without needing anything other than Swift and SourceKit. Let's hope it works.