Upstreaming LKL

[tavip]

This issue is for discussing and tracking our upstream strategy.

[M1cha]

have there ever been any attempts to do so?
I'd really like to read Linus' view on that since this is a rather unusual port.

[thehajime]

For those who're not familiar with the history, there was an attempt of RFC (*1) to LKL back in the fall of 2015. I believe the feedback were mostly nice, comparing the previous my RFC of libos patches, which were reached into a refusal due to the maintainability of the patch.

There were a couple of negative comments (*2) on the basic idea of LKL, as I can see are from xfs subsystem and netdev (in general): both were somehow in the context of performance issue as well as the concerns on the alternate code path provided by LKL (e.g., kernel bypass, etc).

I haven't heard of Linus opinion directly, but I saw an interview report (*3), which he seems to be skeptical about this idea.

My personal feeling is (of course) we can do it.
But we should start with a slow start. That's the way to survive with something new (or crazy) in a huge community, otherwise our chance will be close to zero.

By slow start I mean, we could prepare v1 patch by addressing original comments gathered from RFC email (*1) and see how people react to. It seems we need to resolve something conflicts with UML but we can co-exist for the moment.

Hope it helps to grasp the situation and moves forward the project.

*1
https://lwn.net/Articles/662953/
https://www.phoronix.com/scan.php?page=news_item&px=Intel-Kernel-Library-LKL
http://www.linux-magazine.com/Issues/2016/182/Kernel-News
https://news.ycombinator.com/item?id=11012460

*2
https://mail-archive.com/linux-kernel@vger.kernel.org/msg1024861.html
https://youtu.be/xP9crHI0aAU?t=34m18s

*3 (search 'anykernel' keyword on the page)
https://linux.slashdot.org/story/15/06/30/0058243/interviews-linus-torvalds-answers-your-question

[tavip]

I agree with Hajime. I think we need to start small and ideally work with people that believe in this idea. The UML maintainer has been very open about unifying UML with LKL so I think this is a good path to take.

One of the ideas we discussed was to modify UML so that it gets built into two stages like LKL is: one pure kernel and one for the userspace parts. The UML maintainer was in favor of this and he mentioned that he wanted to do that for a long time.

Unfortunately I didn't had much time for LKL lately (and won't have for a couple of months), but this is what I would start with.

[thehajime]

I am a little bit worried about librarifying UML. It has been in their wish list for a quite long time and it is still not available in 2017.

http://user-mode-linux.sourceforge.net/old/projects.html (section UML as a normal userspace library, written in early 2000s ?)

Of course it is very very nice if those are unified.

My initial thought on small-start doesn't include this unification with UML. If somebody really persists to integrate those two during upstreaming, we can consider this.

@tavip Do you have concern the upstreaming without this unification ?

I can help making v1 patches (making patch series and edit commit messages with a cover page) if you wish.

[thehajime]

any news @tavip ?

[laijs]

any news @tavip ? (no proper emoji for "co ask", so I'm sorry I asked it too)

[tavip]

Hi, sorry, this totally got under my radar.

I think one of the things we should consider before upstreaming is if we want to keep the current syscall optimizations.

In my opinion they are a bit too intrusive and we should try to abstract it away with native operations. I started to add a context switch and syscall native ops and but didn't find the time to complete it.

I also think that it may be possible to improve the syscall latency by using user threads - which we can do easier if we have the context switch native op.

I will cleanup what I have so far and post it and let's see where we can go from there.

[thehajime]

thanks @tavip.

Both syscall refactor and user thread support are definitely nice additions. If you can finalize very soon, I think I agree to include those before upstreaming.

If it's gonna take a bit longer, then I would say again, small-start: do those improvements later.

[tavip]

I agree, I just wonder if it is better to send upstream the original syscall implementation which seems more conventional and straight forward?

Or perhaps I worry too much and we can go with what we have right now?

[thehajime]

Or perhaps I worry too much and we can go with what we have right now?

How much do you expect with this cleanup to reduce the modifications to the original kernel ?

If this is reasonable amount of cleanup so that we can reduce the number of people whom we need to convince, then maybe I would agree to have your patch before upstream.

[tavip]

How much do you expect with this cleanup to reduce the modifications to the original kernel ?

I don't think we have an issue here, Lai's already eliminated the idle loop kernel modifications / exports.

My concern was more about the current syscall implementation complexity. But I don't have a strong preference here, if you think it is OK we can try to upstream it as it is now. The alternatives are to use the old syscall implementation or try to abstract the complexity in new native ops - I'll get back to you next week with an ETA for this.

What do you think?

[thehajime]

My concern was more about the current syscall implementation complexity. But I don't have a strong preference here, if you think it is OK we can try to upstream it as it is now. The alternatives are to use the old syscall implementation or try to abstract the complexity in new native ops - I'll get back to you next week with an ETA for this.

What do you think?

for the particular fix, my opinion is that we can do later.

[thehajime]

FYI.

http://lists.infradead.org/pipermail/linux-um/2019-October/002209.html

[Mic92]

The link is no longer valid. Any other source where can I read it?

[ddiss]

The link is no longer valid. Any other source where can I read it?

IIUC, @thehajime 's last submission was v8:
http://lists.infradead.org/pipermail/linux-um/2021-January/001029.html