Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wip] GitLab evaluation #2218

Open
wants to merge 1 commit into
base: master
from

Conversation

@aparcar
Copy link
Member

commented Jul 9, 2019

This does some basic formal testing, just like done for packages.git.
Later compiling of device tree files, checking kmods or other more
specific tests could be done. However this is a first step to CI.

WIP: not fully tested yet

Signed-off-by: Paul Spooren mail@aparcar.org

@ynezz

This comment has been minimized.

Copy link
Member

commented Jul 15, 2019

Thanks a lot for your persistence, patience and your continuous effort on the CI improvements!

Anyway, as we're probably going to consider switch to GitLab I would really like to avoid any tighter integration with this proprietary services and would rather focus our work towards GitLab integration instead. It's also going to be much easier for you, as you already have all necessary admin rights there, right?

I envision following TODO list:

  • for the start, create just unidirectional GitHub -> GitLab mirroring bridge and start mirroring PRs from GitHub to our testing GitLab instance
  • setup CI and other tools properly at our GitLab instance so we could copy&paste the CI check results back here to the GitHub until we iron out the corner cases
  • once the CI checks are good enough, we could enable bidirectional GitHub <-> GitLab bridge and propagate back GitLab CI check results to GitHub as PR comments (or via some other possible integration hooks)
  • integrate Patchwork/mail workflow with the GitLab instance in the same spirit
  • cast a vote for making our GitLab instance our main platform for development
  • migrate bugs from FlySpray to GitLab issues (this is probably going to be little bit painful as FlySpray has no API access), redirect bugs.openwrt.org to gitlab.openwrt.org/openwrt/issues (or similar), shutdown bugs.openwrt.org
@aparcar

This comment has been minimized.

Copy link
Member Author

commented Jul 15, 2019

Thanks for the writeup, I'll meet lynxis today and try to get some stuff done

@diizzyy

This comment has been minimized.

Copy link
Member

commented Jul 17, 2019

At least do a public query of what people in general want if you want to attract people instead of bulldozing over everyone like last time. Why is there a need to change at all?

@ynezz

This comment has been minimized.

Copy link
Member

commented Jul 18, 2019

tl;dr Don't worry, there's no plan to leave GitHub.

At least do a public query of what people in general want if you want to attract people

Why would we need to do a public query? This is more about team members, core developers. We're just seeing a lot of issues here and there and we would like to fix them or at least attempt to. Some of us simply believe, that GitLab is probably going to help with this regard. You can see the list of the main pain points at the Hamburg 2019 meeting summary.

instead of bulldozing over everyone like last time.

By last time, you mean, when the project moved from SourceForge to GitHub? :-) Putting jokes aside, we first want to play with the GitLab little bit and once we think, that it would be good to proceed, then we're going to vote about it. It's a long term goal, I don't expect, that it would happen this year. We're probably going to make a summary and another talk about it first during next meeting in 2020.

Why is there a need to change at all?

  • unify and integrate currently fragmented workspace (patchwork, GitHub, Flyspray) at one place
  • faster/customizable workflow
  • boost QA/CI integration, automate patch/PR review processes
  • promote, use & develop other OSS project

Someone has asked about this topic on IRC so you can find more details there as well, it starts at Jul 15 09:20:31.

name: Check changes / verify commits
working_directory: ~/openwrt_packages
command: |
cat >> $BASH_ENV <<EOF

This comment has been minimized.

Copy link
@ynezz

ynezz Jul 18, 2019

Member

I think, that we've discussed this already, nobody sane is going to suffer with YAML pox voluntarily :-) so it would be better to keep the YAML file as short as possible and do the rest in the external Bash/Python/Perl scripts instead (probably in separate directory scripts/ci) ? Ideally those checks could be run as make targets make ci-formal-checks (defined in include/ci-commands.mk) even localy by the developers, so they could simply check the contributions before sending patch/PR, debug/develop those CI checks etc. This way it's probably also CI agnostic, so in the end it doesn't matter if it's run on CircleCI or GitLab CI or whatever we choose in the future.

This comment has been minimized.

Copy link
@aparcar

aparcar Jul 18, 2019

Author Member

Good point, I'll surely work on that

@diizzyy

This comment has been minimized.

Copy link
Member

commented Jul 18, 2019

@ynezz
One of the many reason for the split and merge was to become more open and attract more developers/contributors. This appears to have regressed quite a bit in the last months and things are starting to go into a more closed group thing once again but I guess this boils down to if you find yourself more valuable than the project itself. It's also very tiresome when some unwritten rule pops up out of nowhere and everything comes to a grinding halt.

Doing a survey like FreeBSD did would be a better idea rather than "we think" at least if you value the community and contributors..
https://youtu.be/9nc8N6GtAPg?t=1101
https://youtu.be/9nc8N6GtAPg?t=1195 (Q15)
https://youtu.be/9nc8N6GtAPg?t=1306 (Q14)

build: add ci-formal-checks and ci
This does some basic formal testing, just like done for packages.git.
Developers can test their commits locally by running `make
ci-formal-checks`. If working on another branch than `master`, add
`BRANCH=<branch>`. Later compiling of device tree files, checking kmods
or other more specific tests could be done. However this is a first step
to CI.

Signed-off-by: Paul Spooren <mail@aparcar.org>

@aparcar aparcar force-pushed the aparcar:ci branch from 142f5a1 to 581bc2f Jul 18, 2019

@ynezz ynezz changed the title [wip]build: add circleci [wip] GitLab evaluation Jul 19, 2019

@ynezz

This comment has been minimized.

Copy link
Member

commented Jul 19, 2019

tl;dr Nothing is going to change, GitHub would be still used as it is, source code mirror, feeds hosting and PRs would still be accepted

This appears to have regressed quite a bit in the last months and things
are starting to go into a more closed group thing once again

Can you please try to be more explicit, like what was done wrong, what should be improved? As you might guess, english is not my native language, so I probably don't express my thoughts in the understandable way, or I don't clearly understand your remarks and don't answer them as I would wish to.

more closed group thing once again

It might look like this from the outside, but believe me, it's only because there's only 42 hours in the day. It's really hard to find out time for development and then also handle interaction with the outside world. Us developers tend to suck at such tasks, it's like writing documentation. I mean, it took us almost a month to even provide some output from the Hamburg meeting. The project simply needs some kind of a liaison who could clearly communicate the ongoing efforts to the public.

Talking just about GitLab evaluation here. Should I understand, that you (or anybody else) feels to be put on the side regarding the GitLab evaluation? Like you (or anybody else) would like to have access to that testing GitLab instance and help @aparcar with the preparation of the GitLab for the evaluation?

We're simply improvising as we can't have selfhosted public gitlab.openwrt.org from the day one, so we've talked about this situation on the last summit in Hamburg (indeed this was semi-private, core team members mostly) and came to the temporary solution, that we would simply use @lynxis's private GitLab server (we're thankful for that offer) and probably dedicate some CPU slots from the buildslaves to the GitLab CI runners. This private server won't sustain the public traffic/load so it's invite only, but anybody interested in helping with the GitLab evaluation is really more then welcome and should simply just request access and join the effort.

It's also very tiresome when some unwritten rule pops up out of nowhere and
everything comes to a grinding halt.

I can't parse this sentence, what do you mean exactly, can you please be more specific? This GitLab evaluation is well, just evaluation, there's not going to be any rule about that, nothing is going to a grinding halt. I mean, even if the GitLab evaluation pans out and we're going to use it more for the development of the project (after more detailed discussion and proper voting process), we're still going to accept PRs from GitHub, nothing is going to change in this regard. So I really don't understand what is going to be halted.

Doing a survey like FreeBSD did would be a better idea rather than "we
think" at least if you value the community and contributors..

Can you be more specific and give me a hint, what should be in this survey? I mean, for me it boils down to the following:

  • a) should we start evaluating GitLab as possible next development platform (integrating git.openwrt.org, builds.openwrt.org, bugs.openwrt.org and Patchwork under one platform)

  • b) should we stay on GitHub, move projects and staging trees from git.openwrt.org to GitHub, replace bugs.openwrt.org with GitHub issues, integrate Patchwork with GitHub (convert patches sent over email to GitHub PRs)

  • c) provide better solution

But as I'm personally very strongly against b) I'm not going to prepare such survey (or even think about it for a moment), for the following reasons. I'm not going to suggest/recommend anything I'm not 100% sure is reasonable. Although I personally pay GitHub monthly (mainly because I'm lazy to maintain another server for my source code, wiki and issues) and putting aside Microsoft as the owner (I'm really neutral here, I don't care who owns it, really) I simply think, that using it as main development platform for any OSS/FS project is very big risk, because simply of the nature of this proprietary solutions, as in "Those Who Do Not Learn History Are Doomed To Repeat It." (BitKeeper, SourceForge, Google Code to name a few examples) :-)

@CodeFetch

This comment has been minimized.

Copy link
Contributor

commented Jul 25, 2019

@diizzyy It's this developer meritocrazy thing... Give up your life, commit as much as possible no matter how useless the commits are and you are the robot king of FOSS development. You can become a steering gear, but after all you will need to become a gear.

@bobafetthotmail

This comment has been minimized.

Copy link
Contributor

commented Jul 25, 2019

@diizzyy It's this developer meritocrazy thing... Give up your life, commit as much as possible no matter how useless the commits are and you are the robot king of FOSS development. You can become a steering gear, but after all you will need to become a gear.

In what projects you could commit your way to leadership like that? Asking for a friend

@CodeFetch

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2019

The only thing I wanted to say is that FOSS development is neither a question of democracy nor aristocracy and ends in a technocracy in the cloak of anarchy and meritocracy. Those who do, decide what get's done and those who decide can do more what they like to do and thus do more and get affirmation from their dev pals who act as they do and thus do more. There is a German word for this: "Glaubensgefängnis" - faith prison.

diizzyy's displeasure has a certain justification. If you don't ask all people involved, users and devs about or at least try to consider what they want, because they are not the "main devs", the question is: For who do you do FOSS projects? In that case only for yourselves... You are free to do this, but don't think you are selfless, when you do it. And don't expect people to be happy with your decisions. Similarly politicians live in a faith prison - why all of our governments kind of suck. The solution is simple. Don't reply to displeasure with "We do it, because we do it" in a shady long text. That's what politicians do and we should be better than them, because we have computers and free access to knowledge.

ynezz, nothing against you. You are a great person. Just trying to show up a communication problem here...

But as I'm personally very strongly against b) I'm not going to prepare such survey (or even think about it for a moment)

Sounds like ynezz doesn't, so it's likely never going to happen. Actually many people love using Github (and ynezz even pays them, because of laziness) and people more likely use Github issues than the bugtracker, more likely do pull requests than sending patches via email and more likely fork this project, all because of laziness. In this case laziness leads to survival of the fittest and that is Github.

"Those Who Do Not Learn History Are Doomed To Repeat It." (BitKeeper, SourceForge, Google Code to name a few examples) :-)

All very complicated solutions compared to Github. Nothing for lazy people. That's why they starved. Github is better (at least its UI) and thus does not repeat history.

Don't understand me wrong. Personally I prefer Gitlab. Personally I don't like iPhones, Windows, Facebook, Instagram, Twitter and so on... But we need to take the people who use and like it serious and try to understand them. If we think we are more qualified than these people to decide that their solutions are wrong or their opinion does not count, we will loose them. And in the end we live in a Glaubensgefängnis. That's why Linux is still not mainstream. That's why OpenWrt is still not mainstream. I remember an interview with Karl R. Popper where he said something like: "The problem with any ideology is that those who push it forward hope for a greater impact.".

@bobafetthotmail

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2019

If you don't ask all people involved, users and devs about or at least try to consider what they want, because they are not the "main devs", the question is: For who do you do FOSS projects?

That's the crucial point here, people keep thinking that FOSS projects have some kind of obligation to listen to people, or that they are somehow more "democratic" or "selfless" or something.

No they don't. FOSS only means you can fork the code and go on your merry way without issues, and contribute back if your view aligns with the project developers.

It is and will remain some form of oligarchy where the core developers have most power, and contributors have less, random users have near zero as usual.

The goal of most FOSS projects is just to pool development resources between like-minded individuals/organizations, not to be a political system that chooses "what is best for its users".

LEDE at the time of the fork did say that being more transparent was part of their goals, and theoretically this should also still hold true in the "new OpenWrt", but that's their own policy, not a FOSS requirement per-se.

Sounds like ynezz doesn't, so it's likely never going to happen.

Another important aspect people commonly get wrong in FOSS. None is forced to do anything, so if you really want something it's YOU that have to go and talk to people and do stuff.

ynezz said he does not have enough motivation to actually change the status quo. That's fine. Are you aware of the time and effort required to do a proper survey?

If you are interested in these things, the best way is doing them yourself, not shame-guilting people to do them on your behalf.

In this case laziness leads to survival of the fittest and that is Github.

And the last important point people commonly get wrong. FOSS projects are in most cases seriously undermanned and rely on contributor's spare time to do anything. SOME ones do have commercial entities and paid developers to pull the cart, but this isn't the case for OpenWrt.

"Laziness" is a very strong word in this case, even core developers have a very limited amount of time to devote to the project, I think a better term is "efficiency".

@CodeFetch

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2019

Indeed... You've got the point of what I wanted to say. We just need to be aware of these things to not scare people away with taking this for granted and aim to be better (if we have the time to). Most developers were users at some point and they wouldn't have started working on those projects if they wouldn't like them. Thus a successful FOSS project needs to be user focused.

  • cast a vote for making our GitLab instance our main platform for development

This is a very important point and a good way to go.

  • migrate bugs from FlySpray to GitLab issues

And this should be reconsidered as mostly users file issues and a system preferably without authentication that is simple for users and not so much devs is needed.

But those are just my two cents.

@ynezz

This comment has been minimized.

Copy link
Member

commented Jul 28, 2019

"Those Who Do Not Learn History Are Doomed To Repeat It." (BitKeeper, SourceForge, Google Code to name a few examples) :-)

All very complicated solutions compared to Github. Nothing for lazy people. That's why they starved. Github is better (at least its UI) and thus does not repeat history.

Indeed, Microsoft is taking it to next level tkashkin/GameHub#289 (GitHub blocked my account and they think I’m developing nuclear weapons and Github do not ban us from open source world as well)

@filips123

This comment has been minimized.

Copy link

commented Jul 28, 2019

Do you have any specific reason why do you want to switch to GitLab? GitHub has a lot more users and features so it works be better to transfer all current bug trackers and pull requests to it.

If this is because US sanctions, also don't switch to GitLab. It is also US based and hosted on Google servers. And it's even worse because Google is one of the biggest corporations and they also have to obey US sanctions.

Real solution would be to switch to self-hosted GitLab, Gitea, Gogs or other open source program. For more ideas and details see this Reddit topic.

@diizzyy

This comment has been minimized.

Copy link
Member

commented Jul 28, 2019

X is going to be dropped/moved from tree or we're changing X to Y is just a few things and all have a tendency to be "announced" indirectly. Having little to no communication at all doesn't help and are all on the same page or is it something a few individuals have decided?

I do think @CodeFetch's points are crucial for a healthy community in the long run but you are free to pick any direction you want.

As for consolidating services I think it's a great idea but everyone (especially main devs) or at least the vast majority needs be in agreement otherwise you'll have the same issue(s) as you have today. You can of course bulldoze everyone but that's no really ideal.

@ynezz
If you want, I can ask but in short they asked (regarding contribution) how much time people spent, what CVS they used, what they prefered and if changing X to Y would increase their amount of contributions.

@claell

This comment has been minimized.

Copy link

commented Aug 12, 2019

Nice to see this is evaluated. I like their issue tracker and think that having everything in one place will improve the situation.

@diizzyy I am unsure, whether you understand the situation correctly. This is only for evaluation and there will be a voting, although I am not completely sure, this will be public:

Putting jokes aside, we first want to play with the GitLab little bit and once we think, that it would be good to proceed, then we're going to vote about it.

@diizzyy

This comment has been minimized.

Copy link
Member

commented Aug 12, 2019

@claell
..and if you go one step further? Historically it's pretty much never been the case which exactly isn't keeping the community together.

@claell

This comment has been minimized.

Copy link

commented Aug 12, 2019

I don't know about the history, but there is no reason to believe that behaviour would not change.

Additionally I want to mention that according to the chat protocol that @ynezz linked there are no plans to shutdown GitHub, but to keep the service running, like it is currently. So the only affected services I know are the issue tracker and the git hosting. Is this the things you want to keep unchanged or were you just fearing that they will shutdown GitHub?

@adrianschmutzler

This comment has been minimized.

Copy link
Contributor

commented Aug 13, 2019

@aparcar @ynezz Since this was mentioned earlier, I would be interested in taking part ("be invited") in the GitLab evaluation (if a reasonable point for that is reached). I can't promise that I will be helpful, but I would be interested in following the process ...

@ynezz

This comment has been minimized.

Copy link
Member

commented Aug 17, 2019

@aparcar @ynezz Since this was mentioned earlier, I would be interested in taking part ("be invited") in the GitLab evaluation (if a reasonable point for that is reached). I can't promise that I will be helpful, but I would be interested in following the process ...

Invitation sent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.