Collaborte on merges

[esnosy]

I think I can help with merges, I did a merge for practice, got it setup, should I do weekly merges?, maybe we could alternate weeks,

When should I do merges, also I can try to make one today

What do you think?
@ReinerBforartists

[ReinerBforartists]

Thanks for the idea. It's always a good practice to know the whole pipeline. But we don't have this much changes really per week that would make a daily merge necessary.

But more important, the merge job is not only to merge the new code. But also to update the manual, and add the new tasks that arrives with the merge. This means that you have to go through the Blender commits one by one. I have a task here in the Manual tracker for just this:

Bforartists/Manual#231

And you have to make sure that our changes remains, and are not removed by the merge. So this is best done by one person who knows the big picture already.

And there is also building the devbuilds involved. Before the merge. And that's something that i still have to do then, since i am the one with server access.

I personally would say we stay with the current model. One merge per week, and one person who does it. Everything else could quickly escalate in the one or another way. But you are welcome to participate. We just have to coordinate ourselves then.

[esnosy]

Agree, I think yeah if we collaborate on merging we should coordinate, one could do a merge, upload it as a branch that would be part of the work, and the yeah the changes should be in the manual, I have to try adding to the manual too.

Also I think the title "Merge more often" is not right it should be "Helping with weekly merges"

[ReinerBforartists]

You can change the title if you want :)

This job is hard to split. Sometimes i notice issues already when i do the merge. And doing it at two places has also its issues. As told, for me it's the one who does it should do it all.

[Draise14]

Guess you're right, it's like the final gate to a distribution. I guess learning to "merge" maintenance entitles more than just merging code. Gotcha.. hmm.

Tasks to Merge:

• Merging with Bforartists code changes as priority first (so far, success)
• Reporting and fixing merge conflicts
• Prevent bugs
• Report tasks to update anything the Bforartists vision would need to accomodate with the new merge (new icons, new operators, left aligned, new features to expose, etc)
• Updating the manual for the new features
• Going through blender commits to know new features
• Packing a distributable for multiple OS's..etc
• Updating server files (is there an FTP process for that?)

Though, I am all for supporting a second maintainer and core developer of this process - and alternating weeks with the exact same workflow between the two of you would be really comforting for the longevity of Bforartists. And maybe it will give you a bit of a break @ReinerBforartists! Ease up on the weeklies.

Are you down to open up training for that? We could do a simulation merge with the extra tasks at a later date, see if you can be hands free with the end goal of Bforartists dev build merge work, new update tasks and updated manual and distributable all green.

[ReinerBforartists]

I am open for anything. I feel for a break anyways. I feel honestly a bit burned out at the moment. I stare at the code and nothing goes forward. Time to do something else for a while. So when you want to do the merge this week, then you are welcome to do so :)

I will push the latest update to the manual tomorrow, and build the latest devbuild versions. Then it all starts with the merge at sunday. And ends next saturday with pushing the updates to the manual and building the devbuild versions ...

EDIT, mh, this part makes it so complicated. When i commit something, then i usually also fix the commit in the manual already. And the odt files are no text files. You cannot merge them. Only one person can work at it at one time. That's usual me.

Packing a distributable for multiple OS's..etc

At the moment that's windows and a Linux build. For the linux build we are migrating to the docker at the moment. But this requires still a bit work.

Updating server files (is there an FTP process for that?)

Nope. That's why i said i have to do this part. FTP is a dead protocol anyways :)

Mh, we have nextcloud installed at our server. But i haven't logged in for eons. I would first need to relearn it. I could invite you when you prefer this way. But easier solution is to share files by google drive and similar soltions. Just drop me a pm or mail.

[ReinerBforartists]

So let's get started. Here's how i do it. Note that there is more than one working way. Allowed is what works. But the way i do it is already bullet proof. I had more than one trouble at my way. Sorry in case i mention the obvious here and there. Better one time too often ...

First switch to the sync_blender_master branch. You cannot commit to master directly. And even i don't do it. The merge is done in the sync_blender_master branch first.

You said you managed to create a working merge already. So i guess we don't need to cover this detail. And i also assume that you already figured out how to add a remote. I am at Windows, and use Tortoisegit for this part. And not the command line. So i would not be of big help here anyways.

Before merging Blender you need to merge the changes from the Bforartists master. The changes that we did under the week.

git merge bforartists/master

When done commit it to the sync_blender_master branch. Then make a copy of the repo. This step is required since most merges removes the addons folders. Blender uses sub modules for that, we not. And the copy is the place where you can grab and copy back the addons to their place. This way you can also compare code parts in case the merge fails.

If not done already, clone a Blender repository. Sometimes it is necessary to compare the code between Blender and Bforartists directly. And next time, make a copy the repo before you update the Blender repo. Sometimes it is easier to just compare the changes between Blender before and after the merge, and add the changes to the Bforartists repo at a one by one case.

Download Sources and Libraries
Create a folder to store your copy of the Blender source code. This guide will assume your chosen folder is C:\blender-git.

Download the Blender source code:

cd C:\blender-git
git clone git://git.blender.org/blender.git
Download the Blender libraries:

cd C:\blender-git\blender
make update

To update to the latest changes afterwards, run:

cd C:\blender-git\blender
make update
git submodule foreach git pull --rebase origin master
make

Adjust the paths to your needs ...

Then, back at Bforartists, you need to add the blender master as a remote so that you can fetch it for the merge. Merge Blender master.

You will nearly always end in a "Automatic merge failed Please resolve ... " messsage then. So now comes the interesting part, resolving the conflicts. You can do this with a merge tool. I have no idea what possibilities Linux offers here. I work at Windows. I use Tortoisegit here, in combination with Meld to compare the code.

So what i do is to call a list of the conflicts in Tortoisegit, copy it, save it, resolve all conflicts using our Master, and then go though the conflicts one by one. And add the new code by using Meld.

When all conflicts are resolved, compile. Fingers crossed that it compiles without errors.

Now compare the following folders with the Blender repo. That's the sub modules. And they don't update automatically, we don't use the Blender sub modules.

Don't copy over the git, gitignore and arcconfig files!

release\scripts\addons - DON'T blindly copy over. We have more than one Bforartists change here, and own scripts. Some im- and exporters have icons. And we have modified the materials_library_vx addon.
release\scripts\addons_contrib - no content from us. So it is save to copy the files over in case there are changes.
release\datafiles\locale - The language files
source\tools - I have modified the blender_theme_as_c.py here, and added a comment how to use it. Rest should be equal to Blender

Make sure Bforartists is starting before you commit the changes. Also check for errors in the console.

Then make a pull request.

Now comes the cumbersome part. Go through the list of commits, down to the last merge. And check one by one for relevant changes in the manual. The time frame is always the latest week. So for this week it's from 11.04 up to the day you do the merge.

When there is a new tool, or a change in the UI, then we need to update the manual. This one by one comparison gives you also an overview over the latest changes. And you will stumble across new tasks. New props means we have something to align at the left side for example ...

This part can go fast. But it can also go for several days when there are lots of changes. The issue is located here:

Bforartists/Manual#231

I usually commit with this title + the date. For example for the latest commit: Update manual to BFA 2.8.1 / 2.9.0 #231 04.04 to 11.04

I hope i have covered most of the trouble zones. I will commit the latest manual changes now. Good luck. And in case you have questions, i am here :)

[esnosy]

So let's get started. Here's how i do it. Note that there is more than one working way. Allowed is what works. But the way i do it is already bullet proof. I had more than one trouble at my way. Sorry in case i mention the obvious here and there. Better one time too often ...

First switch to the sync_blender_master branch. You cannot commit to master directly. And even i don't do it. The merge is done in the sync_blender_master branch first.

You said you managed to create a working merge already. So i guess we don't need to cover this detail. And i also assume that you already figured out how to add a remote. I am at Windows, and use Tortoisegit for this part. And not the command line. So i would not be of big help here anyways.

Before merging Blender you need to merge the changes from the Bforartists master. The changes that we did under the week.

git merge bforartists/master

When done commit it to the sync_blender_master branch. Then make a copy of the repo. This step is required since most merges removes the addons folders. Blender uses sub modules for that, we not. And the copy is the place where you can grab and copy back the addons to their place. This way you can also compare code parts in case the merge fails.

If not done already, clone a Blender repository. Sometimes it is necessary to compare the code between Blender and Bforartists directly. And next time, make a copy the repo before you update the Blender repo. Sometimes it is easier to just compare the changes between Blender before and after the merge, and add the changes to the Bforartists repo at a one by one case.

Download Sources and Libraries

Create a folder to store your copy of the Blender source code. This guide will assume your chosen folder is C:\blender-git.

Download the Blender source code:

cd C:\blender-git

git clone git://git.blender.org/blender.git

Download the Blender libraries:

cd C:\blender-git\blender

make update

To update to the latest changes afterwards, run:

cd C:\blender-git\blender

make update

git submodule foreach git pull --rebase origin master

make

Adjust the paths to your needs ...

Then, back at Bforartists, you need to add the blender master as a remote so that you can fetch it for the merge. Merge Blender master.

You will nearly always end in a "Automatic merge failed Please resolve ... " messsage then. So now comes the interesting part, resolving the conflicts. You can do this with a merge tool. I have no idea what possibilities Linux offers here. I work at Windows. I use Tortoisegit here, in combination with Meld to compare the code.

So what i do is to call a list of the conflicts in Tortoisegit, copy it, save it, resolve all conflicts using our Master, and then go though the conflicts one by one. And add the new code by using Meld.

When all conflicts are resolved, compile. Fingers crossed that it compiles without errors.

Now compare the following folders with the Blender repo. That's the sub modules. And they don't update automatically, we don't use the Blender sub modules.

Don't copy over the git, gitignore and arcconfig files!

release\scripts\addons - DON'T blindly copy over. We have more than one Bforartists change here, and own scripts. Some im- and exporters have icons. And we have modified the materials_library_vx addon.

release\scripts\addons_contrib - no content from us. So it is save to copy the files over in case there are changes.

release\datafiles\locale - The language files

source\tools - I have modified the blender_theme_as_c.py here, and added a comment how to use it. Rest should be equal to Blender

Make sure Bforartists is starting before you commit the changes. Also check for errors in the console.

Then make a pull request.

Now comes the cumbersome part. Go through the list of commits, down to the last merge. And check one by one for relevant changes in the manual. The time frame is always the latest week. So for this week it's from 11.04 up to the day you do the merge.

When there is a new tool, or a change in the UI, then we need to update the manual. This one by one comparison gives you also an overview over the latest changes. And you will stumble across new tasks. New props means we have something to align at the left side for example ...

This part can go fast. But it can also go for several days when there are lots of changes. The issue is located here:

Bforartists/Manual#231

I usually commit with this title + the date. For example for the latest commit: Update manual to BFA 2.8.1 / 2.9.0 #231 04.04 to 11.04

I hope i have covered most of the trouble zones. I will commit the latest manual changes now. Good luck. And in case you have questions, i am here :)

Many thanks for the extra guide🙏, will start as soon as possible.

[ReinerBforartists]

You're welcome. As told, don't hesitate to ask when you are in trouble. And better ask twice :)

Just because we might need it soon, i have resurrected our nextcloud now. You should be able to register at it. And this makes it easier to share files in our team then. It's our own dropbox so to speak.

I don't want to share our access page in the public to reduce spam. So for further instructions and informations see in our developer section at our homepage

[esnosy]

You're welcome. As told, don't hesitate to ask when you are in trouble. And better ask twice :)

Just because we might need it soon, i have resurrected our nextcloud now. You should be able to register at it. And this makes it easier to share files in our team then. It's our own dropbox so to speak.

I don't want to share our access page in the public to reduce spam. So for further instructions and informations see in our developer section at our homepage

Awesome! :)

[ReinerBforartists]

I just noticed that Meld is also available at Linux. I love this code comparison tool :)

https://meldmerge.org/

[esnosy]

I just noticed that Meld is also available at Linux. I love this code comparison tool :)

https://meldmerge.org/

Oh I didn't use it before will check it out

[ReinerBforartists]

Surprise ...

[ReinerBforartists]

Bites his tongue and stares at the new workspace

I'm really having a hard time at the moment to let this job go for a week :D

[esnosy]

Bites his tongue and stares at the new workspace

I'm really having a hard time at the moment to let this job go for a week :D

Oh 😄, I realised this should have been in discussion section XD

[ReinerBforartists]

No, it fits also here. It is a task :)

How's the merge going?

[esnosy]

No, it fits also here. It is a task :)

How's the merge going?

Going very well, it is pulling updates now, next step is to tackle merge conflicts and update manual

[ReinerBforartists]

It's not this much conflicts this time. Lucky you. But fun with the submodules again :)

When you arrive at the Lock Camera to View conflict, there are a few bfa comments in the way. I would say keep the parts that makes the navigation elements horizontal. And remove the rest. We first have to check the Blender way now. It is one of the tasks that needs to go into the tracker then. To check what happened here to the lock.

This is a general rule. When unsure, return to the Blender code. Don't try to resolve everything in the merge. But make a tracker task then. There might be more involved ...

[esnosy]

It's not this much conflicts this time. Lucky you. But fun with the submodules again :)

When you arrive at the Lock Camera to View conflict, there are a few bfa comments in the way. I would say keep the parts that makes the navigation elements horizontal. And remove the rest. We first have to check the Blender way now. It is one of the tasks that needs to to into the tracker then. To check what happened here to the lock.

This is a general rule. When unsure, return to the Blender code. Don't try to resolve everything in the merge. But make a tracker task then. There might be more involved ...

Great, will follow these tips as said 👌

[esnosy]

what about this change?
it seems a function was removed, should I restore it?
EDIT: restoring it seems harmless, it doesn't depend on anything that was removed, will test too

[esnosy]

found it
blender/blender@4baf396

[esnosy]

they are trying to remove the camera lock to view gizmo operator

[esnosy]

should we remove this feature?

[ReinerBforartists]

When in doubt, return to the Blender code, and make a task. Which you did.

Ah, so they just reverted the code to remove this lock button again. For us this button is super useful. We should readd it. I have opened the task again ...

[esnosy]

Ok great I also found a bug report, they removed the gizmo after it

https://developer.blender.org/T87386

[ReinerBforartists]

I will close this issue now. The merge task is fulfilled. Mh, wouldn't it be a good idea to move this to the discussions instead? It was not really a task. But might be worth a second read for others ....

EDIT, ah, quick decision. Moved to discussion.

[Draise14]

I've noticed that the documentation step of a merge could be delegated a bit more to be more efficient, there is the technical merge, success - but now it's the documentation.

I guess if the person merging can, he could create a quick note/task in the documentation tracker or note in the task he closed (then assigns) to anyone else willing to update documentation.

I can't do a merge, but I can do documentation - like I have helped with some tutorials. As long as it's in bite sized chunks.

Example:

A "Documentation" tag is placed, or a relevant task in the documentation tracker, a documentation task/guide is placed in the relevant task, then you can assign me, and I can do it on Mondays, which is my slow random task day.

An example note:

Documentation Task:

Update Screengrab of View menu in:
16.1.3 Editors - Dope Sheet - View Menu
17.1.2 Editors - Timeline - View Menu
18.1.3 Editors - Graph Editor - View Menu
19.1.3 Editors - Drivers Editor - View Menu
20.1.3 Editors - NLA Editor - View Menu

[ReinerBforartists]

Hmmm. The regular manual updates after a merge is definitely something that should be done by the person who does the merge. This is part of going through the latest commits to document the new features then. It is a check if we have catched everyhing, if the description fits to the feature, and if nothing is broken.

Either way, you ask at the wrong time. We come to the end of the development cylce anyways. And so just the hard and time consuming tasks are left. No small development tasks for you left, sorry. But you could do some community and advertisement work. You could for example post some updates to the facebook page. Or search for facebook groups and make some advertisement for Bforartists :)

[Draise14]

Yeah, I noticed. I do the social rounds every now and then routinely, also mod the discord activity, no problem there.

Was a good effort trying to merge here, and seeing now the next challenge of also getting the documentation and tracker on a merge updated well is the next step.

[ReinerBforartists]

Ah, forgot, @iyadahmed , i will do the manual update tomorrow. I agree that it was a great idea to let you do the merge. It is important to know the whole pipeline. But this lasts simply ways too long now. I don't want to wait anymore until you find the time to finish the manual parts. This blocks me. I want to continue with development. And sunday is already the next merge.

[esnosy]

Great yep it can't be delayed further