Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

⚠ MO2 - Mod Organizer 2 overhaul discussion #598

Closed
frostworx opened this issue Sep 23, 2022 · 67 comments
Closed

⚠ MO2 - Mod Organizer 2 overhaul discussion #598

frostworx opened this issue Sep 23, 2022 · 67 comments
Labels
enhancement New feature or request ModOrganizer 2 Issues related to installing or using ModOrganizer 2 with SteamTinkerLaunch

Comments

@frostworx
Copy link
Collaborator

This issue is meant to be a generic brainstorming issue for a possible MO2 overhaul.
Feel free to contribute any ideas you have to improve it. Of course PRs are very appreciated as well as usual.

Starting with this proposal:
If dotnet is no longer required at all for MO2 (can someone confirm it is no longer required for modding?),
it might make sense to simplify the whole implementation by using one global "install directory" (=MO2 Program Files dir)
and f.e. symlink it into the corresponding compatdata without any further installation.
Imho this would help massively to solve almost all problems people have with the confusing "gamemode" and "standalone modes".

@frostworx frostworx added the ModOrganizer 2 Issues related to installing or using ModOrganizer 2 with SteamTinkerLaunch label Sep 23, 2022
@sonic2kk
Copy link
Owner

sonic2kk commented Sep 23, 2022

DotNet was removed a while ago, right? I modded Oblivion and New Vegas around June-July time and didn't really have any problems (using Proton 7.0, forget what I used for Oblivion but I used 7.0-4 for New Vegas). So in my own experience, DotNet shouldn't be required anymore. Other users have reported modding Oblivion with success too on the issue discussing the recent MO2 download patch.

Maybe any work and testing for this could be done on a separate branch? That can get... Tedious, but to preserve existing mod installs and get feedback having a separate branch once any kind of work starts on this might be nice.

Just my thoughts though, I've only modded 3 games that use similar engines and modding processes. It's very possible there are problems in other areas I haven't encountered. Though I will say the recent link to the Rockerbacon(?) ModOrganizer 2 project mentions Wine having "built-in support" which I assume means that MO2 should be fully compatible with Wine. But this project seems to mainly focus on Bethesda games, which is the extent of my experience too.

I would guess that MO2 might not require DotNet but some of the other GUI applications that you can launch via MO2 for the likes of Skyrim character editing might need DotNet. I couldn't say for sure though.


Just realised the tone of my reply might not get it across (sorry!) but I think being able to clear up any confusion here is a great idea :)

@frostworx
Copy link
Collaborator Author

yeah we recently discussed this already, but I haven't found the corresponding issue anymore to refer it here (haven't searched for long and assumed you'd reply here anyway again. thanks!)
You would not beliebe how much time I spent with testing and writing all that MO2 stuff (without actually using it...).
It once worked fine but apparently it doesn't anymore...
It would be really "nice" if people interested in MO2 would take part in a discussion here and contribute what they know.
Although it seems likely that dotnet is no longer required anymore, I would appreciate it if several people could confirm this at least.

@frostworx frostworx changed the title ⚠ - MO2 - Mod Organizer 2 overhaul discussion ⚠ MO2 - Mod Organizer 2 overhaul discussion Sep 24, 2022
@frostworx
Copy link
Collaborator Author

Of course when removing the Standalone Mode, it is no longer possible to see all modable found games in MO2 and switch between them, because the compatdata of one game doesn't have "the other games installed" (it should work with doing the same symlink voodoo which is currently used in the Standalone Mode compatdata, but messing around with symlinks in the game compatdata should be avoided obviously and therefore isn't an option)

So if you like the existing option to have all modable games in one mo2 running and would like to keep that function_(=Standalone Mode)_ leave a comment.

If you don't care about this and would prefer one MO2 per game (in its own compatdata, so a rewritten game mode and much more simplified as the current implementation - so pretty much without any possible confusing trouble) leave a comment as well.

@sonic2kk
Copy link
Owner

So if you like the existing option to have all modable games in one mo2 running and would like to keep that function

Oh! I do use this every so often and its quite useful. Having a global MO2 install works basically like it does on Windows. Just voicing my support for keeping it 🙇

@frostworx
Copy link
Collaborator Author

Just tried it and agreed, it still seems to work fine.
As this would have been the main task to rewrite the mo2 implementation, I might just cancel the idea to rewrite it generally.
Haven't checked yet why the nxm urls do no longer open, but as the code didn't change here and the desktop files are still identical to those created months ago, I'd think it might be possible, that this is some "external" problem.

After installing mo2 during above test (started with steamtinkerlaunch and mo2 configs), I recognized that the "Connect to Nexus" button under mo2->settings->nexus doesn't do anything. I am not sure if this is related, but it could also explain the missing nxm url functionality (as in communication between wine/proton to the browser does not work at all. If so, there's nothing to fix on steamtinkerlaunch side)

@sonic2kk
Copy link
Owner

I'll have to retest but recently when I was testing this for another issue, I was able to use the "Connect to Nexus" button. It opened my default browser and prompted me to authorise. This happened with both Standalone Mode and Game Mode.

@frostworx
Copy link
Collaborator Author

snippet from WINEDEBUG:

00c4:fixme:system:DisplayConfigGetDeviceInfo Unimplemented packet type: 11
00c4:fixme:uiautomation:UiaRaiseAutomationPropertyChangedEvent (000000002D008DC8, 30005, 0000000000115B20 {VT_EMPTY}, 0000000000115B00 {VT_BSTR: L"Connect to Nexus"}): stub

@sonic2kk
Copy link
Owner

I forget which version of Proton I was using, I think I was just using 7.0-4 in Game Mode. I do my testing for this on my laptop because it's separate from my modded installs on my PC, so when I get a chance to test on my laptop again I'll see what's up on my side 😃

@frostworx
Copy link
Collaborator Author

those lines are written when "Connect to Nexus" is clicked: (with WINEDEBUG enabled):

00c4:fixme:win:GetPointerDevices (00000000001162E0 0000000000000000): partial stub
00c4:fixme:system:DisplayConfigGetDeviceInfo Unimplemented packet type: 11
00c4:fixme:uiautomation:UiaRaiseAutomationPropertyChangedEvent (000000002AF15B28, 30005, 0000000000115AF0 {VT_EMPTY}, 0000000000115AD0 {VT_BSTR: L"Cancel"}): stub
00c4:fixme:bcrypt:BCryptGenRandom ignoring selected algorithm
00c4:fixme:uiautomation:UiaRaiseAutomationPropertyChangedEvent (0000000028402E88, 30047, 00000000001162D0 {VT_EMPTY}, 00000000001162B0 {VT_R8: 317.000000}): stub

The mo2 in-gui log also logs an "opening url" line, so very likely this is the core issue (and therefore not steamtinkerlaunch related)

the mo2 internal logfile

~/.config/steamtinkerlaunch/mo2/compatdata/pfx/drive_c/users/steamuser/AppData/Local/ModOrganizer/nxmhandler.log:
also logs:

[So Sep 25 18:35:38 2022] EnableNonClientDpiScaling() failed for HWND 0x300d6 (120) (Aufruf nicht implementiert.)
[So Sep 25 18:35:38 2022] Unable to enumerate family ' "Noto Sans Inscriptional Parthian" '
[So Sep 25 18:35:41 2022] "C:/Modding/MO2/nxmhandler.exe" "reg" "cyberpunk2077" "C:/Modding/MO2/ModOrganizer.exe"

Aufruf nicht implementiert means function not implemented, which probably is the same issue.

@frostworx
Copy link
Collaborator Author

frostworx commented Sep 27, 2022

played a bit around: I found the problem why the nxm protocol download no longer works. should be easy to fix.

damn, I didn't.

edit:
I actually tested the nxm download handler and it just works for me multiple times reproducible...!?

edit2:
not 100% yet, but seems like something in ModOrganizer.ini enables/borks the functionality.
no time left for today, maybe more tests tomorrow...

edit3: nope, mo2 seems to ignore mo2 sometimes when it is already open, but seems like it is always reproducible that it opens it when closed and starts reliably the download then.

not testing any further for now - works for me and possibly not steamtinkerlaunch related.

@frostworx
Copy link
Collaborator Author

frostworx commented Sep 28, 2022

Yay, impressive feedback overall so far! 👍

I wonder what it takes to simply keep the "global instance" implementation for the "Standalone Mode", and implement a "portable instance" per game for the "Game mode" and sync (symlink?) all required files (e.g. downloads) to avoid duplicates.
Of course the "Modes" would be renamed accordingly to match the upstream global and portable instances.
This would help of course to also add wabbajack support (if feasable)
Undecided yet, if I'll look into this...
I'll dive a bit into a basic approach to see how much work this will be. wip (don't hold your breath)

Current guess: shouldn't be too complicated, but relatively big

edit: I was on a short trip for the last few days and lost the current wip state.
no idea when I can pick it up again for now.

@sonic2kk
Copy link
Owner

sonic2kk commented Oct 6, 2022

Haven't had time to look into the code yet but I just saw on the MO2 wiki that in the desktop file template I added, the STL path in the Exec line was /usr/bin/SteamTinkerLaunch blah blah.

If STL hardcodes its own path in the MO2 desktop file that's something that could be changed. STL on Deck (and with ProtonUp-Qt) is at ~/stl/prefix for example.

Since we also can't always rely on STL being on the path (manual non-root installs where the script could be anywhere), is there a way we can set this to the STL script?

Sorry if this is already being done, it's very possible the template I copied set that path because that was my installation path to STL. But I wanted to flag this up just to make 100% sure 😃

@Retrograde-i486
Copy link

So I'm not sure how helpful I can be but I just wanted to say that I just completed a fully modded run of Fallout 4 with STL and Mod Organizer 2 and it worked perfectly. I've been keeping my system up-to-date (Garuda Linux 5.19.11-zen1-1-zen x86_64) and I didn't think I was updating STL but upon checking I guess it's been getting updated since I'm on v11.11 which appears to be the latest release.

From what I've been seeing it seems people are having the most issues with the flatpack version?

@frostworx
Copy link
Collaborator Author

Thank you very much for confirming @Retrograde-i486!
It is even very helpful, because usually people only say something when something does not work :)
(I'm still working on a rewrite to simplify MO2, but it will need some more time until done)

@Retrograde-i486
Copy link

Retrograde-i486 commented Oct 6, 2022

OK good to hear, I've been running this install of MO2 and STL since July and every time I would update my system I'd check and see if the game still loaded up with the notion that I'd restore from a snapshot if it didn't. Through every Wine and STL update I kept waiting for something to break and it never did so I'm happy to report that I completed all the DLCs and the main quest modded on Linux:

Fallout 4 Boom:
https://imgur.com/01tRe1e

MO2 List:
https://imgur.com/4MP7n5u

If anyone has any questions I can try and help

@frostworx
Copy link
Collaborator Author

Wow, thank you very much! Highly appreciated!

@KBouder
Copy link

KBouder commented Oct 7, 2022

Quick question about MO2 under proton, is it supposed to be a extremely slow and unresponsive? Is there any way to improve performance while using it?

@frostworx
Copy link
Collaborator Author

no, it works normally. not steamtinkerlaunch or topic related though

@KBouder
Copy link

KBouder commented Oct 7, 2022

no, it works normally. not steamtinkerlaunch or topic related though
Sorry, for being off topic. It definitely does not run normally for me for some reason.

@sonic2kk
Copy link
Owner

sonic2kk commented Oct 7, 2022

If you're using a version of GE-Proton the issue might come from its theme, you can disable by enabling Winecfg -> Play -> Desktop Integration -> Theme -> Choose "(No Theme)"

Apart from that I'm not sure why it would run slowly. Might be worth opening a separate issue with more details for this if it persists and if you have the time :-)

@frostworx
Copy link
Collaborator Author

frostworx commented Oct 7, 2022

after some funny issues comments I started spending at least some minutes on the MO2 case before the weekend (yay, finally)

short description:

steamtinkerlaunch generates a small mo2 download script (MODLER in the code if someone is curious) in /dev/shm/steamtinkerlaunch/ which is triggered via
steamtinkerlaunch mo2 u (which in turn is triggered by the nxm mimetype/desktopfile).

When clicking a nxm url, the script is started with the mod url being appended as command line argument though the steamtinkerlaunch function dlMod2nexurl. The script contains game instance specific entries and therefore can't be used as generic script. For that reason, the script is auto-generated on demand and removed when steamtinkerlaunch closes.
I think (not yet tested) that the download only works fine until that script is still available - means the "else part" (the script MODLER is not available) needs to be debugged, tested and possibly replaced.
Just a wild guess so far, but it would explain certain reports/experiences.

edit: of course I will check that and add it to the list of the to-be-rewritten/tested mo2 stuff.

edit2: I think it could be possible, that the download script could be generic (and therefore does not be created) when using a portable instance. I'll elaborate on a separate comment what I'm trying to achieve with the rewrite (hopefully understandable 😏)

@frostworx
Copy link
Collaborator Author

frostworx commented Oct 7, 2022

summary of the current rewrite plan:

  • keep the "global" MO2 implementation (mostly) as-is: one mo2 specific compatdata which has all instances of all found games ready for modding. Only meant for modding and not for playing games and therefore meant to be started independantly (command line or "Mod Organizer 2" button in the settings menu)
  • add "portable" instance mode: when starting MO2 directly through starting a game, steamtinkerlaunch created/opens a "portable" instance inside the game compatdata automatically. This allows directly modding the started game and drops the need of most symlinks and separate compatdata and proton versions
    It will make sense to use the same download path used in the "global" setup.
    If someone has hints what is required for sharing the list of enabled mods with the game instance of the "global" setup with the "portable" setup any hints welcome (for example merging ModOrganizer.ini parts already is enough?)
    (it might be worth to consider opening a portable instance directly on demand without starting the game)
  • update/improve the nxm protocol handler if possible (maybe also separate handling for "portable", which will likely be more stable)
  • (if the "portable" implementation is ready, check if it is worth to also add wabbajack support)

@KBouder

This comment was marked as off-topic.

@sonic2kk

This comment was marked as off-topic.

@KBouder

This comment was marked as off-topic.

@frostworx

This comment was marked as off-topic.

@sonic2kk
Copy link
Owner

The branch for this is still open and pending contributions from any testers to see how stable it is. Once we have more feedback I can take a look at merging this in.

@sonic2kk
Copy link
Owner

sonic2kk commented Nov 2, 2022

Tentatively planning to test the mo2-overhaul branch with my own modded setups sometime on the 25th. Any tests from other users before then would be much appreciated! The branch is forked off from an older version of STL, so if you are testing you probably won't want to use that branch as a full-time replacement for your regular STL usage :-)

If all goes to plan, user feedback is good (or silent 😛) and my existing mod setups still work as expected (from glancing at the branch, they should), I'm hoping to get this merged before the end of the month, in time for broader testing before the next release.

If there are issues and we have to hold off on merging, a couple of fixes from this branch (namely the Wine downloading fixes) will be pulled in for the next release.

@har-nick
Copy link

har-nick commented Nov 10, 2022

Hi there! Two issues I've noticed getting MO2 to work.

  1. STL doesn't seem to recognise any lang file besides english.txt. I'm unsure if I'm doing something wrong as copying german.txt and renaming it to english.txt still shows English strings. Setting the lang= option works, but has the same effect.

  2. Anything to do with MO2 throws an error:

cat: /usr/share/steamtinkerlaunch/misc/mo2games.txt: No such file or directory

Again, I'm unsure if I'm doing something wrong here. I'm just running STL directly from the pulled folder after checking out the overhaul branch.

@frostworx
Copy link
Collaborator Author

https://github.com/sonic2kk/steamtinkerlaunch#how-do-i-install-it

@sonic2kk
Copy link
Owner

Hi! Thanks for testing the MO2 overhaul branch!

Your issue with the language files could be because the MO2 branch is behind master, and likewise your MO2 Games problem is probably also because it's behind master. I guess this doesn't happen for you on master from your other comments :-)

So to test the MO2 branch, you have to use sudo make install to install MO2, taking care to remove any redundant package you may have installed from your package manager. If you're on Steam Deck it should just work by double clicking the script.

I should add a more explicit note somewhere, but to test development branches it's best to do a root install with sudo make install. When you're done you can remove the root install with sudo make uninstall (you may need to go back to the master branch to test this).

Good luck!

@har-nick
Copy link

Thank you detailing the install process out for me! Set MO2 up the same I did previously and downloading through the Nexus works as expected!

@sonic2kk
Copy link
Owner

Thank you detailing the install process out for me! Set MO2 up the same I did previously and downloading through the Nexus works as expected!

That's amazing news!!! I am hoping to get this branch merged in sometime in the next couple of weeks so that is awesome to hear! Hopefully we can get more tests, and on top of that I'm going to test with my existing mod installs when I get the chance. If all works as expected I'm planning to have this merged before the end of the month and in place for STL v12.

Please feel free to report any issues you encounter as the branch is still in development :-)

@sonic2kk
Copy link
Owner

I haven't gotten about this 😅

This branch is almost 2 months old, at this point I don't think there will be many more testers, and the branch has seemed to have some positive improvements.

The plan is to resolve the merge conflicts (of which there are quite a few iirc) and then do some quick testing to ensure the following:

  1. New, fresh mod setups work
  2. Existing mod setups work
  3. There are no other regressions from merging

I will probably spend the next 3-4 days seeing that the Hedge Mod Manager support goes over smoothly enough, and fix any other reported issues as best I can. Then I'm planning on taking a bit of a break, probably 2 weeks or more, to enjoy some games and chill out for a while before merging in the mo2-overhaul branch. I don't want to merge it in and then disappear.

This branch will make its way into v12 if there are no significant problems 👍

@sonic2kk
Copy link
Owner

On break, but a draft PR #674 is up now. Testing is still needed on my end, but the conflicts are resolved.

Probably gonna get it merged in a few weeks from now.

@tetoNidan
Copy link

Not sure if this is the right place to suggest this but since windows is not case sensitive and linux is, I have been creating links for folders in multiple cases. Mod creators dont really have to stick with any naming convention. Dont remember which mod might have been for oldrim that used Skse as their folder name for SKSE. Things can get out of hand in the data folder when you have to manually install some mods.

$ ln -s SKSE skse

$ ln -s Scripts scripts

$ ln -s Plugins plugins

$ ls -s Meshes meshes

$ ln -s Tools tools`

Dont know it this helps at all. Not that big of a deal unless you manually install something though. Like some FNIS mods or maybe an enb or something.

Also I was thinking about setting up a new play through of FO4. If I can help with testing your new MO2 branch without messing with my current play through of Skyrim AE I would gladly do so! Where can I find instructions on how to use your testing branch?

Side note I noticed Skyrim AE was crashing after I reset the prefix with STL and it was because I had the Skyrim SE upgrade version. Fixed it by setting steam comparability to a proton version of gloriouseggroll then launching and letting Skyrim upgrade to anniversary edition. Not much you can do to fix that but might be helpful to know if someone complains about crashing after new pfx with AE upgrade version.

Thanks for all you all do. Love me some STL.

@sonic2kk
Copy link
Owner

sonic2kk commented Dec 1, 2022

Hehe, glad you like STL! Nice to see some positive comments 😄

Yes, the folder naming is certainly an issue and it's something I've kept in the back of my mind. Perhaps there is a way to force all folders recursively in a given mods directory to be lowercase, but I'm not sure if 1) this could cause problems with some mods, and 2) if there is a reasonable way to do this. However, it's 100% a valid thing to bring up, and at the very least it is something I will keep in mind to add to the wiki. It isn't an ideal manual step, but documenting it is better than not, right? :-)

Also I was thinking about setting up a new play through of FO4. If I can help with testing your new MO2 branch without messing with my current play through of Skyrim AE I would gladly do so! Where can I find instructions on how to use your testing branch?

Awesome, thank you! Testing the branch can be done in a few ways:

  • If you're using ProtonUp-Qt, you can install SteamTinkerLaunch-git and choose the mo2-overhaul branch in the version dropdown
  • If you're running from a package i.e. from the AUR, you'll need to uninstall that and then install SteamTinkerLaunch from source. It's as straightforward as cloning the repo to a folder of your choice with git clone https://github.com/sonic2kk/steamtinkerlaunch, running git checkout mo2-overhaul to change to the branch with the MO2 changes, and then running sudo make install to install SteamTinkerLaunch
    • Once you're finished testing, you can run sudo make uninstall to remove the manual installation, and go back to using your distributions package
    • If you're using Pacstall, you may have to keep the package installed, since Pacstall gives the updated Yad dependency that SteamTinkerLaunch needs to run (the version that comes with Ubuntu is too old iirc)
  • If you're already running a manual install, the instructions will be the same as above except you don't have to uninstall anything :-)

A heads up too: You may need jq for this branch to work -- various parts of STL that call out to web services have become reliant on jq to parse JSON, it may not be needed here but it's better to ensure you have it than not :-)

Thank you again for your willingness to test! Full disclosure, I have not had an opportunity to test this myself yet, but a couple of users have reported success (and no users that tested have complained yet more importantly 😉) so I am hoping for the best here too!

While I'm hoping to get the branch merged soon, it is probably still about 2 weeks away before I will even be fully ready to push the merge button heh. So take all the time you need with testing.

Thanks!

@har-nick
Copy link

har-nick commented Dec 4, 2022

@tetoNidan I don't believe SKSE is picky about the case sensitivity of its folders, but from what I hear Proton may be, which results in duplicate folders when different mods use different cases like Skse/SKSE/skse.

If you like living life on the edge, you can recursively rename your skse folders with something like this:

find /PATH/TO/MO2/MODS/FOLDER -maxdepth 2 -type d -iname skse -execdir echo mv {} skse ';'

It'll find all folders named SKSE without case-sensitivity, and rename them using mv.

-execdir runs the command in the containing directory of the found directory. -exec would run it from the MO2 mods folder causing mv to fail. Also, SKSE folders are top-level, so -maxdepth 2 stops it delving into mod folders.

(Note that's a dry-run. Remove echo from that command when you're sure.)

sonic2kk added a commit that referenced this issue Dec 9, 2022
* major mo2 overhaul - testing #598

* minor

* fixed creating list of downloadable wine packages

* case insensitive Steam Collection matching

* Added option to keep the game open after the game closed

* added some more logging for mo2

Co-authored-by: frostworx <frost.zeux@gmail.com>
@tetoNidan
Copy link

A heads up too: You may need jq for this branch to work -- various parts of STL that call out to web services have become reliant on jq to parse JSON, it may not be needed here but it's better to ensure you have it than not :-)

is JQ a jquery lib? id there something I can do for you all in python? Returning json i mean?

Also was thinking about your UI? Are you guys set on YAD? I could give a shot at building a GUI in wxPython. wxPython might be a little overkill but might be nice in case STL is ported to run on windows in the future. The reason I was thinking wxPython was the tabbed window interface for grouping relative settings apart from the whole? Might be nice but might not be something you want to undertake. Been looking for a new project to get me back in the coding game. The biggest problem I can see using wxPython would be compiling a single file executable to pull in all the WX libs every time STL updates. Might not be what you all want to be doing? Thought I'd ask though.

I will try my best to test the dev branch this weekend.

Thanks STL team!

@har-nick
Copy link

har-nick commented Dec 9, 2022

is JQ a jquery lib?

jq is a JSON processor. It's not especially big, but it's used I believe by Proton to format JSON responses when fetching updates.

This is mostly from memory and checking b731f7e, so excuse me if I'm mistaken.

Also was thinking about your UI? Are you guys set on YAD? I could give a shot at building a GUI in wxPython. wxPython might be a little overkill but might be nice in case STL is ported to run on windows in the future.

Not to diminish frostworx amazing efforts with this repo, but to sum it up, STL is basically a giant bash wrapper for various WINE functions.

Given that running Bash scripts on Windows would require WSL, and WINE and its functions aren't necessary on Windows, I'm not sure STL would benefit from a Windows release.

@sonic2kk
Copy link
Owner

sonic2kk commented Dec 9, 2022

Thanks @har-nick, your overview here even down to the specific commit hash is totally correct. jq despite its name is a JSON processor, and yes it's used for parsing Proton versions. It is somewhat unfortunate to mandate it as a dependency but to be totally honest, it's a small dependency and saves a bunch of headaches. It just makes things much more straightforward from a development and maintenance standpoint to use an actual JSON parser.

You're also entirely correct that SteamTinkerLaunch is a big Bash wrapper. This is why we are unfortunately stuck with Yad - SteamTinkerLaunch, to my understanding, started out a lot smaller than it is now and ballooned in scope. Now a full rewrite would be required given how much of the logic and Yad code are intertwined.

And once more, you're exactly correct on the Windows stance. SteamTinkerLaunch is just not needed on Windows. STL is meant as a wrapper to allow for configuration of Linux tools and Wine settings, so I also don't see the benefit of having SteamTinkerLaunch on Windows. If other users do see benefit and want to get STL working with WSL, by all means give it a shot! It can be good fun to just make something for the fun of it and I certainly would not want to take that away from anyone. SteamTinkerLaunch is open source under GPL-3.0, so long as you don't violate the terms of the license I also have no reason to stop anyone from doing whatever they want here - Open source rocks for a reason 😉


So, at the risk of repeating much of what @har-nick has already pointed out, to give a response more directly to you @tetoNidan:

id there something I can do for you all in python?

I'm a Python guy at heart and there was some brief discussion about pulling in Python as a dependency to run some scripts (and also to run a custom headless version of #467 in Python, before plans changed on how to implement this), but in the end it was decided to keep SteamTinkerLaunch as light as possible. So while I appreciate the offer here, it's all good :-)

Are you guys set on YAD? I could give a shot at building a GUI in wxPython

Like with everything you said here, I can't stress enough how much I truly appreciate the effort and initiative here. But as I and @har-nick went over a little above, we are sort of stuck with Yad. STL would need to be entirely rewritten, and honestly having worked with it for about 9 months in-depth, a lot of what SteamTinkerLaunch does is very much suited towards using Bash. Trying to retrofit a Python layer in that calls out to various Bash scripts would cause a lot of headaches and a full rewrite would be better in that case, but a full rewrite of SteamTinkerLaunch is not something I want to think about right now 😅 I am still new as maintainer and don't want to go stripping SteamTinkerLaunch apart just yet.

In future, and this is pie-in-the-sky future we're talking about, I would like to break SteamTinkerLaunch down into several smaller scripts and decouple it a little at a time; "breaking down the monolith" one might say. But ignoring the complexity of some of what SteamTinkerLaunch does, given the sheer size alone it would be a lot of work. I would honestly say if I was going to do it, I would budget over a year before it's fully done once I go down that road. And there are too many other features in the pipeline to think about doing it just yet.

The reason I was thinking wxPython was the tabbed window interface for grouping relative settings apart from the whole?

Yad actually supports this, but not on Wayland. SteamTinkerLaunch used to have tabs but they were removed because Yad does not have support for "notebooks" as they're called under Wayland. And trying to tell when a user is on Wayland vs X11 and changing how the UI is shown, would be a real pain to do. I'm also not sure if wxPython has native Wayland support, but that's a much more minor issue since it would just result in some blur on HiDPI displays. When trying to use notebooks with Yad on Wayland, they spawn as separate window processes and it's just a mess.

I am a Plasma Wayland user, originally by necessity to comfortably use fractional scaling and higher refresh rate displays, and now by choice for the noticeable performance gains in games. So Wayland support is held as a critical for SteamTinkerLaunch in-part selfishly because I use Wayland and I want SteamTinkerLaunch to work for me ;-)

Thought I'd ask though.

And just one more time, I'd like to say thank you for even thinking to offer. This kind of initiative is part of what keeps me motivated to work on SteamTinkerLaunch (even before I was maintainer!).

I will try my best to test the dev branch this weekend.

Actually, I had meant to post this earlier, but this branch is in master now as of a few hours ago! I bit the bullet and merged it. I had actually meant to leave a comment but my personal life caught up with me for a few hours and I didn't get a chance. I'm going to write a more formal message after this monster of a reply heh.


Thanks to both of you for the discussion here, it is much appreciated and something I genuinely love to see 👍

@sonic2kk
Copy link
Owner

sonic2kk commented Dec 9, 2022

With #674 merged, the mo2-overhaul branch is now in master (6a60f8c). I tested my existing New Vegas and Oblivion mod installs, and they worked. Enabling and disabling mods also worked, and installing a new mod worked too. With that and the positive feedback I had already gotten, I felt comfortable merging the changes.

However, please feel free to continue testing the implementation. Just now you can test it master :-) I will keep this issue open probably until SteamTinkerLaunch v12 releases, which should hopefully be very soon pending some further testing! I plan to open an issue to detail the final few steps needed before v12.

Thanks everyone :-)

@har-nick
Copy link

har-nick commented Dec 9, 2022

I did forget to mention, but I've also created a full SSE mod environment with xEdit, Nemesis, Synthesis, etc, and besides some minor graphical issues, it's all worked flawlessly.

I've one issue, but I'm unsure it's related to the overhaul branch specifically, so I'll make a new report.

I'm glad to see the overhaul branch has been pushed. A lot of work was put into it, and I'm sure a lot of users are grateful for it, so thanks!

@sonic2kk
Copy link
Owner

sonic2kk commented Dec 9, 2022

Wow, that is a ringing endorsement that it pretty much works as intended! Lots of work was indeed put into it, I helped out with a little bit of research around it and Frostworx did the actual legwork programming on it.

Small tip that may help with the graphical issues: using Winecfg you can set the Wine theme to (none) and this tends to help with issues like black boxes, though at the expense of GUI programs looking a bit Windows-2000-y. For MO2 itself you can set a custom theme from its settings to get around this, but other programs may not have such luxuries.

And thank you for opening an issue about the problem you were facing :-)

@tetoNidan
Copy link

tetoNidan commented Dec 9, 2022

sonic2kk, I totally understand. A re write isn't something you want to get into at the moment. I dont even use windows so there is no point in me heading a project like that, especially when I'm not familiar with the code base at all.

However if, when you do want to do a rewrite I can be pretty handy in python if you ever do need. My bash is rubbish though! I have done allot of work in python.subprocess and python.psutil. Written a few complex ffmpeg wrappers a few times for encoding multiple videos in different formats and resolutions for web development. with and without real time TTY output from the subprocess. I also have experience with Kivy but none with QT.

If or when the time comes, keep me in your wheelhouse, i'd like to give back.

EDIT: Forgot to mention I have experience with pytinstaller single file executable's compiling on windows, mac and linux compiling from a single source code.

Thanks again!

@frostworx
Copy link
Collaborator Author

Hi all,
great discussion. Good to see new people interested in helping in various ways 👍

If I knew initially that steamtinkerlaunch would become so huge, I would have started directly with writing it in c++ or python.
Writing it in bash to its current scope already was (almost) insane, and rewriting it now from scratch would be as well :)
Getting rid pf all yad/bash barriers the project has currently is very tempting of course, so whenever someone starts an attempt good luck and much fun!

Instead of contributing to steamtinkerlaunch directly, enhancing yad by contributing functions we require in steamtinkerlaunch might be a great idea. For example, a long time ago steamtinkerlaunch had tab menus, which helped a lot for a clear overview.
Unfortunately the implementation used in yad for tabs was (is?) not wayland compatible (required xembed) and therefore I rewrote the whole gui part.

For sure you will have a better readability when splitting steamtinkerlaunch into multiple parts, but you should seriously consider if it is worth the work and time it requires. It would not only require the time for rewriting this, but also for testing every single function and their interoperability and finally the split out parts need to stay available from the SLR sandbox, which might make maintaining the rewrite even harder in the future.

@sonic2kk
Copy link
Owner

but you should seriously consider if it is worth the work and time it requires. It would not only require the time for rewriting this, but also for testing every single function and their interoperability and finally the split out parts need to stay available from the SLR sandbox, which might make maintaining the rewrite even harder in the future.

This is absolutely one of the major concerns :-) And yet another reason why it hasn't even been fully planned. It's just one of those things that I keep in my mind but that I have very little motivation to plan out. Even Proton's proton script is just a behemoth Python script.

Something I would rather do instead is "document" more of the code that already exists. Some big parts of the codebase like MO2 and Vortex have comments around where they begin and end, and I did something similar for HMM support. I think I'll try to do more of this going forward around the code where possible :-)

@sonic2kk
Copy link
Owner

Thanks to every user who participated here in helping with the development of the MO2 overhaul. There is still an outstanding issue that I'm looking into (#682) and other users who have issues are encouraged to file separate bug reports. This issue should no longer be required, so I will be closing it and unpinning it :-)

Thanks to all once again 🥳 Happy tinkering!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request ModOrganizer 2 Issues related to installing or using ModOrganizer 2 with SteamTinkerLaunch
Projects
None yet
Development

No branches or pull requests

7 participants