Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rimsort: init at 1.0.6.2-hf-unstable-2024-04-26 #304943

Closed
wants to merge 7 commits into from

Conversation

vinnymeller
Copy link
Contributor

@vinnymeller vinnymeller commented Apr 18, 2024

Description of changes

Closes #304832

Adds RimSort

Seemingly everything works now

image

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

Copy link
Member

@RossComputerGuy RossComputerGuy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commit history looks good, so far everything looks good for the most part aside from a few nitpicks. You probably should update your top comment on this PR to say Closes #304943 so GitHub links this PR to the related issue.

}:
python3Packages.buildPythonApplication rec {
pname = "rimsort";
version = "unstable-2024-04-15";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably should prefix the version with 0-.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RimSort has tags in a suitable format, so 0 is wrong. 1.0.6.2-hf-unstable-<et cetera> should be used. Checked by letting unstableGitUpdater do a bump via #280501.

image

xmltodict
];

propagatedBuildInputs = [] ++ python-deps;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably don't need the empty list.

Comment on lines 77 to 102
patches = [
./remove-name-main-conditions.patch
./use-writeable-data-folder.patch
];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is common to have patches towards the top, below the src.

}:
buildPythonPackage rec {
pname = "steamfiles";
version = "unstable-2020-10-26";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably should also prefix the version here with 0-.

}:
buildPythonPackage rec {
pname = "steamworks";
version = "unstable-2023-11-25";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here with the prefixing of the version.

}:
buildPythonPackage rec {
pname = "win-inet-pton";
version = "unstable-2019-02-19";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here with the version prefixing.

@vinnymeller vinnymeller force-pushed the init-rimsort branch 3 times, most recently from 17558ea to 15451c7 Compare April 18, 2024 06:19
@vinnymeller
Copy link
Contributor Author

From the current state:

Result of nixpkgs-review pr 304943 run on x86_64-linux 1

22 packages built:
  • python311Packages.gevent-eventemitter
  • python311Packages.gevent-eventemitter.dist
  • python311Packages.steam
  • python311Packages.steam.dist
  • python311Packages.steamfiles
  • python311Packages.steamfiles.dist
  • python311Packages.steamworks
  • python311Packages.steamworks.dist
  • python311Packages.win-inet-pton
  • python311Packages.win-inet-pton.dist
  • python312Packages.gevent-eventemitter
  • python312Packages.gevent-eventemitter.dist
  • python312Packages.steam
  • python312Packages.steam.dist
  • python312Packages.steamfiles
  • python312Packages.steamfiles.dist
  • python312Packages.steamworks
  • python312Packages.steamworks.dist
  • python312Packages.win-inet-pton
  • python312Packages.win-inet-pton.dist
  • rimsort
  • rimsort.dist

@vinnymeller vinnymeller marked this pull request as ready for review April 18, 2024 06:41
@vinnymeller vinnymeller changed the title rimsort: init at unstable-2024-04-15 rimsort: init at 0-unstable-2024-04-15 Apr 18, 2024
@nyabinary
Copy link
Contributor

  File "/nix/store/v30fyz12420nz4ll1jj153dvjma6g9lg-rimsort-0-unstable-2024-04-15/lib/python3.11/site-packages/app/views/main_content_panel.py", line 2099, in _do_steamworks_api_call_animated
    publishedfileids = check_if_pfids_blacklisted(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/v30fyz12420nz4ll1jj153dvjma6g9lg-rimsort-0-unstable-2024-04-15/lib/python3.11/site-packages/app/utils/metadata.py", line 2183, in check_if_pfids_blacklisted
    if steamdb.get(publishedfileid, {}).get("blacklist"):
       ^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'

Trying to run and download a workshop mod from steam through the app causes this (using the Steam app and not SteamCMD)
Clicking Setting up SteamCMD causes a different error:

Traceback (most recent call last):
  File "/nix/store/v30fyz12420nz4ll1jj153dvjma6g9lg-rimsort-0-unstable-2024-04-15/lib/python3.11/site-packages/app/views/main_content_panel.py", line 623, in actions_slot
    self._do_setup_steamcmd()
  File "/nix/store/v30fyz12420nz4ll1jj153dvjma6g9lg-rimsort-0-unstable-2024-04-15/lib/python3.11/site-packages/app/views/main_content_panel.py", line 1912, in _do_setup_steamcmd
    self.steamcmd_wrapper.setup_steamcmd(
  File "/nix/store/v30fyz12420nz4ll1jj153dvjma6g9lg-rimsort-0-unstable-2024-04-15/lib/python3.11/site-packages/app/utils/steam/steamcmd/wrapper.py", line 212, in setup_steamcmd
    os.symlink(
FileNotFoundError: [Errno 2] No such file or directory: '' -> '/home/binary/.local/share/RimSort/steam/steamapps/workshop/content/294100'

@twstagg
Copy link

twstagg commented Apr 18, 2024

  File "/nix/store/v30fyz12420nz4ll1jj153dvjma6g9lg-rimsort-0-unstable-2024-04-15/lib/python3.11/site-packages/app/views/main_content_panel.py", line 2099, in _do_steamworks_api_call_animated
    publishedfileids = check_if_pfids_blacklisted(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/v30fyz12420nz4ll1jj153dvjma6g9lg-rimsort-0-unstable-2024-04-15/lib/python3.11/site-packages/app/utils/metadata.py", line 2183, in check_if_pfids_blacklisted
    if steamdb.get(publishedfileid, {}).get("blacklist"):
       ^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'

Trying to run and download a workshop mod from steam through the app causes this (using the Steam app and not SteamCMD) Clicking Setting up SteamCMD causes a different error:

Traceback (most recent call last):
  File "/nix/store/v30fyz12420nz4ll1jj153dvjma6g9lg-rimsort-0-unstable-2024-04-15/lib/python3.11/site-packages/app/views/main_content_panel.py", line 623, in actions_slot
    self._do_setup_steamcmd()
  File "/nix/store/v30fyz12420nz4ll1jj153dvjma6g9lg-rimsort-0-unstable-2024-04-15/lib/python3.11/site-packages/app/views/main_content_panel.py", line 1912, in _do_setup_steamcmd
    self.steamcmd_wrapper.setup_steamcmd(
  File "/nix/store/v30fyz12420nz4ll1jj153dvjma6g9lg-rimsort-0-unstable-2024-04-15/lib/python3.11/site-packages/app/utils/steam/steamcmd/wrapper.py", line 212, in setup_steamcmd
    os.symlink(
FileNotFoundError: [Errno 2] No such file or directory: '' -> '/home/binary/.local/share/RimSort/steam/steamapps/workshop/content/294100'

Hi @nyabinary , RimSort dev here

Could you please file an Issue on the RimSort repo and I will look into fixing this evening?

@nyabinary
Copy link
Contributor

image
Rimsort doesn't detect git on NixOS.

@nyabinary
Copy link
Contributor

nyabinary commented Apr 20, 2024

image Rimsort doesn't detect git on NixOS.

Downloaded the db and rules manually and tried to download a workshop mod, and it's just infinitely loading (this is via the Steam App)
In the console this popped up though

[S_API FAIL] SteamAPI_Init() failed; connect to global user failed.[S_API FAIL] Tried to access Steam interface SteamUser021 before SteamAPI_Init succeeded.
Received signal 11 SEGV_MAPERR 000000000000
#0 0x7f2acb284ce5 base::debug::CollectStackTrace()
#1 0x7f2acb26cf26 base::debug::StackTrace::StackTrace()
#2 0x7f2acb285169 base::debug::(anonymous namespace)::StackDumpSignalHandler()
#3 0x7f2b6056ff30 (/nix/store/ddwyrxif62r8n6xclvskjyy6szdhvj60-glibc-2.39-5/lib/libc.so.6+0x3ff2f)
#4 0x7f2aaa08de16 <unknown>
#5 0x7f2b5d699052 ffi_call_unix64
#6 0x7f2b5d696ee5 ffi_call_int
#7 0x7f2b5d697ad8 ffi_call
#8 0x7f2b599e31cc _ctypes_callproc
#9 0x7f2b599e53c1 PyCFuncPtr_call
#10 0x7f2b60a016bd _PyObject_MakeTpCall.localalias
#11 0x7f2b608f7c4c _PyEval_EvalFrameDefault.localalias
#12 0x7f2b60b2c0ca _PyEval_Vector
#13 0x7f2b60a4af65 _PyObject_FastCallDictTstate.localalias
#14 0x7f2b60a4b72d slot_tp_init
#15 0x7f2b609af557 type_call
#16 0x7f2b60a016bd _PyObject_MakeTpCall.localalias
#17 0x7f2b608f7c4c _PyEval_EvalFrameDefault.localalias
#18 0x7f2b60b2c0ca _PyEval_Vector
#19 0x7f2b60a1bd0c map_next
#20 0x7f2b60a516c6 list_extend
#21 0x7f2b60a51a84 list_vectorcall
#22 0x7f2b60a5b0e3 PyObject_Vectorcall.localalias
#23 0x7f2b608f7c4c _PyEval_EvalFrameDefault.localalias
#24 0x7f2b60b2c0ca _PyEval_Vector
#25 0x7f2b608fa98c _PyEval_EvalFrameDefault.localalias
#26 0x7f2b60b2c0ca _PyEval_Vector
#27 0x7f2b608fa98c _PyEval_EvalFrameDefault.localalias
#28 0x7f2b60b2c0ca _PyEval_Vector
#29 0x7f2b60a4aeb6 _PyObject_FastCallDictTstate.localalias
#30 0x7f2b60a4b72d slot_tp_init
#31 0x7f2b609af557 type_call
#32 0x7f2b60a016bd _PyObject_MakeTpCall.localalias
#33 0x7f2b608f7c4c _PyEval_EvalFrameDefault.localalias
#34 0x7f2b60b2c0ca _PyEval_Vector
#35 0x7f2b60a4af65 _PyObject_FastCallDictTstate.localalias
#36 0x7f2b60a4b72d slot_tp_init
#37 0x7f2b609af557 type_call
#38 0x7f2b60a016bd _PyObject_MakeTpCall.localalias
#39 0x7f2b608f7c4c _PyEval_EvalFrameDefault.localalias
#40 0x7f2b60b2c0ca _PyEval_Vector
#41 0x7f2b60a4ee88 method_vectorcall
#42 0x7f2b60a34778 _PyVectorcall_Call
#43 0x7f2b60b197bd partial_call
#44 0x7f2b60a016bd _PyObject_MakeTpCall.localalias
#45 0x7f2b60a5b0e3 PyObject_Vectorcall.localalias
#46 0x7f2b608f7c4c _PyEval_EvalFrameDefault.localalias
#47 0x7f2b60b2c0ca _PyEval_Vector
#48 0x7f2b60a4ef24 method_vectorcall
#49 0x7f2b5eaad42f (/nix/store/ib1nrcn38z0p16ar3fx52ah8g68m725c-pyside6-6.7.0/lib/python3.11/site-packages/PySide6/QtCore.abi3.so+0x2ad42e)
#50 0x7f2b5e32cde1 QThreadPrivate::start()
#51 0x7f2b605be272 start_thread
#52 0x7f2b60639dcc __clone3
  r8: 0000000000000000  r9: 0000000000000001 r10: 0000000000000000 r11: 0000000000000293
 r12: 00007f2a773fe690 r13: 0000000000000000 r14: 00007f2a773fe690 r15: 00007f2a773fe600
  di: 00007f2aaa0ec760  si: 0000000000000000  bp: 00007f2a773fe480  bx: 00007f2a773fe600
  dx: 0000000000000000  ax: 0000000000000000  cx: 0000000000000000  sp: 00007f2a773fe470
  ip: 00007f2aaa08de16 efl: 0000000000010206 cgf: 002b000000000033 erf: 0000000000000004
 trp: 000000000000000e msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]

@vinnymeller vinnymeller force-pushed the init-rimsort branch 2 times, most recently from 66c5ac8 to 994bfb1 Compare April 21, 2024 03:03
@vinnymeller
Copy link
Contributor Author

@nyabinary thanks, that + your last comment in the rimsort repo are both things for me to look into

@vinnymeller vinnymeller force-pushed the init-rimsort branch 2 times, most recently from 8cfe341 to 05a4a30 Compare April 21, 2024 20:46
@vinnymeller
Copy link
Contributor Author

image Rimsort doesn't detect git on NixOS.

ok this part is fixed in my latest commit. didn't notice python deps changed, I was still using pygit2 instead of gitpython

@vinnymeller vinnymeller force-pushed the init-rimsort branch 3 times, most recently from 1bc2240 to 63cb445 Compare April 22, 2024 00:20
@vinnymeller
Copy link
Contributor Author

ok as hacky as some of it is i think everything is working for me now. will have to wait for twsta i guess to see if not running rimsort from steam is cause of some of the other issues

@nyabinary
Copy link
Contributor

@nyabinary I think the steamcmd downloads will want to put them in the Steam mods location in settings. it says it saved to your equivalent of the steam mods folder but obviously yeah isn't showing them.

if you go back to settings and autodetect will it detect the path? what if you hit refresh on the bottom right?

current commit fixes the window crashing on keypress thing

unless somebody else objects id want to just get this in & working at the current rimsort version, and then can upstream anything that makes sense to upstream, and then remove my patches after that

image
Yeah, it seems like it wants to download to the rimsort location for some reason instead of the mod location I set it to.

@nyabinary
Copy link
Contributor

ok as hacky as some of it is i think everything is working for me now. will have to wait for twsta i guess to see if not running rimsort from steam is cause of some of the other issues

image
It needs to be symlinked as well, also I think making this into a programs.rimsort module makes sense since some users dont need SteamCMD and such.

@vinnymeller
Copy link
Contributor Author

vinnymeller commented Apr 22, 2024

since some users dont need SteamCMD and such.

not sure how I feel about making it a module primarily because even though some user's indeed won't need to use steamcmd, it doesn't align with how the real project is packaged - since on pretty much any non-nix distro everything builtin to rimsort to download steamcmd and such will just work, it'd be expected that it just works

im not entirely opposed to it though. i just dont like it. im trying to find any official guidance on when I might want to actually make something a module vs not.

a middle ground might be just making some features overrideable? e.g. useSteamCmd ? true in the inputs to just not include steamcmd as a dependency & to the rest.

i still don't love that specifically though because:

  • without my patch, rimsort will try to install steamcmd, which it just wont be able to run
  • with my patch but without steamcmd as a dep, it'll look for a file that doesn't exist. maybe that's better, since the user should know they explicitly turned off steamcmd in an override? idk

edit:
worded differently, my mental model has been that modules are more for when there will be some complex configuration or integration with other programs/services that we want to make simpler, and overrides for more program specific options

but in this case i still dont like the override because the design of rimsort sort of assumes everything either is there or can just be downloaded on the fly, i.e. id be happier with that option if rimsort itself knew that it shouldn't show steamcmd options or a download button for it

@nyabinary
Copy link
Contributor

since some users dont need SteamCMD and such.

not sure how I feel about making it a module primarily because even though some user's indeed won't need to use steamcmd, it doesn't align with how the real project is packaged - since on pretty much any non-nix distro everything builtin to rimsort to download steamcmd and such will just work, it'd be expected that it just works

im not entirely opposed to it though. i just dont like it. im trying to find any official guidance on when I might want to actually make something a module vs not.

a middle ground might be just making some features overrideable? e.g. useSteamCmd ? true in the inputs to just not include steamcmd as a dependency & to the rest.

i still don't love that specifically though because:

* without my patch, rimsort will try to install steamcmd, which it just wont be able to run

* with my patch but without steamcmd as a dep, it'll look for a file that doesn't exist. maybe that's better, since the user should know they explicitly turned off steamcmd in an override? idk

edit: worded differently, my mental model has been that modules are more for when there will be some complex configuration or integration with other programs/services that we want to make simpler, and overrides for more program specific options

but in this case i still dont like the override because the design of rimsort sort of assumes everything either is there or can just be downloaded on the fly, i.e. id be happier with that option if rimsort itself knew that it shouldn't show steamcmd options or a download button for it

You should join the Rimsort Discord since it would be easier to communicate there than clogging this github thread lol

@nyabinary
Copy link
Contributor

RimSort.log
Right-clicking any menu item (eg mods) in the main screen of Rimsorts crashes the app.

@vinnymeller vinnymeller force-pushed the init-rimsort branch 2 times, most recently from d993bff to 752bf4f Compare April 23, 2024 03:04
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/3824


nativeBuildInputs = [
setuptools
wheel
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wheel is probably not required here


nativeBuildInputs = [
setuptools
wheel
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
wheel

];

propagatedBuildInputs = [
coverage
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want to collect coverage as it slows down the tests and we have no use for it

coverage
gevent
pytest
pytest-cov
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pytest-cov


nativeBuildInputs = [
setuptools
wheel
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

protobuf
protobuf3-to-dict
];

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines 12 to 19
@@ -34,4 +34,4 @@ setup(
install_requires = [],
extras_require = {},
zip_safe = True
-)
\ No newline at end of file
+)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
@@ -34,4 +34,4 @@ setup(
install_requires = [],
extras_require = {},
zip_safe = True
-)
\ No newline at end of file
+)

repo = "todds";
rev = version;
hash = "sha256-nyYFYym9ZZskkaTPV30+QavdqpvVopnIXXZC6zkeu7c=";
fetchSubmodules = true;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use the vcpkg from nixpkgs instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

turns out it wasn't needed at all :p didn't think to check if the submodule was actually necessary for the build

};

patches = [
./make-nix-compatible.patch
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you try to upstream the generic fixes in there?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SuperSandro2000 this was my big question, don't know what fixes would be accepted or on what timeline etc.

i can try, i dont mind this pr sitting here people can still use it anyways, i just wasn't sure if it'd be preferable to get it working with ugly patches, get it merged, eventually get stuff upstreamed, then remove patches when the package gets updated, or to try to upstream 1st before ever merging

in either case i am going to try to upstream what i can

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any progress on that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nyabinary no. the rimsort guy didn't reply to my messages about this specifically and seemingly deleted another nix-related thread.

i redid my patches in a way that is much cleaner but it also turned out to be significantly more lines changed to make it pretty

i'm just going to close this pr for now because i realize im not interested in maintaining rimsort as it is now. i'll leave my branch alive on my fork in case either someone else wants to pick it up or if eventually the generic changes can be upstreamed

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello. I am the guy you speak of. I am not sure what you are referring to that i am supposed to have deleted or not responded to.

I am open to supporting NixOS upstream, but am only willing to support code that we work on upstream. I am always open to discussing further, however limited my direct involvement in implementing may be.

I am not really trying to provide support for these patches specifically, and i think that some of what i have looked at warrants discussion. Some of those comments and observations i initially had probably got lost on the discord.

@vinnymeller vinnymeller changed the title rimsort: init at 0-unstable-2024-04-20 rimsort: init at 1.0.6.2-hf-unstable-2024-04-20 Apr 26, 2024
@vinnymeller vinnymeller changed the title rimsort: init at 1.0.6.2-hf-unstable-2024-04-20 rimsort: init at 1.0.6.2-hf-unstable-2024-04-26 Apr 27, 2024
@SigmaSquadron
Copy link
Contributor

Hey @vinnymeller, are you still interested in maintaining RimSort?

@nyabinary
Copy link
Contributor

Hey @vinnymeller, are you still interested in maintaining RimSort?

Prob not so if you want to take over that would be nice :P

@SigmaSquadron
Copy link
Contributor

SigmaSquadron commented Sep 19, 2024 via email

@vinnymeller
Copy link
Contributor Author

@SigmaSquadron sorry I was on vacation when you first tagged me and I guess I missed it

I definitely don't want to maintain this as overall it's just not super... nix friendly?

You're definitely right though before I gave up I was in the middle of debugging that issue but sadly that branch was lost between then and now.

If you try to fix it, good luck 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: python 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 11.by: package-maintainer This PR was created by the maintainer of the package it changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Package request: RimSort
8 participants