[FIX-Linux] Snaps Issues Part 1 #3128

flavioislima · 2023-10-11T08:27:51Z

Fixes:

Legendary access permissions.
Get rid of curl dependency ([Tech] Refactor Download Methods #3129)
Get rid of tar dependency ([Tech] Refactor Extraction methods #3181) since Snaps does not extract tar.xz files by default.
Text not showing correctly on native dialogs (showing squares instead). Need to run on the terminal: rm ~/.cache/fontconfig/* ~/snap/heroic/common/.cache/fontconfig/*

Current Discussions:
https://forum.snapcraft.io/t/help-fixing-heroic-games-launcher-snap/37248/5
#3127

For Part 2

Access to Steam and Lutris folders to look for Proton and Wine.
Gamemode
Mangohud
Gamescope

Use the following Checklist if you have changed something on the Backend or Frontend:

Tested the feature and it's working on a current and clean install.
Tested the main App features and they are still working on a current and clean install. (Login, Install, Play, Uninstall, Move games, etc.)
Created / Updated Tests (If necessary)
Created / Updated documentation (If necessary)

soumyaDghosh · 2023-10-11T08:37:04Z

Hey, these changes probably are not needed. And as said before in the forum, please don't use electron-builder. It'll be worse to maintain later. Please.

flavioislima · 2023-10-11T08:43:22Z

Hey, these changes probably are not needed. And as said before in the forum, please don't use electron-builder. It'll be worse to maintain later. Please.

It is actually worse for us to maintain a separate thing, if we can do it using electron-builder would be easier since we don't need to change our workflows.

Because otherwise we will need a specific workflow just for snaps and find a way to automate the update and release of it on every release as well.

So unless it is impossible to solve with it then we would try a different way.
I think to fix the other issues now I just need to make the layouts work, right? so I can symlink the binaries on the system to snap?

soumyaDghosh · 2023-10-11T08:53:08Z

name: heroic
base: core22 
version: '2.9.2' 
summary: Single-line elevator pitch for your amazing snap 
description: |
  This is my-snap's description. You have a paragraph or two to tell the
  most important story about your snap. Keep it under 100 words though,
  we live in tweetspace and your description wants to look good in the snap
  store.

grade: devel
confinement: devmode 
compression: lzo

parts:
  heroic:
    plugin: dump
    source: https://github.com/Heroic-Games-Launcher/HeroicGamesLauncher/releases/download/v$SNAPCRAFT_PROJECT_VERSION/heroic_$SNAPCRAFT_PROJECT_VERSION_$CRAFT_TARGET_ARCH.deb
  
  deps:
    plugin: nil
    stage-packages:
      - curl
    prime:
      - usr/bin/curl

plugs:
  dot-local-share:
    interface: personal-files
    write:
      - $HOME/.local/share/Steam
      - $HOME/.local/share/lutris
      - 
apps:
  heroic:
    command: opt/Heroic/heroic
    desktop: usr/share/applications/heroic.desktop
    environment:
      TMPDIR: $XDG_RUNTIME_DIR
    extensions:
      - gnome

Bringing up my latest manifest here again to compare. Heroic is a game launcher, a very sensitive app, that needs to be fast and efficient. Now, here with this manifest, curl is working prefectly. I can download wine and proton. Now here, check that I only allowed the curl binary to stage and no other file, this way I can keep the thing in check. Now, regarding the workflow, what I am thinking to build is

New Tag made, Deb package made
After that the github workflow will chaneg the Line 3, version: 2.9.2 to the latest tag
The snap package will be made by the snapcraft builder, released to snapcraft directly from the github workflow

Another alternative way, not using github workflow to build the snap

After 2:
This repository will be connected with snapcraft website
It'll auto build from the latest changed manifest, when the manifest changes, if not, just click the Trigger New Build button and it'll build and release to the edge channel.

EDIT: if the team agrees, I can completely maintain the snap thing, on my own.

flavioislima · 2023-10-11T09:23:24Z

@soumyaDghosh yes I think electron-builder might now allow those options and we will need to go with our own snapcraft file then.

The way that you described could work, but then we trigger the workflow on published like we are doing for the flatpak and the AUR which are totally automated now as well. The snap should be something similar to what we do on the AUR, maybe more simpler since we don't need the even check the md5sum of the package for instance.

About maintaining the Snap, I am fine with it, no problem. But even that might not be needed if we can automate it.

soumyaDghosh · 2023-10-11T12:28:35Z

Forgot to mention, Electron Builder uses core20 where as core22 is the current ongoing base and soon core24 will be there based on Ubuntu 24.04 LTS. Latest bases are necessary for latest software drivers etc. Also, core22 brings significant performance boost for Wayland.

flavioislima · 2023-10-11T12:36:25Z

Forgot to mention, Electron Builder uses core20 where as core22 is the current ongoing base and soon core24 will be there based on Ubuntu 24.04 LTS. Latest bases are necessary for latest software drivers etc. Also, core22 brings significant performance boost for Wayland.

Updated electron-builder to the latest version and using core22 now works fine.
Testing here your snapcraft.yml as well.

soumyaDghosh · 2023-10-11T12:38:05Z

My snap manifest isn't complete yet. I most probably hae one with most of the things working. I will share it soon

arielj · 2023-10-11T13:01:10Z

question about the curl access, do we need curl? what I mean is... we have node, it should be able to do anything we are using curl for without curl

maybe it's easier to remove the curl dependency than to fix the access to curl?

soumyaDghosh · 2023-10-11T13:14:35Z

Forgot to mention, Electron Builder uses core20 where as core22 is the current ongoing base and soon core24 will be there based on Ubuntu 24.04 LTS. Latest bases are necessary for latest software drivers etc. Also, core22 brings significant performance boost for Wayland.

Updated electron-builder to the latest version and using core22 now works fine. Testing here your snapcraft.yml as well.

Kindly give me the snap link. Would like to test the snap.

ashuntu · 2023-10-11T14:11:21Z

@flavioislima Is there a reason you are not staging curl? I.e. stagePackages: ["default", "curl"]

flavioislima · 2023-10-11T14:37:51Z

@flavioislima Is there a reason you are not staging curl? I.e. stagePackages: ["default", "curl"]

I tried but it didn't work. I will try again but on Ubuntu.

flavioislima · 2023-10-11T18:38:00Z

question about the curl access, do we need curl? what I mean is... we have node, it should be able to do anything we are using curl for without curl

maybe it's easier to remove the curl dependency than to fix the access to curl?

For sure. We can use EasyDL lib to download all packages, would be a medium size pr to refactor everything but it's possible.

But we still need to figure it out the rest of the plug-ins like mangohud, etc.

flavioislima · 2023-10-11T18:40:44Z

I will modify this PR so it will generate a snap to download so we can test it.

Etaash-mathamsetty · 2023-10-11T23:00:46Z

src/backend/storeManagers/legendary/library.ts

+
+  // if not on a SNAP environment, set the XDG_CONFIG_HOME to the same location as the config file
+  if (!process.env.SNAP) {
+    options.env.XDG_CONFIG_HOME = dirname(legendaryConfigPath)


does changing XDG_CONFIG_HOME under snap cause issues? if it does does it still cause issues if a different variable is used (like LEGENDARY_CONFIG_PATH)?

it might be better to update legendary and use LEGENDARY_CONFIG_PATH once my PR gets merged

Yes, it breaks the SNAPs because then legendary was trying to write its configs to a place that it is not possible so it was giving permission issues.

imLinguin · 2023-10-12T04:25:37Z

For sure. We can use EasyDL lib to download all packages, would be a medium size pr to refactor everything but it's possible.

Don't we have axios that's capable of that already?

soumyaDghosh · 2023-10-12T06:15:35Z

@flavioislima Is there a reason you are not staging curl? I.e. stagePackages: ["default", "curl"]

All these uncertainties are the reason, I am still on the side of a custom manifest.

flavioislima · 2023-10-12T12:58:52Z

For sure. We can use EasyDL lib to download all packages, would be a medium size pr to refactor everything but it's possible.

Don't we have axios that's capable of that already?

Yes but Axios was not intended to download files, it is possible but it is not stable. EasyDL is a pretty good library that does that and supports progress, split files into chunks and continuing to download in case something fails, different from axios that does not have any of these features.

flavioislima · 2023-10-12T14:18:00Z

@flavioislima Is there a reason you are not staging curl? I.e. stagePackages: ["default", "curl"]

All these uncertainties are the reason, I am still on the side of a custom manifest.

Tried here on Ubuntu and on Arch (with multipass). The curl package is successfully installed but Heroic still cannot access it.
When I try adding gamescope, mangohud or gamemode to the list of stage packages, Snapcraft fails saying that the packages are not available.
So I am starting to think that the only way would be for Heroic to use Classic confinement but it does not meet the requirements for the Snap Store team to approve it.

This leads me to two choices:

Refactor the download part of Heroic that uses Curl so it is not required anymore and show a message to the users saying that some features might not work or are not available on Snaps and recommend it to use a different package if they need that (for Mangohud, Gamemode, and Gamescope), those are not really a HUGE deal to most people so I believe that is fine (perhaps Mangohud only).
Drop the Snap support for good, which I think it is not the best decision since it is the default install method on Ubuntu and Derivates.

For 1, I still need approval from the Snap Store to be able to access those home folders we need. I just tried to publish the package on Edge channel and they denied it because of these new accesses.

ashuntu · 2023-10-12T14:30:12Z

I still would be on the side of a custom snapcraft yaml, as I think you'd have trouble getting GameMode and MangoHUD to work only by adding their packages via electron-builder. Maybe in the meantime, a message to users about functionality wouldn't be a bad idea, though.

We include GameMode and MangoHUD as separate parts in the Steam Snap if you're looking for inspiration there (https://github.com/canonical/steam-snap/blob/main/snap/snapcraft.yaml#L250-L310). Steam is a finicky case since we force the client to run in 32-bit, so functionality isn't very consistent. I'd be curious to see how well something similar would work for Heroic.

flavioislima · 2023-10-12T14:32:16Z

I still would be on the side of a custom snapcraft yaml, as I think you'd have trouble getting GameMode and MangoHUD to work only by adding their packages via electron-builder. Maybe in the meantime, a message to users about functionality wouldn't be a bad idea, though.

We include GameMode and MangoHUD as separate parts in the Steam Snap if you're looking for inspiration there (https://github.com/canonical/steam-snap/blob/main/snap/snapcraft.yaml#L250-L310). Steam is a finicky case since we force the client to run in 32-bit, so functionality isn't very consistent. I'd be curious to see how well something similar would work for Heroic.

Ok, so for now I will remove the need of curl from the code and fix these other issues and in the future we try to bring these other tools to the snap package.

flavioislima · 2023-10-12T15:33:09Z

I am removing the curl dependency here on this PR:
#3129

soumyaDghosh · 2023-10-12T17:51:35Z

I still would be on the side of a custom snapcraft yaml, as I think you'd have trouble getting GameMode and MangoHUD to work only by adding their packages via electron-builder. Maybe in the meantime, a message to users about functionality wouldn't be a bad idea, though.

We include GameMode and MangoHUD as separate parts in the Steam Snap if you're looking for inspiration there (https://github.com/canonical/steam-snap/blob/main/snap/snapcraft.yaml#L250-L310). Steam is a finicky case since we force the client to run in 32-bit, so functionality isn't very consistent. I'd be curious to see how well something similar would work for Heroic.

@flavioislima So, should I share a PR on the snap workflow?

flavioislima · 2023-10-12T18:10:45Z

I still would be on the side of a custom snapcraft yaml, as I think you'd have trouble getting GameMode and MangoHUD to work only by adding their packages via electron-builder. Maybe in the meantime, a message to users about functionality wouldn't be a bad idea, though.
We include GameMode and MangoHUD as separate parts in the Steam Snap if you're looking for inspiration there (https://github.com/canonical/steam-snap/blob/main/snap/snapcraft.yaml#L250-L310). Steam is a finicky case since we force the client to run in 32-bit, so functionality isn't very consistent. I'd be curious to see how well something similar would work for Heroic.

@flavioislima So, should I share a PR on the snap workflow?

sure

soumyaDghosh · 2023-10-13T14:20:09Z

I still would be on the side of a custom snapcraft yaml, as I think you'd have trouble getting GameMode and MangoHUD to work only by adding their packages via electron-builder. Maybe in the meantime, a message to users about functionality wouldn't be a bad idea, though.
We include GameMode and MangoHUD as separate parts in the Steam Snap if you're looking for inspiration there (https://github.com/canonical/steam-snap/blob/main/snap/snapcraft.yaml#L250-L310). Steam is a finicky case since we force the client to run in 32-bit, so functionality isn't very consistent. I'd be curious to see how well something similar would work for Heroic.

@flavioislima So, should I share a PR on the snap workflow?

sure

Problem, I want to test this pr, but where can I get the deb file from?

flavioislima · 2023-10-26T09:34:39Z

I still would be on the side of a custom snapcraft yaml, as I think you'd have trouble getting GameMode and MangoHUD to work only by adding their packages via electron-builder. Maybe in the meantime, a message to users about functionality wouldn't be a bad idea, though.
We include GameMode and MangoHUD as separate parts in the Steam Snap if you're looking for inspiration there (https://github.com/canonical/steam-snap/blob/main/snap/snapcraft.yaml#L250-L310). Steam is a finicky case since we force the client to run in 32-bit, so functionality isn't very consistent. I'd be curious to see how well something similar would work for Heroic.

@flavioislima So, should I share a PR on the snap workflow?

sure

Problem, I want to test this pr, but where can I get the deb file from?

Sorry, missed your reply. I will change this PR so it will generate a .deb during build

…uncher into fix/snaps

flavioislima · 2023-10-30T11:53:36Z

Added message with warning of missing features:

This reverts commit 6ae314b.

…uncher into fix/snaps

soumyaDghosh · 2023-10-31T00:21:16Z

@flavioislima sorry to inform you that heroic being a big project needs a lot of attention and problem is my sem exam is coming soon. So, I'm halting on this thing now. I think if you don't mind, we can restart again after December!

flavioislima · 2023-10-31T19:26:24Z

@flavioislima sorry to inform you that heroic being a big project needs a lot of attention and problem is my sem exam is coming soon. So, I'm halting on this thing now. I think if you don't mind, we can restart again after December!

No problem. This is also getting more complicated. The app cannot even extract a tar.xz file since the core app is so limited on Snap. So I also need to change the extraction methods to use a npm package instead of native tar command.

soumyaDghosh · 2023-10-31T19:28:17Z

@flavioislima sorry to inform you that heroic being a big project needs a lot of attention and problem is my sem exam is coming soon. So, I'm halting on this thing now. I think if you don't mind, we can restart again after December!

No problem. This is also getting more complicated. The app cannot even extract a tar.xz file since the core app is so limited on Snap. So I also need to change the extraction methods to use a npm package instead of native tar command.

Those are possible, and can be done. What I suggest now is to hold any code changes just for the sake of snap. It'll later make a hell for the package.

flavioislima · 2023-10-31T23:03:22Z

Those are possible, and can be done. What I suggest now is to hold any code changes just for the sake of snap. It'll later make a hell for the package.

Yeah, but at the same time having the extraction, download, etc. Done from Heroic itself will be better since we won't rely on external dependencies and packages. So in the end these are necessary refactors.

flavioislima · 2023-11-03T11:22:20Z

Merging this one. Should at least fix part of the issues.
Will continue with more fixes afterwards. Perhaps only by using a snapcraft.yml build file we can have a 100% fully working app.

flavioislima added 2 commits October 11, 2023 09:24

fix: home and config variables

e3c060f

chore: update snap config and electron builder

e2fde38

flavioislima added pr:wip WIP, don't merge. pr:testing This PR is in testing, don't merge. labels Oct 11, 2023

chore: store games and prefixes on real home

1e50a96

Etaash-mathamsetty reviewed Oct 11, 2023

View reviewed changes

flavioislima added 4 commits October 12, 2023 14:52

ci: build and upload snap package on PR

d933074

chore: try build again

fff3d20

fix: not found packages

7c55bf5

fix: build locally

a158dde

flavioislima mentioned this pull request Oct 26, 2023

uncaught exeption error #3065

Closed

flavioislima added 3 commits October 26, 2023 10:36

chore: generate .deb file for testing

6ae314b

Merge branch 'main' of github.com:Heroic-Games-Launcher/HeroicGamesLa…

b2e8fa9

…uncher into fix/snaps

ux: show warning message of missing features

4749599

flavioislima added 2 commits October 30, 2023 13:17

Revert "chore: generate .deb file for testing"

055e38e

This reverts commit 6ae314b.

Merge branch 'main' of github.com:Heroic-Games-Launcher/HeroicGamesLa…

f701116

…uncher into fix/snaps

flavioislima changed the title ~~[FIX-Linux] Snaps Issues~~ [FIX-Linux] Snaps Issues Part 1 Nov 3, 2023

flavioislima merged commit ada9cdb into main Nov 3, 2023
13 checks passed

flavioislima deleted the fix/snaps branch November 3, 2023 11:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FIX-Linux] Snaps Issues Part 1 #3128

[FIX-Linux] Snaps Issues Part 1 #3128

flavioislima commented Oct 11, 2023 •

edited

soumyaDghosh commented Oct 11, 2023

flavioislima commented Oct 11, 2023

soumyaDghosh commented Oct 11, 2023 •

edited

flavioislima commented Oct 11, 2023

soumyaDghosh commented Oct 11, 2023

flavioislima commented Oct 11, 2023

soumyaDghosh commented Oct 11, 2023

arielj commented Oct 11, 2023

soumyaDghosh commented Oct 11, 2023

ashuntu commented Oct 11, 2023 •

edited

flavioislima commented Oct 11, 2023

flavioislima commented Oct 11, 2023

flavioislima commented Oct 11, 2023

Etaash-mathamsetty Oct 11, 2023 •

edited

flavioislima Oct 12, 2023

imLinguin commented Oct 12, 2023

soumyaDghosh commented Oct 12, 2023

flavioislima commented Oct 12, 2023

flavioislima commented Oct 12, 2023

ashuntu commented Oct 12, 2023

flavioislima commented Oct 12, 2023

flavioislima commented Oct 12, 2023

soumyaDghosh commented Oct 12, 2023

flavioislima commented Oct 12, 2023

soumyaDghosh commented Oct 13, 2023

flavioislima commented Oct 26, 2023

flavioislima commented Oct 30, 2023

soumyaDghosh commented Oct 31, 2023

flavioislima commented Oct 31, 2023

soumyaDghosh commented Oct 31, 2023

flavioislima commented Oct 31, 2023

flavioislima commented Nov 3, 2023

[FIX-Linux] Snaps Issues Part 1 #3128

[FIX-Linux] Snaps Issues Part 1 #3128

Conversation

flavioislima commented Oct 11, 2023 • edited

For Part 2

soumyaDghosh commented Oct 11, 2023

flavioislima commented Oct 11, 2023

soumyaDghosh commented Oct 11, 2023 • edited

flavioislima commented Oct 11, 2023

soumyaDghosh commented Oct 11, 2023

flavioislima commented Oct 11, 2023

soumyaDghosh commented Oct 11, 2023

arielj commented Oct 11, 2023

soumyaDghosh commented Oct 11, 2023

ashuntu commented Oct 11, 2023 • edited

flavioislima commented Oct 11, 2023

flavioislima commented Oct 11, 2023

flavioislima commented Oct 11, 2023

Etaash-mathamsetty Oct 11, 2023 • edited

Choose a reason for hiding this comment

flavioislima Oct 12, 2023

Choose a reason for hiding this comment

imLinguin commented Oct 12, 2023

soumyaDghosh commented Oct 12, 2023

flavioislima commented Oct 12, 2023

flavioislima commented Oct 12, 2023

ashuntu commented Oct 12, 2023

flavioislima commented Oct 12, 2023

flavioislima commented Oct 12, 2023

soumyaDghosh commented Oct 12, 2023

flavioislima commented Oct 12, 2023

soumyaDghosh commented Oct 13, 2023

flavioislima commented Oct 26, 2023

flavioislima commented Oct 30, 2023

soumyaDghosh commented Oct 31, 2023

flavioislima commented Oct 31, 2023

soumyaDghosh commented Oct 31, 2023

flavioislima commented Oct 31, 2023

flavioislima commented Nov 3, 2023

flavioislima commented Oct 11, 2023 •

edited

soumyaDghosh commented Oct 11, 2023 •

edited

ashuntu commented Oct 11, 2023 •

edited

Etaash-mathamsetty Oct 11, 2023 •

edited