Chromely cannot be downloaded on linux-arm (Raspberry Pi)

[redplane]

Hi,

I'm using chromely for my application, the below are my system information:

• Raspberry Pi 4

• Raspbian.

• Net 5

• Chromely 5.1.86

• I have built application with linux-arm run time.

Everything seems to be fine, but when I start the application, browser was opened with message:

CEF binaries download completed with error.
Error message: The remote server returned an error: (404) Not Found.
Please close.

I have tried the url you mentioned here, the url can be downloaded on my Raspberry.

What's wrong am I doing ?

[mattkol]

@redplane
The url used for the download should be logged. Can you compare that with what works and you can share it too.

[redplane]

Hi @mattkol ,

I have cloned your repository to my computer and start debugging the application.
I have notice this line , the downloadUrl value is: https://cef-builds.spotifycdn.com/cef_binary_86.0.21%2Bg6a2c8e7%2Bchromium-86.0.4240.183_linuxarm_minimal.tar.bz2.

When I paste the above line into my browser address, the server returns 404.
Then I went to https://cef-builds.spotifycdn.com/ to find the version 86.0.4240.183_linuxarm_minimal.tar.bz2, nothing could I find.

My guess is: the version already deprecated and the developers already removed and replace version 86 with version 87.

I also found the file version.g.cs. It seems that the download link is being generated by reading CefVersion from ChromelyRunTime.

I have few options below:

1. Hard coding the version is fine, but the library should provide a class or interceptor to allow developer to customize their download source (in case Cef cdn server is down or the version is removed from server). For now, what I see is the Download url is hard coded here

2. When I took a look at line 165 - CefLoader.cs, I saw you were downloading the whole CEF spotify cdn page and try finding the version the application needs to download & the download link.

I put a breakpoint there and read the value, the value Chromely downloaded was a raw page, the content seems to be rendered by the below function (which means we can't find any version in the downloaded string - it was client side rendering)

function createBuildTable(tab, build, tag=null)

Actually, there is a json file which lists every available CEF version, and it is: https://cef-builds.spotifycdn.com/index.json (by turning on developer console, we can see this)

I think we can use https://cef-builds.spotifycdn.com/index.json (by turning on developer console, we can see this) to get the available version instead of analyzing Cef spotify web content.

3. For now, because of my project is a little bit rush, I have to fix the Cef download process on raspberry.
   If I have a PR, which branch should I create PR to ?

[mattkol]

@redplane

I have cloned your repository to my computer
I saw you were downloading

This is an open source project. Everyone is encouraged to contribute.

[mattkol]

I think we can use https://cef-builds.spotifycdn.com/index.json (by turning on developer console, we can see this) to get the available version instead of analyzing Cef spotify web content.

I am not sure if the proposal will help. CefGlue specifies version.

Chromely/src/Chromely/CefGlue/Interop/version.g.cs

Line 12 in 3622bc4

public const string CEF_VERSION = "86.0.21+g6a2c8e7+chromium-86.0.4240.183";

So it looks like linux-arm build for v86 is deprecated or removed. So trying to change anything in the code may not help much. What is needed is to upgrade CefGlue. But if you can make it work for the project you are working on, I think you should go ahead. It can help others too. When CefGlue is updated, I believe the error you are seeing will go away.

Alternatively you can also upgrade to CefGlue, v87.

[redplane]

I think we can use https://cef-builds.spotifycdn.com/index.json (by turning on developer console, we can see this) to get the available version instead of analyzing Cef spotify web content.

I am not sure if the proposal will help. CefGlue specifies version.

@mattkol , I have notice that, on Windows PC (I haven't test with another OS on PC), when you download the page, it returns enough information that Chromely can understand & find the version by using regex.

But when I tried on Raspberry PI 4, perhaps, due to its low memory, the render process runs slow, the content was not rendered by Javascript, that's why even I change the version in CefGlue, Chromely cannot find the targeted version.

Btw, can you tell me where I can find information of (on internet)

public const string CEF_API_HASH_PLATFORM_WIN = "3022db6736af26cc38158d38e22f74a51312a5c9";
        public const string CEF_API_HASH_PLATFORM_MACOS = "13e2e2451c2320f0d16b35ba53b3295409888dd8";
        public const string CEF_API_HASH_PLATFORM_LINUX = "abf4cc6ff102e5cd019aaf51f13eb30f9bfeae45";

[mattkol]

@redplane

Btw, can you tell me where I can find information of (on internet)

To change those, you may need to get that from CefGlue Version search.

There is a chance that what you are doing will work, but I have tried making such a change in the past and it just lead to more headaches. My conclusion is if there is not a major code change in the new version, then it will likely work, otherwise you may need an entire source change - this is the guide - here.

[FrankPfattheicher]

@redplane
As i have written the loader there is a long history of the RasPi (linux-arm) builds.
To support raspi i also provided self-build CEF builds for linux-arm (http://chromely.org/cefbuilds/index.html)
Starting with spotify supporting linux-arm end this apporach...

Due to the missing version 86.0.21 on spotify - i see no simple solution here :-(

I dont't like to restart the self-build apporach.

What do you mean?

[redplane]

I think it will be better if Chromely do:

• Allow developer to change the download link of cef_sharp file.
• Every release should include the cef_sharp tar file, the library will be more consistent. :)

[mattkol]

Allow developer to change the download link of cef_sharp file.

Is this still related to CefGlue? The way CefGlue is structured, I do not know what benefit that brings. It is a fixed version. May be as a fallback url, incase it fails like in issue #257. But that is a very very rare occurrence.

Every release should include the cef_sharp tar file, the library will be more consistent. :)

What is cef_sharp tar file?

[redplane]

Allow developer to change the download link of cef_sharp file.

Is this still related to CefGlue? The way CefGlue is structured, I do not know what benefit that brings. It is a fixed version. May be as a fallback url, incase it fails like in issue #257. But that is a very very rare occurrence.

Every release should include the cef_sharp tar file, the library will be more consistent. :)

What is cef_sharp tar file?

I did an application with Chromely before, everything was fine, but after one or 2 months, i built my application again, Chromely cannot download the Cef binary anymore because they removed the old binary version (version 74 arm). I think it will be better to have a fallback url or any kind of mechanism that developer can specify where to download Cef binary file, he/she can copy Cef file to his/her cdn and point the download link there

[FrankPfattheicher]

@redplane i think, with "cef_sharp tar file" you mean the cef_binary archive containing the CEF runtime.
Bundling that with Chromely will result in VERY LARGE packets due there are nine platforms meanwhile...
The idea of the loader was exactly to avoid that.

@mattkol maybe we can "copy" that files for all available platforms and host them on our site or make them available any other way to avoid them missing on spotify if removed there?

Also i think an alternative download link should be possible as options parameter, but that has to be the direct file link (platfor specific).

[redplane]

@FrankPfattheicher ,

About the tar file I mentioned before, actually I didn't say Chromely have to include the tar file in its bundle.
What I thought was to upload the CEF binary together with Chromely release.
Let say, we can have those items in each section of Github release page:

• Released binary file (optional - it can be downloaded from Nuget)
• CEF binary for linux, windows, ... which the current Chromely is rely on.

For now, I notice Chromely heavily depends on CefSharp cdn hosting, that's why after 2 months I built the project, I got error 404. That requires me to update Chromely, it is very time consuming.

[redplane]

@FrankPfattheicher, @mattkol

For now, I'm fixing the issue of missing version 86 on linux-arm for my project to run first.
I'm trying to update to version 87, but when I run the application, after download & decompression completed, I had this exception:

Here what I have changed in version.g.cs

        public const string CEF_VERSION = "87.1.13+g481a82a+chromium-87.0.4280.141";
        public const int CEF_VERSION_MAJOR = 87;
        
        public const int CHROME_VERSION_MAJOR = 87;
        public const int CHROME_VERSION_MINOR = 0;
        public const int CHROME_VERSION_BUILD = 4280;
        public const int CHROME_VERSION_PATCH = 141;
        
//....
        public const string CEF_API_HASH_PLATFORM_MACOS = "13e2e2451c2320f0d16b35ba53b3295409888dd8";

//...

Am I missing anything ?

[mattkol]

Am I missing anything ?

Remember what I wrote earlier:?

There is a chance that what you are doing will work, but I have tried making such a change in the past and it just lead to more headaches.

The easier way I know is to follow the guide I shared earlier - https://github.com/chromelyapps/Chromely/blob/master/src/Chromely/CefGlue/Notes_On_Upgrade.txt

Just note that, if you have to debug anything in CefGlue itself, you are doing something wrong. That is my experience.

[redplane]

Am I missing anything ?

Remember what I wrote earlier:?

There is a chance that what you are doing will work, but I have tried making such a change in the past and it just lead to more headaches.

The easier way I know is to follow the guide I shared earlier - https://github.com/chromelyapps/Chromely/blob/master/src/Chromely/CefGlue/Notes_On_Upgrade.txt

Just note that, if you have to debug anything in CefGlue itself, you are doing something wrong. That is my experience.

Phew, I have found the CefGlue and copy the latest one to the Chromely project.
Don't know why Chromely has to copy the CefGlue library source instead of refering to its dll file.
Finally, with some modifications, Chromely works again on my Raspberry Pi 4, with 2 commented methods that I don't know how to fix with new version of CefGlue:

/// <summary>
        /// Set whether mouse cursor change is disabled.
        /// </summary>
        public void SetMouseCursorChangeDisabled(bool disabled)
        {
            //cef_browser_host_t.set_mouse_cursor_change_disabled(_self, disabled ? 1 : 0);
        }

        /// <summary>
        /// Returns true if mouse cursor change is disabled.
        /// </summary>
        public bool IsMouseCursorChangeDisabled
        {
            get
            {
                // TODO: Implement here.
                return false;
                //return cef_browser_host_t.is_mouse_cursor_change_disabled(_self) != 0;
            }
        }

Still, I hope there will be a fallback url & copy of binary file on Github release to help developer manage their download more easily and don't depend on CEF cdn (even when they delete their distribution)

[mattkol]

Don't know why Chromely has to copy the CefGlue library source instead of refering to its dll file.

Good question.

Answer is simple, CefGlue does not publish nuget packages. Originally it was a separate dll, but since it was a dll created and maintained by us, there was no need creating a dependency. The nuget package are still created for those who still want to use it - https://github.com/chromelyapps/CefGlue.NetStandard

[mattkol]

You can also help create a nuget package for v87 and add it to https://github.com/chromelyapps/CefGlue.NetStandard

[redplane]

@mattkol , I tried to update to version 87 and it worked fine.
But for now, my app cannot focus on any text box or select on linux-arm
I'm commenting this function:

`
///

/// Set whether mouse cursor change is disabled.
///

public void SetMouseCursorChangeDisabled(bool disabled)
{
//cef_browser_host_t.set_mouse_cursor_change_disabled(_self, disabled ? 1 : 0);
}

    /// <summary>
    /// Returns true if mouse cursor change is disabled.
    /// </summary>
    public bool IsMouseCursorChangeDisabled
    {
        get
        {
            // TODO: Implement here.
            return false;
            //return cef_browser_host_t.is_mouse_cursor_change_disabled(_self) != 0;
        }
    }

`

Is it related to my issue ?

[mattkol]

@redplane

Sorry, I really cannot help here. As I stated earlier, my experience is, if you have to debug CefGlue itself, you are doing something wrong. Also I am not familiar with Raspberry Pi. @FrankPfattheicher may have more input on this.

[mattkol]

@redplane
Do you have this file cef_string_t.disabled.cs in https://github.com/chromelyapps/Chromely/tree/master/src/Chromely/CefGlue/Interop? If you do, try delete it. It may help.

[redplane]

@mattkol , I already deleted it because the application failed to compile.
Btw, thank you for your support.

[redplane]

I don't think this issue should be closed.
It isn't resolved yet. What developer needs about this is to manage the download instead of relying on the 3rd party hosting such as: CEF cdn.

[mattkol]

@redplane

If we are going to do that, it is going to be an enhancement. So it is not an issue per se. @FrankPfattheicher and I know what needs to be done if we are going to pursue it. Personally, I have not seen the viability of that. Last time I checked, I do not think Github will allow you transfer a file of libcef.dll file size (even if compressed) - https://docs.github.com/en/github/managing-large-files/conditions-for-large-files

Also we then need to host all CEF binaries for all platforms and all cpu - so we are looking at 8 at least.

But I am not ruling anything out.

[redplane]

@mattkol , @FrankPfattheicher , if you don't want to host every CEF binaries, I think there is no problem.
But Chromely library will be very unstable to include in a production software.

As I mentioned before:

3 months ago, I built a software and delivered to my customer. He was happy when he saw the product works as he expected.
Unfortunately, 2 weeks ago, he told me to update some functions on the source code I gave him.
I thought it was fine, just make some Front-end code and a little Chromely code, build & run, it would be fine.
But when I built my application to run on Windows, it showed 404 because Chrominium version 83 was deleted.
I had to update Chromely to the latest version, changed a lot of code.
It took time and I don't want to update Chromely again and again just because of missing CEF binary.

I already gave a solution for this:

• Chromely release should mention what version of Chrominum it uses (providing download link is very good option)
• Chromely should provide a service, interceptor or any kind of method just to allow developer to change the Download url incase it cannot find CEF binary from cdn.

I don't think this issue should be closed now because the problem is not resolved yet.
We cannot trust CEF cdn, because they can remove the old version at any time.

[mattkol]

@redplane

This is an open source project. There are limitations. If we cannot host CEF binaries on Github then it would require using an external server and service. That has its costs - financial, maintenance and human time dedications.

The way to solve such a problem, is not to rely entirely on auto download. If you release a version ensure either the CEF binaries are packaged with it or you download the CEF binaries and keep or you add a functionality that saves the CEF binaries on the user machine.

I am afraid, I do not see how this can be helped, but I will let @FrankPfattheicher chime in to see if has some ideas.

Chromely release should mention what version of Chrominum it uses (providing download link is very good option)

This is already available, but has not been updated for a while - https://github.com/chromelyapps/Chromely/blob/master/Documents/cef_binaries_download.md

#286

Chromely should provide a service, interceptor or any kind of method just to allow developer to change the Download url incase it cannot find CEF binary from cdn.

The interceptor is a good idea.
#287

On using an alternate url, I will let @FrankPfattheicher chime in on that.