ERROR: The following package names could not be resolved/Julia servers out of date

[MilesCranmer]

tl;dr, delete ~/.julia/registries/General and then run the following commands in Julia:

ENV["JULIA_PKG_SERVER"] = ""
import Pkg
Pkg.update()

---

(original post)
FYI I pushed v0.4.2 of SymbolicRegression.jl to the registry 15 hours ago, which is required for the latest PySR. However, the registry server is still not updating - which seems like an issue that sometimes happens: JuliaRegistries/General#16777.

To get the Julia registry to stay up to date even if the registry server fails, you can use the git version instead. This can be done as follows:

1. Delete your registry folder ~/.julia/registries/General
2. Launch Julia.
3. Run the following commands:

ENV["JULIA_PKG_SERVER"] = ""
import Pkg
Pkg.update()

This will install the git version of the registry, which is always up-to-date.

[MilesCranmer]

Server seems to have updated. Closing.

[MilesCranmer]

Unfortunately appearing again for v0.5.10. I have pinned this issue in case some of the Julia servers are out of date - just follow the above commands to get it working.

[mkitti]

For the impatient, I would just do the following to get the freshest release.

] add https://github.com/MilesCranmer/SymbolicRegression.jl#master

You can also replace master with a tag or commit hash. For example, you might this instead:

] add https://github.com/MilesCranmer/SymbolicRegression.jl#v0.6.0

[MilesCranmer]

Thanks!

I wonder if I should have PySR specify the Manifest.toml file (which could indicate the git repo) rather than just the Project.toml (which only indicates the project hash)? Then I suppose this would never be an issue - could just point directly to SymbolicRegression.jl.

[mkitti]

If this is meant as a library, I would not include the Manifest.toml.

I typically only include the Manifest.toml for terminal projects that I may want to reproduce exactly.

[MilesCranmer]

Oh I meant for the point about including SymbolicRegression.jl based on the git repo URL, rather than the Julia registry. I'm not sure how else to do that, for PySR users, other than to include a fixed Manifest.toml. (is there any way to enforce that in a Project.toml?)

[mkitti]

How users install your package is up to them not you, ultimately. For some, they either have to use specific mirrors or proxies for reliable access in their geographic locations. This is the problem that the pkg server protocol solves by providing a way to mirror and verify packages in a distributed and versioned fashion.

Pointing to your specific repository is a secondary option for those who cannot wait for the pkg servers to catch up and who also have reasonable direct access to Github.

So for the kind of package I think you are providing, I would recommend not including the Manifest.toml.

[MilesCranmer]

How users install your package is up to them not you, ultimately

This would be true if they were using the Julia API, but for PySR, I think this is only true for a handful of power users who know the julia_project param. >99% of PySR users have likely never installed a Julia package themselves, since PySR handles the installation of everything under the hood - it's these users I am concerned about over out-of-date registries.

The current approach to version control of Julia packages from within Python is to literally generate Project.toml in a temp directory, and activate it:

PySR/pysr/sr.py

Lines 340 to 358 in 1178fa3

	def _write_project_file(tmp_dir):
	"""This writes a Julia Project.toml to a temporary directory
	The reason we need this is because sometimes Python will compile a project to binary,
	and then Julia can't read the Project.toml file. It is more reliable to have Python
	simply create the Project.toml from scratch.
	"""
	project_toml = """
	[deps]
	SymbolicRegression = "8254be44-1295-4e6a-a16d-46603ac705cb"
	[compat]
	SymbolicRegression = "0.7.7, 0.7.8"
	julia = "1.5"
	"""
	project_toml_path = tmp_dir / "Project.toml"
	project_toml_path.write_text(project_toml)

[mkitti]

I'm not sure if I'm following why that is necessary. Couldn't you just run this code?

using Pkg
Pkg.activate(; temp=true)
spec = PackageSpec(name = "SymbolicRegression", uuid = "8254be44-1295-4e6a-a16d-46603ac705cb", version = v"0.7.7")
Pkg.add(spec)

[MilesCranmer]

Thanks, that definitely looks like a cleaner way to do it! (note I will be running this from PyJulia, not Julia)

Quick question: I found the url keyword which looks to allow you to specify a GitHub URL. Just to be clear: that would prevent Julia from checking the registry? I think that would finally eliminate this issue of out-of-date registries on user computers.

[mkitti]

Yes, the url keyword allows you to refer to any git repository. Also note the rev keyword is used for the git revision which could be a commit hash, branch, or tag.

[MilesCranmer]

Starting with v0.7.4, #27 will never be an issue again since SymbolicRegression.jl is downloaded directly from the GitHub repo. Unpinning.