New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2019.1: FileNotFoundError content/atlas.sql #2892

Open
apoleon opened this Issue Jan 15, 2019 · 15 comments

Comments

Projects
None yet
6 participants
@apoleon
Copy link

apoleon commented Jan 15, 2019

Hello,

I intented to upgrade UH from Debian version 2017.2+git20180906, a Git snapshot, to 2019. However when I try to start the game I get this error message:
Traceback (most recent call last):
File "/usr/games/unknown-horizons", line 381, in
main()
File "/usr/games/unknown-horizons", line 122, in main
ret = horizons.main.start(options)
File "/usr/lib/python3/dist-packages/horizons/main.py", line 173, in start
horizons.globals.db = _create_main_db()
File "/usr/lib/python3/dist-packages/horizons/main.py", line 587, in _create_main_db
with open(i, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'content/atlas.sql'

I have removed my settings.xml file in ~/.config/unknown-horizons and also deleted ~/.unknown-horizons/

If UH tries to create atlas.sql at startup, then it won't work because the content directory is in /usr/share/ which is read-only. Also the --no-atlas-generation option, documented in the manpage, does not work anymore. (no such option error)

@MarkusHackspacher

This comment has been minimized.

Copy link
Contributor

MarkusHackspacher commented Jan 15, 2019

That is not a good way, let the Debian paket as they are.
Better make a new dir in your home for ex. games.
cd games
git clone git@github.com:unknown-horizons/unknown-horizons.git
cd unknown-horizons
./run_uh.py

if you want the newest version you can update with:
git pull git@github.com:unknown-horizons/unknown-horizons.git

@apoleon

This comment has been minimized.

Copy link
Author

apoleon commented Jan 15, 2019

I maintain UH for Debian.

@MarkusHackspacher

This comment has been minimized.

Copy link
Contributor

MarkusHackspacher commented Jan 15, 2019

https://github.com/unknown-horizons/unknown-horizons/wiki/Databases
try to generate 'content/atlas.sql' with development/generate_atlases.py

@apoleon

This comment has been minimized.

Copy link
Author

apoleon commented Jan 15, 2019

Maybe I was not clear in my initial report. atlas.sql is not part of your source tarball distribution. It used to work with the Git snapshot from September 2018 even without this database present. I simply upgraded to 2019.1 and I expected no bahavioral change. If users are supposed to generate atlas.sql then it should be documented somehow. It also cannot be installed into the content directory because this one is installed into /usr/share/ on Debian systems and most likely all Linux based distributions. If atlas.sql should be writeable it must be installed into a writable directory like the user's home directory or if it should be system-writable, into /var/lib. Ideally your build system should take care of it.

@MarkusHackspacher

This comment has been minimized.

Copy link
Contributor

MarkusHackspacher commented Jan 15, 2019

It is the same case like:
#2716

@LinuxDonald

This comment has been minimized.

Copy link
Member

LinuxDonald commented Jan 15, 2019

The atlas file was an part of the old sourcetarball for example 2017.x releases. For now setup.py create the atlas files.

@apoleon

This comment has been minimized.

Copy link
Author

apoleon commented Jan 15, 2019

I run setup.py build_i18n as documented in https://github.com/unknown-horizons/unknown-horizons/wiki/Linux. The build log says:

The Python Imaging Library (PIL / Pillow) package is needed to run the atlas generator.

Ok, now I have python3-pil installed but then I get the following error message

error: can't copy 'content/gfx/atlas/010.png': doesn't exist or not a regular file

What am I missing?

@LinuxDonald

This comment has been minimized.

Copy link
Member

LinuxDonald commented Jan 15, 2019

I dont get the error in my home folder (using ubuntu)

What you could try is comment out Line 148-152: https://github.com/unknown-horizons/unknown-horizons/blob/master/setup.py#L148

And run development/generate_atlases.py from the unknown-horizons folder.

@apoleon

This comment has been minimized.

Copy link
Author

apoleon commented Jan 15, 2019

When generate_atlases is called from within setup.py I get the following error message:

Recreating all atlases
Traceback (most recent call last):
File "/home/apo/src/todo/unknown-horizons/horizons/engine/generate_atlases.py", line 429, in
generator.recreate()
File "/home/apo/src/todo/unknown-horizons/horizons/engine/generate_atlases.py", line 267, in recreate
self.save()
File "/home/apo/src/todo/unknown-horizons/horizons/engine/generate_atlases.py", line 224, in save
self._save_books(self.books)
File "/home/apo/src/todo/unknown-horizons/horizons/engine/generate_atlases.py", line 210, in _save_books
pool = multiprocessing.Pool(processes=processes)
File "/usr/lib/python3.7/multiprocessing/context.py", line 119, in Pool
context=self.get_context())
File "/usr/lib/python3.7/multiprocessing/pool.py", line 158, in init
self._setup_queues()
File "/usr/lib/python3.7/multiprocessing/pool.py", line 251, in _setup_queues
self._inqueue = self._ctx.SimpleQueue()
File "/usr/lib/python3.7/multiprocessing/context.py", line 112, in SimpleQueue
return SimpleQueue(ctx=self.get_context())
File "/usr/lib/python3.7/multiprocessing/queues.py", line 332, in init
self._rlock = ctx.Lock()
File "/usr/lib/python3.7/multiprocessing/context.py", line 67, in Lock
return Lock(ctx=self.get_context())
File "/usr/lib/python3.7/multiprocessing/synchronize.py", line 162, in init
SemLock.init(self, SEMAPHORE, 1, 1, ctx=ctx)
File "/usr/lib/python3.7/multiprocessing/synchronize.py", line 59, in init
unlink_now)
PermissionError: [Errno 13] Permission denied

If I run generate_atlases.py, which is in horizons/engine by the way, with

python3 horizons/engine/generate_atlases.py

the generation of atlas.sql and the corresponding assets works fine. So at the moment I can work around it by patching setup.py and disabling the generate_atlases function.

@LinuxDonald

This comment has been minimized.

Copy link
Member

LinuxDonald commented Jan 15, 2019

@jmdejong any idea?

@jmdejong

This comment has been minimized.

Copy link
Contributor

jmdejong commented Jan 15, 2019

distribute the atlases with the build anyways?

I think most of the code is structured around having the atlas being distributed with the release.

Is setup.py supposed to run when installing a release too? If so I could try add a check for whether it's a debug or release build for generating the atlases. (sorry, I don't know much of how the release process works)

@apoleon

This comment has been minimized.

Copy link
Author

apoleon commented Jan 15, 2019

I think building the atlas files from source is a good thing no matter if it is a development or release version. You will also ensure that your tooling works and your data is modifiable which is also more in spirit with your licensing terms.

I found out that my error message appears to be releated to POSIX semaphores.

https://stackoverflow.com/questions/2009278/python-multiprocessing-permission-denied

Apparently I have to remount my chroot environment and make /dev/shm read and writable for my normal user. Using chroot environments is quite common when building Debian packages and I don't know if our autobuilders will be affected by it too. Anyways I patched setup.py for now and commented out line 231

self.generate_atlases(2048)

I just run the script from debian/rules which is basically a Makefile. Maybe you can find another way to spawn this script. At the moment that works for me.

@jmdejong

This comment has been minimized.

Copy link
Contributor

jmdejong commented Jan 16, 2019

Atlas generation can safely be skipped, but the game will run slower.

The easiest way to fix the multiprocessing error is to just remove the multiprocessing entirely. All it does is speed up the install a little bit. It looks like it can safely be done synchronously.

@baka0815

This comment has been minimized.

Copy link

baka0815 commented Jan 17, 2019

Just to inform you, yesterday i installed 2019.1 on my Debian box (sid/unstable) and it started without problems.
I didn't start a game or something, just the menu and exited (manually) afterwards.
So looks good!

@theMarix

This comment has been minimized.

Copy link

theMarix commented Feb 3, 2019

I also ran into this when working on the SUSE package. Weirdly, while the setup.py would generate an atlas, that was actually broken. However, this only showed when starting a game. I would get messages like Something was searched, but not found :: content/gfx/atlas/054.png as the atlas.sql contained more entries than there actually were images in content/gfx/atlas. Weirdly, if I run the generation script manually like it is done in the create_release_tarball.sh script, then I get a working atlas and can successfully launch the tutorial.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment