Use XDG Base directories for settings, cache and runtime data

[PureTryOut]

Description

This makes Mycroft use the XDG Base Directory specification.
Data is now saved to:

• settings to $XDG_CONFIG_DIR/mycroft, which most of the time equals to ~/.config/mycroft
• data to $XDG_DATA_DIRS/mycroft, which most of the time equals to ~/.local/share/mycroft
• cache to $XDG_CACHE_HOME/mycroft, which most of the time equals to ~/.cache

Since this changes the default location of pretty much everything, please test it well!

Goal is to support and read from the old ~/.mycroft location if it already exists, but save to the XDG Base Directories and read from it on new installations.

Together with #2559 this should be full XDG Base Directory compliance.

How to test

First of all, run this PR with an existing installation, and check if everything is loaded from the usual locations. Then shut it down and restart to make sure it has saved to the new locations.
Then, completely remove any trace of Mycroft settings from your system. Easiest would be to fully deinstall (remove /opt/mycroft), remove ~/.mycroft, and the XDG Base directories mentioned above.
Now reinstall Mycroft (with this PR applied) and start and connect it. Check if ~/.mycroft is not created anymore and ~/.config/mycroft, ~/.local/share/mycroft and ~/.cache/mycroft are with the correct files and directories in them.

Contributor license agreement signed?

CLA [x]

[devops-mycroft]

Voight Kampff Integration Test Failed (Results)

[pep8speaks]

Hello @PureTryOut! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-01-08 20:50:15 UTC

[forslund]

I still haven't finished writing the specs for this feature you've already implemented so I'll start by sending some of the notes I made on this PR this weekend.

[PureTryOut]

I still haven't finished writing the specs for this feature you've already implemented

Woops, sorry. I got too excited for this 🙈

[devops-mycroft]

Voight Kampff Integration Test Failed (Results)

[devops-mycroft]

Voight Kampff Integration Test Failed (Results)

[devops-mycroft]

Voight Kampff Integration Test Failed (Results)

[PureTryOut]

Made the changes, let's see! Honestly, VK is one big mystery to me.

[forslund]

Just realized this will bork (and did bork) the VK test runner since it moves the identity file (and doesn't restore it)... Could you edit the Jenkinsfile@L60 to be
-v "$HOME/voight-kampff/identity:/root/.config/mycroft/identity" \

to make sure the identity file isn't moved from the shared volume.

[PureTryOut]

Hmm, still failing. Seems testing the skills times out or something?

[forslund]

Right now the identity file is broken (I think) so many of the tests using it (weather, Q and A) are failing and it gets 10 seconds timeout of all of those.

We can retry when the identity file for VK is restored

[krisgesling]

New identity file in place, triggered a new run 🤞

[PureTryOut]

Uh, seems it ignores the updated Jenkinsfile file and keeps mounting the file to /root/.mycroft/identity...

https://hudson.mycroft.team/blue/organizations/jenkins/mycroft-core/detail/PR-2578/35/pipeline#step-39-log-1

[forslund]

Connecting to https://api.github.com to check permissions of obtain list of PureTryOut for MycroftAI/mycroft-core
Loading trusted files from base branch dev at 81eae60b61071746d3f5837c1fe5b8b64ecd0e64 rather than db5a3eba75d86c000240fed66bf26bc30e98cb77
Obtained Jenkinsfile from 81eae60b61071746d3f5837c1fe5b8b64ecd0e64
‘Jenkinsfile’ has been modified in an untrusted revision

Seems like Jenkins doesn't trust you :(

I'll see if I can figure out how to make it trust you / this change of the Jenkinsfile

[forslund]

If you make changes don't push to this branch since it'll destroy the jenkins identity file :/ ... Update in a branch and let me know. I'll pull them into the branch for the PR I created

[PureTryOut]

Done, my branch xdg-config-jarbas contains the changes.

Too bad another review with more requested changes comes in so late 😢

[j1nx]

Done, my branch xdg-config-jarbas contains the changes.

Too bad another review with more requested changes comes in so late 😢

Tried out that branch on OVOS. The following error pops up;

Traceback (most recent call last):
  File "/usr/bin/mycroft-skills", line 11, in <module>
    load_entry_point('mycroft-core==20.8.0', 'console_scripts', 'mycroft-skills')()
  File "/usr/lib/python3.8/site-packages/mycroft/skills/__main__.py", line 197, in main
    event_scheduler = EventScheduler(bus)
  File "/usr/lib/python3.8/site-packages/mycroft/skills/event_scheduler.py", line 70, in __init__
    self.schedule_file = new_path
NameError: name 'new_path' is not defined

[PureTryOut]

Yup known problem @j1nx, fixed in #2794

[j1nx]

Right! To manu different branches / tries because of this VK stuff. Will check/test the other one for you.

[PureTryOut]

Closing this as it's being replaced by #2794 and it seems it causes confusion by being still open. It's the same changes, but with stuff to make VK work (which I don't have permissions for in this PR).