Job fixes for git repositories

[glennmatthews]

Closes: #3856

What's Changed

• Replace import_tasks_from_jobs_root with import_jobs_as_celery_tasks, handling Celery loading of system jobs, local jobs, and git jobs all together.
  • Git jobs are imported as <repo_slug>.jobs.<module>.<ClassName>
  • This change allows removing settings.CELERY_IMPORTS as unnecessary
• Job class_path structure has been changed from "git.repo_slug/submodule_name/ClassName to simply "module_name.ClassName" throughout the code. (TODO: document this)
• Job.source and Job.git_repository have been removed as no longer needed, redundant
  • Job.unique_together is now just ["module_name", "job_class_name"]
• Job.slug has been removed.
• refresh_git_jobs() now relies on the registered Celery tasks instead of on jobs_in_directory(), which is no longer needed and has been removed
• get_jobs() and its helper _get_job_source_paths are no longer needed and have been removed
• Accessing JobModel.job_class now just pulls from the Celery registry instead of looking for the job on disk and potentially resyncing a Git repository
• refresh_job_models() has been reworked to rely on the registered Celery tasks instead of get_jobs()
• I found that the unittest runner recursively imports everything it finds in a /tests/ subdirectory (edit: or a test_*.py file), which was resulting in all of the nautobot/extras/tests/example_jobs being directly imported and registered with Celery when running tests. This is undesirable; I fixed it by moving this directory to nautobot/extras/test_jobs (edit: and renaming all of the job files to remove the test_ prefix from them) so that it no longer gets auto-imported by the runner.

TODO

• Explanation of Change(s)
• Added change log fragment(s) (for more information see the documentation)
• Attached Screenshots, Payload Example
• Unit, Integration Tests
• Documentation Updates (when adding/changing features)
• Example Plugin Updates (when adding/changing features)
• Outline Remaining Work, Constraints from Design
• Does not yet handle ensuring that all workers are up to date with their Git repository clones when receiving Job tasks. I'll be tacking that next.

[gsnider2195]

How does the slugify function/clean handle dots in the slug?

[glennmatthews]

They get discarded:

>>> from nautobot.core.models.fields import slugify_dashes_to_underscores
>>> slugify_dashes_to_underscores("Alpha.Beta-Gamma")
'alphabeta_gamma'

[glennmatthews]

TODO at this time: when syncing a Git repository, jobs from the repository are not executable correctly/without errors until the Celery workers are restarted. This appears to be due to the use of a prefork pool on the workers - i.e. while we're refreshing the task registry in the main worker process, the preforked child processes that actually execute tasks are not refreshed and contain stale or nonexistent registry entries for the refreshed tasks. I don't have a solution in mind at this time.

Resolved in b7e2b0a with some more mucking around in Celery guts.