Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nautilus: mgr/orchestrator: Cache and DeepSea iSCSI + NFS #29060

Merged
merged 9 commits into from Aug 2, 2019

Conversation

@sebastian-philipp
Copy link
Member

commented Jul 16, 2019

#28213 + #27336

  • References tracker ticket
  • Updates documentation if necessary
  • Includes tests for new functionality or reproducer for bug
sebastian-philipp and others added 6 commits May 22, 2019
mgr/orchestrator: Add cache for Inventory and Services
mgr/mgr_module: Added persistent dict
  basically a pythonic interface for a k-v store
mgr/orchestrator: Added common code to implement a cache
mgr/ssh: The cache works by manually adding and removing hosts.
  only the data is invalidated
mgr/deepsea: The cache invalidates all at once and only
  valid objects are cache

Fixes https://tracker.ceph.com/issues/39990

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f49a875)

Conflicts:
	src/pybind/mgr/orchestrator.py
	src/pybind/mgr/ssh/module.py
mgr/orchestrator: Introduce OutdatableDictMixin
No need for a persistent cache in DeepSea

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 5471da5)
mgr/orchestrator: Impove type hint for describe_service
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 5af3a97)
mgr/orchestrator: fix some minor typos/kinks in inventory/service cache
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 2a85362)
mgr/deepsea: rejig service cache
The previous implementation didn't work properly when there was more
than once service on a node (e.g. mgr and mon on the same node).

Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit d472687)
mgr/deepsea: gracefully handle nonexistent nodes in {service,device} ls
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 79d6034)

@sebastian-philipp sebastian-philipp force-pushed the sebastian-philipp:nautilus-orchestrator_cache branch from 6de56bd to f4577b6 Jul 16, 2019

@tserong

This comment has been minimized.

Copy link
Contributor

commented Jul 17, 2019

@sebastian-philipp six approving reviews required?

@sebastian-philipp

This comment has been minimized.

Copy link
Member Author

commented Jul 17, 2019

@sebastian-philipp six approving reviews required?

basically saying no one should merge anything into N

@sebastian-philipp

This comment has been minimized.

Copy link
Member Author

commented Jul 17, 2019

DNM, because I want to wait for #27336 to be merged to master

@smithfarm smithfarm changed the title Nautilus: mgr/orchestrator: Add cache for Inventory and Services nautilus: mgr/orchestrator: Add cache for Inventory and Services Jul 24, 2019

tserong added 3 commits Jul 12, 2019
mgr/deepsea: return ganesha and iscsi endpoint URLs
This updates describe_service() to include nfs and iscsi services
(deepsea internally refers to these as "ganesha" and "igw" roles).
Additionally, if deepsea sets any of container_id, service, version,
rados_config_location, service_url, status or status_desc, these will
come through now too.

This relies on SUSE/DeepSea#1606 for the new
functionality, but if run against an older version of deepsea, it will
continue to operate as it did before, it just won't include
rados_config_location or service_url data.

I've also updated the caching in describe_service() so that if the
cache needs filling, it just requests all services from DeepSea, caches
everything, then does filtering on the results.  Previously, if the
cache was empty and only one service type was requested, the cache
would then *only* include that service and no other items, so subsequent
requests for different service types would return nothing until the
cache expired.

Fixes: https://tracker.ceph.com/issues/39095
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 1af7c08)
mgr/orchestrator: allow listing iscsi services
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit e2bebee)
mgr/orchestrator: add --refresh to `ceph orchestrator service ls`
`ceph orchestrator device ls` already supports --refresh to force
a reload, we should support this for `ceph orchestrator service ls`
as well.

Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit d5e320a)

@sebastian-philipp sebastian-philipp changed the title nautilus: mgr/orchestrator: Add cache for Inventory and Services nautilus: mgr/orchestrator: Cache and DeepSea iSCSI + NFS Jul 31, 2019

@sebastian-philipp sebastian-philipp removed the DNM label Jul 31, 2019

@sebastian-philipp sebastian-philipp requested a review from tserong Jul 31, 2019

@sebastian-philipp

This comment has been minimized.

Copy link
Member Author

commented Jul 31, 2019

Added the commits from #27336

@yuriw

This comment has been minimized.

Copy link
Contributor

commented Jul 31, 2019

@yuriw yuriw merged commit f3a6145 into ceph:nautilus Aug 2, 2019

4 checks passed

Docs: build check OK - docs built
Details
Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
make check make check succeeded
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.