-
Notifications
You must be signed in to change notification settings - Fork 5.9k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #56898 from adk3798/squid-cephadm-smb-backport
squid: cephadm: add a new SMB service to the mgr module Reviewed-by: John Mulligan <jmulligan@redhat.com>
- Loading branch information
Showing
25 changed files
with
2,026 additions
and
128 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,205 @@ | ||
.. _deploy-cephadm-smb-samba: | ||
|
||
=========== | ||
SMB Service | ||
=========== | ||
|
||
.. note:: Only the SMB3 protocol is supported. | ||
|
||
.. warning:: | ||
|
||
SMB support is under active development and many features may be | ||
missing or immature. Additionally, a Manager module to automate | ||
SMB clusters and SMB shares is in development. Once that feature | ||
is developed it will be the preferred method for managing | ||
SMB on ceph. | ||
|
||
|
||
Deploying Samba Containers | ||
========================== | ||
|
||
Cephadm deploys `Samba <http://www.samba.org>`_ servers using container images | ||
built by the `samba-container project <http://github.com/samba-in-kubernetes/samba-container>`_. | ||
|
||
In order to host SMB Shares with access to CephFS file systems, deploy | ||
Samba Containers with the following command: | ||
|
||
.. prompt:: bash # | ||
|
||
orch apply smb <cluster_id> <config_uri> [--features ...] [--placement ...] ... | ||
|
||
There are a number of additional parameters that the command accepts. See | ||
the Service Specification for a description of these options. | ||
|
||
Service Specification | ||
===================== | ||
|
||
An SMB Service can be applied using a specification. An example in YAML follows: | ||
|
||
.. code-block:: yaml | ||
service_type: smb | ||
service_id: tango | ||
placement: | ||
hosts: | ||
- ceph0 | ||
spec: | ||
cluster_id: tango | ||
features: | ||
- domain | ||
config_uri: rados://.smb/tango/scc.toml | ||
custom_dns: | ||
- "192.168.76.204" | ||
join_sources: | ||
- "rados:mon-config-key:smb/config/tango/join1.json" | ||
include_ceph_users: | ||
- client.smb.fs.cluster.tango | ||
The specification can then be applied by running the following command: | ||
|
||
.. prompt:: bash # | ||
|
||
ceph orch apply -i smb.yaml | ||
|
||
|
||
Service Spec Options | ||
-------------------- | ||
|
||
Fields specific to the ``spec`` section of the SMB Service are described below. | ||
|
||
cluster_id | ||
A short name identifying the SMB "cluster". In this case a cluster is | ||
simply a management unit of one or more Samba services sharing a common | ||
configuration, and may not provide actual clustering or availability | ||
mechanisms. | ||
|
||
features | ||
A list of pre-defined terms enabling specific deployment characteristics. | ||
An empty list is valid. Supported terms: | ||
|
||
* ``domain``: Enable domain member mode | ||
|
||
config_uri | ||
A string containing a (standard or de-facto) URI that identifies a | ||
configuration source that should be loaded by the samba-container as the | ||
primary configuration file. | ||
Supported URI schemes include ``http:``, ``https:``, ``rados:``, and | ||
``rados:mon-config-key:``. | ||
|
||
join_sources | ||
A list of strings with (standard or de-facto) URI values that will | ||
be used to identify where authentication data that will be used to | ||
perform domain joins are located. Each join source is tried in sequence | ||
until one succeeds. | ||
See ``config_uri`` for the supported list of URI schemes. | ||
|
||
custom_dns | ||
A list of IP addresses that will be used as the DNS servers for a Samba | ||
container. This features allows Samba Containers to integrate with | ||
Active Directory even if the Ceph host nodes are not tied into the Active | ||
Directory DNS domain(s). | ||
|
||
include_ceph_users: | ||
A list of cephx user (aka entity) names that the Samba Containers may use. | ||
The cephx keys for each user in the list will automatically be added to | ||
the keyring in the container. | ||
|
||
|
||
Configuring an SMB Service | ||
-------------------------- | ||
|
||
.. warning:: | ||
|
||
A Manager module for SMB is under active development. Once that module | ||
is available it will be the preferred method for managing Samba on Ceph | ||
in an end-to-end manner. The following discussion is provided for the sake | ||
of completeness and to explain how the software layers interact. | ||
|
||
Creating an SMB Service spec is not sufficient for complete operation of a | ||
Samba Container on Ceph. It is important to create valid configurations and | ||
place them in locations that the container can read. The complete specification | ||
of these configurations is out of scope for this document. You can refer to the | ||
`documentation for Samba <https://wiki.samba.org/index.php/Main_Page>`_ as | ||
well as the `samba server container | ||
<https://github.com/samba-in-kubernetes/samba-container/blob/master/docs/server.md>`_ | ||
and the `configuation file | ||
<https://github.com/samba-in-kubernetes/sambacc/blob/master/docs/configuration.md>`_ | ||
it accepts. | ||
|
||
When one has composed a configuration it should be stored in a location | ||
that the Samba Container can access. The recommended approach for running | ||
Samba Containers within Ceph orchestration is to store the configuration | ||
in the Ceph cluster. There are two ways to store the configuration | ||
in ceph: | ||
|
||
RADOS | ||
~~~~~ | ||
|
||
A configuration file can be stored as a RADOS object in a pool | ||
named ``.smb``. Within the pool there should be a namespace named after the | ||
``cluster_id`` value. The URI used to identify this resource should be | ||
constructed like ``rados://.smb/<cluster_id>/<object_name>``. Example: | ||
``rados://.smb/tango/config.json``. | ||
|
||
The containers are automatically deployed with cephx keys allowing access to | ||
resources in these pools and namespaces. As long as this scheme is used | ||
no additional configuration to read the object is needed. | ||
|
||
To copy a configuration file to a RADOS pool, use the ``rados`` command line | ||
tool. For example: | ||
|
||
.. prompt:: bash # | ||
|
||
# assuming your config file is /tmp/config.json | ||
rados --pool=.smb --namespace=tango put config.json /tmp/config.json | ||
|
||
MON Key/Value Store | ||
~~~~~~~~~~~~~~~~~~~ | ||
|
||
A configuration file can be stored as a value in the Ceph Monitor Key/Value | ||
store. The key must be named after the cluster like so: | ||
``smb/config/<cluster_id>/<name>``. This results in a URI that can be used to | ||
identify this configuration constructed like | ||
``rados:mon-config-key:smb/config/<cluster_id>/<name>``. | ||
Example: ``rados:mon-config-key:smb/config/tango/config.json``. | ||
|
||
The containers are automatically deployed with cephx keys allowing access to | ||
resources with the key-prefix ``smb/config/<cluster_id>/``. As long as this | ||
scheme is used no additional configuration to read the value is needed. | ||
|
||
To copy a configuration file into the Key/Value store use the ``ceph config-key | ||
put ...`` tool. For example: | ||
|
||
.. prompt:: bash # | ||
|
||
# assuming your config file is /tmp/config.json | ||
ceph config-key set smb/config/tango/config.json -i /tmp/config.json | ||
|
||
|
||
HTTP/HTTPS | ||
~~~~~~~~~~ | ||
|
||
A configuration file can be stored on an HTTP(S) server and automatically read | ||
by the Samba Container. Managing a configuration file on HTTP(S) is left as an | ||
exercise for the reader. | ||
|
||
.. note:: All URI schemes are supported by parameters that accept URIs. Each | ||
scheme has different performance and security characteristics. | ||
|
||
|
||
Limitations | ||
=========== | ||
|
||
A non-exhaustive list of important limitations for the SMB service follows: | ||
|
||
* DNS is a critical component of Active Directory. If one is configuring the | ||
SMB service for domain membership, either the Ceph host node must be | ||
configured so that it can resolve the Active Directory (AD) domain or the | ||
``custom_dns`` option may be used. In both cases DNS hosts for the AD domain | ||
must still be reachable from whatever network segment the ceph cluster is on. | ||
* Proper clustering/high-availability/"transparent state migration" is not yet | ||
supported. If a placement causes more than service to be created these | ||
services will act independently and may lead to unexpected behavior if clients | ||
access the same files at once. | ||
* Services must bind to TCP port 445. Running multiple SMB services on the same | ||
node is not yet supported and will trigger a port-in-use conflict. |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../.qa |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
.qa/distros/supported-container-hosts |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../.qa |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,81 @@ | ||
--- | ||
roles: | ||
# Test is for basic smb deployment & functionality. one node cluster is OK | ||
- - host.a | ||
- mon.a | ||
- mgr.x | ||
- osd.0 | ||
- osd.1 | ||
- client.0 | ||
# Reserve a host for acting as a test client | ||
- - host.b | ||
- cephadm.exclude | ||
overrides: | ||
ceph: | ||
log-only-match: | ||
- CEPHADM_ | ||
tasks: | ||
# TODO: (jjm) I don't think `install` is necessary for this file. Remove? | ||
- install: | ||
- cephadm.configure_samba_client_container: | ||
role: host.b | ||
- cephadm: | ||
- cephadm.shell: | ||
host.a: | ||
- ceph fs volume create cephfs | ||
- cephadm.wait_for_service: | ||
service: mds.cephfs | ||
- cephadm.shell: | ||
host.a: | ||
# create a subvolume so we can verify that we're sharing something | ||
- cmd: ceph fs subvolumegroup create cephfs g1 | ||
- cmd: ceph fs subvolume create cephfs sub1 --group-name=g1 --mode=0777 | ||
# Create a user access the file system from samba | ||
- cmd: ceph fs authorize cephfs client.smbdata / rw | ||
# Create a rados pool and store the config in it | ||
- cmd: ceph osd pool create .smb --yes-i-really-mean-it | ||
- cmd: ceph osd pool application enable .smb smb | ||
- cmd: rados --pool=.smb --namespace=saserv1 put conf.toml /dev/stdin | ||
stdin: | | ||
samba-container-config = "v0" | ||
[configs.saserv1] | ||
shares = ["share1"] | ||
globals = ["default", "domain"] | ||
instance_name = "SAMBA" | ||
[shares.share1.options] | ||
"vfs objects" = "ceph" | ||
path = "/" | ||
"ceph:config_file" = "/etc/ceph/ceph.conf" | ||
"ceph:user_id" = "smbdata" | ||
"kernel share modes" = "no" | ||
"read only" = "no" | ||
"browseable" = "yes" | ||
[globals.default.options] | ||
"server min protocol" = "SMB2" | ||
"load printers" = "no" | ||
"printing" = "bsd" | ||
"printcap name" = "/dev/null" | ||
"disable spoolss" = "yes" | ||
"guest ok" = "no" | ||
[globals.domain.options] | ||
security = "USER" | ||
workgroup = "STANDALONE1" | ||
[[users.all_entries]] | ||
name = "smbuser1" | ||
password = "insecure321" | ||
- cephadm.apply: | ||
specs: | ||
- service_type: smb | ||
service_id: saserv1 | ||
placement: | ||
count: 1 | ||
cluster_id: saserv1 | ||
config_uri: "rados://.smb/saserv1/conf.toml" | ||
include_ceph_users: | ||
- "client.smbdata" | ||
- cephadm.wait_for_service: | ||
service: smb.saserv1 | ||
- cephadm.exec: | ||
host.b: | ||
- sleep 30 | ||
- "{{ctx.samba_client_container_cmd|join(' ')}} smbclient -U smbuser1%insecure321 //{{'host.a'|role_to_remote|attr('ip_address')}}/share1 -c ls" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
roles: | ||
# Test is for basic smb deployment & functionality. one node cluster is OK | ||
- - host.a | ||
- mon.a | ||
- mgr.x | ||
- osd.0 | ||
- osd.1 | ||
- client.0 | ||
# Reserve a host for acting as a domain controller | ||
- - host.b | ||
- cephadm.exclude | ||
overrides: | ||
ceph: | ||
log-only-match: | ||
- CEPHADM_ | ||
tasks: | ||
- cephadm.deploy_samba_ad_dc: | ||
role: host.b | ||
- cephadm: | ||
|
||
- cephadm.shell: | ||
host.a: | ||
- ceph fs volume create cephfs | ||
- cephadm.wait_for_service: | ||
service: mds.cephfs | ||
|
||
- cephadm.shell: | ||
host.a: | ||
# create a subvolume so we can verify that we're sharing something | ||
- cmd: ceph fs subvolumegroup create cephfs g1 | ||
- cmd: ceph fs subvolume create cephfs sub1 --group-name=g1 --mode=0777 | ||
# Create a user access the file system from samba | ||
- cmd: ceph fs authorize cephfs client.smbdata / rw | ||
# Create a rados pool and store the config in it | ||
- cmd: ceph osd pool create .smb --yes-i-really-mean-it | ||
- cmd: ceph osd pool application enable .smb smb | ||
- cmd: rados --pool=.smb --namespace=admem1 put conf.toml /dev/stdin | ||
stdin: | | ||
samba-container-config = "v0" | ||
[configs.admem1] | ||
shares = ["share1"] | ||
globals = ["default", "domain"] | ||
instance_name = "SAMBA" | ||
[shares.share1.options] | ||
"vfs objects" = "ceph" | ||
path = "/" | ||
"ceph:config_file" = "/etc/ceph/ceph.conf" | ||
"ceph:user_id" = "smbdata" | ||
"kernel share modes" = "no" | ||
"read only" = "no" | ||
"browseable" = "yes" | ||
[globals.default.options] | ||
"server min protocol" = "SMB2" | ||
"load printers" = "no" | ||
"printing" = "bsd" | ||
"printcap name" = "/dev/null" | ||
"disable spoolss" = "yes" | ||
"guest ok" = "no" | ||
[globals.domain.options] | ||
security = "ads" | ||
workgroup = "DOMAIN1" | ||
realm = "domain1.sink.test" | ||
"idmap config * : backend" = "autorid" | ||
"idmap config * : range" = "2000-9999999" | ||
# Store the join auth user/pass in the config-key store | ||
- cmd: ceph config-key set smb/config/admem1/join1.json -i - | ||
stdin: | | ||
{"username": "Administrator", "password": "Passw0rd"} | ||
- cephadm.apply: | ||
specs: | ||
- service_type: smb | ||
service_id: admem1 | ||
placement: | ||
count: 1 | ||
cluster_id: admem1 | ||
features: | ||
- domain | ||
config_uri: "rados://.smb/admem1/conf.toml" | ||
custom_dns: | ||
- "{{ctx.samba_ad_dc_ip}}" | ||
join_sources: | ||
- "rados:mon-config-key:smb/config/admem1/join1.json" | ||
include_ceph_users: | ||
- "client.smbdata" | ||
- cephadm.wait_for_service: | ||
service: smb.admem1 | ||
|
||
- cephadm.exec: | ||
host.b: | ||
- sleep 30 | ||
- "{{ctx.samba_client_container_cmd|join(' ')}} smbclient -U DOMAIN1\\\\ckent%1115Rose. //{{'host.a'|role_to_remote|attr('ip_address')}}/share1 -c ls" |
Oops, something went wrong.