diff options
Diffstat (limited to 'doc/cephadm/services')
| -rw-r--r-- | doc/cephadm/services/index.rst | 2 | ||||
| -rw-r--r-- | doc/cephadm/services/mgmt-gateway.rst | 2 | ||||
| -rw-r--r-- | doc/cephadm/services/mon.rst | 6 | ||||
| -rw-r--r-- | doc/cephadm/services/monitoring.rst | 50 | ||||
| -rw-r--r-- | doc/cephadm/services/osd.rst | 58 | ||||
| -rw-r--r-- | doc/cephadm/services/rgw.rst | 26 |
6 files changed, 105 insertions, 39 deletions
diff --git a/doc/cephadm/services/index.rst b/doc/cephadm/services/index.rst index 86a3fad8ab3..4df9933f8e7 100644 --- a/doc/cephadm/services/index.rst +++ b/doc/cephadm/services/index.rst @@ -357,6 +357,8 @@ Or in YAML: * See :ref:`orchestrator-host-labels` +.. _cephadm-services-placement-by-pattern-matching: + Placement by pattern matching ----------------------------- diff --git a/doc/cephadm/services/mgmt-gateway.rst b/doc/cephadm/services/mgmt-gateway.rst index 2b88d55952e..55c024817ae 100644 --- a/doc/cephadm/services/mgmt-gateway.rst +++ b/doc/cephadm/services/mgmt-gateway.rst @@ -183,7 +183,7 @@ The `mgmt-gateway` service internally makes use of nginx reverse proxy. The foll :: - DEFAULT_NGINX_IMAGE = 'quay.io/ceph/nginx:1.26.1' + mgr/cephadm/container_image_nginx = 'quay.io/ceph/nginx:sclorg-nginx-126' Admins can specify the image to be used by changing the `container_image_nginx` cephadm module option. If there were already running daemon(s) you must redeploy the daemon(s) in order to have them actually use the new image. diff --git a/doc/cephadm/services/mon.rst b/doc/cephadm/services/mon.rst index 389dc450e95..86cc121c9d5 100644 --- a/doc/cephadm/services/mon.rst +++ b/doc/cephadm/services/mon.rst @@ -23,8 +23,8 @@ cluster to a particular subnet. ``cephadm`` designates that subnet as the default subnet of the cluster. New monitor daemons will be assigned by default to that subnet unless cephadm is instructed to do otherwise. -If all of the ceph monitor daemons in your cluster are in the same subnet, -manual administration of the ceph monitor daemons is not necessary. +If all of the Ceph monitor daemons in your cluster are in the same subnet, +manual administration of the Ceph monitor daemons is not necessary. ``cephadm`` will automatically add up to five monitors to the subnet, as needed, as new hosts are added to the cluster. @@ -35,7 +35,7 @@ the placement of daemons. Designating a Particular Subnet for Monitors -------------------------------------------- -To designate a particular IP subnet for use by ceph monitor daemons, use a +To designate a particular IP subnet for use by Ceph monitor daemons, use a command of the following form, including the subnet's address in `CIDR`_ format (e.g., ``10.1.2.0/24``): diff --git a/doc/cephadm/services/monitoring.rst b/doc/cephadm/services/monitoring.rst index a0187363b5e..ef987fd7bd3 100644 --- a/doc/cephadm/services/monitoring.rst +++ b/doc/cephadm/services/monitoring.rst @@ -173,24 +173,22 @@ the [ceph-users] mailing list in April of 2024. The thread can be viewed here: ``var/lib/ceph/{FSID}/cephadm.{DIGEST}``, where ``{DIGEST}`` is an alphanumeric string representing the currently-running version of Ceph. -To see the default container images, run a command of the following form: +To see the default container images, run below command: .. prompt:: bash # - grep -E "DEFAULT*IMAGE" /var/lib/ceph/{FSID}/cephadm.{DIGEST} + cephadm list-images -:: - - DEFAULT_PROMETHEUS_IMAGE = 'quay.io/prometheus/prometheus:v2.51.0' - DEFAULT_LOKI_IMAGE = 'docker.io/grafana/loki:2.9.5' - DEFAULT_PROMTAIL_IMAGE = 'docker.io/grafana/promtail:2.9.5' - DEFAULT_NODE_EXPORTER_IMAGE = 'quay.io/prometheus/node-exporter:v1.7.0' - DEFAULT_ALERT_MANAGER_IMAGE = 'quay.io/prometheus/alertmanager:v0.27.0' - DEFAULT_GRAFANA_IMAGE = 'quay.io/ceph/grafana:10.4.0' Default monitoring images are specified in -``/src/cephadm/cephadmlib/constants.py`` and in -``/src/pybind/mgr/cephadm/module.py``. +``/src/python-common/ceph/cephadm/images.py``. + + +.. autoclass:: ceph.cephadm.images.DefaultImages + :members: + :undoc-members: + :exclude-members: desc, image_ref, key + Using custom images ~~~~~~~~~~~~~~~~~~~ @@ -304,14 +302,24 @@ Option names """""""""""" The following templates for files that will be generated by cephadm can be -overridden. These are the names to be used when storing with ``ceph config-key -set``: +overridden. These are the names to be used when storing with ``ceph config-key set``: - ``services/alertmanager/alertmanager.yml`` +- ``services/alertmanager/web.yml`` - ``services/grafana/ceph-dashboard.yml`` - ``services/grafana/grafana.ini`` +- ``services/ingress/haproxy.cfg`` +- ``services/ingress/keepalived.conf`` +- ``services/iscsi/iscsi-gateway.cfg`` +- ``services/mgmt-gateway/external_server.conf`` +- ``services/mgmt-gateway/internal_server.conf`` +- ``services/mgmt-gateway/nginx.conf`` +- ``services/nfs/ganesha.conf`` +- ``services/node-exporter/web.yml`` +- ``services/nvmeof/ceph-nvmeof.conf`` +- ``services/oauth2-proxy/oauth2-proxy.conf`` - ``services/prometheus/prometheus.yml`` -- ``services/prometheus/alerting/custom_alerts.yml`` +- ``services/prometheus/web.yml`` - ``services/loki.yml`` - ``services/promtail.yml`` @@ -319,9 +327,21 @@ You can look up the file templates that are currently used by cephadm in ``src/pybind/mgr/cephadm/templates``: - ``services/alertmanager/alertmanager.yml.j2`` +- ``services/alertmanager/web.yml.j2`` - ``services/grafana/ceph-dashboard.yml.j2`` - ``services/grafana/grafana.ini.j2`` +- ``services/ingress/haproxy.cfg.j2`` +- ``services/ingress/keepalived.conf.j2`` +- ``services/iscsi/iscsi-gateway.cfg.j2`` +- ``services/mgmt-gateway/external_server.conf.j2`` +- ``services/mgmt-gateway/internal_server.conf.j2`` +- ``services/mgmt-gateway/nginx.conf.j2`` +- ``services/nfs/ganesha.conf.j2`` +- ``services/node-exporter/web.yml.j2`` +- ``services/nvmeof/ceph-nvmeof.conf.j2`` +- ``services/oauth2-proxy/oauth2-proxy.conf.j2`` - ``services/prometheus/prometheus.yml.j2`` +- ``services/prometheus/web.yml.j2`` - ``services/loki.yml.j2`` - ``services/promtail.yml.j2`` diff --git a/doc/cephadm/services/osd.rst b/doc/cephadm/services/osd.rst index 9c0b4d2b495..90ebd86f897 100644 --- a/doc/cephadm/services/osd.rst +++ b/doc/cephadm/services/osd.rst @@ -198,6 +198,18 @@ There are a few ways to create new OSDs: .. warning:: When deploying new OSDs with ``cephadm``, ensure that the ``ceph-osd`` package is not already installed on the target host. If it is installed, conflicts may arise in the management and control of the OSD that may lead to errors or unexpected behavior. +* OSDs created via ``ceph orch daemon add`` are by default not added to the orchestrator's OSD service, they get added to 'osd' service. To attach an OSD to a different, existing OSD service, issue a command of the following form: + + .. prompt:: bash * + + ceph orch osd set-spec-affinity <service_name> <osd_id(s)> + + For example: + + .. prompt:: bash # + + ceph orch osd set-spec-affinity osd.default_drive_group 0 1 + Dry Run ------- @@ -478,22 +490,27 @@ for that OSD and also set a specific memory target. For example, Advanced OSD Service Specifications =================================== -:ref:`orchestrator-cli-service-spec`\s of type ``osd`` are a way to describe a -cluster layout, using the properties of disks. Service specifications give the -user an abstract way to tell Ceph which disks should turn into OSDs with which -configurations, without knowing the specifics of device names and paths. +:ref:`orchestrator-cli-service-spec`\s of type ``osd`` provide a way to use the +properties of disks to describe a Ceph cluster's layout. Service specifications +are an abstraction used to tell Ceph which disks it should transform into OSDs +and which configurations to apply to those OSDs. +:ref:`orchestrator-cli-service-spec`\s make it possible to target these disks +for transformation into OSDs even when the Ceph cluster operator does not know +the specific device names and paths associated with those disks. -Service specifications make it possible to define a yaml or json file that can -be used to reduce the amount of manual work involved in creating OSDs. +:ref:`orchestrator-cli-service-spec`\s make it possible to define a ``.yaml`` +or ``.json`` file that can be used to reduce the amount of manual work involved +in creating OSDs. .. note:: - It is recommended that advanced OSD specs include the ``service_id`` field - set. The plain ``osd`` service with no service id is where OSDs created - using ``ceph orch daemon add`` or ``ceph orch apply osd --all-available-devices`` - are placed. Not including a ``service_id`` in your OSD spec would mix - the OSDs from your spec with those OSDs and potentially overwrite services - specs created by cephadm to track them. Newer versions of cephadm will even - block creation of advanced OSD specs without the service_id present + We recommend that advanced OSD specs include the ``service_id`` field set. + OSDs created using ``ceph orch daemon add`` or ``ceph orch apply osd + --all-available-devices`` are placed in the plain ``osd`` service. Failing + to include a ``service_id`` in your OSD spec causes the Ceph cluster to mix + the OSDs from your spec with those OSDs, which can potentially result in the + overwriting of service specs created by ``cephadm`` to track them. Newer + versions of ``cephadm`` will even block creation of advanced OSD specs that + do not include the ``service_id``. For example, instead of running the following command: @@ -501,8 +518,8 @@ For example, instead of running the following command: ceph orch daemon add osd *<host>*:*<path-to-device>* -for each device and each host, we can define a yaml or json file that allows us -to describe the layout. Here's the most basic example. +for each device and each host, we can define a ``.yaml`` or ``.json`` file that +allows us to describe the layout. Here is the most basic example: Create a file called (for example) ``osd_spec.yml``: @@ -520,17 +537,18 @@ This means : #. Turn any available device (ceph-volume decides what 'available' is) into an OSD on all hosts that match the glob pattern '*'. (The glob pattern matches - against the registered hosts from `host ls`) A more detailed section on - host_pattern is available below. + against the registered hosts from `ceph orch host ls`) See + :ref:`cephadm-services-placement-by-pattern-matching` for more on using + ``host_pattern``-matching to turn devices into OSDs. -#. Then pass it to `osd create` like this: +#. Pass ``osd_spec.yml`` to ``osd create`` by using the following command: .. prompt:: bash [monitor.1]# ceph orch apply -i /path/to/osd_spec.yml - This instruction will be issued to all the matching hosts, and will deploy - these OSDs. + This instruction is issued to all the matching hosts, and will deploy these + OSDs. Setups more complex than the one specified by the ``all`` filter are possible. See :ref:`osd_filters` for details. diff --git a/doc/cephadm/services/rgw.rst b/doc/cephadm/services/rgw.rst index ed0b149365a..3df8ed2fc56 100644 --- a/doc/cephadm/services/rgw.rst +++ b/doc/cephadm/services/rgw.rst @@ -173,6 +173,32 @@ Then apply this yaml document: Note the value of ``rgw_frontend_ssl_certificate`` is a literal string as indicated by a ``|`` character preserving newline characters. +Disabling multisite sync traffic +-------------------------------- + +There is an RGW config option called ``rgw_run_sync_thread`` that tells the +RGW daemon to not transmit multisite replication data. This is useful if you want +that RGW daemon to be dedicated to I/O rather than multisite sync operations. +The RGW spec file includes a setting ``disable_multisite_sync_traffic`` that when +set to "True" will tell cephadm to set ``rgw_run_sync_thread`` to false for all +RGW daemons deployed for that RGW service. For example + +.. code-block:: yaml + + service_type: rgw + service_id: foo + placement: + label: rgw + spec: + rgw_realm: myrealm + rgw_zone: myzone + rgw_zonegroup: myzg + disable_multisite_sync_traffic: True + +.. note:: This will only stop the RGW daemon(s) from sending replication data. + The daemon can still receive replication data unless it has been removed + from the zonegroup and zone replication endpoints. + Service specification --------------------- |