diff options
118 files changed, 1929 insertions, 470 deletions
diff --git a/.tito/packages/openshift-ansible b/.tito/packages/openshift-ansible index 200f8d7f3..4e3183de1 100644 --- a/.tito/packages/openshift-ansible +++ b/.tito/packages/openshift-ansible @@ -1 +1 @@ -3.6.68-1 ./ +3.6.80-1 ./ @@ -29,17 +29,34 @@ tito build --rpm To build a container image of `openshift-ansible` using standalone **Docker**: cd openshift-ansible - docker build -t openshift/openshift-ansible . + docker build -f images/installer/Dockerfile -t openshift/openshift-ansible . -Alternatively this can be built using on **OpenShift** using a [build and image stream](https://docs.openshift.org/latest/architecture/core_concepts/builds_and_image_streams.html) with this command: +### Building on OpenShift + +To build an openshift-ansible image using an **OpenShift** [build and image stream](https://docs.openshift.org/latest/architecture/core_concepts/builds_and_image_streams.html) the straightforward command would be: oc new-build docker.io/aweiteka/playbook2image~https://github.com/openshift/openshift-ansible -The progress of the build can be monitored with: +However: because the `Dockerfile` for this repository is not in the top level directory, and because we can't change the build context to the `images/installer` path as it would cause the build to fail, the `oc new-app` command above will create a build configuration using the *source to image* strategy, which is the default approach of the [playbook2image](https://github.com/openshift/playbook2image) base image. This does build an image successfully, but unfortunately the resulting image will be missing some customizations that are handled by the [Dockerfile](images/installer/Dockerfile) in this repo. + +At the time of this writing there is no straightforward option to [set the dockerfilePath](https://docs.openshift.org/latest/dev_guide/builds/build_strategies.html#dockerfile-path) of a `docker` build strategy with `oc new-build`. The alternatives to achieve this are: + +- Use the simple `oc new-build` command above to generate the BuildConfig and ImageStream objects, and then manually edit the generated build configuration to change its strategy to `dockerStrategy` and set `dockerfilePath` to `images/installer/Dockerfile`. + +- Download and pass the `Dockerfile` to `oc new-build` with the `-D` option: + +``` +curl -s https://raw.githubusercontent.com/openshift/openshift-ansible/master/images/installer/Dockerfile | + oc new-build -D - \ + --docker-image=docker.io/aweiteka/playbook2image \ + https://github.com/openshift/openshift-ansible +``` + +Once a build is started, the progress of the build can be monitored with: oc logs -f bc/openshift-ansible -Once built, the image will be visible in the Image Stream created by the same command: +Once built, the image will be visible in the Image Stream created by `oc new-app`: oc describe imagestream openshift-ansible diff --git a/README_CONTAINER_IMAGE.md b/README_CONTAINER_IMAGE.md index b78073100..0d7f7f4af 100644 --- a/README_CONTAINER_IMAGE.md +++ b/README_CONTAINER_IMAGE.md @@ -1,6 +1,6 @@ # Containerized openshift-ansible to run playbooks -The [Dockerfile](Dockerfile) in this repository uses the [playbook2image](https://github.com/aweiteka/playbook2image) source-to-image base image to containerize `openshift-ansible`. The resulting image can run any of the provided playbooks. See [BUILD.md](BUILD.md) for image build instructions. +The [Dockerfile](images/installer/Dockerfile) in this repository uses the [playbook2image](https://github.com/openshift/playbook2image) source-to-image base image to containerize `openshift-ansible`. The resulting image can run any of the provided playbooks. See [BUILD.md](BUILD.md) for image build instructions. The image is designed to **run as a non-root user**. The container's UID is mapped to the username `default` at runtime. Therefore, the container's environment reflects that user's settings, and the configuration should match that. For example `$HOME` is `/opt/app-root/src`, so ssh keys are expected to be under `/opt/app-root/src/.ssh`. If you ran a container as `root` you would have to adjust the container's configuration accordingly, e.g. by placing ssh keys under `/root/.ssh` instead. Nevertheless, the expectation is that containers will be run as non-root; for example, this container image can be run inside OpenShift under the default `restricted` [security context constraint](https://docs.openshift.org/latest/architecture/additional_concepts/authorization.html#security-context-constraints). @@ -8,7 +8,7 @@ The image is designed to **run as a non-root user**. The container's UID is mapp ## Usage -The `playbook2image` base image provides several options to control the behaviour of the containers. For more details on these options see the [playbook2image](https://github.com/aweiteka/playbook2image) documentation. +The `playbook2image` base image provides several options to control the behaviour of the containers. For more details on these options see the [playbook2image](https://github.com/openshift/playbook2image) documentation. At the very least, when running a container you must specify: @@ -24,7 +24,7 @@ Here is an example of how to run a containerized `openshift-ansible` playbook th -v $HOME/.ssh/id_rsa:/opt/app-root/src/.ssh/id_rsa:Z \ -v /etc/ansible/hosts:/tmp/inventory \ -e INVENTORY_FILE=/tmp/inventory \ - -e PLAYBOOK_FILE=playbooks/certificate_expiry/default.yaml \ + -e PLAYBOOK_FILE=playbooks/byo/openshift-checks/certificate_expiry/default.yaml \ -e OPTS="-v" -t \ openshift/openshift-ansible @@ -40,7 +40,7 @@ Here is a detailed explanation of the options used in the command above: * `-v /etc/ansible/hosts:/tmp/inventory` and `-e INVENTORY_FILE=/tmp/inventory` mount the Ansible inventory file into the container as `/tmp/inventory` and set the corresponding environment variable to point at it respectively. The example uses `/etc/ansible/hosts` as the inventory file as this is a default location, but your inventory is likely to be elsewhere so please adjust as needed. Note that depending on the file you point to you might have to handle SELinux labels in a similar way as with the ssh keys, e.g. by adding a `:z` flag to the volume mount, so again you might prefer to copy the inventory to a dedicated location first. -* `-e PLAYBOOK_FILE=playbooks/certificate_expiry/default.yaml` specifies the playbook to run as a relative path from the top level directory of openshift-ansible. +* `-e PLAYBOOK_FILE=playbooks/byo/openshift-checks/certificate_expiry/default.yaml` specifies the playbook to run as a relative path from the top level directory of openshift-ansible. * `-e OPTS="-v"` and `-t` make the output look nicer: the `default.yaml` playbook does not generate results and runs quietly unless we add the `-v` option to the `ansible-playbook` invocation, and a TTY is allocated via `-t` so that Ansible adds color to the output. diff --git a/hack/build-images.sh b/hack/build-images.sh index f6210e239..3e9896caa 100755 --- a/hack/build-images.sh +++ b/hack/build-images.sh @@ -10,7 +10,7 @@ source_root=$(dirname "${0}")/.. prefix="openshift/openshift-ansible" version="latest" verbose=false -options="" +options="-f images/installer/Dockerfile" help=false for args in "$@" diff --git a/images/installer/Dockerfile b/images/installer/Dockerfile index 1df887f32..f6af018ca 100644 --- a/images/installer/Dockerfile +++ b/images/installer/Dockerfile @@ -46,6 +46,6 @@ ADD . /tmp/src RUN /usr/libexec/s2i/assemble # Add files for running as a system container -COPY system-container/root / +COPY images/installer/system-container/root / CMD [ "/usr/libexec/s2i/run" ] diff --git a/inventory/byo/hosts.origin.example b/inventory/byo/hosts.origin.example index 20f342023..d52036930 100644 --- a/inventory/byo/hosts.origin.example +++ b/inventory/byo/hosts.origin.example @@ -345,7 +345,7 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # selector: type=router1 # images: "openshift3/ose-${component}:${version}" # edits: [] -# certificates: +# certificate: # certfile: /path/to/certificate/abc.crt # keyfile: /path/to/certificate/abc.key # cafile: /path/to/certificate/ca.crt @@ -359,7 +359,7 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # serviceaccount: router # selector: type=router2 # images: "openshift3/ose-${component}:${version}" -# certificates: +# certificate: # certfile: /path/to/certificate/xyz.crt # keyfile: /path/to/certificate/xyz.key # cafile: /path/to/certificate/ca.crt @@ -521,6 +521,9 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # Currently, you may only alter the hostname portion of the url, alterting the # `/hawkular/metrics` path will break installation of metrics. #openshift_hosted_metrics_public_url=https://hawkular-metrics.example.com/hawkular/metrics +# Configure the prefix and version for the component images +#openshift_hosted_metrics_deployer_prefix=docker.io/openshift/origin- +#openshift_hosted_metrics_deployer_version=3.6.0 # Logging deployment # @@ -751,6 +754,10 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # Or you may optionally define your own build overrides configuration serialized as json #openshift_buildoverrides_json='{"BuildOverrides":{"configuration":{"apiVersion":"v1","kind":"BuildDefaultsConfig","forcePull":"true"}}}' +# Enable template service broker by specifying one of more namespaces whose +# templates will be served by the broker +#openshift_template_service_broker_namespaces=['openshift'] + # masterConfig.volumeConfig.dynamicProvisioningEnabled, configurable as of 1.2/3.2, enabled by default #openshift_master_dynamic_provisioning_enabled=False diff --git a/inventory/byo/hosts.ose.example b/inventory/byo/hosts.ose.example index f75a47bb8..647dd3d9a 100644 --- a/inventory/byo/hosts.ose.example +++ b/inventory/byo/hosts.ose.example @@ -345,7 +345,7 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # selector: type=router1 # images: "openshift3/ose-${component}:${version}" # edits: [] -# certificates: +# certificate: # certfile: /path/to/certificate/abc.crt # keyfile: /path/to/certificate/abc.key # cafile: /path/to/certificate/ca.crt @@ -359,7 +359,7 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # serviceaccount: router # selector: type=router2 # images: "openshift3/ose-${component}:${version}" -# certificates: +# certificate: # certfile: /path/to/certificate/xyz.crt # keyfile: /path/to/certificate/xyz.key # cafile: /path/to/certificate/ca.crt @@ -522,6 +522,9 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # Currently, you may only alter the hostname portion of the url, alterting the # `/hawkular/metrics` path will break installation of metrics. #openshift_hosted_metrics_public_url=https://hawkular-metrics.example.com/hawkular/metrics +# Configure the prefix and version for the component images +#openshift_hosted_metrics_deployer_prefix=registry.example.com:8888/openshift3/ +#openshift_hosted_metrics_deployer_version=3.6.0 # Logging deployment # @@ -752,6 +755,10 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # Or you may optionally define your own build overrides configuration serialized as json #openshift_buildoverrides_json='{"BuildOverrides":{"configuration":{"apiVersion":"v1","kind":"BuildDefaultsConfig","forcePull":"true"}}}' +# Enable template service broker by specifying one of more namespaces whose +# templates will be served by the broker +#openshift_template_service_broker_namespaces=['openshift'] + # masterConfig.volumeConfig.dynamicProvisioningEnabled, configurable as of 1.2/3.2, enabled by default #openshift_master_dynamic_provisioning_enabled=False diff --git a/openshift-ansible.spec b/openshift-ansible.spec index 19e6356e7..3e967a8e8 100644 --- a/openshift-ansible.spec +++ b/openshift-ansible.spec @@ -9,7 +9,7 @@ %global __requires_exclude ^/usr/bin/ansible-playbook$ Name: openshift-ansible -Version: 3.6.68 +Version: 3.6.80 Release: 1%{?dist} Summary: Openshift and Atomic Enterprise Ansible License: ASL 2.0 @@ -161,23 +161,29 @@ BuildArch: noarch %files playbooks %{_datadir}/ansible/%{name}/playbooks -# We moved playbooks/common/openshift-master/library up to the top and replaced -# it with a symlink. RPM doesn't handle this so we have to do some pre-transaction -# magic. See https://fedoraproject.org/wiki/Packaging:Directory_Replacement +# Along the history of openshift-ansible, some playbook directories had to be +# moved and were replaced with symlinks for backwards compatibility. +# RPM doesn't handle this so we have to do some pre-transaction magic. +# See https://fedoraproject.org/wiki/Packaging:Directory_Replacement %pretrans playbooks -p <lua> --- Define the path to directory being replaced below. +-- Define the paths to directories being replaced below. -- DO NOT add a trailing slash at the end. -path = "/usr/share/ansible/openshift-ansible/playbooks/common/openshift-master/library" -st = posix.stat(path) -if st and st.type == "directory" then - status = os.rename(path, path .. ".rpmmoved") - if not status then - suffix = 0 - while not status do - suffix = suffix + 1 - status = os.rename(path .. ".rpmmoved", path .. ".rpmmoved." .. suffix) +dirs_to_sym = { + "/usr/share/ansible/openshift-ansible/playbooks/common/openshift-master/library", + "/usr/share/ansible/openshift-ansible/playbooks/certificate_expiry" +} +for i,path in ipairs(dirs_to_sym) do + st = posix.stat(path) + if st and st.type == "directory" then + status = os.rename(path, path .. ".rpmmoved") + if not status then + suffix = 0 + while not status do + suffix = suffix + 1 + status = os.rename(path .. ".rpmmoved", path .. ".rpmmoved." .. suffix) + end + os.rename(path, path .. ".rpmmoved") end - os.rename(path, path .. ".rpmmoved") end end @@ -274,6 +280,88 @@ Atomic OpenShift Utilities includes %changelog +* Tue May 23 2017 Jenkins CD Merge Bot <tdawson@redhat.com> 3.6.80-1 +- RPM workaround for the move of cert playbooks (pep@redhat.com) +- health check playbooks: relocate and expand (lmeyer@redhat.com) + +* Tue May 23 2017 Scott Dodson <sdodson@redhat.com> 3.6.69-1 +- preflight int tests: fix for openshift_version dep (lmeyer@redhat.com) +- Removing requirement to pass aws credentials (esauer@redhat.com) +- Workaround sysctl module issue with py3 by converting task to lineinfile. + (abutcher@redhat.com) +- inventory: rename certificates->certificate in router example + (smilner@redhat.com) +- remove skopeo dependency on docker-py (jvallejo@redhat.com) +- improve error handling for missing vars (jvallejo@redhat.com) +- lib/base: Allow for more complex template params (jarrpa@redhat.com) +- Fix yamllint problems (sdodson@redhat.com) +- add ability to expose Elasticsearch as an external route + (rmeggins@redhat.com) +- Parameterized Calico/Node Arguments (vincent.schwarzer@yahoo.de) +- Fix auditConfig for non-HA environments (rteague@redhat.com) +- Added Docker Registry Port 5000 to Firewalld (vincent.schwarzer@yahoo.de) +- Added Calicoctl to deployment of Master Nodes (vincent.schwarzer@yahoo.de) +- move etcd upgrade related code into etcd_upgrade role (jchaloup@redhat.com) +- Localhost TMP Dir Fix (vincent.schwarzer@yahoo.de) +- Adjusted Naming Schema of Calico Roles (vincent.schwarzer@yahoo.de) +- Update hosts.*.example to include openshift_hosted_metrics_deployer_version + (pat2man@gmail.com) +- Fix gpg key path in our repo (sdodson@redhat.com) +- Uninstall: restart docker when container-engine restart hasn't changed. + (abutcher@redhat.com) +- add etcd cluster size check (jvallejo@redhat.com) +- fix etcd_container_version detection (jchaloup@redhat.com) +- systemcontainercustom.conf.j2: use Environment instead of ENVIRONMENT + (gscrivan@redhat.com) +- node, systemd: change Requires to Wants for openvswitch (gscrivan@redhat.com) +- Add teams attribute to github identity provider (dms@redhat.com) +- Don't escalate privileges in local tmpdir creation (skuznets@redhat.com) +- Remove use of local_action with delegate_to and switch 'delegate_to: + localhost' temporary directory cleanup actions to local_actions. + (abutcher@redhat.com) +- Rework openshift_excluders role (rteague@redhat.com) +- Add regexp for container-engine lineinfile (smilner@redhat.com) +- Default image policy on new clusters to on (ccoleman@redhat.com) +- revert role-specific var name (jvallejo@redhat.com) +- Filter non-strings from the oc_adm_ca_server_cert hostnames parameter. + (abutcher@redhat.com) +- Don't set-up origin repositories if they've already been configured + (dms@redhat.com) +- byo inventory versions 1.5 -> 3.6 (smilner@redhat.com) +- byo inventory versions 3.5 -> 3.6 (smilner@redhat.com) +- use dest instead of path for lineinfile (smilner@redhat.com) +- openshift_version: skip rpm version==image version on Atomic + (gscrivan@redhat.com) +- Add NO_PROXY workaround for container-engine atomic command + (smilner@redhat.com) +- Add no_proxy to atomic.conf (smilner@redhat.com) +- Include object validation in 3.6 upgrades (sdodson@redhat.com) +- uninstall: handle container-engine (gscrivan@redhat.com) +- Added Calico BGP Port 179 to Firewalld (vincent.schwarzer@yahoo.de) +- Fixed for python3 with Fedora 25 Atomic (donny@fortnebula.com) +- Add docker package for container-engine install (smilner@redhat.com) +- Fix python3 error in repoquery (jpeeler@redhat.com) +- check if hostname is in list of etcd hosts (jvallejo@redhat.com) +- Fix templating of static service files (rteague@redhat.com) +- Fix container image build references (pep@redhat.com) +- Reset selinux context on /var/lib/origin/openshift.common.volumes + (sdodson@redhat.com) +- Adding assert to check for python-passlib on control host + (ewolinet@redhat.com) +- Update variable name to standard (rhcarvalho@gmail.com) +- Make class attribute name shorter (rhcarvalho@gmail.com) +- Add module docstring (rhcarvalho@gmail.com) +- Update check (rhcarvalho@gmail.com) +- Change based on feedback (vincent.schwarzer@yahoo.de) +- Removed Hardcoded Calico URLs (vincent.schwarzer@yahoo.de) +- int -> float (rhcarvalho@gmail.com) +- Remove vim line (rhcarvalho@gmail.com) +- add etcd volume check (jvallejo@redhat.com) +- Added additional Calico Network Plugin Checks (vincent.schwarzer@yahoo.de) +- Ensure good return code for specific until loops (smilner@redhat.com) +- add template service broker configurable (jminter@redhat.com) +- Prevent line wrap in yaml dump of IDP, fixes #3912 (rikkuness@gmail.com) + * Sat May 13 2017 Jenkins CD Merge Bot <tdawson@redhat.com> 3.6.68-1 - Updating registry-console image version during a post_control_plane upgrade (ewolinet@redhat.com) diff --git a/playbooks/adhoc/uninstall.yml b/playbooks/adhoc/uninstall.yml index 6119990fe..1c8257162 100644 --- a/playbooks/adhoc/uninstall.yml +++ b/playbooks/adhoc/uninstall.yml @@ -313,7 +313,7 @@ - name: restart docker service: name=docker state=restarted ignore_errors: true - when: "container_engine.state != 'started'" + when: not (container_engine | changed) - name: restart NetworkManager service: name=NetworkManager state=restarted diff --git a/playbooks/byo/openshift-checks/README.md b/playbooks/byo/openshift-checks/README.md new file mode 100644 index 000000000..4b2ff1f94 --- /dev/null +++ b/playbooks/byo/openshift-checks/README.md @@ -0,0 +1,65 @@ +# OpenShift health checks + +This directory contains Ansible playbooks for detecting potential problems prior +to an install, as well as health checks to run on existing OpenShift clusters. + +Ansible's default operation mode is to fail fast, on the first error. However, +when performing checks, it is useful to gather as much information about +problems as possible in a single run. + +Thus, the playbooks run a battery of checks against the inventory hosts and have +Ansible gather intermediate errors, giving a more complete diagnostic of the +state of each host. If any check failed, the playbook run will be marked as +failed. + +To facilitate understanding the problems that were encountered, a custom +callback plugin summarizes execution errors at the end of a playbook run. + +# Available playbooks + +1. Pre-install playbook ([pre-install.yml](pre-install.yml)) - verifies system + requirements and look for common problems that can prevent a successful + installation of a production cluster. + +2. Diagnostic playbook ([health.yml](health.yml)) - check an existing cluster + for known signs of problems. + +3. Certificate expiry playbooks ([certificate_expiry](certificate_expiry)) - + check that certificates in use are valid and not expiring soon. + +## Running + +With a [recent installation of Ansible](../../../README.md#setup), run the playbook +against your inventory file. Here is the step-by-step: + +1. If you haven't done it yet, clone this repository: + + ```console + $ git clone https://github.com/openshift/openshift-ansible + $ cd openshift-ansible + ``` + +2. Run the appropriate playbook: + + ```console + $ ansible-playbook -i <inventory file> playbooks/byo/openshift-checks/pre-install.yml + ``` + + or + + ```console + $ ansible-playbook -i <inventory file> playbooks/byo/openshift-checks/health.yml + ``` + + or + + ```console + $ ansible-playbook -i <inventory file> playbooks/byo/openshift-checks/certificate_expiry/default.yaml -v + ``` + +## Running via Docker image + +This repository is built into a Docker image including Ansible so that it can +be run anywhere Docker is available. Instructions for doing so may be found +[in the README](../../README_CONTAINER_IMAGE.md). + diff --git a/playbooks/certificate_expiry/default.yaml b/playbooks/byo/openshift-checks/certificate_expiry/default.yaml index 630135cae..630135cae 100644 --- a/playbooks/certificate_expiry/default.yaml +++ b/playbooks/byo/openshift-checks/certificate_expiry/default.yaml diff --git a/playbooks/certificate_expiry/easy-mode-upload.yaml b/playbooks/byo/openshift-checks/certificate_expiry/easy-mode-upload.yaml index 378d1f154..378d1f154 100644 --- a/playbooks/certificate_expiry/easy-mode-upload.yaml +++ b/playbooks/byo/openshift-checks/certificate_expiry/easy-mode-upload.yaml diff --git a/playbooks/certificate_expiry/easy-mode.yaml b/playbooks/byo/openshift-checks/certificate_expiry/easy-mode.yaml index ae41c7c14..ae41c7c14 100644 --- a/playbooks/certificate_expiry/easy-mode.yaml +++ b/playbooks/byo/openshift-checks/certificate_expiry/easy-mode.yaml diff --git a/playbooks/certificate_expiry/html_and_json_default_paths.yaml b/playbooks/byo/openshift-checks/certificate_expiry/html_and_json_default_paths.yaml index d80cb6ff4..d80cb6ff4 100644 --- a/playbooks/certificate_expiry/html_and_json_default_paths.yaml +++ b/playbooks/byo/openshift-checks/certificate_expiry/html_and_json_default_paths.yaml diff --git a/playbooks/certificate_expiry/html_and_json_timestamp.yaml b/playbooks/byo/openshift-checks/certificate_expiry/html_and_json_timestamp.yaml index 2189455b7..2189455b7 100644 --- a/playbooks/certificate_expiry/html_and_json_timestamp.yaml +++ b/playbooks/byo/openshift-checks/certificate_expiry/html_and_json_timestamp.yaml diff --git a/playbooks/certificate_expiry/longer-warning-period-json-results.yaml b/playbooks/byo/openshift-checks/certificate_expiry/longer-warning-period-json-results.yaml index 87a0f3be4..87a0f3be4 100644 --- a/playbooks/certificate_expiry/longer-warning-period-json-results.yaml +++ b/playbooks/byo/openshift-checks/certificate_expiry/longer-warning-period-json-results.yaml diff --git a/playbooks/certificate_expiry/longer_warning_period.yaml b/playbooks/byo/openshift-checks/certificate_expiry/longer_warning_period.yaml index 960457c4b..960457c4b 100644 --- a/playbooks/certificate_expiry/longer_warning_period.yaml +++ b/playbooks/byo/openshift-checks/certificate_expiry/longer_warning_period.yaml diff --git a/playbooks/byo/openshift-checks/certificate_expiry/roles b/playbooks/byo/openshift-checks/certificate_expiry/roles new file mode 120000 index 000000000..4bdbcbad3 --- /dev/null +++ b/playbooks/byo/openshift-checks/certificate_expiry/roles @@ -0,0 +1 @@ +../../../../roles
\ No newline at end of file diff --git a/playbooks/byo/openshift-checks/health.yml b/playbooks/byo/openshift-checks/health.yml new file mode 100644 index 000000000..dfc1a7db0 --- /dev/null +++ b/playbooks/byo/openshift-checks/health.yml @@ -0,0 +1,3 @@ +--- +- include: ../openshift-cluster/initialize_groups.yml +- include: ../../common/openshift-checks/health.yml diff --git a/playbooks/byo/openshift-checks/pre-install.yml b/playbooks/byo/openshift-checks/pre-install.yml new file mode 100644 index 000000000..5e8c3ab9b --- /dev/null +++ b/playbooks/byo/openshift-checks/pre-install.yml @@ -0,0 +1,3 @@ +--- +- include: ../openshift-cluster/initialize_groups.yml +- include: ../../common/openshift-checks/pre-install.yml diff --git a/playbooks/byo/openshift-preflight/README.md b/playbooks/byo/openshift-preflight/README.md deleted file mode 100644 index b50292eac..000000000 --- a/playbooks/byo/openshift-preflight/README.md +++ /dev/null @@ -1,43 +0,0 @@ -# OpenShift preflight checks - -Here we provide an Ansible playbook for detecting potential roadblocks prior to -an install or upgrade. - -Ansible's default operation mode is to fail fast, on the first error. However, -when performing checks, it is useful to gather as much information about -problems as possible in a single run. - -The `check.yml` playbook runs a battery of checks against the inventory hosts -and tells Ansible to ignore intermediate errors, thus giving a more complete -diagnostic of the state of each host. Still, if any check failed, the playbook -run will be marked as having failed. - -To facilitate understanding the problems that were encountered, we provide a -custom callback plugin to summarize execution errors at the end of a playbook -run. - ---- - -*Note that currently the `check.yml` playbook is only useful for RPM-based -installations. Containerized installs are excluded from checks for now, but -might be included in the future if there is demand for that.* - ---- - -## Running - -With an installation of Ansible 2.2 or greater, run the playbook directly -against your inventory file. Here is the step-by-step: - -1. If you haven't done it yet, clone this repository: - - ```console - $ git clone https://github.com/openshift/openshift-ansible - $ cd openshift-ansible - ``` - -2. Run the playbook: - - ```console - $ ansible-playbook -i <inventory file> playbooks/byo/openshift-preflight/check.yml - ``` diff --git a/playbooks/byo/openshift-preflight/check.yml b/playbooks/byo/openshift-preflight/check.yml index eb763221f..2e53452a6 100644 --- a/playbooks/byo/openshift-preflight/check.yml +++ b/playbooks/byo/openshift-preflight/check.yml @@ -1,15 +1,3 @@ --- -- include: ../openshift-cluster/initialize_groups.yml - -- name: Run OpenShift health checks - # Temporarily reverting to OSEv3 until group standardization is complete - hosts: OSEv3 - roles: - - openshift_health_checker - post_tasks: - # NOTE: we need to use the old "action: name" syntax until - # https://github.com/ansible/ansible/issues/20513 is fixed. - - action: openshift_health_check - args: - checks: - - '@preflight' +# location is moved; this file remains so existing instructions keep working +- include: ../openshift-checks/pre-install.yml diff --git a/playbooks/certificate_expiry b/playbooks/certificate_expiry new file mode 120000 index 000000000..9cf5334a1 --- /dev/null +++ b/playbooks/certificate_expiry @@ -0,0 +1 @@ +byo/openshift-checks/certificate_expiry/
\ No newline at end of file diff --git a/playbooks/certificate_expiry/roles b/playbooks/certificate_expiry/roles deleted file mode 120000 index b741aa3db..000000000 --- a/playbooks/certificate_expiry/roles +++ /dev/null @@ -1 +0,0 @@ -../../roles
\ No newline at end of file diff --git a/playbooks/common/openshift-checks/health.yml b/playbooks/common/openshift-checks/health.yml new file mode 100644 index 000000000..fc0f523d5 --- /dev/null +++ b/playbooks/common/openshift-checks/health.yml @@ -0,0 +1,10 @@ +--- +- name: Run OpenShift health checks + hosts: OSEv3 + roles: + - openshift_health_checker + post_tasks: + - action: openshift_health_check # https://github.com/ansible/ansible/issues/20513 + args: + checks: + - '@health' diff --git a/playbooks/common/openshift-checks/pre-install.yml b/playbooks/common/openshift-checks/pre-install.yml new file mode 100644 index 000000000..c8ffc3d91 --- /dev/null +++ b/playbooks/common/openshift-checks/pre-install.yml @@ -0,0 +1,10 @@ +--- +- hosts: OSEv3 + name: run OpenShift pre-install checks + roles: + - openshift_health_checker + post_tasks: + - action: openshift_health_check # https://github.com/ansible/ansible/issues/20513 + args: + checks: + - '@preflight' diff --git a/playbooks/byo/openshift-preflight/roles b/playbooks/common/openshift-checks/roles index 20c4c58cf..20c4c58cf 120000 --- a/playbooks/byo/openshift-preflight/roles +++ b/playbooks/common/openshift-checks/roles diff --git a/playbooks/common/openshift-cluster/redeploy-certificates/ca.yml b/playbooks/common/openshift-cluster/redeploy-certificates/ca.yml index 4fa7f9cdf..0d0ff798c 100644 --- a/playbooks/common/openshift-cluster/redeploy-certificates/ca.yml +++ b/playbooks/common/openshift-cluster/redeploy-certificates/ca.yml @@ -9,7 +9,8 @@ - name: Backup existing etcd CA certificate directories hosts: oo_etcd_to_config roles: - - etcd_common + - role: etcd_common + r_etcd_common_etcd_runtime: "{{ openshift.common.etcd_runtime }}" tasks: - name: Determine if CA certificate directory exists stat: @@ -52,7 +53,8 @@ vars: etcd_ca_host: "{{ groups.oo_etcd_to_config.0 }}" roles: - - etcd_common + - role: etcd_common + r_etcd_common_etcd_runtime: "{{ openshift.common.etcd_runtime }}" tasks: - name: Create a tarball of the etcd ca certs command: > @@ -98,7 +100,8 @@ - name: Retrieve etcd CA certificate hosts: oo_first_etcd roles: - - etcd_common + - role: etcd_common + r_etcd_common_etcd_runtime: "{{ openshift.common.etcd_runtime }}" tasks: - name: Retrieve etcd CA certificate fetch: diff --git a/playbooks/common/openshift-cluster/redeploy-certificates/etcd.yml b/playbooks/common/openshift-cluster/redeploy-certificates/etcd.yml index 2963a5940..6b5c805e6 100644 --- a/playbooks/common/openshift-cluster/redeploy-certificates/etcd.yml +++ b/playbooks/common/openshift-cluster/redeploy-certificates/etcd.yml @@ -3,7 +3,8 @@ hosts: oo_first_etcd any_errors_fatal: true roles: - - etcd_common + - role: etcd_common + r_etcd_common_etcd_runtime: "{{ openshift.common.etcd_runtime }}" post_tasks: - name: Determine if generated etcd certificates exist stat: @@ -27,7 +28,8 @@ hosts: oo_etcd_to_config any_errors_fatal: true roles: - - etcd_common + - role: etcd_common + r_etcd_common_etcd_runtime: "{{ openshift.common.etcd_runtime }}" post_tasks: - name: Backup etcd certificates command: > @@ -50,6 +52,7 @@ etcd_peers: "{{ groups.oo_etcd_to_config | default([], true) }}" etcd_certificates_etcd_hosts: "{{ groups.oo_etcd_to_config | default([], true) }}" openshift_ca_host: "{{ groups.oo_first_master.0 }}" + r_etcd_common_etcd_runtime: "{{ openshift.common.etcd_runtime }}" - name: Redeploy etcd client certificates for masters hosts: oo_masters_to_config @@ -63,4 +66,5 @@ etcd_cert_prefix: "master.etcd-" openshift_ca_host: "{{ groups.oo_first_master.0 }}" openshift_master_count: "{{ openshift.master.master_count | default(groups.oo_masters | length) }}" + r_etcd_common_etcd_runtime: "{{ openshift.common.etcd_runtime }}" when: groups.oo_etcd_to_config is defined and groups.oo_etcd_to_config diff --git a/playbooks/common/openshift-cluster/upgrades/etcd/backup.yml b/playbooks/common/openshift-cluster/upgrades/etcd/backup.yml index 9d0333ca8..b7fd2c0c5 100644 --- a/playbooks/common/openshift-cluster/upgrades/etcd/backup.yml +++ b/playbooks/common/openshift-cluster/upgrades/etcd/backup.yml @@ -1,84 +1,14 @@ --- - name: Backup etcd hosts: oo_etcd_hosts_to_backup - vars: - embedded_etcd: "{{ groups.oo_etcd_to_config | default([]) | length == 0 }}" - etcdctl_command: "{{ 'etcdctl' if not openshift.common.is_containerized or embedded_etcd else 'docker exec etcd_container etcdctl' if not openshift.common.is_etcd_system_container else 'runc exec etcd etcdctl' }}" - timestamp: "{{ lookup('pipe', 'date +%Y%m%d%H%M%S') }}" roles: - - openshift_facts - tasks: - # Ensure we persist the etcd role for this host in openshift_facts - - openshift_facts: - role: etcd - local_facts: {} - when: "'etcd' not in openshift" - - set_fact: - etcd_backup_dir: "{{ openshift.etcd.etcd_data_dir }}/openshift-backup-{{ backup_tag | default('') }}{{ timestamp }}" - - # TODO: replace shell module with command and update later checks - - name: Check available disk space for etcd backup - shell: df --output=avail -k {{ openshift.etcd.etcd_data_dir }} | tail -n 1 - register: avail_disk - # AUDIT:changed_when: `false` because we are only inspecting - # state, not manipulating anything - changed_when: false - - # TODO: replace shell module with command and update later checks - - name: Check current etcd disk usage - shell: du --exclude='*openshift-backup*' -k {{ openshift.etcd.etcd_data_dir }} | tail -n 1 | cut -f1 - register: etcd_disk_usage - when: embedded_etcd | bool - # AUDIT:changed_when: `false` because we are only inspecting - # state, not manipulating anything - changed_when: false - - - name: Abort if insufficient disk space for etcd backup - fail: - msg: > - {{ etcd_disk_usage.stdout }} Kb disk space required for etcd backup, - {{ avail_disk.stdout }} Kb available. - when: (embedded_etcd | bool) and (etcd_disk_usage.stdout|int > avail_disk.stdout|int) - - # For non containerized and non embedded we should have the correct version of - # etcd installed already. So don't do anything. - # - # For containerized installs we now exec into etcd_container - # - # For embedded non containerized we need to ensure we have the latest version - # etcd on the host. - - name: Install latest etcd for embedded - package: - name: etcd - state: latest - when: - - embedded_etcd | bool - - not openshift.common.is_atomic | bool - - - name: Generate etcd backup - command: > - {{ etcdctl_command }} backup --data-dir={{ openshift.etcd.etcd_data_dir }} - --backup-dir={{ etcd_backup_dir }} - - # According to the docs change you can simply copy snap/db - # https://github.com/openshift/openshift-docs/commit/b38042de02d9780842dce95cfa0ef45d53b58bc6 - - name: Check for v3 data store - stat: - path: "{{ openshift.etcd.etcd_data_dir }}/member/snap/db" - register: v3_db - - - name: Copy etcd v3 data store - command: > - cp -a {{ openshift.etcd.etcd_data_dir }}/member/snap/db - {{ etcd_backup_dir }}/member/snap/ - when: v3_db.stat.exists - - - set_fact: - etcd_backup_complete: True - - - name: Display location of etcd backup - debug: - msg: "Etcd backup created in {{ etcd_backup_dir }}" + - role: openshift_facts + - role: etcd_upgrade + r_etcd_upgrade_action: backup + r_etcd_backup_tag: etcd_backup_tag + r_etcd_common_etcd_runtime: "{{ openshift.common.etcd_runtime }}" + r_etcd_upgrade_embedded_etcd: "{{ groups.oo_etcd_to_config | default([]) | length == 0 }}" + r_etcd_backup_sufix_name: "{{ lookup('pipe', 'date +%Y%m%d%H%M%S') }}" - name: Gate on etcd backup hosts: localhost @@ -88,7 +18,7 @@ - set_fact: etcd_backup_completed: "{{ hostvars | oo_select_keys(groups.oo_etcd_hosts_to_backup) - | oo_collect('inventory_hostname', {'etcd_backup_complete': true}) }}" + | oo_collect('inventory_hostname', {'r_etcd_upgrade_backup_complete': true}) }}" - set_fact: etcd_backup_failed: "{{ groups.oo_etcd_hosts_to_backup | difference(etcd_backup_completed) }}" - fail: diff --git a/playbooks/common/openshift-cluster/upgrades/etcd/fedora_tasks.yml b/playbooks/common/openshift-cluster/upgrades/etcd/fedora_tasks.yml deleted file mode 100644 index 30232110e..000000000 --- a/playbooks/common/openshift-cluster/upgrades/etcd/fedora_tasks.yml +++ /dev/null @@ -1,23 +0,0 @@ ---- -# F23 GA'd with etcd 2.0, currently has 2.2 in updates -# F24 GA'd with etcd-2.2, currently has 2.2 in updates -# F25 Beta currently has etcd 3.0 -- name: Verify cluster is healthy pre-upgrade - command: "etcdctl --cert-file /etc/etcd/peer.crt --key-file /etc/etcd/peer.key --ca-file /etc/etcd/ca.crt -C https://{{ openshift.common.hostname }}:2379 cluster-health" - -- name: Update etcd - package: - name: "etcd" - state: "latest" - -- name: Restart etcd - service: - name: etcd - state: restarted - -- name: Verify cluster is healthy - command: "etcdctl --cert-file /etc/etcd/peer.crt --key-file /etc/etcd/peer.key --ca-file /etc/etcd/ca.crt -C https://{{ openshift.common.hostname }}:2379 cluster-health" - register: etcdctl - until: etcdctl.rc == 0 - retries: 3 - delay: 10 diff --git a/playbooks/common/openshift-cluster/upgrades/etcd/main.yml b/playbooks/common/openshift-cluster/upgrades/etcd/main.yml index d9b59edcb..3e01883ae 100644 --- a/playbooks/common/openshift-cluster/upgrades/etcd/main.yml +++ b/playbooks/common/openshift-cluster/upgrades/etcd/main.yml @@ -8,7 +8,7 @@ - name: Backup etcd before upgrading anything include: backup.yml vars: - backup_tag: "pre-upgrade-" + etcd_backup_tag: "pre-upgrade-" when: openshift_etcd_backup | default(true) | bool - name: Drop etcdctl profiles diff --git a/playbooks/common/openshift-cluster/upgrades/etcd/rhel_tasks.yml b/playbooks/common/openshift-cluster/upgrades/etcd/rhel_tasks.yml deleted file mode 100644 index 3a972e8ab..000000000 --- a/playbooks/common/openshift-cluster/upgrades/etcd/rhel_tasks.yml +++ /dev/null @@ -1,20 +0,0 @@ ---- -- name: Verify cluster is healthy pre-upgrade - command: "etcdctl --cert-file /etc/etcd/peer.crt --key-file /etc/etcd/peer.key --ca-file /etc/etcd/ca.crt -C https://{{ openshift.common.hostname }}:2379 cluster-health" - -- name: Update etcd RPM - package: - name: etcd-{{ upgrade_version }}* - state: latest - -- name: Restart etcd - service: - name: etcd - state: restarted - -- name: Verify cluster is healthy - command: "etcdctl --cert-file /etc/etcd/peer.crt --key-file /etc/etcd/peer.key --ca-file /etc/etcd/ca.crt -C https://{{ openshift.common.hostname }}:2379 cluster-health" - register: etcdctl - until: etcdctl.rc == 0 - retries: 3 - delay: 10 diff --git a/playbooks/common/openshift-cluster/upgrades/etcd/upgrade.yml b/playbooks/common/openshift-cluster/upgrades/etcd/upgrade.yml index 54f9e21a1..0431c1ce0 100644 --- a/playbooks/common/openshift-cluster/upgrades/etcd/upgrade.yml +++ b/playbooks/common/openshift-cluster/upgrades/etcd/upgrade.yml @@ -12,10 +12,10 @@ # AUDIT:changed_when: `false` because we are only inspecting # state, not manipulating anything changed_when: false - - debug: msg: "Etcd rpm version {{ etcd_rpm_version.stdout }} detected" - when: not openshift.common.is_containerized | bool + when: + - not openshift.common.is_containerized | bool - block: - name: Record containerized etcd version (docker) @@ -54,84 +54,57 @@ - debug: msg: "Etcd containerized version {{ etcd_container_version }} detected" - when: - openshift.common.is_containerized | bool -# I really dislike this copy/pasta but I wasn't able to find a way to get it to loop -# through hosts, then loop through tasks only when appropriate -- name: Upgrade to 2.1 - hosts: oo_etcd_hosts_to_upgrade - serial: 1 +- include: upgrade_rpm_members.yml vars: - upgrade_version: '2.1' - tasks: - - include: rhel_tasks.yml - when: etcd_rpm_version.stdout | default('99') | version_compare('2.1','<') and ansible_distribution == 'RedHat' and not openshift.common.is_containerized | bool + etcd_upgrade_version: '2.1' -- name: Upgrade RPM hosts to 2.2 - hosts: oo_etcd_hosts_to_upgrade - serial: 1 +- include: upgrade_rpm_members.yml vars: - upgrade_version: '2.2' - tasks: - - include: rhel_tasks.yml - when: etcd_rpm_version.stdout | default('99') | version_compare('2.2','<') and ansible_distribution == 'RedHat' and not openshift.common.is_containerized | bool + etcd_upgrade_version: '2.2' -- name: Upgrade containerized hosts to 2.2.5 - hosts: oo_etcd_hosts_to_upgrade - serial: 1 +- include: upgrade_image_members.yml vars: - upgrade_version: 2.2.5 - tasks: - - include: containerized_tasks.yml - when: etcd_container_version | default('99') | version_compare('2.2','<') and openshift.common.is_containerized | bool + etcd_upgrade_version: '2.2.5' -- name: Upgrade RPM hosts to 2.3 - hosts: oo_etcd_hosts_to_upgrade - serial: 1 +- include: upgrade_rpm_members.yml vars: - upgrade_version: '2.3' - tasks: - - include: rhel_tasks.yml - when: etcd_rpm_version.stdout | default('99') | version_compare('2.3','<') and ansible_distribution == 'RedHat' and not openshift.common.is_containerized | bool + etcd_upgrade_version: '2.3' -- name: Upgrade containerized hosts to 2.3.7 - hosts: oo_etcd_hosts_to_upgrade - serial: 1 +- include: upgrade_image_members.yml vars: - upgrade_version: 2.3.7 - tasks: - - include: containerized_tasks.yml - when: etcd_container_version | default('99') | version_compare('2.3','<') and openshift.common.is_containerized | bool + etcd_upgrade_version: '2.3.7' -- name: Upgrade RPM hosts to 3.0 - hosts: oo_etcd_hosts_to_upgrade - serial: 1 +- include: upgrade_rpm_members.yml vars: - upgrade_version: '3.0' - tasks: - - include: rhel_tasks.yml - when: etcd_rpm_version.stdout | default('99') | version_compare('3.0','<') and ansible_distribution == 'RedHat' and not openshift.common.is_containerized | bool + etcd_upgrade_version: '3.0' -- name: Upgrade containerized hosts to etcd3 image - hosts: oo_etcd_hosts_to_upgrade - serial: 1 +- include: upgrade_image_members.yml vars: - upgrade_version: 3.0.15 - tasks: - - include: containerized_tasks.yml - when: etcd_container_version | default('99') | version_compare('3.0','<') and openshift.common.is_containerized | bool + etcd_upgrade_version: '3.0.15' + +- include: upgrade_rpm_members.yml + vars: + etcd_upgrade_version: '3.1' + +- include: upgrade_image_members.yml + vars: + etcd_upgrade_version: '3.1.3' - name: Upgrade fedora to latest hosts: oo_etcd_hosts_to_upgrade serial: 1 tasks: - - include: fedora_tasks.yml - when: ansible_distribution == 'Fedora' and not openshift.common.is_containerized | bool + - include_role: + name: etcd_upgrade + when: + - ansible_distribution == 'Fedora' + - not openshift.common.is_containerized | bool - name: Backup etcd include: backup.yml vars: - backup_tag: "post-3.0-" + etcd_backup_tag: "post-3.0-" when: openshift_etcd_backup | default(true) | bool diff --git a/playbooks/common/openshift-cluster/upgrades/etcd/upgrade_image_members.yml b/playbooks/common/openshift-cluster/upgrades/etcd/upgrade_image_members.yml new file mode 100644 index 000000000..831ca8f57 --- /dev/null +++ b/playbooks/common/openshift-cluster/upgrades/etcd/upgrade_image_members.yml @@ -0,0 +1,17 @@ +--- +# INPUT etcd_upgrade_version +# INPUT etcd_container_version +# INPUT openshift.common.is_containerized +- name: Upgrade containerized hosts to {{ etcd_upgrade_version }} + hosts: oo_etcd_hosts_to_upgrade + serial: 1 + roles: + - role: etcd_upgrade + r_etcd_upgrade_action: upgrade + r_etcd_upgrade_mechanism: image + r_etcd_upgrade_version: "{{ etcd_upgrade_version }}" + r_etcd_common_etcd_runtime: "{{ openshift.common.etcd_runtime }}" + etcd_peer: "{{ openshift.common.hostname }}" + when: + - etcd_container_version | default('99') | version_compare(etcd_upgrade_version,'<') + - openshift.common.is_containerized | bool diff --git a/playbooks/common/openshift-cluster/upgrades/etcd/upgrade_rpm_members.yml b/playbooks/common/openshift-cluster/upgrades/etcd/upgrade_rpm_members.yml new file mode 100644 index 000000000..2e79451e0 --- /dev/null +++ b/playbooks/common/openshift-cluster/upgrades/etcd/upgrade_rpm_members.yml @@ -0,0 +1,18 @@ +--- +# INPUT etcd_upgrade_version +# INPUT etcd_rpm_version +# INPUT openshift.common.is_containerized +- name: Upgrade to {{ etcd_upgrade_version }} + hosts: oo_etcd_hosts_to_upgrade + serial: 1 + roles: + - role: etcd_upgrade + r_etcd_upgrade_action: upgrade + r_etcd_upgrade_mechanism: rpm + r_etcd_upgrade_version: "{{ etcd_upgrade_version }}" + r_etcd_common_etcd_runtime: "host" + etcd_peer: "{{ openshift.common.hostname }}" + when: + - etcd_rpm_version.stdout | default('99') | version_compare(etcd_upgrade_version, '<') + - ansible_distribution == 'RedHat' + - not openshift.common.is_containerized | bool diff --git a/playbooks/common/openshift-etcd/config.yml b/playbooks/common/openshift-etcd/config.yml index 1b8106e0e..2cb6197d1 100644 --- a/playbooks/common/openshift-etcd/config.yml +++ b/playbooks/common/openshift-etcd/config.yml @@ -7,4 +7,5 @@ etcd_peers: "{{ groups.oo_etcd_to_config | default([], true) }}" etcd_ca_host: "{{ groups.oo_etcd_to_config.0 }}" etcd_certificates_etcd_hosts: "{{ groups.oo_etcd_to_config | default([], true) }}" + r_etcd_common_etcd_runtime: "{{ openshift.common.etcd_runtime }}" - role: nickhammond.logrotate diff --git a/playbooks/common/openshift-node/restart.yml b/playbooks/common/openshift-node/restart.yml index 441b100e9..01cf948e0 100644 --- a/playbooks/common/openshift-node/restart.yml +++ b/playbooks/common/openshift-node/restart.yml @@ -51,7 +51,7 @@ register: node_output delegate_to: "{{ groups.oo_first_master.0 }}" when: inventory_hostname in groups.oo_nodes_to_config - until: node_output.results.results[0].status.conditions | selectattr('type', 'match', '^Ready$') | map(attribute='status') | join | bool == True + until: node_output.results.returncode == 0 and node_output.results.results[0].status.conditions | selectattr('type', 'match', '^Ready$') | map(attribute='status') | join | bool == True # Give the node two minutes to come back online. retries: 24 delay: 5 diff --git a/requirements.txt b/requirements.txt index 1996a967d..734ee6201 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,7 +1,7 @@ # Versions are pinned to prevent pypi releases arbitrarily breaking # tests with new APIs/semantics. We want to update versions deliberately. ansible==2.2.2.0 -boto==2.45.0 +boto==2.34.0 click==6.7 pyOpenSSL==16.2.0 # We need to disable ruamel.yaml for now because of test failures diff --git a/roles/calico/README.md b/roles/calico/README.md index 99e870521..9b9458bfa 100644 --- a/roles/calico/README.md +++ b/roles/calico/README.md @@ -20,6 +20,15 @@ To install, set the following inventory configuration parameters: * `openshift_use_openshift_sdn=False` * `os_sdn_network_plugin_name='cni'` +## Additional Calico/Node and Felix Configuration Options + +Additional parameters that can be defined in the inventory are: + +| Environment | Description | Schema | Default | +|---------|----------------------|---------|---------| +|CALICO_IPV4POOL_CIDR| The IPv4 Pool to create if none exists at start up. It is invalid to define this variable and NO_DEFAULT_POOLS. |IPv4 CIDR | 192.168.0.0/16 | +| CALICO_IPV4POOL_IPIP | IPIP Mode to use for the IPv4 POOL created at start up. | off, always, cross-subnet | always | +| CALICO_LOG_DIR | Directory on the host machine where Calico Logs are written.| String | /var/log/calico | ### Contact Information diff --git a/roles/calico/defaults/main.yaml b/roles/calico/defaults/main.yaml index a16a7da71..03c612982 100644 --- a/roles/calico/defaults/main.yaml +++ b/roles/calico/defaults/main.yaml @@ -12,3 +12,9 @@ calico_etcd_key_file: "/etc/origin/calico/calico.etcd-client.key" calico_url_cni: "https://github.com/projectcalico/cni-plugin/releases/download/v1.5.5/calico" calico_url_ipam: "https://github.com/projectcalico/cni-plugin/releases/download/v1.5.5/calico-ipam" + +calico_ipv4pool_ipip: "always" +calico_ipv4pool_cidr: "192.168.0.0/16" + +calico_log_dir: "/var/log/calico" +calico_node_image: "calico/node:v1.1.0" diff --git a/roles/calico/templates/calico.service.j2 b/roles/calico/templates/calico.service.j2 index 7a1236392..719d7ba0d 100644 --- a/roles/calico/templates/calico.service.j2 +++ b/roles/calico/templates/calico.service.j2 @@ -10,7 +10,8 @@ ExecStart=/usr/bin/docker run --net=host --privileged \ --name=calico-node \ -e WAIT_FOR_DATASTORE=true \ -e FELIX_DEFAULTENDPOINTTOHOSTACTION=ACCEPT \ - -e CALICO_IPV4POOL_IPIP=always \ + -e CALICO_IPV4POOL_IPIP={{ calico_ipv4pool_ipip }} \ + -e CALICO_IPV4POOL_CIDR={{ calico_ipv4pool_cidr }} \ -e FELIX_IPV6SUPPORT=false \ -e ETCD_ENDPOINTS={{ etcd_endpoints }} \ -v /etc/origin/calico:/etc/origin/calico \ @@ -18,10 +19,11 @@ ExecStart=/usr/bin/docker run --net=host --privileged \ -e ETCD_CERT_FILE={{ calico_etcd_cert_file }} \ -e ETCD_KEY_FILE={{ calico_etcd_key_file }} \ -e NODENAME={{ openshift.common.hostname }} \ - -v /var/log/calico:/var/log/calico \ + -v {{ calico_log_dir }}:/var/log/calico\ -v /lib/modules:/lib/modules \ -v /var/run/calico:/var/run/calico \ - calico/node:v1.1.0 + {{ calico_node_image }} + ExecStop=-/usr/bin/docker stop calico-node diff --git a/roles/calico_master/README.md b/roles/calico_master/README.md index 2d34a967c..6f5ed0664 100644 --- a/roles/calico_master/README.md +++ b/roles/calico_master/README.md @@ -21,6 +21,18 @@ To install, set the following inventory configuration parameters: * `os_sdn_network_plugin_name='cni'` + +## Additional Calico/Node and Felix Configuration Options + +Additional parameters that can be defined in the inventory are: + + +| Environment | Description | Schema | Default | +|---------|----------------------|---------|---------| +|CALICO_IPV4POOL_CIDR| The IPv4 Pool to create if none exists at start up. It is invalid to define this variable and NO_DEFAULT_POOLS. |IPv4 CIDR | 192.168.0.0/16 | +| CALICO_IPV4POOL_IPIP | IPIP Mode to use for the IPv4 POOL created at start up. | off, always, cross-subnet | always | +| CALICO_LOG_DIR | Directory on the host machine where Calico Logs are written.| String | /var/log/calico | + ### Contact Information Author: Dan Osborne <dan@projectcalico.org> diff --git a/roles/etcd/defaults/main.yaml b/roles/etcd/defaults/main.yaml index e45f53219..c0d1d5946 100644 --- a/roles/etcd/defaults/main.yaml +++ b/roles/etcd/defaults/main.yaml @@ -1,10 +1,4 @@ --- -etcd_service: "{{ 'etcd' if openshift.common.is_etcd_system_container | bool or not etcd_is_containerized | bool else 'etcd_container' }}" -etcd_client_port: 2379 -etcd_peer_port: 2380 -etcd_url_scheme: http -etcd_peer_url_scheme: http - etcd_initial_cluster_state: new etcd_initial_cluster_token: etcd-cluster-1 diff --git a/roles/etcd_common/defaults/main.yml b/roles/etcd_common/defaults/main.yml index d12e6a07f..e1a080b34 100644 --- a/roles/etcd_common/defaults/main.yml +++ b/roles/etcd_common/defaults/main.yml @@ -1,6 +1,9 @@ --- +# runc, docker, host +r_etcd_common_etcd_runtime: "docker" + # etcd server vars -etcd_conf_dir: "{{ '/etc/etcd' if not openshift.common.is_etcd_system_container else '/var/lib/etcd/etcd.etcd/etc' }}" +etcd_conf_dir: "{{ '/etc/etcd' if r_etcd_common_etcd_runtime != 'runc' else '/var/lib/etcd/etcd.etcd/etc' }}" etcd_system_container_conf_dir: /var/lib/etcd/etc etcd_conf_file: "{{ etcd_conf_dir }}/etcd.conf" etcd_ca_file: "{{ etcd_conf_dir }}/ca.crt" @@ -38,3 +41,9 @@ etcd_is_thirdparty: False # etcd dir vars etcd_data_dir: /var/lib/etcd/ + +# etcd ports and protocols +etcd_client_port: 2379 +etcd_peer_port: 2380 +etcd_url_scheme: http +etcd_peer_url_scheme: http diff --git a/roles/etcd_common/vars/main.yml b/roles/etcd_common/vars/main.yml new file mode 100644 index 000000000..00d697776 --- /dev/null +++ b/roles/etcd_common/vars/main.yml @@ -0,0 +1,4 @@ +--- +etcd_service: "{{ 'etcd_container' if r_etcd_common_etcd_runtime == 'docker' else 'etcd' }}" +# Location of the service file is fixed and not meant to be changed +etcd_service_file: "/etc/systemd/system/{{ etcd_service }}.service" diff --git a/roles/etcd_upgrade/defaults/main.yml b/roles/etcd_upgrade/defaults/main.yml new file mode 100644 index 000000000..01ad8a268 --- /dev/null +++ b/roles/etcd_upgrade/defaults/main.yml @@ -0,0 +1,9 @@ +--- +r_etcd_upgrade_action: upgrade +r_etcd_upgrade_mechanism: rpm +r_etcd_upgrade_embedded_etcd: False + +# etcd run on a host => use etcdctl command directly +# etcd run as a docker container => use docker exec +# etcd run as a runc container => use runc exec +etcdctl_command: "{{ 'etcdctl' if r_etcd_common_etcd_runtime == 'host' or r_etcd_upgrade_embedded_etcd | bool else 'docker exec etcd_container etcdctl' if r_etcd_common_etcd_runtime == 'docker' else 'runc exec etcd etcdctl' }}" diff --git a/roles/etcd_upgrade/meta/main.yml b/roles/etcd_upgrade/meta/main.yml new file mode 100644 index 000000000..018bdc8d7 --- /dev/null +++ b/roles/etcd_upgrade/meta/main.yml @@ -0,0 +1,16 @@ +--- +galaxy_info: + author: Jan Chaloupka + description: + company: Red Hat, Inc. + license: Apache License, Version 2.0 + min_ansible_version: 1.9 + platforms: + - name: EL + versions: + - 7 + categories: + - cloud + - system +dependencies: +- role: etcd_common diff --git a/roles/etcd_upgrade/tasks/backup.yml b/roles/etcd_upgrade/tasks/backup.yml new file mode 100644 index 000000000..1ea6fc59f --- /dev/null +++ b/roles/etcd_upgrade/tasks/backup.yml @@ -0,0 +1,71 @@ +--- +# INPUT r_etcd_backup_sufix_name +# INPUT r_etcd_backup_tag +# OUTPUT r_etcd_upgrade_backup_complete +- set_fact: + # ORIGIN etcd_data_dir etcd_common.defaults + l_etcd_backup_dir: "{{ etcd_data_dir }}/openshift-backup-{{ r_etcd_backup_tag | default('') }}{{ r_etcd_backup_sufix_name }}" + +# TODO: replace shell module with command and update later checks +- name: Check available disk space for etcd backup + shell: df --output=avail -k {{ etcd_data_dir }} | tail -n 1 + register: avail_disk + # AUDIT:changed_when: `false` because we are only inspecting + # state, not manipulating anything + changed_when: false + +# TODO: replace shell module with command and update later checks +- name: Check current etcd disk usage + shell: du --exclude='*openshift-backup*' -k {{ etcd_data_dir }} | tail -n 1 | cut -f1 + register: etcd_disk_usage + when: r_etcd_upgrade_embedded_etcd | bool + # AUDIT:changed_when: `false` because we are only inspecting + # state, not manipulating anything + changed_when: false + +- name: Abort if insufficient disk space for etcd backup + fail: + msg: > + {{ etcd_disk_usage.stdout }} Kb disk space required for etcd backup, + {{ avail_disk.stdout }} Kb available. + when: (r_etcd_upgrade_embedded_etcd | bool) and (etcd_disk_usage.stdout|int > avail_disk.stdout|int) + +# For non containerized and non embedded we should have the correct version of +# etcd installed already. So don't do anything. +# +# For containerized installs we now exec into etcd_container +# +# For embedded non containerized we need to ensure we have the latest version +# etcd on the host. +- name: Install latest etcd for embedded + package: + name: etcd + state: latest + when: + - r_etcd_upgrade_embedded_etcd | bool + - not l_ostree_booted.stat.exists | bool + +- name: Generate etcd backup + command: > + {{ etcdctl_command }} backup --data-dir={{ etcd_data_dir }} + --backup-dir={{ l_etcd_backup_dir }} + +# According to the docs change you can simply copy snap/db +# https://github.com/openshift/openshift-docs/commit/b38042de02d9780842dce95cfa0ef45d53b58bc6 +- name: Check for v3 data store + stat: + path: "{{ etcd_data_dir }}/member/snap/db" + register: v3_db + +- name: Copy etcd v3 data store + command: > + cp -a {{ etcd_data_dir }}/member/snap/db + {{ l_etcd_backup_dir }}/member/snap/ + when: v3_db.stat.exists + +- set_fact: + r_etcd_upgrade_backup_complete: True + +- name: Display location of etcd backup + debug: + msg: "Etcd backup created in {{ l_etcd_backup_dir }}" diff --git a/roles/etcd_upgrade/tasks/main.yml b/roles/etcd_upgrade/tasks/main.yml new file mode 100644 index 000000000..5178c14e3 --- /dev/null +++ b/roles/etcd_upgrade/tasks/main.yml @@ -0,0 +1,14 @@ +--- +# INPUT r_etcd_upgrade_action +- name: Fail if invalid etcd_upgrade_action provided + fail: + msg: "etcd_upgrade role can only be called with 'upgrade' or 'backup'" + when: + - r_etcd_upgrade_action not in ['upgrade', 'backup'] + +- name: Detecting Atomic Host Operating System + stat: + path: /run/ostree-booted + register: l_ostree_booted + +- include: "{{ r_etcd_upgrade_action }}.yml" diff --git a/roles/etcd_upgrade/tasks/upgrade.yml b/roles/etcd_upgrade/tasks/upgrade.yml new file mode 100644 index 000000000..420c9638e --- /dev/null +++ b/roles/etcd_upgrade/tasks/upgrade.yml @@ -0,0 +1,11 @@ +--- +# INPUT r_etcd_upgrade_version +# INPUT r_etcd_upgrade_mechanism +- name: Failt if r_etcd_upgrade_mechanism is not set during upgrade + fail: + msg: "r_etcd_upgrade_mechanism can be only set to 'rpm' or 'image'" + when: + - r_etcd_upgrade_mechanism not in ['rpm', 'image'] + +- name: "Upgrade {{ r_etcd_upgrade_mechanism }} based etcd" + include: upgrade_{{ r_etcd_upgrade_mechanism }}.yml diff --git a/playbooks/common/openshift-cluster/upgrades/etcd/containerized_tasks.yml b/roles/etcd_upgrade/tasks/upgrade_image.yml index 5f8b59e17..136ec1142 100644 --- a/playbooks/common/openshift-cluster/upgrades/etcd/containerized_tasks.yml +++ b/roles/etcd_upgrade/tasks/upgrade_image.yml @@ -1,27 +1,28 @@ --- +# INPUT r_etcd_upgrade_version - name: Verify cluster is healthy pre-upgrade - command: "etcdctl --cert-file /etc/etcd/peer.crt --key-file /etc/etcd/peer.key --ca-file /etc/etcd/ca.crt -C https://{{ openshift.common.hostname }}:2379 cluster-health" + command: "{{ etcdctlv2 }} cluster-health" - name: Get current image - shell: grep 'ExecStart=' /etc/systemd/system/etcd_container.service | awk '{print $NF}' + shell: "grep 'ExecStart=' {{ etcd_service_file }} | awk '{print $NF}'" register: current_image - name: Set new_etcd_image set_fact: - new_etcd_image: "{{ current_image.stdout | regex_replace('/etcd.*$','/etcd:' ~ upgrade_version ) }}" + new_etcd_image: "{{ current_image.stdout | regex_replace('/etcd.*$','/etcd:' ~ r_etcd_upgrade_version ) }}" - name: Pull new etcd image command: "docker pull {{ new_etcd_image }}" - name: Update to latest etcd image replace: - dest: /etc/systemd/system/etcd_container.service + dest: "{{ etcd_service_file }}" regexp: "{{ current_image.stdout }}$" replace: "{{ new_etcd_image }}" - name: Restart etcd_container systemd: - name: etcd_container + name: "{{ etcd_service }}" daemon_reload: yes state: restarted @@ -30,16 +31,17 @@ ## the container may be newer than etcdctl on the host. Assumes etcd3 obsoletes etcd (7.3.1) - name: Upgrade etcd for etcdctl when not atomic package: name=etcd state=latest - when: not openshift.common.is_atomic | bool + when: not l_ostree_booted.stat.exists | bool - name: Verify cluster is healthy - command: "etcdctl --cert-file /etc/etcd/peer.crt --key-file /etc/etcd/peer.key --ca-file /etc/etcd/ca.crt -C https://{{ openshift.common.hostname }}:2379 cluster-health" + command: "{{ etcdctlv2 }} cluster-health" register: etcdctl until: etcdctl.rc == 0 retries: 3 delay: 10 - name: Store new etcd_image + # DEPENDENCY openshift_facts openshift_facts: role: etcd local_facts: diff --git a/roles/etcd_upgrade/tasks/upgrade_rpm.yml b/roles/etcd_upgrade/tasks/upgrade_rpm.yml new file mode 100644 index 000000000..324b69605 --- /dev/null +++ b/roles/etcd_upgrade/tasks/upgrade_rpm.yml @@ -0,0 +1,32 @@ +--- +# INPUT r_etcd_upgrade_version? + +# F23 GA'd with etcd 2.0, currently has 2.2 in updates +# F24 GA'd with etcd-2.2, currently has 2.2 in updates +# F25 Beta currently has etcd 3.0 +# RHEL 7.3.4 with etcd-3.1.3-1.el7 +# RHEL 7.3.3 with etcd-3.1.0-2.el7 +# RHEL 7.3.2 with etcd-3.0.15-1.el7 + +- name: Verify cluster is healthy pre-upgrade + command: "{{ etcdctlv2 }} cluster-health" + +- set_fact: + l_etcd_target_package: "{{ 'etcd' if r_etcd_upgrade_version is not defined else 'etcd-'+r_etcd_upgrade_version+'*' }}" + +- name: Update etcd RPM to {{ l_etcd_target_package }} + package: + name: "{{ l_etcd_target_package }}" + state: latest + +- name: Restart etcd + service: + name: "{{ etcd_service }}" + state: restarted + +- name: Verify cluster is healthy + command: "{{ etcdctlv2 }} cluster-health" + register: etcdctl + until: etcdctl.rc == 0 + retries: 3 + delay: 10 diff --git a/roles/etcd_upgrade/vars/main.yml b/roles/etcd_upgrade/vars/main.yml new file mode 100644 index 000000000..5ed919d42 --- /dev/null +++ b/roles/etcd_upgrade/vars/main.yml @@ -0,0 +1,3 @@ +--- +# EXPECTS etcd_peer +etcdctlv2: "etcdctl --cert-file {{ etcd_peer_cert_file }} --key-file {{ etcd_peer_key_file }} --ca-file {{ etcd_peer_ca_file }} -C https://{{ etcd_peer }}:{{ etcd_client_port }}" diff --git a/roles/lib_openshift/library/oc_adm_ca_server_cert.py b/roles/lib_openshift/library/oc_adm_ca_server_cert.py index a6273cfe4..7573c5b85 100644 --- a/roles/lib_openshift/library/oc_adm_ca_server_cert.py +++ b/roles/lib_openshift/library/oc_adm_ca_server_cert.py @@ -952,7 +952,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_adm_manage_node.py b/roles/lib_openshift/library/oc_adm_manage_node.py index 7493b5c3d..bb3619081 100644 --- a/roles/lib_openshift/library/oc_adm_manage_node.py +++ b/roles/lib_openshift/library/oc_adm_manage_node.py @@ -938,7 +938,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_adm_policy_group.py b/roles/lib_openshift/library/oc_adm_policy_group.py index 5e72f5954..358d4515b 100644 --- a/roles/lib_openshift/library/oc_adm_policy_group.py +++ b/roles/lib_openshift/library/oc_adm_policy_group.py @@ -924,7 +924,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_adm_policy_user.py b/roles/lib_openshift/library/oc_adm_policy_user.py index 371a3953b..5807f41a8 100644 --- a/roles/lib_openshift/library/oc_adm_policy_user.py +++ b/roles/lib_openshift/library/oc_adm_policy_user.py @@ -924,7 +924,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_adm_registry.py b/roles/lib_openshift/library/oc_adm_registry.py index 7240521c6..e1b79466e 100644 --- a/roles/lib_openshift/library/oc_adm_registry.py +++ b/roles/lib_openshift/library/oc_adm_registry.py @@ -1042,7 +1042,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_adm_router.py b/roles/lib_openshift/library/oc_adm_router.py index a54c62cd4..e3b1bbcbc 100644 --- a/roles/lib_openshift/library/oc_adm_router.py +++ b/roles/lib_openshift/library/oc_adm_router.py @@ -1067,7 +1067,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_clusterrole.py b/roles/lib_openshift/library/oc_clusterrole.py index 78c72ef26..9f3e819a3 100644 --- a/roles/lib_openshift/library/oc_clusterrole.py +++ b/roles/lib_openshift/library/oc_clusterrole.py @@ -916,7 +916,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_configmap.py b/roles/lib_openshift/library/oc_configmap.py index c88f56fc6..3c0e82a09 100644 --- a/roles/lib_openshift/library/oc_configmap.py +++ b/roles/lib_openshift/library/oc_configmap.py @@ -922,7 +922,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_edit.py b/roles/lib_openshift/library/oc_edit.py index 17e3f7dde..008ce6a12 100644 --- a/roles/lib_openshift/library/oc_edit.py +++ b/roles/lib_openshift/library/oc_edit.py @@ -966,7 +966,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_env.py b/roles/lib_openshift/library/oc_env.py index 18ab97bc0..824ad4cb3 100644 --- a/roles/lib_openshift/library/oc_env.py +++ b/roles/lib_openshift/library/oc_env.py @@ -933,7 +933,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_group.py b/roles/lib_openshift/library/oc_group.py index 88c6ef209..7eacac38e 100644 --- a/roles/lib_openshift/library/oc_group.py +++ b/roles/lib_openshift/library/oc_group.py @@ -906,7 +906,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_image.py b/roles/lib_openshift/library/oc_image.py index 45860cbe5..266f8fbcf 100644 --- a/roles/lib_openshift/library/oc_image.py +++ b/roles/lib_openshift/library/oc_image.py @@ -925,7 +925,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_label.py b/roles/lib_openshift/library/oc_label.py index 65923a698..756d7db42 100644 --- a/roles/lib_openshift/library/oc_label.py +++ b/roles/lib_openshift/library/oc_label.py @@ -942,7 +942,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_obj.py b/roles/lib_openshift/library/oc_obj.py index 1d75a21b9..88d4ac8ca 100644 --- a/roles/lib_openshift/library/oc_obj.py +++ b/roles/lib_openshift/library/oc_obj.py @@ -945,7 +945,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_objectvalidator.py b/roles/lib_openshift/library/oc_objectvalidator.py index 72add01f4..8e42083ca 100644 --- a/roles/lib_openshift/library/oc_objectvalidator.py +++ b/roles/lib_openshift/library/oc_objectvalidator.py @@ -877,7 +877,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_process.py b/roles/lib_openshift/library/oc_process.py index 8e1ffe90f..330de07eb 100644 --- a/roles/lib_openshift/library/oc_process.py +++ b/roles/lib_openshift/library/oc_process.py @@ -934,7 +934,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_project.py b/roles/lib_openshift/library/oc_project.py index a06852fd8..b653d9018 100644 --- a/roles/lib_openshift/library/oc_project.py +++ b/roles/lib_openshift/library/oc_project.py @@ -931,7 +931,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_pvc.py b/roles/lib_openshift/library/oc_pvc.py index 79673452d..bab67d499 100644 --- a/roles/lib_openshift/library/oc_pvc.py +++ b/roles/lib_openshift/library/oc_pvc.py @@ -926,7 +926,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_route.py b/roles/lib_openshift/library/oc_route.py index ad705a6c5..7831ec8a4 100644 --- a/roles/lib_openshift/library/oc_route.py +++ b/roles/lib_openshift/library/oc_route.py @@ -976,7 +976,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_scale.py b/roles/lib_openshift/library/oc_scale.py index 291ac8b19..133942e55 100644 --- a/roles/lib_openshift/library/oc_scale.py +++ b/roles/lib_openshift/library/oc_scale.py @@ -920,7 +920,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_secret.py b/roles/lib_openshift/library/oc_secret.py index df28df2bc..8c6877bb2 100644 --- a/roles/lib_openshift/library/oc_secret.py +++ b/roles/lib_openshift/library/oc_secret.py @@ -966,7 +966,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_service.py b/roles/lib_openshift/library/oc_service.py index e98f83cc3..a482e13c1 100644 --- a/roles/lib_openshift/library/oc_service.py +++ b/roles/lib_openshift/library/oc_service.py @@ -972,7 +972,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_serviceaccount.py b/roles/lib_openshift/library/oc_serviceaccount.py index f00e9e4f6..263398e3d 100644 --- a/roles/lib_openshift/library/oc_serviceaccount.py +++ b/roles/lib_openshift/library/oc_serviceaccount.py @@ -918,7 +918,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_serviceaccount_secret.py b/roles/lib_openshift/library/oc_serviceaccount_secret.py index 6691495a6..cc7fda1b5 100644 --- a/roles/lib_openshift/library/oc_serviceaccount_secret.py +++ b/roles/lib_openshift/library/oc_serviceaccount_secret.py @@ -918,7 +918,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_user.py b/roles/lib_openshift/library/oc_user.py index 72f2fbf03..48ac28834 100644 --- a/roles/lib_openshift/library/oc_user.py +++ b/roles/lib_openshift/library/oc_user.py @@ -978,7 +978,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_version.py b/roles/lib_openshift/library/oc_version.py index bc3340a94..21dd5c3c9 100644 --- a/roles/lib_openshift/library/oc_version.py +++ b/roles/lib_openshift/library/oc_version.py @@ -890,7 +890,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/library/oc_volume.py b/roles/lib_openshift/library/oc_volume.py index 9dec0a6d4..be0944843 100644 --- a/roles/lib_openshift/library/oc_volume.py +++ b/roles/lib_openshift/library/oc_volume.py @@ -967,7 +967,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/lib_openshift/src/lib/base.py b/roles/lib_openshift/src/lib/base.py index 2bf795e25..70755187e 100644 --- a/roles/lib_openshift/src/lib/base.py +++ b/roles/lib_openshift/src/lib/base.py @@ -128,7 +128,7 @@ class OpenShiftCLI(object): else: cmd.append(template_name) if params: - param_str = ["{}={}".format(key, value) for key, value in params.items()] + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] cmd.append('-v') cmd.extend(param_str) diff --git a/roles/openshift_certificate_expiry/README.md b/roles/openshift_certificate_expiry/README.md index 107e27f89..f19a421cb 100644 --- a/roles/openshift_certificate_expiry/README.md +++ b/roles/openshift_certificate_expiry/README.md @@ -54,7 +54,7 @@ included in this role, or you can [read on below for more examples](#more-exampl to help you craft you own. ``` -$ ansible-playbook -v -i HOSTS playbooks/certificate_expiry/easy-mode.yaml +$ ansible-playbook -v -i HOSTS playbooks/byo/openshift-checks/certificate_expiry/easy-mode.yaml ``` Using the `easy-mode.yaml` playbook will produce: @@ -65,7 +65,7 @@ Using the `easy-mode.yaml` playbook will produce: > **Note:** If you are running from an RPM install use -> `/usr/share/ansible/openshift-ansible/playbooks/certificate_expiry/easy-mode.yaml` +> `/usr/share/ansible/openshift-ansible/playbooks/byo/openshift-checks/certificate_expiry/easy-mode.yaml` > instead ## Run from a container @@ -80,7 +80,7 @@ There are several [examples](../../examples/README.md) in the `examples` directo ## More Example Playbooks > **Note:** These Playbooks are available to run directly out of the -> [/playbooks/certificate_expiry/](../../playbooks/certificate_expiry/) directory. +> [/playbooks/byo/openshift-checks/certificate_expiry/](../../playbooks/byo/openshift-checks/certificate_expiry/) directory. ### Default behavior @@ -99,14 +99,14 @@ This playbook just invokes the certificate expiration check role with default op **From git:** ``` -$ ansible-playbook -v -i HOSTS playbooks/certificate_expiry/default.yaml +$ ansible-playbook -v -i HOSTS playbooks/byo/openshift-checks/certificate_expiry/default.yaml ``` **From openshift-ansible-playbooks rpm:** ``` -$ ansible-playbook -v -i HOSTS /usr/share/ansible/openshift-ansible/playbooks/certificate_expiry/default.yaml +$ ansible-playbook -v -i HOSTS /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-checks/certificate_expiry/default.yaml ``` -> [View This Playbook](../../playbooks/certificate_expiry/default.yaml) +> [View This Playbook](../../playbooks/byo/openshift-checks/certificate_expiry/default.yaml) ### Easy mode @@ -130,14 +130,14 @@ certificates (healthy or not) are included in the results: **From git:** ``` -$ ansible-playbook -v -i HOSTS playbooks/certificate_expiry/easy-mode.yaml +$ ansible-playbook -v -i HOSTS playbooks/byo/openshift-checks/certificate_expiry/easy-mode.yaml ``` **From openshift-ansible-playbooks rpm:** ``` -$ ansible-playbook -v -i HOSTS /usr/share/ansible/openshift-ansible/playbooks/certificate_expiry/easy-mode.yaml +$ ansible-playbook -v -i HOSTS /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-checks/certificate_expiry/easy-mode.yaml ``` -> [View This Playbook](../../playbooks/certificate_expiry/easy-mode.yaml) +> [View This Playbook](../../playbooks/byo/openshift-checks/certificate_expiry/easy-mode.yaml) ### Easy mode and upload reports to masters @@ -193,14 +193,14 @@ options via environment variables: **From git:** ``` -$ ansible-playbook -v -i HOSTS playbooks/certificate_expiry/easy-mode-upload.yaml +$ ansible-playbook -v -i HOSTS playbooks/byo/openshift-checks/certificate_expiry/easy-mode-upload.yaml ``` **From openshift-ansible-playbooks rpm:** ``` -$ ansible-playbook -v -i HOSTS /usr/share/ansible/openshift-ansible/playbooks/certificate_expiry/easy-mode-upload.yaml +$ ansible-playbook -v -i HOSTS /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-checks/certificate_expiry/easy-mode-upload.yaml ``` -> [View This Playbook](../../playbooks/certificate_expiry/easy-mode-upload.yaml) +> [View This Playbook](../../playbooks/byo/openshift-checks/certificate_expiry/easy-mode-upload.yaml) ### Generate HTML and JSON artifacts in their default paths @@ -219,14 +219,14 @@ $ ansible-playbook -v -i HOSTS /usr/share/ansible/openshift-ansible/playbooks/ce **From git:** ``` -$ ansible-playbook -v -i HOSTS playbooks/certificate_expiry/html_and_json_default_paths.yaml +$ ansible-playbook -v -i HOSTS playbooks/byo/openshift-checks/certificate_expiry/html_and_json_default_paths.yaml ``` **From openshift-ansible-playbooks rpm:** ``` -$ ansible-playbook -v -i HOSTS /usr/share/ansible/openshift-ansible/playbooks/certificate_expiry/html_and_json_default_paths.yaml +$ ansible-playbook -v -i HOSTS /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-checks/certificate_expiry/html_and_json_default_paths.yaml ``` -> [View This Playbook](../../playbooks/certificate_expiry/html_and_json_default_paths.yaml) +> [View This Playbook](../../playbooks/byo/openshift-checks/certificate_expiry/html_and_json_default_paths.yaml) ### Generate HTML and JSON reports in a custom path @@ -250,14 +250,14 @@ This example customizes the report generation path to point to a specific path ( **From git:** ``` -$ ansible-playbook -v -i HOSTS playbooks/certificate_expiry/html_and_json_timestamp.yaml +$ ansible-playbook -v -i HOSTS playbooks/byo/openshift-checks/certificate_expiry/html_and_json_timestamp.yaml ``` **From openshift-ansible-playbooks rpm:** ``` -$ ansible-playbook -v -i HOSTS /usr/share/ansible/openshift-ansible/playbooks/certificate_expiry/html_and_json_timestamp.yaml +$ ansible-playbook -v -i HOSTS /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-checks/certificate_expiry/html_and_json_timestamp.yaml ``` -> [View This Playbook](../../playbooks/certificate_expiry/html_and_json_timestamp.yaml) +> [View This Playbook](../../playbooks/byo/openshift-checks/certificate_expiry/html_and_json_timestamp.yaml) ### Long warning window @@ -278,14 +278,14 @@ the module out): **From git:** ``` -$ ansible-playbook -v -i HOSTS playbooks/certificate_expiry/longer_warning_period.yaml +$ ansible-playbook -v -i HOSTS playbooks/byo/openshift-checks/certificate_expiry/longer_warning_period.yaml ``` **From openshift-ansible-playbooks rpm:** ``` -$ ansible-playbook -v -i HOSTS /usr/share/ansible/openshift-ansible/playbooks/certificate_expiry/longer_warning_period.yaml +$ ansible-playbook -v -i HOSTS /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-checks/certificate_expiry/longer_warning_period.yaml ``` -> [View This Playbook](../../playbooks/certificate_expiry/longer_warning_period.yaml) +> [View This Playbook](../../playbooks/byo/openshift-checks/certificate_expiry/longer_warning_period.yaml) ### Long warning window and JSON report @@ -307,14 +307,14 @@ the module out) and save the results as a JSON file: **From git:** ``` -$ ansible-playbook -v -i HOSTS playbooks/certificate_expiry/longer-warning-period-json-results.yaml +$ ansible-playbook -v -i HOSTS playbooks/byo/openshift-checks/certificate_expiry/longer-warning-period-json-results.yaml ``` **From openshift-ansible-playbooks rpm:** ``` -$ ansible-playbook -v -i HOSTS /usr/share/ansible/openshift-ansible/playbooks/certificate_expiry/longer-warning-period-json-results.yaml +$ ansible-playbook -v -i HOSTS /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-checks/certificate_expiry/longer-warning-period-json-results.yaml ``` -> [View This Playbook](../../playbooks/certificate_expiry/longer-warning-period-json-results.yaml) +> [View This Playbook](../../playbooks/byo/openshift-checks/certificate_expiry/longer-warning-period-json-results.yaml) diff --git a/roles/openshift_facts/tasks/main.yml b/roles/openshift_facts/tasks/main.yml index f657d86cf..1b9bda67e 100644 --- a/roles/openshift_facts/tasks/main.yml +++ b/roles/openshift_facts/tasks/main.yml @@ -15,6 +15,9 @@ l_is_etcd_system_container: "{{ (use_etcd_system_container | default(use_system_containers) | bool) }}" - set_fact: l_any_system_container: "{{ l_is_etcd_system_container or l_is_openvswitch_system_container or l_is_node_system_container or l_is_master_system_container }}" +- set_fact: + l_etcd_runtime: "{{ 'runc' if l_is_etcd_system_container else 'docker' if l_is_containerized else 'host' }}" + - name: Validate python version fail: @@ -80,6 +83,7 @@ is_node_system_container: "{{ l_is_node_system_container | default(false) }}" is_master_system_container: "{{ l_is_master_system_container | default(false) }}" is_etcd_system_container: "{{ l_is_etcd_system_container | default(false) }}" + etcd_runtime: "{{ l_etcd_runtime }}" system_images_registry: "{{ system_images_registry | default('') }}" public_hostname: "{{ openshift_public_hostname | default(None) }}" public_ip: "{{ openshift_public_ip | default(None) }}" diff --git a/roles/openshift_health_checker/library/etcdkeysize.py b/roles/openshift_health_checker/library/etcdkeysize.py new file mode 100644 index 000000000..620e82d87 --- /dev/null +++ b/roles/openshift_health_checker/library/etcdkeysize.py @@ -0,0 +1,122 @@ +#!/usr/bin/python +"""Ansible module that recursively determines if the size of a key in an etcd cluster exceeds a given limit.""" + +from ansible.module_utils.basic import AnsibleModule + + +try: + import etcd + + IMPORT_EXCEPTION_MSG = None +except ImportError as err: + IMPORT_EXCEPTION_MSG = str(err) + + from collections import namedtuple + EtcdMock = namedtuple("etcd", ["EtcdKeyNotFound"]) + etcd = EtcdMock(KeyError) + + +# pylint: disable=too-many-arguments +def check_etcd_key_size(client, key, size_limit, total_size=0, depth=0, depth_limit=1000, visited=None): + """Check size of an etcd path starting at given key. Returns tuple (string, bool)""" + if visited is None: + visited = set() + + if key in visited: + return 0, False + + visited.add(key) + + try: + result = client.read(key, recursive=False) + except etcd.EtcdKeyNotFound: + return 0, False + + size = 0 + limit_exceeded = False + + for node in result.leaves: + if depth >= depth_limit: + raise Exception("Maximum recursive stack depth ({}) exceeded.".format(depth_limit)) + + if size_limit and total_size + size > size_limit: + return size, True + + if not node.dir: + size += len(node.value) + continue + + key_size, limit_exceeded = check_etcd_key_size(client, node.key, + size_limit, + total_size + size, + depth + 1, + depth_limit, visited) + size += key_size + + max_limit_exceeded = limit_exceeded or (total_size + size > size_limit) + return size, max_limit_exceeded + + +def main(): # pylint: disable=missing-docstring,too-many-branches + module = AnsibleModule( + argument_spec=dict( + size_limit_bytes=dict(type="int", default=0), + paths=dict(type="list", default=["/openshift.io/images"]), + host=dict(type="str", default="127.0.0.1"), + port=dict(type="int", default=4001), + protocol=dict(type="str", default="http"), + version_prefix=dict(type="str", default=""), + allow_redirect=dict(type="bool", default=False), + cert=dict(type="dict", default=""), + ca_cert=dict(type="str", default=None), + ), + supports_check_mode=True + ) + + module.params["cert"] = ( + module.params["cert"]["cert"], + module.params["cert"]["key"], + ) + + size_limit = module.params.pop("size_limit_bytes") + paths = module.params.pop("paths") + + limit_exceeded = False + + try: + # pylint: disable=no-member + client = etcd.Client(**module.params) + except AttributeError as attrerr: + msg = str(attrerr) + if IMPORT_EXCEPTION_MSG: + msg = IMPORT_EXCEPTION_MSG + if "No module named etcd" in IMPORT_EXCEPTION_MSG: + # pylint: disable=redefined-variable-type + msg = ('Unable to import the python "etcd" dependency. ' + 'Make sure python-etcd is installed on the host.') + + module.exit_json( + failed=True, + changed=False, + size_limit_exceeded=limit_exceeded, + msg=msg, + ) + + return + + size = 0 + for path in paths: + path_size, limit_exceeded = check_etcd_key_size(client, path, size_limit - size) + size += path_size + + if limit_exceeded: + break + + module.exit_json( + changed=False, + size_limit_exceeded=limit_exceeded, + ) + + +if __name__ == '__main__': + main() diff --git a/roles/openshift_health_checker/meta/main.yml b/roles/openshift_health_checker/meta/main.yml index cd9b55902..4d141974c 100644 --- a/roles/openshift_health_checker/meta/main.yml +++ b/roles/openshift_health_checker/meta/main.yml @@ -2,3 +2,4 @@ dependencies: - role: openshift_facts - role: openshift_repos + - role: openshift_version diff --git a/roles/openshift_health_checker/openshift_checks/docker_image_availability.py b/roles/openshift_health_checker/openshift_checks/docker_image_availability.py index cce289b95..4588ed634 100644 --- a/roles/openshift_health_checker/openshift_checks/docker_image_availability.py +++ b/roles/openshift_health_checker/openshift_checks/docker_image_availability.py @@ -13,41 +13,55 @@ class DockerImageAvailability(OpenShiftCheck): name = "docker_image_availability" tags = ["preflight"] - skopeo_image = "openshift/openshift-ansible" + dependencies = ["skopeo", "python-docker-py"] - # FIXME(juanvallejo): we should consider other possible values of - # `deployment_type` (the key here). See - # https://github.com/openshift/openshift-ansible/blob/8e26f8c/roles/openshift_repos/vars/main.yml#L7 - docker_image_base = { + deployment_image_info = { "origin": { - "repo": "openshift", - "image": "origin", + "namespace": "openshift", + "name": "origin", }, "openshift-enterprise": { - "repo": "openshift3", - "image": "ose", + "namespace": "openshift3", + "name": "ose", }, } - def run(self, tmp, task_vars): - required_images = self.required_images(task_vars) - missing_images = set(required_images) - set(self.local_images(required_images, task_vars)) + @classmethod + def is_active(cls, task_vars): + """Skip hosts with unsupported deployment types.""" + deployment_type = get_var(task_vars, "openshift_deployment_type") + has_valid_deployment_type = deployment_type in cls.deployment_image_info - # exit early if all images were found locally - if not missing_images: - return {"changed": False} + return super(DockerImageAvailability, cls).is_active(task_vars) and has_valid_deployment_type - msg, failed, changed = self.update_skopeo_image(task_vars) + def run(self, tmp, task_vars): + msg, failed, changed = self.ensure_dependencies(task_vars) # exit early if Skopeo update fails if failed: + if "No package matching" in msg: + msg = "Ensure that all required dependencies can be installed via `yum`.\n" return { "failed": True, "changed": changed, - "msg": "Failed to update Skopeo image ({img_name}). {msg}".format(img_name=self.skopeo_image, msg=msg), + "msg": ( + "Unable to update or install required dependency packages on this host;\n" + "These are required in order to check Docker image availability:" + "\n {deps}\n{msg}" + ).format(deps=',\n '.join(self.dependencies), msg=msg), } + required_images = self.required_images(task_vars) + missing_images = set(required_images) - set(self.local_images(required_images, task_vars)) + + # exit early if all images were found locally + if not missing_images: + return {"changed": changed} + registries = self.known_docker_registries(task_vars) + if not registries: + return {"failed": True, "msg": "Unable to retrieve any docker registries.", "changed": changed} + available_images = self.available_images(missing_images, registries, task_vars) unavailable_images = set(missing_images) - set(available_images) @@ -55,44 +69,60 @@ class DockerImageAvailability(OpenShiftCheck): return { "failed": True, "msg": ( - "One or more required images are not available: {}.\n" + "One or more required Docker images are not available:\n {}\n" "Configured registries: {}" - ).format(", ".join(sorted(unavailable_images)), ", ".join(registries)), + ).format(",\n ".join(sorted(unavailable_images)), ", ".join(registries)), "changed": changed, } return {"changed": changed} def required_images(self, task_vars): - deployment_type = get_var(task_vars, "deployment_type") - # FIXME(juanvallejo): we should handle gracefully with a proper error - # message when given an unexpected value for `deployment_type`. - image_base_name = self.docker_image_base[deployment_type] - - openshift_release = get_var(task_vars, "openshift_release") - # FIXME(juanvallejo): this variable is not required when the - # installation is non-containerized. The example inventories have it - # commented out. We should handle gracefully and with a proper error - # message when this variable is required and not set. - openshift_image_tag = get_var(task_vars, "openshift_image_tag") + deployment_type = get_var(task_vars, "openshift_deployment_type") + image_info = self.deployment_image_info[deployment_type] + openshift_release = get_var(task_vars, "openshift_release", default="latest") + openshift_image_tag = get_var(task_vars, "openshift_image_tag") is_containerized = get_var(task_vars, "openshift", "common", "is_containerized") - if is_containerized: - images = set(self.containerized_docker_images(image_base_name, openshift_release)) - else: - images = set(self.rpm_docker_images(image_base_name, openshift_release)) + images = set(self.required_docker_images( + image_info["namespace"], + image_info["name"], + ["registry-console"] if "enterprise" in deployment_type else [], # include enterprise-only image names + openshift_release, + is_containerized, + )) # append images with qualified image tags to our list of required images. # these are images with a (v0.0.0.0) tag, rather than a standard release # format tag (v0.0). We want to check this set in both containerized and # non-containerized installations. images.update( - self.qualified_docker_images(self.image_from_base_name(image_base_name), "v" + openshift_image_tag) + self.required_qualified_docker_images( + image_info["namespace"], + image_info["name"], + openshift_image_tag, + ), ) return images + @staticmethod + def required_docker_images(namespace, name, additional_image_names, version, is_containerized): + if is_containerized: + return ["{}/{}:{}".format(namespace, name, version)] if name else [] + + # include additional non-containerized images specific to the current deployment type + return ["{}/{}:{}".format(namespace, img_name, version) for img_name in additional_image_names] + + @staticmethod + def required_qualified_docker_images(namespace, name, version): + # pylint: disable=invalid-name + return [ + "{}/{}-{}:{}".format(namespace, name, suffix, version) + for suffix in ["haproxy-router", "docker-registry", "deployer", "pod"] + ] + def local_images(self, images, task_vars): """Filter a list of images and return those available locally.""" return [ @@ -107,31 +137,26 @@ class DockerImageAvailability(OpenShiftCheck): return bool(result.get("images", [])) - def known_docker_registries(self, task_vars): - result = self.module_executor("docker_info", {}, task_vars) + @staticmethod + def known_docker_registries(task_vars): + docker_facts = get_var(task_vars, "openshift", "docker") + regs = set(docker_facts["additional_registries"]) - if result.get("failed", False): - return [] + deployment_type = get_var(task_vars, "openshift_deployment_type") + if deployment_type == "origin": + regs.update(["docker.io"]) + elif "enterprise" in deployment_type: + regs.update(["registry.access.redhat.com"]) - # FIXME(juanvallejo): wrong default type, result["info"] is expected to - # contain a dictionary (see how we call `docker_info.get` below). - docker_info = result.get("info", "") - return [registry.get("Name", "") for registry in docker_info.get("Registries", {})] + return list(regs) def available_images(self, images, registries, task_vars): """Inspect existing images using Skopeo and return all images successfully inspected.""" return [ image for image in images - if self.is_image_available(image, registries, task_vars) + if any(self.is_available_skopeo_image(image, registry, task_vars) for registry in registries) ] - def is_image_available(self, image, registries, task_vars): - for registry in registries: - if self.is_available_skopeo_image(image, registry, task_vars): - return True - - return False - def is_available_skopeo_image(self, image, registry, task_vars): """Uses Skopeo to determine if required image exists in a given registry.""" @@ -140,40 +165,15 @@ class DockerImageAvailability(OpenShiftCheck): image=image, ) - args = { - "name": "skopeo_inspect", - "image": self.skopeo_image, - "command": cmd_str, - "detach": False, - "cleanup": True, - } - result = self.module_executor("docker_container", args, task_vars) - return result.get("failed", False) - - def containerized_docker_images(self, base_name, version): - return [ - "{image}:{version}".format(image=self.image_from_base_name(base_name), version=version) - ] + args = {"_raw_params": cmd_str} + result = self.module_executor("command", args, task_vars) + return not result.get("failed", False) and result.get("rc", 0) == 0 - @staticmethod - def rpm_docker_images(base, version): - return [ - "{image_repo}/registry-console:{version}".format(image_repo=base["repo"], version=version) - ] + # ensures that the skopeo and python-docker-py packages exist + # check is skipped on atomic installations + def ensure_dependencies(self, task_vars): + if get_var(task_vars, "openshift", "common", "is_atomic"): + return "", False, False - @staticmethod - def qualified_docker_images(image_name, version): - return [ - "{}-{}:{}".format(image_name, component, version) - for component in "haproxy-router docker-registry deployer pod".split() - ] - - @staticmethod - def image_from_base_name(base): - return "".join([base["repo"], "/", base["image"]]) - - # ensures that the skopeo docker image exists, and updates it - # with latest if image was already present locally. - def update_skopeo_image(self, task_vars): - result = self.module_executor("docker_image", {"name": self.skopeo_image}, task_vars) - return result.get("msg", ""), result.get("failed", False), result.get("changed", False) + result = self.module_executor("yum", {"name": self.dependencies, "state": "latest"}, task_vars) + return result.get("msg", ""), result.get("failed", False) or result.get("rc", 0) != 0, result.get("changed") diff --git a/roles/openshift_health_checker/openshift_checks/etcd_imagedata_size.py b/roles/openshift_health_checker/openshift_checks/etcd_imagedata_size.py new file mode 100644 index 000000000..c04a69765 --- /dev/null +++ b/roles/openshift_health_checker/openshift_checks/etcd_imagedata_size.py @@ -0,0 +1,84 @@ +""" +Ansible module for determining if the size of OpenShift image data exceeds a specified limit in an etcd cluster. +""" + +from openshift_checks import OpenShiftCheck, OpenShiftCheckException, get_var + + +class EtcdImageDataSize(OpenShiftCheck): + """Check that total size of OpenShift image data does not exceed the recommended limit in an etcd cluster""" + + name = "etcd_imagedata_size" + tags = ["etcd"] + + def run(self, tmp, task_vars): + etcd_mountpath = self._get_etcd_mountpath(get_var(task_vars, "ansible_mounts")) + etcd_avail_diskspace = etcd_mountpath["size_available"] + etcd_total_diskspace = etcd_mountpath["size_total"] + + etcd_imagedata_size_limit = get_var(task_vars, + "etcd_max_image_data_size_bytes", + default=int(0.5 * float(etcd_total_diskspace - etcd_avail_diskspace))) + + etcd_is_ssl = get_var(task_vars, "openshift", "master", "etcd_use_ssl", default=False) + etcd_port = get_var(task_vars, "openshift", "master", "etcd_port", default=2379) + etcd_hosts = get_var(task_vars, "openshift", "master", "etcd_hosts") + + config_base = get_var(task_vars, "openshift", "common", "config_base") + + cert = task_vars.get("etcd_client_cert", config_base + "/master/master.etcd-client.crt") + key = task_vars.get("etcd_client_key", config_base + "/master/master.etcd-client.key") + ca_cert = task_vars.get("etcd_client_ca_cert", config_base + "/master/master.etcd-ca.crt") + + for etcd_host in list(etcd_hosts): + args = { + "size_limit_bytes": etcd_imagedata_size_limit, + "paths": ["/openshift.io/images", "/openshift.io/imagestreams"], + "host": etcd_host, + "port": etcd_port, + "protocol": "https" if etcd_is_ssl else "http", + "version_prefix": "/v2", + "allow_redirect": True, + "ca_cert": ca_cert, + "cert": { + "cert": cert, + "key": key, + }, + } + + etcdkeysize = self.module_executor("etcdkeysize", args, task_vars) + + if etcdkeysize.get("rc", 0) != 0 or etcdkeysize.get("failed"): + msg = 'Failed to retrieve stats for etcd host "{host}": {reason}' + reason = etcdkeysize.get("msg") + if etcdkeysize.get("module_stderr"): + reason = etcdkeysize["module_stderr"] + + msg = msg.format(host=etcd_host, reason=reason) + return {"failed": True, "changed": False, "msg": msg} + + if etcdkeysize["size_limit_exceeded"]: + limit = self._to_gigabytes(etcd_imagedata_size_limit) + msg = ("The size of OpenShift image data stored in etcd host " + "\"{host}\" exceeds the maximum recommended limit of {limit:.2f} GB. " + "Use the `oadm prune images` command to cleanup unused Docker images.") + return {"failed": True, "msg": msg.format(host=etcd_host, limit=limit)} + + return {"changed": False} + + @staticmethod + def _get_etcd_mountpath(ansible_mounts): + valid_etcd_mount_paths = ["/var/lib/etcd", "/var/lib", "/var", "/"] + + mount_for_path = {mnt.get("mount"): mnt for mnt in ansible_mounts} + for path in valid_etcd_mount_paths: + if path in mount_for_path: + return mount_for_path[path] + + paths = ', '.join(sorted(mount_for_path)) or 'none' + msg = "Unable to determine a valid etcd mountpath. Paths mounted: {}.".format(paths) + raise OpenShiftCheckException(msg) + + @staticmethod + def _to_gigabytes(byte_size): + return float(byte_size) / 10.0**9 diff --git a/roles/openshift_health_checker/openshift_checks/etcd_volume.py b/roles/openshift_health_checker/openshift_checks/etcd_volume.py new file mode 100644 index 000000000..7452c9cc1 --- /dev/null +++ b/roles/openshift_health_checker/openshift_checks/etcd_volume.py @@ -0,0 +1,58 @@ +"""A health check for OpenShift clusters.""" + +from openshift_checks import OpenShiftCheck, OpenShiftCheckException, get_var + + +class EtcdVolume(OpenShiftCheck): + """Ensures etcd storage usage does not exceed a given threshold.""" + + name = "etcd_volume" + tags = ["etcd", "health"] + + # Default device usage threshold. Value should be in the range [0, 100]. + default_threshold_percent = 90 + # Where to find ectd data, higher priority first. + supported_mount_paths = ["/var/lib/etcd", "/var/lib", "/var", "/"] + + @classmethod + def is_active(cls, task_vars): + etcd_hosts = get_var(task_vars, "groups", "etcd", default=[]) or get_var(task_vars, "groups", "masters", + default=[]) or [] + is_etcd_host = get_var(task_vars, "ansible_ssh_host") in etcd_hosts + return super(EtcdVolume, cls).is_active(task_vars) and is_etcd_host + + def run(self, tmp, task_vars): + mount_info = self._etcd_mount_info(task_vars) + available = mount_info["size_available"] + total = mount_info["size_total"] + used = total - available + + threshold = get_var( + task_vars, + "etcd_device_usage_threshold_percent", + default=self.default_threshold_percent + ) + + used_percent = 100.0 * used / total + + if used_percent > threshold: + device = mount_info.get("device", "unknown") + mount = mount_info.get("mount", "unknown") + msg = "etcd storage usage ({:.1f}%) is above threshold ({:.1f}%). Device: {}, mount: {}.".format( + used_percent, threshold, device, mount + ) + return {"failed": True, "msg": msg} + + return {"changed": False} + + def _etcd_mount_info(self, task_vars): + ansible_mounts = get_var(task_vars, "ansible_mounts") + mounts = {mnt.get("mount"): mnt for mnt in ansible_mounts} + + for path in self.supported_mount_paths: + if path in mounts: + return mounts[path] + + paths = ', '.join(sorted(mounts)) or 'none' + msg = "Unable to find etcd storage mount point. Paths mounted: {}.".format(paths) + raise OpenShiftCheckException(msg) diff --git a/roles/openshift_health_checker/test/docker_image_availability_test.py b/roles/openshift_health_checker/test/docker_image_availability_test.py index 2a9c32f77..0379cafb5 100644 --- a/roles/openshift_health_checker/test/docker_image_availability_test.py +++ b/roles/openshift_health_checker/test/docker_image_availability_test.py @@ -3,26 +3,176 @@ import pytest from openshift_checks.docker_image_availability import DockerImageAvailability -@pytest.mark.xfail(strict=True) # TODO: remove this once this test is fully implemented. -@pytest.mark.parametrize('task_vars,expected_result', [ - ( - dict( - openshift=dict(common=dict( +@pytest.mark.parametrize('deployment_type,is_active', [ + ("origin", True), + ("openshift-enterprise", True), + ("enterprise", False), + ("online", False), + ("invalid", False), + ("", False), +]) +def test_is_active(deployment_type, is_active): + task_vars = dict( + openshift_deployment_type=deployment_type, + ) + assert DockerImageAvailability.is_active(task_vars=task_vars) == is_active + + +@pytest.mark.parametrize("is_containerized,is_atomic", [ + (True, True), + (False, False), + (True, False), + (False, True), +]) +def test_all_images_available_locally(is_containerized, is_atomic): + def execute_module(module_name, args, task_vars): + if module_name == "yum": + return {"changed": True} + + assert module_name == "docker_image_facts" + assert 'name' in args + assert args['name'] + return { + 'images': [args['name']], + } + + result = DockerImageAvailability(execute_module=execute_module).run(tmp=None, task_vars=dict( + openshift=dict( + common=dict( + service_type='origin', + is_containerized=is_containerized, + is_atomic=is_atomic, + ), + docker=dict(additional_registries=["docker.io"]), + ), + openshift_deployment_type='origin', + openshift_release='v3.4', + openshift_image_tag='3.4', + )) + + assert not result.get('failed', False) + + +@pytest.mark.parametrize("available_locally", [ + False, + True, +]) +def test_all_images_available_remotely(available_locally): + def execute_module(module_name, args, task_vars): + if module_name == 'docker_image_facts': + return {'images': [], 'failed': available_locally} + return {'changed': False} + + result = DockerImageAvailability(execute_module=execute_module).run(tmp=None, task_vars=dict( + openshift=dict( + common=dict( service_type='origin', is_containerized=False, - )), - openshift_release='v3.5', - deployment_type='origin', - openshift_image_tag='', # FIXME: should not be required + is_atomic=False, + ), + docker=dict(additional_registries=["docker.io", "registry.access.redhat.com"]), ), - {'changed': False}, + openshift_deployment_type='origin', + openshift_release='3.4', + openshift_image_tag='v3.4', + )) + + assert not result.get('failed', False) + + +def test_all_images_unavailable(): + def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): + if module_name == "command": + return { + 'failed': True, + } + + return { + 'changed': False, + } + + check = DockerImageAvailability(execute_module=execute_module) + actual = check.run(tmp=None, task_vars=dict( + openshift=dict( + common=dict( + service_type='origin', + is_containerized=False, + is_atomic=False, + ), + docker=dict(additional_registries=["docker.io"]), + ), + openshift_deployment_type="openshift-enterprise", + openshift_release=None, + openshift_image_tag='latest' + )) + + assert actual['failed'] + assert "required Docker images are not available" in actual['msg'] + + +@pytest.mark.parametrize("message,extra_words", [ + ( + "docker image update failure", + ["docker image update failure"], + ), + ( + "No package matching 'skopeo' found available, installed or updated", + ["dependencies can be installed via `yum`"] ), - # TODO: add more parameters here to test the multiple possible inputs that affect behavior. ]) -def test_docker_image_availability(task_vars, expected_result): +def test_skopeo_update_failure(message, extra_words): def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): - return {'info': {}} # TODO: this will vary depending on input parameters. + if module_name == "yum": + return { + "failed": True, + "msg": message, + "changed": False, + } - check = DockerImageAvailability(execute_module=execute_module) - result = check.run(tmp=None, task_vars=task_vars) - assert result == expected_result + return {'changed': False} + + actual = DockerImageAvailability(execute_module=execute_module).run(tmp=None, task_vars=dict( + openshift=dict( + common=dict( + service_type='origin', + is_containerized=False, + is_atomic=False, + ), + docker=dict(additional_registries=["unknown.io"]), + ), + openshift_deployment_type="openshift-enterprise", + openshift_release='', + openshift_image_tag='', + )) + + assert actual["failed"] + for word in extra_words: + assert word in actual["msg"] + + +@pytest.mark.parametrize("deployment_type,registries", [ + ("origin", ["unknown.io"]), + ("openshift-enterprise", ["registry.access.redhat.com"]), + ("openshift-enterprise", []), +]) +def test_registry_availability(deployment_type, registries): + def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): + return { + 'changed': False, + } + + actual = DockerImageAvailability(execute_module=execute_module).run(tmp=None, task_vars=dict( + openshift=dict( + common=dict( + service_type='origin', + is_containerized=False, + is_atomic=False, + ), + docker=dict(additional_registries=registries), + ), + openshift_deployment_type=deployment_type, + openshift_release='', + openshift_image_tag='', + )) + + assert not actual.get("failed", False) diff --git a/roles/openshift_health_checker/test/etcd_imagedata_size_test.py b/roles/openshift_health_checker/test/etcd_imagedata_size_test.py new file mode 100644 index 000000000..df9d52d41 --- /dev/null +++ b/roles/openshift_health_checker/test/etcd_imagedata_size_test.py @@ -0,0 +1,328 @@ +import pytest + +from collections import namedtuple +from openshift_checks.etcd_imagedata_size import EtcdImageDataSize, OpenShiftCheckException +from etcdkeysize import check_etcd_key_size + + +def fake_etcd_client(root): + fake_nodes = dict() + fake_etcd_node(root, fake_nodes) + + clientclass = namedtuple("client", ["read"]) + return clientclass(lambda key, recursive: fake_etcd_result(fake_nodes[key])) + + +def fake_etcd_result(fake_node): + resultclass = namedtuple("result", ["leaves"]) + if not fake_node.dir: + return resultclass([fake_node]) + + return resultclass(fake_node.leaves) + + +def fake_etcd_node(node, visited): + min_req_fields = ["dir", "key"] + fields = list(node) + leaves = [] + + if node["dir"] and node.get("leaves"): + for leaf in node["leaves"]: + leaves.append(fake_etcd_node(leaf, visited)) + + if len(set(min_req_fields) - set(fields)) > 0: + raise ValueError("fake etcd nodes require at least {} fields.".format(min_req_fields)) + + if node.get("leaves"): + node["leaves"] = leaves + + nodeclass = namedtuple("node", fields) + nodeinst = nodeclass(**node) + visited[nodeinst.key] = nodeinst + + return nodeinst + + +@pytest.mark.parametrize('ansible_mounts,extra_words', [ + ([], ['none']), # empty ansible_mounts + ([{'mount': '/mnt'}], ['/mnt']), # missing relevant mount paths +]) +def test_cannot_determine_available_mountpath(ansible_mounts, extra_words): + task_vars = dict( + ansible_mounts=ansible_mounts, + ) + check = EtcdImageDataSize(execute_module=fake_execute_module) + + with pytest.raises(OpenShiftCheckException) as excinfo: + check.run(tmp=None, task_vars=task_vars) + + for word in 'determine valid etcd mountpath'.split() + extra_words: + assert word in str(excinfo.value) + + +@pytest.mark.parametrize('ansible_mounts,tree,size_limit,should_fail,extra_words', [ + ( + # test that default image size limit evals to 1/2 * (total size in use) + [{ + 'mount': '/', + 'size_available': 40 * 10**9, + 'size_total': 80 * 10**9, + }], + {"dir": False, "key": "/", "value": "1234"}, + None, + False, + [], + ), + ( + [{ + 'mount': '/', + 'size_available': 40 * 10**9, + 'size_total': 48 * 10**9, + }], + {"dir": False, "key": "/", "value": "1234"}, + None, + False, + [], + ), + ( + # set max size limit for image data to be below total node value + # total node value is defined as the sum of the value field + # from every node + [{ + 'mount': '/', + 'size_available': 40 * 10**9, + 'size_total': 48 * 10**9, + }], + {"dir": False, "key": "/", "value": "12345678"}, + 7, + True, + ["exceeds the maximum recommended limit", "0.00 GB"], + ), + ( + [{ + 'mount': '/', + 'size_available': 48 * 10**9 - 1, + 'size_total': 48 * 10**9, + }], + {"dir": False, "key": "/", "value": "1234"}, + None, + True, + ["exceeds the maximum recommended limit", "0.00 GB"], + ) +]) +def test_check_etcd_key_size_calculates_correct_limit(ansible_mounts, tree, size_limit, should_fail, extra_words): + def execute_module(module_name, args, tmp=None, task_vars=None): + if module_name != "etcdkeysize": + return { + "changed": False, + } + + client = fake_etcd_client(tree) + s, limit_exceeded = check_etcd_key_size(client, tree["key"], args["size_limit_bytes"]) + + return {"size_limit_exceeded": limit_exceeded} + + task_vars = dict( + etcd_max_image_data_size_bytes=size_limit, + ansible_mounts=ansible_mounts, + openshift=dict( + master=dict(etcd_hosts=["localhost"]), + common=dict(config_base="/var/lib/origin") + ) + ) + if size_limit is None: + task_vars.pop("etcd_max_image_data_size_bytes") + + check = EtcdImageDataSize(execute_module=execute_module).run(tmp=None, task_vars=task_vars) + + if should_fail: + assert check["failed"] + + for word in extra_words: + assert word in check["msg"] + else: + assert not check.get("failed", False) + + +@pytest.mark.parametrize('ansible_mounts,tree,root_path,expected_size,extra_words', [ + ( + [{ + 'mount': '/', + 'size_available': 40 * 10**9, + 'size_total': 80 * 10**9, + }], + # test recursive size check on tree with height > 1 + { + "dir": True, + "key": "/", + "leaves": [ + {"dir": False, "key": "/foo1", "value": "1234"}, + {"dir": False, "key": "/foo2", "value": "1234"}, + {"dir": False, "key": "/foo3", "value": "1234"}, + {"dir": False, "key": "/foo4", "value": "1234"}, + { + "dir": True, + "key": "/foo5", + "leaves": [ + {"dir": False, "key": "/foo/bar1", "value": "56789"}, + {"dir": False, "key": "/foo/bar2", "value": "56789"}, + {"dir": False, "key": "/foo/bar3", "value": "56789"}, + { + "dir": True, + "key": "/foo/bar4", + "leaves": [ + {"dir": False, "key": "/foo/bar/baz1", "value": "123"}, + {"dir": False, "key": "/foo/bar/baz2", "value": "123"}, + ] + }, + ] + }, + ] + }, + "/", + 37, + [], + ), + ( + [{ + 'mount': '/', + 'size_available': 40 * 10**9, + 'size_total': 80 * 10**9, + }], + # test correct sub-tree size calculation + { + "dir": True, + "key": "/", + "leaves": [ + {"dir": False, "key": "/foo1", "value": "1234"}, + {"dir": False, "key": "/foo2", "value": "1234"}, + {"dir": False, "key": "/foo3", "value": "1234"}, + {"dir": False, "key": "/foo4", "value": "1234"}, + { + "dir": True, + "key": "/foo5", + "leaves": [ + {"dir": False, "key": "/foo/bar1", "value": "56789"}, + {"dir": False, "key": "/foo/bar2", "value": "56789"}, + {"dir": False, "key": "/foo/bar3", "value": "56789"}, + { + "dir": True, + "key": "/foo/bar4", + "leaves": [ + {"dir": False, "key": "/foo/bar/baz1", "value": "123"}, + {"dir": False, "key": "/foo/bar/baz2", "value": "123"}, + ] + }, + ] + }, + ] + }, + "/foo5", + 21, + [], + ), + ( + [{ + 'mount': '/', + 'size_available': 40 * 10**9, + 'size_total': 80 * 10**9, + }], + # test that a non-existing key is handled correctly + { + "dir": False, + "key": "/", + "value": "1234", + }, + "/missing", + 0, + [], + ), + ( + [{ + 'mount': '/', + 'size_available': 40 * 10**9, + 'size_total': 80 * 10**9, + }], + # test etcd cycle handling + { + "dir": True, + "key": "/", + "leaves": [ + {"dir": False, "key": "/foo1", "value": "1234"}, + {"dir": False, "key": "/foo2", "value": "1234"}, + {"dir": False, "key": "/foo3", "value": "1234"}, + {"dir": False, "key": "/foo4", "value": "1234"}, + { + "dir": True, + "key": "/", + "leaves": [ + {"dir": False, "key": "/foo1", "value": "1"}, + ], + }, + ] + }, + "/", + 16, + [], + ), +]) +def test_etcd_key_size_check_calculates_correct_size(ansible_mounts, tree, root_path, expected_size, extra_words): + def execute_module(module_name, args, tmp=None, task_vars=None): + if module_name != "etcdkeysize": + return { + "changed": False, + } + + client = fake_etcd_client(tree) + size, limit_exceeded = check_etcd_key_size(client, root_path, args["size_limit_bytes"]) + + assert size == expected_size + return { + "size_limit_exceeded": limit_exceeded, + } + + task_vars = dict( + ansible_mounts=ansible_mounts, + openshift=dict( + master=dict(etcd_hosts=["localhost"]), + common=dict(config_base="/var/lib/origin") + ) + ) + + check = EtcdImageDataSize(execute_module=execute_module).run(tmp=None, task_vars=task_vars) + assert not check.get("failed", False) + + +def test_etcdkeysize_module_failure(): + def execute_module(module_name, tmp=None, task_vars=None): + if module_name != "etcdkeysize": + return { + "changed": False, + } + + return { + "rc": 1, + "module_stderr": "failure", + } + + task_vars = dict( + ansible_mounts=[{ + 'mount': '/', + 'size_available': 40 * 10**9, + 'size_total': 80 * 10**9, + }], + openshift=dict( + master=dict(etcd_hosts=["localhost"]), + common=dict(config_base="/var/lib/origin") + ) + ) + + check = EtcdImageDataSize(execute_module=execute_module).run(tmp=None, task_vars=task_vars) + + assert check["failed"] + for word in "Failed to retrieve stats": + assert word in check["msg"] + + +def fake_execute_module(*args): + raise AssertionError('this function should not be called') diff --git a/roles/openshift_health_checker/test/etcd_volume_test.py b/roles/openshift_health_checker/test/etcd_volume_test.py new file mode 100644 index 000000000..917045526 --- /dev/null +++ b/roles/openshift_health_checker/test/etcd_volume_test.py @@ -0,0 +1,149 @@ +import pytest + +from openshift_checks.etcd_volume import EtcdVolume, OpenShiftCheckException + + +@pytest.mark.parametrize('ansible_mounts,extra_words', [ + ([], ['none']), # empty ansible_mounts + ([{'mount': '/mnt'}], ['/mnt']), # missing relevant mount paths +]) +def test_cannot_determine_available_disk(ansible_mounts, extra_words): + task_vars = dict( + ansible_mounts=ansible_mounts, + ) + check = EtcdVolume(execute_module=fake_execute_module) + + with pytest.raises(OpenShiftCheckException) as excinfo: + check.run(tmp=None, task_vars=task_vars) + + for word in 'Unable to find etcd storage mount point'.split() + extra_words: + assert word in str(excinfo.value) + + +@pytest.mark.parametrize('size_limit,ansible_mounts', [ + ( + # if no size limit is specified, expect max usage + # limit to default to 90% of size_total + None, + [{ + 'mount': '/', + 'size_available': 40 * 10**9, + 'size_total': 80 * 10**9 + }], + ), + ( + 1, + [{ + 'mount': '/', + 'size_available': 30 * 10**9, + 'size_total': 30 * 10**9, + }], + ), + ( + 20000000000, + [{ + 'mount': '/', + 'size_available': 20 * 10**9, + 'size_total': 40 * 10**9, + }], + ), + ( + 5000000000, + [{ + # not enough space on / ... + 'mount': '/', + 'size_available': 0, + 'size_total': 0, + }, { + # not enough space on /var/lib ... + 'mount': '/var/lib', + 'size_available': 2 * 10**9, + 'size_total': 21 * 10**9, + }, { + # ... but enough on /var/lib/etcd + 'mount': '/var/lib/etcd', + 'size_available': 36 * 10**9, + 'size_total': 40 * 10**9 + }], + ) +]) +def test_succeeds_with_recommended_disk_space(size_limit, ansible_mounts): + task_vars = dict( + etcd_device_usage_threshold_percent=size_limit, + ansible_mounts=ansible_mounts, + ) + + if task_vars["etcd_device_usage_threshold_percent"] is None: + task_vars.pop("etcd_device_usage_threshold_percent") + + check = EtcdVolume(execute_module=fake_execute_module) + result = check.run(tmp=None, task_vars=task_vars) + + assert not result.get('failed', False) + + +@pytest.mark.parametrize('size_limit_percent,ansible_mounts,extra_words', [ + ( + # if no size limit is specified, expect max usage + # limit to default to 90% of size_total + None, + [{ + 'mount': '/', + 'size_available': 1 * 10**9, + 'size_total': 100 * 10**9, + }], + ['99.0%'], + ), + ( + 70.0, + [{ + 'mount': '/', + 'size_available': 1 * 10**6, + 'size_total': 5 * 10**9, + }], + ['100.0%'], + ), + ( + 40.0, + [{ + 'mount': '/', + 'size_available': 2 * 10**9, + 'size_total': 6 * 10**9, + }], + ['66.7%'], + ), + ( + None, + [{ + # enough space on /var ... + 'mount': '/var', + 'size_available': 20 * 10**9, + 'size_total': 20 * 10**9, + }, { + # .. but not enough on /var/lib + 'mount': '/var/lib', + 'size_available': 1 * 10**9, + 'size_total': 20 * 10**9, + }], + ['95.0%'], + ), +]) +def test_fails_with_insufficient_disk_space(size_limit_percent, ansible_mounts, extra_words): + task_vars = dict( + etcd_device_usage_threshold_percent=size_limit_percent, + ansible_mounts=ansible_mounts, + ) + + if task_vars["etcd_device_usage_threshold_percent"] is None: + task_vars.pop("etcd_device_usage_threshold_percent") + + check = EtcdVolume(execute_module=fake_execute_module) + result = check.run(tmp=None, task_vars=task_vars) + + assert result['failed'] + for word in extra_words: + assert word in result['msg'] + + +def fake_execute_module(*args): + raise AssertionError('this function should not be called') diff --git a/roles/openshift_hosted/defaults/main.yml b/roles/openshift_hosted/defaults/main.yml index e7e62e5e4..089054e2f 100644 --- a/roles/openshift_hosted/defaults/main.yml +++ b/roles/openshift_hosted/defaults/main.yml @@ -30,3 +30,8 @@ openshift_hosted_routers: openshift_hosted_router_certificate: {} openshift_hosted_registry_cert_expire_days: 730 openshift_hosted_router_create_certificate: False + +os_firewall_allow: +- service: Docker Registry Port + port: 5000/tcp + when: openshift.common.use_calico | bool diff --git a/roles/openshift_hosted/meta/main.yml b/roles/openshift_hosted/meta/main.yml index 9626c23c1..9e3f37130 100644 --- a/roles/openshift_hosted/meta/main.yml +++ b/roles/openshift_hosted/meta/main.yml @@ -15,3 +15,8 @@ dependencies: - role: openshift_cli - role: openshift_hosted_facts - role: lib_openshift +- role: os_firewall + os_firewall_allow: + - service: Docker Registry Port + port: 5000/tcp + when: openshift.common.use_calico | bool diff --git a/roles/openshift_hosted/tasks/registry/storage/s3.yml b/roles/openshift_hosted/tasks/registry/storage/s3.yml index 26f921f15..318969885 100644 --- a/roles/openshift_hosted/tasks/registry/storage/s3.yml +++ b/roles/openshift_hosted/tasks/registry/storage/s3.yml @@ -2,14 +2,10 @@ - name: Assert that S3 variables are provided for registry_config template assert: that: - - openshift.hosted.registry.storage.s3.accesskey | default(none) is not none - - openshift.hosted.registry.storage.s3.secretkey | default(none) is not none - openshift.hosted.registry.storage.s3.bucket | default(none) is not none - openshift.hosted.registry.storage.s3.region | default(none) is not none msg: | When using S3 storage, the following variables are required: - openshift_hosted_registry_storage_s3_accesskey - openshift_hosted_registry_storage_s3_secretkey openshift_hosted_registry_storage_s3_bucket openshift_hosted_registry_storage_s3_region diff --git a/roles/openshift_hosted/templates/registry_config.j2 b/roles/openshift_hosted/templates/registry_config.j2 index ca6a23f21..dc8a9f089 100644 --- a/roles/openshift_hosted/templates/registry_config.j2 +++ b/roles/openshift_hosted/templates/registry_config.j2 @@ -10,8 +10,12 @@ storage: blobdescriptor: inmemory {% if openshift_hosted_registry_storage_provider | default('') == 's3' %} s3: +{% if openshift_hosted_registry_storage_s3_accesskey is defined %} accesskey: {{ openshift_hosted_registry_storage_s3_accesskey }} +{% endif %} +{% if openshift_hosted_registry_storage_s3_secretkey is defined %} secretkey: {{ openshift_hosted_registry_storage_s3_secretkey }} +{% endif %} region: {{ openshift_hosted_registry_storage_s3_region }} {% if openshift_hosted_registry_storage_s3_regionendpoint is defined %} regionendpoint: {{ openshift_hosted_registry_storage_s3_regionendpoint }} diff --git a/roles/openshift_logging/README.md b/roles/openshift_logging/README.md index cba0f2de8..3c410eff2 100644 --- a/roles/openshift_logging/README.md +++ b/roles/openshift_logging/README.md @@ -97,3 +97,30 @@ same as above for their non-ops counterparts, but apply to the OPS cluster insta - `openshift_logging_kibana_ops_proxy_cpu_limit`: The amount of CPU to allocate to Kibana proxy or unset if not specified. - `openshift_logging_kibana_ops_proxy_memory_limit`: The amount of memory to allocate to Kibana proxy or unset if not specified. - `openshift_logging_kibana_ops_replica_count`: The number of replicas Kibana ops should be scaled up to. Defaults to 1. + +Elasticsearch can be exposed for external clients outside of the cluster. +- `openshift_logging_es_allow_external`: True (default is False) - if this is + True, Elasticsearch will be exposed as a Route +- `openshift_logging_es_hostname`: The external facing hostname to use for + the route and the TLS server certificate (default is "es." + + `openshift_master_default_subdomain`) +- `openshift_logging_es_cert`: The location of the certificate Elasticsearch + uses for the external TLS server cert (default is a generated cert) +- `openshift_logging_es_key`: The location of the key Elasticsearch + uses for the external TLS server cert (default is a generated key) +- `openshift_logging_es_ca_ext`: The location of the CA cert for the cert + Elasticsearch uses for the external TLS server cert (default is the internal + CA) +Elasticsearch OPS too, if using an OPS cluster: +- `openshift_logging_es_ops_allow_external`: True (default is False) - if this is + True, Elasticsearch will be exposed as a Route +- `openshift_logging_es_ops_hostname`: The external facing hostname to use for + the route and the TLS server certificate (default is "es-ops." + + `openshift_master_default_subdomain`) +- `openshift_logging_es_ops_cert`: The location of the certificate Elasticsearch + uses for the external TLS server cert (default is a generated cert) +- `openshift_logging_es_ops_key`: The location of the key Elasticsearch + uses for the external TLS server cert (default is a generated key) +- `openshift_logging_es_ops_ca_ext`: The location of the CA cert for the cert + Elasticsearch uses for the external TLS server cert (default is the internal + CA) diff --git a/roles/openshift_logging/defaults/main.yml b/roles/openshift_logging/defaults/main.yml index f43336dc4..837c54067 100644 --- a/roles/openshift_logging/defaults/main.yml +++ b/roles/openshift_logging/defaults/main.yml @@ -99,6 +99,22 @@ openshift_logging_es_config: {} openshift_logging_es_number_of_shards: 1 openshift_logging_es_number_of_replicas: 0 +# for exposing es to external (outside of the cluster) clients +openshift_logging_es_allow_external: False +openshift_logging_es_hostname: "{{ 'es.' ~ (openshift_master_default_subdomain | default('router.default.svc.cluster.local', true)) }}" + +#The absolute path on the control node to the cert file to use +#for the public facing es certs +openshift_logging_es_cert: "" + +#The absolute path on the control node to the key file to use +#for the public facing es certs +openshift_logging_es_key: "" + +#The absolute path on the control node to the CA file to use +#for the public facing es certs +openshift_logging_es_ca_ext: "" + # allow cluster-admin or cluster-reader to view operations index openshift_logging_es_ops_allow_cluster_reader: False @@ -118,6 +134,22 @@ openshift_logging_es_ops_recover_after_time: 5m openshift_logging_es_ops_storage_group: "{{ openshift_hosted_logging_elasticsearch_storage_group | default('65534') }}" openshift_logging_es_ops_nodeselector: "{{ openshift_hosted_logging_elasticsearch_ops_nodeselector | default('') | map_from_pairs }}" +# for exposing es-ops to external (outside of the cluster) clients +openshift_logging_es_ops_allow_external: False +openshift_logging_es_ops_hostname: "{{ 'es-ops.' ~ (openshift_master_default_subdomain | default('router.default.svc.cluster.local', true)) }}" + +#The absolute path on the control node to the cert file to use +#for the public facing es-ops certs +openshift_logging_es_ops_cert: "" + +#The absolute path on the control node to the key file to use +#for the public facing es-ops certs +openshift_logging_es_ops_key: "" + +#The absolute path on the control node to the CA file to use +#for the public facing es-ops certs +openshift_logging_es_ops_ca_ext: "" + # storage related defaults openshift_logging_storage_access_modes: "{{ openshift_hosted_logging_storage_access_modes | default(['ReadWriteOnce']) }}" diff --git a/roles/openshift_logging/tasks/generate_certs.yaml b/roles/openshift_logging/tasks/generate_certs.yaml index b34df018d..46a7e82c6 100644 --- a/roles/openshift_logging/tasks/generate_certs.yaml +++ b/roles/openshift_logging/tasks/generate_certs.yaml @@ -60,6 +60,24 @@ - procure_component: mux when: openshift_logging_use_mux +- include: procure_server_certs.yaml + loop_control: + loop_var: cert_info + with_items: + - procure_component: es + hostnames: "es, {{openshift_logging_es_hostname}}" + when: openshift_logging_es_allow_external | bool + +- include: procure_server_certs.yaml + loop_control: + loop_var: cert_info + with_items: + - procure_component: es-ops + hostnames: "es-ops, {{openshift_logging_es_ops_hostname}}" + when: + - openshift_logging_es_allow_external | bool + - openshift_logging_use_ops | bool + - name: Copy proxy TLS configuration file copy: src=server-tls.json dest={{generated_certs_dir}}/server-tls.json when: server_tls_json is undefined @@ -108,6 +126,14 @@ loop_var: node_name when: openshift_logging_use_mux +- name: Generate PEM cert for Elasticsearch external route + include: generate_pems.yaml component={{node_name}} + with_items: + - system.logging.es + loop_control: + loop_var: node_name + when: openshift_logging_es_allow_external | bool + - name: Creating necessary JKS certs include: generate_jks.yaml diff --git a/roles/openshift_logging/tasks/generate_routes.yaml b/roles/openshift_logging/tasks/generate_routes.yaml index f76bb3a0a..ae9a8e023 100644 --- a/roles/openshift_logging/tasks/generate_routes.yaml +++ b/roles/openshift_logging/tasks/generate_routes.yaml @@ -75,3 +75,95 @@ provider: openshift when: openshift_logging_use_ops | bool changed_when: no + +- set_fact: es_key={{ lookup('file', openshift_logging_es_key) | b64encode }} + when: + - openshift_logging_es_key | trim | length > 0 + - openshift_logging_es_allow_external | bool + changed_when: false + +- set_fact: es_cert={{ lookup('file', openshift_logging_es_cert)| b64encode }} + when: + - openshift_logging_es_cert | trim | length > 0 + - openshift_logging_es_allow_external | bool + changed_when: false + +- set_fact: es_ca={{ lookup('file', openshift_logging_es_ca_ext)| b64encode }} + when: + - openshift_logging_es_ca_ext | trim | length > 0 + - openshift_logging_es_allow_external | bool + changed_when: false + +- set_fact: es_ca={{key_pairs | entry_from_named_pair('ca_file') }} + when: + - es_ca is not defined + - openshift_logging_es_allow_external | bool + changed_when: false + +- name: Generating Elasticsearch logging routes + template: src=route_reencrypt.j2 dest={{mktemp.stdout}}/templates/logging-logging-es-route.yaml + tags: routes + vars: + obj_name: "logging-es" + route_host: "{{openshift_logging_es_hostname}}" + service_name: "logging-es" + tls_key: "{{es_key | default('') | b64decode}}" + tls_cert: "{{es_cert | default('') | b64decode}}" + tls_ca_cert: "{{es_ca | b64decode}}" + tls_dest_ca_cert: "{{key_pairs | entry_from_named_pair('ca_file')| b64decode }}" + edge_term_policy: "{{openshift_logging_es_edge_term_policy | default('') }}" + labels: + component: support + logging-infra: support + provider: openshift + changed_when: no + when: openshift_logging_es_allow_external | bool + +- set_fact: es_ops_key={{ lookup('file', openshift_logging_es_ops_key) | b64encode }} + when: + - openshift_logging_es_ops_allow_external | bool + - openshift_logging_use_ops | bool + - "{{ openshift_logging_es_ops_key | trim | length > 0 }}" + changed_when: false + +- set_fact: es_ops_cert={{ lookup('file', openshift_logging_es_ops_cert)| b64encode }} + when: + - openshift_logging_es_ops_allow_external | bool + - openshift_logging_use_ops | bool + - "{{openshift_logging_es_ops_cert | trim | length > 0}}" + changed_when: false + +- set_fact: es_ops_ca={{ lookup('file', openshift_logging_es_ops_ca_ext)| b64encode }} + when: + - openshift_logging_es_ops_allow_external | bool + - openshift_logging_use_ops | bool + - "{{openshift_logging_es_ops_ca_ext | trim | length > 0}}" + changed_when: false + +- set_fact: es_ops_ca={{key_pairs | entry_from_named_pair('ca_file') }} + when: + - openshift_logging_es_ops_allow_external | bool + - openshift_logging_use_ops | bool + - es_ops_ca is not defined + changed_when: false + +- name: Generating Elasticsearch logging ops routes + template: src=route_reencrypt.j2 dest={{mktemp.stdout}}/templates/logging-logging-es-ops-route.yaml + tags: routes + vars: + obj_name: "logging-es-ops" + route_host: "{{openshift_logging_es_ops_hostname}}" + service_name: "logging-es-ops" + tls_key: "{{es_ops_key | default('') | b64decode}}" + tls_cert: "{{es_ops_cert | default('') | b64decode}}" + tls_ca_cert: "{{es_ops_ca | b64decode}}" + tls_dest_ca_cert: "{{key_pairs | entry_from_named_pair('ca_file')| b64decode }}" + edge_term_policy: "{{openshift_logging_es_ops_edge_term_policy | default('') }}" + labels: + component: support + logging-infra: support + provider: openshift + when: + - openshift_logging_es_ops_allow_external | bool + - openshift_logging_use_ops | bool + changed_when: no diff --git a/roles/openshift_logging/tasks/generate_secrets.yaml b/roles/openshift_logging/tasks/generate_secrets.yaml index c1da49fd8..b629bd995 100644 --- a/roles/openshift_logging/tasks/generate_secrets.yaml +++ b/roles/openshift_logging/tasks/generate_secrets.yaml @@ -99,3 +99,31 @@ when: logging_es_secret.stdout is defined check_mode: no changed_when: no + +- name: Retrieving the cert to use when generating secrets for Elasticsearch external route + slurp: src="{{generated_certs_dir}}/{{item.file}}" + register: es_key_pairs + with_items: + - { name: "ca_file", file: "ca.crt" } + - { name: "es_key", file: "system.logging.es.key"} + - { name: "es_cert", file: "system.logging.es.crt"} + when: openshift_logging_es_allow_external | bool + +- name: Generating secrets for Elasticsearch external route + template: src=secret.j2 dest={{mktemp.stdout}}/templates/{{secret_name}}-secret.yaml + vars: + secret_name: "logging-{{component}}" + secret_key_file: "{{component}}_key" + secret_cert_file: "{{component}}_cert" + secrets: + - {key: ca, value: "{{es_key_pairs | entry_from_named_pair('ca_file')| b64decode }}"} + - {key: key, value: "{{es_key_pairs | entry_from_named_pair(secret_key_file)| b64decode }}"} + - {key: cert, value: "{{es_key_pairs | entry_from_named_pair(secret_cert_file)| b64decode }}"} + secret_keys: ["ca", "cert", "key"] + with_items: + - es + loop_control: + loop_var: component + check_mode: no + changed_when: no + when: openshift_logging_es_allow_external | bool diff --git a/roles/openshift_master/templates/master.yaml.v1.j2 b/roles/openshift_master/templates/master.yaml.v1.j2 index 938ac2a12..1935d9592 100644 --- a/roles/openshift_master/templates/master.yaml.v1.j2 +++ b/roles/openshift_master/templates/master.yaml.v1.j2 @@ -44,10 +44,10 @@ assetConfig: - {{ cipher_suite }} {% endfor %} {% endif %} -{% if openshift_master_ha | bool %} {% if openshift.master.audit_config | default(none) is not none and openshift.common.version_gte_3_2_or_1_2 | bool %} auditConfig:{{ openshift.master.audit_config | to_padded_yaml(level=1) }} {% endif %} +{% if openshift_master_ha | bool %} controllerLeaseTTL: {{ openshift.master.controller_lease_ttl | default('30') }} {% endif %} {% if openshift.common.version_gte_3_3_or_1_3 | bool %} @@ -274,5 +274,12 @@ servingInfo: - {{ cipher_suite }} {% endfor %} {% endif %} +{% if openshift_template_service_broker_namespaces is defined %} +templateServiceBrokerConfig: + templateNamespaces: +{% for namespace in openshift_template_service_broker_namespaces %} + - {{ namespace }} +{% endfor %} +{% endif %} volumeConfig: dynamicProvisioningEnabled: {{ openshift.master.dynamic_provisioning_enabled }} diff --git a/roles/openshift_master_facts/filter_plugins/openshift_master.py b/roles/openshift_master_facts/filter_plugins/openshift_master.py index b5be193d2..e767772ce 100644 --- a/roles/openshift_master_facts/filter_plugins/openshift_master.py +++ b/roles/openshift_master_facts/filter_plugins/openshift_master.py @@ -468,7 +468,8 @@ class GitHubIdentityProvider(IdentityProviderOauthBase): """ def __init__(self, api_version, idp): IdentityProviderOauthBase.__init__(self, api_version, idp) - self._optional += [['organizations']] + self._optional += [['organizations'], + ['teams']] class FilterModule(object): diff --git a/roles/openshift_metrics/tasks/main.yaml b/roles/openshift_metrics/tasks/main.yaml index e8b7bea5c..9af10a849 100644 --- a/roles/openshift_metrics/tasks/main.yaml +++ b/roles/openshift_metrics/tasks/main.yaml @@ -1,4 +1,12 @@ --- +- local_action: shell rpm -q python-passlib || echo not installed + register: passlib_result + +- name: Check that python-passlib is available on the control host + assert: + that: + - "'not installed' not in passlib_result.stdout" + msg: "python-passlib rpm must be installed on control host" - name: Set default image variables based on deployment_type include_vars: "{{ item }}" @@ -25,6 +33,7 @@ local_action: command mktemp -d register: local_tmp changed_when: False + become: false - name: Copy the admin client config(s) command: > diff --git a/roles/openshift_node/defaults/main.yml b/roles/openshift_node/defaults/main.yml index bf66ef1d6..5904ca9bc 100644 --- a/roles/openshift_node/defaults/main.yml +++ b/roles/openshift_node/defaults/main.yml @@ -9,3 +9,6 @@ os_firewall_allow: - service: OpenShift OVS sdn port: 4789/udp when: openshift.common.use_openshift_sdn | bool +- service: Calico BGP Port + port: 179/tcp + when: openshift.common.use_calico | bool diff --git a/roles/openshift_node/handlers/main.yml b/roles/openshift_node/handlers/main.yml index cb51416d4..4dcf1eef8 100644 --- a/roles/openshift_node/handlers/main.yml +++ b/roles/openshift_node/handlers/main.yml @@ -12,3 +12,6 @@ - name: restart node systemd: name={{ openshift.common.service_type }}-node state=restarted when: (not skip_node_svc_handlers | default(False) | bool) and not (node_service_status_changed | default(false) | bool) + +- name: reload sysctl.conf + command: /sbin/sysctl -p diff --git a/roles/openshift_node/meta/main.yml b/roles/openshift_node/meta/main.yml index 0da41d0c1..3b7e8126a 100644 --- a/roles/openshift_node/meta/main.yml +++ b/roles/openshift_node/meta/main.yml @@ -33,6 +33,12 @@ dependencies: when: openshift.common.use_openshift_sdn | bool - role: os_firewall os_firewall_allow: + - service: Calico BGP Port + port: 179/tcp + when: openshift.common.use_calico | bool + +- role: os_firewall + os_firewall_allow: - service: Kubernetes service NodePort TCP port: "{{ openshift_node_port_range | default('') }}/tcp" - service: Kubernetes service NodePort UDP diff --git a/roles/openshift_node/tasks/main.yml b/roles/openshift_node/tasks/main.yml index 656874f56..54ba5ba01 100644 --- a/roles/openshift_node/tasks/main.yml +++ b/roles/openshift_node/tasks/main.yml @@ -104,8 +104,14 @@ # The atomic-openshift-node service will set this parameter on # startup, but if the network service is restarted this setting is # lost. Reference: https://bugzilla.redhat.com/show_bug.cgi?id=1372388 +# +# Use lineinfile w/ a handler for this task until +# https://github.com/ansible/ansible/pull/24277 is included in an +# ansible release and we can use the sysctl module. - name: Persist net.ipv4.ip_forward sysctl entry - sysctl: name="net.ipv4.ip_forward" value=1 sysctl_set=yes state=present reload=yes + lineinfile: dest=/etc/sysctl.conf regexp='^net.ipv4.ip_forward' line='net.ipv4.ip_forward=1' + notify: + - reload sysctl.conf - name: Start and enable openvswitch service systemd: diff --git a/roles/openshift_node_upgrade/tasks/main.yml b/roles/openshift_node_upgrade/tasks/main.yml index a41a97e01..7231bdb9d 100644 --- a/roles/openshift_node_upgrade/tasks/main.yml +++ b/roles/openshift_node_upgrade/tasks/main.yml @@ -143,7 +143,7 @@ name: "{{ openshift.common.hostname | lower }}" register: node_output delegate_to: "{{ groups.oo_first_master.0 }}" - until: node_output.results.results[0].status.conditions | selectattr('type', 'match', '^Ready$') | map(attribute='status') | join | bool == True + until: node_output.results.returncode == 0 and node_output.results.results[0].status.conditions | selectattr('type', 'match', '^Ready$') | map(attribute='status') | join | bool == True # Give the node two minutes to come back online. retries: 24 delay: 5 diff --git a/test/integration/openshift_health_checker/preflight/playbooks/package_availability_missing_required.yml b/test/integration/openshift_health_checker/preflight/playbooks/package_availability_missing_required.yml index 31d0d521e..a557282b4 100644 --- a/test/integration/openshift_health_checker/preflight/playbooks/package_availability_missing_required.yml +++ b/test/integration/openshift_health_checker/preflight/playbooks/package_availability_missing_required.yml @@ -1,4 +1,5 @@ --- +# NOTE: this test is probably superfluous since openshift_version already does it - include: ../../setup_container.yml vars: image: preflight-aos-package-checks @@ -7,11 +8,22 @@ - name: Fail as required packages cannot be installed hosts: all + pre_tasks: + + # run before roles to prevent openshift_version breaking + - include: tasks/enable_repo.yml + vars: { repo_name: "ose-3.2" } + roles: - openshift_health_checker - tasks: + + post_tasks: - block: + # put the repo back to disabled + - include: tasks/enable_repo.yml + vars: { repo_name: "ose-3.2", repo_enabled: 0 } + - action: openshift_health_check args: checks: [ 'package_availability' ] diff --git a/test/integration/openshift_health_checker/preflight/playbooks/package_update_dep_missing.yml b/test/integration/openshift_health_checker/preflight/playbooks/package_update_dep_missing.yml index 7b6e71f91..0929b73ce 100644 --- a/test/integration/openshift_health_checker/preflight/playbooks/package_update_dep_missing.yml +++ b/test/integration/openshift_health_checker/preflight/playbooks/package_update_dep_missing.yml @@ -8,9 +8,16 @@ - name: Fails when a dependency required for update is missing hosts: all + pre_tasks: + + # run before roles to prevent openshift_version breaking + - include: tasks/enable_repo.yml + vars: { repo_name: "ose-3.2" } + roles: - openshift_health_checker - tasks: + + post_tasks: - block: - include: tasks/enable_repo.yml diff --git a/test/integration/openshift_health_checker/preflight/playbooks/package_update_repo_broken.yml b/test/integration/openshift_health_checker/preflight/playbooks/package_update_repo_broken.yml index c2e9c3866..f8790358a 100644 --- a/test/integration/openshift_health_checker/preflight/playbooks/package_update_repo_broken.yml +++ b/test/integration/openshift_health_checker/preflight/playbooks/package_update_repo_broken.yml @@ -8,9 +8,16 @@ - name: Fails when a repo definition is completely broken hosts: all + pre_tasks: + + # run before roles to prevent openshift_version breaking + - include: tasks/enable_repo.yml + vars: { repo_name: "ose-3.2" } + roles: - openshift_health_checker - tasks: + + post_tasks: - block: - include: tasks/enable_repo.yml diff --git a/test/integration/openshift_health_checker/preflight/playbooks/package_update_repo_disabled.yml b/test/integration/openshift_health_checker/preflight/playbooks/package_update_repo_disabled.yml index 98d41aad4..e2bb84715 100644 --- a/test/integration/openshift_health_checker/preflight/playbooks/package_update_repo_disabled.yml +++ b/test/integration/openshift_health_checker/preflight/playbooks/package_update_repo_disabled.yml @@ -8,9 +8,16 @@ - name: Succeeds when nothing blocks a yum update hosts: all + pre_tasks: + + # run before roles to prevent openshift_version breaking + - include: tasks/enable_repo.yml + vars: { repo_name: "ose-3.2" } + roles: - openshift_health_checker - tasks: + + post_tasks: - block: - action: openshift_health_check diff --git a/test/integration/openshift_health_checker/preflight/playbooks/package_update_repo_unreachable.yml b/test/integration/openshift_health_checker/preflight/playbooks/package_update_repo_unreachable.yml index 60ab9942a..28efdd81d 100644 --- a/test/integration/openshift_health_checker/preflight/playbooks/package_update_repo_unreachable.yml +++ b/test/integration/openshift_health_checker/preflight/playbooks/package_update_repo_unreachable.yml @@ -8,9 +8,16 @@ - name: Fails when repo content is not available hosts: all + pre_tasks: + + # run before roles to prevent openshift_version breaking + - include: tasks/enable_repo.yml + vars: { repo_name: "ose-3.2" } + roles: - openshift_health_checker - tasks: + + post_tasks: - block: - include: tasks/enable_repo.yml diff --git a/test/integration/openshift_health_checker/preflight/playbooks/package_version_matches.yml b/test/integration/openshift_health_checker/preflight/playbooks/package_version_matches.yml index cd60dee5a..66d43d809 100644 --- a/test/integration/openshift_health_checker/preflight/playbooks/package_version_matches.yml +++ b/test/integration/openshift_health_checker/preflight/playbooks/package_version_matches.yml @@ -8,13 +8,17 @@ - name: Success when AOS version matches openshift_release hosts: all + pre_tasks: + + # run before roles to prevent openshift_version breaking + - include: tasks/enable_repo.yml + vars: { repo_name: "ose-3.2" } + roles: - openshift_health_checker - tasks: - - block: - - include: tasks/enable_repo.yml - vars: { repo_name: "ose-3.2" } + post_tasks: + - block: - action: openshift_health_check args: diff --git a/test/integration/openshift_health_checker/preflight/playbooks/package_version_mismatches.yml b/test/integration/openshift_health_checker/preflight/playbooks/package_version_mismatches.yml index 5939a1ef1..c26413009 100644 --- a/test/integration/openshift_health_checker/preflight/playbooks/package_version_mismatches.yml +++ b/test/integration/openshift_health_checker/preflight/playbooks/package_version_mismatches.yml @@ -1,4 +1,5 @@ --- +# NOTE: this test is probably superfluous since openshift_version already does it - include: ../../setup_container.yml vars: image: preflight-aos-package-checks @@ -8,14 +9,24 @@ - name: Failure when AOS version doesn't match openshift_release hosts: all + pre_tasks: + + # run before roles to prevent openshift_version breaking + - include: tasks/enable_repo.yml + vars: { repo_name: "ose-3.3" } + roles: - openshift_health_checker - tasks: + + post_tasks: - block: + # put the repo back to disabled + - include: tasks/enable_repo.yml + vars: { repo_name: "ose-3.3", repo_enabled: 0 } + # test with wrong repo enabled - include: tasks/enable_repo.yml vars: { repo_name: "ose-3.2" } - - action: openshift_health_check args: checks: [ 'package_version' ] diff --git a/test/integration/openshift_health_checker/preflight/playbooks/package_version_multiple.yml b/test/integration/openshift_health_checker/preflight/playbooks/package_version_multiple.yml index be0f9bc7a..c941413de 100644 --- a/test/integration/openshift_health_checker/preflight/playbooks/package_version_multiple.yml +++ b/test/integration/openshift_health_checker/preflight/playbooks/package_version_multiple.yml @@ -7,14 +7,19 @@ - name: Fails when multiple AOS versions are available hosts: all + pre_tasks: + + # run before roles to prevent openshift_version breaking + - include: tasks/enable_repo.yml + vars: { repo_name: "ose-3.2" } + roles: - openshift_health_checker - tasks: - - block: - - include: tasks/enable_repo.yml - vars: { repo_name: "ose-3.2" } + post_tasks: + - block: + # enable repo with extra minor version available - include: tasks/enable_repo.yml vars: { repo_name: "ose-3.3" } diff --git a/test/integration/openshift_health_checker/preflight/playbooks/tasks/enable_repo.yml b/test/integration/openshift_health_checker/preflight/playbooks/tasks/enable_repo.yml index aaacf205e..a41cb3c9a 100644 --- a/test/integration/openshift_health_checker/preflight/playbooks/tasks/enable_repo.yml +++ b/test/integration/openshift_health_checker/preflight/playbooks/tasks/enable_repo.yml @@ -6,4 +6,4 @@ dest: /etc/yum.repos.d/{{ repo_name }}.repo section: "{{ repo_name }}" option: enabled - value: 1 + value: "{{ repo_enabled | default(1) }}" |