diff options
160 files changed, 4979 insertions, 885 deletions
diff --git a/.dockerignore b/.dockerignore index 968811df5..0a70c5bfa 100644 --- a/.dockerignore +++ b/.dockerignore @@ -1,8 +1,12 @@ .* bin docs +hack +inventory test utils **/*.md *.spec +*.ini +*.txt setup* diff --git a/.tito/packages/openshift-ansible b/.tito/packages/openshift-ansible index fc0fd7ceb..64a35a246 100644 --- a/.tito/packages/openshift-ansible +++ b/.tito/packages/openshift-ansible @@ -1 +1 @@ -3.6.143-1 ./ +3.6.153-1 ./ @@ -33,35 +33,6 @@ To build a container image of `openshift-ansible` using standalone **Docker**: cd openshift-ansible docker build -f images/installer/Dockerfile -t openshift-ansible . -### Building on OpenShift - -To build an openshift-ansible image using an **OpenShift** [build and image stream](https://docs.openshift.org/latest/architecture/core_concepts/builds_and_image_streams.html) the straightforward command would be: - - oc new-build registry.centos.org/openshift/playbook2image~https://github.com/openshift/openshift-ansible - -However: because the `Dockerfile` for this repository is not in the top level directory, and because we can't change the build context to the `images/installer` path as it would cause the build to fail, the `oc new-app` command above will create a build configuration using the *source to image* strategy, which is the default approach of the [playbook2image](https://github.com/openshift/playbook2image) base image. This does build an image successfully, but unfortunately the resulting image will be missing some customizations that are handled by the [Dockerfile](images/installer/Dockerfile) in this repo. - -At the time of this writing there is no straightforward option to [set the dockerfilePath](https://docs.openshift.org/latest/dev_guide/builds/build_strategies.html#dockerfile-path) of a `docker` build strategy with `oc new-build`. The alternatives to achieve this are: - -- Use the simple `oc new-build` command above to generate the BuildConfig and ImageStream objects, and then manually edit the generated build configuration to change its strategy to `dockerStrategy` and set `dockerfilePath` to `images/installer/Dockerfile`. - -- Download and pass the `Dockerfile` to `oc new-build` with the `-D` option: - -``` -curl -s https://raw.githubusercontent.com/openshift/openshift-ansible/master/images/installer/Dockerfile | - oc new-build -D - \ - --docker-image=registry.centos.org/openshift/playbook2image \ - https://github.com/openshift/openshift-ansible -``` - -Once a build is started, the progress of the build can be monitored with: - - oc logs -f bc/openshift-ansible - -Once built, the image will be visible in the Image Stream created by `oc new-app`: - - oc describe imagestream openshift-ansible - ## Build the Atomic System Container A system container runs using runC instead of Docker and it is managed diff --git a/README_CONTAINER_IMAGE.md b/README_CONTAINER_IMAGE.md index cf3b432df..a2151352d 100644 --- a/README_CONTAINER_IMAGE.md +++ b/README_CONTAINER_IMAGE.md @@ -1,6 +1,6 @@ # Containerized openshift-ansible to run playbooks -The [Dockerfile](images/installer/Dockerfile) in this repository uses the [playbook2image](https://github.com/openshift/playbook2image) source-to-image base image to containerize `openshift-ansible`. The resulting image can run any of the provided playbooks. See [BUILD.md](BUILD.md) for image build instructions. +The [Dockerfile](images/installer/Dockerfile) in this repository can be used to build a containerized `openshift-ansible`. The resulting image can run any of the provided playbooks. See [BUILD.md](BUILD.md) for image build instructions. The image is designed to **run as a non-root user**. The container's UID is mapped to the username `default` at runtime. Therefore, the container's environment reflects that user's settings, and the configuration should match that. For example `$HOME` is `/opt/app-root/src`, so ssh keys are expected to be under `/opt/app-root/src/.ssh`. If you ran a container as `root` you would have to adjust the container's configuration accordingly, e.g. by placing ssh keys under `/root/.ssh` instead. Nevertheless, the expectation is that containers will be run as non-root; for example, this container image can be run inside OpenShift under the default `restricted` [security context constraint](https://docs.openshift.org/latest/architecture/additional_concepts/authorization.html#security-context-constraints). @@ -14,8 +14,6 @@ This provides consistency with other images used by the platform and it's also a ## Usage -The `playbook2image` base image provides several options to control the behaviour of the containers. For more details on these options see the [playbook2image](https://github.com/openshift/playbook2image) documentation. - At the very least, when running a container you must specify: 1. An **inventory**. This can be a location inside the container (possibly mounted as a volume) with a path referenced via the `INVENTORY_FILE` environment variable. Alternatively you can serve the inventory file from a web server and use the `INVENTORY_URL` environment variable to fetch it, or `DYNAMIC_SCRIPT_URL` to download a script that provides a dynamic inventory. @@ -52,8 +50,6 @@ Here is a detailed explanation of the options used in the command above: Further usage examples are available in the [examples directory](examples/) with samples of how to use the image from within OpenShift. -Additional usage information for images built from `playbook2image` like this one can be found in the [playbook2image examples](https://github.com/openshift/playbook2image/tree/master/examples). - ## Running openshift-ansible as a System Container Building the System Container: See the [BUILD.md](BUILD.md). diff --git a/README_libvirt.md b/README_libvirt.md index c523d83fb..1661681a0 100644 --- a/README_libvirt.md +++ b/README_libvirt.md @@ -15,7 +15,7 @@ Install dependencies 3. Install [ebtables](http://ebtables.netfilter.org/) 4. Install [qemu and qemu-system-x86](http://wiki.qemu.org/Main_Page) 5. Install [libvirt-python and libvirt](http://libvirt.org/) -6. Install [genisoimage](http://cdrkit.org/) +6. Install [genisoimage](http://cdrkit.org/) or [mkisofs](http://cdrtools.sourceforge.net/private/cdrecord.html) 7. Enable and start the libvirt daemon, e.g: - `systemctl enable libvirtd` - `systemctl start libvirtd` @@ -23,6 +23,7 @@ Install dependencies 9. Check that your `$HOME` is accessible to the qemu user² 10. Configure dns resolution on the host³ 11. Install libselinux-python +12. Ensure you have an SSH private and public keypair at `~/.ssh/id_rsa` and `~/.ssh/id_rsa.pub`⁴ #### ¹ Depending on your distribution, libvirt access may be denied by default or may require a password at each access. @@ -103,6 +104,11 @@ sudo vi /etc/NetworkManager/dnsmasq.d/libvirt_dnsmasq.conf server=/example.com/192.168.55.1 ``` +#### ⁴ Private and public keypair in ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub + +This playbook uses SSH keys to communicate with the libvirt-driven virtual machines. At this time the names of those keys are fixed and cannot be changed. + + Test The Setup -------------- diff --git a/filter_plugins/oo_filters.py b/filter_plugins/oo_filters.py index cff9f8a60..36a90a870 100644 --- a/filter_plugins/oo_filters.py +++ b/filter_plugins/oo_filters.py @@ -194,10 +194,10 @@ def oo_select_keys_from_list(data, keys): """ if not isinstance(data, list): - raise errors.AnsibleFilterError("|failed expects to filter on a list") + raise errors.AnsibleFilterError("|oo_select_keys_from_list failed expects to filter on a list") if not isinstance(keys, list): - raise errors.AnsibleFilterError("|failed expects first param is a list") + raise errors.AnsibleFilterError("|oo_select_keys_from_list failed expects first param is a list") # Gather up the values for the list of keys passed in retval = [oo_select_keys(item, keys) for item in data] @@ -213,10 +213,10 @@ def oo_select_keys(data, keys): """ if not isinstance(data, Mapping): - raise errors.AnsibleFilterError("|failed expects to filter on a dict or object") + raise errors.AnsibleFilterError("|oo_select_keys failed expects to filter on a dict or object") if not isinstance(keys, list): - raise errors.AnsibleFilterError("|failed expects first param is a list") + raise errors.AnsibleFilterError("|oo_select_keys failed expects first param is a list") # Gather up the values for the list of keys passed in retval = [data[key] for key in keys if key in data] @@ -1009,6 +1009,21 @@ def oo_random_word(length, source='abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRS return ''.join(random.choice(source) for i in range(length)) +def oo_contains_rule(source, apiGroups, resources, verbs): + '''Return true if the specified rule is contained within the provided source''' + + rules = source['rules'] + + if rules: + for rule in rules: + if set(rule['apiGroups']) == set(apiGroups): + if set(rule['resources']) == set(resources): + if set(rule['verbs']) == set(verbs): + return True + + return False + + class FilterModule(object): """ Custom ansible filter mapping """ @@ -1049,5 +1064,6 @@ class FilterModule(object): "oo_openshift_loadbalancer_frontends": oo_openshift_loadbalancer_frontends, "oo_openshift_loadbalancer_backends": oo_openshift_loadbalancer_backends, "to_padded_yaml": to_padded_yaml, - "oo_random_word": oo_random_word + "oo_random_word": oo_random_word, + "oo_contains_rule": oo_contains_rule } diff --git a/images/installer/Dockerfile b/images/installer/Dockerfile index 915dfe377..d03f33a1d 100644 --- a/images/installer/Dockerfile +++ b/images/installer/Dockerfile @@ -1,10 +1,18 @@ -# Using playbook2image as a base -# See https://github.com/openshift/playbook2image for details on the image -# including documentation for the settings/env vars referenced below -FROM registry.centos.org/openshift/playbook2image:latest +FROM centos:7 MAINTAINER OpenShift Team <dev@lists.openshift.redhat.com> +USER root + +# install ansible and deps +RUN INSTALL_PKGS="python-lxml pyOpenSSL python2-cryptography openssl java-1.8.0-openjdk-headless httpd-tools openssh-clients" \ + && yum install -y --setopt=tsflags=nodocs $INSTALL_PKGS \ + && EPEL_PKGS="ansible python-passlib python2-boto" \ + && yum install -y epel-release \ + && yum install -y --setopt=tsflags=nodocs $EPEL_PKGS \ + && rpm -q $INSTALL_PKGS $EPEL_PKGS \ + && yum clean all + LABEL name="openshift/origin-ansible" \ summary="OpenShift's installation and configuration tool" \ description="A containerized openshift-ansible image to let you run playbooks to install, upgrade, maintain and check an OpenShift cluster" \ @@ -12,40 +20,24 @@ LABEL name="openshift/origin-ansible" \ io.k8s.display-name="openshift-ansible" \ io.k8s.description="A containerized openshift-ansible image to let you run playbooks to install, upgrade, maintain and check an OpenShift cluster" \ io.openshift.expose-services="" \ - io.openshift.tags="openshift,install,upgrade,ansible" + io.openshift.tags="openshift,install,upgrade,ansible" \ + atomic.run="once" -USER root +ENV USER_UID=1001 \ + HOME=/opt/app-root/src \ + WORK_DIR=/usr/share/ansible/openshift-ansible \ + OPTS="-v" -# Create a symlink to /opt/app-root/src so that files under /usr/share/ansible are accessible. -# This is required since the system-container uses by default the playbook under -# /usr/share/ansible/openshift-ansible. With this change we won't need to keep two different -# configurations for the two images. -RUN mkdir -p /usr/share/ansible/ && ln -s /opt/app-root/src /usr/share/ansible/openshift-ansible +# Add image scripts and files for running as a system container +COPY images/installer/root / +# Include playbooks, roles, plugins, etc. from this repo +COPY . ${WORK_DIR} -RUN INSTALL_PKGS="skopeo openssl java-1.8.0-openjdk-headless httpd-tools" && \ - yum install -y --setopt=tsflags=nodocs $INSTALL_PKGS && \ - rpm -V $INSTALL_PKGS && \ - yum clean all +RUN /usr/local/bin/user_setup \ + && rm /usr/local/bin/usage.ocp USER ${USER_UID} -# The playbook to be run is specified via the PLAYBOOK_FILE env var. -# This sets a default of openshift_facts.yml as it's an informative playbook -# that can help test that everything is set properly (inventory, sshkeys) -ENV PLAYBOOK_FILE=playbooks/byo/openshift_facts.yml \ - OPTS="-v" \ - INSTALL_OC=true - -# playbook2image's assemble script expects the source to be available in -# /tmp/src (as per the source-to-image specs) so we import it there -ADD . /tmp/src - -# Running the 'assemble' script provided by playbook2image will install -# dependencies specified in requirements.txt and install the 'oc' client -# as per the INSTALL_OC environment setting above -RUN /usr/libexec/s2i/assemble - -# Add files for running as a system container -COPY images/installer/system-container/root / - -CMD [ "/usr/libexec/s2i/run" ] +WORKDIR ${WORK_DIR} +ENTRYPOINT [ "/usr/local/bin/entrypoint" ] +CMD [ "/usr/local/bin/run" ] diff --git a/images/installer/Dockerfile.rhel7 b/images/installer/Dockerfile.rhel7 index f861d4bcf..3110f409c 100644 --- a/images/installer/Dockerfile.rhel7 +++ b/images/installer/Dockerfile.rhel7 @@ -1,55 +1,46 @@ -FROM openshift3/playbook2image +FROM rhel7.3:7.3-released MAINTAINER OpenShift Team <dev@lists.openshift.redhat.com> -# override env vars from base image -ENV SUMMARY="OpenShift's installation and configuration tool" \ - DESCRIPTION="A containerized openshift-ansible image to let you run playbooks to install, upgrade, maintain and check an OpenShift cluster" +USER root + +# Playbooks, roles, and their dependencies are installed from packages. +RUN INSTALL_PKGS="atomic-openshift-utils atomic-openshift-clients python-boto openssl java-1.8.0-openjdk-headless httpd-tools" \ + && yum repolist > /dev/null \ + && yum-config-manager --enable rhel-7-server-ose-3.6-rpms \ + && yum-config-manager --enable rhel-7-server-rh-common-rpms \ + && yum install -y --setopt=tsflags=nodocs $INSTALL_PKGS \ + && rpm -q $INSTALL_PKGS \ + && yum clean all LABEL name="openshift3/ose-ansible" \ - summary="$SUMMARY" \ - description="$DESCRIPTION" \ + summary="OpenShift's installation and configuration tool" \ + description="A containerized openshift-ansible image to let you run playbooks to install, upgrade, maintain and check an OpenShift cluster" \ url="https://github.com/openshift/openshift-ansible" \ io.k8s.display-name="openshift-ansible" \ - io.k8s.description="$DESCRIPTION" \ + io.k8s.description="A containerized openshift-ansible image to let you run playbooks to install, upgrade, maintain and check an OpenShift cluster" \ io.openshift.expose-services="" \ io.openshift.tags="openshift,install,upgrade,ansible" \ com.redhat.component="aos3-installation-docker" \ version="v3.6.0" \ release="1" \ - architecture="x86_64" - -# Playbooks, roles and their dependencies are installed from packages. -# Unlike in Dockerfile, we don't invoke the 'assemble' script here -# because all content and dependencies (like 'oc') is already -# installed via yum. -USER root -RUN INSTALL_PKGS="atomic-openshift-utils atomic-openshift-clients python-boto skopeo openssl java-1.8.0-openjdk-headless httpd-tools" && \ - yum repolist > /dev/null && \ - yum-config-manager --enable rhel-7-server-ose-3.6-rpms && \ - yum-config-manager --enable rhel-7-server-rh-common-rpms && \ - yum install -y $INSTALL_PKGS && \ - yum clean all - -# The symlinks below are a (hopefully temporary) hack to work around the fact that this -# image is based on python s2i which uses the python27 SCL instead of system python, -# and so the system python modules we need would otherwise not be in the path. -RUN ln -s /usr/lib/python2.7/site-packages/{boto,passlib} /opt/app-root/lib64/python2.7/ - -USER ${USER_UID} + architecture="x86_64" \ + atomic.run="once" -# The playbook to be run is specified via the PLAYBOOK_FILE env var. -# This sets a default of openshift_facts.yml as it's an informative playbook -# that can help test that everything is set properly (inventory, sshkeys). -# As the playbooks are installed via packages instead of being copied to -# $APP_HOME by the 'assemble' script, we set the WORK_DIR env var to the -# location of openshift-ansible. -ENV PLAYBOOK_FILE=playbooks/byo/openshift_facts.yml \ - ANSIBLE_CONFIG=/usr/share/atomic-openshift-utils/ansible.cfg \ +ENV USER_UID=1001 \ + HOME=/opt/app-root/src \ WORK_DIR=/usr/share/ansible/openshift-ansible \ + ANSIBLE_CONFIG=/usr/share/atomic-openshift-utils/ansible.cfg \ OPTS="-v" -# Add files for running as a system container -COPY system-container/root / +# Add image scripts and files for running as a system container +COPY root / + +RUN /usr/local/bin/user_setup \ + && mv /usr/local/bin/usage{.ocp,} + +USER ${USER_UID} -CMD [ "/usr/libexec/s2i/run" ] +WORKDIR ${WORK_DIR} +ENTRYPOINT [ "/usr/local/bin/entrypoint" ] +CMD [ "/usr/local/bin/run" ] diff --git a/images/installer/system-container/README.md b/images/installer/README_CONTAINER_IMAGE.md index fbcd47c4a..bc1ebb4a8 100644 --- a/images/installer/system-container/README.md +++ b/images/installer/README_CONTAINER_IMAGE.md @@ -1,17 +1,34 @@ -# System container installer +ORIGIN-ANSIBLE IMAGE INSTALLER +=============================== + +Contains Dockerfile information for building an openshift/origin-ansible image +based on `centos:7` or `rhel7.3:7.3-released`. + +Read additional setup information for this image at: https://hub.docker.com/r/openshift/origin-ansible/ + +Read additional information about the `openshift/origin-ansible` at: https://github.com/openshift/openshift-ansible/blob/master/README_CONTAINER_IMAGE.md + +Also contains necessary components for running the installer using an Atomic System Container. + + +System container installer +========================== These files are needed to run the installer using an [Atomic System container](http://www.projectatomic.io/blog/2016/09/intro-to-system-containers/). +These files can be found under `root/exports`: * config.json.template - Template of the configuration file used for running containers. -* manifest.json - Used to define various settings for the system container, such as the default values to use for the installation. - -* run-system-container.sh - Entrypoint to the container. +* manifest.json - Used to define various settings for the system container, such as the default values to use for the installation. * service.template - Template file for the systemd service. * tmpfiles.template - Template file for systemd-tmpfiles. +These files can be found under `root/usr/local/bin`: + +* run-system-container.sh - Entrypoint to the container. + ## Options These options may be set via the ``atomic`` ``--set`` flag. For defaults see ``root/exports/manifest.json`` @@ -28,4 +45,4 @@ These options may be set via the ``atomic`` ``--set`` flag. For defaults see ``r * ANSIBLE_CONFIG - Full path for the ansible configuration file to use inside the container -* INVENTORY_FILE - Full path for the inventory to use from the host +* INVENTORY_FILE - Full path for the inventory to use from the host
\ No newline at end of file diff --git a/images/installer/system-container/root/exports/config.json.template b/images/installer/root/exports/config.json.template index 739c0080f..739c0080f 100644 --- a/images/installer/system-container/root/exports/config.json.template +++ b/images/installer/root/exports/config.json.template diff --git a/images/installer/system-container/root/exports/manifest.json b/images/installer/root/exports/manifest.json index 8b984d7a3..8b984d7a3 100644 --- a/images/installer/system-container/root/exports/manifest.json +++ b/images/installer/root/exports/manifest.json diff --git a/images/installer/system-container/root/exports/service.template b/images/installer/root/exports/service.template index bf5316af6..bf5316af6 100644 --- a/images/installer/system-container/root/exports/service.template +++ b/images/installer/root/exports/service.template diff --git a/images/installer/system-container/root/exports/tmpfiles.template b/images/installer/root/exports/tmpfiles.template index b1f6caf47..b1f6caf47 100644 --- a/images/installer/system-container/root/exports/tmpfiles.template +++ b/images/installer/root/exports/tmpfiles.template diff --git a/images/installer/root/usr/local/bin/entrypoint b/images/installer/root/usr/local/bin/entrypoint new file mode 100755 index 000000000..777bf3f11 --- /dev/null +++ b/images/installer/root/usr/local/bin/entrypoint @@ -0,0 +1,17 @@ +#!/bin/bash -e +# +# This file serves as the main entrypoint to the openshift-ansible image. +# +# For more information see the documentation: +# https://github.com/openshift/openshift-ansible/blob/master/README_CONTAINER_IMAGE.md + + +# Patch /etc/passwd file with the current user info. +# The current user's entry must be correctly defined in this file in order for +# the `ssh` command to work within the created container. + +if ! whoami &>/dev/null; then + echo "${USER:-default}:x:$(id -u):$(id -g):Default User:$HOME:/sbin/nologin" >> /etc/passwd +fi + +exec "$@" diff --git a/images/installer/root/usr/local/bin/run b/images/installer/root/usr/local/bin/run new file mode 100755 index 000000000..9401ea118 --- /dev/null +++ b/images/installer/root/usr/local/bin/run @@ -0,0 +1,46 @@ +#!/bin/bash -e +# +# This file serves as the default command to the openshift-ansible image. +# Runs a playbook with inventory as specified by environment variables. +# +# For more information see the documentation: +# https://github.com/openshift/openshift-ansible/blob/master/README_CONTAINER_IMAGE.md + +# SOURCE and HOME DIRECTORY: /opt/app-root/src + +if [[ -z "${PLAYBOOK_FILE}" ]]; then + echo + echo "PLAYBOOK_FILE must be provided." + exec /usr/local/bin/usage +fi + +INVENTORY="$(mktemp)" +if [[ -v INVENTORY_FILE ]]; then + # Make a copy so that ALLOW_ANSIBLE_CONNECTION_LOCAL below + # does not attempt to modify the original + cp -a ${INVENTORY_FILE} ${INVENTORY} +elif [[ -v INVENTORY_URL ]]; then + curl -o ${INVENTORY} ${INVENTORY_URL} +elif [[ -v DYNAMIC_SCRIPT_URL ]]; then + curl -o ${INVENTORY} ${DYNAMIC_SCRIPT_URL} + chmod 755 ${INVENTORY} +else + echo + echo "One of INVENTORY_FILE, INVENTORY_URL or DYNAMIC_SCRIPT_URL must be provided." + exec /usr/local/bin/usage +fi +INVENTORY_ARG="-i ${INVENTORY}" + +if [[ "$ALLOW_ANSIBLE_CONNECTION_LOCAL" = false ]]; then + sed -i s/ansible_connection=local// ${INVENTORY} +fi + +if [[ -v VAULT_PASS ]]; then + VAULT_PASS_FILE=.vaultpass + echo ${VAULT_PASS} > ${VAULT_PASS_FILE} + VAULT_PASS_ARG="--vault-password-file ${VAULT_PASS_FILE}" +fi + +cd ${WORK_DIR} + +exec ansible-playbook ${INVENTORY_ARG} ${VAULT_PASS_ARG} ${OPTS} ${PLAYBOOK_FILE} diff --git a/images/installer/system-container/root/usr/local/bin/run-system-container.sh b/images/installer/root/usr/local/bin/run-system-container.sh index 9ce7c7328..9ce7c7328 100755 --- a/images/installer/system-container/root/usr/local/bin/run-system-container.sh +++ b/images/installer/root/usr/local/bin/run-system-container.sh diff --git a/images/installer/root/usr/local/bin/usage b/images/installer/root/usr/local/bin/usage new file mode 100755 index 000000000..3518d7f19 --- /dev/null +++ b/images/installer/root/usr/local/bin/usage @@ -0,0 +1,33 @@ +#!/bin/bash -e +cat <<"EOF" + +The origin-ansible image provides several options to control the behaviour of the containers. +For more details on these options see the documentation: + + https://github.com/openshift/openshift-ansible/blob/master/README_CONTAINER_IMAGE.md + +At a minimum, when running a container using this image you must provide: + +* ssh keys so that Ansible can reach your hosts. These should be mounted as a volume under + /opt/app-root/src/.ssh +* An inventory file. This can be mounted inside the container as a volume and specified with the + INVENTORY_FILE environment variable. Alternatively you can serve the inventory file from a web + server and use the INVENTORY_URL environment variable to fetch it. +* The playbook to run. This is set using the PLAYBOOK_FILE environment variable. + +Here is an example of how to run a containerized origin-ansible with +the openshift_facts playbook, which collects and displays facts about your +OpenShift environment. The inventory and ssh keys are mounted as volumes +(the latter requires setting the uid in the container and SELinux label +in the key file via :Z so they can be accessed) and the PLAYBOOK_FILE +environment variable is set to point to the playbook within the image: + +docker run -tu `id -u` \ + -v $HOME/.ssh/id_rsa:/opt/app-root/src/.ssh/id_rsa:Z,ro \ + -v /etc/ansible/hosts:/tmp/inventory:Z,ro \ + -e INVENTORY_FILE=/tmp/inventory \ + -e OPTS="-v" \ + -e PLAYBOOK_FILE=playbooks/byo/openshift_facts.yml \ + openshift/origin-ansible + +EOF diff --git a/images/installer/root/usr/local/bin/usage.ocp b/images/installer/root/usr/local/bin/usage.ocp new file mode 100755 index 000000000..50593af6e --- /dev/null +++ b/images/installer/root/usr/local/bin/usage.ocp @@ -0,0 +1,33 @@ +#!/bin/bash -e +cat <<"EOF" + +The ose-ansible image provides several options to control the behaviour of the containers. +For more details on these options see the documentation: + + https://github.com/openshift/openshift-ansible/blob/master/README_CONTAINER_IMAGE.md + +At a minimum, when running a container using this image you must provide: + +* ssh keys so that Ansible can reach your hosts. These should be mounted as a volume under + /opt/app-root/src/.ssh +* An inventory file. This can be mounted inside the container as a volume and specified with the + INVENTORY_FILE environment variable. Alternatively you can serve the inventory file from a web + server and use the INVENTORY_URL environment variable to fetch it. +* The playbook to run. This is set using the PLAYBOOK_FILE environment variable. + +Here is an example of how to run a containerized ose-ansible with +the openshift_facts playbook, which collects and displays facts about your +OpenShift environment. The inventory and ssh keys are mounted as volumes +(the latter requires setting the uid in the container and SELinux label +in the key file via :Z so they can be accessed) and the PLAYBOOK_FILE +environment variable is set to point to the playbook within the image: + +docker run -tu `id -u` \ + -v $HOME/.ssh/id_rsa:/opt/app-root/src/.ssh/id_rsa:Z,ro \ + -v /etc/ansible/hosts:/tmp/inventory:Z,ro \ + -e INVENTORY_FILE=/tmp/inventory \ + -e OPTS="-v" \ + -e PLAYBOOK_FILE=playbooks/byo/openshift_facts.yml \ + openshift3/ose-ansible + +EOF diff --git a/images/installer/root/usr/local/bin/user_setup b/images/installer/root/usr/local/bin/user_setup new file mode 100755 index 000000000..b76e60a4d --- /dev/null +++ b/images/installer/root/usr/local/bin/user_setup @@ -0,0 +1,17 @@ +#!/bin/sh +set -x + +# ensure $HOME exists and is accessible by group 0 (we don't know what the runtime UID will be) +mkdir -p ${HOME} +chown ${USER_UID}:0 ${HOME} +chmod ug+rwx ${HOME} + +# runtime user will need to be able to self-insert in /etc/passwd +chmod g+rw /etc/passwd + +# ensure that the ansible content is accessible +chmod -R g+r ${WORK_DIR} +find ${WORK_DIR} -type d -exec chmod g+x {} + + +# no need for this script to remain in the image after running +rm $0 diff --git a/inventory/byo/hosts.byo.glusterfs.external.example b/inventory/byo/hosts.byo.glusterfs.external.example new file mode 100644 index 000000000..5a284ce97 --- /dev/null +++ b/inventory/byo/hosts.byo.glusterfs.external.example @@ -0,0 +1,56 @@ +# This is an example of a bring your own (byo) host inventory for a cluster +# with natively hosted, containerized GlusterFS storage. +# +# This inventory may be used with the byo/config.yml playbook to deploy a new +# cluster with GlusterFS storage, which will use that storage to create a +# volume that will provide backend storage for a hosted Docker registry. +# +# This inventory may also be used with byo/openshift-glusterfs/config.yml to +# deploy GlusterFS storage on an existing cluster. With this playbook, the +# registry backend volume will be created but the administrator must then +# either deploy a hosted registry or change an existing hosted registry to use +# that volume. +# +# There are additional configuration parameters that can be specified to +# control the deployment and state of a GlusterFS cluster. Please see the +# documentation in playbooks/byo/openshift-glusterfs/README.md and +# roles/openshift_storage_glusterfs/README.md for additional details. + +[OSEv3:children] +masters +nodes +# Specify there will be GlusterFS nodes +glusterfs + +[OSEv3:vars] +ansible_ssh_user=root +openshift_deployment_type=origin +# Specify that we want to use an external GlusterFS cluster +openshift_storage_glusterfs_is_native=False +# Specify the IP address or hostname of the external heketi service +openshift_storage_glusterfs_heketi_url=172.0.0.1 + +[masters] +master + +[nodes] +master openshift_schedulable=False +node0 openshift_schedulable=True +node1 openshift_schedulable=True +node2 openshift_schedulable=True + +# Specify the glusterfs group, which contains the nodes of the external +# GlusterFS cluster. At a minimum, each node must have "glusterfs_hostname" +# and "glusterfs_devices" variables defined. +# +# The first variable indicates the hostname of the external GLusterFS node, +# and must be reachable by the external heketi service. +# +# The second variable is a list of block devices the node will have access to +# that are intended solely for use as GlusterFS storage. These block devices +# must be bare (e.g. have no data, not be marked as LVM PVs), and will be +# formatted. +[glusterfs] +node0.local glusterfs_ip='172.0.0.10' glusterfs_devices='[ "/dev/vdb" ]' +node1.local glusterfs_ip='172.0.0.11' glusterfs_devices='[ "/dev/vdb", "/dev/vdc" ]' +node2.local glusterfs_ip='172.0.0.11' glusterfs_devices='[ "/dev/vdd" ]' diff --git a/inventory/byo/hosts.byo.glusterfs.mixed.example b/inventory/byo/hosts.byo.glusterfs.mixed.example new file mode 100644 index 000000000..d16df6470 --- /dev/null +++ b/inventory/byo/hosts.byo.glusterfs.mixed.example @@ -0,0 +1,59 @@ +# This is an example of a bring your own (byo) host inventory for a cluster +# with natively hosted, containerized GlusterFS storage. +# +# This inventory may be used with the byo/config.yml playbook to deploy a new +# cluster with GlusterFS storage, which will use that storage to create a +# volume that will provide backend storage for a hosted Docker registry. +# +# This inventory may also be used with byo/openshift-glusterfs/config.yml to +# deploy GlusterFS storage on an existing cluster. With this playbook, the +# registry backend volume will be created but the administrator must then +# either deploy a hosted registry or change an existing hosted registry to use +# that volume. +# +# There are additional configuration parameters that can be specified to +# control the deployment and state of a GlusterFS cluster. Please see the +# documentation in playbooks/byo/openshift-glusterfs/README.md and +# roles/openshift_storage_glusterfs/README.md for additional details. + +[OSEv3:children] +masters +nodes +# Specify there will be GlusterFS nodes +glusterfs + +[OSEv3:vars] +ansible_ssh_user=root +openshift_deployment_type=origin +# Specify that we want to use an external GlusterFS cluster and a native +# heketi service +openshift_storage_glusterfs_is_native=False +openshift_storage_glusterfs_heketi_is_native=True +# Specify that heketi will use SSH to communicate to the GlusterFS nodes and +# the private key file it will use for authentication +openshift_storage_glusterfs_heketi_executor=ssh +openshift_storage_glusterfs_heketi_ssh_keyfile=/root/id_rsa +[masters] +master + +[nodes] +master openshift_schedulable=False +node0 openshift_schedulable=True +node1 openshift_schedulable=True +node2 openshift_schedulable=True + +# Specify the glusterfs group, which contains the nodes of the external +# GlusterFS cluster. At a minimum, each node must have "glusterfs_hostname" +# and "glusterfs_devices" variables defined. +# +# The first variable indicates the hostname of the external GLusterFS node, +# and must be reachable by the external heketi service. +# +# The second variable is a list of block devices the node will have access to +# that are intended solely for use as GlusterFS storage. These block devices +# must be bare (e.g. have no data, not be marked as LVM PVs), and will be +# formatted. +[glusterfs] +node0.local glusterfs_ip='172.0.0.10' glusterfs_devices='[ "/dev/vdb" ]' +node1.local glusterfs_ip='172.0.0.11' glusterfs_devices='[ "/dev/vdb", "/dev/vdc" ]' +node2.local glusterfs_ip='172.0.0.11' glusterfs_devices='[ "/dev/vdd" ]' diff --git a/inventory/byo/hosts.byo.glusterfs.native.example b/inventory/byo/hosts.byo.glusterfs.native.example new file mode 100644 index 000000000..c1a1f6f84 --- /dev/null +++ b/inventory/byo/hosts.byo.glusterfs.native.example @@ -0,0 +1,46 @@ +# This is an example of a bring your own (byo) host inventory for a cluster +# with natively hosted, containerized GlusterFS storage for applications. It +# will also autmatically create a StorageClass for this purpose. +# +# This inventory may be used with the byo/config.yml playbook to deploy a new +# cluster with GlusterFS storage. +# +# This inventory may also be used with byo/openshift-glusterfs/config.yml to +# deploy GlusterFS storage on an existing cluster. +# +# There are additional configuration parameters that can be specified to +# control the deployment and state of a GlusterFS cluster. Please see the +# documentation in playbooks/byo/openshift-glusterfs/README.md and +# roles/openshift_storage_glusterfs/README.md for additional details. + +[OSEv3:children] +masters +nodes +# Specify there will be GlusterFS nodes +glusterfs + +[OSEv3:vars] +ansible_ssh_user=root +openshift_deployment_type=origin + +[masters] +master + +[nodes] +master openshift_schedulable=False +# A hosted registry, by default, will only be deployed on nodes labeled +# "region=infra". +node0 openshift_schedulable=True +node1 openshift_schedulable=True +node2 openshift_schedulable=True + +# Specify the glusterfs group, which contains the nodes that will host +# GlusterFS storage pods. At a minimum, each node must have a +# "glusterfs_devices" variable defined. This variable is a list of block +# devices the node will have access to that is intended solely for use as +# GlusterFS storage. These block devices must be bare (e.g. have no data, not +# be marked as LVM PVs), and will be formatted. +[glusterfs] +node0 glusterfs_devices='[ "/dev/vdb", "/dev/vdc", "/dev/vdd" ]' +node1 glusterfs_devices='[ "/dev/vdb", "/dev/vdc", "/dev/vdd" ]' +node2 glusterfs_devices='[ "/dev/vdb", "/dev/vdc", "/dev/vdd" ]' diff --git a/inventory/byo/hosts.byo.native-glusterfs.example b/inventory/byo/hosts.byo.glusterfs.registry-only.example index dc847a5b2..31a85ee42 100644 --- a/inventory/byo/hosts.byo.native-glusterfs.example +++ b/inventory/byo/hosts.byo.glusterfs.registry-only.example @@ -1,11 +1,12 @@ # This is an example of a bring your own (byo) host inventory for a cluster -# with natively hosted, containerized GlusterFS storage. +# with natively hosted, containerized GlusterFS storage for exclusive use +# as storage for a natively hosted Docker registry. # # This inventory may be used with the byo/config.yml playbook to deploy a new # cluster with GlusterFS storage, which will use that storage to create a # volume that will provide backend storage for a hosted Docker registry. # -# This inventory may also be used with byo/openshift-glusterfs/config.yml to +# This inventory may also be used with byo/openshift-glusterfs/registry.yml to # deploy GlusterFS storage on an existing cluster. With this playbook, the # registry backend volume will be created but the administrator must then # either deploy a hosted registry or change an existing hosted registry to use @@ -20,7 +21,7 @@ masters nodes # Specify there will be GlusterFS nodes -glusterfs +glusterfs_registry [OSEv3:vars] ansible_ssh_user=root @@ -29,15 +30,15 @@ openshift_deployment_type=origin openshift_hosted_registry_storage_kind=glusterfs [masters] -master node=True storage=True master=True +master [nodes] -master node=True storage=True master=True openshift_schedulable=False +master openshift_schedulable=False # A hosted registry, by default, will only be deployed on nodes labeled # "region=infra". -node0 node=True openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True -node1 node=True openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True -node2 node=True openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True +node0 openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True +node1 openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True +node2 openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True # Specify the glusterfs group, which contains the nodes that will host # GlusterFS storage pods. At a minimum, each node must have a @@ -45,7 +46,7 @@ node2 node=True openshift_node_labels="{'region': 'infra'}" openshift_schedula # devices the node will have access to that is intended solely for use as # GlusterFS storage. These block devices must be bare (e.g. have no data, not # be marked as LVM PVs), and will be formatted. -[glusterfs] +[glusterfs_registry] node0 glusterfs_devices='[ "/dev/vdb", "/dev/vdc", "/dev/vdd" ]' node1 glusterfs_devices='[ "/dev/vdb", "/dev/vdc", "/dev/vdd" ]' node2 glusterfs_devices='[ "/dev/vdb", "/dev/vdc", "/dev/vdd" ]' diff --git a/inventory/byo/hosts.byo.glusterfs.storage-and-registry.example b/inventory/byo/hosts.byo.glusterfs.storage-and-registry.example new file mode 100644 index 000000000..54bd89ddc --- /dev/null +++ b/inventory/byo/hosts.byo.glusterfs.storage-and-registry.example @@ -0,0 +1,63 @@ +# This is an example of a bring your own (byo) host inventory for a cluster +# with natively hosted, containerized GlusterFS storage for both general +# application use and a natively hosted Docker registry. It will also create a +# StorageClass for the general storage. +# +# This inventory may be used with the byo/config.yml playbook to deploy a new +# cluster with GlusterFS storage. +# +# This inventory may also be used with byo/openshift-glusterfs/config.yml to +# deploy GlusterFS storage on an existing cluster. With this playbook, the +# registry backend volume will be created but the administrator must then +# either deploy a hosted registry or change an existing hosted registry to use +# that volume. +# +# There are additional configuration parameters that can be specified to +# control the deployment and state of a GlusterFS cluster. Please see the +# documentation in playbooks/byo/openshift-glusterfs/README.md and +# roles/openshift_storage_glusterfs/README.md for additional details. + +[OSEv3:children] +masters +nodes +# Specify there will be GlusterFS nodes +glusterfs +glusterfs_registry + +[OSEv3:vars] +ansible_ssh_user=root +openshift_deployment_type=origin +# Specify that we want to use GlusterFS storage for a hosted registry +openshift_hosted_registry_storage_kind=glusterfs + +[masters] +master + +[nodes] +master openshift_schedulable=False +# It is recommended to not use a single cluster for both general and registry +# storage, so two three-node clusters will be required. +node0 openshift_schedulable=True +node1 openshift_schedulable=True +node2 openshift_schedulable=True +# A hosted registry, by default, will only be deployed on nodes labeled +# "region=infra". +node3 openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True +node4 openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True +node5 openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True + +# Specify the glusterfs group, which contains the nodes that will host +# GlusterFS storage pods. At a minimum, each node must have a +# "glusterfs_devices" variable defined. This variable is a list of block +# devices the node will have access to that is intended solely for use as +# GlusterFS storage. These block devices must be bare (e.g. have no data, not +# be marked as LVM PVs), and will be formatted. +[glusterfs] +node0 glusterfs_devices='[ "/dev/vdb", "/dev/vdc", "/dev/vdd" ]' +node1 glusterfs_devices='[ "/dev/vdb", "/dev/vdc", "/dev/vdd" ]' +node2 glusterfs_devices='[ "/dev/vdb", "/dev/vdc", "/dev/vdd" ]' + +[glusterfs_registry] +node3 glusterfs_devices='[ "/dev/vdb", "/dev/vdc", "/dev/vdd" ]' +node4 glusterfs_devices='[ "/dev/vdb", "/dev/vdc", "/dev/vdd" ]' +node5 glusterfs_devices='[ "/dev/vdb", "/dev/vdc", "/dev/vdd" ]' diff --git a/inventory/byo/hosts.origin.example b/inventory/byo/hosts.origin.example index e6bc6c829..de7493f71 100644 --- a/inventory/byo/hosts.origin.example +++ b/inventory/byo/hosts.origin.example @@ -113,6 +113,11 @@ openshift_release=v3.6 # Downgrades are not supported and will error out. Be careful when upgrading docker from < 1.10 to > 1.10. # docker_version="1.12.1" +# Specify whether to run Docker daemon with SELinux enabled in containers. Default is True. +# Uncomment below to disable; for example if your kernel does not support the +# Docker overlay/overlay2 storage drivers with SELinux enabled. +#openshift_docker_selinux_enabled=False + # Skip upgrading Docker during an OpenShift upgrade, leaves the current Docker version alone. # docker_upgrade=False @@ -151,8 +156,9 @@ openshift_release=v3.6 # modify image streams to point at that registry by setting the following to true #openshift_examples_modify_imagestreams=true -# Origin copr repo +# OpenShift repository configuration #openshift_additional_repos=[{'id': 'openshift-origin-copr', 'name': 'OpenShift Origin COPR', 'baseurl': 'https://copr-be.cloud.fedoraproject.org/results/maxamillion/origin-next/epel-7-$basearch/', 'enabled': 1, 'gpgcheck': 1, 'gpgkey': 'https://copr-be.cloud.fedoraproject.org/results/maxamillion/origin-next/pubkey.gpg'}] +#openshift_repos_enable_testing=false # htpasswd auth openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider', 'filename': '/etc/origin/master/htpasswd'}] @@ -468,6 +474,8 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # S3 bucket must already exist. #openshift_hosted_registry_storage_kind=object #openshift_hosted_registry_storage_provider=s3 +#openshift_hosted_registry_storage_s3_encrypt=false +#openshift_hosted_registry_storage_s3_kmskeyid=aws_kms_key_id #openshift_hosted_registry_storage_s3_accesskey=aws_access_key_id #openshift_hosted_registry_storage_s3_secretkey=aws_secret_access_key #openshift_hosted_registry_storage_s3_bucket=bucket_name @@ -552,6 +560,11 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # Configure the prefix and version for the component images #openshift_hosted_metrics_deployer_prefix=docker.io/openshift/origin- #openshift_hosted_metrics_deployer_version=3.6.0 +# +# StorageClass +# openshift_storageclass_name=gp2 +# openshift_storageclass_parameters={'type': 'gp2', 'encrypted': false} +# # Logging deployment # @@ -714,7 +727,7 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', #openshift_dns_ip=172.30.0.1 # Configure node kubelet arguments. pods-per-core is valid in OpenShift Origin 1.3 or OpenShift Container Platform 3.3 and later. -#openshift_node_kubelet_args={'pods-per-core': ['10'], 'max-pods': ['250'], 'image-gc-high-threshold': ['90'], 'image-gc-low-threshold': ['80']} +#openshift_node_kubelet_args={'pods-per-core': ['10'], 'max-pods': ['250'], 'image-gc-high-threshold': ['85'], 'image-gc-low-threshold': ['80']} # Configure logrotate scripts # See: https://github.com/nickhammond/ansible-logrotate @@ -740,6 +753,10 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # Global Proxy Configuration # These options configure HTTP_PROXY, HTTPS_PROXY, and NOPROXY environment # variables for docker and master services. +# +# Hosts in the openshift_no_proxy list will NOT use any globally +# configured HTTP(S)_PROXYs. openshift_no_proxy accepts domains +# (.example.com), and hosts (example.com), and IP addresses. #openshift_http_proxy=http://USER:PASSWORD@IPADDR:PORT #openshift_https_proxy=https://USER:PASSWORD@IPADDR:PORT #openshift_no_proxy='.hosts.example.com,some-host.com' @@ -747,7 +764,11 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # Most environments don't require a proxy between openshift masters, nodes, and # etcd hosts. So automatically add those hostnames to the openshift_no_proxy list. # If all of your hosts share a common domain you may wish to disable this and -# specify that domain above. +# specify that domain above instead. +# +# For example, having hosts with FQDNs: m1.ex.com, n1.ex.com, and +# n2.ex.com, one would simply add '.ex.com' to the openshift_no_proxy +# variable (above) and set this value to False #openshift_generate_no_proxy_hosts=True # # These options configure the BuildDefaults admission controller which injects @@ -862,6 +883,14 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # openshift_upgrade_nodes_serial=4 openshift_upgrade_nodes_max_fail_percentage=49 # where as this would not # openshift_upgrade_nodes_serial=4 openshift_upgrade_nodes_max_fail_percentage=50 +# +# Multiple data migrations take place and if they fail they will fail the upgrade +# You may wish to disable these or make them non fatal +# +# openshift_upgrade_pre_storage_migration_enabled=true +# openshift_upgrade_pre_storage_migration_fatal==true +# openshift_upgrade_post_storage_migration_enabled=true +# openshift_upgrade_post_storage_migration_fatal==false # host group for masters [masters] diff --git a/inventory/byo/hosts.ose.example b/inventory/byo/hosts.ose.example index 928da40fa..62a364e0d 100644 --- a/inventory/byo/hosts.ose.example +++ b/inventory/byo/hosts.ose.example @@ -109,6 +109,11 @@ openshift_release=v3.6 # Default value: "--log-driver=journald" #openshift_docker_options="-l warn --ipv6=false" +# Specify whether to run Docker daemon with SELinux enabled in containers. Default is True. +# Uncomment below to disable; for example if your kernel does not support the +# Docker overlay/overlay2 storage drivers with SELinux enabled. +#openshift_docker_selinux_enabled=False + # Specify exact version of Docker to configure or upgrade to. # Downgrades are not supported and will error out. Be careful when upgrading docker from < 1.10 to > 1.10. # docker_version="1.12.1" @@ -150,8 +155,9 @@ openshift_release=v3.6 # modify image streams to point at that registry by setting the following to true #openshift_examples_modify_imagestreams=true -# Additional yum repos to install +# OpenShift repository configuration #openshift_additional_repos=[{'id': 'ose-devel', 'name': 'ose-devel', 'baseurl': 'http://example.com/puddle/build/AtomicOpenShift/3.1/latest/RH7-RHOSE-3.0/$basearch/os', 'enabled': 1, 'gpgcheck': 0}] +#openshift_repos_enable_testing=false # htpasswd auth openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider', 'filename': '/etc/origin/master/htpasswd'}] @@ -468,6 +474,8 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # S3 bucket must already exist. #openshift_hosted_registry_storage_kind=object #openshift_hosted_registry_storage_provider=s3 +#openshift_hosted_registry_storage_s3_encrypt=false +#openshift_hosted_registry_storage_s3_kmskeyid=aws_kms_key_id #openshift_hosted_registry_storage_s3_accesskey=aws_access_key_id #openshift_hosted_registry_storage_s3_secretkey=aws_secret_access_key #openshift_hosted_registry_storage_s3_bucket=bucket_name @@ -552,6 +560,11 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # Configure the prefix and version for the component images #openshift_hosted_metrics_deployer_prefix=registry.example.com:8888/openshift3/ #openshift_hosted_metrics_deployer_version=3.6.0 +# +# StorageClass +# openshift_storageclass_name=gp2 +# openshift_storageclass_parameters={'type': 'gp2', 'encrypted': false} +# # Logging deployment # @@ -714,7 +727,7 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', #openshift_dns_ip=172.30.0.1 # Configure node kubelet arguments. pods-per-core is valid in OpenShift Origin 1.3 or OpenShift Container Platform 3.3 and later. -#openshift_node_kubelet_args={'pods-per-core': ['10'], 'max-pods': ['250'], 'image-gc-high-threshold': ['90'], 'image-gc-low-threshold': ['80']} +#openshift_node_kubelet_args={'pods-per-core': ['10'], 'max-pods': ['250'], 'image-gc-high-threshold': ['85'], 'image-gc-low-threshold': ['80']} # Configure logrotate scripts # See: https://github.com/nickhammond/ansible-logrotate @@ -740,6 +753,10 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # Global Proxy Configuration # These options configure HTTP_PROXY, HTTPS_PROXY, and NOPROXY environment # variables for docker and master services. +# +# Hosts in the openshift_no_proxy list will NOT use any globally +# configured HTTP(S)_PROXYs. openshift_no_proxy accepts domains +# (.example.com), hosts (example.com), and IP addresses. #openshift_http_proxy=http://USER:PASSWORD@IPADDR:PORT #openshift_https_proxy=https://USER:PASSWORD@IPADDR:PORT #openshift_no_proxy='.hosts.example.com,some-host.com' @@ -747,7 +764,11 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # Most environments don't require a proxy between openshift masters, nodes, and # etcd hosts. So automatically add those hostnames to the openshift_no_proxy list. # If all of your hosts share a common domain you may wish to disable this and -# specify that domain above. +# specify that domain above instead. +# +# For example, having hosts with FQDNs: m1.ex.com, n1.ex.com, and +# n2.ex.com, one would simply add '.ex.com' to the openshift_no_proxy +# variable (above) and set this value to False #openshift_generate_no_proxy_hosts=True # # These options configure the BuildDefaults admission controller which injects @@ -858,6 +879,14 @@ openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', # openshift_upgrade_nodes_serial=4 openshift_upgrade_nodes_max_fail_percentage=49 # where as this would not # openshift_upgrade_nodes_serial=4 openshift_upgrade_nodes_max_fail_percentage=50 +# +# Multiple data migrations take place and if they fail they will fail the upgrade +# You may wish to disable these or make them non fatal +# +# openshift_upgrade_pre_storage_migration_enabled=true +# openshift_upgrade_pre_storage_migration_fatal==true +# openshift_upgrade_post_storage_migration_enabled=true +# openshift_upgrade_post_storage_migration_fatal==false # host group for masters [masters] diff --git a/openshift-ansible.spec b/openshift-ansible.spec index d36edad5f..3a59ab25b 100644 --- a/openshift-ansible.spec +++ b/openshift-ansible.spec @@ -9,7 +9,7 @@ %global __requires_exclude ^/usr/bin/ansible-playbook$ Name: openshift-ansible -Version: 3.6.143 +Version: 3.6.153 Release: 1%{?dist} Summary: Openshift and Atomic Enterprise Ansible License: ASL 2.0 @@ -280,6 +280,49 @@ Atomic OpenShift Utilities includes %changelog +* Tue Jul 18 2017 Jenkins CD Merge Bot <smunilla@redhat.com> 3.6.153-1 +- Updating to compare sets instead of sorted lists (ewolinet@redhat.com) +- Adding ability to create podpreset for service-catalog-controller for + bz1471881 (ewolinet@redhat.com) +- Updating to use oc replace and conditionally update edit and admin roles + (ewolinet@redhat.com) +- Other playbooks maybe expecting this to be at least an empty string. I think + they default it to an empty list if its not found. (tbielawa@redhat.com) +- Fix NO_PROXY environment variable setting (tbielawa@redhat.com) +- Changing the passing of data for sc creation. (kwoodson@redhat.com) +- Fixed variable name. (kwoodson@redhat.com) +- Adding disk encryption to storageclasses and to openshift registry + (kwoodson@redhat.com) + +* Mon Jul 17 2017 Jenkins CD Merge Bot <smunilla@redhat.com> 3.6.152-1 +- + +* Sun Jul 16 2017 Jenkins CD Merge Bot <smunilla@redhat.com> 3.6.151-1 +- + +* Sun Jul 16 2017 Jenkins CD Merge Bot <smunilla@redhat.com> 3.6.150-1 +- + +* Sat Jul 15 2017 Jenkins CD Merge Bot <smunilla@redhat.com> 3.6.149-1 +- Config was missed before replace. (jkaur@redhat.com) +- Redeploy-certificates will fail for registry and router if user is not + system:admin (jkaur@redhat.com) + +* Fri Jul 14 2017 Jenkins CD Merge Bot <smunilla@redhat.com> 3.6.148-1 +- Adding in permissions to edit and admin cluster roles (ewolinet@redhat.com) +- making kube-service-catalog project network global when using redhat + /openshift-ovs-multitenant plugin (ewolinet@redhat.com) +- set KillMode to process in node service file (jchaloup@redhat.com) +- Upgrade fails when "Drain Node for Kubelet upgrade" (jkaur@redhat.com) +- openvswitch, syscontainer: specify the Docker service name + (gscrivan@redhat.com) + +* Thu Jul 13 2017 Jenkins CD Merge Bot <smunilla@redhat.com> 3.6.144-1 +- Created js file for enabling tech preview for console, updated master-config + for pod presets and console tech preview (ewolinet@redhat.com) +- GlusterFS: Add updated example hosts files (jarrpa@redhat.com) +- GlusterFS: Fix SSH-based heketi configuration (jarrpa@redhat.com) + * Wed Jul 12 2017 Jenkins CD Merge Bot <smunilla@redhat.com> 3.6.143-1 - diff --git a/playbooks/adhoc/uninstall.yml b/playbooks/adhoc/uninstall.yml index a1f541712..58b3a7835 100644 --- a/playbooks/adhoc/uninstall.yml +++ b/playbooks/adhoc/uninstall.yml @@ -325,6 +325,10 @@ service: name=docker state=restarted failed_when: false when: not (container_engine | changed) + register: l_docker_restart_docker_in_pb_result + until: not l_docker_restart_docker_in_pb_result | failed + retries: 3 + delay: 30 - hosts: masters become: yes diff --git a/playbooks/aws/openshift-cluster/config.yml b/playbooks/aws/openshift-cluster/config.yml index 8d64b0521..821a0f30e 100644 --- a/playbooks/aws/openshift-cluster/config.yml +++ b/playbooks/aws/openshift-cluster/config.yml @@ -35,4 +35,3 @@ openshift_use_flannel: "{{ lookup('oo_option', 'use_flannel') }}" openshift_use_calico: "{{ lookup('oo_option', 'use_calico') }}" openshift_use_fluentd: "{{ lookup('oo_option', 'use_fluentd') }}" - openshift_use_dnsmasq: false diff --git a/playbooks/byo/openshift-checks/README.md b/playbooks/byo/openshift-checks/README.md index 4b2ff1f94..f0f14b268 100644 --- a/playbooks/byo/openshift-checks/README.md +++ b/playbooks/byo/openshift-checks/README.md @@ -39,7 +39,9 @@ against your inventory file. Here is the step-by-step: $ cd openshift-ansible ``` -2. Run the appropriate playbook: +2. Install the [dependencies](../../../README.md#setup) + +3. Run the appropriate playbook: ```console $ ansible-playbook -i <inventory file> playbooks/byo/openshift-checks/pre-install.yml @@ -57,9 +59,8 @@ against your inventory file. Here is the step-by-step: $ ansible-playbook -i <inventory file> playbooks/byo/openshift-checks/certificate_expiry/default.yaml -v ``` -## Running via Docker image +## Running in a container This repository is built into a Docker image including Ansible so that it can -be run anywhere Docker is available. Instructions for doing so may be found -[in the README](../../README_CONTAINER_IMAGE.md). - +be run anywhere Docker is available, without the need to manually install dependencies. +Instructions for doing so may be found [in the README](../../../README_CONTAINER_IMAGE.md). diff --git a/playbooks/common/openshift-cluster/openshift_hosted.yml b/playbooks/common/openshift-cluster/openshift_hosted.yml index 8d94b6509..ce7f981ab 100644 --- a/playbooks/common/openshift-cluster/openshift_hosted.yml +++ b/playbooks/common/openshift-cluster/openshift_hosted.yml @@ -26,6 +26,8 @@ logging_elasticsearch_cluster_size: "{{ openshift_hosted_logging_elasticsearch_cluster_size | default(1) }}" logging_elasticsearch_ops_cluster_size: "{{ openshift_hosted_logging_elasticsearch_ops_cluster_size | default(1) }}" roles: + - role: openshift_default_storage_class + when: openshift_cloudprovider_kind is defined and (openshift_cloudprovider_kind == 'aws' or openshift_cloudprovider_kind == 'gce') - role: openshift_hosted - role: openshift_metrics when: openshift_hosted_metrics_deploy | default(false) | bool @@ -45,8 +47,6 @@ - role: cockpit-ui when: ( openshift.common.version_gte_3_3_or_1_3 | bool ) and ( openshift_hosted_manage_registry | default(true) | bool ) and not (openshift.docker.hosted_registry_insecure | default(false) | bool) - - role: openshift_default_storage_class - when: openshift_cloudprovider_kind is defined and (openshift_cloudprovider_kind == 'aws' or openshift_cloudprovider_kind == 'gce') - name: Update master-config for publicLoggingURL hosts: oo_masters_to_config:!oo_first_master diff --git a/playbooks/common/openshift-cluster/redeploy-certificates/registry.yml b/playbooks/common/openshift-cluster/redeploy-certificates/registry.yml index 8c8062585..afd5463b2 100644 --- a/playbooks/common/openshift-cluster/redeploy-certificates/registry.yml +++ b/playbooks/common/openshift-cluster/redeploy-certificates/registry.yml @@ -66,6 +66,7 @@ --signer-cert={{ openshift.common.config_base }}/master/ca.crt --signer-key={{ openshift.common.config_base }}/master/ca.key --signer-serial={{ openshift.common.config_base }}/master/ca.serial.txt + --config={{ mktemp.stdout }}/admin.kubeconfig --hostnames="{{ docker_registry_service_ip.results.clusterip }},docker-registry.default.svc,docker-registry.default.svc.cluster.local,{{ docker_registry_route_hostname }}" --cert={{ openshift.common.config_base }}/master/registry.crt --key={{ openshift.common.config_base }}/master/registry.key diff --git a/playbooks/common/openshift-cluster/redeploy-certificates/router.yml b/playbooks/common/openshift-cluster/redeploy-certificates/router.yml index 9f14f2d69..748bbbf91 100644 --- a/playbooks/common/openshift-cluster/redeploy-certificates/router.yml +++ b/playbooks/common/openshift-cluster/redeploy-certificates/router.yml @@ -116,8 +116,9 @@ tls.crt="{{ mktemp.stdout }}/openshift-hosted-router-certificate.pem" tls.key="{{ mktemp.stdout }}/openshift-hosted-router-certificate.key" --type=kubernetes.io/tls + --config={{ mktemp.stdout }}/admin.kubeconfig --confirm - -o json | {{ openshift.common.client_binary }} replace -f - + -o json | {{ openshift.common.client_binary }} --config={{ mktemp.stdout }}/admin.kubeconfig replace -f - - name: Remove temporary router certificate and key files file: diff --git a/playbooks/common/openshift-cluster/upgrades/docker/docker_upgrade.yml b/playbooks/common/openshift-cluster/upgrades/docker/docker_upgrade.yml index 07db071ce..02b8a9d3c 100644 --- a/playbooks/common/openshift-cluster/upgrades/docker/docker_upgrade.yml +++ b/playbooks/common/openshift-cluster/upgrades/docker/docker_upgrade.yml @@ -52,9 +52,13 @@ - name: Drain Node for Kubelet upgrade command: > - {{ openshift.common.admin_binary }} drain {{ openshift.node.nodename }} --force --delete-local-data --ignore-daemonsets + {{ openshift.common.admin_binary }} drain {{ openshift.node.nodename }} --config={{ openshift.common.config_base }}/master/admin.kubeconfig --force --delete-local-data --ignore-daemonsets delegate_to: "{{ groups.oo_first_master.0 }}" when: l_docker_upgrade is defined and l_docker_upgrade | bool and inventory_hostname in groups.oo_nodes_to_upgrade + register: l_docker_upgrade_drain_result + until: not l_docker_upgrade_drain_result | failed + retries: 60 + delay: 60 - include: upgrade.yml when: l_docker_upgrade is defined and l_docker_upgrade | bool diff --git a/playbooks/common/openshift-cluster/upgrades/docker/restart.yml b/playbooks/common/openshift-cluster/upgrades/docker/restart.yml index 96c729d79..13313377e 100644 --- a/playbooks/common/openshift-cluster/upgrades/docker/restart.yml +++ b/playbooks/common/openshift-cluster/upgrades/docker/restart.yml @@ -1,6 +1,10 @@ --- - name: Restart docker service: name=docker state=restarted + register: l_docker_restart_docker_in_upgrade_result + until: not l_docker_restart_docker_in_upgrade_result | failed + retries: 3 + delay: 30 - name: Update docker facts openshift_facts: diff --git a/playbooks/common/openshift-cluster/upgrades/docker/upgrade.yml b/playbooks/common/openshift-cluster/upgrades/docker/upgrade.yml index 17f8fc6e9..35d000e49 100644 --- a/playbooks/common/openshift-cluster/upgrades/docker/upgrade.yml +++ b/playbooks/common/openshift-cluster/upgrades/docker/upgrade.yml @@ -32,7 +32,13 @@ - debug: var=docker_image_count.stdout when: docker_upgrade_nuke_images is defined and docker_upgrade_nuke_images | bool -- service: name=docker state=stopped +- service: + name: docker + state: stopped + register: l_pb_docker_upgrade_stop_result + until: not l_pb_docker_upgrade_stop_result | failed + retries: 3 + delay: 30 - name: Upgrade Docker package: name=docker{{ '-' + docker_version }} state=present diff --git a/playbooks/common/openshift-cluster/upgrades/upgrade_control_plane.yml b/playbooks/common/openshift-cluster/upgrades/upgrade_control_plane.yml index 86f2cfa54..695dc3140 100644 --- a/playbooks/common/openshift-cluster/upgrades/upgrade_control_plane.yml +++ b/playbooks/common/openshift-cluster/upgrades/upgrade_control_plane.yml @@ -12,6 +12,12 @@ command: > {{ openshift.common.client_binary }} adm --config={{ openshift.common.config_base }}/master/admin.kubeconfig migrate storage --include=* --confirm + register: l_pb_upgrade_control_plane_pre_upgrade_storage + when: openshift_upgrade_pre_storage_migration_enabled | default(true,true) | bool + failed_when: + - openshift_upgrade_pre_storage_migration_enabled | default(true,true) | bool + - l_pb_upgrade_control_plane_pre_upgrade_storage.rc != 0 + - openshift_upgrade_pre_storage_migration_fatal | default(true,true) | bool # If facts cache were for some reason deleted, this fact may not be set, and if not set # it will always default to true. This causes problems for the etcd data dir fact detection @@ -140,16 +146,21 @@ - include: "{{ openshift_master_upgrade_post_hook }}" when: openshift_master_upgrade_post_hook is defined - - set_fact: - master_update_complete: True - -- name: Post master upgrade - Upgrade clusterpolicies storage - hosts: oo_first_master - tasks: - - name: Upgrade clusterpolicies storage + - name: Post master upgrade - Upgrade clusterpolicies storage command: > {{ openshift.common.client_binary }} adm --config={{ openshift.common.config_base }}/master/admin.kubeconfig migrate storage --include=clusterpolicies --confirm + register: l_pb_upgrade_control_plane_post_upgrade_storage + when: openshift_upgrade_post_storage_migration_enabled | default(true,true) | bool + failed_when: + - openshift_upgrade_post_storage_migration_enabled | default(true,true) | bool + - l_pb_upgrade_control_plane_post_upgrade_storage.rc != 0 + - openshift_upgrade_post_storage_migration_fatal | default(false,true) | bool + run_once: true + delegate_to: oo_first_master + + - set_fact: + master_update_complete: True ############################################################################## # Gate on master update complete @@ -230,11 +241,17 @@ - reconcile_scc_result.rc == 0 run_once: true - - name: Upgrade job storage + - name: Migrate storage post policy reconciliation command: > {{ openshift.common.client_binary }} adm --config={{ openshift.common.config_base }}/master/admin.kubeconfig migrate storage --include=* --confirm run_once: true + register: l_pb_upgrade_control_plane_post_upgrade_storage + when: openshift_upgrade_post_storage_migration_enabled | default(true,true) | bool + failed_when: + - openshift_upgrade_post_storage_migration_enabled | default(true,true) | bool + - l_pb_upgrade_control_plane_post_upgrade_storage.rc != 0 + - openshift_upgrade_post_storage_migration_fatal | default(false,true) | bool - set_fact: reconcile_complete: True @@ -294,8 +311,12 @@ - name: Drain Node for Kubelet upgrade command: > - {{ hostvars[groups.oo_first_master.0].openshift.common.admin_binary }} drain {{ openshift.node.nodename | lower }} --force --delete-local-data --ignore-daemonsets + {{ hostvars[groups.oo_first_master.0].openshift.common.admin_binary }} drain {{ openshift.node.nodename | lower }} --config={{ openshift.common.config_base }}/master/admin.kubeconfig --force --delete-local-data --ignore-daemonsets delegate_to: "{{ groups.oo_first_master.0 }}" + register: l_upgrade_control_plane_drain_result + until: not l_upgrade_control_plane_drain_result | failed + retries: 60 + delay: 60 roles: - lib_openshift diff --git a/playbooks/common/openshift-cluster/upgrades/upgrade_nodes.yml b/playbooks/common/openshift-cluster/upgrades/upgrade_nodes.yml index 1d1e440d4..c93a5d89c 100644 --- a/playbooks/common/openshift-cluster/upgrades/upgrade_nodes.yml +++ b/playbooks/common/openshift-cluster/upgrades/upgrade_nodes.yml @@ -26,8 +26,12 @@ - name: Drain Node for Kubelet upgrade command: > - {{ hostvars[groups.oo_first_master.0].openshift.common.admin_binary }} drain {{ openshift.node.nodename | lower }} --force --delete-local-data --ignore-daemonsets + {{ hostvars[groups.oo_first_master.0].openshift.common.admin_binary }} drain {{ openshift.node.nodename | lower }} --config={{ openshift.common.config_base }}/master/admin.kubeconfig --force --delete-local-data --ignore-daemonsets delegate_to: "{{ groups.oo_first_master.0 }}" + register: l_upgrade_nodes_drain_result + until: not l_upgrade_nodes_drain_result | failed + retries: 60 + delay: 60 roles: - lib_openshift diff --git a/playbooks/common/openshift-glusterfs/config.yml b/playbooks/common/openshift-glusterfs/config.yml index 1efdfb336..edc15a3f2 100644 --- a/playbooks/common/openshift-glusterfs/config.yml +++ b/playbooks/common/openshift-glusterfs/config.yml @@ -1,6 +1,6 @@ --- -- name: Open firewall ports for GlusterFS - hosts: oo_glusterfs_to_config +- name: Open firewall ports for GlusterFS nodes + hosts: glusterfs vars: os_firewall_allow: - service: glusterfs_sshd @@ -14,7 +14,24 @@ roles: - role: os_firewall when: - - openshift_storage_glusterfs_is_native | default(True) + - openshift_storage_glusterfs_is_native | default(True) | bool + +- name: Open firewall ports for GlusterFS registry nodes + hosts: glusterfs_registry + vars: + os_firewall_allow: + - service: glusterfs_sshd + port: "2222/tcp" + - service: glusterfs_daemon + port: "24007/tcp" + - service: glusterfs_management + port: "24008/tcp" + - service: glusterfs_bricks + port: "49152-49251/tcp" + roles: + - role: os_firewall + when: + - openshift_storage_glusterfs_registry_is_native | default(True) | bool - name: Configure GlusterFS hosts: oo_first_master diff --git a/playbooks/common/openshift-loadbalancer/config.yml b/playbooks/common/openshift-loadbalancer/config.yml index c414913bf..2dacc1218 100644 --- a/playbooks/common/openshift-loadbalancer/config.yml +++ b/playbooks/common/openshift-loadbalancer/config.yml @@ -12,5 +12,6 @@ openshift_use_nuage | default(false), nuage_mon_rest_server_port | default(none))) + openshift_loadbalancer_additional_backends | default([]) }}" + openshift_image_tag: "{{ hostvars[groups.oo_first_master.0].openshift_image_tag }}" roles: - role: openshift_loadbalancer diff --git a/playbooks/common/openshift-master/config.yml b/playbooks/common/openshift-master/config.yml index 7d3a371e3..5de03951c 100644 --- a/playbooks/common/openshift-master/config.yml +++ b/playbooks/common/openshift-master/config.yml @@ -127,6 +127,9 @@ | union(groups['oo_etcd_to_config'] | default([]))) | oo_collect('openshift.common.hostname') | default([]) | join (',') }}" + openshift_no_proxy_etcd_host_ips: "{{ hostvars | oo_select_keys(groups['oo_etcd_to_config'] | default([])) + | oo_collect('openshift.common.ip') | default([]) | join(',') + }}" roles: - role: openshift_master openshift_ca_host: "{{ groups.oo_first_master.0 }}" diff --git a/playbooks/common/openshift-node/restart.yml b/playbooks/common/openshift-node/restart.yml index 63273cb78..ed2473a43 100644 --- a/playbooks/common/openshift-node/restart.yml +++ b/playbooks/common/openshift-node/restart.yml @@ -11,6 +11,10 @@ service: name: docker state: restarted + register: l_docker_restart_docker_in_node_result + until: not l_docker_restart_docker_in_node_result | failed + retries: 3 + delay: 30 - name: Update docker facts openshift_facts: diff --git a/playbooks/libvirt/openshift-cluster/config.yml b/playbooks/libvirt/openshift-cluster/config.yml index 477213f4e..569e00da2 100644 --- a/playbooks/libvirt/openshift-cluster/config.yml +++ b/playbooks/libvirt/openshift-cluster/config.yml @@ -37,4 +37,3 @@ openshift_use_flannel: "{{ lookup('oo_option', 'use_flannel') }}" openshift_use_calico: "{{ lookup('oo_option', 'use_calico') }}" openshift_use_fluentd: "{{ lookup('oo_option', 'use_fluentd') }}" - openshift_use_dnsmasq: false diff --git a/playbooks/libvirt/openshift-cluster/tasks/launch_instances.yml b/playbooks/libvirt/openshift-cluster/tasks/launch_instances.yml index ccd29be29..4df86effa 100644 --- a/playbooks/libvirt/openshift-cluster/tasks/launch_instances.yml +++ b/playbooks/libvirt/openshift-cluster/tasks/launch_instances.yml @@ -49,11 +49,15 @@ - '{{ instances }}' - [ user-data, meta-data ] +- name: Check for genisoimage + command: which genisoimage + register: which_genisoimage + - name: Create the cloud-init config drive - command: 'genisoimage -output {{ libvirt_storage_pool_path }}/{{ item }}_cloud-init.iso -volid cidata -joliet -rock user-data meta-data' + command: "{{ 'genisoimage' if which_genisoimage.rc == 0 else 'mkisofs' }} -output {{ libvirt_storage_pool_path }}/{{ item }}_cloud-init.iso -volid cidata -joliet -rock user-data meta-data" args: - chdir: '{{ libvirt_storage_pool_path }}/{{ item }}_configdrive/' - creates: '{{ libvirt_storage_pool_path }}/{{ item }}_cloud-init.iso' + chdir: "{{ libvirt_storage_pool_path }}/{{ item }}_configdrive/" + creates: "{{ libvirt_storage_pool_path }}/{{ item }}_cloud-init.iso" with_items: '{{ instances }}' - name: Refresh the libvirt storage pool for openshift diff --git a/requirements.txt b/requirements.txt index dae460713..bf95b4ff9 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,6 +1,6 @@ # Versions are pinned to prevent pypi releases arbitrarily breaking # tests with new APIs/semantics. We want to update versions deliberately. -ansible==2.2.2.0 +ansible==2.3.1.0 boto==2.34.0 click==6.7 pyOpenSSL==16.2.0 diff --git a/roles/ansible_service_broker/defaults/main.yml b/roles/ansible_service_broker/defaults/main.yml index aa1c5022b..12929b354 100644 --- a/roles/ansible_service_broker/defaults/main.yml +++ b/roles/ansible_service_broker/defaults/main.yml @@ -4,6 +4,7 @@ ansible_service_broker_remove: false ansible_service_broker_log_level: info ansible_service_broker_output_request: false ansible_service_broker_recovery: true +ansible_service_broker_bootstrap_on_startup: true # Recommended you do not enable this for now ansible_service_broker_dev_broker: false ansible_service_broker_launch_apb_on_bind: false diff --git a/roles/ansible_service_broker/tasks/install.yml b/roles/ansible_service_broker/tasks/install.yml index 65dffc89b..b3797ef96 100644 --- a/roles/ansible_service_broker/tasks/install.yml +++ b/roles/ansible_service_broker/tasks/install.yml @@ -42,6 +42,14 @@ namespace: openshift-ansible-service-broker state: present +- name: Set SA cluster-role + oc_adm_policy_user: + state: present + namespace: "openshift-ansible-service-broker" + resource_kind: cluster-role + resource_name: admin + user: "system:serviceaccount:openshift-ansible-service-broker:asb" + - name: create ansible-service-broker service oc_service: name: asb @@ -254,6 +262,7 @@ launch_apb_on_bind: {{ ansible_service_broker_launch_apb_on_bind | bool | lower }} recovery: {{ ansible_service_broker_recovery | bool | lower }} output_request: {{ ansible_service_broker_output_request | bool | lower }} + bootstrap_on_startup: {{ ansible_service_broker_bootstrap_on_startup | bool | lower }} - name: Create the Broker resource in the catalog oc_obj: diff --git a/roles/ansible_service_broker/vars/openshift-enterprise.yml b/roles/ansible_service_broker/vars/openshift-enterprise.yml index f672760aa..0b3a2a69d 100644 --- a/roles/ansible_service_broker/vars/openshift-enterprise.yml +++ b/roles/ansible_service_broker/vars/openshift-enterprise.yml @@ -1,6 +1,6 @@ --- -__ansible_service_broker_image_prefix: registry.access.redhat.com/openshift3/ +__ansible_service_broker_image_prefix: registry.access.redhat.com/openshift3/ose- __ansible_service_broker_image_tag: latest __ansible_service_broker_etcd_image_prefix: rhel7/ diff --git a/roles/calico/handlers/main.yml b/roles/calico/handlers/main.yml index 53cecfcc3..67fc0065f 100644 --- a/roles/calico/handlers/main.yml +++ b/roles/calico/handlers/main.yml @@ -8,3 +8,7 @@ systemd: name: "{{ openshift.docker.service_name }}" state: restarted + register: l_docker_restart_docker_in_calico_result + until: not l_docker_restart_docker_in_calico_result | failed + retries: 3 + delay: 30 diff --git a/roles/contiv/tasks/netplugin.yml b/roles/contiv/tasks/netplugin.yml index 0847c92bc..e861a2591 100644 --- a/roles/contiv/tasks/netplugin.yml +++ b/roles/contiv/tasks/netplugin.yml @@ -108,6 +108,10 @@ name: "{{ openshift.docker.service_name }}" state: restarted when: docker_updated|changed + register: l_docker_restart_docker_in_contiv_result + until: not l_docker_restart_docker_in_contiv_result | failed + retries: 3 + delay: 30 - name: Netplugin | Enable Netplugin service: diff --git a/roles/docker/handlers/main.yml b/roles/docker/handlers/main.yml index 3a4f4ba92..591367467 100644 --- a/roles/docker/handlers/main.yml +++ b/roles/docker/handlers/main.yml @@ -6,9 +6,8 @@ state: restarted register: r_docker_restart_docker_result until: not r_docker_restart_docker_result | failed - retries: 1 + retries: 3 delay: 30 - when: not docker_service_status_changed | default(false) | bool - name: restart udev diff --git a/roles/docker/tasks/package_docker.yml b/roles/docker/tasks/package_docker.yml index c82d8659a..bc52ab60c 100644 --- a/roles/docker/tasks/package_docker.yml +++ b/roles/docker/tasks/package_docker.yml @@ -93,7 +93,7 @@ dest: /etc/sysconfig/docker regexp: '^OPTIONS=.*$' line: "OPTIONS='\ - {% if ansible_selinux.status | default(None) == '''enabled''' and docker_selinux_enabled | default(true) %} --selinux-enabled {% endif %}\ + {% if ansible_selinux.status | default(None) == 'enabled' and docker_selinux_enabled | default(true) | bool %} --selinux-enabled {% endif %}\ {% if docker_log_driver is defined %} --log-driver {{ docker_log_driver }}{% endif %}\ {% if docker_log_options is defined %} {{ docker_log_options | oo_split() | oo_prepend_strings_in_list('--log-opt ') | join(' ')}}{% endif %}\ {% if docker_options is defined %} {{ docker_options }}{% endif %}\ @@ -123,9 +123,12 @@ enabled: yes state: started daemon_reload: yes - register: start_result + register: r_docker_package_docker_start_result + until: not r_docker_package_docker_start_result | failed + retries: 3 + delay: 30 - set_fact: - docker_service_status_changed: start_result | changed + docker_service_status_changed: "{{ r_docker_package_docker_start_result | changed }}" - meta: flush_handlers diff --git a/roles/docker/tasks/systemcontainer_docker.yml b/roles/docker/tasks/systemcontainer_docker.yml index d8c5ccfd3..57a84bc2c 100644 --- a/roles/docker/tasks/systemcontainer_docker.yml +++ b/roles/docker/tasks/systemcontainer_docker.yml @@ -46,6 +46,11 @@ state: stopped daemon_reload: yes ignore_errors: True + register: r_docker_systemcontainer_docker_stop_result + until: not r_docker_systemcontainer_docker_stop_result | failed + retries: 3 + delay: 30 + # Set http_proxy, https_proxy, and no_proxy in /etc/atomic.conf # regexp: the line starts with or without #, followed by the string @@ -160,9 +165,12 @@ enabled: yes state: started daemon_reload: yes - register: start_result + register: r_docker_systemcontainer_docker_start_result + until: not r_docker_systemcontainer_docker_start_result | failed + retries: 3 + delay: 30 - set_fact: - docker_service_status_changed: start_result | changed + docker_service_status_changed: "{{ r_docker_systemcontainer_docker_start_result | changed }}" - meta: flush_handlers diff --git a/roles/etcd/tasks/main.yml b/roles/etcd/tasks/main.yml index f0661209f..8c2f392ee 100644 --- a/roles/etcd/tasks/main.yml +++ b/roles/etcd/tasks/main.yml @@ -14,7 +14,8 @@ name: etcd_common vars: r_etcd_common_action: drop_etcdctl - when: openshift_etcd_etcdctl_profile | default(true) | bool + when: + - openshift_etcd_etcdctl_profile | default(true) | bool - block: - name: Pull etcd container diff --git a/roles/flannel/handlers/main.yml b/roles/flannel/handlers/main.yml index c60c2115a..02f5a5f64 100644 --- a/roles/flannel/handlers/main.yml +++ b/roles/flannel/handlers/main.yml @@ -8,3 +8,7 @@ systemd: name: "{{ openshift.docker.service_name }}" state: restarted + register: l_docker_restart_docker_in_flannel_result + until: not l_docker_restart_docker_in_flannel_result | failed + retries: 3 + delay: 30 diff --git a/roles/lib_openshift/library/oc_storageclass.py b/roles/lib_openshift/library/oc_storageclass.py new file mode 100644 index 000000000..d5375e27a --- /dev/null +++ b/roles/lib_openshift/library/oc_storageclass.py @@ -0,0 +1,1688 @@ +#!/usr/bin/env python +# pylint: disable=missing-docstring +# flake8: noqa: T001 +# ___ ___ _ _ ___ ___ _ _____ ___ ___ +# / __| __| \| | __| _ \ /_\_ _| __| \ +# | (_ | _|| .` | _|| / / _ \| | | _|| |) | +# \___|___|_|\_|___|_|_\/_/_\_\_|_|___|___/_ _____ +# | \ / _ \ | \| |/ _ \_ _| | __| \_ _|_ _| +# | |) | (_) | | .` | (_) || | | _|| |) | | | | +# |___/ \___/ |_|\_|\___/ |_| |___|___/___| |_| +# +# Copyright 2016 Red Hat, Inc. and/or its affiliates +# and other contributors as indicated by the @author tags. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# -*- -*- -*- Begin included fragment: lib/import.py -*- -*- -*- +''' + OpenShiftCLI class that wraps the oc commands in a subprocess +''' +# pylint: disable=too-many-lines + +from __future__ import print_function +import atexit +import copy +import json +import os +import re +import shutil +import subprocess +import tempfile +# pylint: disable=import-error +try: + import ruamel.yaml as yaml +except ImportError: + import yaml + +from ansible.module_utils.basic import AnsibleModule + +# -*- -*- -*- End included fragment: lib/import.py -*- -*- -*- + +# -*- -*- -*- Begin included fragment: doc/storageclass -*- -*- -*- + +DOCUMENTATION = ''' +--- +module: oc_storageclass +short_description: Create, modify, and idempotently manage openshift storageclasses. +description: + - Manage openshift storageclass objects programmatically. +options: + state: + description: + - State represents whether to create, modify, delete, or list + required: False + default: present + choices: ["present", "absent", "list"] + aliases: [] + kubeconfig: + description: + - The path for the kubeconfig file to use for authentication + required: false + default: /etc/origin/master/admin.kubeconfig + aliases: [] + debug: + description: + - Turn on debug output. + required: false + default: False + aliases: [] + name: + description: + - Name of the object that is being queried. + required: false + default: None + aliases: [] + provisioner: + description: + - Any annotations to add to the storageclass + required: false + default: 'aws-ebs' + aliases: [] + default_storage_class: + description: + - Whether or not this is the default storage class + required: false + default: False + aliases: [] + parameters: + description: + - A dictionary with the parameters to configure the storageclass. This will be based on provisioner + required: false + default: None + aliases: [] + api_version: + description: + - The api version. + required: false + default: v1 + aliases: [] +author: +- "Kenny Woodson <kwoodson@redhat.com>" +extends_documentation_fragment: [] +''' + +EXAMPLES = ''' +- name: get storageclass + run_once: true + oc_storageclass: + name: gp2 + state: list + register: registry_sc_out + +- name: create the storageclass + oc_storageclass: + run_once: true + name: gp2 + parameters: + type: gp2 + encrypted: 'true' + kmsKeyId: '<full kms key arn>' + provisioner: aws-ebs + default_storage_class: False + register: sc_out + notify: + - restart openshift master services +''' + +# -*- -*- -*- End included fragment: doc/storageclass -*- -*- -*- + +# -*- -*- -*- Begin included fragment: ../../lib_utils/src/class/yedit.py -*- -*- -*- + + +class YeditException(Exception): # pragma: no cover + ''' Exception class for Yedit ''' + pass + + +# pylint: disable=too-many-public-methods +class Yedit(object): # pragma: no cover + ''' Class to modify yaml files ''' + re_valid_key = r"(((\[-?\d+\])|([0-9a-zA-Z%s/_-]+)).?)+$" + re_key = r"(?:\[(-?\d+)\])|([0-9a-zA-Z{}/_-]+)" + com_sep = set(['.', '#', '|', ':']) + + # pylint: disable=too-many-arguments + def __init__(self, + filename=None, + content=None, + content_type='yaml', + separator='.', + backup=False): + self.content = content + self._separator = separator + self.filename = filename + self.__yaml_dict = content + self.content_type = content_type + self.backup = backup + self.load(content_type=self.content_type) + if self.__yaml_dict is None: + self.__yaml_dict = {} + + @property + def separator(self): + ''' getter method for separator ''' + return self._separator + + @separator.setter + def separator(self, inc_sep): + ''' setter method for separator ''' + self._separator = inc_sep + + @property + def yaml_dict(self): + ''' getter method for yaml_dict ''' + return self.__yaml_dict + + @yaml_dict.setter + def yaml_dict(self, value): + ''' setter method for yaml_dict ''' + self.__yaml_dict = value + + @staticmethod + def parse_key(key, sep='.'): + '''parse the key allowing the appropriate separator''' + common_separators = list(Yedit.com_sep - set([sep])) + return re.findall(Yedit.re_key.format(''.join(common_separators)), key) + + @staticmethod + def valid_key(key, sep='.'): + '''validate the incoming key''' + common_separators = list(Yedit.com_sep - set([sep])) + if not re.match(Yedit.re_valid_key.format(''.join(common_separators)), key): + return False + + return True + + @staticmethod + def remove_entry(data, key, sep='.'): + ''' remove data at location key ''' + if key == '' and isinstance(data, dict): + data.clear() + return True + elif key == '' and isinstance(data, list): + del data[:] + return True + + if not (key and Yedit.valid_key(key, sep)) and \ + isinstance(data, (list, dict)): + return None + + key_indexes = Yedit.parse_key(key, sep) + for arr_ind, dict_key in key_indexes[:-1]: + if dict_key and isinstance(data, dict): + data = data.get(dict_key) + elif (arr_ind and isinstance(data, list) and + int(arr_ind) <= len(data) - 1): + data = data[int(arr_ind)] + else: + return None + + # process last index for remove + # expected list entry + if key_indexes[-1][0]: + if isinstance(data, list) and int(key_indexes[-1][0]) <= len(data) - 1: # noqa: E501 + del data[int(key_indexes[-1][0])] + return True + + # expected dict entry + elif key_indexes[-1][1]: + if isinstance(data, dict): + del data[key_indexes[-1][1]] + return True + + @staticmethod + def add_entry(data, key, item=None, sep='.'): + ''' Get an item from a dictionary with key notation a.b.c + d = {'a': {'b': 'c'}}} + key = a#b + return c + ''' + if key == '': + pass + elif (not (key and Yedit.valid_key(key, sep)) and + isinstance(data, (list, dict))): + return None + + key_indexes = Yedit.parse_key(key, sep) + for arr_ind, dict_key in key_indexes[:-1]: + if dict_key: + if isinstance(data, dict) and dict_key in data and data[dict_key]: # noqa: E501 + data = data[dict_key] + continue + + elif data and not isinstance(data, dict): + raise YeditException("Unexpected item type found while going through key " + + "path: {} (at key: {})".format(key, dict_key)) + + data[dict_key] = {} + data = data[dict_key] + + elif (arr_ind and isinstance(data, list) and + int(arr_ind) <= len(data) - 1): + data = data[int(arr_ind)] + else: + raise YeditException("Unexpected item type found while going through key path: {}".format(key)) + + if key == '': + data = item + + # process last index for add + # expected list entry + elif key_indexes[-1][0] and isinstance(data, list) and int(key_indexes[-1][0]) <= len(data) - 1: # noqa: E501 + data[int(key_indexes[-1][0])] = item + + # expected dict entry + elif key_indexes[-1][1] and isinstance(data, dict): + data[key_indexes[-1][1]] = item + + # didn't add/update to an existing list, nor add/update key to a dict + # so we must have been provided some syntax like a.b.c[<int>] = "data" for a + # non-existent array + else: + raise YeditException("Error adding to object at path: {}".format(key)) + + return data + + @staticmethod + def get_entry(data, key, sep='.'): + ''' Get an item from a dictionary with key notation a.b.c + d = {'a': {'b': 'c'}}} + key = a.b + return c + ''' + if key == '': + pass + elif (not (key and Yedit.valid_key(key, sep)) and + isinstance(data, (list, dict))): + return None + + key_indexes = Yedit.parse_key(key, sep) + for arr_ind, dict_key in key_indexes: + if dict_key and isinstance(data, dict): + data = data.get(dict_key) + elif (arr_ind and isinstance(data, list) and + int(arr_ind) <= len(data) - 1): + data = data[int(arr_ind)] + else: + return None + + return data + + @staticmethod + def _write(filename, contents): + ''' Actually write the file contents to disk. This helps with mocking. ''' + + tmp_filename = filename + '.yedit' + + with open(tmp_filename, 'w') as yfd: + yfd.write(contents) + + os.rename(tmp_filename, filename) + + def write(self): + ''' write to file ''' + if not self.filename: + raise YeditException('Please specify a filename.') + + if self.backup and self.file_exists(): + shutil.copy(self.filename, self.filename + '.orig') + + # Try to set format attributes if supported + try: + self.yaml_dict.fa.set_block_style() + except AttributeError: + pass + + # Try to use RoundTripDumper if supported. + try: + Yedit._write(self.filename, yaml.dump(self.yaml_dict, Dumper=yaml.RoundTripDumper)) + except AttributeError: + Yedit._write(self.filename, yaml.safe_dump(self.yaml_dict, default_flow_style=False)) + + return (True, self.yaml_dict) + + def read(self): + ''' read from file ''' + # check if it exists + if self.filename is None or not self.file_exists(): + return None + + contents = None + with open(self.filename) as yfd: + contents = yfd.read() + + return contents + + def file_exists(self): + ''' return whether file exists ''' + if os.path.exists(self.filename): + return True + + return False + + def load(self, content_type='yaml'): + ''' return yaml file ''' + contents = self.read() + + if not contents and not self.content: + return None + + if self.content: + if isinstance(self.content, dict): + self.yaml_dict = self.content + return self.yaml_dict + elif isinstance(self.content, str): + contents = self.content + + # check if it is yaml + try: + if content_type == 'yaml' and contents: + # Try to set format attributes if supported + try: + self.yaml_dict.fa.set_block_style() + except AttributeError: + pass + + # Try to use RoundTripLoader if supported. + try: + self.yaml_dict = yaml.safe_load(contents, yaml.RoundTripLoader) + except AttributeError: + self.yaml_dict = yaml.safe_load(contents) + + # Try to set format attributes if supported + try: + self.yaml_dict.fa.set_block_style() + except AttributeError: + pass + + elif content_type == 'json' and contents: + self.yaml_dict = json.loads(contents) + except yaml.YAMLError as err: + # Error loading yaml or json + raise YeditException('Problem with loading yaml file. {}'.format(err)) + + return self.yaml_dict + + def get(self, key): + ''' get a specified key''' + try: + entry = Yedit.get_entry(self.yaml_dict, key, self.separator) + except KeyError: + entry = None + + return entry + + def pop(self, path, key_or_item): + ''' remove a key, value pair from a dict or an item for a list''' + try: + entry = Yedit.get_entry(self.yaml_dict, path, self.separator) + except KeyError: + entry = None + + if entry is None: + return (False, self.yaml_dict) + + if isinstance(entry, dict): + # AUDIT:maybe-no-member makes sense due to fuzzy types + # pylint: disable=maybe-no-member + if key_or_item in entry: + entry.pop(key_or_item) + return (True, self.yaml_dict) + return (False, self.yaml_dict) + + elif isinstance(entry, list): + # AUDIT:maybe-no-member makes sense due to fuzzy types + # pylint: disable=maybe-no-member + ind = None + try: + ind = entry.index(key_or_item) + except ValueError: + return (False, self.yaml_dict) + + entry.pop(ind) + return (True, self.yaml_dict) + + return (False, self.yaml_dict) + + def delete(self, path): + ''' remove path from a dict''' + try: + entry = Yedit.get_entry(self.yaml_dict, path, self.separator) + except KeyError: + entry = None + + if entry is None: + return (False, self.yaml_dict) + + result = Yedit.remove_entry(self.yaml_dict, path, self.separator) + if not result: + return (False, self.yaml_dict) + + return (True, self.yaml_dict) + + def exists(self, path, value): + ''' check if value exists at path''' + try: + entry = Yedit.get_entry(self.yaml_dict, path, self.separator) + except KeyError: + entry = None + + if isinstance(entry, list): + if value in entry: + return True + return False + + elif isinstance(entry, dict): + if isinstance(value, dict): + rval = False + for key, val in value.items(): + if entry[key] != val: + rval = False + break + else: + rval = True + return rval + + return value in entry + + return entry == value + + def append(self, path, value): + '''append value to a list''' + try: + entry = Yedit.get_entry(self.yaml_dict, path, self.separator) + except KeyError: + entry = None + + if entry is None: + self.put(path, []) + entry = Yedit.get_entry(self.yaml_dict, path, self.separator) + if not isinstance(entry, list): + return (False, self.yaml_dict) + + # AUDIT:maybe-no-member makes sense due to loading data from + # a serialized format. + # pylint: disable=maybe-no-member + entry.append(value) + return (True, self.yaml_dict) + + # pylint: disable=too-many-arguments + def update(self, path, value, index=None, curr_value=None): + ''' put path, value into a dict ''' + try: + entry = Yedit.get_entry(self.yaml_dict, path, self.separator) + except KeyError: + entry = None + + if isinstance(entry, dict): + # AUDIT:maybe-no-member makes sense due to fuzzy types + # pylint: disable=maybe-no-member + if not isinstance(value, dict): + raise YeditException('Cannot replace key, value entry in dict with non-dict type. ' + + 'value=[{}] type=[{}]'.format(value, type(value))) + + entry.update(value) + return (True, self.yaml_dict) + + elif isinstance(entry, list): + # AUDIT:maybe-no-member makes sense due to fuzzy types + # pylint: disable=maybe-no-member + ind = None + if curr_value: + try: + ind = entry.index(curr_value) + except ValueError: + return (False, self.yaml_dict) + + elif index is not None: + ind = index + + if ind is not None and entry[ind] != value: + entry[ind] = value + return (True, self.yaml_dict) + + # see if it exists in the list + try: + ind = entry.index(value) + except ValueError: + # doesn't exist, append it + entry.append(value) + return (True, self.yaml_dict) + + # already exists, return + if ind is not None: + return (False, self.yaml_dict) + return (False, self.yaml_dict) + + def put(self, path, value): + ''' put path, value into a dict ''' + try: + entry = Yedit.get_entry(self.yaml_dict, path, self.separator) + except KeyError: + entry = None + + if entry == value: + return (False, self.yaml_dict) + + # deepcopy didn't work + # Try to use ruamel.yaml and fallback to pyyaml + try: + tmp_copy = yaml.load(yaml.round_trip_dump(self.yaml_dict, + default_flow_style=False), + yaml.RoundTripLoader) + except AttributeError: + tmp_copy = copy.deepcopy(self.yaml_dict) + + # set the format attributes if available + try: + tmp_copy.fa.set_block_style() + except AttributeError: + pass + + result = Yedit.add_entry(tmp_copy, path, value, self.separator) + if result is None: + return (False, self.yaml_dict) + + # When path equals "" it is a special case. + # "" refers to the root of the document + # Only update the root path (entire document) when its a list or dict + if path == '': + if isinstance(result, list) or isinstance(result, dict): + self.yaml_dict = result + return (True, self.yaml_dict) + + return (False, self.yaml_dict) + + self.yaml_dict = tmp_copy + + return (True, self.yaml_dict) + + def create(self, path, value): + ''' create a yaml file ''' + if not self.file_exists(): + # deepcopy didn't work + # Try to use ruamel.yaml and fallback to pyyaml + try: + tmp_copy = yaml.load(yaml.round_trip_dump(self.yaml_dict, + default_flow_style=False), + yaml.RoundTripLoader) + except AttributeError: + tmp_copy = copy.deepcopy(self.yaml_dict) + + # set the format attributes if available + try: + tmp_copy.fa.set_block_style() + except AttributeError: + pass + + result = Yedit.add_entry(tmp_copy, path, value, self.separator) + if result is not None: + self.yaml_dict = tmp_copy + return (True, self.yaml_dict) + + return (False, self.yaml_dict) + + @staticmethod + def get_curr_value(invalue, val_type): + '''return the current value''' + if invalue is None: + return None + + curr_value = invalue + if val_type == 'yaml': + curr_value = yaml.load(invalue) + elif val_type == 'json': + curr_value = json.loads(invalue) + + return curr_value + + @staticmethod + def parse_value(inc_value, vtype=''): + '''determine value type passed''' + true_bools = ['y', 'Y', 'yes', 'Yes', 'YES', 'true', 'True', 'TRUE', + 'on', 'On', 'ON', ] + false_bools = ['n', 'N', 'no', 'No', 'NO', 'false', 'False', 'FALSE', + 'off', 'Off', 'OFF'] + + # It came in as a string but you didn't specify value_type as string + # we will convert to bool if it matches any of the above cases + if isinstance(inc_value, str) and 'bool' in vtype: + if inc_value not in true_bools and inc_value not in false_bools: + raise YeditException('Not a boolean type. str=[{}] vtype=[{}]'.format(inc_value, vtype)) + elif isinstance(inc_value, bool) and 'str' in vtype: + inc_value = str(inc_value) + + # There is a special case where '' will turn into None after yaml loading it so skip + if isinstance(inc_value, str) and inc_value == '': + pass + # If vtype is not str then go ahead and attempt to yaml load it. + elif isinstance(inc_value, str) and 'str' not in vtype: + try: + inc_value = yaml.safe_load(inc_value) + except Exception: + raise YeditException('Could not determine type of incoming value. ' + + 'value=[{}] vtype=[{}]'.format(type(inc_value), vtype)) + + return inc_value + + @staticmethod + def process_edits(edits, yamlfile): + '''run through a list of edits and process them one-by-one''' + results = [] + for edit in edits: + value = Yedit.parse_value(edit['value'], edit.get('value_type', '')) + if edit.get('action') == 'update': + # pylint: disable=line-too-long + curr_value = Yedit.get_curr_value( + Yedit.parse_value(edit.get('curr_value')), + edit.get('curr_value_format')) + + rval = yamlfile.update(edit['key'], + value, + edit.get('index'), + curr_value) + + elif edit.get('action') == 'append': + rval = yamlfile.append(edit['key'], value) + + else: + rval = yamlfile.put(edit['key'], value) + + if rval[0]: + results.append({'key': edit['key'], 'edit': rval[1]}) + + return {'changed': len(results) > 0, 'results': results} + + # pylint: disable=too-many-return-statements,too-many-branches + @staticmethod + def run_ansible(params): + '''perform the idempotent crud operations''' + yamlfile = Yedit(filename=params['src'], + backup=params['backup'], + separator=params['separator']) + + state = params['state'] + + if params['src']: + rval = yamlfile.load() + + if yamlfile.yaml_dict is None and state != 'present': + return {'failed': True, + 'msg': 'Error opening file [{}]. Verify that the '.format(params['src']) + + 'file exists, that it is has correct permissions, and is valid yaml.'} + + if state == 'list': + if params['content']: + content = Yedit.parse_value(params['content'], params['content_type']) + yamlfile.yaml_dict = content + + if params['key']: + rval = yamlfile.get(params['key']) or {} + + return {'changed': False, 'result': rval, 'state': state} + + elif state == 'absent': + if params['content']: + content = Yedit.parse_value(params['content'], params['content_type']) + yamlfile.yaml_dict = content + + if params['update']: + rval = yamlfile.pop(params['key'], params['value']) + else: + rval = yamlfile.delete(params['key']) + + if rval[0] and params['src']: + yamlfile.write() + + return {'changed': rval[0], 'result': rval[1], 'state': state} + + elif state == 'present': + # check if content is different than what is in the file + if params['content']: + content = Yedit.parse_value(params['content'], params['content_type']) + + # We had no edits to make and the contents are the same + if yamlfile.yaml_dict == content and \ + params['value'] is None: + return {'changed': False, 'result': yamlfile.yaml_dict, 'state': state} + + yamlfile.yaml_dict = content + + # If we were passed a key, value then + # we enapsulate it in a list and process it + # Key, Value passed to the module : Converted to Edits list # + edits = [] + _edit = {} + if params['value'] is not None: + _edit['value'] = params['value'] + _edit['value_type'] = params['value_type'] + _edit['key'] = params['key'] + + if params['update']: + _edit['action'] = 'update' + _edit['curr_value'] = params['curr_value'] + _edit['curr_value_format'] = params['curr_value_format'] + _edit['index'] = params['index'] + + elif params['append']: + _edit['action'] = 'append' + + edits.append(_edit) + + elif params['edits'] is not None: + edits = params['edits'] + + if edits: + results = Yedit.process_edits(edits, yamlfile) + + # if there were changes and a src provided to us we need to write + if results['changed'] and params['src']: + yamlfile.write() + + return {'changed': results['changed'], 'result': results['results'], 'state': state} + + # no edits to make + if params['src']: + # pylint: disable=redefined-variable-type + rval = yamlfile.write() + return {'changed': rval[0], + 'result': rval[1], + 'state': state} + + # We were passed content but no src, key or value, or edits. Return contents in memory + return {'changed': False, 'result': yamlfile.yaml_dict, 'state': state} + return {'failed': True, 'msg': 'Unkown state passed'} + +# -*- -*- -*- End included fragment: ../../lib_utils/src/class/yedit.py -*- -*- -*- + +# -*- -*- -*- Begin included fragment: lib/base.py -*- -*- -*- +# pylint: disable=too-many-lines +# noqa: E301,E302,E303,T001 + + +class OpenShiftCLIError(Exception): + '''Exception class for openshiftcli''' + pass + + +ADDITIONAL_PATH_LOOKUPS = ['/usr/local/bin', os.path.expanduser('~/bin')] + + +def locate_oc_binary(): + ''' Find and return oc binary file ''' + # https://github.com/openshift/openshift-ansible/issues/3410 + # oc can be in /usr/local/bin in some cases, but that may not + # be in $PATH due to ansible/sudo + paths = os.environ.get("PATH", os.defpath).split(os.pathsep) + ADDITIONAL_PATH_LOOKUPS + + oc_binary = 'oc' + + # Use shutil.which if it is available, otherwise fallback to a naive path search + try: + which_result = shutil.which(oc_binary, path=os.pathsep.join(paths)) + if which_result is not None: + oc_binary = which_result + except AttributeError: + for path in paths: + if os.path.exists(os.path.join(path, oc_binary)): + oc_binary = os.path.join(path, oc_binary) + break + + return oc_binary + + +# pylint: disable=too-few-public-methods +class OpenShiftCLI(object): + ''' Class to wrap the command line tools ''' + def __init__(self, + namespace, + kubeconfig='/etc/origin/master/admin.kubeconfig', + verbose=False, + all_namespaces=False): + ''' Constructor for OpenshiftCLI ''' + self.namespace = namespace + self.verbose = verbose + self.kubeconfig = Utils.create_tmpfile_copy(kubeconfig) + self.all_namespaces = all_namespaces + self.oc_binary = locate_oc_binary() + + # Pylint allows only 5 arguments to be passed. + # pylint: disable=too-many-arguments + def _replace_content(self, resource, rname, content, force=False, sep='.'): + ''' replace the current object with the content ''' + res = self._get(resource, rname) + if not res['results']: + return res + + fname = Utils.create_tmpfile(rname + '-') + + yed = Yedit(fname, res['results'][0], separator=sep) + changes = [] + for key, value in content.items(): + changes.append(yed.put(key, value)) + + if any([change[0] for change in changes]): + yed.write() + + atexit.register(Utils.cleanup, [fname]) + + return self._replace(fname, force) + + return {'returncode': 0, 'updated': False} + + def _replace(self, fname, force=False): + '''replace the current object with oc replace''' + # We are removing the 'resourceVersion' to handle + # a race condition when modifying oc objects + yed = Yedit(fname) + results = yed.delete('metadata.resourceVersion') + if results[0]: + yed.write() + + cmd = ['replace', '-f', fname] + if force: + cmd.append('--force') + return self.openshift_cmd(cmd) + + def _create_from_content(self, rname, content): + '''create a temporary file and then call oc create on it''' + fname = Utils.create_tmpfile(rname + '-') + yed = Yedit(fname, content=content) + yed.write() + + atexit.register(Utils.cleanup, [fname]) + + return self._create(fname) + + def _create(self, fname): + '''call oc create on a filename''' + return self.openshift_cmd(['create', '-f', fname]) + + def _delete(self, resource, name=None, selector=None): + '''call oc delete on a resource''' + cmd = ['delete', resource] + if selector is not None: + cmd.append('--selector={}'.format(selector)) + elif name is not None: + cmd.append(name) + else: + raise OpenShiftCLIError('Either name or selector is required when calling delete.') + + return self.openshift_cmd(cmd) + + def _process(self, template_name, create=False, params=None, template_data=None): # noqa: E501 + '''process a template + + template_name: the name of the template to process + create: whether to send to oc create after processing + params: the parameters for the template + template_data: the incoming template's data; instead of a file + ''' + cmd = ['process'] + if template_data: + cmd.extend(['-f', '-']) + else: + cmd.append(template_name) + if params: + param_str = ["{}={}".format(key, str(value).replace("'", r'"')) for key, value in params.items()] + cmd.append('-v') + cmd.extend(param_str) + + results = self.openshift_cmd(cmd, output=True, input_data=template_data) + + if results['returncode'] != 0 or not create: + return results + + fname = Utils.create_tmpfile(template_name + '-') + yed = Yedit(fname, results['results']) + yed.write() + + atexit.register(Utils.cleanup, [fname]) + + return self.openshift_cmd(['create', '-f', fname]) + + def _get(self, resource, name=None, selector=None): + '''return a resource by name ''' + cmd = ['get', resource] + if selector is not None: + cmd.append('--selector={}'.format(selector)) + elif name is not None: + cmd.append(name) + + cmd.extend(['-o', 'json']) + + rval = self.openshift_cmd(cmd, output=True) + + # Ensure results are retuned in an array + if 'items' in rval: + rval['results'] = rval['items'] + elif not isinstance(rval['results'], list): + rval['results'] = [rval['results']] + + return rval + + def _schedulable(self, node=None, selector=None, schedulable=True): + ''' perform oadm manage-node scheduable ''' + cmd = ['manage-node'] + if node: + cmd.extend(node) + else: + cmd.append('--selector={}'.format(selector)) + + cmd.append('--schedulable={}'.format(schedulable)) + + return self.openshift_cmd(cmd, oadm=True, output=True, output_type='raw') # noqa: E501 + + def _list_pods(self, node=None, selector=None, pod_selector=None): + ''' perform oadm list pods + + node: the node in which to list pods + selector: the label selector filter if provided + pod_selector: the pod selector filter if provided + ''' + cmd = ['manage-node'] + if node: + cmd.extend(node) + else: + cmd.append('--selector={}'.format(selector)) + + if pod_selector: + cmd.append('--pod-selector={}'.format(pod_selector)) + + cmd.extend(['--list-pods', '-o', 'json']) + + return self.openshift_cmd(cmd, oadm=True, output=True, output_type='raw') + + # pylint: disable=too-many-arguments + def _evacuate(self, node=None, selector=None, pod_selector=None, dry_run=False, grace_period=None, force=False): + ''' perform oadm manage-node evacuate ''' + cmd = ['manage-node'] + if node: + cmd.extend(node) + else: + cmd.append('--selector={}'.format(selector)) + + if dry_run: + cmd.append('--dry-run') + + if pod_selector: + cmd.append('--pod-selector={}'.format(pod_selector)) + + if grace_period: + cmd.append('--grace-period={}'.format(int(grace_period))) + + if force: + cmd.append('--force') + + cmd.append('--evacuate') + + return self.openshift_cmd(cmd, oadm=True, output=True, output_type='raw') + + def _version(self): + ''' return the openshift version''' + return self.openshift_cmd(['version'], output=True, output_type='raw') + + def _import_image(self, url=None, name=None, tag=None): + ''' perform image import ''' + cmd = ['import-image'] + + image = '{0}'.format(name) + if tag: + image += ':{0}'.format(tag) + + cmd.append(image) + + if url: + cmd.append('--from={0}/{1}'.format(url, image)) + + cmd.append('-n{0}'.format(self.namespace)) + + cmd.append('--confirm') + return self.openshift_cmd(cmd) + + def _run(self, cmds, input_data): + ''' Actually executes the command. This makes mocking easier. ''' + curr_env = os.environ.copy() + curr_env.update({'KUBECONFIG': self.kubeconfig}) + proc = subprocess.Popen(cmds, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + env=curr_env) + + stdout, stderr = proc.communicate(input_data) + + return proc.returncode, stdout.decode('utf-8'), stderr.decode('utf-8') + + # pylint: disable=too-many-arguments,too-many-branches + def openshift_cmd(self, cmd, oadm=False, output=False, output_type='json', input_data=None): + '''Base command for oc ''' + cmds = [self.oc_binary] + + if oadm: + cmds.append('adm') + + cmds.extend(cmd) + + if self.all_namespaces: + cmds.extend(['--all-namespaces']) + elif self.namespace is not None and self.namespace.lower() not in ['none', 'emtpy']: # E501 + cmds.extend(['-n', self.namespace]) + + if self.verbose: + print(' '.join(cmds)) + + try: + returncode, stdout, stderr = self._run(cmds, input_data) + except OSError as ex: + returncode, stdout, stderr = 1, '', 'Failed to execute {}: {}'.format(subprocess.list2cmdline(cmds), ex) + + rval = {"returncode": returncode, + "cmd": ' '.join(cmds)} + + if output_type == 'json': + rval['results'] = {} + if output and stdout: + try: + rval['results'] = json.loads(stdout) + except ValueError as verr: + if "No JSON object could be decoded" in verr.args: + rval['err'] = verr.args + elif output_type == 'raw': + rval['results'] = stdout if output else '' + + if self.verbose: + print("STDOUT: {0}".format(stdout)) + print("STDERR: {0}".format(stderr)) + + if 'err' in rval or returncode != 0: + rval.update({"stderr": stderr, + "stdout": stdout}) + + return rval + + +class Utils(object): # pragma: no cover + ''' utilities for openshiftcli modules ''' + + @staticmethod + def _write(filename, contents): + ''' Actually write the file contents to disk. This helps with mocking. ''' + + with open(filename, 'w') as sfd: + sfd.write(contents) + + @staticmethod + def create_tmp_file_from_contents(rname, data, ftype='yaml'): + ''' create a file in tmp with name and contents''' + + tmp = Utils.create_tmpfile(prefix=rname) + + if ftype == 'yaml': + # AUDIT:no-member makes sense here due to ruamel.YAML/PyYAML usage + # pylint: disable=no-member + if hasattr(yaml, 'RoundTripDumper'): + Utils._write(tmp, yaml.dump(data, Dumper=yaml.RoundTripDumper)) + else: + Utils._write(tmp, yaml.safe_dump(data, default_flow_style=False)) + + elif ftype == 'json': + Utils._write(tmp, json.dumps(data)) + else: + Utils._write(tmp, data) + + # Register cleanup when module is done + atexit.register(Utils.cleanup, [tmp]) + return tmp + + @staticmethod + def create_tmpfile_copy(inc_file): + '''create a temporary copy of a file''' + tmpfile = Utils.create_tmpfile('lib_openshift-') + Utils._write(tmpfile, open(inc_file).read()) + + # Cleanup the tmpfile + atexit.register(Utils.cleanup, [tmpfile]) + + return tmpfile + + @staticmethod + def create_tmpfile(prefix='tmp'): + ''' Generates and returns a temporary file name ''' + + with tempfile.NamedTemporaryFile(prefix=prefix, delete=False) as tmp: + return tmp.name + + @staticmethod + def create_tmp_files_from_contents(content, content_type=None): + '''Turn an array of dict: filename, content into a files array''' + if not isinstance(content, list): + content = [content] + files = [] + for item in content: + path = Utils.create_tmp_file_from_contents(item['path'] + '-', + item['data'], + ftype=content_type) + files.append({'name': os.path.basename(item['path']), + 'path': path}) + return files + + @staticmethod + def cleanup(files): + '''Clean up on exit ''' + for sfile in files: + if os.path.exists(sfile): + if os.path.isdir(sfile): + shutil.rmtree(sfile) + elif os.path.isfile(sfile): + os.remove(sfile) + + @staticmethod + def exists(results, _name): + ''' Check to see if the results include the name ''' + if not results: + return False + + if Utils.find_result(results, _name): + return True + + return False + + @staticmethod + def find_result(results, _name): + ''' Find the specified result by name''' + rval = None + for result in results: + if 'metadata' in result and result['metadata']['name'] == _name: + rval = result + break + + return rval + + @staticmethod + def get_resource_file(sfile, sfile_type='yaml'): + ''' return the service file ''' + contents = None + with open(sfile) as sfd: + contents = sfd.read() + + if sfile_type == 'yaml': + # AUDIT:no-member makes sense here due to ruamel.YAML/PyYAML usage + # pylint: disable=no-member + if hasattr(yaml, 'RoundTripLoader'): + contents = yaml.load(contents, yaml.RoundTripLoader) + else: + contents = yaml.safe_load(contents) + elif sfile_type == 'json': + contents = json.loads(contents) + + return contents + + @staticmethod + def filter_versions(stdout): + ''' filter the oc version output ''' + + version_dict = {} + version_search = ['oc', 'openshift', 'kubernetes'] + + for line in stdout.strip().split('\n'): + for term in version_search: + if not line: + continue + if line.startswith(term): + version_dict[term] = line.split()[-1] + + # horrible hack to get openshift version in Openshift 3.2 + # By default "oc version in 3.2 does not return an "openshift" version + if "openshift" not in version_dict: + version_dict["openshift"] = version_dict["oc"] + + return version_dict + + @staticmethod + def add_custom_versions(versions): + ''' create custom versions strings ''' + + versions_dict = {} + + for tech, version in versions.items(): + # clean up "-" from version + if "-" in version: + version = version.split("-")[0] + + if version.startswith('v'): + versions_dict[tech + '_numeric'] = version[1:].split('+')[0] + # "v3.3.0.33" is what we have, we want "3.3" + versions_dict[tech + '_short'] = version[1:4] + + return versions_dict + + @staticmethod + def openshift_installed(): + ''' check if openshift is installed ''' + import yum + + yum_base = yum.YumBase() + if yum_base.rpmdb.searchNevra(name='atomic-openshift'): + return True + + return False + + # Disabling too-many-branches. This is a yaml dictionary comparison function + # pylint: disable=too-many-branches,too-many-return-statements,too-many-statements + @staticmethod + def check_def_equal(user_def, result_def, skip_keys=None, debug=False): + ''' Given a user defined definition, compare it with the results given back by our query. ''' + + # Currently these values are autogenerated and we do not need to check them + skip = ['metadata', 'status'] + if skip_keys: + skip.extend(skip_keys) + + for key, value in result_def.items(): + if key in skip: + continue + + # Both are lists + if isinstance(value, list): + if key not in user_def: + if debug: + print('User data does not have key [%s]' % key) + print('User data: %s' % user_def) + return False + + if not isinstance(user_def[key], list): + if debug: + print('user_def[key] is not a list key=[%s] user_def[key]=%s' % (key, user_def[key])) + return False + + if len(user_def[key]) != len(value): + if debug: + print("List lengths are not equal.") + print("key=[%s]: user_def[%s] != value[%s]" % (key, len(user_def[key]), len(value))) + print("user_def: %s" % user_def[key]) + print("value: %s" % value) + return False + + for values in zip(user_def[key], value): + if isinstance(values[0], dict) and isinstance(values[1], dict): + if debug: + print('sending list - list') + print(type(values[0])) + print(type(values[1])) + result = Utils.check_def_equal(values[0], values[1], skip_keys=skip_keys, debug=debug) + if not result: + print('list compare returned false') + return False + + elif value != user_def[key]: + if debug: + print('value should be identical') + print(user_def[key]) + print(value) + return False + + # recurse on a dictionary + elif isinstance(value, dict): + if key not in user_def: + if debug: + print("user_def does not have key [%s]" % key) + return False + if not isinstance(user_def[key], dict): + if debug: + print("dict returned false: not instance of dict") + return False + + # before passing ensure keys match + api_values = set(value.keys()) - set(skip) + user_values = set(user_def[key].keys()) - set(skip) + if api_values != user_values: + if debug: + print("keys are not equal in dict") + print(user_values) + print(api_values) + return False + + result = Utils.check_def_equal(user_def[key], value, skip_keys=skip_keys, debug=debug) + if not result: + if debug: + print("dict returned false") + print(result) + return False + + # Verify each key, value pair is the same + else: + if key not in user_def or value != user_def[key]: + if debug: + print("value not equal; user_def does not have key") + print(key) + print(value) + if key in user_def: + print(user_def[key]) + return False + + if debug: + print('returning true') + return True + +class OpenShiftCLIConfig(object): + '''Generic Config''' + def __init__(self, rname, namespace, kubeconfig, options): + self.kubeconfig = kubeconfig + self.name = rname + self.namespace = namespace + self._options = options + + @property + def config_options(self): + ''' return config options ''' + return self._options + + def to_option_list(self, ascommalist=''): + '''return all options as a string + if ascommalist is set to the name of a key, and + the value of that key is a dict, format the dict + as a list of comma delimited key=value pairs''' + return self.stringify(ascommalist) + + def stringify(self, ascommalist=''): + ''' return the options hash as cli params in a string + if ascommalist is set to the name of a key, and + the value of that key is a dict, format the dict + as a list of comma delimited key=value pairs ''' + rval = [] + for key in sorted(self.config_options.keys()): + data = self.config_options[key] + if data['include'] \ + and (data['value'] or isinstance(data['value'], int)): + if key == ascommalist: + val = ','.join(['{}={}'.format(kk, vv) for kk, vv in sorted(data['value'].items())]) + else: + val = data['value'] + rval.append('--{}={}'.format(key.replace('_', '-'), val)) + + return rval + + +# -*- -*- -*- End included fragment: lib/base.py -*- -*- -*- + +# -*- -*- -*- Begin included fragment: lib/storageclass.py -*- -*- -*- + + +# pylint: disable=too-many-instance-attributes +class StorageClassConfig(object): + ''' Handle service options ''' + # pylint: disable=too-many-arguments + def __init__(self, + name, + provisioner=None, + parameters=None, + annotations=None, + default_storage_class="false", + api_version='v1', + kubeconfig='/etc/origin/master/admin.kubeconfig'): + ''' constructor for handling storageclass options ''' + self.name = name + self.parameters = parameters + self.annotations = annotations + self.provisioner = provisioner + self.api_version = api_version + self.default_storage_class = str(default_storage_class).lower() + self.kubeconfig = kubeconfig + self.data = {} + + self.create_dict() + + def create_dict(self): + ''' instantiates a storageclass dict ''' + self.data['apiVersion'] = self.api_version + self.data['kind'] = 'StorageClass' + self.data['metadata'] = {} + self.data['metadata']['name'] = self.name + + self.data['metadata']['annotations'] = {} + if self.annotations is not None: + self.data['metadata']['annotations'] = self.annotations + + self.data['metadata']['annotations']['storageclass.beta.kubernetes.io/is-default-class'] = \ + self.default_storage_class + + if self.provisioner is None: + self.data['provisioner'] = 'kubernetes.io/aws-ebs' + else: + self.data['provisioner'] = self.provisioner + + self.data['parameters'] = {} + if self.parameters is not None: + self.data['parameters'].update(self.parameters) + + # default to aws if no params were passed + else: + self.data['parameters']['type'] = 'gp2' + + + +# pylint: disable=too-many-instance-attributes,too-many-public-methods +class StorageClass(Yedit): + ''' Class to model the oc storageclass object ''' + annotations_path = "metadata.annotations" + provisioner_path = "provisioner" + parameters_path = "parameters" + kind = 'StorageClass' + + def __init__(self, content): + '''StorageClass constructor''' + super(StorageClass, self).__init__(content=content) + + def get_annotations(self): + ''' get a list of ports ''' + return self.get(StorageClass.annotations_path) or {} + + def get_parameters(self): + ''' get the service selector''' + return self.get(StorageClass.parameters_path) or {} + +# -*- -*- -*- End included fragment: lib/storageclass.py -*- -*- -*- + +# -*- -*- -*- Begin included fragment: class/oc_storageclass.py -*- -*- -*- + +# pylint: disable=too-many-instance-attributes +class OCStorageClass(OpenShiftCLI): + ''' Class to wrap the oc command line tools ''' + kind = 'storageclass' + + # pylint allows 5 + # pylint: disable=too-many-arguments + def __init__(self, + config, + verbose=False): + ''' Constructor for OCStorageClass ''' + super(OCStorageClass, self).__init__(None, kubeconfig=config.kubeconfig, verbose=verbose) + self.config = config + self.storage_class = None + + def exists(self): + ''' return whether a storageclass exists''' + if self.storage_class: + return True + + return False + + def get(self): + '''return storageclass ''' + result = self._get(self.kind, self.config.name) + if result['returncode'] == 0: + self.storage_class = StorageClass(content=result['results'][0]) + elif '\"%s\" not found' % self.config.name in result['stderr']: + result['returncode'] = 0 + result['results'] = [{}] + + return result + + def delete(self): + '''delete the object''' + return self._delete(self.kind, self.config.name) + + def create(self): + '''create the object''' + return self._create_from_content(self.config.name, self.config.data) + + def update(self): + '''update the object''' + # parameters are currently unable to be updated. need to delete and recreate + self.delete() + # pause here and attempt to wait for delete. + # Better option would be to poll + import time + time.sleep(5) + return self.create() + + def needs_update(self): + ''' verify an update is needed ''' + # check if params have updated + if self.storage_class.get_parameters() != self.config.parameters: + return True + + for anno_key, anno_value in self.storage_class.get_annotations().items(): + if 'is-default-class' in anno_key and anno_value != self.config.default_storage_class: + return True + + return False + + @staticmethod + # pylint: disable=too-many-return-statements,too-many-branches + # TODO: This function should be refactored into its individual parts. + def run_ansible(params, check_mode): + '''run the ansible idempotent code''' + + rconfig = StorageClassConfig(params['name'], + provisioner="kubernetes.io/{}".format(params['provisioner']), + parameters=params['parameters'], + annotations=params['annotations'], + api_version="storage.k8s.io/{}".format(params['api_version']), + default_storage_class=params.get('default_storage_class', 'false'), + kubeconfig=params['kubeconfig'], + ) + + oc_sc = OCStorageClass(rconfig, verbose=params['debug']) + + state = params['state'] + + api_rval = oc_sc.get() + + ##### + # Get + ##### + if state == 'list': + return {'changed': False, 'results': api_rval['results'], 'state': 'list'} + + ######## + # Delete + ######## + if state == 'absent': + if oc_sc.exists(): + + if check_mode: + return {'changed': True, 'msg': 'Would have performed a delete.'} + + api_rval = oc_sc.delete() + + return {'changed': True, 'results': api_rval, 'state': 'absent'} + + return {'changed': False, 'state': 'absent'} + + if state == 'present': + ######## + # Create + ######## + if not oc_sc.exists(): + + if check_mode: + return {'changed': True, 'msg': 'Would have performed a create.'} + + # Create it here + api_rval = oc_sc.create() + + if api_rval['returncode'] != 0: + return {'failed': True, 'msg': api_rval} + + # return the created object + api_rval = oc_sc.get() + + if api_rval['returncode'] != 0: + return {'failed': True, 'msg': api_rval} + + return {'changed': True, 'results': api_rval, 'state': 'present'} + + ######## + # Update + ######## + if oc_sc.needs_update(): + api_rval = oc_sc.update() + + if api_rval['returncode'] != 0: + return {'failed': True, 'msg': api_rval} + + # return the created object + api_rval = oc_sc.get() + + if api_rval['returncode'] != 0: + return {'failed': True, 'msg': api_rval} + + return {'changed': True, 'results': api_rval, 'state': 'present'} + + return {'changed': False, 'results': api_rval, 'state': 'present'} + + + return {'failed': True, + 'changed': False, + 'msg': 'Unknown state passed. %s' % state, + 'state': 'unknown'} + +# -*- -*- -*- End included fragment: class/oc_storageclass.py -*- -*- -*- + +# -*- -*- -*- Begin included fragment: ansible/oc_storageclass.py -*- -*- -*- + +def main(): + ''' + ansible oc module for storageclass + ''' + + module = AnsibleModule( + argument_spec=dict( + kubeconfig=dict(default='/etc/origin/master/admin.kubeconfig', type='str'), + state=dict(default='present', type='str', choices=['present', 'absent', 'list']), + debug=dict(default=False, type='bool'), + name=dict(default=None, type='str'), + annotations=dict(default=None, type='dict'), + parameters=dict(default=None, type='dict'), + provisioner=dict(default='aws-ebs', type='str', choices=['aws-ebs', 'gce-pd', 'glusterfs', 'cinder']), + api_version=dict(default='v1', type='str'), + default_storage_class=dict(default="false", type='str'), + ), + supports_check_mode=True, + ) + + rval = OCStorageClass.run_ansible(module.params, module.check_mode) + if 'failed' in rval: + return module.fail_json(**rval) + + return module.exit_json(**rval) + + +if __name__ == '__main__': + main() + +# -*- -*- -*- End included fragment: ansible/oc_storageclass.py -*- -*- -*- diff --git a/roles/lib_openshift/src/ansible/oc_storageclass.py b/roles/lib_openshift/src/ansible/oc_storageclass.py new file mode 100644 index 000000000..2bd8f18d5 --- /dev/null +++ b/roles/lib_openshift/src/ansible/oc_storageclass.py @@ -0,0 +1,32 @@ +# pylint: skip-file +# flake8: noqa + +def main(): + ''' + ansible oc module for storageclass + ''' + + module = AnsibleModule( + argument_spec=dict( + kubeconfig=dict(default='/etc/origin/master/admin.kubeconfig', type='str'), + state=dict(default='present', type='str', choices=['present', 'absent', 'list']), + debug=dict(default=False, type='bool'), + name=dict(default=None, type='str'), + annotations=dict(default=None, type='dict'), + parameters=dict(default=None, type='dict'), + provisioner=dict(default='aws-ebs', type='str', choices=['aws-ebs', 'gce-pd', 'glusterfs', 'cinder']), + api_version=dict(default='v1', type='str'), + default_storage_class=dict(default="false", type='str'), + ), + supports_check_mode=True, + ) + + rval = OCStorageClass.run_ansible(module.params, module.check_mode) + if 'failed' in rval: + return module.fail_json(**rval) + + return module.exit_json(**rval) + + +if __name__ == '__main__': + main() diff --git a/roles/lib_openshift/src/class/oc_storageclass.py b/roles/lib_openshift/src/class/oc_storageclass.py new file mode 100644 index 000000000..aced586ae --- /dev/null +++ b/roles/lib_openshift/src/class/oc_storageclass.py @@ -0,0 +1,155 @@ +# pylint: skip-file +# flake8: noqa + +# pylint: disable=too-many-instance-attributes +class OCStorageClass(OpenShiftCLI): + ''' Class to wrap the oc command line tools ''' + kind = 'storageclass' + + # pylint allows 5 + # pylint: disable=too-many-arguments + def __init__(self, + config, + verbose=False): + ''' Constructor for OCStorageClass ''' + super(OCStorageClass, self).__init__(None, kubeconfig=config.kubeconfig, verbose=verbose) + self.config = config + self.storage_class = None + + def exists(self): + ''' return whether a storageclass exists''' + if self.storage_class: + return True + + return False + + def get(self): + '''return storageclass ''' + result = self._get(self.kind, self.config.name) + if result['returncode'] == 0: + self.storage_class = StorageClass(content=result['results'][0]) + elif '\"%s\" not found' % self.config.name in result['stderr']: + result['returncode'] = 0 + result['results'] = [{}] + + return result + + def delete(self): + '''delete the object''' + return self._delete(self.kind, self.config.name) + + def create(self): + '''create the object''' + return self._create_from_content(self.config.name, self.config.data) + + def update(self): + '''update the object''' + # parameters are currently unable to be updated. need to delete and recreate + self.delete() + # pause here and attempt to wait for delete. + # Better option would be to poll + import time + time.sleep(5) + return self.create() + + def needs_update(self): + ''' verify an update is needed ''' + # check if params have updated + if self.storage_class.get_parameters() != self.config.parameters: + return True + + for anno_key, anno_value in self.storage_class.get_annotations().items(): + if 'is-default-class' in anno_key and anno_value != self.config.default_storage_class: + return True + + return False + + @staticmethod + # pylint: disable=too-many-return-statements,too-many-branches + # TODO: This function should be refactored into its individual parts. + def run_ansible(params, check_mode): + '''run the ansible idempotent code''' + + rconfig = StorageClassConfig(params['name'], + provisioner="kubernetes.io/{}".format(params['provisioner']), + parameters=params['parameters'], + annotations=params['annotations'], + api_version="storage.k8s.io/{}".format(params['api_version']), + default_storage_class=params.get('default_storage_class', 'false'), + kubeconfig=params['kubeconfig'], + ) + + oc_sc = OCStorageClass(rconfig, verbose=params['debug']) + + state = params['state'] + + api_rval = oc_sc.get() + + ##### + # Get + ##### + if state == 'list': + return {'changed': False, 'results': api_rval['results'], 'state': 'list'} + + ######## + # Delete + ######## + if state == 'absent': + if oc_sc.exists(): + + if check_mode: + return {'changed': True, 'msg': 'Would have performed a delete.'} + + api_rval = oc_sc.delete() + + return {'changed': True, 'results': api_rval, 'state': 'absent'} + + return {'changed': False, 'state': 'absent'} + + if state == 'present': + ######## + # Create + ######## + if not oc_sc.exists(): + + if check_mode: + return {'changed': True, 'msg': 'Would have performed a create.'} + + # Create it here + api_rval = oc_sc.create() + + if api_rval['returncode'] != 0: + return {'failed': True, 'msg': api_rval} + + # return the created object + api_rval = oc_sc.get() + + if api_rval['returncode'] != 0: + return {'failed': True, 'msg': api_rval} + + return {'changed': True, 'results': api_rval, 'state': 'present'} + + ######## + # Update + ######## + if oc_sc.needs_update(): + api_rval = oc_sc.update() + + if api_rval['returncode'] != 0: + return {'failed': True, 'msg': api_rval} + + # return the created object + api_rval = oc_sc.get() + + if api_rval['returncode'] != 0: + return {'failed': True, 'msg': api_rval} + + return {'changed': True, 'results': api_rval, 'state': 'present'} + + return {'changed': False, 'results': api_rval, 'state': 'present'} + + + return {'failed': True, + 'changed': False, + 'msg': 'Unknown state passed. %s' % state, + 'state': 'unknown'} diff --git a/roles/lib_openshift/src/doc/storageclass b/roles/lib_openshift/src/doc/storageclass new file mode 100644 index 000000000..5a7320d55 --- /dev/null +++ b/roles/lib_openshift/src/doc/storageclass @@ -0,0 +1,86 @@ +# flake8: noqa +# pylint: skip-file + +DOCUMENTATION = ''' +--- +module: oc_storageclass +short_description: Create, modify, and idempotently manage openshift storageclasses. +description: + - Manage openshift storageclass objects programmatically. +options: + state: + description: + - State represents whether to create, modify, delete, or list + required: False + default: present + choices: ["present", "absent", "list"] + aliases: [] + kubeconfig: + description: + - The path for the kubeconfig file to use for authentication + required: false + default: /etc/origin/master/admin.kubeconfig + aliases: [] + debug: + description: + - Turn on debug output. + required: false + default: False + aliases: [] + name: + description: + - Name of the object that is being queried. + required: false + default: None + aliases: [] + provisioner: + description: + - Any annotations to add to the storageclass + required: false + default: 'aws-ebs' + aliases: [] + default_storage_class: + description: + - Whether or not this is the default storage class + required: false + default: False + aliases: [] + parameters: + description: + - A dictionary with the parameters to configure the storageclass. This will be based on provisioner + required: false + default: None + aliases: [] + api_version: + description: + - The api version. + required: false + default: v1 + aliases: [] +author: +- "Kenny Woodson <kwoodson@redhat.com>" +extends_documentation_fragment: [] +''' + +EXAMPLES = ''' +- name: get storageclass + run_once: true + oc_storageclass: + name: gp2 + state: list + register: registry_sc_out + +- name: create the storageclass + oc_storageclass: + run_once: true + name: gp2 + parameters: + type: gp2 + encrypted: 'true' + kmsKeyId: '<full kms key arn>' + provisioner: aws-ebs + default_storage_class: False + register: sc_out + notify: + - restart openshift master services +''' diff --git a/roles/lib_openshift/src/lib/storageclass.py b/roles/lib_openshift/src/lib/storageclass.py new file mode 100644 index 000000000..ef12a8d2d --- /dev/null +++ b/roles/lib_openshift/src/lib/storageclass.py @@ -0,0 +1,76 @@ +# pylint: skip-file +# flake8: noqa + + +# pylint: disable=too-many-instance-attributes +class StorageClassConfig(object): + ''' Handle service options ''' + # pylint: disable=too-many-arguments + def __init__(self, + name, + provisioner=None, + parameters=None, + annotations=None, + default_storage_class="false", + api_version='v1', + kubeconfig='/etc/origin/master/admin.kubeconfig'): + ''' constructor for handling storageclass options ''' + self.name = name + self.parameters = parameters + self.annotations = annotations + self.provisioner = provisioner + self.api_version = api_version + self.default_storage_class = str(default_storage_class).lower() + self.kubeconfig = kubeconfig + self.data = {} + + self.create_dict() + + def create_dict(self): + ''' instantiates a storageclass dict ''' + self.data['apiVersion'] = self.api_version + self.data['kind'] = 'StorageClass' + self.data['metadata'] = {} + self.data['metadata']['name'] = self.name + + self.data['metadata']['annotations'] = {} + if self.annotations is not None: + self.data['metadata']['annotations'] = self.annotations + + self.data['metadata']['annotations']['storageclass.beta.kubernetes.io/is-default-class'] = \ + self.default_storage_class + + if self.provisioner is None: + self.data['provisioner'] = 'kubernetes.io/aws-ebs' + else: + self.data['provisioner'] = self.provisioner + + self.data['parameters'] = {} + if self.parameters is not None: + self.data['parameters'].update(self.parameters) + + # default to aws if no params were passed + else: + self.data['parameters']['type'] = 'gp2' + + + +# pylint: disable=too-many-instance-attributes,too-many-public-methods +class StorageClass(Yedit): + ''' Class to model the oc storageclass object ''' + annotations_path = "metadata.annotations" + provisioner_path = "provisioner" + parameters_path = "parameters" + kind = 'StorageClass' + + def __init__(self, content): + '''StorageClass constructor''' + super(StorageClass, self).__init__(content=content) + + def get_annotations(self): + ''' get a list of ports ''' + return self.get(StorageClass.annotations_path) or {} + + def get_parameters(self): + ''' get the service selector''' + return self.get(StorageClass.parameters_path) or {} diff --git a/roles/lib_openshift/src/sources.yml b/roles/lib_openshift/src/sources.yml index 9fa2a6c0e..e9b6bf261 100644 --- a/roles/lib_openshift/src/sources.yml +++ b/roles/lib_openshift/src/sources.yml @@ -263,6 +263,17 @@ oc_service.py: - class/oc_service.py - ansible/oc_service.py +oc_storageclass.py: +- doc/generated +- doc/license +- lib/import.py +- doc/storageclass +- ../../lib_utils/src/class/yedit.py +- lib/base.py +- lib/storageclass.py +- class/oc_storageclass.py +- ansible/oc_storageclass.py + oc_user.py: - doc/generated - doc/license diff --git a/roles/lib_openshift/src/test/integration/oc_storageclass.yml b/roles/lib_openshift/src/test/integration/oc_storageclass.yml new file mode 100755 index 000000000..c82f9dedb --- /dev/null +++ b/roles/lib_openshift/src/test/integration/oc_storageclass.yml @@ -0,0 +1,87 @@ +#!/usr/bin/ansible-playbook --module-path=../../../library/ +# ./oc_storageclass.yml -M ../../../library -e "cli_master_test=$OPENSHIFT_MASTER +--- +- hosts: "{{ cli_master_test }}" + gather_facts: no + user: root + tasks: + - name: create a storageclass + oc_storageclass: + name: testsc + parameters: + type: gp2 + default_storage_class: "true" + register: sc_out + - debug: var=sc_out + + - assert: + that: + - "sc_out.results.results[0]['metadata']['name'] == 'testsc'" + - sc_out.changed + - "sc_out.results.results[0]['parameters']['type'] == 'gp2'" + msg: storageclass create failed. + + # Test idempotent create + - name: NOOP create the storageclass + oc_storageclass: + name: testsc + parameters: + type: gp2 + default_storage_class: "true" + register: sc_out + + - assert: + that: + - "sc_out.results.results[0]['metadata']['name'] == 'testsc'" + - sc_out.changed == False + msg: storageclass create failed. No changes expected + + - name: test list storageclass + oc_storageclass: + name: testsc + state: list + register: sc_out + - debug: var=sc_out + + - assert: + that: "sc_out.results[0]['metadata']['name'] == 'testsc'" + msg: storageclass list failed + + - name: update the storageclass + oc_storageclass: + name: testsc + parameters: + type: gp2 + encrypted: "true" + default_storage_class: "true" + register: sc_out + + - assert: + that: "sc_out.results.results[0]['parameters']['encrypted'] == 'true'" + msg: storageclass update failed + + - name: oc delete storageclass + oc_storageclass: + name: testsc + state: absent + register: sc_out + - debug: var=sc_out + + - assert: + that: + - "sc_out.results['returncode'] == 0" + - "sc_out.results.results == {}" + msg: storageclass delete failed + + - name: oc get storageclass + oc_storageclass: + name: testsc + state: list + register: sc_out + - debug: var=sc_out + + - assert: + that: + - sc_out.changed == False + - "sc_out.results == [{}]" + msg: storageclass get failed diff --git a/roles/lib_openshift/src/test/unit/test_oc_storageclass.py b/roles/lib_openshift/src/test/unit/test_oc_storageclass.py new file mode 100755 index 000000000..4fd02a8b1 --- /dev/null +++ b/roles/lib_openshift/src/test/unit/test_oc_storageclass.py @@ -0,0 +1,93 @@ +''' + Unit tests for oc serviceaccount +''' + +import os +import sys +import unittest +import mock + +# Removing invalid variable names for tests so that I can +# keep them brief +# pylint: disable=invalid-name,no-name-in-module +# Disable import-error b/c our libraries aren't loaded in jenkins +# pylint: disable=import-error +# place class in our python path +module_path = os.path.join('/'.join(os.path.realpath(__file__).split('/')[:-4]), 'library') # noqa: E501 +sys.path.insert(0, module_path) +from oc_storageclass import OCStorageClass # noqa: E402 + + +class OCStorageClassTest(unittest.TestCase): + ''' + Test class for OCStorageClass + ''' + params = {'kubeconfig': '/etc/origin/master/admin.kubeconfig', + 'state': 'present', + 'debug': False, + 'name': 'testsc', + 'provisioner': 'kubernetes.io/aws-ebs', + 'annotations': {'storageclass.beta.kubernetes.io/is-default-class': "true"}, + 'parameters': {'type': 'gp2'}, + 'api_version': 'v1', + 'default_storage_class': 'true'} + + @mock.patch('oc_storageclass.locate_oc_binary') + @mock.patch('oc_storageclass.Utils.create_tmpfile_copy') + @mock.patch('oc_storageclass.OCStorageClass._run') + def test_adding_a_storageclass(self, mock_cmd, mock_tmpfile_copy, mock_oc_binary): + ''' Testing adding a storageclass ''' + + # Arrange + + # run_ansible input parameters + + valid_result_json = '''{ + "kind": "StorageClass", + "apiVersion": "v1", + "metadata": { + "name": "testsc", + "selfLink": "/apis/storage.k8s.io/v1/storageclasses/gp2", + "uid": "4d8320c9-e66f-11e6-8edc-0eece8f2ce22", + "resourceVersion": "2828", + "creationTimestamp": "2017-01-29T22:07:19Z", + "annotations": {"storageclass.beta.kubernetes.io/is-default-class": "true"} + }, + "provisioner": "kubernetes.io/aws-ebs", + "parameters": {"type": "gp2"} + }''' + + # Return values of our mocked function call. These get returned once per call. + mock_cmd.side_effect = [ + # First call to mock + (1, '', 'Error from server: storageclass "testsc" not found'), + + # Second call to mock + (0, 'storageclass "testsc" created', ''), + + # Third call to mock + (0, valid_result_json, ''), + ] + + mock_oc_binary.side_effect = [ + 'oc' + ] + + mock_tmpfile_copy.side_effect = [ + '/tmp/mocked_kubeconfig', + ] + + # Act + results = OCStorageClass.run_ansible(OCStorageClassTest.params, False) + + # Assert + self.assertTrue(results['changed']) + self.assertEqual(results['results']['returncode'], 0) + self.assertEqual(results['state'], 'present') + + # Making sure our mock was called as we expected + mock_cmd.assert_has_calls([ + mock.call(['oc', 'get', 'storageclass', 'testsc', '-o', 'json'], None), + mock.call(['oc', 'create', '-f', mock.ANY], None), + mock.call(['oc', 'get', 'storageclass', 'testsc', '-o', 'json'], None), + ]) diff --git a/roles/openshift_default_storage_class/defaults/main.yml b/roles/openshift_default_storage_class/defaults/main.yml index 66ffd2a73..bdece7640 100644 --- a/roles/openshift_default_storage_class/defaults/main.yml +++ b/roles/openshift_default_storage_class/defaults/main.yml @@ -1,14 +1,19 @@ --- openshift_storageclass_defaults: aws: + provisioner: aws-ebs name: gp2 - provisioner: kubernetes.io/aws-ebs - type: gp2 + parameters: + type: gp2 + kmsKeyId: '' + encrypted: 'false' gce: name: standard - provisioner: kubernetes.io/gce-pd - type: pd-standard + provisioner: gce-pd + parameters: + type: pd-standard +openshift_storageclass_default: "true" openshift_storageclass_name: "{{ openshift_storageclass_defaults[openshift_cloudprovider_kind]['name'] }}" openshift_storageclass_provisioner: "{{ openshift_storageclass_defaults[openshift_cloudprovider_kind]['provisioner'] }}" -openshift_storageclass_type: "{{ openshift_storageclass_defaults[openshift_cloudprovider_kind]['type'] }}" +openshift_storageclass_parameters: "{{ openshift_storageclass_defaults[openshift_cloudprovider_kind]['parameters'] }}" diff --git a/roles/openshift_default_storage_class/tasks/main.yml b/roles/openshift_default_storage_class/tasks/main.yml index 408fc17c7..172e2ac25 100644 --- a/roles/openshift_default_storage_class/tasks/main.yml +++ b/roles/openshift_default_storage_class/tasks/main.yml @@ -1,19 +1,9 @@ --- # Install default storage classes in GCE & AWS - name: Ensure storageclass object - oc_obj: - kind: storageclass + oc_storageclass: name: "{{ openshift_storageclass_name }}" - content: - path: /tmp/openshift_storageclass - data: - kind: StorageClass - apiVersion: storage.k8s.io/v1beta1 - metadata: - name: "{{ openshift_storageclass_name }}" - annotations: - storageclass.beta.kubernetes.io/is-default-class: "true" - provisioner: "{{ openshift_storageclass_provisioner }}" - parameters: - type: "{{ openshift_storageclass_type }}" + default_storage_class: "{{ openshift_storageclass_default | default('true') | string}}" + parameters: "{{ openshift_storageclass_parameters }}" + provisioner: "{{ openshift_storageclass_provisioner }}" run_once: true diff --git a/roles/openshift_examples/tasks/main.yml b/roles/openshift_examples/tasks/main.yml index 551e21e72..1a4562776 100644 --- a/roles/openshift_examples/tasks/main.yml +++ b/roles/openshift_examples/tasks/main.yml @@ -53,7 +53,7 @@ # RHEL and Centos image streams are mutually exclusive - name: Import RHEL streams command: > - {{ openshift.common.client_binary }} {{ openshift_examples_import_command }} -n openshift -f {{ item }} + {{ openshift.common.client_binary }} {{ openshift_examples_import_command }} --config={{ openshift.common.config_base }}/master/admin.kubeconfig -n openshift -f {{ item }} when: openshift_examples_load_rhel | bool with_items: - "{{ rhel_image_streams }}" @@ -63,7 +63,7 @@ - name: Import Centos Image streams command: > - {{ openshift.common.client_binary }} {{ openshift_examples_import_command }} -n openshift -f {{ centos_image_streams }} + {{ openshift.common.client_binary }} {{ openshift_examples_import_command }} --config={{ openshift.common.config_base }}/master/admin.kubeconfig -n openshift -f {{ centos_image_streams }} when: openshift_examples_load_centos | bool register: oex_import_centos_streams failed_when: "'already exists' not in oex_import_centos_streams.stderr and oex_import_centos_streams.rc != 0" @@ -71,7 +71,7 @@ - name: Import db templates command: > - {{ openshift.common.client_binary }} {{ openshift_examples_import_command }} -n openshift -f {{ db_templates_base }} + {{ openshift.common.client_binary }} {{ openshift_examples_import_command }} --config={{ openshift.common.config_base }}/master/admin.kubeconfig -n openshift -f {{ db_templates_base }} when: openshift_examples_load_db_templates | bool register: oex_import_db_templates failed_when: "'already exists' not in oex_import_db_templates.stderr and oex_import_db_templates.rc != 0" @@ -88,7 +88,7 @@ - "{{ quickstarts_base }}/django.json" - name: Remove defunct quickstart templates from openshift namespace - command: "{{ openshift.common.client_binary }} -n openshift delete templates/{{ item }}" + command: "{{ openshift.common.client_binary }} --config={{ openshift.common.config_base }}/master/admin.kubeconfig -n openshift delete templates/{{ item }}" with_items: - nodejs-example - cakephp-example @@ -100,7 +100,7 @@ - name: Import quickstart-templates command: > - {{ openshift.common.client_binary }} {{ openshift_examples_import_command }} -n openshift -f {{ quickstarts_base }} + {{ openshift.common.client_binary }} {{ openshift_examples_import_command }} --config={{ openshift.common.config_base }}/master/admin.kubeconfig -n openshift -f {{ quickstarts_base }} when: openshift_examples_load_quickstarts | bool register: oex_import_quickstarts failed_when: "'already exists' not in oex_import_quickstarts.stderr and oex_import_quickstarts.rc != 0" @@ -114,7 +114,7 @@ - "{{ xpaas_templates_base }}/sso70-basic.json" - name: Remove old xPaas templates from openshift namespace - command: "{{ openshift.common.client_binary }} -n openshift delete templates/{{ item }}" + command: "{{ openshift.common.client_binary }} --config={{ openshift.common.config_base }}/master/admin.kubeconfig -n openshift delete templates/{{ item }}" with_items: - sso70-basic register: oex_delete_old_xpaas_templates @@ -123,7 +123,7 @@ - name: Import xPaas image streams command: > - {{ openshift.common.client_binary }} {{ openshift_examples_import_command }} -n openshift -f {{ xpaas_image_streams }} + {{ openshift.common.client_binary }} {{ openshift_examples_import_command }} --config={{ openshift.common.config_base }}/master/admin.kubeconfig -n openshift -f {{ xpaas_image_streams }} when: openshift_examples_load_xpaas | bool register: oex_import_xpaas_streams failed_when: "'already exists' not in oex_import_xpaas_streams.stderr and oex_import_xpaas_streams.rc != 0" @@ -131,7 +131,7 @@ - name: Import xPaas templates command: > - {{ openshift.common.client_binary }} {{ openshift_examples_import_command }} -n openshift -f {{ xpaas_templates_base }} + {{ openshift.common.client_binary }} {{ openshift_examples_import_command }} --config={{ openshift.common.config_base }}/master/admin.kubeconfig -n openshift -f {{ xpaas_templates_base }} when: openshift_examples_load_xpaas | bool register: oex_import_xpaas_templates failed_when: "'already exists' not in oex_import_xpaas_templates.stderr and oex_import_xpaas_templates.rc != 0" diff --git a/roles/openshift_excluder/tasks/exclude.yml b/roles/openshift_excluder/tasks/exclude.yml index 934f1b2d2..1b4818df9 100644 --- a/roles/openshift_excluder/tasks/exclude.yml +++ b/roles/openshift_excluder/tasks/exclude.yml @@ -5,7 +5,7 @@ register: docker_excluder_stat - name: Enable docker excluder - command: "{{ r_openshift_excluder_service_type }}-docker-excluder exclude" + command: "/sbin/{{ r_openshift_excluder_service_type }}-docker-excluder exclude" when: - r_openshift_excluder_enable_docker_excluder | bool - docker_excluder_stat.stat.exists @@ -16,7 +16,7 @@ register: openshift_excluder_stat - name: Enable openshift excluder - command: "{{ r_openshift_excluder_service_type }}-excluder exclude" + command: "/sbin/{{ r_openshift_excluder_service_type }}-excluder exclude" when: - r_openshift_excluder_enable_openshift_excluder | bool - openshift_excluder_stat.stat.exists diff --git a/roles/openshift_excluder/tasks/unexclude.yml b/roles/openshift_excluder/tasks/unexclude.yml index a5ce8d5c7..a68165bde 100644 --- a/roles/openshift_excluder/tasks/unexclude.yml +++ b/roles/openshift_excluder/tasks/unexclude.yml @@ -9,7 +9,7 @@ register: docker_excluder_stat - name: disable docker excluder - command: "{{ r_openshift_excluder_service_type }}-docker-excluder unexclude" + command: "/sbin/{{ r_openshift_excluder_service_type }}-docker-excluder unexclude" when: - unexclude_docker_excluder | default(false) | bool - docker_excluder_stat.stat.exists @@ -20,7 +20,7 @@ register: openshift_excluder_stat - name: disable openshift excluder - command: "{{ r_openshift_excluder_service_type }}-excluder unexclude" + command: "/sbin/{{ r_openshift_excluder_service_type }}-excluder unexclude" when: - unexclude_openshift_excluder | default(false) | bool - openshift_excluder_stat.stat.exists diff --git a/roles/openshift_facts/library/openshift_facts.py b/roles/openshift_facts/library/openshift_facts.py index 04b5dc86b..49cc51b48 100755 --- a/roles/openshift_facts/library/openshift_facts.py +++ b/roles/openshift_facts/library/openshift_facts.py @@ -1642,39 +1642,28 @@ def set_proxy_facts(facts): """ if 'common' in facts: common = facts['common'] + if 'http_proxy' in common or 'https_proxy' in common or 'no_proxy' in common: + if 'no_proxy' in common and isinstance(common['no_proxy'], string_types): + common['no_proxy'] = common['no_proxy'].split(",") + elif 'no_proxy' not in common: + common['no_proxy'] = [] + + # See https://bugzilla.redhat.com/show_bug.cgi?id=1466783 + # masters behind a proxy need to connect to etcd via IP + if 'no_proxy_etcd_host_ips' in common: + if isinstance(common['no_proxy_etcd_host_ips'], string_types): + common['no_proxy'].extend(common['no_proxy_etcd_host_ips'].split(',')) - # No openshift_no_proxy settings detected, empty list for now - if 'no_proxy' not in common: - common['no_proxy'] = [] - - # _no_proxy settings set. It is just a simple string, not a - # list or anything - elif 'no_proxy' in common and isinstance(common['no_proxy'], string_types): - # no_proxy is now a list of all the comma-separated items - # in the _no_proxy value - common['no_proxy'] = common['no_proxy'].split(",") - - # at this point common['no_proxy'] is a LIST datastructure. It - # may be empty, or it may contain some hostnames or ranges. - - # We always add local dns domain, the service domain, and - # ourselves, no matter what (if you are setting any - # NO_PROXY values) - common['no_proxy'].append('.svc') - common['no_proxy'].append('.' + common['dns_domain']) - common['no_proxy'].append(common['hostname']) - - # You are also setting system proxy vars, openshift_http_proxy/openshift_https_proxy - if 'http_proxy' in common or 'https_proxy' in common: - # You want to generate no_proxy hosts and it's a boolean value if 'generate_no_proxy_hosts' in common and safe_get_bool(common['generate_no_proxy_hosts']): - # And you want to set up no_proxy for internal hostnames if 'no_proxy_internal_hostnames' in common: - # Split the internal_hostnames string by a comma - # and add that list to the overall no_proxy list common['no_proxy'].extend(common['no_proxy_internal_hostnames'].split(',')) + # We always add local dns domain and ourselves no matter what + common['no_proxy'].append('.' + common['dns_domain']) + common['no_proxy'].append('.svc') + common['no_proxy'].append(common['hostname']) + common['no_proxy'] = ','.join(sort_unique(common['no_proxy'])) + facts['common'] = common - common['no_proxy'] = ','.join(sort_unique(common['no_proxy'])) return facts diff --git a/roles/openshift_health_checker/action_plugins/openshift_health_check.py b/roles/openshift_health_checker/action_plugins/openshift_health_check.py index 0390dc82e..581dd7d15 100644 --- a/roles/openshift_health_checker/action_plugins/openshift_health_check.py +++ b/roles/openshift_health_checker/action_plugins/openshift_health_check.py @@ -37,7 +37,7 @@ class ActionModule(ActionBase): return result try: - known_checks = self.load_known_checks() + known_checks = self.load_known_checks(tmp, task_vars) args = self._task.args resolved_checks = resolve_checks(args.get("checks", []), known_checks.values()) except OpenShiftCheckException as e: @@ -56,13 +56,13 @@ class ActionModule(ActionBase): display.banner("CHECK [{} : {}]".format(check_name, task_vars["ansible_host"])) check = known_checks[check_name] - if not check.is_active(task_vars): + if not check.is_active(): r = dict(skipped=True, skipped_reason="Not active for this host") elif check_name in user_disabled_checks: r = dict(skipped=True, skipped_reason="Disabled by user request") else: try: - r = check.run(tmp, task_vars) + r = check.run() except OpenShiftCheckException as e: r = dict( failed=True, @@ -78,7 +78,7 @@ class ActionModule(ActionBase): result["changed"] = any(r.get("changed", False) for r in check_results.values()) return result - def load_known_checks(self): + def load_known_checks(self, tmp, task_vars): load_checks() known_checks = {} @@ -91,7 +91,7 @@ class ActionModule(ActionBase): check_name, cls.__module__, cls.__name__, other_cls.__module__, other_cls.__name__)) - known_checks[check_name] = cls(execute_module=self._execute_module) + known_checks[check_name] = cls(execute_module=self._execute_module, tmp=tmp, task_vars=task_vars) return known_checks diff --git a/roles/openshift_health_checker/callback_plugins/zz_failure_summary.py b/roles/openshift_health_checker/callback_plugins/zz_failure_summary.py index 443b76ea1..d10200719 100644 --- a/roles/openshift_health_checker/callback_plugins/zz_failure_summary.py +++ b/roles/openshift_health_checker/callback_plugins/zz_failure_summary.py @@ -1,6 +1,6 @@ -''' -Ansible callback plugin. -''' +""" +Ansible callback plugin to give a nicely formatted summary of failures. +""" # Reason: In several locations below we disable pylint protected-access # for Ansible objects that do not give us any public way @@ -16,11 +16,11 @@ from ansible.utils.color import stringc class CallbackModule(CallbackBase): - ''' + """ This callback plugin stores task results and summarizes failures. The file name is prefixed with `zz_` to make this plugin be loaded last by Ansible, thus making its output the last thing that users see. - ''' + """ CALLBACK_VERSION = 2.0 CALLBACK_TYPE = 'aggregate' @@ -48,7 +48,7 @@ class CallbackModule(CallbackBase): self._print_failure_details(self.__failures) def _print_failure_details(self, failures): - '''Print a summary of failed tasks or checks.''' + """Print a summary of failed tasks or checks.""" self._display.display(u'\nFailure summary:\n') width = len(str(len(failures))) @@ -69,7 +69,9 @@ class CallbackModule(CallbackBase): playbook_context = None # re: result attrs see top comment # pylint: disable=protected-access for failure in failures: - # get context from check task result since callback plugins cannot access task vars + # Get context from check task result since callback plugins cannot access task vars. + # NOTE: thus context is not known unless checks run. Failures prior to checks running + # don't have playbook_context in the results. But we only use it now when checks fail. playbook_context = playbook_context or failure['result']._result.get('playbook_context') failed_checks.update( name @@ -81,8 +83,11 @@ class CallbackModule(CallbackBase): def _print_check_failure_summary(self, failed_checks, context): checks = ','.join(sorted(failed_checks)) - # NOTE: context is not set if all failures occurred prior to checks task - summary = ( + # The purpose of specifying context is to vary the output depending on what the user was + # expecting to happen (based on which playbook they ran). The only use currently is to + # vary the message depending on whether the user was deliberately running checks or was + # trying to install/upgrade and checks are just included. Other use cases may arise. + summary = ( # default to explaining what checks are in the first place '\n' 'The execution of "{playbook}"\n' 'includes checks designed to fail early if the requirements\n' @@ -94,27 +99,26 @@ class CallbackModule(CallbackBase): 'Some checks may be configurable by variables if your requirements\n' 'are different from the defaults; consult check documentation.\n' 'Variables can be set in the inventory or passed on the\n' - 'command line using the -e flag to ansible-playbook.\n' + 'command line using the -e flag to ansible-playbook.\n\n' ).format(playbook=self._playbook_file, checks=checks) if context in ['pre-install', 'health']: - summary = ( + summary = ( # user was expecting to run checks, less explanation needed '\n' 'You may choose to configure or disable failing checks by\n' 'setting Ansible variables. To disable those above:\n\n' ' openshift_disable_check={checks}\n\n' 'Consult check documentation for configurable variables.\n' 'Variables can be set in the inventory or passed on the\n' - 'command line using the -e flag to ansible-playbook.\n' + 'command line using the -e flag to ansible-playbook.\n\n' ).format(checks=checks) - # other expected contexts: install, upgrade self._display.display(summary) # re: result attrs see top comment # pylint: disable=protected-access def _format_failure(failure): - '''Return a list of pretty-formatted text entries describing a failure, including + """Return a list of pretty-formatted text entries describing a failure, including relevant information about it. Expect that the list of text entries will be joined - by a newline separator when output to the user.''' + by a newline separator when output to the user.""" result = failure['result'] host = result._host.get_name() play = _get_play(result._task) @@ -135,7 +139,7 @@ def _format_failure(failure): def _format_failed_checks(checks): - '''Return pretty-formatted text describing checks that failed.''' + """Return pretty-formatted text describing checks that failed.""" failed_check_msgs = [] for check, body in checks.items(): if body.get('failed', False): # only show the failed checks @@ -150,7 +154,7 @@ def _format_failed_checks(checks): # This is inspired by ansible.playbook.base.Base.dump_me. # re: play/task/block attrs see top comment # pylint: disable=protected-access def _get_play(obj): - '''Given a task or block, recursively tries to find its parent play.''' + """Given a task or block, recursively try to find its parent play.""" if hasattr(obj, '_play'): return obj._play if getattr(obj, '_parent'): diff --git a/roles/openshift_health_checker/library/aos_version.py b/roles/openshift_health_checker/library/aos_version.py index 4c205e48c..f9babebb9 100755..100644 --- a/roles/openshift_health_checker/library/aos_version.py +++ b/roles/openshift_health_checker/library/aos_version.py @@ -1,5 +1,5 @@ #!/usr/bin/python -''' +""" Ansible module for yum-based systems determining if multiple releases of an OpenShift package are available, and if the release requested (if any) is available down to the given precision. @@ -16,9 +16,13 @@ of release availability already. Without duplicating all that, we would like the user to have a helpful error message if we detect things will not work out right. Note that if openshift_release is not specified in the inventory, the version comparison checks just pass. -''' +""" from ansible.module_utils.basic import AnsibleModule +# NOTE: because of the dependency on yum (Python 2-only), this module does not +# work under Python 3. But since we run unit tests against both Python 2 and +# Python 3, we use six for cross compatibility in this module alone: +from ansible.module_utils.six import string_types IMPORT_EXCEPTION = None try: @@ -28,7 +32,7 @@ except ImportError as err: class AosVersionException(Exception): - '''Base exception class for package version problems''' + """Base exception class for package version problems""" def __init__(self, message, problem_pkgs=None): Exception.__init__(self, message) self.problem_pkgs = problem_pkgs @@ -122,12 +126,15 @@ def _check_precise_version_found(pkgs, expected_pkgs_dict): for pkg in pkgs: if pkg.name not in expected_pkgs_dict: continue - # does the version match, to the precision requested? - # and, is it strictly greater, at the precision requested? - expected_pkg_version = expected_pkgs_dict[pkg.name]["version"] - match_version = '.'.join(pkg.version.split('.')[:expected_pkg_version.count('.') + 1]) - if match_version == expected_pkg_version: - pkgs_precise_version_found.add(pkg.name) + expected_pkg_versions = expected_pkgs_dict[pkg.name]["version"] + if isinstance(expected_pkg_versions, string_types): + expected_pkg_versions = [expected_pkg_versions] + for expected_pkg_version in expected_pkg_versions: + # does the version match, to the precision requested? + # and, is it strictly greater, at the precision requested? + match_version = '.'.join(pkg.version.split('.')[:expected_pkg_version.count('.') + 1]) + if match_version == expected_pkg_version: + pkgs_precise_version_found.add(pkg.name) not_found = [] for name, pkg in expected_pkgs_dict.items(): @@ -157,8 +164,13 @@ def _check_higher_version_found(pkgs, expected_pkgs_dict): for pkg in pkgs: if pkg.name not in expected_pkg_names: continue - expected_pkg_version = expected_pkgs_dict[pkg.name]["version"] - req_release_arr = [int(segment) for segment in expected_pkg_version.split(".")] + expected_pkg_versions = expected_pkgs_dict[pkg.name]["version"] + if isinstance(expected_pkg_versions, string_types): + expected_pkg_versions = [expected_pkg_versions] + # NOTE: the list of versions is assumed to be sorted so that the highest + # desirable version is the last. + highest_desirable_version = expected_pkg_versions[-1] + req_release_arr = [int(segment) for segment in highest_desirable_version.split(".")] version = [int(segment) for segment in pkg.version.split(".")] too_high = version[:len(req_release_arr)] > req_release_arr higher_than_seen = version > higher_version_for_pkg.get(pkg.name, []) diff --git a/roles/openshift_health_checker/library/check_yum_update.py b/roles/openshift_health_checker/library/check_yum_update.py index 433795b67..433795b67 100755..100644 --- a/roles/openshift_health_checker/library/check_yum_update.py +++ b/roles/openshift_health_checker/library/check_yum_update.py diff --git a/roles/openshift_health_checker/library/docker_info.py b/roles/openshift_health_checker/library/docker_info.py index 7f712bcff..0d0ddae8b 100644 --- a/roles/openshift_health_checker/library/docker_info.py +++ b/roles/openshift_health_checker/library/docker_info.py @@ -1,4 +1,3 @@ -# pylint: disable=missing-docstring """ Ansible module for determining information about the docker host. @@ -13,6 +12,7 @@ from ansible.module_utils.docker_common import AnsibleDockerClient def main(): + """Entrypoint for running an Ansible module.""" client = AnsibleDockerClient() client.module.exit_json( diff --git a/roles/openshift_health_checker/library/search_journalctl.py b/roles/openshift_health_checker/library/search_journalctl.py new file mode 100644 index 000000000..3631f71c8 --- /dev/null +++ b/roles/openshift_health_checker/library/search_journalctl.py @@ -0,0 +1,150 @@ +#!/usr/bin/python +"""Interface to journalctl.""" + +from time import time +import json +import re +import subprocess + +from ansible.module_utils.basic import AnsibleModule + + +class InvalidMatcherRegexp(Exception): + """Exception class for invalid matcher regexp.""" + pass + + +class InvalidLogEntry(Exception): + """Exception class for invalid / non-json log entries.""" + pass + + +class LogInputSubprocessError(Exception): + """Exception class for errors that occur while executing a subprocess.""" + pass + + +def main(): + """Scan a given list of "log_matchers" for journalctl messages containing given patterns. + "log_matchers" is a list of dicts consisting of three keys that help fine-tune log searching: + 'start_regexp', 'regexp', and 'unit'. + + Sample "log_matchers" list: + + [ + { + 'start_regexp': r'Beginning of systemd unit', + 'regexp': r'the specific log message to find', + 'unit': 'etcd', + } + ] + """ + module = AnsibleModule( + argument_spec=dict( + log_count_limit=dict(type="int", default=500), + log_matchers=dict(type="list", required=True), + ), + ) + + timestamp_limit_seconds = time() - 60 * 60 # 1 hour + + log_count_limit = module.params["log_count_limit"] + log_matchers = module.params["log_matchers"] + + matched_regexp, errors = get_log_matches(log_matchers, log_count_limit, timestamp_limit_seconds) + + module.exit_json( + changed=False, + failed=bool(errors), + errors=errors, + matched=matched_regexp, + ) + + +def get_log_matches(matchers, log_count_limit, timestamp_limit_seconds): + """Return a list of up to log_count_limit matches for each matcher. + + Log entries are only considered if newer than timestamp_limit_seconds. + """ + matched_regexp = [] + errors = [] + + for matcher in matchers: + try: + log_output = get_log_output(matcher) + except LogInputSubprocessError as err: + errors.append(str(err)) + continue + + try: + matched = find_matches(log_output, matcher, log_count_limit, timestamp_limit_seconds) + if matched: + matched_regexp.append(matcher.get("regexp", "")) + except InvalidMatcherRegexp as err: + errors.append(str(err)) + except InvalidLogEntry as err: + errors.append(str(err)) + + return matched_regexp, errors + + +def get_log_output(matcher): + """Return an iterator on the logs of a given matcher.""" + try: + cmd_output = subprocess.Popen(list([ + '/bin/journalctl', + '-ru', matcher.get("unit", ""), + '--output', 'json', + ]), stdout=subprocess.PIPE) + + return iter(cmd_output.stdout.readline, '') + + except subprocess.CalledProcessError as exc: + msg = "Could not obtain journalctl logs for the specified systemd unit: {}: {}" + raise LogInputSubprocessError(msg.format(matcher.get("unit", "<missing>"), str(exc))) + except OSError as exc: + raise LogInputSubprocessError(str(exc)) + + +def find_matches(log_output, matcher, log_count_limit, timestamp_limit_seconds): + """Return log messages matched in iterable log_output by a given matcher. + + Ignore any log_output items older than timestamp_limit_seconds. + """ + try: + regexp = re.compile(matcher.get("regexp", "")) + start_regexp = re.compile(matcher.get("start_regexp", "")) + except re.error as err: + msg = "A log matcher object was provided with an invalid regular expression: {}" + raise InvalidMatcherRegexp(msg.format(str(err))) + + matched = None + + for log_count, line in enumerate(log_output): + if log_count >= log_count_limit: + break + + try: + obj = json.loads(line) + + # don't need to look past the most recent service restart + if start_regexp.match(obj["MESSAGE"]): + break + + log_timestamp_seconds = float(obj["__REALTIME_TIMESTAMP"]) / 1000000 + if log_timestamp_seconds < timestamp_limit_seconds: + break + + if regexp.match(obj["MESSAGE"]): + matched = line + break + + except ValueError: + msg = "Log entry for systemd unit {} contained invalid json syntax: {}" + raise InvalidLogEntry(msg.format(matcher.get("unit"), line)) + + return matched + + +if __name__ == '__main__': + main() diff --git a/roles/openshift_health_checker/openshift_checks/__init__.py b/roles/openshift_health_checker/openshift_checks/__init__.py index 5c9949ced..40a28cde5 100644 --- a/roles/openshift_health_checker/openshift_checks/__init__.py +++ b/roles/openshift_health_checker/openshift_checks/__init__.py @@ -19,15 +19,21 @@ class OpenShiftCheckException(Exception): @six.add_metaclass(ABCMeta) class OpenShiftCheck(object): - """A base class for defining checks for an OpenShift cluster environment.""" + """ + A base class for defining checks for an OpenShift cluster environment. + + Expect optional params: method execute_module, dict task_vars, and string tmp. + execute_module is expected to have a signature compatible with _execute_module + from ansible plugins/action/__init__.py, e.g.: + def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None, *args): + This is stored so that it can be invoked in subclasses via check.execute_module("name", args) + which provides the check's stored task_vars and tmp. + """ - def __init__(self, execute_module=None, module_executor=None): - if execute_module is module_executor is None: - raise TypeError( - "__init__() takes either execute_module (recommended) " - "or module_executor (deprecated), none given") - self.execute_module = execute_module or module_executor - self.module_executor = self.execute_module + def __init__(self, execute_module=None, task_vars=None, tmp=None): + self._execute_module = execute_module + self.task_vars = task_vars or {} + self.tmp = tmp @abstractproperty def name(self): @@ -43,13 +49,13 @@ class OpenShiftCheck(object): """ return [] - @classmethod - def is_active(cls, task_vars): # pylint: disable=unused-argument + @staticmethod + def is_active(): """Returns true if this check applies to the ansible-playbook run.""" return True @abstractmethod - def run(self, tmp, task_vars): + def run(self): """Executes a check, normally implemented as a module.""" return {} @@ -62,6 +68,43 @@ class OpenShiftCheck(object): for subclass in subclass.subclasses(): yield subclass + def execute_module(self, module_name=None, module_args=None): + """Invoke an Ansible module from a check. + + Invoke stored _execute_module, normally copied from the action + plugin, with its params and the task_vars and tmp given at + check initialization. No positional parameters beyond these + are specified. If it's necessary to specify any of the other + parameters to _execute_module then that should just be invoked + directly (with awareness of changes in method signature per + Ansible version). + + So e.g. check.execute_module("foo", dict(arg1=...)) + Return: result hash from module execution. + """ + if self._execute_module is None: + raise NotImplementedError( + self.__class__.__name__ + + " invoked execute_module without providing the method at initialization." + ) + return self._execute_module(module_name, module_args, self.tmp, self.task_vars) + + def get_var(self, *keys, **kwargs): + """Get deeply nested values from task_vars. + + Ansible task_vars structures are Python dicts, often mapping strings to + other dicts. This helper makes it easier to get a nested value, raising + OpenShiftCheckException when a key is not found or returning a default value + provided as a keyword argument. + """ + try: + value = reduce(operator.getitem, keys, self.task_vars) + except (KeyError, TypeError): + if "default" in kwargs: + return kwargs["default"] + raise OpenShiftCheckException("'{}' is undefined".format(".".join(map(str, keys)))) + return value + LOADER_EXCLUDES = ( "__init__.py", @@ -86,20 +129,3 @@ def load_checks(path=None, subpkg=""): modules.append(import_module(__package__ + subpkg + "." + name[:-3])) return modules - - -def get_var(task_vars, *keys, **kwargs): - """Helper function to get deeply nested values from task_vars. - - Ansible task_vars structures are Python dicts, often mapping strings to - other dicts. This helper makes it easier to get a nested value, raising - OpenShiftCheckException when a key is not found or returning a default value - provided as a keyword argument. - """ - try: - value = reduce(operator.getitem, keys, task_vars) - except (KeyError, TypeError): - if "default" in kwargs: - return kwargs["default"] - raise OpenShiftCheckException("'{}' is undefined".format(".".join(map(str, keys)))) - return value diff --git a/roles/openshift_health_checker/openshift_checks/disk_availability.py b/roles/openshift_health_checker/openshift_checks/disk_availability.py index e93e81efa..283461294 100644 --- a/roles/openshift_health_checker/openshift_checks/disk_availability.py +++ b/roles/openshift_health_checker/openshift_checks/disk_availability.py @@ -3,7 +3,7 @@ import os.path import tempfile -from openshift_checks import OpenShiftCheck, OpenShiftCheckException, get_var +from openshift_checks import OpenShiftCheck, OpenShiftCheckException class DiskAvailability(OpenShiftCheck): @@ -35,22 +35,21 @@ class DiskAvailability(OpenShiftCheck): }, } - @classmethod - def is_active(cls, task_vars): + def is_active(self): """Skip hosts that do not have recommended disk space requirements.""" - group_names = get_var(task_vars, "group_names", default=[]) + group_names = self.get_var("group_names", default=[]) active_groups = set() - for recommendation in cls.recommended_disk_space_bytes.values(): + for recommendation in self.recommended_disk_space_bytes.values(): active_groups.update(recommendation.keys()) has_disk_space_recommendation = bool(active_groups.intersection(group_names)) - return super(DiskAvailability, cls).is_active(task_vars) and has_disk_space_recommendation + return super(DiskAvailability, self).is_active() and has_disk_space_recommendation - def run(self, tmp, task_vars): - group_names = get_var(task_vars, "group_names") - ansible_mounts = get_var(task_vars, "ansible_mounts") + def run(self): + group_names = self.get_var("group_names") + ansible_mounts = self.get_var("ansible_mounts") ansible_mounts = {mount['mount']: mount for mount in ansible_mounts} - user_config = get_var(task_vars, "openshift_check_min_host_disk_gb", default={}) + user_config = self.get_var("openshift_check_min_host_disk_gb", default={}) try: # For backwards-compatibility, if openshift_check_min_host_disk_gb # is a number, then it overrides the required config for '/var'. diff --git a/roles/openshift_health_checker/openshift_checks/docker_image_availability.py b/roles/openshift_health_checker/openshift_checks/docker_image_availability.py index bde81ad2c..77180223e 100644 --- a/roles/openshift_health_checker/openshift_checks/docker_image_availability.py +++ b/roles/openshift_health_checker/openshift_checks/docker_image_availability.py @@ -1,6 +1,6 @@ """Check that required Docker images are available.""" -from openshift_checks import OpenShiftCheck, get_var +from openshift_checks import OpenShiftCheck from openshift_checks.mixins import DockerHostMixin @@ -22,25 +22,26 @@ DEPLOYMENT_IMAGE_INFO = { class DockerImageAvailability(DockerHostMixin, OpenShiftCheck): """Check that required Docker images are available. - This check attempts to ensure that required docker images are - either present locally, or able to be pulled down from available - registries defined in a host machine. + Determine docker images that an install would require and check that they + are either present in the host's docker index, or available for the host to pull + with known registries as defined in our inventory file (or defaults). """ name = "docker_image_availability" tags = ["preflight"] - dependencies = ["skopeo", "python-docker-py"] + # we use python-docker-py to check local docker for images, and skopeo + # to look for images available remotely without waiting to pull them. + dependencies = ["python-docker-py", "skopeo"] - @classmethod - def is_active(cls, task_vars): + def is_active(self): """Skip hosts with unsupported deployment types.""" - deployment_type = get_var(task_vars, "openshift_deployment_type") + deployment_type = self.get_var("openshift_deployment_type") has_valid_deployment_type = deployment_type in DEPLOYMENT_IMAGE_INFO - return super(DockerImageAvailability, cls).is_active(task_vars) and has_valid_deployment_type + return super(DockerImageAvailability, self).is_active() and has_valid_deployment_type - def run(self, tmp, task_vars): - msg, failed, changed = self.ensure_dependencies(task_vars) + def run(self): + msg, failed, changed = self.ensure_dependencies() if failed: return { "failed": True, @@ -48,18 +49,18 @@ class DockerImageAvailability(DockerHostMixin, OpenShiftCheck): "msg": "Some dependencies are required in order to check Docker image availability.\n" + msg } - required_images = self.required_images(task_vars) - missing_images = set(required_images) - set(self.local_images(required_images, task_vars)) + required_images = self.required_images() + missing_images = set(required_images) - set(self.local_images(required_images)) # exit early if all images were found locally if not missing_images: return {"changed": changed} - registries = self.known_docker_registries(task_vars) + registries = self.known_docker_registries() if not registries: return {"failed": True, "msg": "Unable to retrieve any docker registries.", "changed": changed} - available_images = self.available_images(missing_images, registries, task_vars) + available_images = self.available_images(missing_images, registries) unavailable_images = set(missing_images) - set(available_images) if unavailable_images: @@ -74,8 +75,7 @@ class DockerImageAvailability(DockerHostMixin, OpenShiftCheck): return {"changed": changed} - @staticmethod - def required_images(task_vars): + def required_images(self): """ Determine which images we expect to need for this host. Returns: a set of required images like 'openshift/origin:v3.6' @@ -92,17 +92,17 @@ class DockerImageAvailability(DockerHostMixin, OpenShiftCheck): Registry is not included in constructed images. It may be in oreg_url or etcd image. """ required = set() - deployment_type = get_var(task_vars, "openshift_deployment_type") - host_groups = get_var(task_vars, "group_names") + deployment_type = self.get_var("openshift_deployment_type") + host_groups = self.get_var("group_names") # containerized etcd may not have openshift_image_tag, see bz 1466622 - image_tag = get_var(task_vars, "openshift_image_tag", default="latest") + image_tag = self.get_var("openshift_image_tag", default="latest") image_info = DEPLOYMENT_IMAGE_INFO[deployment_type] if not image_info: return required # template for images that run on top of OpenShift image_url = "{}/{}-{}:{}".format(image_info["namespace"], image_info["name"], "${component}", "${version}") - image_url = get_var(task_vars, "oreg_url", default="") or image_url + image_url = self.get_var("oreg_url", default="") or image_url if 'nodes' in host_groups: for suffix in NODE_IMAGE_SUFFIXES: required.add(image_url.replace("${component}", suffix).replace("${version}", image_tag)) @@ -112,7 +112,7 @@ class DockerImageAvailability(DockerHostMixin, OpenShiftCheck): required.add(image_info["registry_console_image"]) # images for containerized components - if get_var(task_vars, "openshift", "common", "is_containerized"): + if self.get_var("openshift", "common", "is_containerized"): components = set() if 'nodes' in host_groups: components.update(["node", "openvswitch"]) @@ -125,28 +125,27 @@ class DockerImageAvailability(DockerHostMixin, OpenShiftCheck): return required - def local_images(self, images, task_vars): + def local_images(self, images): """Filter a list of images and return those available locally.""" return [ image for image in images - if self.is_image_local(image, task_vars) + if self.is_image_local(image) ] - def is_image_local(self, image, task_vars): + def is_image_local(self, image): """Check if image is already in local docker index.""" - result = self.execute_module("docker_image_facts", {"name": image}, task_vars=task_vars) + result = self.execute_module("docker_image_facts", {"name": image}) if result.get("failed", False): return False return bool(result.get("images", [])) - @staticmethod - def known_docker_registries(task_vars): + def known_docker_registries(self): """Build a list of docker registries available according to inventory vars.""" - docker_facts = get_var(task_vars, "openshift", "docker") + docker_facts = self.get_var("openshift", "docker") regs = set(docker_facts["additional_registries"]) - deployment_type = get_var(task_vars, "openshift_deployment_type") + deployment_type = self.get_var("openshift_deployment_type") if deployment_type == "origin": regs.update(["docker.io"]) elif "enterprise" in deployment_type: @@ -154,24 +153,25 @@ class DockerImageAvailability(DockerHostMixin, OpenShiftCheck): return list(regs) - def available_images(self, images, registries, task_vars): - """Inspect existing images using Skopeo and return all images successfully inspected.""" + def available_images(self, images, default_registries): + """Search remotely for images. Returns: list of images found.""" return [ image for image in images - if self.is_available_skopeo_image(image, registries, task_vars) + if self.is_available_skopeo_image(image, default_registries) ] - def is_available_skopeo_image(self, image, registries, task_vars): + def is_available_skopeo_image(self, image, default_registries): """Use Skopeo to determine if required image exists in known registry(s).""" + registries = default_registries - # if image does already includes a registry, just use that + # if image already includes a registry, only use that if image.count("/") > 1: registry, image = image.split("/", 1) registries = [registry] for registry in registries: args = {"_raw_params": "skopeo inspect --tls-verify=false docker://{}/{}".format(registry, image)} - result = self.execute_module("command", args, task_vars=task_vars) + result = self.execute_module("command", args) if result.get("rc", 0) == 0 and not result.get("failed"): return True diff --git a/roles/openshift_health_checker/openshift_checks/docker_storage.py b/roles/openshift_health_checker/openshift_checks/docker_storage.py index e80691ef3..dea15a56e 100644 --- a/roles/openshift_health_checker/openshift_checks/docker_storage.py +++ b/roles/openshift_health_checker/openshift_checks/docker_storage.py @@ -1,7 +1,8 @@ """Check Docker storage driver and usage.""" import json +import os.path import re -from openshift_checks import OpenShiftCheck, OpenShiftCheckException, get_var +from openshift_checks import OpenShiftCheck, OpenShiftCheckException from openshift_checks.mixins import DockerHostMixin @@ -20,12 +21,29 @@ class DockerStorage(DockerHostMixin, OpenShiftCheck): storage_drivers = ["devicemapper", "overlay", "overlay2"] max_thinpool_data_usage_percent = 90.0 max_thinpool_meta_usage_percent = 90.0 + max_overlay_usage_percent = 90.0 - # pylint: disable=too-many-return-statements - # Reason: permanent stylistic exception; - # it is clearer to return on failures and there are just many ways to fail here. - def run(self, tmp, task_vars): - msg, failed, changed = self.ensure_dependencies(task_vars) + # TODO(lmeyer): mention these in the output when check fails + configuration_variables = [ + ( + "max_thinpool_data_usage_percent", + "For 'devicemapper' storage driver, usage threshold percentage for data. " + "Format: float. Default: {:.1f}".format(max_thinpool_data_usage_percent), + ), + ( + "max_thinpool_meta_usage_percent", + "For 'devicemapper' storage driver, usage threshold percentage for metadata. " + "Format: float. Default: {:.1f}".format(max_thinpool_meta_usage_percent), + ), + ( + "max_overlay_usage_percent", + "For 'overlay' or 'overlay2' storage driver, usage threshold percentage. " + "Format: float. Default: {:.1f}".format(max_overlay_usage_percent), + ), + ] + + def run(self): + msg, failed, changed = self.ensure_dependencies() if failed: return { "failed": True, @@ -34,17 +52,17 @@ class DockerStorage(DockerHostMixin, OpenShiftCheck): } # attempt to get the docker info hash from the API - info = self.execute_module("docker_info", {}, task_vars=task_vars) - if info.get("failed"): + docker_info = self.execute_module("docker_info", {}) + if docker_info.get("failed"): return {"failed": True, "changed": changed, "msg": "Failed to query Docker API. Is docker running on this host?"} - if not info.get("info"): # this would be very strange + if not docker_info.get("info"): # this would be very strange return {"failed": True, "changed": changed, - "msg": "Docker API query missing info:\n{}".format(json.dumps(info))} - info = info["info"] + "msg": "Docker API query missing info:\n{}".format(json.dumps(docker_info))} + docker_info = docker_info["info"] # check if the storage driver we saw is valid - driver = info.get("Driver", "[NONE]") + driver = docker_info.get("Driver", "[NONE]") if driver not in self.storage_drivers: msg = ( "Detected unsupported Docker storage driver '{driver}'.\n" @@ -53,26 +71,34 @@ class DockerStorage(DockerHostMixin, OpenShiftCheck): return {"failed": True, "changed": changed, "msg": msg} # driver status info is a list of tuples; convert to dict and validate based on driver - driver_status = {item[0]: item[1] for item in info.get("DriverStatus", [])} + driver_status = {item[0]: item[1] for item in docker_info.get("DriverStatus", [])} + + result = {} + if driver == "devicemapper": - if driver_status.get("Data loop file"): - msg = ( - "Use of loopback devices with the Docker devicemapper storage driver\n" - "(the default storage configuration) is unsupported in production.\n" - "Please use docker-storage-setup to configure a backing storage volume.\n" - "See http://red.ht/2rNperO for further information." - ) - return {"failed": True, "changed": changed, "msg": msg} - result = self._check_dm_usage(driver_status, task_vars) - result['changed'] = result.get('changed', False) or changed - return result + result = self.check_devicemapper_support(driver_status) - # TODO(lmeyer): determine how to check usage for overlay2 + if driver in ['overlay', 'overlay2']: + result = self.check_overlay_support(docker_info, driver_status) - return {"changed": changed} + result['changed'] = result.get('changed', False) or changed + return result - def _check_dm_usage(self, driver_status, task_vars): - """ + def check_devicemapper_support(self, driver_status): + """Check if dm storage driver is supported as configured. Return: result dict.""" + if driver_status.get("Data loop file"): + msg = ( + "Use of loopback devices with the Docker devicemapper storage driver\n" + "(the default storage configuration) is unsupported in production.\n" + "Please use docker-storage-setup to configure a backing storage volume.\n" + "See http://red.ht/2rNperO for further information." + ) + return {"failed": True, "msg": msg} + result = self.check_dm_usage(driver_status) + return result + + def check_dm_usage(self, driver_status): + """Check usage thresholds for Docker dm storage driver. Return: result dict. Backing assumptions: We expect devicemapper to be backed by an auto-expanding thin pool implemented as an LV in an LVM2 VG. This is how docker-storage-setup currently configures devicemapper storage. The LV is "thin" because it does not use all available storage @@ -83,7 +109,7 @@ class DockerStorage(DockerHostMixin, OpenShiftCheck): could run out of space first; so we check both. """ vals = dict( - vg_free=self._get_vg_free(driver_status.get("Pool Name"), task_vars), + vg_free=self.get_vg_free(driver_status.get("Pool Name")), data_used=driver_status.get("Data Space Used"), data_total=driver_status.get("Data Space Total"), metadata_used=driver_status.get("Metadata Space Used"), @@ -93,7 +119,7 @@ class DockerStorage(DockerHostMixin, OpenShiftCheck): # convert all human-readable strings to bytes for key, value in vals.copy().items(): try: - vals[key + "_bytes"] = self._convert_to_bytes(value) + vals[key + "_bytes"] = self.convert_to_bytes(value) except ValueError as err: # unlikely to hit this from API info, but just to be safe return { "failed": True, @@ -104,7 +130,7 @@ class DockerStorage(DockerHostMixin, OpenShiftCheck): # determine the threshold percentages which usage should not exceed for name, default in [("data", self.max_thinpool_data_usage_percent), ("metadata", self.max_thinpool_meta_usage_percent)]: - percent = get_var(task_vars, "max_thinpool_" + name + "_usage_percent", default=default) + percent = self.get_var("max_thinpool_" + name + "_usage_percent", default=default) try: vals[name + "_threshold"] = float(percent) except ValueError: @@ -131,10 +157,12 @@ class DockerStorage(DockerHostMixin, OpenShiftCheck): vals["msg"] = "\n".join(messages or ["Thinpool usage is within thresholds."]) return vals - def _get_vg_free(self, pool, task_vars): - # Determine which VG to examine according to the pool name, the only indicator currently - # available from the Docker API driver info. We assume a name that looks like - # "vg--name-docker--pool"; vg and lv names with inner hyphens doubled, joined by a hyphen. + def get_vg_free(self, pool): + """Determine which VG to examine according to the pool name. Return: size vgs reports. + Pool name is the only indicator currently available from the Docker API driver info. + We assume a name that looks like "vg--name-docker--pool"; + vg and lv names with inner hyphens doubled, joined by a hyphen. + """ match = re.match(r'((?:[^-]|--)+)-(?!-)', pool) # matches up to the first single hyphen if not match: # unlikely, but... be clear if we assumed wrong raise OpenShiftCheckException( @@ -146,7 +174,7 @@ class DockerStorage(DockerHostMixin, OpenShiftCheck): vgs_cmd = "/sbin/vgs --noheadings -o vg_free --units g --select vg_name=" + vg_name # should return free space like " 12.00g" if the VG exists; empty if it does not - ret = self.execute_module("command", {"_raw_params": vgs_cmd}, task_vars=task_vars) + ret = self.execute_module("command", {"_raw_params": vgs_cmd}) if ret.get("failed") or ret.get("rc", 0) != 0: raise OpenShiftCheckException( "Is LVM installed? Failed to run /sbin/vgs " @@ -163,7 +191,8 @@ class DockerStorage(DockerHostMixin, OpenShiftCheck): return size @staticmethod - def _convert_to_bytes(string): + def convert_to_bytes(string): + """Convert string like "10.3 G" to bytes (binary units assumed). Return: float bytes.""" units = dict( b=1, k=1024, @@ -183,3 +212,87 @@ class DockerStorage(DockerHostMixin, OpenShiftCheck): raise ValueError("Cannot convert to a byte size: " + string) return float(number) * multiplier + + def check_overlay_support(self, docker_info, driver_status): + """Check if overlay storage driver is supported for this host. Return: result dict.""" + # check for xfs as backing store + backing_fs = driver_status.get("Backing Filesystem", "[NONE]") + if backing_fs != "xfs": + msg = ( + "Docker storage drivers 'overlay' and 'overlay2' are only supported with\n" + "'xfs' as the backing storage, but this host's storage is type '{fs}'." + ).format(fs=backing_fs) + return {"failed": True, "msg": msg} + + # check support for OS and kernel version + o_s = docker_info.get("OperatingSystem", "[NONE]") + if "Red Hat Enterprise Linux" in o_s or "CentOS" in o_s: + # keep it simple, only check enterprise kernel versions; assume everyone else is good + kernel = docker_info.get("KernelVersion", "[NONE]") + kernel_arr = [int(num) for num in re.findall(r'\d+', kernel)] + if kernel_arr < [3, 10, 0, 514]: # rhel < 7.3 + msg = ( + "Docker storage drivers 'overlay' and 'overlay2' are only supported beginning with\n" + "kernel version 3.10.0-514; but Docker reports kernel version {version}." + ).format(version=kernel) + return {"failed": True, "msg": msg} + # NOTE: we could check for --selinux-enabled here but docker won't even start with + # that option until it's supported in the kernel so we don't need to. + + return self.check_overlay_usage(docker_info) + + def check_overlay_usage(self, docker_info): + """Check disk usage on OverlayFS backing store volume. Return: result dict.""" + path = docker_info.get("DockerRootDir", "/var/lib/docker") + "/" + docker_info["Driver"] + + threshold = self.get_var("max_overlay_usage_percent", default=self.max_overlay_usage_percent) + try: + threshold = float(threshold) + except ValueError: + return { + "failed": True, + "msg": "Specified 'max_overlay_usage_percent' is not a percentage: {}".format(threshold), + } + + mount = self.find_ansible_mount(path, self.get_var("ansible_mounts")) + try: + free_bytes = mount['size_available'] + total_bytes = mount['size_total'] + usage = 100.0 * (total_bytes - free_bytes) / total_bytes + except (KeyError, ZeroDivisionError): + return { + "failed": True, + "msg": "The ansible_mount found for path {} is invalid.\n" + "This is likely to be an Ansible bug. The record was:\n" + "{}".format(path, json.dumps(mount, indent=2)), + } + + if usage > threshold: + return { + "failed": True, + "msg": ( + "For Docker OverlayFS mount point {path},\n" + "usage percentage {pct:.1f} is higher than threshold {thresh:.1f}." + ).format(path=mount["mount"], pct=usage, thresh=threshold) + } + + return {} + + # TODO(lmeyer): migrate to base class + @staticmethod + def find_ansible_mount(path, ansible_mounts): + """Return the mount point for path from ansible_mounts.""" + + mount_for_path = {mount['mount']: mount for mount in ansible_mounts} + mount_point = path + while mount_point not in mount_for_path: + if mount_point in ["/", ""]: # "/" not in ansible_mounts??? + break + mount_point = os.path.dirname(mount_point) + + try: + return mount_for_path[mount_point] + except KeyError: + known_mounts = ', '.join('"{}"'.format(mount) for mount in sorted(mount_for_path)) or 'none' + msg = 'Unable to determine mount point for path "{}". Known mount points: {}.' + raise OpenShiftCheckException(msg.format(path, known_mounts)) diff --git a/roles/openshift_health_checker/openshift_checks/etcd_imagedata_size.py b/roles/openshift_health_checker/openshift_checks/etcd_imagedata_size.py index c04a69765..28c38504d 100644 --- a/roles/openshift_health_checker/openshift_checks/etcd_imagedata_size.py +++ b/roles/openshift_health_checker/openshift_checks/etcd_imagedata_size.py @@ -2,7 +2,7 @@ Ansible module for determining if the size of OpenShift image data exceeds a specified limit in an etcd cluster. """ -from openshift_checks import OpenShiftCheck, OpenShiftCheckException, get_var +from openshift_checks import OpenShiftCheck, OpenShiftCheckException class EtcdImageDataSize(OpenShiftCheck): @@ -11,24 +11,25 @@ class EtcdImageDataSize(OpenShiftCheck): name = "etcd_imagedata_size" tags = ["etcd"] - def run(self, tmp, task_vars): - etcd_mountpath = self._get_etcd_mountpath(get_var(task_vars, "ansible_mounts")) + def run(self): + etcd_mountpath = self._get_etcd_mountpath(self.get_var("ansible_mounts")) etcd_avail_diskspace = etcd_mountpath["size_available"] etcd_total_diskspace = etcd_mountpath["size_total"] - etcd_imagedata_size_limit = get_var(task_vars, - "etcd_max_image_data_size_bytes", - default=int(0.5 * float(etcd_total_diskspace - etcd_avail_diskspace))) + etcd_imagedata_size_limit = self.get_var( + "etcd_max_image_data_size_bytes", + default=int(0.5 * float(etcd_total_diskspace - etcd_avail_diskspace)) + ) - etcd_is_ssl = get_var(task_vars, "openshift", "master", "etcd_use_ssl", default=False) - etcd_port = get_var(task_vars, "openshift", "master", "etcd_port", default=2379) - etcd_hosts = get_var(task_vars, "openshift", "master", "etcd_hosts") + etcd_is_ssl = self.get_var("openshift", "master", "etcd_use_ssl", default=False) + etcd_port = self.get_var("openshift", "master", "etcd_port", default=2379) + etcd_hosts = self.get_var("openshift", "master", "etcd_hosts") - config_base = get_var(task_vars, "openshift", "common", "config_base") + config_base = self.get_var("openshift", "common", "config_base") - cert = task_vars.get("etcd_client_cert", config_base + "/master/master.etcd-client.crt") - key = task_vars.get("etcd_client_key", config_base + "/master/master.etcd-client.key") - ca_cert = task_vars.get("etcd_client_ca_cert", config_base + "/master/master.etcd-ca.crt") + cert = self.get_var("etcd_client_cert", default=config_base + "/master/master.etcd-client.crt") + key = self.get_var("etcd_client_key", default=config_base + "/master/master.etcd-client.key") + ca_cert = self.get_var("etcd_client_ca_cert", default=config_base + "/master/master.etcd-ca.crt") for etcd_host in list(etcd_hosts): args = { @@ -46,7 +47,7 @@ class EtcdImageDataSize(OpenShiftCheck): }, } - etcdkeysize = self.module_executor("etcdkeysize", args, task_vars) + etcdkeysize = self.execute_module("etcdkeysize", args) if etcdkeysize.get("rc", 0) != 0 or etcdkeysize.get("failed"): msg = 'Failed to retrieve stats for etcd host "{host}": {reason}' diff --git a/roles/openshift_health_checker/openshift_checks/etcd_traffic.py b/roles/openshift_health_checker/openshift_checks/etcd_traffic.py new file mode 100644 index 000000000..cc1b14d8a --- /dev/null +++ b/roles/openshift_health_checker/openshift_checks/etcd_traffic.py @@ -0,0 +1,44 @@ +"""Check that scans journalctl for messages caused as a symptom of increased etcd traffic.""" + +from openshift_checks import OpenShiftCheck + + +class EtcdTraffic(OpenShiftCheck): + """Check if host is being affected by an increase in etcd traffic.""" + + name = "etcd_traffic" + tags = ["health", "etcd"] + + def is_active(self): + """Skip hosts that do not have etcd in their group names.""" + group_names = self.get_var("group_names", default=[]) + valid_group_names = "etcd" in group_names + + version = self.get_var("openshift", "common", "short_version") + valid_version = version in ("3.4", "3.5", "1.4", "1.5") + + return super(EtcdTraffic, self).is_active() and valid_group_names and valid_version + + def run(self): + is_containerized = self.get_var("openshift", "common", "is_containerized") + unit = "etcd_container" if is_containerized else "etcd" + + log_matchers = [{ + "start_regexp": r"Starting Etcd Server", + "regexp": r"etcd: sync duration of [^,]+, expected less than 1s", + "unit": unit + }] + + match = self.execute_module("search_journalctl", {"log_matchers": log_matchers}) + + if match.get("matched"): + msg = ("Higher than normal etcd traffic detected.\n" + "OpenShift 3.4 introduced an increase in etcd traffic.\n" + "Upgrading to OpenShift 3.6 is recommended in order to fix this issue.\n" + "Please refer to https://access.redhat.com/solutions/2916381 for more information.") + return {"failed": True, "msg": msg} + + if match.get("failed"): + return {"failed": True, "msg": "\n".join(match.get("errors"))} + + return {} diff --git a/roles/openshift_health_checker/openshift_checks/etcd_volume.py b/roles/openshift_health_checker/openshift_checks/etcd_volume.py index 7452c9cc1..da7d0364a 100644 --- a/roles/openshift_health_checker/openshift_checks/etcd_volume.py +++ b/roles/openshift_health_checker/openshift_checks/etcd_volume.py @@ -1,6 +1,6 @@ """A health check for OpenShift clusters.""" -from openshift_checks import OpenShiftCheck, OpenShiftCheckException, get_var +from openshift_checks import OpenShiftCheck, OpenShiftCheckException class EtcdVolume(OpenShiftCheck): @@ -14,21 +14,18 @@ class EtcdVolume(OpenShiftCheck): # Where to find ectd data, higher priority first. supported_mount_paths = ["/var/lib/etcd", "/var/lib", "/var", "/"] - @classmethod - def is_active(cls, task_vars): - etcd_hosts = get_var(task_vars, "groups", "etcd", default=[]) or get_var(task_vars, "groups", "masters", - default=[]) or [] - is_etcd_host = get_var(task_vars, "ansible_ssh_host") in etcd_hosts - return super(EtcdVolume, cls).is_active(task_vars) and is_etcd_host + def is_active(self): + etcd_hosts = self.get_var("groups", "etcd", default=[]) or self.get_var("groups", "masters", default=[]) or [] + is_etcd_host = self.get_var("ansible_ssh_host") in etcd_hosts + return super(EtcdVolume, self).is_active() and is_etcd_host - def run(self, tmp, task_vars): - mount_info = self._etcd_mount_info(task_vars) + def run(self): + mount_info = self._etcd_mount_info() available = mount_info["size_available"] total = mount_info["size_total"] used = total - available - threshold = get_var( - task_vars, + threshold = self.get_var( "etcd_device_usage_threshold_percent", default=self.default_threshold_percent ) @@ -45,8 +42,8 @@ class EtcdVolume(OpenShiftCheck): return {"changed": False} - def _etcd_mount_info(self, task_vars): - ansible_mounts = get_var(task_vars, "ansible_mounts") + def _etcd_mount_info(self): + ansible_mounts = self.get_var("ansible_mounts") mounts = {mnt.get("mount"): mnt for mnt in ansible_mounts} for path in self.supported_mount_paths: diff --git a/roles/openshift_health_checker/openshift_checks/logging/curator.py b/roles/openshift_health_checker/openshift_checks/logging/curator.py index c9fc59896..f82ae64d7 100644 --- a/roles/openshift_health_checker/openshift_checks/logging/curator.py +++ b/roles/openshift_health_checker/openshift_checks/logging/curator.py @@ -1,28 +1,21 @@ -""" -Module for performing checks on an Curator logging deployment -""" +"""Check for an aggregated logging Curator deployment""" -from openshift_checks import get_var from openshift_checks.logging.logging import LoggingCheck class Curator(LoggingCheck): - """Module that checks an integrated logging Curator deployment""" + """Check for an aggregated logging Curator deployment""" name = "curator" tags = ["health", "logging"] logging_namespace = None - def run(self, tmp, task_vars): - """Check various things and gather errors. Returns: result as hash""" - - self.logging_namespace = get_var(task_vars, "openshift_logging_namespace", default="logging") + def run(self): + self.logging_namespace = self.get_var("openshift_logging_namespace", default="logging") curator_pods, error = super(Curator, self).get_pods_for_component( - self.module_executor, self.logging_namespace, "curator", - task_vars ) if error: return {"failed": True, "changed": False, "msg": error} diff --git a/roles/openshift_health_checker/openshift_checks/logging/elasticsearch.py b/roles/openshift_health_checker/openshift_checks/logging/elasticsearch.py index 01cb35b81..1e478c04d 100644 --- a/roles/openshift_health_checker/openshift_checks/logging/elasticsearch.py +++ b/roles/openshift_health_checker/openshift_checks/logging/elasticsearch.py @@ -1,35 +1,30 @@ -""" -Module for performing checks on an Elasticsearch logging deployment -""" +"""Check for an aggregated logging Elasticsearch deployment""" import json import re -from openshift_checks import get_var from openshift_checks.logging.logging import LoggingCheck class Elasticsearch(LoggingCheck): - """Module that checks an integrated logging Elasticsearch deployment""" + """Check for an aggregated logging Elasticsearch deployment""" name = "elasticsearch" tags = ["health", "logging"] logging_namespace = None - def run(self, tmp, task_vars): + def run(self): """Check various things and gather errors. Returns: result as hash""" - self.logging_namespace = get_var(task_vars, "openshift_logging_namespace", default="logging") + self.logging_namespace = self.get_var("openshift_logging_namespace", default="logging") es_pods, error = super(Elasticsearch, self).get_pods_for_component( - self.execute_module, self.logging_namespace, "es", - task_vars, ) if error: return {"failed": True, "changed": False, "msg": error} - check_error = self.check_elasticsearch(es_pods, task_vars) + check_error = self.check_elasticsearch(es_pods) if check_error: msg = ("The following Elasticsearch deployment issue was found:" @@ -41,7 +36,7 @@ class Elasticsearch(LoggingCheck): return {"failed": False, "changed": False, "msg": 'No problems found with Elasticsearch deployment.'} def _not_running_elasticsearch_pods(self, es_pods): - """Returns: list of running pods, list of errors about non-running pods""" + """Returns: list of pods that are not running, list of errors about non-running pods""" not_running = super(Elasticsearch, self).not_running_pods(es_pods) if not_running: return not_running, [( @@ -54,7 +49,7 @@ class Elasticsearch(LoggingCheck): ))] return not_running, [] - def check_elasticsearch(self, es_pods, task_vars): + def check_elasticsearch(self, es_pods): """Various checks for elasticsearch. Returns: error string""" not_running_pods, error_msgs = self._not_running_elasticsearch_pods(es_pods) running_pods = [pod for pod in es_pods if pod not in not_running_pods] @@ -65,10 +60,10 @@ class Elasticsearch(LoggingCheck): } if not pods_by_name: return 'No logging Elasticsearch pods were found. Is logging deployed?' - error_msgs += self._check_elasticsearch_masters(pods_by_name, task_vars) - error_msgs += self._check_elasticsearch_node_list(pods_by_name, task_vars) - error_msgs += self._check_es_cluster_health(pods_by_name, task_vars) - error_msgs += self._check_elasticsearch_diskspace(pods_by_name, task_vars) + error_msgs += self._check_elasticsearch_masters(pods_by_name) + error_msgs += self._check_elasticsearch_node_list(pods_by_name) + error_msgs += self._check_es_cluster_health(pods_by_name) + error_msgs += self._check_elasticsearch_diskspace(pods_by_name) return '\n'.join(error_msgs) @staticmethod @@ -76,14 +71,14 @@ class Elasticsearch(LoggingCheck): base = "exec {name} -- curl -s --cert {base}cert --key {base}key --cacert {base}ca -XGET '{url}'" return base.format(base="/etc/elasticsearch/secret/admin-", name=pod_name, url=url) - def _check_elasticsearch_masters(self, pods_by_name, task_vars): + def _check_elasticsearch_masters(self, pods_by_name): """Check that Elasticsearch masters are sane. Returns: list of error strings""" es_master_names = set() error_msgs = [] for pod_name in pods_by_name.keys(): # Compare what each ES node reports as master and compare for split brain get_master_cmd = self._build_es_curl_cmd(pod_name, "https://localhost:9200/_cat/master") - master_name_str = self._exec_oc(get_master_cmd, [], task_vars) + master_name_str = self._exec_oc(get_master_cmd, []) master_names = (master_name_str or '').split(' ') if len(master_names) > 1: es_master_names.add(master_names[1]) @@ -108,7 +103,7 @@ class Elasticsearch(LoggingCheck): return error_msgs - def _check_elasticsearch_node_list(self, pods_by_name, task_vars): + def _check_elasticsearch_node_list(self, pods_by_name): """Check that reported ES masters are accounted for by pods. Returns: list of error strings""" if not pods_by_name: @@ -116,7 +111,7 @@ class Elasticsearch(LoggingCheck): # get ES cluster nodes node_cmd = self._build_es_curl_cmd(list(pods_by_name.keys())[0], 'https://localhost:9200/_nodes') - cluster_node_data = self._exec_oc(node_cmd, [], task_vars) + cluster_node_data = self._exec_oc(node_cmd, []) try: cluster_nodes = json.loads(cluster_node_data)['nodes'] except (ValueError, KeyError): @@ -138,12 +133,12 @@ class Elasticsearch(LoggingCheck): return error_msgs - def _check_es_cluster_health(self, pods_by_name, task_vars): + def _check_es_cluster_health(self, pods_by_name): """Exec into the elasticsearch pods and check the cluster health. Returns: list of errors""" error_msgs = [] for pod_name in pods_by_name.keys(): cluster_health_cmd = self._build_es_curl_cmd(pod_name, 'https://localhost:9200/_cluster/health?pretty=true') - cluster_health_data = self._exec_oc(cluster_health_cmd, [], task_vars) + cluster_health_data = self._exec_oc(cluster_health_cmd, []) try: health_res = json.loads(cluster_health_data) if not health_res or not health_res.get('status'): @@ -162,7 +157,7 @@ class Elasticsearch(LoggingCheck): return error_msgs - def _check_elasticsearch_diskspace(self, pods_by_name, task_vars): + def _check_elasticsearch_diskspace(self, pods_by_name): """ Exec into an ES pod and query the diskspace on the persistent volume. Returns: list of errors @@ -170,7 +165,7 @@ class Elasticsearch(LoggingCheck): error_msgs = [] for pod_name in pods_by_name.keys(): df_cmd = 'exec {} -- df --output=ipcent,pcent /elasticsearch/persistent'.format(pod_name) - disk_output = self._exec_oc(df_cmd, [], task_vars) + disk_output = self._exec_oc(df_cmd, []) lines = disk_output.splitlines() # expecting one header looking like 'IUse% Use%' and one body line body_re = r'\s*(\d+)%?\s+(\d+)%?\s*$' @@ -182,7 +177,7 @@ class Elasticsearch(LoggingCheck): continue inode_pct, disk_pct = re.match(body_re, lines[1]).groups() - inode_pct_thresh = get_var(task_vars, 'openshift_check_efk_es_inode_pct', default='90') + inode_pct_thresh = self.get_var('openshift_check_efk_es_inode_pct', default='90') if int(inode_pct) >= int(inode_pct_thresh): error_msgs.append( 'Inode percent usage on the storage volume for logging ES pod "{pod}"\n' @@ -193,7 +188,7 @@ class Elasticsearch(LoggingCheck): limit=str(inode_pct_thresh), param='openshift_check_efk_es_inode_pct', )) - disk_pct_thresh = get_var(task_vars, 'openshift_check_efk_es_storage_pct', default='80') + disk_pct_thresh = self.get_var('openshift_check_efk_es_storage_pct', default='80') if int(disk_pct) >= int(disk_pct_thresh): error_msgs.append( 'Disk percent usage on the storage volume for logging ES pod "{pod}"\n' @@ -207,11 +202,9 @@ class Elasticsearch(LoggingCheck): return error_msgs - def _exec_oc(self, cmd_str, extra_args, task_vars): + def _exec_oc(self, cmd_str, extra_args): return super(Elasticsearch, self).exec_oc( - self.execute_module, self.logging_namespace, cmd_str, extra_args, - task_vars, ) diff --git a/roles/openshift_health_checker/openshift_checks/logging/fluentd.py b/roles/openshift_health_checker/openshift_checks/logging/fluentd.py index 627567293..063e707a9 100644 --- a/roles/openshift_health_checker/openshift_checks/logging/fluentd.py +++ b/roles/openshift_health_checker/openshift_checks/logging/fluentd.py @@ -1,33 +1,29 @@ -""" -Module for performing checks on an Fluentd logging deployment -""" +"""Check for an aggregated logging Fluentd deployment""" import json -from openshift_checks import get_var from openshift_checks.logging.logging import LoggingCheck class Fluentd(LoggingCheck): - """Module that checks an integrated logging Fluentd deployment""" + """Check for an aggregated logging Fluentd deployment""" + name = "fluentd" tags = ["health", "logging"] logging_namespace = None - def run(self, tmp, task_vars): + def run(self): """Check various things and gather errors. Returns: result as hash""" - self.logging_namespace = get_var(task_vars, "openshift_logging_namespace", default="logging") + self.logging_namespace = self.get_var("openshift_logging_namespace", default="logging") fluentd_pods, error = super(Fluentd, self).get_pods_for_component( - self.execute_module, self.logging_namespace, "fluentd", - task_vars, ) if error: return {"failed": True, "changed": False, "msg": error} - check_error = self.check_fluentd(fluentd_pods, task_vars) + check_error = self.check_fluentd(fluentd_pods) if check_error: msg = ("The following Fluentd deployment issue was found:" @@ -53,10 +49,9 @@ class Fluentd(LoggingCheck): ).format(label=node_selector) return fluentd_nodes, None - @staticmethod - def _check_node_labeling(nodes_by_name, fluentd_nodes, node_selector, task_vars): + def _check_node_labeling(self, nodes_by_name, fluentd_nodes, node_selector): """Note if nodes are not labeled as expected. Returns: error string""" - intended_nodes = get_var(task_vars, 'openshift_logging_fluentd_hosts', default=['--all']) + intended_nodes = self.get_var('openshift_logging_fluentd_hosts', default=['--all']) if not intended_nodes or '--all' in intended_nodes: intended_nodes = nodes_by_name.keys() nodes_missing_labels = set(intended_nodes) - set(fluentd_nodes.keys()) @@ -114,13 +109,15 @@ class Fluentd(LoggingCheck): )) return None - def check_fluentd(self, pods, task_vars): + def check_fluentd(self, pods): """Verify fluentd is running everywhere. Returns: error string""" - node_selector = get_var(task_vars, 'openshift_logging_fluentd_nodeselector', - default='logging-infra-fluentd=true') + node_selector = self.get_var( + 'openshift_logging_fluentd_nodeselector', + default='logging-infra-fluentd=true' + ) - nodes_by_name, error = self.get_nodes_by_name(task_vars) + nodes_by_name, error = self.get_nodes_by_name() if error: return error @@ -129,7 +126,7 @@ class Fluentd(LoggingCheck): return error error_msgs = [] - error = self._check_node_labeling(nodes_by_name, fluentd_nodes, node_selector, task_vars) + error = self._check_node_labeling(nodes_by_name, fluentd_nodes, node_selector) if error: error_msgs.append(error) error = self._check_nodes_have_fluentd(pods, fluentd_nodes) @@ -148,9 +145,9 @@ class Fluentd(LoggingCheck): return '\n'.join(error_msgs) - def get_nodes_by_name(self, task_vars): + def get_nodes_by_name(self): """Retrieve all the node definitions. Returns: dict(name: node), error string""" - nodes_json = self._exec_oc("get nodes -o json", [], task_vars) + nodes_json = self._exec_oc("get nodes -o json", []) try: nodes = json.loads(nodes_json) except ValueError: # no valid json - should not happen @@ -162,9 +159,9 @@ class Fluentd(LoggingCheck): for node in nodes['items'] }, None - def _exec_oc(self, cmd_str, extra_args, task_vars): - return super(Fluentd, self).exec_oc(self.execute_module, - self.logging_namespace, - cmd_str, - extra_args, - task_vars) + def _exec_oc(self, cmd_str, extra_args): + return super(Fluentd, self).exec_oc( + self.logging_namespace, + cmd_str, + extra_args, + ) diff --git a/roles/openshift_health_checker/openshift_checks/logging/kibana.py b/roles/openshift_health_checker/openshift_checks/logging/kibana.py index 551e8dfa0..60f94e106 100644 --- a/roles/openshift_health_checker/openshift_checks/logging/kibana.py +++ b/roles/openshift_health_checker/openshift_checks/logging/kibana.py @@ -12,7 +12,6 @@ except ImportError: from urllib.error import HTTPError, URLError import urllib.request as urllib2 -from openshift_checks import get_var from openshift_checks.logging.logging import LoggingCheck @@ -24,22 +23,20 @@ class Kibana(LoggingCheck): logging_namespace = None - def run(self, tmp, task_vars): + def run(self): """Check various things and gather errors. Returns: result as hash""" - self.logging_namespace = get_var(task_vars, "openshift_logging_namespace", default="logging") + self.logging_namespace = self.get_var("openshift_logging_namespace", default="logging") kibana_pods, error = super(Kibana, self).get_pods_for_component( - self.execute_module, self.logging_namespace, "kibana", - task_vars, ) if error: return {"failed": True, "changed": False, "msg": error} check_error = self.check_kibana(kibana_pods) if not check_error: - check_error = self._check_kibana_route(task_vars) + check_error = self._check_kibana_route() if check_error: msg = ("The following Kibana deployment issue was found:" @@ -50,7 +47,7 @@ class Kibana(LoggingCheck): # TODO(lmeyer): run it all again for the ops cluster return {"failed": False, "changed": False, "msg": 'No problems found with Kibana deployment.'} - def _verify_url_internal(self, url, task_vars): + def _verify_url_internal(self, url): """ Try to reach a URL from the host. Returns: success (bool), reason (for failure) @@ -62,7 +59,7 @@ class Kibana(LoggingCheck): # TODO(lmeyer): give users option to validate certs status_code=302, ) - result = self.execute_module('uri', args, None, task_vars) + result = self.execute_module('uri', args) if result.get('failed'): return result['msg'] return None @@ -114,14 +111,14 @@ class Kibana(LoggingCheck): return None - def _get_kibana_url(self, task_vars): + def _get_kibana_url(self): """ Get kibana route or report error. Returns: url (or empty), reason for failure """ # Get logging url - get_route = self._exec_oc("get route logging-kibana -o json", [], task_vars) + get_route = self._exec_oc("get route logging-kibana -o json", []) if not get_route: return None, 'no_route_exists' @@ -139,7 +136,7 @@ class Kibana(LoggingCheck): return 'https://{}/'.format(host), None - def _check_kibana_route(self, task_vars): + def _check_kibana_route(self): """ Check to see if kibana route is up and working. Returns: error string @@ -160,12 +157,12 @@ class Kibana(LoggingCheck): ), ) - kibana_url, error = self._get_kibana_url(task_vars) + kibana_url, error = self._get_kibana_url() if not kibana_url: return known_errors.get(error, error) # first, check that kibana is reachable from the master. - error = self._verify_url_internal(kibana_url, task_vars) + error = self._verify_url_internal(kibana_url) if error: if 'urlopen error [Errno 111] Connection refused' in error: error = ( @@ -190,7 +187,7 @@ class Kibana(LoggingCheck): # in production we would like the kibana route to work from outside the # cluster too; but that may not be the case, so allow disabling just this part. - if not get_var(task_vars, "openshift_check_efk_kibana_external", default=True): + if not self.get_var("openshift_check_efk_kibana_external", default=True): return None error = self._verify_url_external(kibana_url) if error: @@ -221,9 +218,9 @@ class Kibana(LoggingCheck): return error return None - def _exec_oc(self, cmd_str, extra_args, task_vars): - return super(Kibana, self).exec_oc(self.execute_module, - self.logging_namespace, - cmd_str, - extra_args, - task_vars) + def _exec_oc(self, cmd_str, extra_args): + return super(Kibana, self).exec_oc( + self.logging_namespace, + cmd_str, + extra_args, + ) diff --git a/roles/openshift_health_checker/openshift_checks/logging/logging.py b/roles/openshift_health_checker/openshift_checks/logging/logging.py index 6e951e82c..a48e1c728 100644 --- a/roles/openshift_health_checker/openshift_checks/logging/logging.py +++ b/roles/openshift_health_checker/openshift_checks/logging/logging.py @@ -5,39 +5,36 @@ Util functions for performing checks on an Elasticsearch, Fluentd, and Kibana st import json import os -from openshift_checks import OpenShiftCheck, OpenShiftCheckException, get_var +from openshift_checks import OpenShiftCheck, OpenShiftCheckException class LoggingCheck(OpenShiftCheck): - """Base class for logging component checks""" + """Base class for OpenShift aggregated logging component checks""" name = "logging" + logging_namespace = "logging" - @classmethod - def is_active(cls, task_vars): - return super(LoggingCheck, cls).is_active(task_vars) and cls.is_first_master(task_vars) + def is_active(self): + logging_deployed = self.get_var("openshift_hosted_logging_deploy", default=False) + return logging_deployed and super(LoggingCheck, self).is_active() and self.is_first_master() - @staticmethod - def is_first_master(task_vars): - """Run only on first master and only when logging is configured. Returns: bool""" - logging_deployed = get_var(task_vars, "openshift_hosted_logging_deploy", default=True) + def is_first_master(self): + """Determine if running on first master. Returns: bool""" # Note: It would be nice to use membership in oo_first_master group, however for now it # seems best to avoid requiring that setup and just check this is the first master. - hostname = get_var(task_vars, "ansible_ssh_host") or [None] - masters = get_var(task_vars, "groups", "masters", default=None) or [None] - return logging_deployed and masters[0] == hostname + hostname = self.get_var("ansible_ssh_host") or [None] + masters = self.get_var("groups", "masters", default=None) or [None] + return masters[0] == hostname - def run(self, tmp, task_vars): + def run(self): pass - def get_pods_for_component(self, execute_module, namespace, logging_component, task_vars): + def get_pods_for_component(self, namespace, logging_component): """Get all pods for a given component. Returns: list of pods for component, error string""" pod_output = self.exec_oc( - execute_module, namespace, "get pods -l component={} -o json".format(logging_component), [], - task_vars ) try: pods = json.loads(pod_output) @@ -45,7 +42,7 @@ class LoggingCheck(OpenShiftCheck): raise ValueError() except ValueError: # successful run but non-parsing data generally means there were no pods in the namespace - return None, 'There are no pods in the {} namespace. Is logging deployed?'.format(namespace) + return None, 'No pods were found for the "{}" logging component.'.format(logging_component) return pods['items'], None @@ -63,14 +60,13 @@ class LoggingCheck(OpenShiftCheck): ) ] - @staticmethod - def exec_oc(execute_module=None, namespace="logging", cmd_str="", extra_args=None, task_vars=None): + def exec_oc(self, namespace="logging", cmd_str="", extra_args=None): """ Execute an 'oc' command in the remote host. Returns: output of command and namespace, or raises OpenShiftCheckException on error """ - config_base = get_var(task_vars, "openshift", "common", "config_base") + config_base = self.get_var("openshift", "common", "config_base") args = { "namespace": namespace, "config_file": os.path.join(config_base, "master", "admin.kubeconfig"), @@ -78,7 +74,7 @@ class LoggingCheck(OpenShiftCheck): "extra_args": list(extra_args) if extra_args else [], } - result = execute_module("ocutil", args, None, task_vars) + result = self.execute_module("ocutil", args) if result.get("failed"): msg = ( 'Unexpected error using `oc` to validate the logging stack components.\n' diff --git a/roles/openshift_health_checker/openshift_checks/logging/logging_index_time.py b/roles/openshift_health_checker/openshift_checks/logging/logging_index_time.py new file mode 100644 index 000000000..b24e88e05 --- /dev/null +++ b/roles/openshift_health_checker/openshift_checks/logging/logging_index_time.py @@ -0,0 +1,130 @@ +""" +Check for ensuring logs from pods can be queried in a reasonable amount of time. +""" + +import json +import time + +from uuid import uuid4 + +from openshift_checks import OpenShiftCheckException +from openshift_checks.logging.logging import LoggingCheck + + +ES_CMD_TIMEOUT_SECONDS = 30 + + +class LoggingIndexTime(LoggingCheck): + """Check that pod logs are aggregated and indexed in ElasticSearch within a reasonable amount of time.""" + name = "logging_index_time" + tags = ["health", "logging"] + + logging_namespace = "logging" + + def run(self): + """Add log entry by making unique request to Kibana. Check for unique entry in the ElasticSearch pod logs.""" + try: + log_index_timeout = int( + self.get_var("openshift_check_logging_index_timeout_seconds", default=ES_CMD_TIMEOUT_SECONDS) + ) + except ValueError: + return { + "failed": True, + "msg": ('Invalid value provided for "openshift_check_logging_index_timeout_seconds". ' + 'Value must be an integer representing an amount in seconds.'), + } + + running_component_pods = dict() + + # get all component pods + self.logging_namespace = self.get_var("openshift_logging_namespace", default=self.logging_namespace) + for component, name in (['kibana', 'Kibana'], ['es', 'Elasticsearch']): + pods, error = self.get_pods_for_component(self.logging_namespace, component) + + if error: + msg = 'Unable to retrieve pods for the {} logging component: {}' + return {"failed": True, "changed": False, "msg": msg.format(name, error)} + + running_pods = self.running_pods(pods) + + if not running_pods: + msg = ('No {} pods in the "Running" state were found.' + 'At least one pod is required in order to perform this check.') + return {"failed": True, "changed": False, "msg": msg.format(name)} + + running_component_pods[component] = running_pods + + uuid = self.curl_kibana_with_uuid(running_component_pods["kibana"][0]) + self.wait_until_cmd_or_err(running_component_pods["es"][0], uuid, log_index_timeout) + return {} + + def wait_until_cmd_or_err(self, es_pod, uuid, timeout_secs): + """Retry an Elasticsearch query every second until query success, or a defined + length of time has passed.""" + deadline = time.time() + timeout_secs + interval = 1 + while not self.query_es_from_es(es_pod, uuid): + if time.time() + interval > deadline: + msg = "expecting match in Elasticsearch for message with uuid {}, but no matches were found after {}s." + raise OpenShiftCheckException(msg.format(uuid, timeout_secs)) + time.sleep(interval) + + def curl_kibana_with_uuid(self, kibana_pod): + """curl Kibana with a unique uuid.""" + uuid = self.generate_uuid() + pod_name = kibana_pod["metadata"]["name"] + exec_cmd = "exec {pod_name} -c kibana -- curl --max-time 30 -s http://localhost:5601/{uuid}" + exec_cmd = exec_cmd.format(pod_name=pod_name, uuid=uuid) + + error_str = self.exec_oc(self.logging_namespace, exec_cmd, []) + + try: + error_code = json.loads(error_str)["statusCode"] + except KeyError: + msg = ('invalid response returned from Kibana request (Missing "statusCode" key):\n' + 'Command: {}\nResponse: {}').format(exec_cmd, error_str) + raise OpenShiftCheckException(msg) + except ValueError: + msg = ('invalid response returned from Kibana request (Non-JSON output):\n' + 'Command: {}\nResponse: {}').format(exec_cmd, error_str) + raise OpenShiftCheckException(msg) + + if error_code != 404: + msg = 'invalid error code returned from Kibana request. Expecting error code "404", but got "{}" instead.' + raise OpenShiftCheckException(msg.format(error_code)) + + return uuid + + def query_es_from_es(self, es_pod, uuid): + """curl the Elasticsearch pod and look for a unique uuid in its logs.""" + pod_name = es_pod["metadata"]["name"] + exec_cmd = ( + "exec {pod_name} -- curl --max-time 30 -s -f " + "--cacert /etc/elasticsearch/secret/admin-ca " + "--cert /etc/elasticsearch/secret/admin-cert " + "--key /etc/elasticsearch/secret/admin-key " + "https://logging-es:9200/project.{namespace}*/_count?q=message:{uuid}" + ) + exec_cmd = exec_cmd.format(pod_name=pod_name, namespace=self.logging_namespace, uuid=uuid) + result = self.exec_oc(self.logging_namespace, exec_cmd, []) + + try: + count = json.loads(result)["count"] + except KeyError: + msg = 'invalid response from Elasticsearch query:\n"{}"\nMissing "count" key:\n{}' + raise OpenShiftCheckException(msg.format(exec_cmd, result)) + except ValueError: + msg = 'invalid response from Elasticsearch query:\n"{}"\nNon-JSON output:\n{}' + raise OpenShiftCheckException(msg.format(exec_cmd, result)) + + return count + + @staticmethod + def running_pods(pods): + """Filter pods that are running.""" + return [pod for pod in pods if pod['status']['phase'] == 'Running'] + + @staticmethod + def generate_uuid(): + """Wrap uuid generator. Allows for testing with expected values.""" + return str(uuid4()) diff --git a/roles/openshift_health_checker/openshift_checks/memory_availability.py b/roles/openshift_health_checker/openshift_checks/memory_availability.py index f4e31065f..765ba072d 100644 --- a/roles/openshift_health_checker/openshift_checks/memory_availability.py +++ b/roles/openshift_health_checker/openshift_checks/memory_availability.py @@ -1,5 +1,5 @@ -# pylint: disable=missing-docstring -from openshift_checks import OpenShiftCheck, get_var +"""Check that recommended memory is available.""" +from openshift_checks import OpenShiftCheck MIB = 2**20 GIB = 2**30 @@ -21,19 +21,18 @@ class MemoryAvailability(OpenShiftCheck): # https://access.redhat.com/solutions/3006511 physical RAM is partly reserved from memtotal memtotal_adjustment = 1 * GIB - @classmethod - def is_active(cls, task_vars): + def is_active(self): """Skip hosts that do not have recommended memory requirements.""" - group_names = get_var(task_vars, "group_names", default=[]) - has_memory_recommendation = bool(set(group_names).intersection(cls.recommended_memory_bytes)) - return super(MemoryAvailability, cls).is_active(task_vars) and has_memory_recommendation + group_names = self.get_var("group_names", default=[]) + has_memory_recommendation = bool(set(group_names).intersection(self.recommended_memory_bytes)) + return super(MemoryAvailability, self).is_active() and has_memory_recommendation - def run(self, tmp, task_vars): - group_names = get_var(task_vars, "group_names") - total_memory_bytes = get_var(task_vars, "ansible_memtotal_mb") * MIB + def run(self): + group_names = self.get_var("group_names") + total_memory_bytes = self.get_var("ansible_memtotal_mb") * MIB recommended_min = max(self.recommended_memory_bytes.get(name, 0) for name in group_names) - configured_min = float(get_var(task_vars, "openshift_check_min_host_memory_gb", default=0)) * GIB + configured_min = float(self.get_var("openshift_check_min_host_memory_gb", default=0)) * GIB min_memory_bytes = configured_min or recommended_min if total_memory_bytes + self.memtotal_adjustment < min_memory_bytes: diff --git a/roles/openshift_health_checker/openshift_checks/mixins.py b/roles/openshift_health_checker/openshift_checks/mixins.py index 2cb2e21aa..3b2c64e6a 100644 --- a/roles/openshift_health_checker/openshift_checks/mixins.py +++ b/roles/openshift_health_checker/openshift_checks/mixins.py @@ -2,19 +2,16 @@ Mixin classes meant to be used with subclasses of OpenShiftCheck. """ -from openshift_checks import get_var - class NotContainerizedMixin(object): """Mixin for checks that are only active when not in containerized mode.""" # permanent # pylint: disable=too-few-public-methods # Reason: The mixin is not intended to stand on its own as a class. - @classmethod - def is_active(cls, task_vars): + def is_active(self): """Only run on non-containerized hosts.""" - is_containerized = get_var(task_vars, "openshift", "common", "is_containerized") - return super(NotContainerizedMixin, cls).is_active(task_vars) and not is_containerized + is_containerized = self.get_var("openshift", "common", "is_containerized") + return super(NotContainerizedMixin, self).is_active() and not is_containerized class DockerHostMixin(object): @@ -22,28 +19,26 @@ class DockerHostMixin(object): dependencies = [] - @classmethod - def is_active(cls, task_vars): + def is_active(self): """Only run on hosts that depend on Docker.""" - is_containerized = get_var(task_vars, "openshift", "common", "is_containerized") - is_node = "nodes" in get_var(task_vars, "group_names", default=[]) - return super(DockerHostMixin, cls).is_active(task_vars) and (is_containerized or is_node) + is_containerized = self.get_var("openshift", "common", "is_containerized") + is_node = "nodes" in self.get_var("group_names", default=[]) + return super(DockerHostMixin, self).is_active() and (is_containerized or is_node) - def ensure_dependencies(self, task_vars): + def ensure_dependencies(self): """ Ensure that docker-related packages exist, but not on atomic hosts (which would not be able to install but should already have them). Returns: msg, failed, changed """ - if get_var(task_vars, "openshift", "common", "is_atomic"): + if self.get_var("openshift", "common", "is_atomic"): return "", False, False # NOTE: we would use the "package" module but it's actually an action plugin # and it's not clear how to invoke one of those. This is about the same anyway: result = self.execute_module( - get_var(task_vars, "ansible_pkg_mgr", default="yum"), + self.get_var("ansible_pkg_mgr", default="yum"), {"name": self.dependencies, "state": "present"}, - task_vars=task_vars, ) msg = result.get("msg", "") if result.get("failed"): diff --git a/roles/openshift_health_checker/openshift_checks/ovs_version.py b/roles/openshift_health_checker/openshift_checks/ovs_version.py index 2dd045f1f..cd6ebd493 100644 --- a/roles/openshift_health_checker/openshift_checks/ovs_version.py +++ b/roles/openshift_health_checker/openshift_checks/ovs_version.py @@ -3,7 +3,7 @@ Ansible module for determining if an installed version of Open vSwitch is incomp currently installed version of OpenShift. """ -from openshift_checks import OpenShiftCheck, OpenShiftCheckException, get_var +from openshift_checks import OpenShiftCheck, OpenShiftCheckException from openshift_checks.mixins import NotContainerizedMixin @@ -27,27 +27,26 @@ class OvsVersion(NotContainerizedMixin, OpenShiftCheck): "1": "3", } - @classmethod - def is_active(cls, task_vars): + def is_active(self): """Skip hosts that do not have package requirements.""" - group_names = get_var(task_vars, "group_names", default=[]) + group_names = self.get_var("group_names", default=[]) master_or_node = 'masters' in group_names or 'nodes' in group_names - return super(OvsVersion, cls).is_active(task_vars) and master_or_node + return super(OvsVersion, self).is_active() and master_or_node - def run(self, tmp, task_vars): + def run(self): args = { "package_list": [ { "name": "openvswitch", - "version": self.get_required_ovs_version(task_vars), + "version": self.get_required_ovs_version(), }, ], } - return self.execute_module("rpm_version", args, task_vars=task_vars) + return self.execute_module("rpm_version", args) - def get_required_ovs_version(self, task_vars): + def get_required_ovs_version(self): """Return the correct Open vSwitch version for the current OpenShift version""" - openshift_version = self._get_openshift_version(task_vars) + openshift_version = self._get_openshift_version() if float(openshift_version) < 3.5: return self.openshift_to_ovs_version["3.4"] @@ -59,8 +58,8 @@ class OvsVersion(NotContainerizedMixin, OpenShiftCheck): msg = "There is no recommended version of Open vSwitch for the current version of OpenShift: {}" raise OpenShiftCheckException(msg.format(openshift_version)) - def _get_openshift_version(self, task_vars): - openshift_version = get_var(task_vars, "openshift_image_tag") + def _get_openshift_version(self): + openshift_version = self.get_var("openshift_image_tag") if openshift_version and openshift_version[0] == 'v': openshift_version = openshift_version[1:] diff --git a/roles/openshift_health_checker/openshift_checks/package_availability.py b/roles/openshift_health_checker/openshift_checks/package_availability.py index 0dd2b1286..a86180b00 100644 --- a/roles/openshift_health_checker/openshift_checks/package_availability.py +++ b/roles/openshift_health_checker/openshift_checks/package_availability.py @@ -1,5 +1,6 @@ -# pylint: disable=missing-docstring -from openshift_checks import OpenShiftCheck, get_var +"""Check that required RPM packages are available.""" + +from openshift_checks import OpenShiftCheck from openshift_checks.mixins import NotContainerizedMixin @@ -9,13 +10,13 @@ class PackageAvailability(NotContainerizedMixin, OpenShiftCheck): name = "package_availability" tags = ["preflight"] - @classmethod - def is_active(cls, task_vars): - return super(PackageAvailability, cls).is_active(task_vars) and task_vars["ansible_pkg_mgr"] == "yum" + def is_active(self): + """Run only when yum is the package manager as the code is specific to it.""" + return super(PackageAvailability, self).is_active() and self.get_var("ansible_pkg_mgr") == "yum" - def run(self, tmp, task_vars): - rpm_prefix = get_var(task_vars, "openshift", "common", "service_type") - group_names = get_var(task_vars, "group_names", default=[]) + def run(self): + rpm_prefix = self.get_var("openshift", "common", "service_type") + group_names = self.get_var("group_names", default=[]) packages = set() @@ -25,10 +26,11 @@ class PackageAvailability(NotContainerizedMixin, OpenShiftCheck): packages.update(self.node_packages(rpm_prefix)) args = {"packages": sorted(set(packages))} - return self.execute_module("check_yum_update", args, tmp=tmp, task_vars=task_vars) + return self.execute_module("check_yum_update", args) @staticmethod def master_packages(rpm_prefix): + """Return a list of RPMs that we expect a master install to have available.""" return [ "{rpm_prefix}".format(rpm_prefix=rpm_prefix), "{rpm_prefix}-clients".format(rpm_prefix=rpm_prefix), @@ -44,6 +46,7 @@ class PackageAvailability(NotContainerizedMixin, OpenShiftCheck): @staticmethod def node_packages(rpm_prefix): + """Return a list of RPMs that we expect a node install to have available.""" return [ "{rpm_prefix}".format(rpm_prefix=rpm_prefix), "{rpm_prefix}-node".format(rpm_prefix=rpm_prefix), diff --git a/roles/openshift_health_checker/openshift_checks/package_update.py b/roles/openshift_health_checker/openshift_checks/package_update.py index f432380c6..1e9aecbe0 100644 --- a/roles/openshift_health_checker/openshift_checks/package_update.py +++ b/roles/openshift_health_checker/openshift_checks/package_update.py @@ -1,14 +1,14 @@ -# pylint: disable=missing-docstring +"""Check that a yum update would not run into conflicts with available packages.""" from openshift_checks import OpenShiftCheck from openshift_checks.mixins import NotContainerizedMixin class PackageUpdate(NotContainerizedMixin, OpenShiftCheck): - """Check that there are no conflicts in RPM packages.""" + """Check that a yum update would not run into conflicts with available packages.""" name = "package_update" tags = ["preflight"] - def run(self, tmp, task_vars): + def run(self): args = {"packages": []} - return self.execute_module("check_yum_update", args, tmp=tmp, task_vars=task_vars) + return self.execute_module("check_yum_update", args) diff --git a/roles/openshift_health_checker/openshift_checks/package_version.py b/roles/openshift_health_checker/openshift_checks/package_version.py index 6a76bb93d..020786804 100644 --- a/roles/openshift_health_checker/openshift_checks/package_version.py +++ b/roles/openshift_health_checker/openshift_checks/package_version.py @@ -1,5 +1,5 @@ -# pylint: disable=missing-docstring -from openshift_checks import OpenShiftCheck, OpenShiftCheckException, get_var +"""Check that available RPM packages match the required versions.""" +from openshift_checks import OpenShiftCheck, OpenShiftCheckException from openshift_checks.mixins import NotContainerizedMixin @@ -10,8 +10,8 @@ class PackageVersion(NotContainerizedMixin, OpenShiftCheck): tags = ["preflight"] openshift_to_ovs_version = { - "3.6": "2.6", - "3.5": "2.6", + "3.6": ["2.6", "2.7"], + "3.5": ["2.6", "2.7"], "3.4": "2.4", } @@ -28,29 +28,28 @@ class PackageVersion(NotContainerizedMixin, OpenShiftCheck): "1": "3", } - @classmethod - def is_active(cls, task_vars): + def is_active(self): """Skip hosts that do not have package requirements.""" - group_names = get_var(task_vars, "group_names", default=[]) + group_names = self.get_var("group_names", default=[]) master_or_node = 'masters' in group_names or 'nodes' in group_names - return super(PackageVersion, cls).is_active(task_vars) and master_or_node + return super(PackageVersion, self).is_active() and master_or_node - def run(self, tmp, task_vars): - rpm_prefix = get_var(task_vars, "openshift", "common", "service_type") - openshift_release = get_var(task_vars, "openshift_release", default='') - deployment_type = get_var(task_vars, "openshift_deployment_type") + def run(self): + rpm_prefix = self.get_var("openshift", "common", "service_type") + openshift_release = self.get_var("openshift_release", default='') + deployment_type = self.get_var("openshift_deployment_type") check_multi_minor_release = deployment_type in ['openshift-enterprise'] args = { "package_list": [ { "name": "openvswitch", - "version": self.get_required_ovs_version(task_vars), + "version": self.get_required_ovs_version(), "check_multi": False, }, { "name": "docker", - "version": self.get_required_docker_version(task_vars), + "version": self.get_required_docker_version(), "check_multi": False, }, { @@ -71,13 +70,13 @@ class PackageVersion(NotContainerizedMixin, OpenShiftCheck): ], } - return self.execute_module("aos_version", args, tmp=tmp, task_vars=task_vars) + return self.execute_module("aos_version", args) - def get_required_ovs_version(self, task_vars): + def get_required_ovs_version(self): """Return the correct Open vSwitch version for the current OpenShift version. If the current OpenShift version is >= 3.5, ensure Open vSwitch version 2.6, Else ensure Open vSwitch version 2.4""" - openshift_version = self.get_openshift_version(task_vars) + openshift_version = self.get_openshift_version() if float(openshift_version) < 3.5: return self.openshift_to_ovs_version["3.4"] @@ -89,12 +88,12 @@ class PackageVersion(NotContainerizedMixin, OpenShiftCheck): msg = "There is no recommended version of Open vSwitch for the current version of OpenShift: {}" raise OpenShiftCheckException(msg.format(openshift_version)) - def get_required_docker_version(self, task_vars): + def get_required_docker_version(self): """Return the correct Docker version for the current OpenShift version. If the OpenShift version is 3.1, ensure Docker version 1.8. If the OpenShift version is 3.2 or 3.3, ensure Docker version 1.10. If the current OpenShift version is >= 3.4, ensure Docker version 1.12.""" - openshift_version = self.get_openshift_version(task_vars) + openshift_version = self.get_openshift_version() if float(openshift_version) >= 3.4: return self.openshift_to_docker_version["3.4"] @@ -106,14 +105,16 @@ class PackageVersion(NotContainerizedMixin, OpenShiftCheck): msg = "There is no recommended version of Docker for the current version of OpenShift: {}" raise OpenShiftCheckException(msg.format(openshift_version)) - def get_openshift_version(self, task_vars): - openshift_version = get_var(task_vars, "openshift_image_tag") + def get_openshift_version(self): + """Return received image tag as a normalized X.Y minor version string.""" + openshift_version = self.get_var("openshift_image_tag") if openshift_version and openshift_version[0] == 'v': openshift_version = openshift_version[1:] return self.parse_version(openshift_version) def parse_version(self, version): + """Return a normalized X.Y minor version string.""" components = version.split(".") if not components or len(components) < 2: msg = "An invalid version of OpenShift was found for this host: {}" diff --git a/roles/openshift_health_checker/test/action_plugin_test.py b/roles/openshift_health_checker/test/action_plugin_test.py index 9383b233c..2d068be3d 100644 --- a/roles/openshift_health_checker/test/action_plugin_test.py +++ b/roles/openshift_health_checker/test/action_plugin_test.py @@ -15,14 +15,13 @@ def fake_check(name='fake_check', tags=None, is_active=True, run_return=None, ru name = _name tags = _tags or [] - def __init__(self, execute_module=None): + def __init__(self, execute_module=None, task_vars=None, tmp=None): pass - @classmethod - def is_active(cls, task_vars): + def is_active(self): return is_active - def run(self, tmp, task_vars): + def run(self): if run_exception is not None: raise run_exception return run_return @@ -124,7 +123,7 @@ def test_action_plugin_skip_disabled_checks(plugin, task_vars, monkeypatch): def test_action_plugin_run_check_ok(plugin, task_vars, monkeypatch): check_return_value = {'ok': 'test'} check_class = fake_check(run_return=check_return_value) - monkeypatch.setattr(plugin, 'load_known_checks', lambda: {'fake_check': check_class()}) + monkeypatch.setattr(plugin, 'load_known_checks', lambda tmp, task_vars: {'fake_check': check_class()}) monkeypatch.setattr('openshift_health_check.resolve_checks', lambda *args: ['fake_check']) result = plugin.run(tmp=None, task_vars=task_vars) @@ -138,7 +137,7 @@ def test_action_plugin_run_check_ok(plugin, task_vars, monkeypatch): def test_action_plugin_run_check_changed(plugin, task_vars, monkeypatch): check_return_value = {'ok': 'test', 'changed': True} check_class = fake_check(run_return=check_return_value) - monkeypatch.setattr(plugin, 'load_known_checks', lambda: {'fake_check': check_class()}) + monkeypatch.setattr(plugin, 'load_known_checks', lambda tmp, task_vars: {'fake_check': check_class()}) monkeypatch.setattr('openshift_health_check.resolve_checks', lambda *args: ['fake_check']) result = plugin.run(tmp=None, task_vars=task_vars) @@ -152,7 +151,7 @@ def test_action_plugin_run_check_changed(plugin, task_vars, monkeypatch): def test_action_plugin_run_check_fail(plugin, task_vars, monkeypatch): check_return_value = {'failed': True} check_class = fake_check(run_return=check_return_value) - monkeypatch.setattr(plugin, 'load_known_checks', lambda: {'fake_check': check_class()}) + monkeypatch.setattr(plugin, 'load_known_checks', lambda tmp, task_vars: {'fake_check': check_class()}) monkeypatch.setattr('openshift_health_check.resolve_checks', lambda *args: ['fake_check']) result = plugin.run(tmp=None, task_vars=task_vars) @@ -167,7 +166,7 @@ def test_action_plugin_run_check_exception(plugin, task_vars, monkeypatch): exception_msg = 'fake check has an exception' run_exception = OpenShiftCheckException(exception_msg) check_class = fake_check(run_exception=run_exception) - monkeypatch.setattr(plugin, 'load_known_checks', lambda: {'fake_check': check_class()}) + monkeypatch.setattr(plugin, 'load_known_checks', lambda tmp, task_vars: {'fake_check': check_class()}) monkeypatch.setattr('openshift_health_check.resolve_checks', lambda *args: ['fake_check']) result = plugin.run(tmp=None, task_vars=task_vars) @@ -179,7 +178,7 @@ def test_action_plugin_run_check_exception(plugin, task_vars, monkeypatch): def test_action_plugin_resolve_checks_exception(plugin, task_vars, monkeypatch): - monkeypatch.setattr(plugin, 'load_known_checks', lambda: {}) + monkeypatch.setattr(plugin, 'load_known_checks', lambda tmp, task_vars: {}) result = plugin.run(tmp=None, task_vars=task_vars) diff --git a/roles/openshift_health_checker/test/aos_version_test.py b/roles/openshift_health_checker/test/aos_version_test.py index 697805dd2..4100f6c70 100644 --- a/roles/openshift_health_checker/test/aos_version_test.py +++ b/roles/openshift_health_checker/test/aos_version_test.py @@ -18,7 +18,43 @@ expected_pkgs = { } -@pytest.mark.parametrize('pkgs, expect_not_found', [ +@pytest.mark.parametrize('pkgs,expected_pkgs_dict', [ + ( + # all found + [Package('spam', '3.2.1'), Package('eggs', '3.2.1')], + expected_pkgs, + ), + ( + # found with more specific version + [Package('spam', '3.2.1'), Package('eggs', '3.2.1.5')], + expected_pkgs, + ), + ( + [Package('ovs', '2.6'), Package('ovs', '2.4')], + { + "ovs": { + "name": "ovs", + "version": ["2.6", "2.7"], + "check_multi": False, + } + }, + ), + ( + [Package('ovs', '2.7')], + { + "ovs": { + "name": "ovs", + "version": ["2.6", "2.7"], + "check_multi": False, + } + }, + ), +]) +def test_check_precise_version_found(pkgs, expected_pkgs_dict): + aos_version._check_precise_version_found(pkgs, expected_pkgs_dict) + + +@pytest.mark.parametrize('pkgs,expect_not_found', [ ( [], { @@ -55,14 +91,6 @@ expected_pkgs = { }, # not the right version ), ( - [Package('spam', '3.2.1'), Package('eggs', '3.2.1')], - {}, # all found - ), - ( - [Package('spam', '3.2.1'), Package('eggs', '3.2.1.5')], - {}, # found with more specific version - ), - ( [Package('eggs', '1.2.3'), Package('eggs', '3.2.1.5')], { "spam": { @@ -73,64 +101,86 @@ expected_pkgs = { }, # eggs found with multiple versions ), ]) -def test_check_pkgs_for_precise_version(pkgs, expect_not_found): - if expect_not_found: - with pytest.raises(aos_version.PreciseVersionNotFound) as e: - aos_version._check_precise_version_found(pkgs, expected_pkgs) - - assert list(expect_not_found.values()) == e.value.problem_pkgs - else: +def test_check_precise_version_found_fail(pkgs, expect_not_found): + with pytest.raises(aos_version.PreciseVersionNotFound) as e: aos_version._check_precise_version_found(pkgs, expected_pkgs) + assert list(expect_not_found.values()) == e.value.problem_pkgs -@pytest.mark.parametrize('pkgs, expect_higher', [ +@pytest.mark.parametrize('pkgs,expected_pkgs_dict', [ ( [], - [], + expected_pkgs, ), ( + # more precise but not strictly higher [Package('spam', '3.2.1.9')], - [], # more precise but not strictly higher + expected_pkgs, ), ( + [Package('ovs', '2.7')], + { + "ovs": { + "name": "ovs", + "version": ["2.6", "2.7"], + "check_multi": False, + } + }, + ), +]) +def test_check_higher_version_found(pkgs, expected_pkgs_dict): + aos_version._check_higher_version_found(pkgs, expected_pkgs_dict) + + +@pytest.mark.parametrize('pkgs,expected_pkgs_dict,expect_higher', [ + ( [Package('spam', '3.3')], + expected_pkgs, ['spam-3.3'], # lower precision, but higher ), ( [Package('spam', '3.2.1'), Package('eggs', '3.3.2')], + expected_pkgs, ['eggs-3.3.2'], # one too high ), ( [Package('eggs', '1.2.3'), Package('eggs', '3.2.1.5'), Package('eggs', '3.4')], + expected_pkgs, ['eggs-3.4'], # multiple versions, one is higher ), ( [Package('eggs', '3.2.1'), Package('eggs', '3.4'), Package('eggs', '3.3')], + expected_pkgs, ['eggs-3.4'], # multiple versions, two are higher ), + ( + [Package('ovs', '2.8')], + { + "ovs": { + "name": "ovs", + "version": ["2.6", "2.7"], + "check_multi": False, + } + }, + ['ovs-2.8'], + ), ]) -def test_check_pkgs_for_greater_version(pkgs, expect_higher): - if expect_higher: - with pytest.raises(aos_version.FoundHigherVersion) as e: - aos_version._check_higher_version_found(pkgs, expected_pkgs) - assert set(expect_higher) == set(e.value.problem_pkgs) - else: - aos_version._check_higher_version_found(pkgs, expected_pkgs) +def test_check_higher_version_found_fail(pkgs, expected_pkgs_dict, expect_higher): + with pytest.raises(aos_version.FoundHigherVersion) as e: + aos_version._check_higher_version_found(pkgs, expected_pkgs_dict) + assert set(expect_higher) == set(e.value.problem_pkgs) -@pytest.mark.parametrize('pkgs, expect_to_flag_pkgs', [ - ( - [], - [], - ), - ( - [Package('spam', '3.2.1')], - [], - ), - ( - [Package('spam', '3.2.1'), Package('eggs', '3.2.2')], - [], - ), +@pytest.mark.parametrize('pkgs', [ + [], + [Package('spam', '3.2.1')], + [Package('spam', '3.2.1'), Package('eggs', '3.2.2')], +]) +def test_check_multi_minor_release(pkgs): + aos_version._check_multi_minor_release(pkgs, expected_pkgs) + + +@pytest.mark.parametrize('pkgs,expect_to_flag_pkgs', [ ( [Package('spam', '3.2.1'), Package('spam', '3.3.2')], ['spam'], @@ -140,10 +190,7 @@ def test_check_pkgs_for_greater_version(pkgs, expect_higher): ['eggs'], ), ]) -def test_check_pkgs_for_multi_release(pkgs, expect_to_flag_pkgs): - if expect_to_flag_pkgs: - with pytest.raises(aos_version.FoundMultiRelease) as e: - aos_version._check_multi_minor_release(pkgs, expected_pkgs) - assert set(expect_to_flag_pkgs) == set(e.value.problem_pkgs) - else: +def test_check_multi_minor_release_fail(pkgs, expect_to_flag_pkgs): + with pytest.raises(aos_version.FoundMultiRelease) as e: aos_version._check_multi_minor_release(pkgs, expected_pkgs) + assert set(expect_to_flag_pkgs) == set(e.value.problem_pkgs) diff --git a/roles/openshift_health_checker/test/disk_availability_test.py b/roles/openshift_health_checker/test/disk_availability_test.py index 945b9eafc..e98d02c58 100644 --- a/roles/openshift_health_checker/test/disk_availability_test.py +++ b/roles/openshift_health_checker/test/disk_availability_test.py @@ -17,7 +17,7 @@ def test_is_active(group_names, is_active): task_vars = dict( group_names=group_names, ) - assert DiskAvailability.is_active(task_vars=task_vars) == is_active + assert DiskAvailability(None, task_vars).is_active() == is_active @pytest.mark.parametrize('ansible_mounts,extra_words', [ @@ -30,10 +30,9 @@ def test_cannot_determine_available_disk(ansible_mounts, extra_words): group_names=['masters'], ansible_mounts=ansible_mounts, ) - check = DiskAvailability(execute_module=fake_execute_module) with pytest.raises(OpenShiftCheckException) as excinfo: - check.run(tmp=None, task_vars=task_vars) + DiskAvailability(fake_execute_module, task_vars).run() for word in 'determine disk availability'.split() + extra_words: assert word in str(excinfo.value) @@ -93,8 +92,7 @@ def test_succeeds_with_recommended_disk_space(group_names, configured_min, ansib ansible_mounts=ansible_mounts, ) - check = DiskAvailability(execute_module=fake_execute_module) - result = check.run(tmp=None, task_vars=task_vars) + result = DiskAvailability(fake_execute_module, task_vars).run() assert not result.get('failed', False) @@ -168,8 +166,7 @@ def test_fails_with_insufficient_disk_space(group_names, configured_min, ansible ansible_mounts=ansible_mounts, ) - check = DiskAvailability(execute_module=fake_execute_module) - result = check.run(tmp=None, task_vars=task_vars) + result = DiskAvailability(fake_execute_module, task_vars).run() assert result['failed'] for word in 'below recommended'.split() + extra_words: diff --git a/roles/openshift_health_checker/test/docker_image_availability_test.py b/roles/openshift_health_checker/test/docker_image_availability_test.py index 3b9e097fb..8d0a53df9 100644 --- a/roles/openshift_health_checker/test/docker_image_availability_test.py +++ b/roles/openshift_health_checker/test/docker_image_availability_test.py @@ -21,7 +21,7 @@ def test_is_active(deployment_type, is_containerized, group_names, expect_active openshift_deployment_type=deployment_type, group_names=group_names, ) - assert DockerImageAvailability.is_active(task_vars=task_vars) == expect_active + assert DockerImageAvailability(None, task_vars).is_active() == expect_active @pytest.mark.parametrize("is_containerized,is_atomic", [ @@ -31,7 +31,7 @@ def test_is_active(deployment_type, is_containerized, group_names, expect_active (False, True), ]) def test_all_images_available_locally(is_containerized, is_atomic): - def execute_module(module_name, module_args, task_vars): + def execute_module(module_name, module_args, *_): if module_name == "yum": return {"changed": True} @@ -42,7 +42,7 @@ def test_all_images_available_locally(is_containerized, is_atomic): 'images': [module_args['name']], } - result = DockerImageAvailability(execute_module=execute_module).run(tmp=None, task_vars=dict( + result = DockerImageAvailability(execute_module, task_vars=dict( openshift=dict( common=dict( service_type='origin', @@ -54,7 +54,7 @@ def test_all_images_available_locally(is_containerized, is_atomic): openshift_deployment_type='origin', openshift_image_tag='3.4', group_names=['nodes', 'masters'], - )) + )).run() assert not result.get('failed', False) @@ -64,12 +64,12 @@ def test_all_images_available_locally(is_containerized, is_atomic): True, ]) def test_all_images_available_remotely(available_locally): - def execute_module(module_name, module_args, task_vars): + def execute_module(module_name, *_): if module_name == 'docker_image_facts': return {'images': [], 'failed': available_locally} return {'changed': False} - result = DockerImageAvailability(execute_module=execute_module).run(tmp=None, task_vars=dict( + result = DockerImageAvailability(execute_module, task_vars=dict( openshift=dict( common=dict( service_type='origin', @@ -81,13 +81,13 @@ def test_all_images_available_remotely(available_locally): openshift_deployment_type='origin', openshift_image_tag='v3.4', group_names=['nodes', 'masters'], - )) + )).run() assert not result.get('failed', False) def test_all_images_unavailable(): - def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): + def execute_module(module_name=None, *_): if module_name == "command": return { 'failed': True, @@ -97,8 +97,7 @@ def test_all_images_unavailable(): 'changed': False, } - check = DockerImageAvailability(execute_module=execute_module) - actual = check.run(tmp=None, task_vars=dict( + actual = DockerImageAvailability(execute_module, task_vars=dict( openshift=dict( common=dict( service_type='origin', @@ -110,7 +109,7 @@ def test_all_images_unavailable(): openshift_deployment_type="openshift-enterprise", openshift_image_tag='latest', group_names=['nodes', 'masters'], - )) + )).run() assert actual['failed'] assert "required Docker images are not available" in actual['msg'] @@ -127,7 +126,7 @@ def test_all_images_unavailable(): ), ]) def test_skopeo_update_failure(message, extra_words): - def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): + def execute_module(module_name=None, *_): if module_name == "yum": return { "failed": True, @@ -137,7 +136,7 @@ def test_skopeo_update_failure(message, extra_words): return {'changed': False} - actual = DockerImageAvailability(execute_module=execute_module).run(tmp=None, task_vars=dict( + actual = DockerImageAvailability(execute_module, task_vars=dict( openshift=dict( common=dict( service_type='origin', @@ -149,7 +148,7 @@ def test_skopeo_update_failure(message, extra_words): openshift_deployment_type="openshift-enterprise", openshift_image_tag='', group_names=['nodes', 'masters'], - )) + )).run() assert actual["failed"] for word in extra_words: @@ -162,12 +161,12 @@ def test_skopeo_update_failure(message, extra_words): ("openshift-enterprise", []), ]) def test_registry_availability(deployment_type, registries): - def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): + def execute_module(module_name=None, *_): return { 'changed': False, } - actual = DockerImageAvailability(execute_module=execute_module).run(tmp=None, task_vars=dict( + actual = DockerImageAvailability(execute_module, task_vars=dict( openshift=dict( common=dict( service_type='origin', @@ -179,7 +178,7 @@ def test_registry_availability(deployment_type, registries): openshift_deployment_type=deployment_type, openshift_image_tag='', group_names=['nodes', 'masters'], - )) + )).run() assert not actual.get("failed", False) @@ -258,7 +257,7 @@ def test_required_images(deployment_type, is_containerized, groups, oreg_url, ex openshift_image_tag='vtest', ) - assert expected == DockerImageAvailability("DUMMY").required_images(task_vars) + assert expected == DockerImageAvailability("DUMMY", task_vars).required_images() def test_containerized_etcd(): @@ -272,4 +271,4 @@ def test_containerized_etcd(): group_names=['etcd'], ) expected = set(['registry.access.redhat.com/rhel7/etcd']) - assert expected == DockerImageAvailability("DUMMY").required_images(task_vars) + assert expected == DockerImageAvailability("DUMMY", task_vars).required_images() diff --git a/roles/openshift_health_checker/test/docker_storage_test.py b/roles/openshift_health_checker/test/docker_storage_test.py index bb25e3f66..e0dccc062 100644 --- a/roles/openshift_health_checker/test/docker_storage_test.py +++ b/roles/openshift_health_checker/test/docker_storage_test.py @@ -4,12 +4,6 @@ from openshift_checks import OpenShiftCheckException from openshift_checks.docker_storage import DockerStorage -def dummy_check(execute_module=None): - def dummy_exec(self, status, task_vars): - raise Exception("dummy executor called") - return DockerStorage(execute_module=execute_module or dummy_exec) - - @pytest.mark.parametrize('is_containerized, group_names, is_active', [ (False, ["masters", "etcd"], False), (False, ["masters", "nodes"], True), @@ -20,10 +14,11 @@ def test_is_active(is_containerized, group_names, is_active): openshift=dict(common=dict(is_containerized=is_containerized)), group_names=group_names, ) - assert DockerStorage.is_active(task_vars=task_vars) == is_active + assert DockerStorage(None, task_vars).is_active() == is_active -non_atomic_task_vars = {"openshift": {"common": {"is_atomic": False}}} +def non_atomic_task_vars(): + return {"openshift": {"common": {"is_atomic": False}}} @pytest.mark.parametrize('docker_info, failed, expect_msg', [ @@ -56,7 +51,7 @@ non_atomic_task_vars = {"openshift": {"common": {"is_atomic": False}}} ( dict(info={ "Driver": "overlay2", - "DriverStatus": [] + "DriverStatus": [("Backing Filesystem", "xfs")], }), False, [], @@ -64,6 +59,27 @@ non_atomic_task_vars = {"openshift": {"common": {"is_atomic": False}}} ( dict(info={ "Driver": "overlay", + "DriverStatus": [("Backing Filesystem", "btrfs")], + }), + True, + ["storage is type 'btrfs'", "only supported with\n'xfs'"], + ), + ( + dict(info={ + "Driver": "overlay2", + "DriverStatus": [("Backing Filesystem", "xfs")], + "OperatingSystem": "Red Hat Enterprise Linux Server release 7.2 (Maipo)", + "KernelVersion": "3.10.0-327.22.2.el7.x86_64", + }), + True, + ["Docker reports kernel version 3.10.0-327"], + ), + ( + dict(info={ + "Driver": "overlay", + "DriverStatus": [("Backing Filesystem", "xfs")], + "OperatingSystem": "CentOS", + "KernelVersion": "3.10.0-514", }), False, [], @@ -77,16 +93,17 @@ non_atomic_task_vars = {"openshift": {"common": {"is_atomic": False}}} ), ]) def test_check_storage_driver(docker_info, failed, expect_msg): - def execute_module(module_name, module_args, tmp=None, task_vars=None): + def execute_module(module_name, *_): if module_name == "yum": return {} if module_name != "docker_info": raise ValueError("not expecting module " + module_name) return docker_info - check = dummy_check(execute_module=execute_module) - check._check_dm_usage = lambda status, task_vars: dict() # stub out for this test - result = check.run(tmp=None, task_vars=non_atomic_task_vars) + check = DockerStorage(execute_module, non_atomic_task_vars()) + check.check_dm_usage = lambda status: dict() # stub out for this test + check.check_overlay_usage = lambda info: dict() # stub out for this test + result = check.run() if failed: assert result["failed"] @@ -145,9 +162,9 @@ not_enough_space = { ), ]) def test_dm_usage(task_vars, driver_status, vg_free, success, expect_msg): - check = dummy_check() - check._get_vg_free = lambda pool, task_vars: vg_free - result = check._check_dm_usage(driver_status, task_vars) + check = DockerStorage(None, task_vars) + check.get_vg_free = lambda pool: vg_free + result = check.check_dm_usage(driver_status) result_success = not result.get("failed") assert result_success is success @@ -187,18 +204,18 @@ def test_dm_usage(task_vars, driver_status, vg_free, success, expect_msg): ) ]) def test_vg_free(pool, command_returns, raises, returns): - def execute_module(module_name, module_args, tmp=None, task_vars=None): + def execute_module(module_name, *_): if module_name != "command": raise ValueError("not expecting module " + module_name) return command_returns - check = dummy_check(execute_module=execute_module) + check = DockerStorage(execute_module) if raises: with pytest.raises(OpenShiftCheckException) as err: - check._get_vg_free(pool, {}) + check.get_vg_free(pool) assert raises in str(err.value) else: - ret = check._get_vg_free(pool, {}) + ret = check.get_vg_free(pool) assert ret == returns @@ -209,7 +226,7 @@ def test_vg_free(pool, command_returns, raises, returns): ("12g", 12.0 * 1024**3), ]) def test_convert_to_bytes(string, expect_bytes): - got = DockerStorage._convert_to_bytes(string) + got = DockerStorage.convert_to_bytes(string) assert got == expect_bytes @@ -219,6 +236,70 @@ def test_convert_to_bytes(string, expect_bytes): ]) def test_convert_to_bytes_error(string): with pytest.raises(ValueError) as err: - DockerStorage._convert_to_bytes(string) + DockerStorage.convert_to_bytes(string) assert "Cannot convert" in str(err.value) assert string in str(err.value) + + +ansible_mounts_enough = [{ + 'mount': '/var/lib/docker', + 'size_available': 50 * 10**9, + 'size_total': 50 * 10**9, +}] +ansible_mounts_not_enough = [{ + 'mount': '/var/lib/docker', + 'size_available': 0, + 'size_total': 50 * 10**9, +}] +ansible_mounts_missing_fields = [dict(mount='/var/lib/docker')] +ansible_mounts_zero_size = [{ + 'mount': '/var/lib/docker', + 'size_available': 0, + 'size_total': 0, +}] + + +@pytest.mark.parametrize('ansible_mounts, threshold, expect_fail, expect_msg', [ + ( + ansible_mounts_enough, + None, + False, + [], + ), + ( + ansible_mounts_not_enough, + None, + True, + ["usage percentage", "higher than threshold"], + ), + ( + ansible_mounts_not_enough, + "bogus percent", + True, + ["is not a percentage"], + ), + ( + ansible_mounts_missing_fields, + None, + True, + ["Ansible bug"], + ), + ( + ansible_mounts_zero_size, + None, + True, + ["Ansible bug"], + ), +]) +def test_overlay_usage(ansible_mounts, threshold, expect_fail, expect_msg): + task_vars = non_atomic_task_vars() + task_vars["ansible_mounts"] = ansible_mounts + if threshold is not None: + task_vars["max_overlay_usage_percent"] = threshold + check = DockerStorage(None, task_vars) + docker_info = dict(DockerRootDir="/var/lib/docker", Driver="overlay") + result = check.check_overlay_usage(docker_info) + + assert expect_fail == bool(result.get("failed")) + for msg in expect_msg: + assert msg in result["msg"] diff --git a/roles/openshift_health_checker/test/elasticsearch_test.py b/roles/openshift_health_checker/test/elasticsearch_test.py index b9d375d8c..9edfc17c7 100644 --- a/roles/openshift_health_checker/test/elasticsearch_test.py +++ b/roles/openshift_health_checker/test/elasticsearch_test.py @@ -6,9 +6,9 @@ from openshift_checks.logging.elasticsearch import Elasticsearch task_vars_config_base = dict(openshift=dict(common=dict(config_base='/etc/origin'))) -def canned_elasticsearch(exec_oc=None): +def canned_elasticsearch(task_vars=None, exec_oc=None): """Create an Elasticsearch check object with canned exec_oc method""" - check = Elasticsearch("dummy") # fails if a module is actually invoked + check = Elasticsearch("dummy", task_vars or {}) # fails if a module is actually invoked if exec_oc: check._exec_oc = exec_oc return check @@ -50,10 +50,10 @@ split_es_pod = { def test_check_elasticsearch(): - assert 'No logging Elasticsearch pods' in canned_elasticsearch().check_elasticsearch([], {}) + assert 'No logging Elasticsearch pods' in canned_elasticsearch().check_elasticsearch([]) # canned oc responses to match so all the checks pass - def _exec_oc(cmd, args, task_vars): + def _exec_oc(cmd, args): if '_cat/master' in cmd: return 'name logging-es' elif '/_nodes' in cmd: @@ -65,7 +65,7 @@ def test_check_elasticsearch(): else: raise Exception(cmd) - assert not canned_elasticsearch(_exec_oc).check_elasticsearch([plain_es_pod], {}) + assert not canned_elasticsearch({}, _exec_oc).check_elasticsearch([plain_es_pod]) def pods_by_name(pods): @@ -88,9 +88,9 @@ def pods_by_name(pods): ]) def test_check_elasticsearch_masters(pods, expect_error): test_pods = list(pods) - check = canned_elasticsearch(lambda cmd, args, task_vars: test_pods.pop(0)['_test_master_name_str']) + check = canned_elasticsearch(task_vars_config_base, lambda cmd, args: test_pods.pop(0)['_test_master_name_str']) - errors = check._check_elasticsearch_masters(pods_by_name(pods), task_vars_config_base) + errors = check._check_elasticsearch_masters(pods_by_name(pods)) assert_error(''.join(errors), expect_error) @@ -124,9 +124,9 @@ es_node_list = { ), ]) def test_check_elasticsearch_node_list(pods, node_list, expect_error): - check = canned_elasticsearch(lambda cmd, args, task_vars: json.dumps(node_list)) + check = canned_elasticsearch(task_vars_config_base, lambda cmd, args: json.dumps(node_list)) - errors = check._check_elasticsearch_node_list(pods_by_name(pods), task_vars_config_base) + errors = check._check_elasticsearch_node_list(pods_by_name(pods)) assert_error(''.join(errors), expect_error) @@ -149,9 +149,9 @@ def test_check_elasticsearch_node_list(pods, node_list, expect_error): ]) def test_check_elasticsearch_cluster_health(pods, health_data, expect_error): test_health_data = list(health_data) - check = canned_elasticsearch(lambda cmd, args, task_vars: json.dumps(test_health_data.pop(0))) + check = canned_elasticsearch(task_vars_config_base, lambda cmd, args: json.dumps(test_health_data.pop(0))) - errors = check._check_es_cluster_health(pods_by_name(pods), task_vars_config_base) + errors = check._check_es_cluster_health(pods_by_name(pods)) assert_error(''.join(errors), expect_error) @@ -174,7 +174,7 @@ def test_check_elasticsearch_cluster_health(pods, health_data, expect_error): ), ]) def test_check_elasticsearch_diskspace(disk_data, expect_error): - check = canned_elasticsearch(lambda cmd, args, task_vars: disk_data) + check = canned_elasticsearch(task_vars_config_base, lambda cmd, args: disk_data) - errors = check._check_elasticsearch_diskspace(pods_by_name([plain_es_pod]), task_vars_config_base) + errors = check._check_elasticsearch_diskspace(pods_by_name([plain_es_pod])) assert_error(''.join(errors), expect_error) diff --git a/roles/openshift_health_checker/test/etcd_imagedata_size_test.py b/roles/openshift_health_checker/test/etcd_imagedata_size_test.py index df9d52d41..e3d6706fa 100644 --- a/roles/openshift_health_checker/test/etcd_imagedata_size_test.py +++ b/roles/openshift_health_checker/test/etcd_imagedata_size_test.py @@ -51,10 +51,10 @@ def test_cannot_determine_available_mountpath(ansible_mounts, extra_words): task_vars = dict( ansible_mounts=ansible_mounts, ) - check = EtcdImageDataSize(execute_module=fake_execute_module) + check = EtcdImageDataSize(fake_execute_module, task_vars) with pytest.raises(OpenShiftCheckException) as excinfo: - check.run(tmp=None, task_vars=task_vars) + check.run() for word in 'determine valid etcd mountpath'.split() + extra_words: assert word in str(excinfo.value) @@ -111,14 +111,14 @@ def test_cannot_determine_available_mountpath(ansible_mounts, extra_words): ) ]) def test_check_etcd_key_size_calculates_correct_limit(ansible_mounts, tree, size_limit, should_fail, extra_words): - def execute_module(module_name, args, tmp=None, task_vars=None): + def execute_module(module_name, module_args, *_): if module_name != "etcdkeysize": return { "changed": False, } client = fake_etcd_client(tree) - s, limit_exceeded = check_etcd_key_size(client, tree["key"], args["size_limit_bytes"]) + s, limit_exceeded = check_etcd_key_size(client, tree["key"], module_args["size_limit_bytes"]) return {"size_limit_exceeded": limit_exceeded} @@ -133,7 +133,7 @@ def test_check_etcd_key_size_calculates_correct_limit(ansible_mounts, tree, size if size_limit is None: task_vars.pop("etcd_max_image_data_size_bytes") - check = EtcdImageDataSize(execute_module=execute_module).run(tmp=None, task_vars=task_vars) + check = EtcdImageDataSize(execute_module, task_vars).run() if should_fail: assert check["failed"] @@ -267,14 +267,14 @@ def test_check_etcd_key_size_calculates_correct_limit(ansible_mounts, tree, size ), ]) def test_etcd_key_size_check_calculates_correct_size(ansible_mounts, tree, root_path, expected_size, extra_words): - def execute_module(module_name, args, tmp=None, task_vars=None): + def execute_module(module_name, module_args, *_): if module_name != "etcdkeysize": return { "changed": False, } client = fake_etcd_client(tree) - size, limit_exceeded = check_etcd_key_size(client, root_path, args["size_limit_bytes"]) + size, limit_exceeded = check_etcd_key_size(client, root_path, module_args["size_limit_bytes"]) assert size == expected_size return { @@ -289,12 +289,12 @@ def test_etcd_key_size_check_calculates_correct_size(ansible_mounts, tree, root_ ) ) - check = EtcdImageDataSize(execute_module=execute_module).run(tmp=None, task_vars=task_vars) + check = EtcdImageDataSize(execute_module, task_vars).run() assert not check.get("failed", False) def test_etcdkeysize_module_failure(): - def execute_module(module_name, tmp=None, task_vars=None): + def execute_module(module_name, *_): if module_name != "etcdkeysize": return { "changed": False, @@ -317,7 +317,7 @@ def test_etcdkeysize_module_failure(): ) ) - check = EtcdImageDataSize(execute_module=execute_module).run(tmp=None, task_vars=task_vars) + check = EtcdImageDataSize(execute_module, task_vars).run() assert check["failed"] for word in "Failed to retrieve stats": diff --git a/roles/openshift_health_checker/test/etcd_traffic_test.py b/roles/openshift_health_checker/test/etcd_traffic_test.py new file mode 100644 index 000000000..f4316c423 --- /dev/null +++ b/roles/openshift_health_checker/test/etcd_traffic_test.py @@ -0,0 +1,74 @@ +import pytest + +from openshift_checks.etcd_traffic import EtcdTraffic + + +@pytest.mark.parametrize('group_names,version,is_active', [ + (['masters'], "3.5", False), + (['masters'], "3.6", False), + (['nodes'], "3.4", False), + (['etcd'], "3.4", True), + (['etcd'], "3.5", True), + (['etcd'], "3.1", False), + (['masters', 'nodes'], "3.5", False), + (['masters', 'etcd'], "3.5", True), + ([], "3.4", False), +]) +def test_is_active(group_names, version, is_active): + task_vars = dict( + group_names=group_names, + openshift=dict( + common=dict(short_version=version), + ), + ) + assert EtcdTraffic(task_vars=task_vars).is_active() == is_active + + +@pytest.mark.parametrize('group_names,matched,failed,extra_words', [ + (["masters"], True, True, ["Higher than normal", "traffic"]), + (["masters", "etcd"], False, False, []), + (["etcd"], False, False, []), +]) +def test_log_matches_high_traffic_msg(group_names, matched, failed, extra_words): + def execute_module(module_name, *_): + return { + "matched": matched, + "failed": failed, + } + + task_vars = dict( + group_names=group_names, + openshift=dict( + common=dict(service_type="origin", is_containerized=False), + ) + ) + + result = EtcdTraffic(execute_module, task_vars).run() + + for word in extra_words: + assert word in result.get("msg", "") + + assert result.get("failed", False) == failed + + +@pytest.mark.parametrize('is_containerized,expected_unit_value', [ + (False, "etcd"), + (True, "etcd_container"), +]) +def test_systemd_unit_matches_deployment_type(is_containerized, expected_unit_value): + task_vars = dict( + openshift=dict( + common=dict(is_containerized=is_containerized), + ) + ) + + def execute_module(module_name, args, *_): + assert module_name == "search_journalctl" + matchers = args["log_matchers"] + + for matcher in matchers: + assert matcher["unit"] == expected_unit_value + + return {"failed": False} + + EtcdTraffic(execute_module, task_vars).run() diff --git a/roles/openshift_health_checker/test/etcd_volume_test.py b/roles/openshift_health_checker/test/etcd_volume_test.py index 917045526..0b255136e 100644 --- a/roles/openshift_health_checker/test/etcd_volume_test.py +++ b/roles/openshift_health_checker/test/etcd_volume_test.py @@ -11,10 +11,9 @@ def test_cannot_determine_available_disk(ansible_mounts, extra_words): task_vars = dict( ansible_mounts=ansible_mounts, ) - check = EtcdVolume(execute_module=fake_execute_module) with pytest.raises(OpenShiftCheckException) as excinfo: - check.run(tmp=None, task_vars=task_vars) + EtcdVolume(fake_execute_module, task_vars).run() for word in 'Unable to find etcd storage mount point'.split() + extra_words: assert word in str(excinfo.value) @@ -76,8 +75,7 @@ def test_succeeds_with_recommended_disk_space(size_limit, ansible_mounts): if task_vars["etcd_device_usage_threshold_percent"] is None: task_vars.pop("etcd_device_usage_threshold_percent") - check = EtcdVolume(execute_module=fake_execute_module) - result = check.run(tmp=None, task_vars=task_vars) + result = EtcdVolume(fake_execute_module, task_vars).run() assert not result.get('failed', False) @@ -137,8 +135,7 @@ def test_fails_with_insufficient_disk_space(size_limit_percent, ansible_mounts, if task_vars["etcd_device_usage_threshold_percent"] is None: task_vars.pop("etcd_device_usage_threshold_percent") - check = EtcdVolume(execute_module=fake_execute_module) - result = check.run(tmp=None, task_vars=task_vars) + result = EtcdVolume(fake_execute_module, task_vars).run() assert result['failed'] for word in extra_words: diff --git a/roles/openshift_health_checker/test/fluentd_test.py b/roles/openshift_health_checker/test/fluentd_test.py index d151c0b19..9cee57868 100644 --- a/roles/openshift_health_checker/test/fluentd_test.py +++ b/roles/openshift_health_checker/test/fluentd_test.py @@ -103,7 +103,7 @@ fluentd_node3_unlabeled = { ), ]) def test_get_fluentd_pods(pods, nodes, expect_error): - check = canned_fluentd(lambda cmd, args, task_vars: json.dumps(dict(items=nodes))) + check = canned_fluentd(exec_oc=lambda cmd, args: json.dumps(dict(items=nodes))) - error = check.check_fluentd(pods, {}) + error = check.check_fluentd(pods) assert_error(error, expect_error) diff --git a/roles/openshift_health_checker/test/kibana_test.py b/roles/openshift_health_checker/test/kibana_test.py index 40a5d19d8..3a880d300 100644 --- a/roles/openshift_health_checker/test/kibana_test.py +++ b/roles/openshift_health_checker/test/kibana_test.py @@ -13,7 +13,7 @@ from openshift_checks.logging.kibana import Kibana def canned_kibana(exec_oc=None): """Create a Kibana check object with canned exec_oc method""" - check = Kibana("dummy") # fails if a module is actually invoked + check = Kibana() # fails if a module is actually invoked if exec_oc: check._exec_oc = exec_oc return check @@ -137,9 +137,9 @@ def test_check_kibana(pods, expect_error): ), ]) def test_get_kibana_url(route, expect_url, expect_error): - check = canned_kibana(lambda cmd, args, task_vars: json.dumps(route) if route else "") + check = canned_kibana(exec_oc=lambda cmd, args: json.dumps(route) if route else "") - url, error = check._get_kibana_url({}) + url, error = check._get_kibana_url() if expect_url: assert url == expect_url else: @@ -169,10 +169,10 @@ def test_get_kibana_url(route, expect_url, expect_error): ), ]) def test_verify_url_internal_failure(exec_result, expect): - check = Kibana(execute_module=lambda module_name, args, tmp, task_vars: dict(failed=True, msg=exec_result)) - check._get_kibana_url = lambda task_vars: ('url', None) + check = Kibana(execute_module=lambda *_: dict(failed=True, msg=exec_result)) + check._get_kibana_url = lambda: ('url', None) - error = check._check_kibana_route({}) + error = check._check_kibana_route() assert_error(error, expect) @@ -211,8 +211,8 @@ def test_verify_url_external_failure(lib_result, expect, monkeypatch): monkeypatch.setattr(urllib2, 'urlopen', urlopen) check = canned_kibana() - check._get_kibana_url = lambda task_vars: ('url', None) - check._verify_url_internal = lambda url, task_vars: None + check._get_kibana_url = lambda: ('url', None) + check._verify_url_internal = lambda url: None - error = check._check_kibana_route({}) + error = check._check_kibana_route() assert_error(error, expect) diff --git a/roles/openshift_health_checker/test/logging_check_test.py b/roles/openshift_health_checker/test/logging_check_test.py index 128b76b12..6f1697ee6 100644 --- a/roles/openshift_health_checker/test/logging_check_test.py +++ b/roles/openshift_health_checker/test/logging_check_test.py @@ -11,7 +11,7 @@ logging_namespace = "logging" def canned_loggingcheck(exec_oc=None): """Create a LoggingCheck object with canned exec_oc method""" - check = LoggingCheck("dummy") # fails if a module is actually invoked + check = LoggingCheck() # fails if a module is actually invoked check.logging_namespace = 'logging' if exec_oc: check.exec_oc = exec_oc @@ -90,15 +90,15 @@ plain_curator_pod = { ("Permission denied", "Unexpected error using `oc`"), ]) def test_oc_failure(problem, expect): - def execute_module(module_name, args, tmp, task_vars): + def execute_module(module_name, *_): if module_name == "ocutil": return dict(failed=True, result=problem) return dict(changed=False) - check = LoggingCheck({}) + check = LoggingCheck(execute_module, task_vars_config_base) with pytest.raises(OpenShiftCheckException) as excinfo: - check.exec_oc(execute_module, logging_namespace, 'get foo', [], task_vars=task_vars_config_base) + check.exec_oc(logging_namespace, 'get foo', []) assert expect in str(excinfo) @@ -121,14 +121,14 @@ def test_is_active(groups, logging_deployed, is_active): openshift_hosted_logging_deploy=logging_deployed, ) - assert LoggingCheck.is_active(task_vars=task_vars) == is_active + assert LoggingCheck(None, task_vars).is_active() == is_active @pytest.mark.parametrize('pod_output, expect_pods, expect_error', [ ( 'No resources found.', None, - 'There are no pods in the logging namespace', + 'No pods were found for the "es"', ), ( json.dumps({'items': [plain_kibana_pod, plain_es_pod, plain_curator_pod, fluentd_pod_node1]}), @@ -137,12 +137,10 @@ def test_is_active(groups, logging_deployed, is_active): ), ]) def test_get_pods_for_component(pod_output, expect_pods, expect_error): - check = canned_loggingcheck(lambda exec_module, namespace, cmd, args, task_vars: pod_output) + check = canned_loggingcheck(lambda namespace, cmd, args: pod_output) pods, error = check.get_pods_for_component( - lambda name, args, task_vars: {}, logging_namespace, "es", - {} ) assert_error(error, expect_error) diff --git a/roles/openshift_health_checker/test/logging_index_time_test.py b/roles/openshift_health_checker/test/logging_index_time_test.py new file mode 100644 index 000000000..178d7cd84 --- /dev/null +++ b/roles/openshift_health_checker/test/logging_index_time_test.py @@ -0,0 +1,170 @@ +import json + +import pytest + +from openshift_checks.logging.logging_index_time import LoggingIndexTime, OpenShiftCheckException + + +SAMPLE_UUID = "unique-test-uuid" + + +def canned_loggingindextime(exec_oc=None): + """Create a check object with a canned exec_oc method""" + check = LoggingIndexTime() # fails if a module is actually invoked + if exec_oc: + check.exec_oc = exec_oc + return check + + +plain_running_elasticsearch_pod = { + "metadata": { + "labels": {"component": "es", "deploymentconfig": "logging-es-data-master"}, + "name": "logging-es-data-master-1", + }, + "status": { + "containerStatuses": [{"ready": True}, {"ready": True}], + "phase": "Running", + } +} +plain_running_kibana_pod = { + "metadata": { + "labels": {"component": "kibana", "deploymentconfig": "logging-kibana"}, + "name": "logging-kibana-1", + }, + "status": { + "containerStatuses": [{"ready": True}, {"ready": True}], + "phase": "Running", + } +} +not_running_kibana_pod = { + "metadata": { + "labels": {"component": "kibana", "deploymentconfig": "logging-kibana"}, + "name": "logging-kibana-2", + }, + "status": { + "containerStatuses": [{"ready": True}, {"ready": False}], + "conditions": [{"status": "True", "type": "Ready"}], + "phase": "pending", + } +} + + +@pytest.mark.parametrize('pods, expect_pods', [ + ( + [not_running_kibana_pod], + [], + ), + ( + [plain_running_kibana_pod], + [plain_running_kibana_pod], + ), + ( + [], + [], + ) +]) +def test_check_running_pods(pods, expect_pods): + check = canned_loggingindextime() + pods = check.running_pods(pods) + assert pods == expect_pods + + +@pytest.mark.parametrize('name, json_response, uuid, timeout, extra_words', [ + ( + 'valid count in response', + { + "count": 1, + }, + SAMPLE_UUID, + 0.001, + [], + ), +], ids=lambda argval: argval[0]) +def test_wait_until_cmd_or_err_succeeds(name, json_response, uuid, timeout, extra_words): + check = canned_loggingindextime(lambda *_: json.dumps(json_response)) + check.wait_until_cmd_or_err(plain_running_elasticsearch_pod, uuid, timeout) + + +@pytest.mark.parametrize('name, json_response, uuid, timeout, extra_words', [ + ( + 'invalid json response', + { + "invalid_field": 1, + }, + SAMPLE_UUID, + 0.001, + ["invalid response", "Elasticsearch"], + ), + ( + 'empty response', + {}, + SAMPLE_UUID, + 0.001, + ["invalid response", "Elasticsearch"], + ), + ( + 'valid response but invalid match count', + { + "count": 0, + }, + SAMPLE_UUID, + 0.005, + ["expecting match", SAMPLE_UUID, "0.005s"], + ) +], ids=lambda argval: argval[0]) +def test_wait_until_cmd_or_err(name, json_response, uuid, timeout, extra_words): + check = canned_loggingindextime(lambda *_: json.dumps(json_response)) + with pytest.raises(OpenShiftCheckException) as error: + check.wait_until_cmd_or_err(plain_running_elasticsearch_pod, uuid, timeout) + + for word in extra_words: + assert word in str(error) + + +@pytest.mark.parametrize('name, json_response, uuid, extra_words', [ + ( + 'correct response code, found unique id is returned', + { + "statusCode": 404, + }, + "sample unique id", + ["sample unique id"], + ), +], ids=lambda argval: argval[0]) +def test_curl_kibana_with_uuid(name, json_response, uuid, extra_words): + check = canned_loggingindextime(lambda *_: json.dumps(json_response)) + check.generate_uuid = lambda: uuid + + result = check.curl_kibana_with_uuid(plain_running_kibana_pod) + + for word in extra_words: + assert word in result + + +@pytest.mark.parametrize('name, json_response, uuid, extra_words', [ + ( + 'invalid json response', + { + "invalid_field": "invalid", + }, + SAMPLE_UUID, + ["invalid response returned", 'Missing "statusCode" key'], + ), + ( + 'wrong error code in response', + { + "statusCode": 500, + }, + SAMPLE_UUID, + ["Expecting error code", "500"], + ), +], ids=lambda argval: argval[0]) +def test_failed_curl_kibana_with_uuid(name, json_response, uuid, extra_words): + check = canned_loggingindextime(lambda *_: json.dumps(json_response)) + check.generate_uuid = lambda: uuid + + with pytest.raises(OpenShiftCheckException) as error: + check.curl_kibana_with_uuid(plain_running_kibana_pod) + + for word in extra_words: + assert word in str(error) diff --git a/roles/openshift_health_checker/test/memory_availability_test.py b/roles/openshift_health_checker/test/memory_availability_test.py index 4fbaea0a9..aee2f0416 100644 --- a/roles/openshift_health_checker/test/memory_availability_test.py +++ b/roles/openshift_health_checker/test/memory_availability_test.py @@ -17,7 +17,7 @@ def test_is_active(group_names, is_active): task_vars = dict( group_names=group_names, ) - assert MemoryAvailability.is_active(task_vars=task_vars) == is_active + assert MemoryAvailability(None, task_vars).is_active() == is_active @pytest.mark.parametrize('group_names,configured_min,ansible_memtotal_mb', [ @@ -59,8 +59,7 @@ def test_succeeds_with_recommended_memory(group_names, configured_min, ansible_m ansible_memtotal_mb=ansible_memtotal_mb, ) - check = MemoryAvailability(execute_module=fake_execute_module) - result = check.run(tmp=None, task_vars=task_vars) + result = MemoryAvailability(fake_execute_module, task_vars).run() assert not result.get('failed', False) @@ -117,8 +116,7 @@ def test_fails_with_insufficient_memory(group_names, configured_min, ansible_mem ansible_memtotal_mb=ansible_memtotal_mb, ) - check = MemoryAvailability(execute_module=fake_execute_module) - result = check.run(tmp=None, task_vars=task_vars) + result = MemoryAvailability(fake_execute_module, task_vars).run() assert result.get('failed', False) for word in 'below recommended'.split() + extra_words: diff --git a/roles/openshift_health_checker/test/mixins_test.py b/roles/openshift_health_checker/test/mixins_test.py index 2d83e207d..b1a41ca3c 100644 --- a/roles/openshift_health_checker/test/mixins_test.py +++ b/roles/openshift_health_checker/test/mixins_test.py @@ -14,10 +14,10 @@ class NotContainerizedCheck(NotContainerizedMixin, OpenShiftCheck): (dict(openshift=dict(common=dict(is_containerized=True))), False), ]) def test_is_active(task_vars, expected): - assert NotContainerizedCheck.is_active(task_vars) == expected + assert NotContainerizedCheck(None, task_vars).is_active() == expected def test_is_active_missing_task_vars(): with pytest.raises(OpenShiftCheckException) as excinfo: - NotContainerizedCheck.is_active(task_vars={}) + NotContainerizedCheck().is_active() assert 'is_containerized' in str(excinfo.value) diff --git a/roles/openshift_health_checker/test/openshift_check_test.py b/roles/openshift_health_checker/test/openshift_check_test.py index e3153979c..43aa875f4 100644 --- a/roles/openshift_health_checker/test/openshift_check_test.py +++ b/roles/openshift_health_checker/test/openshift_check_test.py @@ -1,7 +1,7 @@ import pytest from openshift_checks import OpenShiftCheck, OpenShiftCheckException -from openshift_checks import load_checks, get_var +from openshift_checks import load_checks # Fixtures @@ -28,34 +28,23 @@ def test_OpenShiftCheck_init(): name = "test_check" run = NotImplemented - # initialization requires at least one argument (apart from self) - with pytest.raises(TypeError) as excinfo: - TestCheck() + # execute_module required at init if it will be used + with pytest.raises(RuntimeError) as excinfo: + TestCheck().execute_module("foo") assert 'execute_module' in str(excinfo.value) - assert 'module_executor' in str(excinfo.value) execute_module = object() # initialize with positional argument check = TestCheck(execute_module) - # new recommended name - assert check.execute_module == execute_module - # deprecated attribute name - assert check.module_executor == execute_module + assert check._execute_module == execute_module - # initialize with keyword argument, recommended name + # initialize with keyword argument check = TestCheck(execute_module=execute_module) - # new recommended name - assert check.execute_module == execute_module - # deprecated attribute name - assert check.module_executor == execute_module + assert check._execute_module == execute_module - # initialize with keyword argument, deprecated name - check = TestCheck(module_executor=execute_module) - # new recommended name - assert check.execute_module == execute_module - # deprecated attribute name - assert check.module_executor == execute_module + assert check.task_vars == {} + assert check.tmp is None def test_subclasses(): @@ -81,19 +70,27 @@ def test_load_checks(): assert modules +def dummy_check(task_vars): + class TestCheck(OpenShiftCheck): + name = "dummy" + run = NotImplemented + + return TestCheck(task_vars=task_vars) + + @pytest.mark.parametrize("keys,expected", [ (("foo",), 42), (("bar", "baz"), "openshift"), ]) def test_get_var_ok(task_vars, keys, expected): - assert get_var(task_vars, *keys) == expected + assert dummy_check(task_vars).get_var(*keys) == expected def test_get_var_error(task_vars, missing_keys): with pytest.raises(OpenShiftCheckException): - get_var(task_vars, *missing_keys) + dummy_check(task_vars).get_var(*missing_keys) def test_get_var_default(task_vars, missing_keys): default = object() - assert get_var(task_vars, *missing_keys, default=default) == default + assert dummy_check(task_vars).get_var(*missing_keys, default=default) == default diff --git a/roles/openshift_health_checker/test/ovs_version_test.py b/roles/openshift_health_checker/test/ovs_version_test.py index 6494e1c06..b6acef5a6 100644 --- a/roles/openshift_health_checker/test/ovs_version_test.py +++ b/roles/openshift_health_checker/test/ovs_version_test.py @@ -4,7 +4,7 @@ from openshift_checks.ovs_version import OvsVersion, OpenShiftCheckException def test_openshift_version_not_supported(): - def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): + def execute_module(*_): return {} openshift_release = '111.7.0' @@ -16,15 +16,14 @@ def test_openshift_version_not_supported(): openshift_deployment_type='origin', ) - check = OvsVersion(execute_module=execute_module) with pytest.raises(OpenShiftCheckException) as excinfo: - check.run(tmp=None, task_vars=task_vars) + OvsVersion(execute_module, task_vars).run() assert "no recommended version of Open vSwitch" in str(excinfo.value) def test_invalid_openshift_release_format(): - def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): + def execute_module(*_): return {} task_vars = dict( @@ -33,9 +32,8 @@ def test_invalid_openshift_release_format(): openshift_deployment_type='origin', ) - check = OvsVersion(execute_module=execute_module) with pytest.raises(OpenShiftCheckException) as excinfo: - check.run(tmp=None, task_vars=task_vars) + OvsVersion(execute_module, task_vars).run() assert "invalid version" in str(excinfo.value) @@ -54,7 +52,7 @@ def test_ovs_package_version(openshift_release, expected_ovs_version): ) return_value = object() - def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): + def execute_module(module_name=None, module_args=None, *_): assert module_name == 'rpm_version' assert "package_list" in module_args @@ -64,8 +62,7 @@ def test_ovs_package_version(openshift_release, expected_ovs_version): return return_value - check = OvsVersion(execute_module=execute_module) - result = check.run(tmp=None, task_vars=task_vars) + result = OvsVersion(execute_module, task_vars).run() assert result is return_value @@ -86,4 +83,4 @@ def test_ovs_version_skip_when_not_master_nor_node(group_names, is_containerized group_names=group_names, openshift=dict(common=dict(is_containerized=is_containerized)), ) - assert OvsVersion.is_active(task_vars=task_vars) == is_active + assert OvsVersion(None, task_vars).is_active() == is_active diff --git a/roles/openshift_health_checker/test/package_availability_test.py b/roles/openshift_health_checker/test/package_availability_test.py index f7e916a46..1fe648b75 100644 --- a/roles/openshift_health_checker/test/package_availability_test.py +++ b/roles/openshift_health_checker/test/package_availability_test.py @@ -14,7 +14,7 @@ def test_is_active(pkg_mgr, is_containerized, is_active): ansible_pkg_mgr=pkg_mgr, openshift=dict(common=dict(is_containerized=is_containerized)), ) - assert PackageAvailability.is_active(task_vars=task_vars) == is_active + assert PackageAvailability(None, task_vars).is_active() == is_active @pytest.mark.parametrize('task_vars,must_have_packages,must_not_have_packages', [ @@ -51,13 +51,12 @@ def test_is_active(pkg_mgr, is_containerized, is_active): def test_package_availability(task_vars, must_have_packages, must_not_have_packages): return_value = object() - def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): + def execute_module(module_name=None, module_args=None, *_): assert module_name == 'check_yum_update' assert 'packages' in module_args assert set(module_args['packages']).issuperset(must_have_packages) assert not set(module_args['packages']).intersection(must_not_have_packages) return return_value - check = PackageAvailability(execute_module=execute_module) - result = check.run(tmp=None, task_vars=task_vars) + result = PackageAvailability(execute_module, task_vars).run() assert result is return_value diff --git a/roles/openshift_health_checker/test/package_update_test.py b/roles/openshift_health_checker/test/package_update_test.py index 5e000cff5..06489b0d7 100644 --- a/roles/openshift_health_checker/test/package_update_test.py +++ b/roles/openshift_health_checker/test/package_update_test.py @@ -4,13 +4,12 @@ from openshift_checks.package_update import PackageUpdate def test_package_update(): return_value = object() - def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): + def execute_module(module_name=None, module_args=None, *_): assert module_name == 'check_yum_update' assert 'packages' in module_args # empty list of packages means "generic check if 'yum update' will work" assert module_args['packages'] == [] return return_value - check = PackageUpdate(execute_module=execute_module) - result = check.run(tmp=None, task_vars=None) + result = PackageUpdate(execute_module).run() assert result is return_value diff --git a/roles/openshift_health_checker/test/package_version_test.py b/roles/openshift_health_checker/test/package_version_test.py index 91eace512..1ddb9cecb 100644 --- a/roles/openshift_health_checker/test/package_version_test.py +++ b/roles/openshift_health_checker/test/package_version_test.py @@ -8,7 +8,7 @@ from openshift_checks.package_version import PackageVersion, OpenShiftCheckExcep ('0.0.0', ["no recommended version of Docker"]), ]) def test_openshift_version_not_supported(openshift_release, extra_words): - def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): + def execute_module(*_): return {} task_vars = dict( @@ -18,16 +18,16 @@ def test_openshift_version_not_supported(openshift_release, extra_words): openshift_deployment_type='origin', ) - check = PackageVersion(execute_module=execute_module) + check = PackageVersion(execute_module, task_vars) with pytest.raises(OpenShiftCheckException) as excinfo: - check.run(tmp=None, task_vars=task_vars) + check.run() for word in extra_words: assert word in str(excinfo.value) def test_invalid_openshift_release_format(): - def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): + def execute_module(*_): return {} task_vars = dict( @@ -36,9 +36,9 @@ def test_invalid_openshift_release_format(): openshift_deployment_type='origin', ) - check = PackageVersion(execute_module=execute_module) + check = PackageVersion(execute_module, task_vars) with pytest.raises(OpenShiftCheckException) as excinfo: - check.run(tmp=None, task_vars=task_vars) + check.run() assert "invalid version" in str(excinfo.value) @@ -57,7 +57,7 @@ def test_package_version(openshift_release): ) return_value = object() - def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): + def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None, *_): assert module_name == 'aos_version' assert "package_list" in module_args @@ -67,38 +67,8 @@ def test_package_version(openshift_release): return return_value - check = PackageVersion(execute_module=execute_module) - result = check.run(tmp=None, task_vars=task_vars) - assert result is return_value - - -@pytest.mark.parametrize('deployment_type,openshift_release,expected_ovs_version', [ - ("openshift-enterprise", "3.5", "2.6"), - ("origin", "3.6", "2.6"), - ("openshift-enterprise", "3.4", "2.4"), - ("origin", "3.3", "2.4"), -]) -def test_ovs_package_version(deployment_type, openshift_release, expected_ovs_version): - task_vars = dict( - openshift=dict(common=dict(service_type='origin')), - openshift_release=openshift_release, - openshift_image_tag='v' + openshift_release, - openshift_deployment_type=deployment_type, - ) - return_value = object() - - def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): - assert module_name == 'aos_version' - assert "package_list" in module_args - - for pkg in module_args["package_list"]: - if pkg["name"] == "openvswitch": - assert pkg["version"] == expected_ovs_version - - return return_value - - check = PackageVersion(execute_module=execute_module) - result = check.run(tmp=None, task_vars=task_vars) + check = PackageVersion(execute_module, task_vars) + result = check.run() assert result is return_value @@ -119,7 +89,7 @@ def test_docker_package_version(deployment_type, openshift_release, expected_doc ) return_value = object() - def execute_module(module_name=None, module_args=None, tmp=None, task_vars=None): + def execute_module(module_name=None, module_args=None, *_): assert module_name == 'aos_version' assert "package_list" in module_args @@ -129,8 +99,8 @@ def test_docker_package_version(deployment_type, openshift_release, expected_doc return return_value - check = PackageVersion(execute_module=execute_module) - result = check.run(tmp=None, task_vars=task_vars) + check = PackageVersion(execute_module, task_vars) + result = check.run() assert result is return_value @@ -151,4 +121,4 @@ def test_package_version_skip_when_not_master_nor_node(group_names, is_container group_names=group_names, openshift=dict(common=dict(is_containerized=is_containerized)), ) - assert PackageVersion.is_active(task_vars=task_vars) == is_active + assert PackageVersion(None, task_vars).is_active() == is_active diff --git a/roles/openshift_health_checker/test/search_journalctl_test.py b/roles/openshift_health_checker/test/search_journalctl_test.py new file mode 100644 index 000000000..724928aa1 --- /dev/null +++ b/roles/openshift_health_checker/test/search_journalctl_test.py @@ -0,0 +1,157 @@ +import pytest +import search_journalctl + + +def canned_search_journalctl(get_log_output=None): + """Create a search_journalctl object with canned get_log_output method""" + module = search_journalctl + if get_log_output: + module.get_log_output = get_log_output + return module + + +DEFAULT_TIMESTAMP = 1496341364 + + +def get_timestamp(modifier=0): + return DEFAULT_TIMESTAMP + modifier + + +def get_timestamp_microseconds(modifier=0): + return get_timestamp(modifier) * 1000000 + + +def create_test_log_object(stamp, msg): + return '{{"__REALTIME_TIMESTAMP": "{}", "MESSAGE": "{}"}}'.format(stamp, msg) + + +@pytest.mark.parametrize('name,matchers,log_input,expected_matches,expected_errors', [ + ( + 'test with valid params', + [ + { + "start_regexp": r"Sample Logs Beginning", + "regexp": r"test log message", + "unit": "test", + }, + ], + [ + create_test_log_object(get_timestamp_microseconds(), "test log message"), + create_test_log_object(get_timestamp_microseconds(), "Sample Logs Beginning"), + ], + ["test log message"], + [], + ), + ( + 'test with invalid json in log input', + [ + { + "start_regexp": r"Sample Logs Beginning", + "regexp": r"test log message", + "unit": "test-unit", + }, + ], + [ + '{__REALTIME_TIMESTAMP: ' + str(get_timestamp_microseconds()) + ', "MESSAGE": "test log message"}', + ], + [], + [ + ["invalid json", "test-unit", "test log message"], + ], + ), + ( + 'test with invalid regexp', + [ + { + "start_regexp": r"Sample Logs Beginning", + "regexp": r"test [ log message", + "unit": "test", + }, + ], + [ + create_test_log_object(get_timestamp_microseconds(), "test log message"), + create_test_log_object(get_timestamp_microseconds(), "sample log message"), + create_test_log_object(get_timestamp_microseconds(), "fake log message"), + create_test_log_object(get_timestamp_microseconds(), "dummy log message"), + create_test_log_object(get_timestamp_microseconds(), "Sample Logs Beginning"), + ], + [], + [ + ["invalid regular expression"], + ], + ), +], ids=lambda argval: argval[0]) +def test_get_log_matches(name, matchers, log_input, expected_matches, expected_errors): + def get_log_output(matcher): + return log_input + + module = canned_search_journalctl(get_log_output) + matched_regexp, errors = module.get_log_matches(matchers, 500, 60 * 60) + + assert set(matched_regexp) == set(expected_matches) + assert len(expected_errors) == len(errors) + + for idx, partial_err_set in enumerate(expected_errors): + for partial_err_msg in partial_err_set: + assert partial_err_msg in errors[idx] + + +@pytest.mark.parametrize('name,matcher,log_count_lim,stamp_lim_seconds,log_input,expected_match', [ + ( + 'test with matching log message, but out of bounds of log_count_lim', + { + "start_regexp": r"Sample Logs Beginning", + "regexp": r"dummy log message", + "unit": "test", + }, + 3, + get_timestamp(-100 * 60 * 60), + [ + create_test_log_object(get_timestamp_microseconds(), "test log message"), + create_test_log_object(get_timestamp_microseconds(), "sample log message"), + create_test_log_object(get_timestamp_microseconds(), "fake log message"), + create_test_log_object(get_timestamp_microseconds(), "dummy log message"), + create_test_log_object(get_timestamp_microseconds(), "Sample Logs Beginning"), + ], + None, + ), + ( + 'test with matching log message, but with timestamp too old', + { + "start_regexp": r"Sample Logs Beginning", + "regexp": r"dummy log message", + "unit": "test", + }, + 100, + get_timestamp(-10), + [ + create_test_log_object(get_timestamp_microseconds(), "test log message"), + create_test_log_object(get_timestamp_microseconds(), "sample log message"), + create_test_log_object(get_timestamp_microseconds(), "fake log message"), + create_test_log_object(get_timestamp_microseconds(-1000), "dummy log message"), + create_test_log_object(get_timestamp_microseconds(-1000), "Sample Logs Beginning"), + ], + None, + ), + ( + 'test with matching log message, and timestamp within time limit', + { + "start_regexp": r"Sample Logs Beginning", + "regexp": r"dummy log message", + "unit": "test", + }, + 100, + get_timestamp(-1010), + [ + create_test_log_object(get_timestamp_microseconds(), "test log message"), + create_test_log_object(get_timestamp_microseconds(), "sample log message"), + create_test_log_object(get_timestamp_microseconds(), "fake log message"), + create_test_log_object(get_timestamp_microseconds(-1000), "dummy log message"), + create_test_log_object(get_timestamp_microseconds(-1000), "Sample Logs Beginning"), + ], + create_test_log_object(get_timestamp_microseconds(-1000), "dummy log message"), + ), +], ids=lambda argval: argval[0]) +def test_find_matches_skips_logs(name, matcher, log_count_lim, stamp_lim_seconds, log_input, expected_match): + match = search_journalctl.find_matches(log_input, matcher, log_count_lim, stamp_lim_seconds) + assert match == expected_match diff --git a/roles/openshift_hosted/defaults/main.yml b/roles/openshift_hosted/defaults/main.yml index 089054e2f..0391e5602 100644 --- a/roles/openshift_hosted/defaults/main.yml +++ b/roles/openshift_hosted/defaults/main.yml @@ -29,7 +29,7 @@ openshift_hosted_routers: openshift_hosted_router_certificate: {} openshift_hosted_registry_cert_expire_days: 730 -openshift_hosted_router_create_certificate: False +openshift_hosted_router_create_certificate: True os_firewall_allow: - service: Docker Registry Port diff --git a/roles/openshift_hosted/tasks/registry/storage/glusterfs.yml b/roles/openshift_hosted/tasks/registry/storage/glusterfs.yml index c504bfb80..c2954fde1 100644 --- a/roles/openshift_hosted/tasks/registry/storage/glusterfs.yml +++ b/roles/openshift_hosted/tasks/registry/storage/glusterfs.yml @@ -35,7 +35,7 @@ mount: state: mounted fstype: glusterfs - src: "{% if 'glusterfs_registry' in groups %}{{ groups.glusterfs_registry[0] }}{% else %}{{ groups.glusterfs[0] }}{% endif %}:/{{ openshift.hosted.registry.storage.glusterfs.path }}" + src: "{% if 'glusterfs_registry' in groups %}{% set node = groups.glusterfs_registry[0] %}{% else %}{% set node = groups.glusterfs[0] %}{% endif %}{% if 'glusterfs_hostname' in hostvars[node] %}{{ hostvars[node].glusterfs_hostname }}{% elif 'openshift' in hostvars[node] %}{{ hostvars[node].openshift.node.nodename }}{% else %}{{ node }}{% endif %}:/{{ openshift.hosted.registry.storage.glusterfs.path }}" name: "{{ mktemp.stdout }}" - name: Set registry volume permissions diff --git a/roles/openshift_hosted/tasks/router/router.yml b/roles/openshift_hosted/tasks/router/router.yml index c60b67862..dd485a64a 100644 --- a/roles/openshift_hosted/tasks/router/router.yml +++ b/roles/openshift_hosted/tasks/router/router.yml @@ -23,8 +23,8 @@ signer_key: "{{ openshift_master_config_dir }}/ca.key" signer_serial: "{{ openshift_master_config_dir }}/ca.serial.txt" hostnames: - - "{{ openshift_master_default_subdomain }}" - - "*.{{ openshift_master_default_subdomain }}" + - "{{ openshift_master_default_subdomain | default('router.default.svc.cluster.local') }}" + - "*.{{ openshift_master_default_subdomain | default('router.default.svc.cluster.local') }}" cert: "{{ ('/etc/origin/master/' ~ (item.certificate.certfile | basename)) if 'certfile' in item.certificate else ((openshift_master_config_dir) ~ '/openshift-router.crt') }}" key: "{{ ('/etc/origin/master/' ~ (item.certificate.keyfile | basename)) if 'keyfile' in item.certificate else ((openshift_master_config_dir) ~ '/openshift-router.key') }}" with_items: "{{ openshift_hosted_routers }}" @@ -37,7 +37,7 @@ cafile: "{{ openshift_master_config_dir ~ '/ca.crt' }}" # End Block - when: openshift_hosted_router_create_certificate | bool + when: ( openshift_hosted_router_create_certificate | bool ) and openshift_hosted_router_certificate == {} - name: Get the certificate contents for router copy: diff --git a/roles/openshift_hosted/templates/registry_config.j2 b/roles/openshift_hosted/templates/registry_config.j2 index dc8a9f089..fc9272679 100644 --- a/roles/openshift_hosted/templates/registry_config.j2 +++ b/roles/openshift_hosted/templates/registry_config.j2 @@ -21,7 +21,10 @@ storage: regionendpoint: {{ openshift_hosted_registry_storage_s3_regionendpoint }} {% endif %} bucket: {{ openshift_hosted_registry_storage_s3_bucket }} - encrypt: false + encrypt: {{ openshift_hosted_registry_storage_s3_encrypt | default(false) }} +{% if openshift_hosted_registry_storage_s3_kmskeyid is defined %} + keyid: {{ openshift_hosted_registry_storage_s3_kmskeyid }} +{% endif %} secure: true v4auth: true rootdirectory: {{ openshift_hosted_registry_storage_s3_rootdirectory | default('/registry') }} diff --git a/roles/openshift_loadbalancer/README.md b/roles/openshift_loadbalancer/README.md index bea4c509b..330895f20 100644 --- a/roles/openshift_loadbalancer/README.md +++ b/roles/openshift_loadbalancer/README.md @@ -25,6 +25,7 @@ From this role: | openshift_loadbalancer_default_maxconn | 20000 | Maximum per-process number of concurrent connections. | | openshift_loadbalancer_frontends | none | List of frontends. See example below. | | openshift_loadbalancer_backends | none | List of backends. See example below. | +| openshift_image_tag | none | Image tag for containerized haproxy image. | Dependencies ------------ @@ -64,6 +65,7 @@ Example Playbook - name: master3 address: "192.168.122.223:8443" opts: check + openshift_image_tag: v3.6.153 ``` License diff --git a/roles/openshift_logging/tasks/install_logging.yaml b/roles/openshift_logging/tasks/install_logging.yaml index 5c5bbf84c..464e8594f 100644 --- a/roles/openshift_logging/tasks/install_logging.yaml +++ b/roles/openshift_logging/tasks/install_logging.yaml @@ -60,6 +60,9 @@ - set_fact: openshift_logging_es_pvc_prefix="logging-es" when: openshift_logging_es_pvc_prefix == "" +- set_fact: + elasticsearch_storage_type: "{{ openshift_logging_elasticsearch_storage_type | default('pvc' if ( openshift_logging_es_pvc_dynamic | bool or openshift_hosted_logging_storage_kind | default('') == 'nfs' or openshift_logging_es_pvc_size | length > 0) else 'emptydir') }}" + # We don't allow scaling down of ES nodes currently - include_role: name: openshift_logging_elasticsearch @@ -70,7 +73,7 @@ openshift_logging_elasticsearch_pvc_name: "{{ openshift_logging_es_pvc_prefix ~ '-' ~ item.2 if item.1 is none else item.1 }}" openshift_logging_elasticsearch_replica_count: "{{ openshift_logging_es_cluster_size | int }}" - openshift_logging_elasticsearch_storage_type: "{{ 'pvc' if ( openshift_logging_es_pvc_dynamic | bool or openshift_hosted_logging_storage_kind | default('') == 'nfs') else 'emptydir' }}" + openshift_logging_elasticsearch_storage_type: "{{ elasticsearch_storage_type }}" openshift_logging_elasticsearch_pvc_size: "{{ openshift_logging_es_pvc_size }}" openshift_logging_elasticsearch_pvc_dynamic: "{{ openshift_logging_es_pvc_dynamic }}" openshift_logging_elasticsearch_pvc_pv_selector: "{{ openshift_logging_es_pv_selector }}" @@ -91,7 +94,7 @@ openshift_logging_elasticsearch_pvc_name: "{{ openshift_logging_es_pvc_prefix }}-{{ item | int + openshift_logging_facts.elasticsearch.deploymentconfigs | count - 1 }}" openshift_logging_elasticsearch_replica_count: "{{ openshift_logging_es_cluster_size | int }}" - openshift_logging_elasticsearch_storage_type: "{{ 'pvc' if ( openshift_logging_es_pvc_dynamic | bool or openshift_hosted_logging_storage_kind | default('') == 'nfs') else 'emptydir' }}" + openshift_logging_elasticsearch_storage_type: "{{ elasticsearch_storage_type }}" openshift_logging_elasticsearch_pvc_size: "{{ openshift_logging_es_pvc_size }}" openshift_logging_elasticsearch_pvc_dynamic: "{{ openshift_logging_es_pvc_dynamic }}" openshift_logging_elasticsearch_pvc_pv_selector: "{{ openshift_logging_es_pv_selector }}" @@ -110,6 +113,11 @@ - set_fact: openshift_logging_es_ops_pvc_prefix="logging-es-ops" when: openshift_logging_es_ops_pvc_prefix == "" +- set_fact: + elasticsearch_storage_type: "{{ openshift_logging_elasticsearch_storage_type | default('pvc' if ( openshift_logging_es_ops_pvc_dynamic | bool or openshift_hosted_logging_storage_kind | default('') == 'nfs' or openshift_logging_es_ops_pvc_size | length > 0) else 'emptydir') }}" + when: + - openshift_logging_use_ops | bool + - include_role: name: openshift_logging_elasticsearch vars: @@ -120,7 +128,7 @@ openshift_logging_elasticsearch_ops_deployment: true openshift_logging_elasticsearch_replica_count: "{{ openshift_logging_es_ops_cluster_size | int }}" - openshift_logging_elasticsearch_storage_type: "{{ 'pvc' if ( openshift_logging_es_ops_pvc_dynamic | bool or openshift_hosted_logging_storage_kind | default('') == 'nfs') else 'emptydir' }}" + openshift_logging_elasticsearch_storage_type: "{{ elasticsearch_storage_type }}" openshift_logging_elasticsearch_pvc_size: "{{ openshift_logging_es_ops_pvc_size }}" openshift_logging_elasticsearch_pvc_dynamic: "{{ openshift_logging_es_ops_pvc_dynamic }}" openshift_logging_elasticsearch_pvc_pv_selector: "{{ openshift_logging_es_ops_pv_selector }}" @@ -149,7 +157,7 @@ openshift_logging_elasticsearch_ops_deployment: true openshift_logging_elasticsearch_replica_count: "{{ openshift_logging_es_ops_cluster_size | int }}" - openshift_logging_elasticsearch_storage_type: "{{ 'pvc' if ( openshift_logging_es_ops_pvc_dynamic | bool or openshift_hosted_logging_storage_kind | default('') == 'nfs') else 'emptydir' }}" + openshift_logging_elasticsearch_storage_type: "{{ elasticsearch_storage_type }}" openshift_logging_elasticsearch_pvc_size: "{{ openshift_logging_es_ops_pvc_size }}" openshift_logging_elasticsearch_pvc_dynamic: "{{ openshift_logging_es_ops_pvc_dynamic }}" openshift_logging_elasticsearch_pvc_pv_selector: "{{ openshift_logging_es_ops_pv_selector }}" diff --git a/roles/openshift_logging_elasticsearch/tasks/main.yaml b/roles/openshift_logging_elasticsearch/tasks/main.yaml index 68726aa78..532f4a85d 100644 --- a/roles/openshift_logging_elasticsearch/tasks/main.yaml +++ b/roles/openshift_logging_elasticsearch/tasks/main.yaml @@ -206,7 +206,7 @@ storage_class_name: "{{ openshift_logging_elasticsearch_pvc_storage_class_name | default('', true) }}" when: - openshift_logging_elasticsearch_storage_type == "pvc" - - not openshift_logging_elasticsearch_pvc_dynamic + - not openshift_logging_elasticsearch_pvc_dynamic | bool # Storageclasses are used by default if configured - name: Creating ES storage template - dynamic @@ -220,7 +220,7 @@ pv_selector: "{{ openshift_logging_elasticsearch_pvc_pv_selector }}" when: - openshift_logging_elasticsearch_storage_type == "pvc" - - openshift_logging_elasticsearch_pvc_dynamic + - openshift_logging_elasticsearch_pvc_dynamic | bool - name: Set ES storage oc_obj: diff --git a/roles/openshift_logging_kibana/defaults/main.yml b/roles/openshift_logging_kibana/defaults/main.yml index 23337bcd2..b2556fd71 100644 --- a/roles/openshift_logging_kibana/defaults/main.yml +++ b/roles/openshift_logging_kibana/defaults/main.yml @@ -11,7 +11,7 @@ openshift_logging_kibana_nodeselector: "" openshift_logging_kibana_cpu_limit: null openshift_logging_kibana_memory_limit: 736Mi -openshift_logging_kibana_hostname: "kibana.router.default.svc.cluster.local" +openshift_logging_kibana_hostname: "{{ openshift_hosted_logging_hostname | default('kibana.' ~ (openshift_master_default_subdomain | default('router.default.svc.cluster.local', true))) }}" openshift_logging_kibana_es_host: "logging-es" openshift_logging_kibana_es_port: 9200 diff --git a/roles/openshift_master/tasks/main.yml b/roles/openshift_master/tasks/main.yml index 9b7125240..0c4ee319c 100644 --- a/roles/openshift_master/tasks/main.yml +++ b/roles/openshift_master/tasks/main.yml @@ -140,6 +140,12 @@ - set_fact: openshift_push_via_dns: "{{ (openshift_use_dnsmasq | default(true) and openshift.common.version_gte_3_6) or (already_set.stdout | match('OPENSHIFT_DEFAULT_REGISTRY=docker-registry.default.svc:5000')) }}" +- name: Set fact of all etcd host IPs + openshift_facts: + role: common + local_facts: + no_proxy_etcd_host_ips: "{{ openshift_no_proxy_etcd_host_ips }}" + - name: Install the systemd units include: systemd_units.yml @@ -200,6 +206,10 @@ delay: 60 notify: Verify API Server +- name: Dump logs from master service if it failed + command: journalctl --no-pager -n 100 -u {{ openshift.common.service_type }}-master + when: start_result | failed + - name: Stop and disable non-HA master when running HA systemd: name: "{{ openshift.common.service_type }}-master" @@ -233,6 +243,10 @@ retries: 1 delay: 60 +- name: Dump logs from master-api if it failed + command: journalctl --no-pager -n 100 -u {{ openshift.common.service_type }}-master-api + when: start_result | failed + - set_fact: master_api_service_status_changed: "{{ start_result | changed }}" when: openshift_master_ha | bool and openshift.master.cluster_method == 'native' and inventory_hostname == openshift_master_hosts[0] @@ -252,6 +266,10 @@ retries: 1 delay: 60 +- name: Dump logs from master-api if it failed + command: journalctl --no-pager -n 100 -u {{ openshift.common.service_type }}-master-api + when: start_result | failed + - set_fact: master_api_service_status_changed: "{{ start_result | changed }}" when: openshift_master_ha | bool and openshift.master.cluster_method == 'native' and inventory_hostname != openshift_master_hosts[0] @@ -288,6 +306,10 @@ retries: 1 delay: 60 +- name: Dump logs from master-controllers if it failed + command: journalctl --no-pager -n 100 -u {{ openshift.common.service_type }}-master-controllers + when: start_result | failed + - name: Wait for master controller service to start on first master pause: seconds: 15 @@ -304,6 +326,10 @@ retries: 1 delay: 60 +- name: Dump logs from master-controllers if it failed + command: journalctl --no-pager -n 100 -u {{ openshift.common.service_type }}-master-controllers + when: start_result | failed + - set_fact: master_controllers_service_status_changed: "{{ start_result | changed }}" when: openshift_master_ha | bool and openshift.master.cluster_method == 'native' diff --git a/roles/openshift_metrics/tasks/generate_hawkular_certificates.yaml b/roles/openshift_metrics/tasks/generate_hawkular_certificates.yaml index 8d7ee00ed..31129a6ac 100644 --- a/roles/openshift_metrics/tasks/generate_hawkular_certificates.yaml +++ b/roles/openshift_metrics/tasks/generate_hawkular_certificates.yaml @@ -26,7 +26,6 @@ - name: generate htpasswd file for hawkular metrics local_action: htpasswd path="{{ local_tmp.stdout }}/hawkular-metrics.htpasswd" name=hawkular password="{{ hawkular_metrics_pwd.content | b64decode }}" - no_log: true become: false - name: copy local generated passwords to target diff --git a/roles/openshift_metrics/tasks/generate_rolebindings.yaml b/roles/openshift_metrics/tasks/generate_rolebindings.yaml index e050c8eb2..1304ab8b5 100644 --- a/roles/openshift_metrics/tasks/generate_rolebindings.yaml +++ b/roles/openshift_metrics/tasks/generate_rolebindings.yaml @@ -13,3 +13,27 @@ - kind: ServiceAccount name: hawkular changed_when: no + +- name: generate hawkular-metrics cluster role binding for the hawkular service account + template: + src: rolebinding.j2 + dest: "{{ mktemp.stdout }}/templates/hawkular-cluster-rolebinding.yaml" + vars: + cluster: True + obj_name: hawkular-namespace-watcher + labels: + metrics-infra: hawkular + roleRef: + kind: ClusterRole + name: hawkular-metrics + subjects: + - kind: ServiceAccount + name: hawkular + namespace: "{{openshift_metrics_project}}" + changed_when: no + +- name: generate the hawkular cluster role + template: + src: hawkular_metrics_role.j2 + dest: "{{ mktemp.stdout }}/templates/hawkular-cluster-role.yaml" + changed_when: no diff --git a/roles/openshift_metrics/tasks/uninstall_metrics.yaml b/roles/openshift_metrics/tasks/uninstall_metrics.yaml index 9a5d52eb6..403b1252c 100644 --- a/roles/openshift_metrics/tasks/uninstall_metrics.yaml +++ b/roles/openshift_metrics/tasks/uninstall_metrics.yaml @@ -6,7 +6,7 @@ command: > {{ openshift.common.client_binary }} -n {{ openshift_metrics_project }} --config={{ mktemp.stdout }}/admin.kubeconfig delete --ignore-not-found --selector=metrics-infra - all,sa,secrets,templates,routes,pvc,rolebindings,clusterrolebindings + all,sa,secrets,templates,routes,pvc,rolebindings,clusterrolebindings,clusterrole register: delete_metrics changed_when: delete_metrics.stdout != 'No resources found' @@ -16,4 +16,5 @@ delete --ignore-not-found rolebinding/hawkular-view clusterrolebinding/heapster-cluster-reader + clusterrolebinding/hawkular-metrics changed_when: delete_metrics.stdout != 'No resources found' diff --git a/roles/openshift_metrics/templates/hawkular_metrics_role.j2 b/roles/openshift_metrics/templates/hawkular_metrics_role.j2 new file mode 100644 index 000000000..6c9dbf5d6 --- /dev/null +++ b/roles/openshift_metrics/templates/hawkular_metrics_role.j2 @@ -0,0 +1,15 @@ +apiVersion: v1 +kind: ClusterRole +metadata: + name: hawkular-metrics + labels: + metrics-infra: hawkular-metrics +rules: +- apiGroups: + - "" + resources: + - namespaces + verbs: + - list + - get + - watch diff --git a/roles/openshift_node/handlers/main.yml b/roles/openshift_node/handlers/main.yml index a6bd12d4e..6b38da7f8 100644 --- a/roles/openshift_node/handlers/main.yml +++ b/roles/openshift_node/handlers/main.yml @@ -4,9 +4,14 @@ name: openvswitch state: restarted when: (not skip_node_svc_handlers | default(False) | bool) and not (ovs_service_status_changed | default(false) | bool) and openshift.common.use_openshift_sdn | bool + register: l_openshift_node_stop_openvswitch_result + until: not l_openshift_node_stop_openvswitch_result | failed + retries: 3 + delay: 30 notify: - restart openvswitch pause + - name: restart openvswitch pause pause: seconds=15 when: (not skip_node_svc_handlers | default(False) | bool) and openshift.common.is_containerized | bool @@ -15,7 +20,13 @@ systemd: name: "{{ openshift.common.service_type }}-node" state: restarted - when: (not skip_node_svc_handlers | default(False) | bool) and not (node_service_status_changed | default(false) | bool) + register: l_openshift_node_restart_node_result + until: not l_openshift_node_restart_node_result | failed + retries: 3 + delay: 30 + when: + - (not skip_node_svc_handlers | default(False) | bool) + - not (node_service_status_changed | default(false) | bool) - name: reload sysctl.conf command: /sbin/sysctl -p diff --git a/roles/openshift_node/tasks/main.yml b/roles/openshift_node/tasks/main.yml index 573051504..0133533fc 100644 --- a/roles/openshift_node/tasks/main.yml +++ b/roles/openshift_node/tasks/main.yml @@ -118,8 +118,12 @@ name: openvswitch.service enabled: yes state: started + daemon_reload: yes when: openshift.common.is_containerized | bool and openshift.common.use_openshift_sdn | bool register: ovs_start_result + until: not ovs_start_result | failed + retries: 3 + delay: 30 - set_fact: ovs_service_status_changed: "{{ ovs_start_result | changed }}" @@ -212,15 +216,27 @@ state: started when: openshift.common.is_containerized | bool + - name: Start and enable node systemd: name: "{{ openshift.common.service_type }}-node" enabled: yes state: started + daemon_reload: yes register: node_start_result until: not node_start_result | failed retries: 1 delay: 30 + ignore_errors: true + +- name: Dump logs from node service if it failed + command: journalctl --no-pager -n 100 -u {{ openshift.common.service_type }}-node + when: node_start_result | failed + +- name: Abort if node failed to start + fail: + msg: Node failed to start please inspect the logs and try again + when: node_start_result | failed - set_fact: node_service_status_changed: "{{ node_start_result | changed }}" diff --git a/roles/openshift_node/tasks/openvswitch_system_container.yml b/roles/openshift_node/tasks/openvswitch_system_container.yml index 8cfa5a026..c8d653880 100644 --- a/roles/openshift_node/tasks/openvswitch_system_container.yml +++ b/roles/openshift_node/tasks/openvswitch_system_container.yml @@ -10,3 +10,5 @@ name: openvswitch image: "{{ openshift.common.system_images_registry }}/{{ openshift.node.ovs_system_image }}:{{ openshift_image_tag }}" state: latest + values: + - "DOCKER_SERVICE={{ openshift.docker.service_name }}.service" diff --git a/roles/openshift_node/templates/node.service.j2 b/roles/openshift_node/templates/node.service.j2 index d4f0b7762..e12a52c15 100644 --- a/roles/openshift_node/templates/node.service.j2 +++ b/roles/openshift_node/templates/node.service.j2 @@ -24,6 +24,7 @@ WorkingDirectory=/var/lib/origin/ SyslogIdentifier={{ openshift.common.service_type }}-node Restart=always RestartSec=5s +TimeoutStartSec=300 OOMScoreAdjust=-999 [Install] diff --git a/roles/openshift_node_certificates/handlers/main.yml b/roles/openshift_node_certificates/handlers/main.yml index 502f80434..4abe8bcaf 100644 --- a/roles/openshift_node_certificates/handlers/main.yml +++ b/roles/openshift_node_certificates/handlers/main.yml @@ -9,3 +9,7 @@ name: "{{ openshift.docker.service_name }}" state: restarted when: not openshift_certificates_redeploy | default(false) | bool + register: l_docker_restart_docker_in_cert_result + until: not l_docker_restart_docker_in_cert_result | failed + retries: 3 + delay: 30 diff --git a/roles/openshift_node_upgrade/README.md b/roles/openshift_node_upgrade/README.md index 8b388cc6a..4e6229bfb 100644 --- a/roles/openshift_node_upgrade/README.md +++ b/roles/openshift_node_upgrade/README.md @@ -84,6 +84,11 @@ Including an example of how to use your role (for instance, with variables passe command: > {{ hostvars[groups.oo_first_master.0].openshift.common.admin_binary }} drain {{ openshift.node.nodename | lower }} --force --delete-local-data --ignore-daemonsets delegate_to: "{{ groups.oo_first_master.0 }}" + register: l_docker_upgrade_drain_result + until: not l_docker_upgrade_drain_result | failed + retries: 60 + delay: 60 + roles: - openshift_facts diff --git a/roles/openshift_node_upgrade/handlers/main.yml b/roles/openshift_node_upgrade/handlers/main.yml index cb51416d4..110dfe5ce 100644 --- a/roles/openshift_node_upgrade/handlers/main.yml +++ b/roles/openshift_node_upgrade/handlers/main.yml @@ -1,7 +1,13 @@ --- - name: restart openvswitch - systemd: name=openvswitch state=restarted + systemd: + name: openvswitch + state: restarted when: (not skip_node_svc_handlers | default(False) | bool) and not (ovs_service_status_changed | default(false) | bool) and openshift.common.use_openshift_sdn | bool + register: l_openshift_node_upgrade_stop_openvswitch_result + until: not l_openshift_node_upgrade_stop_openvswitch_result | failed + retries: 3 + delay: 30 notify: - restart openvswitch pause @@ -10,5 +16,13 @@ when: (not skip_node_svc_handlers | default(False) | bool) and openshift.common.is_containerized | bool - name: restart node - systemd: name={{ openshift.common.service_type }}-node state=restarted - when: (not skip_node_svc_handlers | default(False) | bool) and not (node_service_status_changed | default(false) | bool) + systemd: + name: "{{ openshift.common.service_type }}-node" + state: restarted + register: l_openshift_node_upgrade_restart_node_result + until: not l_openshift_node_upgrade_restart_node_result | failed + retries: 3 + delay: 30 + when: + - (not skip_node_svc_handlers | default(False) | bool) + - not (node_service_status_changed | default(false) | bool) diff --git a/roles/openshift_node_upgrade/tasks/docker/upgrade.yml b/roles/openshift_node_upgrade/tasks/docker/upgrade.yml index 416cf605a..ebe87d6fd 100644 --- a/roles/openshift_node_upgrade/tasks/docker/upgrade.yml +++ b/roles/openshift_node_upgrade/tasks/docker/upgrade.yml @@ -26,7 +26,13 @@ - debug: var=docker_image_count.stdout when: docker_upgrade_nuke_images is defined and docker_upgrade_nuke_images | bool -- service: name=docker state=stopped +- service: + name: docker + state: stopped + register: l_openshift_node_upgrade_docker_stop_result + until: not l_openshift_node_upgrade_docker_stop_result | failed + retries: 3 + delay: 30 - name: Upgrade Docker package: name=docker{{ '-' + docker_version }} state=present diff --git a/roles/openshift_node_upgrade/tasks/restart.yml b/roles/openshift_node_upgrade/tasks/restart.yml index 6947223af..f228b6e08 100644 --- a/roles/openshift_node_upgrade/tasks/restart.yml +++ b/roles/openshift_node_upgrade/tasks/restart.yml @@ -19,7 +19,7 @@ state: started register: docker_start_result until: not docker_start_result | failed - retries: 1 + retries: 3 delay: 30 - name: Update docker facts diff --git a/roles/openshift_node_upgrade/templates/node.service.j2 b/roles/openshift_node_upgrade/templates/node.service.j2 index d4f0b7762..e12a52c15 100644 --- a/roles/openshift_node_upgrade/templates/node.service.j2 +++ b/roles/openshift_node_upgrade/templates/node.service.j2 @@ -24,6 +24,7 @@ WorkingDirectory=/var/lib/origin/ SyslogIdentifier={{ openshift.common.service_type }}-node Restart=always RestartSec=5s +TimeoutStartSec=300 OOMScoreAdjust=-999 [Install] diff --git a/roles/openshift_repos/defaults/main.yaml b/roles/openshift_repos/defaults/main.yaml index 7c5a14cd7..44f34ea7b 100644 --- a/roles/openshift_repos/defaults/main.yaml +++ b/roles/openshift_repos/defaults/main.yaml @@ -1,2 +1,3 @@ --- openshift_additional_repos: {} +openshift_repos_enable_testing: false diff --git a/roles/openshift_repos/tasks/main.yaml b/roles/openshift_repos/tasks/main.yaml index 8f8550e2d..7458db87e 100644 --- a/roles/openshift_repos/tasks/main.yaml +++ b/roles/openshift_repos/tasks/main.yaml @@ -33,7 +33,7 @@ # "centos-release-openshift-origin" package which configures the repository. # This task matches the file names provided by the package so that they are # not installed twice in different files and remains idempotent. - - name: Configure origin gpg keys if needed + - name: Configure origin repositories and gpg keys if needed copy: src: "{{ item.src }}" dest: "{{ item.dest }}" @@ -49,6 +49,10 @@ - openshift_deployment_type == 'origin' - openshift_enable_origin_repo | default(true) | bool + - name: Enable centos-openshift-origin-testing repository + command: yum-config-manager --enable centos-openshift-origin-testing + when: openshift_repos_enable_testing | bool + - name: Ensure clean repo cache in the event repos have been changed manually debug: msg: "First run of openshift_repos" diff --git a/roles/openshift_service_catalog/files/kubeservicecatalog_roles_bindings.yml b/roles/openshift_service_catalog/files/kubeservicecatalog_roles_bindings.yml index 2e0dcfd97..71e21a269 100644 --- a/roles/openshift_service_catalog/files/kubeservicecatalog_roles_bindings.yml +++ b/roles/openshift_service_catalog/files/kubeservicecatalog_roles_bindings.yml @@ -99,7 +99,6 @@ objects: - "" resources: - secrets - - podpresets verbs: - create - update @@ -137,6 +136,23 @@ objects: - serviceclasses verbs: - create + - delete + - update + - patch + - get + - list + - watch + - apiGroups: + - settings.k8s.io + resources: + - podpresets + verbs: + - create + - update + - delete + - get + - list + - watch - kind: ClusterRoleBinding apiVersion: v1 diff --git a/roles/openshift_service_catalog/files/openshift-ansible-catalog-console.js b/roles/openshift_service_catalog/files/openshift-ansible-catalog-console.js new file mode 100644 index 000000000..1f25cc39f --- /dev/null +++ b/roles/openshift_service_catalog/files/openshift-ansible-catalog-console.js @@ -0,0 +1,2 @@ +window.OPENSHIFT_CONSTANTS.ENABLE_TECH_PREVIEW_FEATURE.service_catalog_landing_page = true; +window.OPENSHIFT_CONSTANTS.ENABLE_TECH_PREVIEW_FEATURE.pod_presets = true; diff --git a/roles/openshift_service_catalog/tasks/install.yml b/roles/openshift_service_catalog/tasks/install.yml index 1342c3d30..686857d94 100644 --- a/roles/openshift_service_catalog/tasks/install.yml +++ b/roles/openshift_service_catalog/tasks/install.yml @@ -23,7 +23,12 @@ oc_project: state: present name: "kube-service-catalog" -# node_selector: "{{ openshift_service_catalog_nodeselector | default(null) }}" + node_selector: "" + +- name: Make kube-service-catalog project network global + command: > + oc adm pod-network make-projects-global kube-service-catalog + when: os_sdn_network_plugin_name | default('') == 'redhat/openshift-ovs-multitenant' - include: generate_certs.yml @@ -61,6 +66,52 @@ template_name: kube-system-service-catalog namespace: kube-system +- oc_obj: + name: edit + kind: clusterrole + state: list + register: edit_yaml + +# only do this if we don't already have the updated role info +- name: Generate apply template for clusterrole/edit + template: + src: sc_role_patching.j2 + dest: "{{ mktemp.stdout }}/edit_sc_patch.yml" + vars: + original_content: "{{ edit_yaml.results.results[0] | to_yaml }}" + when: + - not edit_yaml.results.results[0] | oo_contains_rule(['servicecatalog.k8s.io'], ['instances', 'bindings'], ['create', 'update', 'delete', 'get', 'list', 'watch']) or not edit_yaml.results.results[0] | oo_contains_rule(['settings.k8s.io'], ['podpresets'], ['create', 'update', 'delete', 'get', 'list', 'watch']) + +# only do this if we don't already have the updated role info +- name: update edit role for service catalog and pod preset access + command: > + oc replace -f {{ mktemp.stdout }}/edit_sc_patch.yml + when: + - not edit_yaml.results.results[0] | oo_contains_rule(['servicecatalog.k8s.io'], ['instances', 'bindings'], ['create', 'update', 'delete', 'get', 'list', 'watch']) or not edit_yaml.results.results[0] | oo_contains_rule(['settings.k8s.io'], ['podpresets'], ['create', 'update', 'delete', 'get', 'list', 'watch']) + +- oc_obj: + name: admin + kind: clusterrole + state: list + register: admin_yaml + +# only do this if we don't already have the updated role info +- name: Generate apply template for clusterrole/admin + template: + src: sc_role_patching.j2 + dest: "{{ mktemp.stdout }}/admin_sc_patch.yml" + vars: + original_content: "{{ admin_yaml.results.results[0] | to_yaml }}" + when: + - not admin_yaml.results.results[0] | oo_contains_rule(['servicecatalog.k8s.io'], ['instances', 'bindings'], ['create', 'update', 'delete', 'get', 'list', 'watch']) or not admin_yaml.results.results[0] | oo_contains_rule(['settings.k8s.io'], ['podpresets'], ['create', 'update', 'delete', 'get', 'list', 'watch']) + +# only do this if we don't already have the updated role info +- name: update admin role for service catalog and pod preset access + command: > + oc replace -f {{ mktemp.stdout }}/admin_sc_patch.yml + when: + - not admin_yaml.results.results[0] | oo_contains_rule(['servicecatalog.k8s.io'], ['instances', 'bindings'], ['create', 'update', 'delete', 'get', 'list', 'watch']) or not admin_yaml.results.results[0] | oo_contains_rule(['settings.k8s.io'], ['podpresets'], ['create', 'update', 'delete', 'get', 'list', 'watch']) + - shell: > oc get policybindings/kube-system:default -n kube-system || echo "not found" register: get_kube_system diff --git a/roles/openshift_service_catalog/tasks/wire_aggregator.yml b/roles/openshift_service_catalog/tasks/wire_aggregator.yml index cbb29e623..d5291a99a 100644 --- a/roles/openshift_service_catalog/tasks/wire_aggregator.yml +++ b/roles/openshift_service_catalog/tasks/wire_aggregator.yml @@ -119,6 +119,11 @@ when: - not front_proxy_kubeconfig.stat.exists +- name: copy tech preview extension file for service console UI + copy: + src: openshift-ansible-catalog-console.js + dest: /etc/origin/master/openshift-ansible-catalog-console.js + - name: Update master config yedit: state: present @@ -138,6 +143,16 @@ value: [X-Remote-Group] - key: authConfig.requestHeader.extraHeaderPrefixes value: [X-Remote-Extra-] + - key: assetConfig.extensionScripts + value: [/etc/origin/master/openshift-ansible-catalog-console.js] + - key: kubernetesMasterConfig.apiServerArguments.runtime-config + value: [apis/settings.k8s.io/v1alpha1=true] + - key: admissionConfig.pluginConfig.PodPreset.configuration.kind + value: DefaultAdmissionConfig + - key: admissionConfig.pluginConfig.PodPreset.configuration.apiVersion + value: v1 + - key: admissionConfig.pluginConfig.PodPreset.configuration.disable + value: false register: yedit_output #restart master serially here diff --git a/roles/openshift_service_catalog/templates/sc_role_patching.j2 b/roles/openshift_service_catalog/templates/sc_role_patching.j2 new file mode 100644 index 000000000..69b062b3f --- /dev/null +++ b/roles/openshift_service_catalog/templates/sc_role_patching.j2 @@ -0,0 +1,26 @@ +{{ original_content }} +- apiGroups: + - "servicecatalog.k8s.io" + attributeRestrictions: null + resources: + - instances + - bindings + verbs: + - create + - update + - delete + - get + - list + - watch +- apiGroups: + - "settings.k8s.io" + attributeRestrictions: null + resources: + - podpresets + verbs: + - create + - update + - delete + - get + - list + - watch diff --git a/roles/openshift_storage_glusterfs/README.md b/roles/openshift_storage_glusterfs/README.md index 4b9a5f42c..b367e7daf 100644 --- a/roles/openshift_storage_glusterfs/README.md +++ b/roles/openshift_storage_glusterfs/README.md @@ -64,7 +64,7 @@ their configuration as GlusterFS nodes: |--------------------|---------------------------|-----------------------------------------| | glusterfs_cluster | 1 | The ID of the cluster this node should belong to. This is useful when a single heketi service is expected to manage multiple distinct clusters. **NOTE:** For natively-hosted clusters, all pods will be in the same OpenShift namespace | glusterfs_hostname | openshift.node.nodename | A hostname (or IP address) that will be used for internal GlusterFS communication -| glusterfs_ip | openshift.common.ip | An IP address that will be used by pods to communicate with the GlusterFS node +| glusterfs_ip | openshift.common.ip | An IP address that will be used by pods to communicate with the GlusterFS node. **NOTE:** Required for external GlusterFS nodes | glusterfs_zone | 1 | A zone number for the node. Zones are used within the cluster for determining how to distribute the bricks of GlusterFS volumes. heketi will try to spread each volumes' bricks as evenly as possible across all zones Role Variables @@ -76,7 +76,7 @@ GlusterFS cluster into a new or existing OpenShift cluster: | Name | Default value | Description | |--------------------------------------------------|-------------------------|-----------------------------------------| | openshift_storage_glusterfs_timeout | 300 | Seconds to wait for pods to become ready -| openshift_storage_glusterfs_namespace | 'default' | Namespace in which to create GlusterFS resources +| openshift_storage_glusterfs_namespace | 'glusterfs' | Namespace in which to create GlusterFS resources | openshift_storage_glusterfs_is_native | True | GlusterFS should be containerized | openshift_storage_glusterfs_name | 'storage' | A name to identify the GlusterFS cluster, which will be used in resource names | openshift_storage_glusterfs_nodeselector | 'glusterfs=storage-host'| Selector to determine which nodes will host GlusterFS pods in native mode. **NOTE:** The label value is taken from the cluster name @@ -85,6 +85,7 @@ GlusterFS cluster into a new or existing OpenShift cluster: | openshift_storage_glusterfs_version | 'latest' | Container image version to use for GlusterFS pods | openshift_storage_glusterfs_wipe | False | Destroy any existing GlusterFS resources and wipe storage devices. **WARNING: THIS WILL DESTROY ANY DATA ON THOSE DEVICES.** | openshift_storage_glusterfs_heketi_is_native | True | heketi should be containerized +| openshift_storage_glusterfs_heketi_cli | 'heketi-cli' | Command/Path to invoke the heketi-cli tool **NOTE:** Change this only for **non-native heketi** if heketi-cli is not in the global `$PATH` of the machine running openshift-ansible | openshift_storage_glusterfs_heketi_image | 'heketi/heketi' | Container image to use for heketi pods, enterprise default is 'rhgs3/rhgs-volmanager-rhel7' | openshift_storage_glusterfs_heketi_version | 'latest' | Container image version to use for heketi pods | openshift_storage_glusterfs_heketi_admin_key | auto-generated | String to use as secret key for performing heketi commands as admin @@ -92,6 +93,11 @@ GlusterFS cluster into a new or existing OpenShift cluster: | openshift_storage_glusterfs_heketi_topology_load | True | Load the GlusterFS topology information into heketi | openshift_storage_glusterfs_heketi_url | Undefined | When heketi is native, this sets the hostname portion of the final heketi route URL. When heketi is external, this is the full URL to the heketi service. | openshift_storage_glusterfs_heketi_port | 8080 | TCP port for external heketi service **NOTE:** This has no effect in native mode +| openshift_storage_glusterfs_heketi_executor | 'kubernetes' | Selects how a native heketi service will manage GlusterFS nodes: 'kubernetes' for native nodes, 'ssh' for external nodes +| openshift_storage_glusterfs_heketi_ssh_port | 22 | SSH port for external GlusterFS nodes via native heketi +| openshift_storage_glusterfs_heketi_ssh_user | 'root' | SSH user for external GlusterFS nodes via native heketi +| openshift_storage_glusterfs_heketi_ssh_sudo | False | Whether to sudo (if non-root user) for SSH to external GlusterFS nodes via native heketi +| openshift_storage_glusterfs_heketi_ssh_keyfile | '/dev/null' | Path to a private key file for use with SSH connections to external GlusterFS nodes via native heketi **NOTE:** This must be an absolute path | openshift_storage_glusterfs_heketi_wipe | False | Destroy any existing heketi resources, defaults to the value of `openshift_storage_glusterfs_wipe` Each role variable also has a corresponding variable to optionally configure a @@ -103,7 +109,7 @@ are an exception: | Name | Default value | Description | |-------------------------------------------------------|-----------------------|-----------------------------------------| -| openshift_storage_glusterfs_registry_namespace | registry namespace | Default is to use the hosted registry's namespace, otherwise 'default' +| openshift_storage_glusterfs_registry_namespace | registry namespace | Default is to use the hosted registry's namespace, otherwise 'glusterfs' | openshift_storage_glusterfs_registry_name | 'registry' | This allows for the logical separation of the registry GlusterFS cluster from other GlusterFS clusters | openshift_storage_glusterfs_registry_storageclass | False | It is recommended to not create a StorageClass for GlusterFS clusters serving registry storage, so as to avoid performance penalties | openshift_storage_glusterfs_registry_heketi_admin_key | auto-generated | Separate from the above diff --git a/roles/openshift_storage_glusterfs/defaults/main.yml b/roles/openshift_storage_glusterfs/defaults/main.yml index 4ff56af9e..a846889ca 100644 --- a/roles/openshift_storage_glusterfs/defaults/main.yml +++ b/roles/openshift_storage_glusterfs/defaults/main.yml @@ -1,6 +1,6 @@ --- openshift_storage_glusterfs_timeout: 300 -openshift_storage_glusterfs_namespace: 'default' +openshift_storage_glusterfs_namespace: 'glusterfs' openshift_storage_glusterfs_is_native: True openshift_storage_glusterfs_name: 'storage' openshift_storage_glusterfs_nodeselector: "glusterfs={{ openshift_storage_glusterfs_name }}-host" @@ -8,9 +8,10 @@ openshift_storage_glusterfs_storageclass: True openshift_storage_glusterfs_image: "{{ 'rhgs3/rhgs-server-rhel7' | quote if deployment_type == 'openshift-enterprise' else 'gluster/gluster-centos' | quote }}" openshift_storage_glusterfs_version: 'latest' openshift_storage_glusterfs_wipe: False -openshift_storage_glusterfs_heketi_is_native: True +openshift_storage_glusterfs_heketi_is_native: "{{ openshift_storage_glusterfs_is_native }}" openshift_storage_glusterfs_heketi_is_missing: True openshift_storage_glusterfs_heketi_deploy_is_missing: True +openshift_storage_glusterfs_heketi_cli: 'heketi-cli' openshift_storage_glusterfs_heketi_image: "{{ 'rhgs3/rhgs-volmanager-rhel7' | quote if deployment_type == 'openshift-enterprise' else 'heketi/heketi' | quote }}" openshift_storage_glusterfs_heketi_version: 'latest' openshift_storage_glusterfs_heketi_admin_key: "{{ omit }}" @@ -19,9 +20,14 @@ openshift_storage_glusterfs_heketi_topology_load: True openshift_storage_glusterfs_heketi_wipe: "{{ openshift_storage_glusterfs_wipe }}" openshift_storage_glusterfs_heketi_url: "{{ omit }}" openshift_storage_glusterfs_heketi_port: 8080 +openshift_storage_glusterfs_heketi_executor: 'kubernetes' +openshift_storage_glusterfs_heketi_ssh_port: 22 +openshift_storage_glusterfs_heketi_ssh_user: 'root' +openshift_storage_glusterfs_heketi_ssh_sudo: False +openshift_storage_glusterfs_heketi_ssh_keyfile: '/dev/null' openshift_storage_glusterfs_registry_timeout: "{{ openshift_storage_glusterfs_timeout }}" -openshift_storage_glusterfs_registry_namespace: "{{ openshift.hosted.registry.namespace | default('default') }}" +openshift_storage_glusterfs_registry_namespace: "{{ openshift.hosted.registry.namespace | default(openshift_storage_glusterfs_namespace) }}" openshift_storage_glusterfs_registry_is_native: "{{ openshift_storage_glusterfs_is_native }}" openshift_storage_glusterfs_registry_name: 'registry' openshift_storage_glusterfs_registry_nodeselector: "glusterfs={{ openshift_storage_glusterfs_registry_name }}-host" @@ -29,9 +35,10 @@ openshift_storage_glusterfs_registry_storageclass: False openshift_storage_glusterfs_registry_image: "{{ openshift_storage_glusterfs_image }}" openshift_storage_glusterfs_registry_version: "{{ openshift_storage_glusterfs_version }}" openshift_storage_glusterfs_registry_wipe: "{{ openshift_storage_glusterfs_wipe }}" -openshift_storage_glusterfs_registry_heketi_is_native: "{{ openshift_storage_glusterfs_heketi_is_native }}" +openshift_storage_glusterfs_registry_heketi_is_native: "{{ openshift_storage_glusterfs_registry_is_native }}" openshift_storage_glusterfs_registry_heketi_is_missing: "{{ openshift_storage_glusterfs_heketi_is_missing }}" openshift_storage_glusterfs_registry_heketi_deploy_is_missing: "{{ openshift_storage_glusterfs_heketi_deploy_is_missing }}" +openshift_storage_glusterfs_registry_heketi_cli: "{{ openshift_storage_glusterfs_heketi_cli }}" openshift_storage_glusterfs_registry_heketi_image: "{{ openshift_storage_glusterfs_heketi_image }}" openshift_storage_glusterfs_registry_heketi_version: "{{ openshift_storage_glusterfs_heketi_version }}" openshift_storage_glusterfs_registry_heketi_admin_key: "{{ omit }}" @@ -39,4 +46,9 @@ openshift_storage_glusterfs_registry_heketi_user_key: "{{ omit }}" openshift_storage_glusterfs_registry_heketi_topology_load: "{{ openshift_storage_glusterfs_heketi_topology_load }}" openshift_storage_glusterfs_registry_heketi_wipe: "{{ openshift_storage_glusterfs_heketi_wipe }}" openshift_storage_glusterfs_registry_heketi_url: "{{ openshift_storage_glusterfs_heketi_url | default(omit) }}" -openshift_storage_glusterfs_registry_heketi_port: 8080 +openshift_storage_glusterfs_registry_heketi_port: "{{ openshift_storage_glusterfs_heketi_port }}" +openshift_storage_glusterfs_registry_heketi_executor: "{{ openshift_storage_glusterfs_heketi_executor }}" +openshift_storage_glusterfs_registry_heketi_ssh_port: "{{ openshift_storage_glusterfs_heketi_ssh_port }}" +openshift_storage_glusterfs_registry_heketi_ssh_user: "{{ openshift_storage_glusterfs_heketi_ssh_user }}" +openshift_storage_glusterfs_registry_heketi_ssh_sudo: "{{ openshift_storage_glusterfs_heketi_ssh_sudo }}" +openshift_storage_glusterfs_registry_heketi_ssh_keyfile: "{{ openshift_storage_glusterfs_heketi_ssh_keyfile }}" diff --git a/roles/openshift_storage_glusterfs/files/v3.6/deploy-heketi-template.yml b/roles/openshift_storage_glusterfs/files/v3.6/deploy-heketi-template.yml index 4434f750c..9ebb0d5ec 100644 --- a/roles/openshift_storage_glusterfs/files/v3.6/deploy-heketi-template.yml +++ b/roles/openshift_storage_glusterfs/files/v3.6/deploy-heketi-template.yml @@ -71,7 +71,7 @@ objects: - name: HEKETI_ADMIN_KEY value: ${HEKETI_ADMIN_KEY} - name: HEKETI_EXECUTOR - value: kubernetes + value: ${HEKETI_EXECUTOR} - name: HEKETI_FSTAB value: /var/lib/heketi/fstab - name: HEKETI_SNAPSHOT_LIMIT @@ -87,6 +87,8 @@ objects: mountPath: /var/lib/heketi - name: topology mountPath: ${TOPOLOGY_PATH} + - name: config + mountPath: /etc/heketi readinessProbe: timeoutSeconds: 3 initialDelaySeconds: 3 @@ -104,6 +106,9 @@ objects: - name: topology secret: secretName: heketi-${CLUSTER_NAME}-topology-secret + - name: config + secret: + secretName: heketi-${CLUSTER_NAME}-config-secret parameters: - name: HEKETI_USER_KEY displayName: Heketi User Secret @@ -111,6 +116,10 @@ parameters: - name: HEKETI_ADMIN_KEY displayName: Heketi Administrator Secret description: Set secret for administration of the Heketi service as user _admin_ +- name: HEKETI_EXECUTOR + displayName: heketi executor type + description: Set the executor type, kubernetes or ssh + value: kubernetes - name: HEKETI_KUBE_NAMESPACE displayName: Namespace description: Set the namespace where the GlusterFS pods reside diff --git a/roles/openshift_storage_glusterfs/files/v3.6/heketi-template.yml b/roles/openshift_storage_glusterfs/files/v3.6/heketi-template.yml index e3fa0a9fb..61b6a8c13 100644 --- a/roles/openshift_storage_glusterfs/files/v3.6/heketi-template.yml +++ b/roles/openshift_storage_glusterfs/files/v3.6/heketi-template.yml @@ -67,7 +67,7 @@ objects: - name: HEKETI_ADMIN_KEY value: ${HEKETI_ADMIN_KEY} - name: HEKETI_EXECUTOR - value: kubernetes + value: ${HEKETI_EXECUTOR} - name: HEKETI_FSTAB value: /var/lib/heketi/fstab - name: HEKETI_SNAPSHOT_LIMIT @@ -81,6 +81,8 @@ objects: volumeMounts: - name: db mountPath: /var/lib/heketi + - name: config + mountPath: /etc/heketi readinessProbe: timeoutSeconds: 3 initialDelaySeconds: 3 @@ -98,6 +100,9 @@ objects: glusterfs: endpoints: heketi-db-${CLUSTER_NAME}-endpoints path: heketidbstorage + - name: config + secret: + secretName: heketi-${CLUSTER_NAME}-config-secret parameters: - name: HEKETI_USER_KEY displayName: Heketi User Secret @@ -105,6 +110,10 @@ parameters: - name: HEKETI_ADMIN_KEY displayName: Heketi Administrator Secret description: Set secret for administration of the Heketi service as user _admin_ +- name: HEKETI_EXECUTOR + displayName: heketi executor type + description: Set the executor type, kubernetes or ssh + value: kubernetes - name: HEKETI_KUBE_NAMESPACE displayName: Namespace description: Set the namespace where the GlusterFS pods reside diff --git a/roles/openshift_storage_glusterfs/tasks/glusterfs_common.yml b/roles/openshift_storage_glusterfs/tasks/glusterfs_common.yml index af901103e..600d8f676 100644 --- a/roles/openshift_storage_glusterfs/tasks/glusterfs_common.yml +++ b/roles/openshift_storage_glusterfs/tasks/glusterfs_common.yml @@ -1,4 +1,16 @@ --- +- name: Make sure heketi-client is installed + package: name=heketi-client state=present + when: + - not openshift.common.is_atomic | bool + - not glusterfs_heketi_is_native | bool + +- name: Verify heketi-cli is installed + shell: "command -v {{ glusterfs_heketi_cli }} >/dev/null 2>&1 || { echo >&2 'ERROR: Make sure heketi-cli is available, then re-run the installer'; exit 1; }" + changed_when: False + when: + - not glusterfs_heketi_is_native | bool + - name: Verify target namespace exists oc_project: state: present @@ -19,6 +31,8 @@ name: "heketi-storage-endpoints" - kind: "secret" name: "heketi-{{ glusterfs_name }}-topology-secret" + - kind: "secret" + name: "heketi-{{ glusterfs_name }}-config-secret" - kind: "template,route,service,dc" name: "heketi-{{ glusterfs_name }}" - kind: "svc" @@ -125,6 +139,13 @@ when: - glusterfs_heketi_topology_load +- name: Generate heketi config file + template: + src: "{{ openshift.common.examples_content_version }}/heketi.json.j2" + dest: "{{ mktemp.stdout }}/heketi.json" + when: + - glusterfs_heketi_is_native + - name: Generate heketi admin key set_fact: glusterfs_heketi_admin_key: "{{ 32 | oo_generate_secret }}" @@ -142,6 +163,20 @@ - glusterfs_heketi_is_native - glusterfs_heketi_user_key is undefined +- name: Create heketi config secret + oc_secret: + namespace: "{{ glusterfs_namespace }}" + state: present + name: "heketi-{{ glusterfs_name }}-config-secret" + force: True + files: + - name: heketi.json + path: "{{ mktemp.stdout }}/heketi.json" + - name: private_key + path: "{{ glusterfs_heketi_ssh_keyfile }}" + when: + - glusterfs_heketi_is_native + - include: heketi_deploy_part1.yml when: - glusterfs_heketi_is_native @@ -150,7 +185,7 @@ - name: Set heketi-cli command set_fact: - glusterfs_heketi_client: "{% if glusterfs_heketi_is_native %}{{ openshift.common.client_binary }} rsh --namespace={{ glusterfs_namespace }} {{ heketi_pod.results.results[0]['items'][0]['metadata']['name'] }} {% endif %}heketi-cli -s http://{% if glusterfs_heketi_is_native %}localhost:8080{% else %}{{ glusterfs_heketi_url }}:{{ glusterfs_heketi_port }}{% endif %} --user admin --secret '{{ glusterfs_heketi_admin_key }}'" + glusterfs_heketi_client: "{% if glusterfs_heketi_is_native %}{{ openshift.common.client_binary }} rsh --namespace={{ glusterfs_namespace }} {{ heketi_pod.results.results[0]['items'][0]['metadata']['name'] }} {% endif %}{{ glusterfs_heketi_cli }} -s http://{% if glusterfs_heketi_is_native %}localhost:8080{% else %}{{ glusterfs_heketi_url }}:{{ glusterfs_heketi_port }}{% endif %} --user admin {% if glusterfs_heketi_admin_key is defined %}--secret '{{ glusterfs_heketi_admin_key }}'{% endif %}" - name: Verify heketi service command: "{{ glusterfs_heketi_client }} cluster list" @@ -180,6 +215,7 @@ data: "{{ glusterfs_heketi_admin_key }}" when: - glusterfs_storageclass + - glusterfs_heketi_admin_key is defined - name: Get heketi route oc_obj: diff --git a/roles/openshift_storage_glusterfs/tasks/glusterfs_config.yml b/roles/openshift_storage_glusterfs/tasks/glusterfs_config.yml index dbfe126a4..7a2987883 100644 --- a/roles/openshift_storage_glusterfs/tasks/glusterfs_config.yml +++ b/roles/openshift_storage_glusterfs/tasks/glusterfs_config.yml @@ -2,24 +2,30 @@ - set_fact: glusterfs_timeout: "{{ openshift_storage_glusterfs_timeout }}" glusterfs_namespace: "{{ openshift_storage_glusterfs_namespace }}" - glusterfs_is_native: "{{ openshift_storage_glusterfs_is_native }}" + glusterfs_is_native: "{{ openshift_storage_glusterfs_is_native | bool }}" glusterfs_name: "{{ openshift_storage_glusterfs_name }}" glusterfs_nodeselector: "{{ openshift_storage_glusterfs_nodeselector | default(['storagenode', openshift_storage_glusterfs_name] | join('=')) | map_from_pairs }}" glusterfs_storageclass: "{{ openshift_storage_glusterfs_storageclass }}" glusterfs_image: "{{ openshift_storage_glusterfs_image }}" glusterfs_version: "{{ openshift_storage_glusterfs_version }}" - glusterfs_wipe: "{{ openshift_storage_glusterfs_wipe }}" - glusterfs_heketi_is_native: "{{ openshift_storage_glusterfs_heketi_is_native }}" - glusterfs_heketi_is_missing: "{{ openshift_storage_glusterfs_heketi_is_missing }}" - glusterfs_heketi_deploy_is_missing: "{{ openshift_storage_glusterfs_heketi_deploy_is_missing }}" + glusterfs_wipe: "{{ openshift_storage_glusterfs_wipe | bool }}" + glusterfs_heketi_is_native: "{{ openshift_storage_glusterfs_heketi_is_native | bool }}" + glusterfs_heketi_is_missing: "{{ openshift_storage_glusterfs_heketi_is_missing | bool }}" + glusterfs_heketi_deploy_is_missing: "{{ openshift_storage_glusterfs_heketi_deploy_is_missing | bool }}" + glusterfs_heketi_cli: "{{ openshift_storage_glusterfs_heketi_cli }}" glusterfs_heketi_image: "{{ openshift_storage_glusterfs_heketi_image }}" glusterfs_heketi_version: "{{ openshift_storage_glusterfs_heketi_version }}" glusterfs_heketi_admin_key: "{{ openshift_storage_glusterfs_heketi_admin_key }}" glusterfs_heketi_user_key: "{{ openshift_storage_glusterfs_heketi_user_key }}" - glusterfs_heketi_topology_load: "{{ openshift_storage_glusterfs_heketi_topology_load }}" - glusterfs_heketi_wipe: "{{ openshift_storage_glusterfs_heketi_wipe }}" + glusterfs_heketi_topology_load: "{{ openshift_storage_glusterfs_heketi_topology_load | bool }}" + glusterfs_heketi_wipe: "{{ openshift_storage_glusterfs_heketi_wipe | bool }}" glusterfs_heketi_url: "{{ openshift_storage_glusterfs_heketi_url }}" glusterfs_heketi_port: "{{ openshift_storage_glusterfs_heketi_port }}" + glusterfs_heketi_executor: "{{ openshift_storage_glusterfs_heketi_executor }}" + glusterfs_heketi_ssh_port: "{{ openshift_storage_glusterfs_heketi_ssh_port }}" + glusterfs_heketi_ssh_user: "{{ openshift_storage_glusterfs_heketi_ssh_user }}" + glusterfs_heketi_ssh_sudo: "{{ openshift_storage_glusterfs_heketi_ssh_sudo | bool }}" + glusterfs_heketi_ssh_keyfile: "{{ openshift_storage_glusterfs_heketi_ssh_keyfile }}" glusterfs_nodes: "{{ groups.glusterfs }}" - include: glusterfs_common.yml diff --git a/roles/openshift_storage_glusterfs/tasks/glusterfs_registry.yml b/roles/openshift_storage_glusterfs/tasks/glusterfs_registry.yml index 0849f2a2e..e46cec378 100644 --- a/roles/openshift_storage_glusterfs/tasks/glusterfs_registry.yml +++ b/roles/openshift_storage_glusterfs/tasks/glusterfs_registry.yml @@ -2,24 +2,30 @@ - set_fact: glusterfs_timeout: "{{ openshift_storage_glusterfs_registry_timeout }}" glusterfs_namespace: "{{ openshift_storage_glusterfs_registry_namespace }}" - glusterfs_is_native: "{{ openshift_storage_glusterfs_registry_is_native }}" + glusterfs_is_native: "{{ openshift_storage_glusterfs_registry_is_native | bool }}" glusterfs_name: "{{ openshift_storage_glusterfs_registry_name }}" glusterfs_nodeselector: "{{ openshift_storage_glusterfs_registry_nodeselector | default(['storagenode', openshift_storage_glusterfs_registry_name] | join('=')) | map_from_pairs }}" glusterfs_storageclass: "{{ openshift_storage_glusterfs_registry_storageclass }}" glusterfs_image: "{{ openshift_storage_glusterfs_registry_image }}" glusterfs_version: "{{ openshift_storage_glusterfs_registry_version }}" - glusterfs_wipe: "{{ openshift_storage_glusterfs_registry_wipe }}" - glusterfs_heketi_is_native: "{{ openshift_storage_glusterfs_registry_heketi_is_native }}" - glusterfs_heketi_is_missing: "{{ openshift_storage_glusterfs_registry_heketi_is_missing }}" - glusterfs_heketi_deploy_is_missing: "{{ openshift_storage_glusterfs_registry_heketi_deploy_is_missing }}" + glusterfs_wipe: "{{ openshift_storage_glusterfs_registry_wipe | bool }}" + glusterfs_heketi_is_native: "{{ openshift_storage_glusterfs_registry_heketi_is_native | bool }}" + glusterfs_heketi_is_missing: "{{ openshift_storage_glusterfs_registry_heketi_is_missing | bool }}" + glusterfs_heketi_deploy_is_missing: "{{ openshift_storage_glusterfs_registry_heketi_deploy_is_missing | bool }}" + glusterfs_heketi_cli: "{{ openshift_storage_glusterfs_registry_heketi_cli }}" glusterfs_heketi_image: "{{ openshift_storage_glusterfs_registry_heketi_image }}" glusterfs_heketi_version: "{{ openshift_storage_glusterfs_registry_heketi_version }}" glusterfs_heketi_admin_key: "{{ openshift_storage_glusterfs_registry_heketi_admin_key }}" glusterfs_heketi_user_key: "{{ openshift_storage_glusterfs_registry_heketi_user_key }}" - glusterfs_heketi_topology_load: "{{ openshift_storage_glusterfs_registry_heketi_topology_load }}" - glusterfs_heketi_wipe: "{{ openshift_storage_glusterfs_registry_heketi_wipe }}" + glusterfs_heketi_topology_load: "{{ openshift_storage_glusterfs_registry_heketi_topology_load | bool }}" + glusterfs_heketi_wipe: "{{ openshift_storage_glusterfs_registry_heketi_wipe | bool }}" glusterfs_heketi_url: "{{ openshift_storage_glusterfs_registry_heketi_url }}" glusterfs_heketi_port: "{{ openshift_storage_glusterfs_registry_heketi_port }}" + glusterfs_heketi_executor: "{{ openshift_storage_glusterfs_registry_heketi_executor }}" + glusterfs_heketi_ssh_port: "{{ openshift_storage_glusterfs_registry_heketi_ssh_port }}" + glusterfs_heketi_ssh_user: "{{ openshift_storage_glusterfs_registry_heketi_ssh_user }}" + glusterfs_heketi_ssh_sudo: "{{ openshift_storage_glusterfs_registry_heketi_ssh_sudo | bool }}" + glusterfs_heketi_ssh_keyfile: "{{ openshift_storage_glusterfs_registry_heketi_ssh_keyfile }}" glusterfs_nodes: "{{ groups.glusterfs_registry | default(groups.glusterfs) }}" - include: glusterfs_common.yml @@ -50,7 +56,7 @@ - name: Create GlusterFS registry endpoints oc_obj: - namespace: "{{ glusterfs_namespace }}" + namespace: "{{ openshift.hosted.registry.namespace | default('default') }}" state: present kind: endpoints name: "glusterfs-{{ glusterfs_name }}-endpoints" @@ -59,7 +65,7 @@ - name: Create GlusterFS registry service oc_obj: - namespace: "{{ glusterfs_namespace }}" + namespace: "{{ openshift.hosted.registry.namespace | default('default') }}" state: present kind: service name: "glusterfs-{{ glusterfs_name }}-endpoints" diff --git a/roles/openshift_storage_glusterfs/tasks/heketi_deploy_part1.yml b/roles/openshift_storage_glusterfs/tasks/heketi_deploy_part1.yml index ea9b1fe1f..3ba1eb2d2 100644 --- a/roles/openshift_storage_glusterfs/tasks/heketi_deploy_part1.yml +++ b/roles/openshift_storage_glusterfs/tasks/heketi_deploy_part1.yml @@ -36,6 +36,7 @@ HEKETI_ROUTE: "{{ glusterfs_heketi_url | default(['heketi-',glusterfs_name]|join) }}" HEKETI_USER_KEY: "{{ glusterfs_heketi_user_key }}" HEKETI_ADMIN_KEY: "{{ glusterfs_heketi_admin_key }}" + HEKETI_EXECUTOR: "{{ glusterfs_heketi_executor }}" HEKETI_KUBE_NAMESPACE: "{{ glusterfs_namespace }}" CLUSTER_NAME: "{{ glusterfs_name }}" TOPOLOGY_PATH: "{{ mktemp.stdout }}" diff --git a/roles/openshift_storage_glusterfs/tasks/heketi_deploy_part2.yml b/roles/openshift_storage_glusterfs/tasks/heketi_deploy_part2.yml index 63009c539..37d3e6ba2 100644 --- a/roles/openshift_storage_glusterfs/tasks/heketi_deploy_part2.yml +++ b/roles/openshift_storage_glusterfs/tasks/heketi_deploy_part2.yml @@ -106,6 +106,7 @@ HEKETI_ROUTE: "{{ glusterfs_heketi_url | default(['heketi-',glusterfs_name]|join) }}" HEKETI_USER_KEY: "{{ glusterfs_heketi_user_key }}" HEKETI_ADMIN_KEY: "{{ glusterfs_heketi_admin_key }}" + HEKETI_EXECUTOR: "{{ glusterfs_heketi_executor }}" HEKETI_KUBE_NAMESPACE: "{{ glusterfs_namespace }}" CLUSTER_NAME: "{{ glusterfs_name }}" @@ -125,7 +126,7 @@ - name: Set heketi-cli command set_fact: - glusterfs_heketi_client: "{{ openshift.common.client_binary }} rsh --namespace={{ glusterfs_namespace }} {{ heketi_pod.results.results[0]['items'][0]['metadata']['name'] }} heketi-cli -s http://localhost:8080 --user admin --secret '{{ glusterfs_heketi_admin_key }}'" + glusterfs_heketi_client: "{{ openshift.common.client_binary }} rsh --namespace={{ glusterfs_namespace }} {{ heketi_pod.results.results[0]['items'][0]['metadata']['name'] }} {{ glusterfs_heketi_cli }} -s http://localhost:8080 --user admin --secret '{{ glusterfs_heketi_admin_key }}'" - name: Verify heketi service command: "{{ glusterfs_heketi_client }} cluster list" diff --git a/roles/openshift_storage_glusterfs/templates/v3.6/glusterfs-storageclass.yml.j2 b/roles/openshift_storage_glusterfs/templates/v3.6/glusterfs-storageclass.yml.j2 index 2ec9a9e9a..095fb780f 100644 --- a/roles/openshift_storage_glusterfs/templates/v3.6/glusterfs-storageclass.yml.j2 +++ b/roles/openshift_storage_glusterfs/templates/v3.6/glusterfs-storageclass.yml.j2 @@ -7,5 +7,7 @@ provisioner: kubernetes.io/glusterfs parameters: resturl: "http://{% if glusterfs_heketi_is_native %}{{ glusterfs_heketi_route }}{% else %}{{ glusterfs_heketi_url }}:{{ glusterfs_heketi_port }}{% endif %}" restuser: "admin" +{% if glusterfs_heketi_admin_key is defined %} secretNamespace: "{{ glusterfs_namespace }}" secretName: "heketi-{{ glusterfs_name }}-admin-secret" +{%- endif -%} diff --git a/roles/openshift_storage_glusterfs/templates/v3.6/heketi.json.j2 b/roles/openshift_storage_glusterfs/templates/v3.6/heketi.json.j2 new file mode 100644 index 000000000..579b11bb7 --- /dev/null +++ b/roles/openshift_storage_glusterfs/templates/v3.6/heketi.json.j2 @@ -0,0 +1,36 @@ +{ + "_port_comment": "Heketi Server Port Number", + "port" : "8080", + + "_use_auth": "Enable JWT authorization. Please enable for deployment", + "use_auth" : false, + + "_jwt" : "Private keys for access", + "jwt" : { + "_admin" : "Admin has access to all APIs", + "admin" : { + "key" : "My Secret" + }, + "_user" : "User only has access to /volumes endpoint", + "user" : { + "key" : "My Secret" + } + }, + + "_glusterfs_comment": "GlusterFS Configuration", + "glusterfs" : { + + "_executor_comment": "Execute plugin. Possible choices: mock, kubernetes, ssh", + "executor" : "{{ glusterfs_heketi_executor }}", + + "_db_comment": "Database file name", + "db" : "/var/lib/heketi/heketi.db", + + "sshexec" : { + "keyfile" : "/etc/heketi/private_key", + "port" : "{{ glusterfs_heketi_ssh_port }}", + "user" : "{{ glusterfs_heketi_ssh_user }}", + "sudo" : {{ glusterfs_heketi_ssh_sudo | lower }} + } + } +} diff --git a/roles/openshift_storage_glusterfs/templates/v3.6/topology.json.j2 b/roles/openshift_storage_glusterfs/templates/v3.6/topology.json.j2 index 3aac68e2f..d6c28f6dd 100644 --- a/roles/openshift_storage_glusterfs/templates/v3.6/topology.json.j2 +++ b/roles/openshift_storage_glusterfs/templates/v3.6/topology.json.j2 @@ -17,10 +17,20 @@ "node": { "hostnames": { "manage": [ - "{{ hostvars[node].glusterfs_hostname | default(hostvars[node].openshift.node.nodename) }}" +{%- if 'glusterfs_hostname' in hostvars[node] -%} + "{{ hostvars[node].glusterfs_hostname }}" +{%- elif 'openshift' in hostvars[node] -%} + "{{ hostvars[node].openshift.node.nodename }}" +{%- else -%} + "{{ node }}" +{%- endif -%} ], "storage": [ - "{{ hostvars[node].glusterfs_ip | default(hostvars[node].openshift.common.ip) }}" +{%- if 'glusterfs_ip' in hostvars[node] -%} + "{{ hostvars[node].glusterfs_ip }}" +{%- else -%} + "{{ hostvars[node].openshift.common.ip }}" +{%- endif -%} ] }, "zone": {{ hostvars[node].glusterfs_zone | default(1) }} diff --git a/roles/openshift_storage_nfs/tasks/main.yml b/roles/openshift_storage_nfs/tasks/main.yml index 0d6b8b7d4..019ada2fb 100644 --- a/roles/openshift_storage_nfs/tasks/main.yml +++ b/roles/openshift_storage_nfs/tasks/main.yml @@ -30,7 +30,7 @@ - "{{ openshift.hosted.metrics }}" - "{{ openshift.hosted.logging }}" - "{{ openshift.hosted.loggingops }}" - + - "{{ openshift.hosted.etcd }}" - name: Configure exports template: diff --git a/roles/openshift_storage_nfs/templates/exports.j2 b/roles/openshift_storage_nfs/templates/exports.j2 index 8c6d4105c..7e8f70b23 100644 --- a/roles/openshift_storage_nfs/templates/exports.j2 +++ b/roles/openshift_storage_nfs/templates/exports.j2 @@ -2,3 +2,4 @@ {{ openshift.hosted.metrics.storage.nfs.directory }}/{{ openshift.hosted.metrics.storage.volume.name }} {{ openshift.hosted.metrics.storage.nfs.options }} {{ openshift.hosted.logging.storage.nfs.directory }}/{{ openshift.hosted.logging.storage.volume.name }} {{ openshift.hosted.logging.storage.nfs.options }} {{ openshift.hosted.loggingops.storage.nfs.directory }}/{{ openshift.hosted.loggingops.storage.volume.name }} {{ openshift.hosted.loggingops.storage.nfs.options }} +{{ openshift.hosted.etcd.storage.nfs.directory }}/{{ openshift.hosted.etcd.storage.volume.name }} {{ openshift.hosted.etcd.storage.nfs.options }} @@ -24,9 +24,4 @@ commands = generate_validation: python setup.py generate_validation # TODO(rhcarvalho): check syntax of other important entrypoint playbooks ansible_syntax: python setup.py ansible_syntax - # ansible 2.2.2+ unfortunately breaks the integration test runner - # because it can no longer set facts on the test docker hosts. - # So for now, install separate ansible version for integration. - # PR that fixes it: https://github.com/ansible/ansible/pull/23599 - # Once that PR is available, drop this and use same ansible. - integration: pip install ansible==2.2.1.0 + integration: python -c 'print "run test/integration/run-tests.sh"' |