| Age | Commit message (Collapse) | Author |
|---|
|
Signed-off-by: Vadim Rutkovsky <vrutkovs@redhat.com>
|
|
Initial support for 3.10
|
|
|
|
|
|
Move more plugins to lib_utils
|
|
Bug 1527178 - installation of logging stack failed: Invalid version s…
|
|
Automatic merge from submit-queue.
logging: fix jinja filters to support py3
|
|
This commit continues moving plugins into lib_utils.
This commit does not move any plugins for add-on roles
such as logging and metrics.
|
|
Since py3 returns `dict_items` for dict.keys() call instead of a list,
it should be converted into a list for compatibility
Signed-off-by: Vadim Rutkovsky <vrutkovs@redhat.com>
|
|
This commit removes become:no statements that break
the installer in various ways.
|
|
specified for Elasticsearch
openshift_logging_{curator,elasicsearch,fluentd,kibana,mux}/vars/main.yml:
- adding "3_9" to __allowed_.*_versions
- bumping __latest_.*_version to "3_9"
|
|
In Ansible 2.2, the include_role directive came into existence as
a Tech Preview. It is still a Tech Preview through Ansible 2.4
(and in current devel branch), but with a noteable change. The
default behavior switched from static: true to static: false
because that functionality moved to the newly introduced
import_role directive (in order to stay consistent with include*
being dynamic in nature and `import* being static in nature).
The dynamic include is considerably more memory intensive as it will
dynamically create a role import for every host in the inventory
list to be used. (Also worth noting, there is at the time of this
writing an object allocation inefficiency in the dynamic include
that can in certain situations amplify this effect considerably)
This change is meant to mitigate the pressure on memory for the
Ansible control host.
We need to evaluate where it makes sense to dynamically include roles
and revert back to dynamic inclusion if and where it makes sense to do
so.
|
|
Automatic merge from submit-queue.
Update logging to use existing cluster deployment for defaults
This will allow us to use logging facts to set defaults of specific configurations such as ES index replicas and shard count.
The update to logging facts yields us output like:
```json
"elasticsearch": {
"clusterrolebindings": {},
"configmaps": {
"logging-elasticsearch": {
"elasticsearch.yml": {
"cloud": {
"kubernetes": {
"namespace": "${NAMESPACE}",
"pod_label": "${POD_LABEL}",
"pod_port": 9300
}
},
"cluster": {
"name": "${CLUSTER_NAME}"
},
"discovery": {
"type": "kubernetes",
"zen.minimum_master_nodes": "${NODE_QUORUM}",
"zen.ping.multicast.enabled": false
},
"gateway": {
"expected_nodes": "${RECOVER_EXPECTED_NODES}",
"recover_after_nodes": "${NODE_QUORUM}",
"recover_after_time": "${RECOVER_AFTER_TIME}"
},
"index": {
"number_of_replicas": 0,
"number_of_shards": 1,
"translog": {
"flush_threshold_period": "5m",
"flush_threshold_size": "256mb"
},
"unassigned.node_left.delayed_timeout": "2m"
},
"io.fabric8.elasticsearch.authentication.users": [
"system.logging.kibana",
"system.logging.fluentd",
"system.logging.curator",
"system.admin"
],
```
TODO:
- [x] Update logging facts to pull out settings from config maps
- [x] Move `openshift_sanitize_inventory/library/conditional_set_fact.py` up to repo level
- [x] Generate diffs against currently deployed configs and correctly patch in custom changes from customers
- [x] Use `conditional_set_fact` to easily set defaults for logging based on logging facts, or falling back to role defaults when not specified in the inventory
- [x] Update all components to follow patching configmaps
|
|
files, use diffs to keep custom changes, white list certain settings when creating diffs
|
|
specified for Elasticsearch
openshift_logging_{curator,elasicsearch,fluentd,kibana,mux}/vars/main.yml:
- adding "3_8" to __allowed_.*_versions
- replacing the value of __latest_.*_version "3_6" with "3_8".
|
|
Move openshift_deployment_type check into sanity_check
action plugin. Remove compatibility for deployment_type.
deployment_type has been deprecated for some time now.
|
|
This commit relocates filter_plugings to lib_utils,
changes the namespacing to prevent unintended use of
older versions that may be present in filter_plugins/
directory on existing installs.
Add lib_utils to meta depends for roles
Also consolidate some plugins into lib_utils from
various other areas.
Update rpm spec, obsolete plugin rpms.
|
|
. fixing a typo in fluentd template
. adjusting the default value of openshift_logging_mux_file_buffer_limit
|
|
Do not escalate privileges in logging stack deployment task
|
|
|
|
Remove hosted vars from openshift_facts.
The current pattern is causing a bunch of undesired sideffects.
|
|
|
|
Automatic merge from submit-queue.
Add enclosing <store> tags to sample secure-forward.conf files
This fixes [BZ#1498398](https://bugzilla.redhat.com/show_bug.cgi?id=1498398)
The examples provided in the documentation to configure the secure forward mention the configuration must be enclosed between the <store> tag. For that I think the examples should include it as well.
See [3.6 Documentation](https://docs.openshift.com/container-platform/3.6/install_config/aggregate_logging.html#aggregated-fluentd)
~~~
<store>
@type secure_forward
self_hostname pod-${HOSTNAME}
shared_key thisisasharedkey
secure yes
enable_strict_verification yes
ca_cert_path /etc/fluent/keys/your_ca_cert
ca_private_key_path /etc/fluent/keys/your_private_key
ca_private_key_passphrase passphrase
<server>
host ose1.example.com
port 24284
</server>
<server>
host ose2.example.com
port 24284
standby
</server>
<server>
host ose3.example.com
port 24284
standby
</server>
</store>
~~~
|
|
|
|
openshift_logging pattern
|
|
|
|
- all images logging and metrics change their default imagePullPolicy
from Always to IfNotPresent
|
|
We now use a CPU request to ensure logging infrastructure pods are
not capped by default for CPU usage. It is still important to ensure
we have a minimum amount of CPU.
We keep the use of the variables *_cpu_limit so that the existing
behavior is maintained.
Note that we don't want to cap an infra pod's CPU usage by default,
since we want to be able to use the necessary resources to complete
it's tasks.
Bug 1501960 (https://bugzilla.redhat.com/show_bug.cgi?id=1501960)
|
|
If enabled, tho logs are stored in ES' operations index, accesible only by cluster admins.
|
|
fails to start due to missing
If openshift_logging_use_mux=False and openshift_logging_mux_allow_external=False,
then all other mux related parameters should be set to False (if boolean) or
removed (e.g. openshift_logging_mux_client_mode should be undefined).
|
|
|
|
|
|
Includes the following code review fixes, and a rebase with upstream/master
- mux.j2 template needed mux specific parameters for syslog configs
- fixed bug in IF condition not checking for true-ness
- updated README wording to remove ambiguity of payload_key's datatype
|
|
Automatic merge from submit-queue
logging: introducing event router
Initial concept integrating kubernetes events to EFK stack using [eventrouter](https://github.com/heptio/eventrouter)
- **eventrouter** is deployed to logging project, has a service account and its own role to read events
- **eventrouter** watches kubernetes events, marshalls them to JSON and outputs to its STDOUT
- **fluentd** picks them up and inserts to elastic search logging project index
Kubernetes events could be easily stored into different index. Among other ways, it could be achieved either by a fluentd filter plugin or custom eventrouter sink.
Pending actions:
- [x] - get our downstream and upstream images for eventrouter, don't use heptio's
cc: @josefkarasek , @jcantrill , @richm
|
|
Automatic merge from submit-queue
Fluentd: one output tag, one output plugin (origin-aggregated-logging)
|
|
- eventrouter has its own role 'openshift_logging_eventrouter' written as
a template
- 'openshift_logging_install_eventrouter' controls whether it gets included
in the playbook or not
- deployed by default to 'default' namespace
|
|
Adding <label @OUTPUT> to fluent.conf.
|
|
|
|
Merged by openshift-bot
|
|
|
|
|
|
|
|
logging set memory request to limit
|
|
Merged by openshift-bot
|
|
|
|
Allowing to specify an image version for each logging component
https://bugzilla.redhat.com/show_bug.cgi?id=1471322
|
|
Without that, playbook runs print warnings such as this:
[WARNING]: when statements should not include jinja2 templating
delimiters such as {{ }} or {% %}. Found: {{ g_etcd_hosts is not
defined and g_new_etcd_hosts is not
defined}}
|
|
Instead of the `openshift_logging_use_mux_client` boolean parameter,
use `openshift_logging_mux_client_mode` which will allow us to support
different mux client use cases:
The value `maximal` will cause Fluentd to perform as much of the
processing as possible at the local node. This currently means all of
the processing *except* for the Kubernetes metadata processing, which will
be done by mux. This is the currently recommended mode to use due to
current scaling issues.
The value `minimal` means that Fluentd will do *no* processing at all,
and send the raw logs to mux for processing. This is currently not
recommended to use due to current scaling issues. Ansible will warn
you if you try to use this mode.
`MUX_ALLOW_EXTERNAL` is no longer needed in the mux dc. mux now always
operates to process external logs. The ansible setting
`openshift_logging_mux_allow_external` is still required in order to
set up the mux service to accept connections from outside of the
cluster.
|
|
(cherry picked from commit 231a7c4ce27ff944d9e14169062f487f57c94dd6)
|
|
"openshift_logging_fluentd_use_journal=false" nor omitted collects the log entries
https://bugzilla.redhat.com/show_bug.cgi?id=1466152
Do not set openshift_logging_fluentd_use_journal or USE_JOURNAL at
all unless it is explicitly set as an ansible param. It is almost
always better to let fluentd figure out which log driver docker
is using.
|
