Commit Graph

2736 Commits (cc77a8c3950e32cc8955c8db484e619d59f64115)

Author SHA1 Message Date
Qasim Sarfraz 0a3cf1a087 Fix CA cert environment variable for ectd v3 (#4381) 2019-03-28 00:18:43 -07:00
Bart Verwilst 0efa3e6392
Upgrade to k8s 1.13.5 2019-03-27 11:16:21 +01:00
Matthew Mosesohn 6d7f3c4405 Reduce jinja2 filters in coredns templates (#4390) 2019-03-26 11:09:17 -07:00
Etienne d0ae316934 Use proxy_env with kubeadm phase commands (#4325) 2019-03-26 03:03:19 -07:00
Matthew Mosesohn b7fd462944 Fix support for ansible 2.7.9 (#4375) 2019-03-20 11:29:42 -07:00
Matthew Mosesohn ec08303f82 Revert "Fix #4237: update kube cert path (#4354)" (#4369)
This reverts commit ea7a6f1cf1.

This change modified the certs dir for Kubernetes, but did not move the directories for existing clusters.
2019-03-20 05:56:57 -07:00
Dmitry Chepurovskiy ea7a6f1cf1 Fix #4237: update kube cert path (#4354) 2019-03-17 23:55:11 -07:00
Matthew Mosesohn 150a969cf4
Forcefully delete pods when necessary (#4328)
Pods on down/unresponsive nodes can't be deleted without
--force --grace-period=0.

Fixes #4314
2019-03-14 07:45:46 -07:00
Manuel Cintron 3c4cbf133e Adding ability to override dashboard replica count (#4344) 2019-03-13 13:58:25 -07:00
Matthew Mosesohn fd2c47b56a Move most coredns templates to static files (#4341)
* Move most coredns templates to static files

This should speed up the task slightly

* yaml lint fixes
2019-03-12 21:17:31 -07:00
Bort Verwilst 33024731e4 Upgrade to k8s 1.13.4 (#4319) 2019-03-06 23:16:56 -08:00
chadswilson d469282f1c add blockSize to IPPool spec for Calico >= v3.3.0 (#4224)
* add blockSize to IPPool spec for Calico >= v3.3.0

* fix "cidr" spec in Calico IPPool resource for my PR
2019-03-06 12:42:48 -08:00
Matthew Mosesohn acbf3db233 Remove hard dependence on facts for all nodes (#4304)
* Remove hard dependence on facts for all nodes

* Update main.yaml

* Update main.yaml
2019-03-05 03:04:39 -08:00
Matthew Mosesohn adf6a7121f Reenable set_facts task for dns_late (#4312) 2019-03-01 05:39:30 -08:00
Bort Verwilst bbfd2dc2bd Add 1.12.6, sort arm64 descending (#4308)
* Add 1.12.6, sort arm64 descending

* remove 1.10.x checksums (EOL anyways)
2019-02-28 05:55:19 -08:00
Matthew Mosesohn 4fe61968cf Set default value for local_path_provisioner_enabled in role (#4309) 2019-02-28 05:36:08 -08:00
Anupam Basak 9e8e069b23 remove kube bridge on reset (#4250) 2019-02-26 00:32:00 -08:00
Peter Metz 26ca58419f feat(external-provisioner): adds support for local-path-provisioner (#4232)
* feat(external-provisioner/local-path-provisioner): adds support for local path provisioner

Helpful for local development but also in production workloads (once the
permission model is worked out) where you have redundancy built into the
software uses the PVCs (e.g. database cluster with synchronous
replication)

* feat(local-path-provisioner): adds debug flag, image tag group var

* fix(local-path-provisioner): moves image repo/tag to download role

* test(gce_centos7-flannel): enables local-path-provisioner in test case

* fix(addons): add image repo/tag to commented default values

* fix(local-path-provisioner): typo in jinja template for local path provisioner

* style(local-path-provisioner): debug flag condition re-formatted

* fix(local-path-provisioner): adds missing default value for debug flag

* fix(local-path-provisioner): syntax fix for debug if condition end

* fix(local-path-provisioner): jinja template syntax: if condition white space
2019-02-25 22:45:30 -08:00
hikoz 67832aada9 changed_when:false (#4189) 2019-02-25 20:09:30 -08:00
hikoz 3d25b4dfc1 30MiB for gpu-device-plugin (#4227)
* 30MiB for gpu-device-plugin

* use vars for easier configuration
2019-02-25 20:03:53 -08:00
Wong Hoi Sing Edison 1c12c19150 weave: Upgrade to 2.5.1 (#4248)
Upstream Changes:

  - weave 2.5.1 (https://github.com/weaveworks/weave/releases/tag/v2.5.1)

Our Changes:

  - Sync templates with upstream changes
2019-02-25 20:02:00 -08:00
Ryler Hockenbury 88249308a0 Add labels to vsphere cloud config (#4275) 2019-02-25 19:58:15 -08:00
Gabor Lekeny b4aaa7b908 Speed up tasks (#4278)
* fact gathering should run only once per node
* eliminate ansible version check, it is at the beginning of each
  playbook
2019-02-25 19:56:23 -08:00
Vasilis Remmas 81801ce23b Add master toleration flag in dashboard deployment (#4290) 2019-02-25 19:34:47 -08:00
Etienne 7dfa39483f Make container storage repository configurable (#4284) 2019-02-25 19:29:32 -08:00
Matthew Mosesohn b07641c3f3 Move kube_proxy_remove out of set_facts and set default (#4180) 2019-02-25 00:08:06 -08:00
Matthew Mosesohn 4638acfe81 Retry applying podsecurity policies (#4279) 2019-02-24 22:50:55 -08:00
Kaoet aadef80404 Upgrade to latest version of ubuntu-nvidia-driver-installer. (#4296)
The lastest version of ubuntu-nvidia-driver-installer contains a fix for
https://github.com/GoogleCloudPlatform/container-engine-accelerators/issues/90
which causes the installer pod to crash when driver is already loaded.
2019-02-24 22:22:48 -08:00
Frank Ritchie 9805fb7a34 Add flexvolume plugin dir to kubeadm kubelet (#4168)
This was already approved in #4106 but there are CI issues
with that PR due to references to kubernetes incubator.

After upgrading to Kubespray 2.8.1 with Kubeadm enabled Rook
Ceph volume provision failed due to the flexvolume plugin dir not
being correct. Adding the var fixed the issue
2019-02-20 15:02:02 -08:00
Maxime Guyot 323d788f48 Add support for --enable-skip-login in Dashboard (#4265) 2019-02-19 23:24:29 -08:00
Abdulaziz AlMalki eafab9636f fix wrong indent of oidc-username-prefix and oidc-groups-prefix in kubeadm config template (#4263) 2019-02-19 23:22:32 -08:00
Seungkyu Ahn 107bfb259a This PS is to fix the bug when Workers can't join the cluster (#4276)
because of etc-kubernetes-manifests not empty.
2019-02-19 22:13:59 -08:00
Rong Zhang d4a36aa55b
Merge pull request #4027 from riverzhang/kube-proxy
Add update server field in kube-proxy kubeconfig
2019-02-20 13:41:06 +08:00
Manuel Cintron 07b2894080 Adding ability to maintain existing Encryption Secrets at Rest. (#4255)
* Adding ability to maintain existing Encryption Secrets at Rest.

If secrets_encryption.yaml is present it will not be overriten with a new kube_encrypt_token.

This should allow for it to be set ahead of a playbook running or maintain it if cluster.yml is ran on the same cluster and the ansible host does not have access to the secrets.

* Setting existing kube_encrypt_token across all master nodes in case it was missing in one or more nodes.
2019-02-19 07:31:45 -08:00
Florent Monbillard 802ac377b8 Fix typo in task description (#4243) 2019-02-19 06:06:29 -08:00
Kaoet 23685b4537 Add image tag in "pause" container of nvidia driver installer. (#4247) 2019-02-17 21:02:30 -08:00
Chad Swenson e552be76ce Docker apt repo name fix (again) (#4246)
For some reason 18.09 packages are now prefixed with `5:` in the download.docker.com apt repos
Followup to #4236
2019-02-14 10:19:19 -08:00
Ryler Hockenbury eea22dfd40 Fix typo with docker-ce package versions (#4236) 2019-02-14 07:32:12 -08:00
Kaoet 192f4c4e96 Allow customizing container image path used in NVIDIA GPU addon. (#4229) 2019-02-14 03:51:38 -08:00
hikoz e03588f431 use swapon -s (#4216) 2019-02-14 02:35:17 -08:00
Chad Swenson 8872b2e0c6 Fix calico when kube_override_hostname is set (#4235)
This fixes an issue where the `nodename` in calico's cni config json can fall out of sync with the k8s node name used by the calico pod if `kube_override_hostname` is set
2019-02-13 16:02:48 -08:00
Florent Monbillard 061f5a313b Explicitely set etcd endpoint in kubeadm-images.yaml (#4063)
Currently, the task `container_download | download images for kubeadm config images` fetches etcd image even though it's not required (etcd is bootstrapped by kubespray, not kubeadm).

`kubeadm-images.yaml` is only a subset of `kubeadm-config.yaml`, therefore ``kubeadm config images pull` will try to get all this list (including etcd)

```
# kubeadm config images list --config /etc/kubernetes/kubeadm-images.yaml
k8s.gcr.io/kube-apiserver:v1.13.2
k8s.gcr.io/kube-controller-manager:v1.13.2
k8s.gcr.io/kube-scheduler:v1.13.2
k8s.gcr.io/kube-proxy:v1.13.2
k8s.gcr.io/pause:3.1
k8s.gcr.io/etcd:3.2.24
k8s.gcr.io/coredns:1.2.6
```

When using the `kubeadm-config.yaml` though, it doesn't list etcd image:

```
# kubeadm config images list --config /etc/kubernetes/kubeadm-config.yaml
k8s.gcr.io/kube-apiserver:v1.13.2
k8s.gcr.io/kube-controller-manager:v1.13.2
k8s.gcr.io/kube-scheduler:v1.13.2
k8s.gcr.io/kube-proxy:v1.13.2
k8s.gcr.io/pause:3.1
k8s.gcr.io/coredns:1.2.6
```

This change just adds the etcd endpoints in the `kubeadm-images.yaml` to give a hint to kubeadm it doesn't need etcd image for its boostrapping as etcd is "external".
I confess it is a ugly hack, a better way would be to use a single `kubeadm-config.yaml` for both tasks, but they are triggered by different roles (`kubeadm-images.yaml` is used by download, `kubeadm-config.yaml` by kubernetes/master) at different steps and I didn't want to refactor too many things to prevent breakage. 

This is specially useful for offline installation where a whitelist of container images is mirrored on a local private container registry. `k8s.gcr.io/etcd` and `quay.io/coreos/etcd`  are two different repositories hosting the same images but using *different tags*! 
* coreos/etcd:v3.2.24   
* k8s.gcr.io/etcd:3.2.24 (note the missing 'v' in the tag name)
2019-02-13 12:44:12 -08:00
Chad Swenson 2e2ed3bd35 [SECURITY] Docker patches for CVE-2019-5736 (#4223)
This updates docker 18.06 and 18.09 with the two patches released
yesterday to address the new runc exploit. Details here:
https://kubernetes.io/blog/2019/02/11/runc-and-cve-2019-5736/
2019-02-13 01:50:53 -08:00
Manuel Cintron 7697baf0da Omit does not work in the context of yum_repository proxy. The ansible documentation specifies to use _none_ to disable the global proxy setting. (#4225) 2019-02-12 16:46:32 -08:00
Sorin Sbarnea 22a5a00c49 Improve kubeadm join tasks (#4206)
Fix issue where `kubeadm join` could wait forever for joining.

Fix issue where `kubeadm join` were not reaching the user, making
impossible to find the cause of the failure.

New behaviour is to first attempt to join without bypassing the
verifications checks and to display them if needed.

If this fails it still attempts to join by ignoring the check in
order to make previous behavior.

A timeout of 60 seconds is allocated for a joining.

Related-bug: #3973
2019-02-12 13:42:56 -08:00
Maxime Guyot 6a33411d65 Add an option for helm init --wait (#4202) 2019-02-11 14:32:26 -08:00
hikoz 9a91ef8628 change permission after unarchive (#4191) 2019-02-11 14:21:38 -08:00
Sergey fbce6349c4 check kube_pods_subnet and kube_service_addresses to valid ip network range, not single ip address (#4188) 2019-02-11 14:12:06 -08:00
MarkusTeufelberger e2ad6aad5a bootstrap: rework role (#4045)
* bootstrap: rework role

* support being called from a non-root user
* run some commands in check mode
* unify spelling/task names

* bootstrap: fix wording of comments for check_mode: false

* bootstrap: remove setup-pipelining task
2019-02-11 14:04:27 -08:00
Chad Swenson 038a2eb862
Merge pull request #3949 from trogeat/patch-fix-missing-ca-cert-apiserver
kubespray: fix missing ca-certificate path in apiserver
2019-02-11 15:40:04 -06:00
Manuel Cintron 5d146e52fe If a centos or rhel node is not configured with the extras repo installation of required packages (python-httplib2 in particular) will fail later on. (#4213) 2019-02-11 13:27:02 -08:00
Jeff Bornemann c41c1e771f OCI Cloud Provider Update (#4186)
* OCI subnet AD 2 is not required for CCM >= 0.7.0

Reorganize OCI provider to generate configuration, rather than pull

Add pull secret option to OCI cloud provider

* Updated oci example to document new parameters
2019-02-11 12:08:53 -08:00
Karl 85b77f7c22 Remove Ubuntu Bionic specific vars file - breaks multi-arch (#3974) 2019-02-11 00:04:27 -08:00
Maxime Guyot 6b3f7306a4 Add support for arm64 images for hyperkube, kubeadm and cni_binary (#4176) 2019-02-09 02:08:57 -08:00
joakimr-axis 01d70f2c7c Update flannel version to v0.11.0 (#4190)
Change-Id: I27d670803bea82a68d5eb0e49d4677f4afdce55f
2019-02-07 04:33:01 -08:00
Chad Swenson 6878c2af4e Fix kube_hostname_override inconsistencies (#4185) 2019-02-06 22:20:11 -08:00
Bort Verwilst db2b76a22a update k8s to 1.13.3 (#4192)
* update k8s to 1.13.3

* update README as well
2019-02-06 10:48:05 -08:00
peerapach 69e5deeccc Fix newline issue of priorityClassName when enable tolerations (#4164) 2019-02-04 12:59:01 -08:00
Danny Kulchinsky 226d5ed7de [Calico] Define FELIX_KUBENODEPORTRANGES when kube-proxy in ipvs mode (#4173)
* Define FELIX_KUBENODEPORTRANGES when kube-proxy in ipvs mode

* ensure kube_apiserver_node_port_range is defined
2019-02-04 12:42:40 -08:00
Earl C. Ruby III 52e0aa7a80 Install the latest filesystem creation packages (#3904)
This PR ensures that the e2fsprogs and xfsprogs packages are
installed on all Kubernetes nodes and that the packages are
the latest versions. It also ensures that the nodes can
create XFS filesystems when necessary, since not all distros
install xfsprogs by default.

e2fsprogs - ext2/ext3/ext4 file system utilities
xfsprogs - Utilities for managing the XFS filesystem
2019-02-04 12:23:33 -08:00
peerapach bd9474bafd fix kubeadm-setup when enable access_ip (#4145) 2019-02-01 20:10:34 -08:00
Sorin Sbarnea 316b73178d Add timeout to Get current version of calico cluster version (#4149)
Avoid waiting forever for this task that should be very quick.

Fixes: #4148
2019-02-01 20:09:04 -08:00
Manuel Cintron 143e2272ff Fixing an issue where trying to install docker-ce-18.09 on rhel7 nodes (or potentially centos 7) without an enabled extras repo the installation will fail because container-selinux >= 2.9 is required. The check for container-selinux upfront should obviate the need for adding an extras repo if the node is able to find it from another source. (#4161) 2019-01-31 16:19:48 -08:00
Vasilis Remmas cd7924f8c9 Add oidc prefixes to kubeadm templates (#4159) 2019-01-31 15:31:43 -08:00
Erwan Miran 7f93a5a0f5 Fix deprecation warnings (#4130)
* use not deprecated ansible_play_hosts variable

* Using tests as filters is deprecated

* Fix deprecation warning about pkg list
2019-01-31 14:57:22 -08:00
Erwan Miran f6d60a7e89 Calico: Ability to define the default IPPool CIDR (instead of kube_pods_subnet) (#4131)
* Calico: Ability to define the default IPPool CIDR (instead of kube_pods_subnet)

* Documentation for calico_pool_cidr (and calico_advertise_cluster_ips which has been forgotten...)
2019-01-31 13:39:13 -08:00
Thomas Nys 68fd7e39da Set cluster DNS correctly in case of nodelocal dns cache (#3879)
* Set cluster DNS correctly in case of nodelocal dns cache

* Pass in cluster_ip based on dns mode

* Disable nodelocaldns by default

* Fix syntax error

* Fix syntax issue

* Add nodelocadns ip to vars of node installation

* Change location of nodelocaldns_ip

* Try to remove newlines from jinja template

* Add debug for config file

* Move parameter logic outside of template

* Adapt templates after feedback

* Remove debugging
2019-01-28 23:39:27 -08:00
wangxf a096761306 [PR-Calico]Support calico 3.4.0 (#4102)
* Suport calico 3.4.0

Signed-off-by: wangxf1987 <xiaofeix.wang@gmail.com>

* Remove symlink + cni conflist template when 3.3.0+, handle Canal, addition of install-cni: sidecar(3.3.0) or initontainer(3.4.0), KUBECONFIG_FILEPATH, calico_cert_dir, advertise cluster ips

* scheduler.alpha.kubernetes.io/critical-pod deprecated since 1.12
2019-01-28 11:03:49 -08:00
Florent Monbillard 2054a98cf7 Run kubeadm and hyperkube outside of local_release_dir (#4098)
Addressing the discussion started in #4064, this PR moves kubeadm and
hyperkube binaries to /usr/local/bin before running them on the master
nodes.

It is to address the case where local_release_dir points to /tmp
(kubespray default) and /tmp is mounted with noexec mode, preventing
any binaries to be run in that partition.

In role "node", we still move kubeadm to bin_dir only on the worker
nodes.
2019-01-28 02:00:49 -08:00
Sergey ce8ba1f170 create artifacts_dir (#4079) 2019-01-28 01:59:15 -08:00
Danny Kulchinsky 595d6427ac [Nodelocal DNS cache] Mount host /run/xtables.lock in nodelocaldns container (#4074)
* Mount host /run/xtables.lock in nodelocaldns container

* fix typo in nodelocaldns daemonset manifest yml

* Add prometheus scrape annotation, updateStrategy and reduce termination grace period

* fix indentation

* actually fix it..

* Bump k8s-dns-node-cache tag to 1.15.1 (fixes https://github.com/kubernetes/dns/issues/282)
2019-01-28 01:57:40 -08:00
Danny Kulchinsky 96688269f8 Support both --address and --bind-address for scheduler and controller-manager (#4112) 2019-01-27 23:43:34 -08:00
Rong Zhang 55aa58ee2e
Merge pull request #4025 from riverzhang/download-images
Fix kubeadm config images pull
2019-01-28 15:41:15 +08:00
Erwan Miran 556a8d68bc Set IP env var to autodetect when calico_ip_auto_method is defined (#4105) 2019-01-27 23:09:18 -08:00
rongzhang 3ed5f89cf5 Add update server field in kube-proxy kubeconfig
I know this is a bit hack.
If you use cloud LB, you can use kubeadm's controlPlaneEndpoint to configure kube-proxy's server field.
But for nginx-proxy, it didn't start when kubeadm init.
2019-01-28 14:45:43 +08:00
rongzhang 8d0158ceeb Fix kubeadm config images pull
Supported by kubeadm v1.11
2019-01-28 14:42:55 +08:00
Douglas Hellinger 4479cc48fe Introduce `calico_upgrade_url` var for Calico upgrade tool.
So that binary can be sourced from anywhere - not only github.
2019-01-23 16:19:27 +08:00
Chad Swenson 881be9b741 Fix epel_enabled and RHEL support in bootstrap-os
Looks like `epel_enabled` was not configured for the epel install in `bootstrap-centos.yml`. Also, there were no conditionals that would trigger bootstrap for RHEL.
2019-01-22 16:40:02 -06:00
Chad Swenson e2592f1ce2 Fix docker 18.09.1 systemd service
The `docker-ce` 18.09.1 packaging missed an `After` dependency on containerd in the systemd service. Upstream PR: https://github.com/docker/docker-ce-packaging/pull/290
2019-01-22 11:19:54 -06:00
Matthew Mosesohn 77d31e679a
fixup external kube-apiserver port (#4075) 2019-01-21 14:43:27 +03:00
Florent Monbillard decbcdc423 Use external LB IP for external api endpoint (#4060)
* Use external LB IP for external api endpoint

Use loadbalancer_apiserver.address instead of apiserver_loadbalancer_domain_name for kudadm init --apiserver-advertise-address argument

https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/#options states apiserver-advertise-address needs to be a IPv4 or IPv6 address

* only use loadbalancer IP if it is defined
2019-01-21 12:27:42 +03:00
Chad Swenson e3ffa21303
Merge pull request #4019 from chadswen/kubeadm-env
Fix PATH for kubeadm init
2019-01-18 11:27:57 -06:00
Chad Swenson f2ecda6f0f
Merge pull request #4059 from chadswen/helm-version-bump
Update helm version for security and stablity fixes
2019-01-18 11:25:42 -06:00
Chad Swenson 26f6f1f62e
Merge pull request #4050 from chadswen/docker-18.09.1
Bump docker 18.09 to the latest patch
2019-01-18 11:23:44 -06:00
Bort Verwilst f97cb4e761 Add 1.12.5 checksums (#4067) 2019-01-18 07:16:43 -08:00
Chad Swenson 405198acd0 Update helm version for security and stablity fixes
Helm v2.12.2 has fixes for a security vuln, and there have been several improvements since our last update.
2019-01-16 11:03:23 -06:00
Matthew Mosesohn eecaba6b84
Generate external admin.conf with kubeadm (#4056)
* Generate external admin.conf with kubeadm

* Fix apiserver sans
2019-01-16 16:30:50 +03:00
Thomas Rogeat 83e11f9ef7 kubespray: fix missing ca-certificate path in apiserver 2019-01-16 11:48:24 +01:00
Chad Swenson 5a7ac7e5c1
Merge pull request #3984 from dannyk81/calico_xtables_lock
[calico/canal] mount host's xtables lock and enable calico locking for <v3.2.1
2019-01-15 23:13:02 -06:00
Chad Swenson c15c933ce8 Bump docker 18.09 to the latest patch
Docker 18.09.1 is out and it includes some fixes that are quite critical for RHEL distros, details here: https://docs.docker.com/engine/release-notes/#18091
2019-01-15 13:54:58 -06:00
Chad Swenson 0697ab4b4f
Merge pull request #4048 from chadswen/readonly-writable-fix
Fix kubeadm config extra volumes
2019-01-15 13:02:04 -06:00
Chad Swenson 13e3e867ac Fix kubeadm config extra volumes
I found a potential use case where `writable` could be null and therfore
not treated like a boolean, so this adds an extra default statement to
avoid negating a non-boolean as boolean which would lead to undefined. refs #4020
2019-01-15 12:35:22 -06:00
Chad Swenson cc30220f01
Merge pull request #4044 from chadswen/lvp-cm-fix
Fix local-volume-provisioner configmap template
2019-01-15 09:08:08 -06:00
Danny Kulchinsky 257019d424 Mount host's xtable lock and enable calico lokcing for <v3.2.1 2019-01-14 17:16:29 -05:00
Chad Swenson 4959bfc1b3
Merge pull request #3950 from elementyang/pr-registry
fix registry_storage_class equals empty string
2019-01-14 15:45:09 -06:00
Chad Swenson 301671ae19
Merge pull request #4026 from riverzhang/bind-address
Use --bind-address instead of --address
2019-01-14 15:35:00 -06:00
Chad Swenson f10f7d0e84
Merge pull request #3975 from kskewes/arm64-urls
Update kubectl and etcd download urls for mult-arch
2019-01-14 15:04:29 -06:00
Chad Swenson 3ee5aa0d6b Fix local-volume-provisioner configmap template
Looks like the template is removing the trailing space between storage
class entries, and since CI only has one storage class we never hit this
issue. This change will prevent the yaml from printing on a single line
when multiple storage classes are defined.
2019-01-14 14:28:00 -06:00
Chad Swenson fce8712bff
Merge pull request #4033 from MarkusTeufelberger/pypy_portable
Use Pypy portable on coreos
2019-01-14 12:30:47 -06:00
Markus Teufelberger 87c9a871b9 bootstrap-os: use the systemd module to stop and mask locksmithd 2019-01-12 15:06:01 +01:00