You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

144 lines
7.2 KiB

  1. K8s DNS stack by Kubespray
  2. ======================
  3. For K8s cluster nodes, Kubespray configures a [Kubernetes DNS](http://kubernetes.io/docs/admin/dns/)
  4. [cluster add-on](http://releases.k8s.io/master/cluster/addons/README.md)
  5. to serve as an authoritative DNS server for a given ``dns_domain`` and its
  6. ``svc, default.svc`` default subdomains (a total of ``ndots: 5`` max levels).
  7. Other nodes in the inventory, like external storage nodes or a separate etcd cluster
  8. node group, considered non-cluster and left up to the user to configure DNS resolve.
  9. DNS variables
  10. =============
  11. There are several global variables which can be used to modify DNS settings:
  12. #### ndots
  13. ndots value to be used in ``/etc/resolv.conf``
  14. It is important to note that multiple search domains combined with high ``ndots``
  15. values lead to poor performance of DNS stack, so please choose it wisely.
  16. #### searchdomains
  17. Custom search domains to be added in addition to the cluster search domains (``default.svc.{{ dns_domain }}, svc.{{ dns_domain }}``).
  18. Most Linux systems limit the total number of search domains to 6 and the total length of all search domains
  19. to 256 characters. Depending on the length of ``dns_domain``, you're limitted to less then the total limit.
  20. Please note that ``resolvconf_mode: docker_dns`` will automatically add your systems search domains as
  21. additional search domains. Please take this into the accounts for the limits.
  22. #### nameservers
  23. This variable is only used by ``resolvconf_mode: host_resolvconf``. These nameservers are added to the hosts
  24. ``/etc/resolv.conf`` *after* ``upstream_dns_servers`` and thus serve as backup nameservers. If this variable
  25. is not set, a default resolver is chosen (depending on cloud provider or 8.8.8.8 when no cloud provider is specified).
  26. #### upstream_dns_servers
  27. DNS servers to be added *after* the cluster DNS. Used by all ``resolvconf_mode`` modes. These serve as backup
  28. DNS servers in early cluster deployment when no cluster DNS is available yet.
  29. DNS modes supported by Kubespray
  30. ============================
  31. You can modify how Kubespray sets up DNS for your cluster with the variables ``dns_mode`` and ``resolvconf_mode``.
  32. ## dns_mode
  33. ``dns_mode`` configures how Kubespray will setup cluster DNS. There are four modes available:
  34. #### coredns (default)
  35. This installs CoreDNS as the default cluster DNS for all queries.
  36. #### coredns_dual
  37. This installs CoreDNS as the default cluster DNS for all queries, plus a secondary CoreDNS stack.
  38. #### manual
  39. This does not install coredns, but allows you to specify
  40. `manual_dns_server`, which will be configured on nodes for handling Pod DNS.
  41. Use this method if you plan to install your own DNS server in the cluster after
  42. initial deployment.
  43. #### none
  44. This does not install any of DNS solution at all. This basically disables cluster DNS completely and
  45. leaves you with a non functional cluster.
  46. ## resolvconf_mode
  47. ``resolvconf_mode`` configures how Kubespray will setup DNS for ``hostNetwork: true`` PODs and non-k8s containers.
  48. There are three modes available:
  49. #### docker_dns (default)
  50. This sets up the docker daemon with additional --dns/--dns-search/--dns-opt flags.
  51. The following nameservers are added to the docker daemon (in the same order as listed here):
  52. * cluster nameserver (depends on dns_mode)
  53. * content of optional upstream_dns_servers variable
  54. * host system nameservers (read from hosts /etc/resolv.conf)
  55. The following search domains are added to the docker daemon (in the same order as listed here):
  56. * cluster domains (``default.svc.{{ dns_domain }}``, ``svc.{{ dns_domain }}``)
  57. * content of optional searchdomains variable
  58. * host system search domains (read from hosts /etc/resolv.conf)
  59. The following dns options are added to the docker daemon
  60. * ndots:{{ ndots }}
  61. * timeout:2
  62. * attempts:2
  63. For normal PODs, k8s will ignore these options and setup its own DNS settings for the PODs, taking
  64. the --cluster_dns (either coredns or coredns_dual, depending on dns_mode) kubelet option into account.
  65. For ``hostNetwork: true`` PODs however, k8s will let docker setup DNS settings. Docker containers which
  66. are not started/managed by k8s will also use these docker options.
  67. The host system name servers are added to ensure name resolution is also working while cluster DNS is not
  68. running yet. This is especially important in early stages of cluster deployment. In this early stage,
  69. DNS queries to the cluster DNS will timeout after a few seconds, resulting in the system nameserver being
  70. used as a backup nameserver. After cluster DNS is running, all queries will be answered by the cluster DNS
  71. servers, which in turn will forward queries to the system nameserver if required.
  72. #### host_resolvconf
  73. This activates the classic Kubespray behaviour that modifies the hosts ``/etc/resolv.conf`` file and dhclient
  74. configuration to point to the cluster dns server (either coredns or coredns_dual, depending on dns_mode).
  75. As cluster DNS is not available on early deployment stage, this mode is split into 2 stages. In the first
  76. stage (``dns_early: true``), ``/etc/resolv.conf`` is configured to use the DNS servers found in ``upstream_dns_servers``
  77. and ``nameservers``. Later, ``/etc/resolv.conf`` is reconfigured to use the cluster DNS server first, leaving
  78. the other nameservers as backups.
  79. Also note, existing records will be purged from the `/etc/resolv.conf`,
  80. including resolvconf's base/head/cloud-init config files and those that come from dhclient.
  81. #### none
  82. Does nothing regarding ``/etc/resolv.conf``. This leaves you with a cluster that works as expected in most cases.
  83. The only exception is that ``hostNetwork: true`` PODs and non-k8s managed containers will not be able to resolve
  84. cluster service names.
  85. ## Nodelocal DNS cache
  86. Setting ``enable_nodelocaldns`` to ``true`` will make pods reach out to the dns (core-dns) caching agent running on the same node, thereby avoiding iptables DNAT rules and connection tracking. The local caching agent will query kube-dns / core-dns (depending on what main DNS plugin is configured in your cluster) for cache misses of cluster hostnames(cluster.local suffix by default).
  87. More information on the rationale behind this implementation can be found [here](https://github.com/kubernetes/enhancements/blob/master/keps/sig-network/0030-nodelocal-dns-cache.md).
  88. Limitations
  89. -----------
  90. * Kubespray has yet ways to configure Kubedns addon to forward requests SkyDns can
  91. not answer with authority to arbitrary recursive resolvers. This task is left
  92. for future. See [official SkyDns docs](https://github.com/skynetservices/skydns)
  93. for details.
  94. * There is
  95. [no way to specify a custom value](https://github.com/kubernetes/kubernetes/issues/33554)
  96. for the SkyDNS ``ndots`` param via an
  97. [option for KubeDNS](https://github.com/kubernetes/kubernetes/blob/master/cmd/kube-dns/app/options/options.go)
  98. add-on, while SkyDNS supports it though.
  99. * the ``searchdomains`` have a limitation of a 6 names and 256 chars
  100. length. Due to default ``svc, default.svc`` subdomains, the actual
  101. limits are a 4 names and 239 chars respectively.
  102. * the ``nameservers`` have a limitation of a 3 servers, although there
  103. is a way to mitigate that with the ``upstream_dns_servers``,
  104. see below. Anyway, the ``nameservers`` can take no more than a two
  105. custom DNS servers because of one slot is reserved for a Kubernetes
  106. cluster needs.