You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

142 lines
7.3 KiB

  1. # Vagrant
  2. Assuming you have Vagrant 2.0+ installed with virtualbox or libvirt/qemu
  3. (vmware may work, but is untested) you should be able to launch a 3 node
  4. Kubernetes cluster by simply running `vagrant up`.
  5. This will spin up 3 VMs and install kubernetes on them.
  6. Once they are completed you can connect to any of them by running `vagrant ssh k8s-[1..3]`.
  7. To give an estimate of the expected duration of a provisioning run:
  8. On a dual core i5-6300u laptop with an SSD, provisioning takes around 13
  9. to 15 minutes, once the container images and other files are cached.
  10. Note that libvirt/qemu is recommended over virtualbox as it is quite a bit
  11. faster, especially during boot-up time.
  12. For proper performance a minimum of 12GB RAM is recommended.
  13. It is possible to run a 3 node cluster on a laptop with 8GB of RAM using
  14. the default Vagrantfile, provided you have 8GB zram swap configured and
  15. not much more than a browser and a mail client running.
  16. If you decide to run on such a machine, then also make sure that any tmpfs
  17. devices, that are mounted, are mostly empty and disable any swapfiles
  18. mounted on HDD/SSD or you will be in for some serious swap-madness.
  19. Things can get a bit sluggish during provisioning, but when that's done,
  20. the system will actually be able to perform quite well.
  21. ## Customize Vagrant
  22. You can override the default settings in the `Vagrantfile` either by
  23. directly modifying the `Vagrantfile` or through an override file.
  24. In the same directory as the `Vagrantfile`, create a folder called
  25. `vagrant` and create `config.rb` file in it.
  26. Example:
  27. ```ruby
  28. # vagrant/config.rb
  29. $instance_name_prefix = "kub"
  30. $vm_cpus = 1
  31. $num_instances = 3
  32. $os = "centos8-bento"
  33. $subnet = "10.0.20"
  34. $network_plugin = "flannel"
  35. $extra_vars = {
  36. dns_domain: my.custom.domain
  37. }
  38. # or
  39. $extra_vars = "path/to/extra/vars/file.yml"
  40. ```
  41. For all available options look at the Vagrantfile (search for "CONFIG")
  42. ## Use alternative OS for Vagrant
  43. By default, Vagrant uses Ubuntu 18.04 box to provision a local cluster.
  44. You may use an alternative supported operating system for your local cluster.
  45. Customize `$os` variable in `Vagrantfile` or as override, e.g.,:
  46. ```ShellSession
  47. echo '$os = "flatcar-stable"' >> vagrant/config.rb
  48. ```
  49. The supported operating systems for vagrant are defined in the `SUPPORTED_OS`
  50. constant in the `Vagrantfile`.
  51. ## File and image caching
  52. Kubespray can take quite a while to start on a laptop. To improve provisioning
  53. speed, the variable 'download_run_once' is set. This will make kubespray
  54. download all files and containers just once and then redistributes them to
  55. the other nodes and as a bonus, also cache all downloads locally and re-use
  56. them on the next provisioning run. For more information on download settings
  57. see [download documentation](/docs/advanced/downloads.md).
  58. ## Example use of Vagrant
  59. The following is an example of setting up and running kubespray using `vagrant`.
  60. Customize your settings as shown, above, then run the commands:
  61. ```ShellSession
  62. # use virtualenv to install all python requirements
  63. VENVDIR=venv
  64. $ virtualenv --python=/usr/bin/python3.7 $VENVDIR
  65. $ source $VENVDIR/bin/activate
  66. $ pip install -r requirements.txt
  67. $ vagrant up
  68. # Access the cluster
  69. $ export INV=.vagrant/provisionners/ansible/inventory
  70. $ export KUBECONFIG=${INV}/artifacts/admin.conf
  71. # make the kubectl binary available
  72. $ export PATH=$PATH:$PWD/$INV/artifacts
  73. ```
  74. If a vagrant run failed and you've made some changes to fix the issue causing
  75. the fail, here is how you would re-run ansible:
  76. ```ShellSession
  77. vagrant provision
  78. ```
  79. If all went well, you check if it's all working as expected:
  80. ```ShellSession
  81. $ kubectl get nodes
  82. NAME STATUS ROLES AGE VERSION
  83. kub-1 Ready control-plane,master 4m37s v1.22.5
  84. kub-2 Ready control-plane,master 4m7s v1.22.5
  85. kub-3 Ready <none> 3m7s v1.22.5
  86. ```
  87. Another nice test is the following:
  88. ```ShellSession
  89. $ kubectl get pods --all-namespaces -o wide
  90. NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
  91. kube-system coredns-8474476ff8-m2469 1/1 Running 0 2m45s 10.233.65.2 kub-2 <none> <none>
  92. kube-system coredns-8474476ff8-v5wzj 1/1 Running 0 2m41s 10.233.64.3 kub-1 <none> <none>
  93. kube-system dns-autoscaler-5ffdc7f89d-76tnv 1/1 Running 0 2m43s 10.233.64.2 kub-1 <none> <none>
  94. kube-system kube-apiserver-kub-1 1/1 Running 1 4m54s 10.0.20.101 kub-1 <none> <none>
  95. kube-system kube-apiserver-kub-2 1/1 Running 1 4m33s 10.0.20.102 kub-2 <none> <none>
  96. kube-system kube-controller-manager-kub-1 1/1 Running 1 5m1s 10.0.20.101 kub-1 <none> <none>
  97. kube-system kube-controller-manager-kub-2 1/1 Running 1 4m33s 10.0.20.102 kub-2 <none> <none>
  98. kube-system kube-flannel-9xgf5 1/1 Running 0 3m10s 10.0.20.102 kub-2 <none> <none>
  99. kube-system kube-flannel-l8jbl 1/1 Running 0 3m10s 10.0.20.101 kub-1 <none> <none>
  100. kube-system kube-flannel-zss4t 1/1 Running 0 3m10s 10.0.20.103 kub-3 <none> <none>
  101. kube-system kube-multus-ds-amd64-bhpc9 1/1 Running 0 3m2s 10.0.20.103 kub-3 <none> <none>
  102. kube-system kube-multus-ds-amd64-n6vl8 1/1 Running 0 3m2s 10.0.20.102 kub-2 <none> <none>
  103. kube-system kube-multus-ds-amd64-qttgs 1/1 Running 0 3m2s 10.0.20.101 kub-1 <none> <none>
  104. kube-system kube-proxy-2x4jl 1/1 Running 0 3m33s 10.0.20.101 kub-1 <none> <none>
  105. kube-system kube-proxy-d48r7 1/1 Running 0 3m33s 10.0.20.103 kub-3 <none> <none>
  106. kube-system kube-proxy-f45lp 1/1 Running 0 3m33s 10.0.20.102 kub-2 <none> <none>
  107. kube-system kube-scheduler-kub-1 1/1 Running 1 4m54s 10.0.20.101 kub-1 <none> <none>
  108. kube-system kube-scheduler-kub-2 1/1 Running 1 4m33s 10.0.20.102 kub-2 <none> <none>
  109. kube-system nginx-proxy-kub-3 1/1 Running 0 3m33s 10.0.20.103 kub-3 <none> <none>
  110. kube-system nodelocaldns-cg9tz 1/1 Running 0 2m41s 10.0.20.102 kub-2 <none> <none>
  111. kube-system nodelocaldns-htswt 1/1 Running 0 2m41s 10.0.20.103 kub-3 <none> <none>
  112. kube-system nodelocaldns-nsp7s 1/1 Running 0 2m41s 10.0.20.101 kub-1 <none> <none>
  113. local-path-storage local-path-provisioner-66df45bfdd-km4zg 1/1 Running 0 2m54s 10.233.66.2 kub-3 <none> <none>
  114. ```