Compare commits

..

122 Commits

Author SHA1 Message Date
Aleksandr Didenko ec77f046fb Update CCP installation instructions 2016-07-13 16:48:11 +02:00
Aleksandr Didenko 66a178c614 Another minor fix in readme 2016-07-13 16:08:02 +02:00
Aleksandr Didenko 95a2bcdd9d Fixed docs 2016-07-13 16:07:11 +02:00
Aleksandr Didenko 7ab62170e0 All patches are merged, no need to pull reviews 2016-07-13 15:33:02 +02:00
Aleksandr Didenko 8c37d0aa1f Update the list of patchsets for ccp-neutron 2016-07-13 13:38:12 +02:00
Aleksandr Didenko 438a4bdeca Update ccp-neutron patch sets 2016-07-12 19:14:17 +02:00
Aleksandr Didenko da0a973dd4 Added a list of openstack cli commands for demo 2016-07-12 18:57:50 +02:00
root d8bef773ee Add pulling of custom patches for ccp-neutron 2016-07-12 16:56:41 +00:00
Aleksandr Didenko 64608f06cf Bugfixes
- Don't create registry pod if it already exists
- Fix shell commands
2016-07-12 17:56:40 +02:00
Aleksandr Didenko d450e1f06f Refactor CCP deployment part 2016-07-12 17:20:58 +02:00
Aleksandr Didenko 569d0081d3 Minor fixes in deploy-config.yaml style and doc 2016-07-12 12:15:01 +02:00
Aleksandr Didenko ba5466cacf Added some OpenStack CLI command examples 2016-07-11 17:26:06 +02:00
Aleksandr Didenko 6b094db607 Fix nodes in label-nodes.sh 2016-07-11 14:16:02 +02:00
Aleksandr Didenko d0dd69399e Change nodes generation for Vagrant
Master IP is .10, nodes start from .11
2016-07-11 13:00:57 +02:00
root f33f447b3d Fix dynamic ansible inventory 2016-07-11 10:28:25 +00:00
Aleksandr Didenko efaf6328a2 Add python to provision 2016-07-08 19:11:13 +02:00
Aleksandr Didenko a5a34c98a5 Move node bootstrap to ansible 2016-07-08 18:52:42 +02:00
Aleksandr Didenko 6a287973d9 Update default nodes specs for vagrant lab 2016-07-08 18:32:30 +02:00
root d066f0c9e9 Move args to env 2016-07-08 16:04:47 +00:00
Aleksandr Didenko b7f3ff5ce9 We need python-ipaddr on master node for dynamic inv 2016-07-08 17:52:26 +02:00
root 48ec698314 Merge branch 'master' of https://github.com/adidenko/vagrant-k8s 2016-07-08 15:46:31 +00:00
root 737a83788f Add reading group variables 2016-07-08 15:45:02 +00:00
Aleksandr Didenko 9471173f6a Update docs 2016-07-08 17:23:43 +02:00
Aleksandr Didenko c4e3266031 Fix a bug in nodes_to_inv.py 2016-07-08 17:22:52 +02:00
Aleksandr Didenko 8d3abdb489 Switch to dynamic ansible inventory
We don't need to install and use kargo cli anymore.
2016-07-08 17:10:09 +02:00
Aleksandr Didenko e89f4ac7ee Add labeling nodes 2016-07-08 15:04:31 +02:00
Aleksandr Didenko 99db440287 Added check for CCP images build 2016-07-08 14:32:43 +02:00
Aleksandr Didenko e6358d825e Bugfixes in ccp playbooks 2016-07-08 14:23:00 +02:00
Aleksandr Didenko 8b3112d287 Switch to using upstream fuel-ccp project 2016-07-07 19:50:16 +02:00
Aleksandr Didenko b4dfd8c973 Update to deploy of ccp 2016-07-06 13:17:08 +02:00
Aleksandr Didenko a153ac231a Refactor to use new deploy config in ccp 2016-07-06 13:14:23 +02:00
Aleksandr Didenko 5b4c365b8c Change kube version to 1.2.4 2016-07-05 16:26:51 +02:00
Aleksandr Didenko a08fb131fb Remove unneeded curl from netchecker deploy script 2016-07-05 15:24:09 +02:00
Aleksandr Didenko ba2c3f052f Fix namespace deletion 2016-07-05 15:14:14 +02:00
Aleksandr Didenko c8a488cfbe Added missing parameter to sed command 2016-07-05 14:53:28 +02:00
Aleksandr Didenko fad80d8595 Add another workaround for hostnetwork pods 2016-07-05 14:51:01 +02:00
Aleksandr Didenko 21f1c82fb0 Setup a ip ro workaround for cluste IPs 2016-07-05 12:43:35 +02:00
Aleksandr Didenko 6ec957a255 Add DNS servers 2016-07-04 17:16:56 +02:00
Aleksandr Didenko 76b49bfe30 Enable build back 2016-07-04 15:51:35 +02:00
Aleksandr Didenko 7d14763cf0 Add prebuilt images as option 2016-07-04 15:22:30 +02:00
Aleksandr Didenko 4c300a57b5 Add ip route workaround for DNS clusterIP 2016-07-04 15:13:49 +02:00
Aleksandr Didenko e68d6575cd Add dnsutils to vagrant nodes 2016-07-04 11:32:33 +02:00
Aleksandr Didenko 11b6e31c55 Fix mcp.conf for prebuilt packages 2016-07-01 18:13:19 +02:00
Aleksandr Didenko 4d295d567b Switch to using prebuilt CCP images 2016-07-01 17:19:41 +02:00
Aleksandr Didenko ca8ef29ae4 Add temp fix for bug in mcp builder 2016-07-01 16:34:20 +02:00
Aleksandr Didenko 9be65f8c19 Update mcp.conf according to upstream 2016-07-01 16:24:42 +02:00
Alex Didenko b70b8a7c39 Update README.md 2016-07-01 16:02:47 +02:00
Aleksandr Didenko 687cc01151 Fix typos in readme 2016-07-01 15:59:21 +02:00
Aleksandr Didenko 25b986ede7 Fix list nesting 2016-07-01 15:58:39 +02:00
Aleksandr Didenko 1e294b25c1 Update readme 2016-07-01 15:57:41 +02:00
Aleksandr Didenko 5c369d6d40 Added deploy-netchecker.sh script 2016-07-01 15:55:57 +02:00
Aleksandr Didenko 7b1e29f855 Comment out hacking of resolv.conf 2016-07-01 15:32:45 +02:00
Aleksandr Didenko 80ee1f2d9e Hack resolv.conf for ALL containers 2016-06-30 19:04:41 +02:00
Aleksandr Didenko 87856513c6 Hack for resolv.conf is back, a bit different this time 2016-06-30 19:01:01 +02:00
Aleksandr Didenko f7f560de2e No need to hack resolv.conf in dockerfiles 2016-06-30 18:18:33 +02:00
Aleksandr Didenko 7a8ead07d8 Added some packages for provisioning 2016-06-30 18:01:45 +02:00
Aleksandr Didenko 46f99befee Rename chapter in readme 2016-06-30 17:19:21 +02:00
Aleksandr Didenko 3563dbe9e8 Add useful k8s commands 2016-06-30 17:18:36 +02:00
Aleksandr Didenko fec601a238 Minor improvements in commands and readme 2016-06-30 16:56:52 +02:00
Aleksandr Didenko aad4edaf47 Fix paths in playbooks 2016-06-30 16:50:29 +02:00
Aleksandr Didenko bb3a57a719 Hack all problem images and put resolv.conf there 2016-06-30 16:12:26 +02:00
Aleksandr Didenko 6be93a3b87 Another fix in path for inventory 2016-06-30 15:56:57 +02:00
Aleksandr Didenko 333d9daea8 Fix readme 2016-06-30 15:47:28 +02:00
Aleksandr Didenko 8b53ff8ef7 Remove kpm from provisioning, kargo installs it 2016-06-30 15:46:02 +02:00
Aleksandr Didenko 8334a9e1e4 Fix paths to kargo 2016-06-30 15:35:00 +02:00
Aleksandr Didenko f1e5bc81f8 Fix path to nodes list 2016-06-30 15:32:46 +02:00
Aleksandr Didenko 26646b4a79 Fix cwd 2016-06-30 15:31:30 +02:00
Aleksandr Didenko eddd1251eb Fix paths 2016-06-30 15:28:36 +02:00
Aleksandr Didenko ba710ade23 Fix typo 2016-06-30 15:27:25 +02:00
Aleksandr Didenko 25d19720c0 Huge refactoring
Split scripts and instructuins into two parts: lab preparation and
deployment.
2016-06-30 15:22:39 +02:00
Aleksandr Didenko 17e3108b0c Save output of mcp-microservice to separate logs 2016-06-30 12:08:14 +02:00
Aleksandr Didenko f304dd4cf3 Switch to new vagrant image and update ccp-pull 2016-06-30 11:53:54 +02:00
Aleksandr Didenko 7dcc7c31f6 Update CCP repos 2016-06-29 17:45:51 +02:00
Aleksandr Didenko 4ca2931ae9 Add new virtual drive for /var/lib/docker
Otherwise 10G is not enough to host all the CCP images and it
leads to deployment failures.
2016-06-29 16:40:34 +02:00
Aleksandr Didenko 9744972f4a Replace sleep with wait loop 2016-06-29 14:21:34 +02:00
Aleksandr Didenko f770ae82e6 Updated readme in examples 2016-06-29 12:48:21 +02:00
Aleksandr Didenko aa9578ba99 Updated examples readme 2016-06-29 12:46:32 +02:00
Aleksandr Didenko 898e79a49e Update CCP refs 2016-06-29 11:30:29 +02:00
Aleksandr Didenko 8d80265392 Use ansible instead of kargo-cli to deploy k8s 2016-06-29 11:24:20 +02:00
Aleksandr Didenko 8acd4396d6 Remove some hardcode for CCP installation 2016-06-28 18:26:51 +02:00
Aleksandr Didenko a47f9394bb Updated CCP example with nodePort 2016-06-28 16:56:34 +02:00
Aleksandr Didenko 5cc37db4bf Minor improvements in nodes list generation 2016-06-28 16:54:34 +02:00
Aleksandr Didenko 3eb2ec101e Added curl to bootstrap scripts 2016-06-28 15:32:15 +02:00
Aleksandr Didenko 84d85e41a9 Added example for Horizone exposing via nodePort 2016-06-28 14:42:36 +02:00
Aleksandr Didenko 96add56527 Update node labels according to new CCP nodeSelector 2016-06-28 14:06:53 +02:00
Aleksandr Didenko d7f9d4a590 Fix bug in shell command 2016-06-28 12:58:05 +02:00
Aleksandr Didenko 26e61fc9be Another update related to CCP upstream changes 2016-06-28 10:56:46 +02:00
Aleksandr Didenko 7546d75513 Update reviews for ccp and fix missing script 2016-06-28 10:55:23 +02:00
Aleksandr Didenko 1fb6f36e9c Remove extra space 2016-06-27 18:04:14 +02:00
Aleksandr Didenko df4fe074f0 Added CCP deployment scripts 2016-06-27 17:57:29 +02:00
Aleksandr Didenko 0c9826c60f Added kubedash external service to examples 2016-06-23 16:53:13 +02:00
Aleksandr Didenko d7a11887f6 Added example how to expose k8s dashboard 2016-06-23 16:43:11 +02:00
Aleksandr Didenko 39dd4c1aaa New playbooks for k8s service and examples
- kubedns moved to playbooks dir
- new ansible playbooks added for kubedash and kube-dashboard
- examples for k8s deployments and services added
2016-06-22 18:43:39 +02:00
Aleksandr Didenko 9c5c0f2697 Minor update in README 2016-06-20 16:12:18 +02:00
Aleksandr Didenko 62a1925664 Fix in network names 2016-06-20 15:03:24 +02:00
Aleksandr Didenko 7f247754f9 Add network names 2016-06-20 15:02:11 +02:00
Aleksandr Didenko 7a53c32a3b Fix networks and add some debug tools
- Create two networks: public (NATed) and private (isolated)
- Add some debug tools to minion nodes
2016-06-20 14:43:20 +02:00
Aleksandr Didenko 0e48ce51ce Multiple fixes to deployment scripts and lab
- Clean up private/pub keys from k8s nodes
- Install kpm on k8s nodes, not on master node
2016-06-20 12:02:25 +02:00
Aleksandr Didenko bc29db7bd2 Fixes and improvements
- Move kubedns to a separate mini playbook
- Fix custom.yaml location
2016-06-20 11:06:01 +02:00
Aleksandr Didenko 8d8622bbb9 Switch to ansible-2.1 2016-06-20 09:58:09 +02:00
Aleksandr Didenko 943edb6dd2 Fix permissions and kargo custom.yaml 2016-06-20 09:13:07 +02:00
Aleksandr Didenko c81457c617 Update README 2016-06-16 17:50:44 +02:00
Aleksandr Didenko 4c6f85b8ae Start counting k8s nodes from 2
First node is our master node.
2016-06-16 14:42:53 +02:00
Aleksandr Didenko 60fa68e5f7 Bugfix: return missing deploy command 2016-06-16 14:36:07 +02:00
Aleksandr Didenko ea5b40ae0e Fix ssh config 2016-06-16 14:28:29 +02:00
Aleksandr Didenko daf02e029c Fix nodes list 2016-06-16 13:00:51 +02:00
Aleksandr Didenko 3824493b1d Added README 2016-06-16 12:36:58 +02:00
Aleksandr Didenko cdfbcc1046 Remove SSH keys and generate them instead 2016-06-16 12:29:33 +02:00
Aleksandr Didenko 62e98bd4b0 Removing deprecated stuff 2016-06-16 12:19:27 +02:00
Aleksandr Didenko 43b2b5b464 Bugfixing for newly added code with kargo-clu support 2016-06-16 12:00:18 +02:00
Aleksandr Didenko 531f611ea3 Add support for deployment via kargo-cli 2016-06-16 11:49:17 +02:00
Aleksandr Didenko b9ed54812b Remove controlled by puppet line 2016-06-15 11:50:09 +02:00
Aleksandr Didenko 338749be16 Added README for master node with usefill commands 2016-06-15 11:48:50 +02:00
Aleksandr Didenko a2f3048e7a Added some packages to bootstrap script 2016-06-15 10:26:44 +02:00
Aleksandr Didenko b009ca3ff8 Fix inv dir copy command 2016-06-14 19:10:20 +02:00
Aleksandr Didenko c714660c0b Updates to use new inventory settings 2016-06-14 19:04:09 +02:00
Aleksandr Didenko 70519e2c5a Refactor inventroty for the lab 2016-06-14 19:02:27 +02:00
Aleksandr Didenko d365fab9ec Add inventory.cfg download to bootstrap script 2016-06-14 18:51:43 +02:00
Aleksandr Didenko 785169b934 Added inventory.cfg for the lab 2016-06-14 18:49:31 +02:00
Aleksandr Didenko 48e2062d92 Added apt-get commands to bootstrap-node 2016-06-14 18:40:41 +02:00
Aleksandr Didenko d894529f07 Update ssh/config on master node 2016-06-14 18:30:30 +02:00
Oleksandr Didenko 7e08800876 Initial commit 2016-06-14 18:21:56 +02:00
113 changed files with 909 additions and 22348 deletions
+2
View File
@@ -0,0 +1,2 @@
ssh
nodes
-49
View File
@@ -1,49 +0,0 @@
[submodule "roles/apps/k8s-kube-ui"]
path = roles/apps/k8s-kube-ui
url = https://github.com/ansibl8s/k8s-kube-ui.git
branch = v1.0
[submodule "roles/apps/k8s-kubedns"]
path = roles/apps/k8s-kubedns
url = https://github.com/ansibl8s/k8s-kubedns.git
branch = v1.0
[submodule "roles/apps/k8s-common"]
path = roles/apps/k8s-common
url = https://github.com/ansibl8s/k8s-common.git
branch = v1.0
[submodule "roles/apps/k8s-redis"]
path = roles/apps/k8s-redis
url = https://github.com/ansibl8s/k8s-redis.git
branch = v1.0
[submodule "roles/apps/k8s-elasticsearch"]
path = roles/apps/k8s-elasticsearch
url = https://github.com/ansibl8s/k8s-elasticsearch.git
[submodule "roles/apps/k8s-fabric8"]
path = roles/apps/k8s-fabric8
url = https://github.com/ansibl8s/k8s-fabric8.git
branch = v1.0
[submodule "roles/apps/k8s-memcached"]
path = roles/apps/k8s-memcached
url = https://github.com/ansibl8s/k8s-memcached.git
branch = v1.0
[submodule "roles/apps/k8s-postgres"]
path = roles/apps/k8s-postgres
url = https://github.com/ansibl8s/k8s-postgres.git
branch = v1.0
[submodule "roles/apps/k8s-kubedash"]
path = roles/apps/k8s-kubedash
url = https://github.com/ansibl8s/k8s-kubedash.git
[submodule "roles/apps/k8s-heapster"]
path = roles/apps/k8s-heapster
url = https://github.com/ansibl8s/k8s-heapster.git
[submodule "roles/apps/k8s-influxdb"]
path = roles/apps/k8s-influxdb
url = https://github.com/ansibl8s/k8s-influxdb.git
[submodule "roles/apps/k8s-kube-logstash"]
path = roles/apps/k8s-kube-logstash
url = https://github.com/ansibl8s/k8s-kube-logstash.git
[submodule "roles/apps/k8s-etcd"]
path = roles/apps/k8s-etcd
url = https://github.com/ansibl8s/k8s-etcd.git
[submodule "roles/apps/k8s-rabbitmq"]
path = roles/apps/k8s-rabbitmq
url = https://github.com/ansibl8s/k8s-rabbitmq.git
-41
View File
@@ -1,41 +0,0 @@
sudo: required
dist: trusty
language: python
python: "2.7"
addons:
hosts:
- node1
env:
- SITE=cluster.yml
before_install:
- sudo apt-get update -qq
install:
# Install Ansible.
- sudo -H pip install ansible
- sudo -H pip install netaddr
cache:
directories:
- $HOME/releases
- $HOME/.cache/pip
before_script:
- export PATH=$PATH:/usr/local/bin
script:
# Check the role/playbook's syntax.
- "sudo -H ansible-playbook -i inventory/local-tests.cfg $SITE --syntax-check"
# Run the role/playbook with ansible-playbook.
- "sudo -H ansible-playbook -i inventory/local-tests.cfg $SITE --connection=local"
# Run the role/playbook again, checking to make sure it's idempotent.
- >
sudo -H ansible-playbook -i inventory/local-tests.cfg $SITE --connection=local
| tee /dev/stderr | grep -q 'changed=0.*failed=0'
&& (echo 'Idempotence test: pass' && exit 0)
|| (echo 'Idempotence test: fail' && exit 1)
+115 -233
View File
@@ -1,279 +1,161 @@
[![Build Status](https://travis-ci.org/ansibl8s/setup-kubernetes.svg)](https://travis-ci.org/ansibl8s/setup-kubernetes)
kubernetes-ansible
========
vagrant-k8s
===========
Scripts to create libvirt lab with vagrant and prepare some stuff for `k8s` deployment with `kargo`.
Install and configure a Multi-Master/HA kubernetes cluster including network plugin.
### Requirements
Tested on **Debian Wheezy/Jessie** and **Ubuntu** (14.10, 15.04, 15.10).
Should work on **RedHat/Fedora/Centos** platforms (to be tested)
* The target servers must have access to the Internet in order to pull docker imaqes.
* The firewalls are not managed, you'll need to implement your own rules the way you used to.
* Ansible v1.9.x and python-netaddr
Requirements
------------
### Components
* [kubernetes](https://github.com/kubernetes/kubernetes/releases) v1.1.3
* [etcd](https://github.com/coreos/etcd/releases) v2.2.2
* [calicoctl](https://github.com/projectcalico/calico-docker/releases) v0.13.0
* [flanneld](https://github.com/coreos/flannel/releases) v0.5.5
* [docker](https://www.docker.com/) v1.9.1
* `libvirt`
* `vagrant`
* `vagrant-libvirt` plugin (`vagrant plugin install vagrant-libvirt`)
* `$USER` should be able to connect to libvirt (test with `virsh list --all`)
Quickstart
-------------------------
The following steps will quickly setup a kubernetes cluster with default configuration.
These defaults are good for tests purposes.
Vargant lab preparation
-----------------------
Edit the inventory according to the number of servers
```
[downloader]
localhost ansible_connection=local ansible_python_interpreter=python2
* Change default IP pool for vagrant networks if you want:
[kube-master]
10.115.99.31
[etcd]
10.115.99.31
10.115.99.32
10.115.99.33
[kube-node]
10.115.99.32
10.115.99.33
[k8s-cluster:children]
kube-node
kube-master
```bash
export VAGRANT_POOL="10.100.0.0/16"
```
Run the playbook
```
ansible-playbook -i inventory/inventory.cfg cluster.yml -u root
* Clone this repo
```bash
git clone https://github.com/adidenko/vagrant-k8s
cd vagrant-k8s
```
You can jump directly to "*Available apps, installation procedure*"
Ansible
-------------------------
### Variables
The main variables to change are located in the directory ```inventory/group_vars/all.yml```.
### Inventory
Below is an example of an inventory.
Note : The bgp vars local_as and peers are not mandatory if the var **'peer_with_router'** is set to false
By default this variable is set to false and therefore all the nodes are configure in **'node-mesh'** mode.
In node-mesh mode the nodes peers with all the nodes in order to exchange routes.
* Prepare the virtual lab:
```bash
vagrant up
```
[downloader]
localhost ansible_connection=local ansible_python_interpreter=python2
Deployment on a lab
-------------------
[kube-master]
node1 ansible_ssh_host=10.99.0.26
node2 ansible_ssh_host=10.99.0.27
* Login to master node and sudo to root:
[etcd]
node1 ansible_ssh_host=10.99.0.26
node2 ansible_ssh_host=10.99.0.27
node3 ansible_ssh_host=10.99.0.4
[kube-node]
node2 ansible_ssh_host=10.99.0.27
node3 ansible_ssh_host=10.99.0.4
node4 ansible_ssh_host=10.99.0.5
node5 ansible_ssh_host=10.99.0.36
node6 ansible_ssh_host=10.99.0.37
[paris]
node1 ansible_ssh_host=10.99.0.26
node3 ansible_ssh_host=10.99.0.4 local_as=xxxxxxxx
node4 ansible_ssh_host=10.99.0.5 local_as=xxxxxxxx
[new-york]
node2 ansible_ssh_host=10.99.0.27
node5 ansible_ssh_host=10.99.0.36 local_as=xxxxxxxx
node6 ansible_ssh_host=10.99.0.37 local_as=xxxxxxxx
[k8s-cluster:children]
kube-node
kube-master
```bash
vagrant ssh $USER-k8s-00
sudo su -
```
### Playbook
```
---
- hosts: downloader
sudo: no
roles:
- { role: download, tags: download }
- hosts: k8s-cluster
roles:
- { role: kubernetes/preinstall, tags: preinstall }
- { role: docker, tags: docker }
- { role: kubernetes/node, tags: node }
- { role: etcd, tags: etcd }
- { role: dnsmasq, tags: dnsmasq }
- { role: network_plugin, tags: ['calico', 'flannel', 'network'] }
- hosts: kube-master
roles:
- { role: kubernetes/master, tags: master }
* Clone this repo
```bash
git clone https://github.com/adidenko/vagrant-k8s ~/mcp
```
### Run
It is possible to define variables for different environments.
For instance, in order to deploy the cluster on 'dev' environment run the following command.
```
ansible-playbook -i inventory/dev/inventory.cfg cluster.yml -u root
* Install required software and pull needed repos:
```bash
cd ~/mcp
./bootstrap-master.sh
```
Kubernetes
-------------------------
### Multi master notes
* You can choose where to install the master components. If you want your master node to act both as master (api,scheduler,controller) and node (e.g. accept workloads, create pods ...),
the server address has to be present on both groups 'kube-master' and 'kube-node'.
* Check `nodes` list and make sure you have SSH access to them
* Almost all kubernetes components are running into pods except *kubelet*. These pods are managed by kubelet which ensure they're always running
* For safety reasons, you should have at least two master nodes and 3 etcd servers
* Kube-proxy doesn't support multiple apiservers on startup ([Issue 18174](https://github.com/kubernetes/kubernetes/issues/18174)). An external loadbalancer needs to be configured.
In order to do so, some variables have to be used '**loadbalancer_apiserver**' and '**apiserver_loadbalancer_domain_name**'
### Network Overlay
You can choose between 2 network plugins. Only one must be chosen.
* **flannel**: gre/vxlan (layer 2) networking. ([official docs](https://github.com/coreos/flannel))
* **calico**: bgp (layer 3) networking. ([official docs](http://docs.projectcalico.org/en/0.13/))
The choice is defined with the variable '**kube_network_plugin**'
### Expose a service
There are several loadbalancing solutions.
The one i found suitable for kubernetes are [Vulcand](http://vulcand.io/) and [Haproxy](http://www.haproxy.org/)
My cluster is working with haproxy and kubernetes services are configured with the loadbalancing type '**nodePort**'.
eg: each node opens the same tcp port and forwards the traffic to the target pod wherever it is located.
Then Haproxy can be configured to request kubernetes's api in order to loadbalance on the proper tcp port on the nodes.
Please refer to the proper kubernetes documentation on [Services](https://github.com/kubernetes/kubernetes/blob/release-1.0/docs/user-guide/services.md)
### Check cluster status
#### Kubernetes components
* Check the status of the processes
```
systemctl status kubelet
```bash
cd ~/mcp
cat nodes
ansible all -m ping -i nodes_to_inv.py
```
* Check the logs
```
journalctl -ae -u kubelet
* Deploy k8s using kargo playbooks
```bash
cd ~/mcp
./deploy-k8s.kargo.sh
```
* Check the NAT rules
```
iptables -nLv -t nat
* Deploy OpenStack CCP:
```bash
cd ~/mcp
# Build CCP images
ansible-playbook -i nodes_to_inv.py playbooks/ccp-build.yaml
# Deploy CCP
ansible-playbook -i nodes_to_inv.py playbooks/ccp-deploy.yaml
```
For the master nodes you'll have to see the docker logs for the apiserver
```
docker logs [apiserver docker id]
* Wait for CCP deployment to complete
```bash
# On k8s master node
# Check CCP pods, all should become running
kubectl --namespace=openstack get pods -o wide
# Check CCP jobs status, wait until all complete
kubectl --namespace=openstack get jobs
```
* Check Horizon:
### Available apps, installation procedure
```bash
# On k8s master node check nodePort of Horizon service
HORIZON_PORT=$(kubectl --namespace=openstack get svc/horizon -o go-template='{{(index .spec.ports 0).nodePort}}')
echo $HORIZON_PORT
There are two ways of installing new apps
#### Ansible galaxy
Additionnal apps can be installed with ```ansible-galaxy```.
ou'll need to edit the file '*requirements.yml*' in order to chose needed apps.
The list of available apps are available [there](https://github.com/ansibl8s)
For instance it is **strongly recommanded** to install a dns server which resolves kubernetes service names.
In order to use this role you'll need the following entries in the file '*requirements.yml*'
Please refer to the [k8s-kubedns readme](https://github.com/ansibl8s/k8s-kubedns) for additionnal info.
```
- src: https://github.com/ansibl8s/k8s-common.git
path: roles/apps
# version: v1.0
- src: https://github.com/ansibl8s/k8s-kubedns.git
path: roles/apps
# version: v1.0
```
**Note**: the role common is required by all the apps and provides the tasks and libraries needed.
And empty the apps directory
```
rm -rf roles/apps/*
# Access Horizon via nodePort
curl -i -s $ANY_K8S_NODE_IP:$HORIZON_PORT
```
Then download the roles with ansible-galaxy
```
ansible-galaxy install -r requirements.yml
Working with kubernetes
-----------------------
* Login to one of your kube-master nodes and run:
```bash
# List images in registry
curl -s 127.0.0.1:31500/v2/_catalog | python -mjson.tool
# Check CCP jobs status
kubectl --namespace=openstack get jobs
# Check CCP pods
kubectl --namespace=openstack get pods -o wide
```
#### Git submodules
Alternatively the roles can be installed as git submodules.
That way is easier if you want to do some changes and commit them.
* Troubleshooting
You can list available submodules with the following command:
```
grep path .gitmodules | sed 's/.*= //'
```bash
# Get logs from pod
kubectl --namespace=openstack logs $POD_NAME
# Exec command from pod
kubectl --namespace=openstack exec $POD_NAME -- cat /etc/resolv.conf
kubectl --namespace=openstack exec $POD_NAME -- curl http://etcd-client:2379/health
# Run a container
docker run -t -i 127.0.0.1:31500/mcp/neutron-dhcp-agent /bin/bash
```
In order to install the dns addon you'll need to follow these steps
```
git submodule init roles/apps/k8s-common roles/apps/k8s-kubedns
git submodule update
* Network checker
```bash
cd ~/mcp
./deploy-netchecker.sh
# or in openstack namespace
./deploy-netchecker.sh openstack
```
Finally update the playbook ```apps.yml``` with the chosen roles, and run it
```
...
- hosts: kube-master
roles:
- { role: apps/k8s-kubedns, tags: ['kubedns', 'apps'] }
...
```
* CCP
```
ansible-playbook -i inventory/inventory.cfg apps.yml -u root
```
```bash
# Run a bash in one of containers
docker run -t -i 127.0.0.1:31500/mcp/nova-base /bin/bash
# Inside container export credentials
export OS_USERNAME=admin
export OS_PASSWORD=password
export OS_TENANT_NAME=admin
export OS_REGION_NAME=RegionOne
export OS_AUTH_URL=http://keystone:35357
#### Calico networking
Check if the calico-node container is running
# Run CLI commands
openstack service list
neutron agent-list
```
docker ps | grep calico
```
The **calicoctl** command allows to check the status of the network workloads.
* Check the status of Calico nodes
```
calicoctl status
```
* Show the configured network subnet for containers
```
calicoctl pool show
```
* Show the workloads (ip addresses of containers and their located)
```
calicoctl endpoint show --detail
```
#### Flannel networking
Congrats ! now you can walk through [kubernetes basics](http://kubernetes.io/v1.1/basicstutorials.html)
Vendored
+115
View File
@@ -0,0 +1,115 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :
pool = ENV["VAGRANT_POOL"] || "10.250.0.0/16"
ENV["VAGRANT_DEFAULT_PROVIDER"] = "libvirt"
prefix = pool.gsub(/\.\d+\.\d+\/16$/, "")
$num_instances = 4
$vm_memory = 6144
$vm_cpus = 2
$master_memory = 1024
$master_cpus = 1
$user = ENV["USER"]
$public_subnet = prefix.to_s + ".0"
$private_subnet = prefix.to_s + ".1"
$mgmt_cidr = prefix.to_s + ".2.0/24"
$neutron_subnet = "172.30.250"
$instance_name_prefix = "#{$user}-k8s"
# Boxes with libvirt provider support:
#$box = "yk0/ubuntu-xenial" #900M
#$box = "centos/7"
#$box = "nrclark/xenial64-minimal-libvirt"
$box = "peru/ubuntu-16.04-server-amd64"
# Create SSH keys for future lab
system 'bash vagrant-scripts/ssh-keygen.sh'
# Create nodes list for future kargo deployment
nodes=""
(1..$num_instances-1).each do |i|
ip = "#{$private_subnet}.#{i+10}"
nodes = "#{nodes}#{ip}\n"
end
File.open("nodes", 'w') { |file| file.write(nodes) }
# Create the lab
Vagrant.configure("2") do |config|
(0..$num_instances-1).each do |i|
# First node would be master node
master = i == 0
config.ssh.insert_key = false
vm_name = "%s-%02d" % [$instance_name_prefix, i]
config.vm.define vm_name do |test_vm|
test_vm.vm.box = $box
test_vm.vm.hostname = vm_name
# Libvirt provider settings
test_vm.vm.provider :libvirt do |domain|
domain.uri = "qemu+unix:///system"
if master
domain.memory = $master_memory
domain.cpus = $master_cpus
else
domain.memory = $vm_memory
domain.cpus = $vm_cpus
end
domain.driver = "kvm"
domain.host = "localhost"
domain.connect_via_ssh = false
domain.username = $user
domain.storage_pool_name = "default"
domain.nic_model_type = "e1000"
domain.management_network_name = "#{$instance_name_prefix}-mgmt-net"
domain.management_network_address = $mgmt_cidr
domain.nested = true
domain.cpu_mode = "host-passthrough"
domain.volume_cache = "unsafe"
domain.disk_bus = "virtio"
# DISABLED: switched to new box which has 100G / partition
#domain.storage :file, :type => 'qcow2', :bus => 'virtio', :size => '20G', :device => 'vdb'
end
# Networks and interfaces
ip = "#{$private_subnet}.#{i+10}"
pub_ip = "#{$public_subnet}.#{i+10}"
# "public" network with nat forwarding
test_vm.vm.network :private_network,
:ip => pub_ip,
:model_type => "e1000",
:libvirt__network_name => "#{$instance_name_prefix}-public",
:libvirt__dhcp_enabled => false,
:libvirt__forward_mode => "nat"
# "private" isolated network
test_vm.vm.network :private_network,
:ip => ip,
:model_type => "e1000",
:libvirt__network_name => "#{$instance_name_prefix}-private",
:libvirt__dhcp_enabled => false,
:libvirt__forward_mode => "none"
# "neutron" isolated network
test_vm.vm.network :private_network,
:ip => "#{$neutron_subnet}.#{i+10}",
:model_type => "e1000",
:libvirt__network_name => "#{$instance_name_prefix}-neutron",
:libvirt__dhcp_enabled => false,
:libvirt__forward_mode => "none"
# Provisioning
config.vm.provision "file", source: "ssh", destination: "~/ssh"
if master
config.vm.provision "nodes", type: "file", source: "nodes", destination: "/var/tmp/nodes"
config.vm.provision "bootstrap", type: "shell", path: "vagrant-scripts/provision-master.sh"
else
config.vm.provision "bootstrap", type: "shell", path: "vagrant-scripts/provision-node.sh"
end
end
end
end
-29
View File
@@ -1,29 +0,0 @@
---
- hosts: kube-master
roles:
# System
- { role: apps/k8s-kubedns, tags: ['kubedns', 'kube-system'] }
# Databases
- { role: apps/k8s-postgres, tags: 'postgres' }
- { role: apps/k8s-elasticsearch, tags: 'elasticsearch' }
- { role: apps/k8s-memcached, tags: 'memcached' }
- { role: apps/k8s-redis, tags: 'redis' }
# Msg Broker
- { role: apps/k8s-rabbitmq, tags: 'rabbitmq' }
# Monitoring
- { role: apps/k8s-influxdb, tags: ['influxdb', 'kube-system']}
- { role: apps/k8s-heapster, tags: ['heapster', 'kube-system']}
- { role: apps/k8s-kubedash, tags: ['kubedash', 'kube-system']}
# logging
- { role: apps/k8s-kube-logstash, tags: 'kube-logstash'}
# Console
- { role: apps/k8s-fabric8, tags: 'fabric8' }
- { role: apps/k8s-kube-ui, tags: ['kube-ui', 'kube-system']}
# ETCD
- { role: apps/k8s-etcd, tags: 'etcd'}
+11
View File
@@ -0,0 +1,11 @@
#!/bin/bash
set -e
INVENTORY="nodes_to_inv.py"
echo "Createing repository and CCP images, it may take a while..."
ansible-playbook -i $INVENTORY playbooks/ccp-build.yaml
echo "Deploying up OpenStack CCP..."
ansible-playbook -i $INVENTORY playbooks/ccp-deploy.yaml
+22
View File
@@ -0,0 +1,22 @@
#!/bin/bash
# Packages
apt-get --yes update
apt-get --yes upgrade
apt-get --yes install git screen vim telnet tcpdump python-setuptools gcc python-dev python-pip libssl-dev libffi-dev software-properties-common curl python-netaddr
# Get ansible-2.1+, vanilla ubuntu-16.04 ansible (2.0.0.2) is broken due to https://github.com/ansible/ansible/issues/13876
ansible --version || (
apt-add-repository -y ppa:ansible/ansible
apt-get update
apt-get install -y ansible
)
# Copy/create nodes list
test -f ./nodes || cp /var/tmp/nodes ./nodes
# Either pull or copy microservices repos
cp -a /var/tmp/microservices* ./ccp/ || touch /var/tmp/ccp-download
# Pull kargo
git clone https://github.com/kubespray/kargo ~/kargo
+2
View File
@@ -0,0 +1,2 @@
microservices-repos
microservices
+16
View File
@@ -0,0 +1,16 @@
[DEFAULT]
deploy_config = /root/ccp/deploy-config.yaml
[builder]
push = True
[registry]
address = "127.0.0.1:31500"
[kubernetes]
namespace = "openstack"
[repositories]
skip_empty = True
protocol = https
port = 443
+6
View File
@@ -0,0 +1,6 @@
configs:
public_interface: "eth1"
private_interface: "eth2"
neutron_external_interface: "eth3"
neutron_logging_debug: "true"
neutron_plugin_agent: "openvswitch"
+25
View File
@@ -0,0 +1,25 @@
#!/bin/bash
set -e
# FIXME: hardcoded roles
declare -A nodes
nodes=( \
["node1"]="openstack-controller=true"
["node2"]="openstack-compute=true"
["node3"]="openstack-compute=true"
)
label_nodes() {
all_label='openstack-compute-controller=true'
for i in "${!nodes[@]}"
do
node=$i
label=${nodes[$i]}
kubectl get nodes $node --show-labels | grep -q "$label" || kubectl label nodes $node $label
kubectl get nodes $node --show-labels | grep -q "$all_label" || kubectl label nodes $node $all_label
done
}
label_nodes
+16
View File
@@ -0,0 +1,16 @@
apiVersion: v1
kind: Pod
metadata:
name: registry
labels:
app: registry
spec:
containers:
- name: registry
image: registry:2
env:
imagePullPolicy: Always
ports:
- containerPort: 5000
hostPort: 5000
+15
View File
@@ -0,0 +1,15 @@
kind: "Service"
apiVersion: "v1"
metadata:
name: "registry"
spec:
selector:
app: "registry"
ports:
-
protocol: "TCP"
port: 5000
targetPort: 5000
nodePort: 31500
type: "NodePort"
-18
View File
@@ -1,18 +0,0 @@
---
- hosts: downloader
sudo: no
roles:
- { role: download, tags: download }
- hosts: k8s-cluster
roles:
- { role: kubernetes/preinstall, tags: preinstall }
- { role: docker, tags: docker }
- { role: kubernetes/node, tags: node }
- { role: etcd, tags: etcd }
- { role: dnsmasq, tags: dnsmasq }
- { role: network_plugin, tags: ['calico', 'flannel', 'network'] }
- hosts: kube-master
roles:
- { role: kubernetes/master, tags: master }
+13
View File
@@ -0,0 +1,13 @@
# Kubernetes version
kube_version: "v1.2.4"
# Switch network to calico
kube_network_plugin: "calico"
# Kube-proxy should be iptables for calico
kube_proxy_mode: "iptables"
# Use non-tmpfs tmp dir
local_release_dir: "/var/tmp/releases"
# Upstream DNS servers with mirantis.net
upstream_dns_servers:
- 8.8.8.8
- 8.8.4.4
- /mirantis.net/172.18.32.6
+19
View File
@@ -0,0 +1,19 @@
#!/bin/bash
INVENTORY="nodes_to_inv.py"
echo "Installing requirements on nodes..."
ansible-playbook -i $INVENTORY playbooks/bootstrap-nodes.yaml
echo "Running deployment..."
ansible-playbook -i $INVENTORY /root/kargo/cluster.yml -e @custom.yaml
deploy_res=$?
if [ "$deploy_res" -eq "0" ]; then
echo "Setting up kubedns..."
ansible-playbook -i $INVENTORY playbooks/kubedns.yaml
echo "Setting up kubedashboard..."
ansible-playbook -i $INVENTORY playbooks/kubedashboard.yaml
echo "Setting up ip route work-around for DNS clusterIP availability..."
ansible-playbook -i $INVENTORY playbooks/ipro_for_cluster_ips.yaml
fi
+36
View File
@@ -0,0 +1,36 @@
#!/bin/bash
if [ -n "$1" ] ; then
NS="--namespace=$1"
fi
kubectl get nodes || exit 1
echo "Installing netchecker server"
git clone https://github.com/adidenko/netchecker-server
pushd netchecker-server
pushd docker
docker build -t 127.0.0.1:31500/netchecker/server:latest .
docker push 127.0.0.1:31500/netchecker/server:latest
popd
kubectl create -f netchecker-server_pod.yaml $NS
kubectl create -f netchecker-server_svc.yaml $NS
popd
echo "Installing netchecker agents"
git clone https://github.com/adidenko/netchecker-agent
pushd netchecker-agent
pushd docker
docker build -t 127.0.0.1:31500/netchecker/agent:latest .
docker push 127.0.0.1:31500/netchecker/agent:latest
popd
kubectl get nodes | grep Ready | awk '{print $1}' | xargs -I {} kubectl label nodes {} netchecker=agent
NUMNODES=`kubectl get nodes --show-labels | grep Ready | grep netchecker=agent | wc -l`
sed -e "s/replicas:.*/replicas: $NUMNODES/g" -i netchecker-agent_rc.yaml
kubectl create -f netchecker-agent_rc.yaml $NS
popd
echo "DONE"
echo
echo "use the following command to check agents:"
echo "curl -s -X GET 'http://localhost:31081/api/v1/agents/' | python -mjson.tool"
+25
View File
@@ -0,0 +1,25 @@
CCP examples
============
Some examples for Openstack CCP.
Expose Horizon
==============
* Get nodePort of Horizon service:
```bash
echo $(kubectl --namespace=openstack get svc/horizon -o go-template='{{(index .spec.ports 0).nodePort}}')
```
* NAT on your router/jump-box to any k8s minion public IP and nodePort to provide external access:
```bash
iptables -t nat -I PREROUTING -p tcp --dport 8080 -j DNAT --to-destination 10.210.0.12:32643
iptables -t nat -I POSTROUTING -d 10.210.0.12 ! -s 10.210.0.0/24 -j MASQUERADE
iptables -I FORWARD -d 10.210.0.12 -j ACCEPT
```
Where `10.210.0.12` is IP of one of your k8s minions and `32643` is nodePort of Horizon service.
* You can do the same for novnc:
```bash
echo $(kubectl --namespace=openstack get svc/nova-novncproxy -o go-template='{{(index .spec.ports 0).nodePort}}')
```
+36
View File
@@ -0,0 +1,36 @@
# This script should be executed inside k8s:
# docker run -t -i 127.0.0.1:31500/mcp/nova-base /bin/bash
export OS_USERNAME=admin
export OS_PASSWORD=password
export OS_TENANT_NAME=admin
export OS_REGION_NAME=RegionOne
export OS_AUTH_URL=http://keystone:35357
# Key
nova keypair-add test > test.pem
chmod 600 test.pem
# Flavor
nova flavor-create demo --is-public true auto 128 2 1
# Image
curl -O http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img
glance image-create --name cirros --disk-format qcow2 --container-format bare --file cirros-0.3.4-x86_64-disk.img
# Aggregates
node2=`openstack hypervisor list | grep -o '[a-z]\+-k8s-02'`
node3=`openstack hypervisor list | grep -o '[a-z]\+-k8s-03'`
nova aggregate-create n2 n2
nova aggregate-add-host n2 $node2
nova aggregate-create n3 n3
nova aggregate-add-host n3 $node3
# Network
neutron net-create net1 --provider:network-type vxlan
neutron subnet-create net1 172.20.0.0/24 --name subnet1
# Instances
net_id=`neutron net-list | grep net1 | awk '{print $2}'`
nova boot ti02 --image cirros --flavor demo --nic net-id=$net_id --key-name test --availability-zone n2
nova boot ti03 --image cirros --flavor demo --nic net-id=$net_id --key-name test --availability-zone n3
@@ -0,0 +1,45 @@
Examples how to expose k8s services
===================================
Exposing dashboard via frontend and externalIPs
-----------------------------------------------
* Edit `kubernetes-dashboard.yaml` and update `externalIPs` to the list of external IPs of your k8s minions
* Run:
```bash
kubectl create -f kubernetes-dashboard.yaml --namespace=kube-system
```
* Access:
```bash
curl $ANY_MINION_EXTERNAL_IP:9090
```
Exposing dashboard via nodePort
-------------------------------
* Get nodePort of the service:
```bash
echo $(kubectl --namespace=kube-system get svc/kubernetes-dashboard -o go-template='{{(index .spec.ports 0).nodePort}}')
```
* NAT on your router/jump-box to any k8s minion public IP and nodePort to provide external access:
```bash
iptables -t nat -I PREROUTING -p tcp --dport 9090 -j DNAT --to-destination 10.210.0.12:32005
iptables -t nat -I POSTROUTING -d 10.210.0.12 ! -s 10.210.0.0/24 -j MASQUERADE
iptables -I FORWARD -d 10.210.0.12 -j ACCEPT
```
Where `10.210.0.12` is public IP of one of your k8s minions and `32005` is nodePort of `kubernetes-dashboard` service.
* Access:
```bash
curl 10.210.0.12:9090
```
@@ -0,0 +1,22 @@
apiVersion: v1
kind: Service
metadata:
name: kubedash-frontend
labels:
app: kubedash-frontend
tier: frontend
spec:
externalIPs:
- 10.210.0.12
- 10.210.0.13
- 10.210.0.14
- 10.210.0.15
- 10.210.0.16
- 10.210.0.17
ports:
- name: http
port: 8289
protocol: TCP
targetPort: 8289
selector:
name: kubedash
@@ -0,0 +1,22 @@
apiVersion: v1
kind: Service
metadata:
name: dashboard-frontend
labels:
app: dashboard-frontend
tier: frontend
spec:
externalIPs:
- 10.210.0.12
- 10.210.0.13
- 10.210.0.14
- 10.210.0.15
- 10.210.0.16
- 10.210.0.17
ports:
- name: http
port: 9090
protocol: TCP
targetPort: 9090
selector:
app: kubernetes-dashboard
@@ -0,0 +1,18 @@
Nginx example with external IPs
===============================
* Edit `nginx-frontend.yaml` and update `externalIPs` to the list of external IPs of your k8s minions
* Deploy:
```bash
kubectl create -f nginx-backends.yaml
kubectl create -f nginx-frontend.yaml
```
* Check:
```bash
curl $ANY_MINION_EXTERNAL_IP
```
@@ -0,0 +1,24 @@
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-backend
spec:
replicas: 3
template:
metadata:
labels:
app: nginx-backend
tier: backend
spec:
containers:
- name: nginx
image: nginx
resources:
requests:
cpu: 100m
memory: 100Mi
env:
- name: GET_HOSTS_FROM
value: dns
ports:
- containerPort: 80
@@ -0,0 +1,22 @@
apiVersion: v1
kind: Service
metadata:
name: nginx-frontend
labels:
app: nginx-frontend
tier: frontend
spec:
externalIPs:
- 10.210.0.12
- 10.210.0.13
- 10.210.0.14
- 10.210.0.15
- 10.210.0.16
- 10.210.0.17
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx-backend
-86
View File
@@ -1,86 +0,0 @@
# Directory where the binaries will be installed
bin_dir: /usr/local/bin
# Where the binaries will be downloaded.
# Note: ensure that you've enough disk space (about 1G)
local_release_dir: "/tmp/releases"
# Cluster Loglevel configuration
kube_log_level: 2
# Users to create for basic auth in Kubernetes API via HTTP
kube_users:
kube:
pass: changeme
role: admin
# root:
# pass: changeme
# role: admin
# Kubernetes cluster name, also will be used as DNS domain
cluster_name: cluster.local
# set this variable to calico if needed. keep it empty if flannel is used
kube_network_plugin: calico
# Kubernetes internal network for services, unused block of space.
kube_service_addresses: 10.233.0.0/18
# internal network. When used, it will assign IP
# addresses from this range to individual pods.
# This network must be unused in your network infrastructure!
kube_pods_subnet: 10.233.64.0/18
# internal network total size (optional). This is the prefix of the
# entire network. Must be unused in your environment.
# kube_network_prefix: 18
# internal network node size allocation (optional). This is the size allocated
# to each node on your network. With these defaults you should have
# room for 4096 nodes with 254 pods per node.
kube_network_node_prefix: 24
# With calico it is possible to distributed routes with border routers of the datacenter.
peer_with_router: false
# Warning : enabling router peering will disable calico's default behavior ('node mesh').
# The subnets of each nodes will be distributed by the datacenter router
# The port the API Server will be listening on.
kube_apiserver_ip: "{{ kube_service_addresses|ipaddr('net')|ipaddr(1)|ipaddr('address') }}"
kube_apiserver_port: 443 # (https)
kube_apiserver_insecure_port: 8080 # (http)
# Internal DNS configuration.
# Kubernetes can create and mainatain its own DNS server to resolve service names
# into appropriate IP addresses. It's highly advisable to run such DNS server,
# as it greatly simplifies configuration of your applications - you can use
# service names instead of magic environment variables.
# You still must manually configure all your containers to use this DNS server,
# Kubernetes won't do this for you (yet).
# Upstream dns servers used by dnsmasq
upstream_dns_servers:
- 8.8.8.8
- 4.4.8.8
#
# # Use dns server : https://github.com/ansibl8s/k8s-skydns/blob/master/skydns-README.md
dns_setup: true
dns_domain: "{{ cluster_name }}"
#
# # Ip address of the kubernetes dns service
dns_server: "{{ kube_service_addresses|ipaddr('net')|ipaddr(2)|ipaddr('address') }}"
# For multi masters architecture:
# kube-proxy doesn't support multiple apiservers for the time being so you'll need to configure your own loadbalancer
# This domain name will be inserted into the /etc/hosts file of all servers
# configuration example with haproxy :
# listen kubernetes-apiserver-https
# bind 10.99.0.21:8383
# option ssl-hello-chk
# mode tcp
# timeout client 3h
# timeout server 3h
# server master1 10.99.0.26:443
# server master2 10.99.0.27:443
# balance roundrobin
# apiserver_loadbalancer_domain_name: "lb-apiserver.kubernetes.local"
-10
View File
@@ -1,10 +0,0 @@
#---
#peers:
# -router_id: "10.99.0.34"
# as: "65xxx"
# - router_id: "10.99.0.35"
# as: "65xxx"
#
#loadbalancer_apiserver:
# address: "10.99.0.44"
# port: "8383"
-10
View File
@@ -1,10 +0,0 @@
#---
#peers:
# -router_id: "10.99.0.2"
# as: "65xxx"
# - router_id: "10.99.0.3"
# as: "65xxx"
#
#loadbalancer_apiserver:
# address: "10.99.0.21"
# port: "8383"
-32
View File
@@ -1,32 +0,0 @@
[downloader]
localhost ansible_connection=local ansible_python_interpreter=python2
[kube-master]
node1 ansible_ssh_host=10.99.0.26
node2 ansible_ssh_host=10.99.0.27
[etcd]
node1 ansible_ssh_host=10.99.0.26
node2 ansible_ssh_host=10.99.0.27
node3 ansible_ssh_host=10.99.0.4
[kube-node]
node2 ansible_ssh_host=10.99.0.27
node3 ansible_ssh_host=10.99.0.4
node4 ansible_ssh_host=10.99.0.5
node5 ansible_ssh_host=10.99.0.36
node6 ansible_ssh_host=10.99.0.37
[paris]
node1 ansible_ssh_host=10.99.0.26
node3 ansible_ssh_host=10.99.0.4 local_as=xxxxxxxx
node4 ansible_ssh_host=10.99.0.5 local_as=xxxxxxxx
[new-york]
node2 ansible_ssh_host=10.99.0.27
node5 ansible_ssh_host=10.99.0.36 local_as=xxxxxxxx
node6 ansible_ssh_host=10.99.0.37 local_as=xxxxxxxx
[k8s-cluster:children]
kube-node
kube-master
-17
View File
@@ -1,17 +0,0 @@
node1 ansible_connection=local local_release_dir={{ansible_env.HOME}}/releases
[downloader]
node1
[kube-master]
node1
[etcd]
node1
[kube-node]
node1
[k8s-cluster:children]
kube-node
kube-master
+97
View File
@@ -0,0 +1,97 @@
#!/usr/bin/env python
# A simple dynamic replacemant of 'kargo prepare'
# Generates ansible inventory from a list of IPs in 'nodes' file.
import argparse
import json
import os
import yaml
def read_nodes_from_file(filename):
f = open(filename, 'r')
content = [x.strip('\n') for x in f.readlines()]
return content
def read_vars_from_file(src="/root/kargo/inventory/group_vars/all.yml"):
with open(src, 'r') as f:
content = yaml.load(f)
return content
def nodes_to_hash(nodes_list, masters, group_vars):
nodes = {
'all': {
'hosts': [],
'vars': group_vars
},
'etcd': {
'hosts': [],
},
'kube-master': {
'hosts': [],
},
'kube-node': {
'hosts': [],
},
'k8s-cluster': {
'children': ['kube-node', 'kube-master']
},
'_meta': {
'hostvars': {}
}
}
i = 1
for node_ip in nodes_list:
node_name = "node%s" % i
nodes['all']['hosts'].append(node_name)
nodes['_meta']['hostvars'][node_name] = {
'ansible_ssh_host': node_ip,
'ip': node_ip,
}
nodes['kube-node']['hosts'].append(node_name)
if i <= masters:
nodes['kube-master']['hosts'].append(node_name)
if i <= 3:
nodes['etcd']['hosts'].append(node_name)
i += 1
return nodes
def main():
parser = argparse.ArgumentParser(description='Kargo inventory simulator')
parser.add_argument('--list', action='store_true')
parser.add_argument('--host', default=False)
args = parser.parse_args()
# Read params from ENV since ansible does not support passing args to dynamic inv scripts
if os.environ.get('K8S_NODES_FILE'):
nodes_file = os.environ['K8S_NODES_FILE']
else:
nodes_file = 'nodes'
if os.environ.get('K8S_MASTERS'):
masters = int(os.environ['K8S_MASTERS'])
else:
masters = 2
if os.environ.get('KARGO_GROUP_VARS'):
vars_file = os.environ['KARGO_GROUP_VARS']
else:
vars_file = "/root/kargo/inventory/group_vars/all.yml"
nodes_list = read_nodes_from_file(nodes_file)
if len(nodes_list) < 3:
print "Error: requires at least 3 nodes"
return
nodes = nodes_to_hash(nodes_list, masters, read_vars_from_file(vars_file))
if args.host:
print json.dumps(nodes['_meta']['hostvars'][args.host])
else:
print json.dumps(nodes)
if __name__ == "__main__":
main()
+17
View File
@@ -0,0 +1,17 @@
- hosts: all
tasks:
- name: Install packages
package: name={{ item }} state=latest
with_items:
- python-pip
- screen
- vim
- telnet
- tcpdump
- traceroute
- iperf3
- nmap
- ethtool
- curl
- git
- dnsutils
+69
View File
@@ -0,0 +1,69 @@
- hosts: kube-master
pre_tasks:
- name: Download fuel-ccp
git:
repo: https://git.openstack.org/openstack/fuel-ccp
dest: /usr/local/src/fuel-ccp
version: master
- name: Upload ccp configs to master nodes
synchronize:
src: ../ccp/
dest: /root/ccp/
tasks:
- name: Install CCP cli tool
shell: pip install -U fuel-ccp/
args:
chdir: /usr/local/src
creates: /usr/local/bin/mcp-microservices
- name: Get pods
shell: kubectl get pods
register: get_pod
run_once: true
- name: Get services
shell: kubectl get svc
register: get_svc
run_once: true
- name: Create registry pod
shell: kubectl create -f registry_pod.yaml
args:
chdir: /root/ccp
run_once: true
when: get_pod.stdout.find('registry') == -1
- name: Create registry svc
shell: kubectl create -f registry_svc.yaml
args:
chdir: /root/ccp
run_once: true
when: get_svc.stdout.find('registry') == -1
- name: Fetch CCP images
shell: mcp-microservices --config-file=/root/ccp/ccp.conf fetch
run_once: true
# - name: Patch fuel-ccp-neutron
# run_once: true
# args:
# chdir: /root/microservices-repos/fuel-ccp-neutron
# shell: git fetch https://git.openstack.org/openstack/fuel-ccp-neutron {{ item }} && git cherry-pick FETCH_HEAD
# with_items:
# - "refs/changes/96/340496/6"
- name: Build CCP images
shell: mcp-microservices --config-file=/root/ccp/ccp.conf build
run_once: true
- hosts: k8s-cluster
tasks:
- name: Check number of built images
shell: test $(curl -s 127.0.0.1:31500/v2/_catalog | python -mjson.tool | grep mcp/ | wc -l) -ge 29
+27
View File
@@ -0,0 +1,27 @@
- hosts: kube-master
pre_tasks:
- name: Rsync CCP configs
synchronize:
src: ../ccp/
dest: /root/ccp/
tasks:
- name: Label nodes
shell: ./label-nodes.sh
args:
chdir: /root/ccp
run_once: true
- name: Get namespaces
shell: kubectl get namespace
register: get_ns
run_once: true
- name: Deploy CCP
shell: mcp-microservices --config-file=/root/ccp/ccp.conf deploy
args:
chdir: /root/ccp
run_once: true
when: get_ns.stdout.find('openstack') == -1
+24
View File
@@ -0,0 +1,24 @@
# FXIME: add persistent routing rule
- hosts: kube-master
tasks:
- name: Get kube service net
shell: grep KUBE_SERVICE_ADDRESSES /etc/kubernetes/kube-apiserver.env | grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}/[0-9]{1,2}\b"
register: kube_service_addresses
run_once: true
- hosts: all
tasks:
- name: Get local IP
shell: "calicoctl status | grep IP: | awk '{print $2}'"
register: local_ip
- name: Get route
shell: ip ro ls | grep "^{{ hostvars[groups['kube-master'][0]]['kube_service_addresses']['stdout'] }}" || echo ""
register: local_route
- name: Clean up route
shell: ip ro del {{ hostvars[groups['kube-master'][0]]['kube_service_addresses']['stdout'] }} || true
when: local_route.stdout.find('{{ local_ip.stdout }}') == -1
- name: Setup route
shell: ip ro add {{ hostvars[groups['kube-master'][0]]['kube_service_addresses']['stdout'] }} via {{ local_ip.stdout }}
when: local_route.stdout.find('{{ local_ip.stdout }}') == -1
- name: Add openstack namespace to resolv.conf
shell: grep openstack.svc.cluster.local /etc/resolv.conf || sed '/^search / s/$/ openstack.svc.cluster.local/' -i /etc/resolv.conf
+5
View File
@@ -0,0 +1,5 @@
- hosts: kube-master
tasks:
- name: setup-kubedns
shell: kpm deploy kube-system/kubedash --namespace=kube-system
run_once: true
+5
View File
@@ -0,0 +1,5 @@
- hosts: kube-master
tasks:
- name: setup-kubedns
shell: kpm deploy kube-system/kubernetes-dashboard --namespace=kube-system
run_once: true
+5
View File
@@ -0,0 +1,5 @@
- hosts: kube-master
tasks:
- name: setup-kubedns
shell: kpm deploy kube-system/kubedns --namespace=kube-system
run_once: true
-41
View File
@@ -1,41 +0,0 @@
---
- src: https://github.com/ansibl8s/k8s-common.git
path: roles/apps
version: v1.0
- src: https://github.com/ansibl8s/k8s-kubedns.git
path: roles/apps
version: v1.0
#- src: https://github.com/ansibl8s/k8s-kube-ui.git
# path: roles/apps
# version: v1.0
#
#- src: https://github.com/ansibl8s/k8s-fabric8.git
# path: roles/apps
# version: v1.0
#
#- src: https://github.com/ansibl8s/k8s-elasticsearch.git
# path: roles/apps
# # version: v1.0
#
#- src: https://github.com/ansibl8s/k8s-redis.git
# path: roles/apps
# # version: v1.0
#
#- src: https://github.com/ansibl8s/k8s-memcached.git
# path: roles/apps
# version: v1.0
#
#- src: https://github.com/ansibl8s/k8s-postgres.git
# path: roles/apps
# version: v1.0
#
#- src: https://github.com/ansibl8s/k8s-heapster.git
# path: roles/apps
#
#- src: https://github.com/ansibl8s/k8s-influxdb.git
# path: roles/apps
#
#- src: https://github.com/ansibl8s/k8s-kubedash.git
# path: roles/apps
-4
View File
@@ -1,4 +0,0 @@
#!/bin/sh
make_resolv_conf() {
:
}
-101
View File
@@ -1,101 +0,0 @@
---
- name: populate inventory into hosts file
lineinfile:
dest: /etc/hosts
regexp: "^{{ hostvars[item].ansible_default_ipv4.address }} {{ item }}$"
line: "{{ hostvars[item].ansible_default_ipv4.address }} {{ item }}"
state: present
backup: yes
when: hostvars[item].ansible_default_ipv4.address is defined
with_items: groups['all']
- name: populate kubernetes loadbalancer address into hosts file
lineinfile:
dest: /etc/hosts
regexp: ".*{{ apiserver_loadbalancer_domain_name }}$"
line: "{{ loadbalancer_apiserver.address }} lb-apiserver.kubernetes.local"
state: present
backup: yes
when: loadbalancer_apiserver is defined and apiserver_loadbalancer_domain_name is defined
- name: clean hosts file
lineinfile:
dest: /etc/hosts
regexp: "{{ item }}"
state: absent
backup: yes
with_items:
- '^127\.0\.0\.1(\s+){{ inventory_hostname }}.*'
- '^::1(\s+){{ inventory_hostname }}.*'
- name: ensure dnsmasq.d directory exists
file:
path: /etc/dnsmasq.d
state: directory
when: inventory_hostname in groups['kube-master']
- name: configure dnsmasq
template:
src: 01-kube-dns.conf.j2
dest: /etc/dnsmasq.d/01-kube-dns.conf
mode: 755
backup: yes
when: inventory_hostname in groups['kube-master']
- name: create dnsmasq pod template
template: src=dnsmasq-pod.yml dest=/etc/kubernetes/manifests/dnsmasq-pod.manifest
when: inventory_hostname in groups['kube-master']
- name: Check for dnsmasq port
wait_for:
port: 53
delay: 5
timeout: 100
when: inventory_hostname in groups['kube-master']
- name: check resolvconf
stat: path=/etc/resolvconf/resolv.conf.d/head
register: resolvconf
- name: target resolv.conf file
set_fact:
resolvconffile: >
{%- if resolvconf.stat.exists == True -%}
/etc/resolvconf/resolv.conf.d/head
{%- else -%}
/etc/resolv.conf
{%- endif -%}
- name: Add search resolv.conf
lineinfile:
line: search {{ [ 'default.svc.' + dns_domain, 'svc.' + dns_domain, dns_domain ] | join(' ') }}
dest: "{{resolvconffile}}"
state: present
insertafter: EOF
backup: yes
follow: yes
- name: Add all masters as nameserver
lineinfile:
line: nameserver {{ hostvars[item]['ansible_default_ipv4']['address'] }}
dest: "{{resolvconffile}}"
state: present
insertafter: EOF
backup: yes
follow: yes
with_items: groups['kube-master']
- name: disable resolv.conf modification by dhclient
copy: src=dhclient_nodnsupdate dest=/etc/dhcp/dhclient-enter-hooks.d/nodnsupdate mode=u+x backup=yes
when: ansible_os_family == "Debian"
- name: disable resolv.conf modification by dhclient
copy: src=dhclient_nodnsupdate dest=/etc/dhcp/dhclient.d/nodnsupdate mode=u+x backup=yes
when: ansible_os_family == "RedHat"
- name: update resolvconf
command: resolvconf -u
changed_when: False
when: resolvconf.stat.exists == True
- meta: flush_handlers
@@ -1,19 +0,0 @@
#Listen on all interfaces
interface=*
addn-hosts=/etc/hosts
bogus-priv
#Set upstream dns servers
{% if upstream_dns_servers is defined %}
{% for srv in upstream_dns_servers %}
server={{ srv }}
{% endfor %}
{% else %}
server=8.8.8.8
server=8.8.4.4
{% endif %}
# Forward k8s domain to kube-dns
server=/{{ dns_domain }}/{{ dns_server }}
-49
View File
@@ -1,49 +0,0 @@
---
apiVersion: v1
kind: Pod
metadata:
name: dnsmasq
namespace: kube-system
spec:
hostNetwork: true
containers:
- name: dnsmasq
image: andyshinn/dnsmasq:2.72
command:
- dnsmasq
args:
- -k
- "-7"
- /etc/dnsmasq.d
- --local-service
securityContext:
capabilities:
add:
- NET_ADMIN
imagePullPolicy: Always
resources:
limits:
cpu: 100m
memory: 256M
ports:
- name: dns
containerPort: 53
hostPort: 53
protocol: UDP
- name: dns-tcp
containerPort: 53
hostPort: 53
protocol: TCP
volumeMounts:
- name: etcdnsmasqd
mountPath: /etc/dnsmasq.d
- name: etcdnsmasqdavailable
mountPath: /etc/dnsmasq.d-available
volumes:
- name: etcdnsmasqd
hostPath:
path: /etc/dnsmasq.d
- name: etcdnsmasqdavailable
hostPath:
path: /etc/dnsmasq.d-available
-2
View File
@@ -1,2 +0,0 @@
.*.swp
.vagrant
-53
View File
@@ -1,53 +0,0 @@
---
- name: gather os specific variables
include_vars: "{{ item }}"
with_first_found:
- files:
- "{{ ansible_distribution|lower }}-{{ ansible_distribution_version|lower|replace('/', '_') }}.yml"
- "{{ ansible_distribution|lower }}-{{ ansible_distribution_release }}.yml"
- "{{ ansible_distribution|lower }}-{{ ansible_distribution_major_version|lower|replace('/', '_') }}.yml"
- "{{ ansible_distribution|lower }}.yml"
- "{{ ansible_os_family|lower }}.yml"
- defaults.yml
paths:
- ../vars
- name: check for minimum kernel version
fail:
msg: >
docker requires a minimum kernel version of
{{ docker_kernel_min_version }} on
{{ ansible_distribution }}-{{ ansible_distribution_version }}
when: ansible_kernel|version_compare(docker_kernel_min_version, "<")
- name: ensure docker requirements packages are installed
action: "{{ docker_package_info.pkg_mgr }}"
args: docker_package_info.args
with_items: docker_package_info.pre_pkgs
when: docker_package_info.pre_pkgs|length > 0
- name: ensure docker repository public key is installed
action: "{{ docker_repo_key_info.pkg_key }}"
args: docker_repo_key_info.args
with_items: docker_repo_key_info.repo_keys
when: docker_repo_key_info.repo_keys|length > 0
- name: ensure docker repository is enabled
action: "{{ docker_repo_info.pkg_repo }}"
args: docker_repo_info.args
with_items: docker_repo_info.repos
when: docker_repo_info.repos|length > 0
- name: ensure docker packages are installed
action: "{{ docker_package_info.pkg_mgr }}"
args: docker_package_info.args
with_items: docker_package_info.pkgs
when: docker_package_info.pkgs|length > 0
- name: ensure docker service is started and enabled
service:
name: "{{ item }}"
enabled: yes
state: started
with_items:
- docker
-24
View File
@@ -1,24 +0,0 @@
docker_kernel_min_version: '2.6.32-431'
docker_package_info:
pkg_mgr: yum
args:
name: "{{ item }}"
state: latest
update_cache: yes
pre_pkgs:
- epel-release
- curl
- device-mapper-libs
pkgs:
- docker-io
docker_repo_key_info:
pkg_key: ''
args: {}
repo_keys: []
docker_repo_info:
pkg_repo: ''
args: {}
repos: []
-36
View File
@@ -1,36 +0,0 @@
docker_kernel_min_version: '3.2'
docker_package_info:
pkg_mgr: apt
args:
pkg: "{{ item }}"
update_cache: yes
cache_valid_time: 600
state: latest
pre_pkgs:
- apt-transport-https
- curl
- software-properties-common
pkgs:
- docker-engine
docker_repo_key_info:
pkg_key: apt_key
args:
id: "{{ item }}"
keyserver: hkp://p80.pool.sks-keyservers.net:80
state: present
repo_keys:
- 58118E89F3A912897C070ADBF76221572C52609D
docker_repo_info:
pkg_repo: apt_repository
args:
repo: "{{ item }}"
update_cache: yes
state: present
repos:
- >
deb https://apt.dockerproject.org/repo
{{ ansible_distribution|lower }}-{{ ansible_distribution_release|lower }}
main
-22
View File
@@ -1,22 +0,0 @@
docker_kernel_min_version: '0'
docker_package_info:
pkg_mgr: yum
args:
name: "{{ item }}"
state: latest
update_cache: yes
pre_pkgs:
- curl
pkgs:
- docker-io
docker_repo_key_info:
pkg_key: ''
args: {}
repo_keys: []
docker_repo_info:
pkg_repo: ''
args: {}
repos: []
-22
View File
@@ -1,22 +0,0 @@
docker_kernel_min_version: '0'
docker_package_info:
pkg_mgr: yum
args:
name: "{{ item }}"
state: latest
update_cache: yes
pre_pkgs:
- curl
pkgs:
- docker
docker_repo_key_info:
pkg_key: ''
args: {}
repo_keys: []
docker_repo_info:
pkg_repo: ''
args: {}
repos: []
-42
View File
@@ -1,42 +0,0 @@
---
local_release_dir: /tmp
flannel_version: 0.5.5
calico_version: v0.13.0
calico_plugin_version: v0.7.0
kube_version: v1.1.3
kubectl_checksum: "01b9bea18061a27b1cf30e34fd8ab45cfc096c9a9d57d0ed21072abb40dd3d1d"
kubelet_checksum: "62191c66f2d670dd52ddf1d88ef81048977abf1ffaa95ee6333299447eb6a482"
kube_download_url: "https://storage.googleapis.com/kubernetes-release/release/{{ kube_version }}/bin/linux/amd64"
flannel_download_url: "https://github.com/coreos/flannel/releases/download/v{{ flannel_version }}/flannel-{{ flannel_version }}-linux-amd64.tar.gz"
calico_download_url: "https://github.com/Metaswitch/calico-docker/releases/download/{{calico_version}}/calicoctl"
calico_plugin_download_url: "https://github.com/projectcalico/calico-kubernetes/releases/download/{{calico_plugin_version}}/calico_kubernetes"
downloads:
- name: calico
dest: calico/bin/calicoctl
url: "{{calico_download_url}}"
- name: calico-plugin
dest: calico/bin/calico
url: "{{calico_plugin_download_url}}"
- name: flannel
dest: flannel/flannel-{{ flannel_version }}-linux-amd64.tar.gz
url: "{{flannel_download_url}}"
unarchive: yes
- name: kubernetes-kubelet
dest: kubernetes/bin/kubelet
sha256: "{{kubelet_checksum}}"
url: "{{ kube_download_url }}/kubelet"
- name: kubernetes-kubectl
dest: kubernetes/bin/kubectl
sha256: "{{kubectl_checksum}}"
url: "{{ kube_download_url }}/kubectl"
-19
View File
@@ -1,19 +0,0 @@
---
- name: Create dest directories
file: path={{local_release_dir}}/{{item.dest|dirname}} state=directory recurse=yes
with_items: downloads
- name: Download items
get_url:
url: "{{item.url}}"
dest: "{{local_release_dir}}/{{item.dest}}"
sha256sum: "{{item.sha256 | default(omit)}}"
with_items: downloads
- name: Extract archives
unarchive:
src: "{{ local_release_dir }}/{{item.dest}}"
dest: "{{ local_release_dir }}/{{item.dest|dirname}}"
copy: no
when: "{{item.unarchive is defined and item.unarchive == True}}"
with_items: downloads
-13
View File
@@ -1,13 +0,0 @@
---
- name: ETCD2 | Stop etcd2 service
service: name=etcd state=stopped
ignore_errors: yes
- name: ETCD2 | create etcd pod template
template: src=etcd-pod.yml dest=/etc/kubernetes/manifests/etcd-pod.manifest
- name: ETCD2 | Check for etcd2 port
wait_for:
port: 2379
delay: 5
timeout: 100
-54
View File
@@ -1,54 +0,0 @@
---
apiVersion: v1
kind: Pod
metadata:
name: etcd
namespace: kube-system
spec:
hostNetwork: true
containers:
- name: etcd
image: quay.io/coreos/etcd:v2.2.2
resources:
limits:
cpu: 100m
memory: 256M
args:
{% if inventory_hostname in groups['etcd'] %}
- --name
- etcd-{{inventory_hostname}}-master
- --advertise-client-urls
- "http://{{ hostvars[inventory_hostname]['ip'] | default( ansible_default_ipv4.address) }}:2379"
- --listen-peer-urls
- http://0.0.0.0:2380
- --initial-advertise-peer-urls
- http://{{ hostvars[inventory_hostname]['ip'] | default( ansible_default_ipv4.address) }}:2380
- --data-dir
- /var/etcd/data
- --initial-cluster-state
- new
{% else %}
- --proxy
- 'on'
{% endif %}
- --listen-client-urls
- "http://{{ hostvars[inventory_hostname]['ip'] | default( ansible_default_ipv4.address) }}:2379,http://127.0.0.1:2379"
- --initial-cluster
- "{% for host in groups['etcd'] %}etcd-{{host}}-master=http://{{ hostvars[host]['ip'] | default( hostvars[host]['ansible_default_ipv4']['address']) }}:2380{% if not loop.last %},{% endif %}{% endfor %}"
- --initial-cluster-token
- etcd-k8s-cluster
ports:
- name: etcd-client
containerPort: 2379
hostPort: 2379
- name: etcd-peer
containerPort: 2380
hostPort: 2380
volumeMounts:
- name: varetcd
mountPath: /var/etcd
readOnly: false
volumes:
- name: varetcd
hostPath:
path: /containers/pods/etcd-{{inventory_hostname}}/rootfs/var/etcd
File diff suppressed because it is too large Load Diff
-14
View File
@@ -1,14 +0,0 @@
---
- name: reload systemd
command: systemctl daemon-reload
- name: restart systemd-kubelet
command: /bin/true
notify:
- reload systemd
- restart kubelet
- name: restart kubelet
service:
name: kubelet
state: restarted
-4
View File
@@ -1,4 +0,0 @@
---
dependencies:
- { role: etcd }
- { role: kubernetes/node }
-82
View File
@@ -1,82 +0,0 @@
---
- name: Copy kubectl bash completion
copy:
src: kubectl_bash_completion.sh
dest: /etc/bash_completion.d/kubectl.sh
- name: Install kubectl binary
synchronize:
src: "{{ local_release_dir }}/kubernetes/bin/kubectl"
dest: "{{ bin_dir }}/kubectl"
archive: no
checksum: yes
times: yes
delegate_to: "{{ groups['downloader'][0] }}"
- name: Perms kubectl binary
file: path={{ bin_dir }}/kubectl owner=kube mode=0755 state=file
- name: populate users for basic auth in API
lineinfile:
dest: "{{ kube_users_dir }}/known_users.csv"
create: yes
line: '{{ item.value.pass }},{{ item.key }},{{ item.value.role }}'
backup: yes
with_dict: "{{ kube_users }}"
# Sync masters
- name: synchronize auth directories for masters
synchronize:
src: "{{ item }}"
dest: "{{ kube_config_dir }}"
recursive: yes
delete: yes
rsync_opts: [ '--one-file-system']
set_remote_user: false
with_items:
- "{{ kube_token_dir }}"
- "{{ kube_cert_dir }}"
- "{{ kube_users_dir }}"
delegate_to: "{{ groups['kube-master'][0] }}"
when: inventory_hostname != "{{ groups['kube-master'][0] }}"
# Write manifests
- name: Write kube-apiserver manifest
template:
src: manifests/kube-apiserver.manifest.j2
dest: "{{ kube_manifest_dir }}/kube-apisever.manifest"
notify:
- restart kubelet
- meta: flush_handlers
- name: wait for the apiserver to be running (pulling image and running container)
wait_for:
port: "{{kube_apiserver_insecure_port}}"
delay: 10
timeout: 60
- name: Create 'kube-system' namespace
uri:
url: http://127.0.0.1:{{ kube_apiserver_insecure_port }}/api/v1/namespaces
method: POST
body: '{"apiVersion":"v1","kind":"Namespace","metadata":{"name":"kube-system"}}'
status_code: 201,409
body_format: json
run_once: yes
when: inventory_hostname == groups['kube-master'][0]
- name: Write kube-controller-manager manifest
template:
src: manifests/kube-controller-manager.manifest.j2
dest: "{{ kube_config_dir }}/kube-controller-manager.manifest"
- name: Write kube-scheduler manifest
template:
src: manifests/kube-scheduler.manifest.j2
dest: "{{ kube_config_dir }}/kube-scheduler.manifest"
- name: Write podmaster manifest
template:
src: manifests/kube-podmaster.manifest.j2
dest: "{{ kube_manifest_dir }}/kube-podmaster.manifest"
@@ -1,18 +0,0 @@
apiVersion: v1
kind: Config
current-context: kubectl-to-{{ cluster_name }}
preferences: {}
clusters:
- cluster:
certificate-authority-data: {{ kube_node_cert|b64encode }}
server: https://{{ groups['kube-master'][0] }}:{{ kube_apiserver_port }}
name: {{ cluster_name }}
contexts:
- context:
cluster: {{ cluster_name }}
user: kubectl
name: kubectl-to-{{ cluster_name }}
users:
- name: kubectl
user:
token: {{ kubectl_token }}
@@ -1,52 +0,0 @@
apiVersion: v1
kind: Pod
metadata:
name: kube-apiserver
spec:
hostNetwork: true
containers:
- name: kube-apiserver
image: {{ hyperkube_image_repo }}:{{ hyperkube_image_tag }}
command:
- /hyperkube
- apiserver
- --etcd-servers={% for srv in groups['etcd'] %}http://{{ srv }}:2379{% if not loop.last %},{% endif %}{% endfor %}
- --admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota
- --service-cluster-ip-range={{ kube_service_addresses }}
- --client-ca-file={{ kube_cert_dir }}/ca.pem
- --basic-auth-file={{ kube_users_dir }}/known_users.csv
- --tls-cert-file={{ kube_cert_dir }}/apiserver.pem
- --tls-private-key-file={{ kube_cert_dir }}/apiserver-key.pem
- --service-account-key-file={{ kube_cert_dir }}/apiserver-key.pem
- --secure-port={{ kube_apiserver_port }}
- --insecure-port={{ kube_apiserver_insecure_port }}
{% if kube_api_runtime_config is defined %}
{% for conf in kube_api_runtime_config %}
- --runtime-config={{ conf }}
{% endfor %}
{% endif %}
- --token-auth-file={{ kube_token_dir }}/known_tokens.csv
- --v={{ kube_log_level | default('2') }}
- --allow-privileged=true
ports:
- containerPort: {{ kube_apiserver_port }}
hostPort: {{ kube_apiserver_port }}
name: https
- containerPort: {{ kube_apiserver_insecure_port }}
hostPort: {{ kube_apiserver_insecure_port }}
name: local
volumeMounts:
- mountPath: {{ kube_config_dir }}
name: kubernetes-config
readOnly: true
- mountPath: /etc/ssl/certs
name: ssl-certs-host
readOnly: true
volumes:
- hostPath:
path: {{ kube_config_dir }}
name: kubernetes-config
- hostPath:
path: /usr/share/ca-certificates
name: ssl-certs-host
@@ -1,38 +0,0 @@
apiVersion: v1
kind: Pod
metadata:
name: kube-controller-manager
namespace: kube-system
spec:
hostNetwork: true
containers:
- name: kube-controller-manager
image: {{ hyperkube_image_repo }}:{{ hyperkube_image_tag }}
command:
- /hyperkube
- controller-manager
- --master=http://127.0.0.1:{{kube_apiserver_insecure_port}}
- --service-account-private-key-file={{ kube_cert_dir }}/apiserver-key.pem
- --root-ca-file={{ kube_cert_dir }}/ca.pem
- --v={{ kube_log_level | default('2') }}
livenessProbe:
httpGet:
host: 127.0.0.1
path: /healthz
port: 10252
initialDelaySeconds: 15
timeoutSeconds: 1
volumeMounts:
- mountPath: {{ kube_cert_dir }}
name: ssl-certs-kubernetes
readOnly: true
- mountPath: /etc/ssl/certs
name: ssl-certs-host
readOnly: true
volumes:
- hostPath:
path: {{ kube_cert_dir }}
name: ssl-certs-kubernetes
- hostPath:
path: /usr/share/ca-certificates
name: ssl-certs-host
@@ -1,46 +0,0 @@
apiVersion: v1
kind: Pod
metadata:
name: kube-podmaster
namespace: kube-system
spec:
hostNetwork: true
containers:
- name: scheduler-elector
image: gcr.io/google_containers/podmaster:1.1
command:
- /podmaster
- --etcd-servers={% for srv in groups['etcd'] %}http://{{ srv }}:2379{% if not loop.last %},{% endif %}{% endfor %}
- --key=scheduler
- --source-file={{ kube_config_dir}}/kube-scheduler.manifest
- --dest-file={{ kube_manifest_dir }}/kube-scheduler.manifest
volumeMounts:
- mountPath: {{ kube_config_dir }}
name: manifest-src
readOnly: true
- mountPath: {{ kube_manifest_dir }}
name: manifest-dst
- name: controller-manager-elector
image: gcr.io/google_containers/podmaster:1.1
command:
- /podmaster
- --etcd-servers={% for srv in groups['etcd'] %}http://{{ srv }}:2379{% if not loop.last %},{% endif %}{% endfor %}
- --key=controller
- --source-file={{ kube_config_dir }}/kube-controller-manager.manifest
- --dest-file={{ kube_manifest_dir }}/kube-controller-manager.manifest
terminationMessagePath: /dev/termination-log
volumeMounts:
- mountPath: {{ kube_config_dir }}
name: manifest-src
readOnly: true
- mountPath: {{ kube_manifest_dir }}
name: manifest-dst
volumes:
- hostPath:
path: {{ kube_config_dir }}
name: manifest-src
- hostPath:
path: {{ kube_manifest_dir }}
name: manifest-dst
@@ -1,22 +0,0 @@
apiVersion: v1
kind: Pod
metadata:
name: kube-scheduler
namespace: kube-system
spec:
hostNetwork: true
containers:
- name: kube-scheduler
image: {{ hyperkube_image_repo }}:{{ hyperkube_image_tag }}
command:
- /hyperkube
- scheduler
- --master=http://127.0.0.1:{{kube_apiserver_insecure_port}}
- --v={{ kube_log_level | default('2') }}
livenessProbe:
httpGet:
host: 127.0.0.1
path: /healthz
port: 10251
initialDelaySeconds: 15
timeoutSeconds: 1
-49
View File
@@ -1,49 +0,0 @@
# This directory is where all the additional scripts go
# that Kubernetes normally puts in /srv/kubernetes.
# This puts them in a sane location
kube_script_dir: "{{ bin_dir }}/kubernetes-scripts"
# This directory is where all the additional config stuff goes
# the kubernetes normally puts in /srv/kubernets.
# This puts them in a sane location.
# Editting this value will almost surely break something. Don't
# change it. Things like the systemd scripts are hard coded to
# look in here. Don't do it.
kube_config_dir: /etc/kubernetes
# This is where all the cert scripts and certs will be located
kube_cert_dir: "{{ kube_config_dir }}/ssl"
# This is where all of the bearer tokens will be stored
kube_token_dir: "{{ kube_config_dir }}/tokens"
# This is where to save basic auth file
kube_users_dir: "{{ kube_config_dir }}/users"
# This is where you can drop yaml/json files and the kubelet will run those
# pods on startup
kube_manifest_dir: "{{ kube_config_dir }}/manifests"
# This is the group that the cert creation scripts chgrp the
# cert files to. Not really changable...
kube_cert_group: kube-cert
dns_domain: "{{ cluster_name }}"
kube_proxy_mode: userspace
# Temporary image, waiting for official google release
# hyperkube_image_repo: gcr.io/google_containers/hyperkube
hyperkube_image_repo: quay.io/smana/hyperkube
hyperkube_image_tag: v1.1.3
# IP address of the DNS server.
# Kubernetes will create a pod with several containers, serving as the DNS
# server and expose it under this IP address. The IP address must be from
# the range specified as kube_service_addresses. This magic will actually
# pick the 10th ip address in the kube_service_addresses range and use that.
dns_server: "{{ kube_service_addresses|ipaddr('net')|ipaddr(253)|ipaddr('address') }}"
kube_api_runtime_config:
- extensions/v1beta1/daemonsets=true
- extensions/v1beta1/deployments=true
@@ -1,34 +0,0 @@
#!/bin/bash
# Copyright 2015 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
token_dir=${TOKEN_DIR:-/var/srv/kubernetes}
token_file="${token_dir}/known_tokens.csv"
create_accounts=($@)
if [ ! -e "${token_file}" ]; then
touch "${token_file}"
fi
for account in "${create_accounts[@]}"; do
if grep ",${account}," "${token_file}" ; then
continue
fi
token=$(dd if=/dev/urandom bs=128 count=1 2>/dev/null | base64 | tr -d "=+/" | dd bs=32 count=1 2>/dev/null)
echo "${token},${account},${account}" >> "${token_file}"
echo "${token}" > "${token_dir}/${account}.token"
echo "Added ${account}"
done
-107
View File
@@ -1,107 +0,0 @@
#!/bin/bash
# Author: skahlouc@skahlouc-laptop
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -o errexit
set -o pipefail
usage()
{
cat << EOF
Create self signed certificates
Usage : $(basename $0) -f <config> [-c <cloud_provider>] [-d <ssldir>] [-g <ssl_group>]
-h | --help : Show this message
-f | --config : Openssl configuration file
-c | --cloud : Cloud provider (GCE, AWS or AZURE)
-d | --ssldir : Directory where the certificates will be installed
-g | --sslgrp : Group of the certificates
ex :
$(basename $0) -f openssl.conf -c GCE -d /srv/ssl -g kube
EOF
}
# Options parsing
while (($#)); do
case "$1" in
-h | --help) usage; exit 0;;
-f | --config) CONFIG=${2}; shift 2;;
-c | --cloud) CLOUD=${2}; shift 2;;
-d | --ssldir) SSLDIR="${2}"; shift 2;;
-g | --group) SSLGRP="${2}"; shift 2;;
*)
usage
echo "ERROR : Unknown option"
exit 3
;;
esac
done
if [ -z ${CONFIG} ]; then
echo "ERROR: the openssl configuration file is missing. option -f"
exit 1
fi
if [ -z ${SSLDIR} ]; then
SSLDIR="/etc/kubernetes/certs"
fi
if [ -z ${SSLGRP} ]; then
SSLGRP="kube-cert"
fi
#echo "config=$CONFIG, cloud=$CLOUD, certdir=$SSLDIR, certgroup=$SSLGRP"
SUPPORTED_CLOUDS="GCE AWS AZURE"
# TODO: Add support for discovery on other providers?
if [ "${CLOUD}" == "GCE" ]; then
CLOUD_IP=$(curl -s -H Metadata-Flavor:Google http://metadata.google.internal./computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip)
fi
if [ "${CLOUD}" == "AWS" ]; then
CLOUD_IP=$(curl -s http://169.254.169.254/latest/meta-data/public-ipv4)
fi
if [ "${CLOUD}" == "AZURE" ]; then
CLOUD_IP=$(uname -n | awk -F. '{ print $2 }').cloudapp.net
fi
tmpdir=$(mktemp -d --tmpdir kubernetes_cacert.XXXXXX)
trap 'rm -rf "${tmpdir}"' EXIT
cd "${tmpdir}"
mkdir -p "${SSLDIR}"
# Root CA
openssl genrsa -out ca-key.pem 2048 > /dev/null 2>&1
openssl req -x509 -new -nodes -key ca-key.pem -days 10000 -out ca.pem -subj "/CN=kube-ca" > /dev/null 2>&1
# Apiserver
openssl genrsa -out apiserver-key.pem 2048 > /dev/null 2>&1
openssl req -new -key apiserver-key.pem -out apiserver.csr -subj "/CN=kube-apiserver" -config ${CONFIG} > /dev/null 2>&1
openssl x509 -req -in apiserver.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out apiserver.pem -days 365 -extensions v3_req -extfile ${CONFIG} > /dev/null 2>&1
# Nodes and Admin
for i in node admin; do
openssl genrsa -out ${i}-key.pem 2048 > /dev/null 2>&1
openssl req -new -key ${i}-key.pem -out ${i}.csr -subj "/CN=kube-${i}" > /dev/null 2>&1
openssl x509 -req -in ${i}.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out ${i}.pem -days 365 > /dev/null 2>&1
done
# Install certs
mv *.pem ${SSLDIR}/
chgrp ${SSLGRP} ${SSLDIR}/*
chmod 600 ${SSLDIR}/*-key.pem
chown root:root ${SSLDIR}/*-key.pem
-14
View File
@@ -1,14 +0,0 @@
---
- name: reload systemd
command: systemctl daemon-reload
- name: restart systemd-kubelet
command: /bin/true
notify:
- reload systemd
- restart kubelet
- name: restart kubelet
service:
name: kubelet
state: restarted
-28
View File
@@ -1,28 +0,0 @@
---
- name: certs | install cert generation script
copy:
src=make-ssl.sh
dest={{ kube_script_dir }}
mode=0500
changed_when: false
- name: certs | write openssl config
template:
src: "openssl.conf.j2"
dest: "{{ kube_config_dir }}/.openssl.conf"
- name: certs | run cert generation script
shell: >
{{ kube_script_dir }}/make-ssl.sh
-f {{ kube_config_dir }}/.openssl.conf
-g {{ kube_cert_group }}
-d {{ kube_cert_dir }}
args:
creates: "{{ kube_cert_dir }}/apiserver.pem"
- name: certs | check certificate permissions
file:
path={{ kube_cert_dir }}
group={{ kube_cert_group }}
owner=kube
recurse=yes
@@ -1,48 +0,0 @@
---
- name: tokens | copy the token gen script
copy:
src=kube-gen-token.sh
dest={{ kube_script_dir }}
mode=u+x
when: inventory_hostname == groups['kube-master'][0]
- name: tokens | generate tokens for master components
command: "{{ kube_script_dir }}/kube-gen-token.sh {{ item[0] }}-{{ item[1] }}"
environment:
TOKEN_DIR: "{{ kube_token_dir }}"
with_nested:
- [ "system:kubectl" ]
- "{{ groups['kube-master'] }}"
register: gentoken
changed_when: "'Added' in gentoken.stdout"
when: inventory_hostname == groups['kube-master'][0]
- name: tokens | generate tokens for node components
command: "{{ kube_script_dir }}/kube-gen-token.sh {{ item[0] }}-{{ item[1] }}"
environment:
TOKEN_DIR: "{{ kube_token_dir }}"
with_nested:
- [ 'system:kubelet' ]
- "{{ groups['kube-node'] }}"
register: gentoken
changed_when: "'Added' in gentoken.stdout"
when: inventory_hostname == groups['kube-master'][0]
- name: tokens | generate tokens for calico
command: "{{ kube_script_dir }}/kube-gen-token.sh {{ item[0] }}-{{ item[1] }}"
environment:
TOKEN_DIR: "{{ kube_token_dir }}"
with_nested:
- [ "system:calico" ]
- "{{ groups['k8s-cluster'] }}"
register: gentoken
changed_when: "'Added' in gentoken.stdout"
when: kube_network_plugin == "calico"
delegate_to: "{{ groups['kube-master'][0] }}"
- name: tokens | get the calico token values
slurp:
src: "{{ kube_token_dir }}/system:calico-{{ inventory_hostname }}.token"
register: calico_token
when: kube_network_plugin == "calico"
delegate_to: "{{ groups['kube-master'][0] }}"
-48
View File
@@ -1,48 +0,0 @@
---
- debug: msg="{{init_system == "systemd"}}"
- debug: msg="{{init_system}}"
- name: install | Write kubelet systemd init file
template: src=kubelet.service.j2 dest=/etc/systemd/system/kubelet.service backup=yes
when: init_system == "systemd"
notify: restart systemd-kubelet
- name: install | Write kubelet initd script
template: src=deb-kubelet.initd.j2 dest=/etc/init.d/kubelet owner=root mode=755 backup=yes
when: init_system == "sysvinit" and ansible_os_family == "Debian"
notify: restart kubelet
- name: install | Write kubelet initd script
template: src=rh-kubelet.initd.j2 dest=/etc/init.d/kubelet owner=root mode=755 backup=yes
when: init_system == "sysvinit" and ansible_os_family == "RedHat"
notify: restart kubelet
- name: install | Install kubelet binary
synchronize:
src: "{{ local_release_dir }}/kubernetes/bin/kubelet"
dest: "{{ bin_dir }}/kubelet"
times: yes
archive: no
delegate_to: "{{ groups['downloader'][0] }}"
notify:
- restart kubelet
- name: install | Perms kubelet binary
file: path={{ bin_dir }}/kubelet owner=kube mode=0755 state=file
- name: install | Calico-plugin | Directory
file: path=/usr/libexec/kubernetes/kubelet-plugins/net/exec/calico/ state=directory
when: kube_network_plugin == "calico"
- name: install | Calico-plugin | Binary
synchronize:
src: "{{ local_release_dir }}/calico/bin/calico"
dest: "/usr/libexec/kubernetes/kubelet-plugins/net/exec/calico/calico"
times: yes
archive: no
delegate_to: "{{ groups['downloader'][0] }}"
when: kube_network_plugin == "calico"
notify: restart kubelet
- name: install | Perms calico plugin binary
file: path=/usr/libexec/kubernetes/kubelet-plugins/net/exec/calico/calico owner=kube mode=0755 state=file
-49
View File
@@ -1,49 +0,0 @@
---
- name: create kubernetes config directory
file: path={{ kube_config_dir }} state=directory
- name: create kubernetes script directory
file: path={{ kube_script_dir }} state=directory
- name: Make sure manifest directory exists
file: path={{ kube_manifest_dir }} state=directory
- name: certs | create system kube-cert groups
group: name={{ kube_cert_group }} state=present system=yes
- name: create system kube user
user:
name=kube
comment="Kubernetes user"
shell=/sbin/nologin
state=present
system=yes
groups={{ kube_cert_group }}
- include: secrets.yml
tags:
- secrets
- include: install.yml
- name: Write kubelet config file
template: src=kubelet.j2 dest={{ kube_config_dir }}/kubelet backup=yes
notify:
- restart kubelet
- name: write the kubecfg (auth) file for kubelet
template: src=node-kubeconfig.yaml.j2 dest={{ kube_config_dir }}/node-kubeconfig.yaml backup=yes
notify:
- restart kubelet
- name: Write proxy manifest
template:
src: manifests/kube-proxy.manifest.j2
dest: "{{ kube_manifest_dir }}/kube-proxy.manifest"
- name: Enable kubelet
service:
name: kubelet
enabled: yes
state: started
-52
View File
@@ -1,52 +0,0 @@
---
- name: certs | make sure the certificate directory exits
file:
path={{ kube_cert_dir }}
state=directory
mode=o-rwx
group={{ kube_cert_group }}
- name: tokens | make sure the tokens directory exits
file:
path={{ kube_token_dir }}
state=directory
mode=o-rwx
group={{ kube_cert_group }}
- include: gen_certs.yml
run_once: true
when: inventory_hostname == groups['kube-master'][0]
- include: gen_tokens.yml
# Sync certs between nodes
- user:
name: '{{ansible_user_id}}'
generate_ssh_key: yes
delegate_to: "{{ groups['kube-master'][0] }}"
run_once: yes
- name: 'get ssh keypair'
slurp: path=~/.ssh/id_rsa.pub
register: public_key
delegate_to: "{{ groups['kube-master'][0] }}"
- name: 'setup keypair on nodes'
authorized_key:
user: '{{ansible_user_id}}'
key: "{{public_key.content|b64decode }}"
- name: synchronize certificates for nodes
synchronize:
src: "{{ item }}"
dest: "{{ kube_cert_dir }}"
recursive: yes
delete: yes
rsync_opts: [ '--one-file-system']
set_remote_user: false
with_items:
- "{{ kube_cert_dir}}/ca.pem"
- "{{ kube_cert_dir}}/node.pem"
- "{{ kube_cert_dir}}/node-key.pem"
delegate_to: "{{ groups['kube-master'][0] }}"
when: inventory_hostname not in "{{ groups['kube-master'] }}"
@@ -1,119 +0,0 @@
#!/bin/bash
#
### BEGIN INIT INFO
# Provides: kubelet
# Required-Start: $local_fs $network $syslog
# Required-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: The Kubernetes node container manager
# Description:
# The Kubernetes container manager maintains docker state against a state file.
### END INIT INFO
# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="The Kubernetes container manager"
NAME=kubelet
DAEMON={{ bin_dir }}/kubelet
DAEMON_ARGS=""
DAEMON_LOG_FILE=/var/log/$NAME.log
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
DAEMON_USER=root
# Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0
# Read configuration variable file if it is present
[ -r /etc/kubernetes/$NAME ] && . /etc/kubernetes/$NAME
# Define LSB log_* functions.
# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
# and status_of_proc is working.
. /lib/lsb/init-functions
#
# Function that starts the daemon/service
#
do_start()
{
# Return
# 0 if daemon has been started
# 1 if daemon was already running
# 2 if daemon could not be started
start-stop-daemon --start --quiet --background --no-close \
--make-pidfile --pidfile $PIDFILE \
--exec $DAEMON -c $DAEMON_USER --test > /dev/null \
|| return 1
start-stop-daemon --start --quiet --background --no-close \
--make-pidfile --pidfile $PIDFILE \
--exec $DAEMON -c $DAEMON_USER -- \
$DAEMON_ARGS >> $DAEMON_LOG_FILE 2>&1 \
|| return 2
}
#
# Function that stops the daemon/service
#
do_stop()
{
# Return
# 0 if daemon has been stopped
# 1 if daemon was already stopped
# 2 if daemon could not be stopped
# other if a failure occurred
start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
RETVAL="$?"
[ "$RETVAL" = 2 ] && return 2
# Many daemons don't delete their pidfiles when they exit.
rm -f $PIDFILE
return "$RETVAL"
}
case "$1" in
start)
log_daemon_msg "Starting $DESC" "$NAME"
do_start
case "$?" in
0|1) log_end_msg 0 || exit 0 ;;
2) log_end_msg 1 || exit 1 ;;
esac
;;
stop)
log_daemon_msg "Stopping $DESC" "$NAME"
do_stop
case "$?" in
0|1) log_end_msg 0 ;;
2) exit 1 ;;
esac
;;
status)
status_of_proc -p $PIDFILE "$DAEMON" "$NAME" && exit 0 || exit $?
;;
restart|force-reload)
log_daemon_msg "Restarting $DESC" "$NAME"
do_stop
case "$?" in
0|1)
do_start
case "$?" in
0) log_end_msg 0 ;;
1) log_end_msg 1 ;; # Old process is still running
*) log_end_msg 1 ;; # Failed to start
esac
;;
*)
# Failed to stop
log_end_msg 1
;;
esac
;;
*)
echo "Usage: $SCRIPTNAME {start|stop|status|restart|force-reload}" >&2
exit 3
;;
esac
@@ -1,28 +0,0 @@
KUBE_LOGTOSTDERR="--logtostderr=true"
KUBE_LOG_LEVEL="--v={{ kube_log_level | default('2') }}"
KUBE_ALLOW_PRIV="--allow_privileged=true"
KUBELET_API_SERVER="--api_servers={% for host in groups['kube-master'] %}https://{{ hostvars[host]['ip'] | default(hostvars[host]['ansible_default_ipv4']['address']) }}:{{ kube_apiserver_port }}{% if not loop.last %},{% endif %}{% endfor %}"
# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=0.0.0.0"
# The port for the info server to serve on
# KUBELET_PORT="--port=10250"
# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname_override={{ inventory_hostname }}"
{% if inventory_hostname in groups['kube-master'] and inventory_hostname not in groups['kube-node'] %}
KUBELET_REGISTER_NODE="--register-node=false"
{% endif %}
# location of the api-server
{% if dns_setup %}
KUBELET_ARGS="--cluster_dns={{ dns_server }} --cluster_domain={{ dns_domain }} --kubeconfig={{ kube_config_dir}}/node-kubeconfig.yaml --config={{ kube_manifest_dir }}"
{% else %}
KUBELET_ARGS="--kubeconfig={{ kube_config_dir}}/kubelet.kubeconfig --config={{ kube_manifest_dir }}"
{% endif %}
{% if kube_network_plugin is defined and kube_network_plugin == "calico" %}
KUBELET_NETWORK_PLUGIN="--network_plugin={{ kube_network_plugin }}"
{% endif %}
# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow_privileged=true"
{% if init_system == "sysvinit" %}
DAEMON_ARGS="$KUBE_LOGTOSTDERR $KUBE_LOG_LEVEL $KUBE_ALLOW_PRIV $KUBELET_API_SERVER $KUBELET_ADDRESS \
$KUBELET_HOSTNAME $KUBELET_REGISTER_NODE $KUBELET_ARGS $KUBELET_ARGS $KUBELET_NETWORK_PLUGIN"
{% endif %}
@@ -1,26 +0,0 @@
[Unit]
Description=Kubernetes Kubelet Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
{% if kube_network_plugin is defined and kube_network_plugin == "calico" %}
After=docker.service calico-node.service
{% else %}
After=docker.service
{% endif %}
[Service]
EnvironmentFile=/etc/kubernetes/kubelet
ExecStart={{ bin_dir }}/kubelet \
$KUBE_LOGTOSTDERR \
$KUBE_LOG_LEVEL \
$KUBELET_API_SERVER \
$KUBELET_ADDRESS \
$KUBELET_PORT \
$KUBELET_HOSTNAME \
$KUBE_ALLOW_PRIV \
$KUBELET_ARGS \
$KUBELET_REGISTER_NODE \
$KUBELET_NETWORK_PLUGIN
Restart=on-failure
[Install]
WantedBy=multi-user.target
@@ -1,46 +0,0 @@
apiVersion: v1
kind: Pod
metadata:
name: kube-proxy
namespace: kube-system
spec:
hostNetwork: true
containers:
- name: kube-proxy
image: {{ hyperkube_image_repo }}:{{ hyperkube_image_tag }}
command:
- /hyperkube
- proxy
- --v={{ kube_log_level | default('2') }}
{% if inventory_hostname in groups['kube-master'] %}
- --master=http://127.0.0.1:{{kube_apiserver_insecure_port}}
{% else %}
{% if loadbalancer_apiserver is defined and apiserver_loadbalancer_domain_name is defined %}
- --master=https://{{ apiserver_loadbalancer_domain_name }}:{{ loadbalancer_apiserver.port }}
{% else %}
- --master=https://{{ hostvars[groups['kube-master'][0]]['ip'] | default(hostvars[groups['kube-master'][0]]['ansible_default_ipv4']['address']) }}:{{ kube_apiserver_port }}
{% endif%}
- --kubeconfig=/etc/kubernetes/node-kubeconfig.yaml
{% endif %}
securityContext:
privileged: true
volumeMounts:
- mountPath: /etc/ssl/certs
name: ssl-certs-host
readOnly: true
- mountPath: /etc/kubernetes/node-kubeconfig.yaml
name: "kubeconfig"
readOnly: true
- mountPath: /etc/kubernetes/ssl
name: "etc-kube-ssl"
readOnly: true
volumes:
- name: ssl-certs-host
hostPath:
path: /usr/share/ca-certificates
- name: "kubeconfig"
hostPath:
path: "/etc/kubernetes/node-kubeconfig.yaml"
- name: "etc-kube-ssl"
hostPath:
path: "/etc/kubernetes/ssl"
@@ -1,17 +0,0 @@
apiVersion: v1
kind: Config
clusters:
- name: local
cluster:
certificate-authority: {{ kube_cert_dir }}/ca.pem
users:
- name: kubelet
user:
client-certificate: {{ kube_cert_dir }}/node.pem
client-key: {{ kube_cert_dir }}/node-key.pem
contexts:
- context:
cluster: local
user: kubelet
name: kubelet-{{ cluster_name }}
current-context: kubelet-{{ cluster_name }}
@@ -1,20 +0,0 @@
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc.{{ dns_domain }}
{% if loadbalancer_apiserver is defined and apiserver_loadbalancer_domain_name is defined %}
DNS.4 = {{ apiserver_loadbalancer_domain_name }}
{% endif %}
{% for host in groups['kube-master'] %}
IP.{{ loop.index }} = {{ hostvars[host]['ip'] | default(hostvars[host]['ansible_default_ipv4']['address']) }}
{% endfor %}
{% set idx = groups['kube-master'] | length | int + 1 %}
IP.{{ idx | string }} = {{ kube_apiserver_ip }}
@@ -1,129 +0,0 @@
#!/bin/bash
#
# /etc/rc.d/init.d/kubelet
#
# chkconfig: 2345 95 95
# description: Daemon for kubelet (kubernetes.io)
### BEGIN INIT INFO
# Provides: kubelet
# Required-Start: $local_fs $network $syslog cgconfig
# Required-Stop:
# Should-Start:
# Should-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: start and stop kubelet
# Description:
# The Kubernetes container manager maintains docker state against a state file.
### END INIT INFO
# Source function library.
. /etc/rc.d/init.d/functions
prog="kubelet"
exec="{{ bin_dir }}/$prog"
pidfile="/var/run/$prog.pid"
lockfile="/var/lock/subsys/$prog"
logfile="/var/log/$prog"
[ -e /etc/kubernetes/$prog ] && . /etc/kubernetes/$prog
start() {
if [ ! -x $exec ]; then
if [ ! -e $exec ]; then
echo "Docker executable $exec not found"
else
echo "You do not have permission to execute the Docker executable $exec"
fi
exit 5
fi
check_for_cleanup
if ! [ -f $pidfile ]; then
printf "Starting $prog:\t"
echo "\n$(date)\n" >> $logfile
$exec $DAEMON_ARGS &>> $logfile &
pid=$!
echo $pid >> $pidfile
touch $lockfile
success
echo
else
failure
echo
printf "$pidfile still exists...\n"
exit 7
fi
}
stop() {
echo -n $"Stopping $prog: "
killproc -p $pidfile -d 300 $prog
retval=$?
echo
[ $retval -eq 0 ] && rm -f $lockfile
return $retval
}
restart() {
stop
start
}
reload() {
restart
}
force_reload() {
restart
}
rh_status() {
status -p $pidfile $prog
}
rh_status_q() {
rh_status >/dev/null 2>&1
}
check_for_cleanup() {
if [ -f ${pidfile} ]; then
/bin/ps -fp $(cat ${pidfile}) > /dev/null || rm ${pidfile}
fi
}
case "$1" in
start)
rh_status_q && exit 0
$1
;;
stop)
rh_status_q || exit 0
$1
;;
restart)
$1
;;
reload)
rh_status_q || exit 7
$1
;;
force-reload)
force_reload
;;
status)
rh_status
;;
condrestart|try-restart)
rh_status_q || exit 0
restart
;;
*)
echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload}"
exit 2
esac
exit $?
@@ -1,15 +0,0 @@
---
common_required_pkgs:
- python-httplib2
- openssl
- curl
debian_required_pkgs:
- python-apt
- python-pip
rh_required_pkgs:
- libselinux-python
pypy_version: 2.4.0
python_pypy_url: "https://bitbucket.org/pypy/pypy/downloads/pypy-{{ pypy_version }}.tar.bz2"
@@ -1,29 +0,0 @@
#/bin/bash
set -e
BINDIR="/usr/local/bin"
cd $BINDIR
if [[ -e $BINDIR/.bootstrapped ]]; then
exit 0
fi
PYPY_VERSION=2.4.0
wget -O - https://bitbucket.org/pypy/pypy/downloads/pypy-$PYPY_VERSION-linux64.tar.bz2 |tar -xjf -
mv -n pypy-$PYPY_VERSION-linux64 pypy
## library fixup
mkdir -p pypy/lib
ln -snf /lib64/libncurses.so.5.9 $BINDIR/pypy/lib/libtinfo.so.5
cat > $BINDIR/python <<EOF
#!/bin/bash
LD_LIBRARY_PATH=$BINDIR/pypy/lib:$LD_LIBRARY_PATH exec $BINDIR/pypy/bin/pypy "\$@"
EOF
chmod +x $BINDIR/python
$BINDIR/python --version
touch $BINDIR/.bootstrapped
File diff suppressed because it is too large Load Diff
-3
View File
@@ -1,3 +0,0 @@
#!/bin/bash
BINDIR="/usr/local/bin"
LD_LIBRARY_PATH=$BINDIR/pypy/lib:$LD_LIBRARY_PATH $BINDIR/pypy/bin/$(basename $0) $@
@@ -1,40 +0,0 @@
---
- name: "Identify init system"
shell: >
$(pgrep systemd > /dev/null && systemctl status > /dev/null);
if [ $? -eq 0 ] ; then
echo systemd;
else
echo sysvinit;
fi
always_run: True
register: init_system_output
changed_when: False
- set_fact:
init_system: "{{ init_system_output.stdout }}"
- name: Install packages requirements
action:
module: "{{ ansible_pkg_mgr }}"
name: "{{ item }}"
state: latest
with_items: common_required_pkgs
- name: Install debian packages requirements
apt:
name: "{{ item }}"
state: latest
when: ansible_os_family == "Debian"
with_items: debian_required_pkgs
- name: Install redhat packages requirements
action:
module: "{{ ansible_pkg_mgr }}"
name: "{{ item }}"
state: latest
when: ansible_os_family == "RedHat"
with_items: rh_required_pkgs
- include: python-bootstrap.yml
when: ansible_os_family not in [ "Debian", "RedHat" ]
@@ -1,41 +0,0 @@
---
- name: Python | Check if bootstrap is needed
raw: stat {{ bin_dir}}/.bootstrapped
register: need_bootstrap
ignore_errors: True
- name: Python | Run bootstrap.sh
script: bootstrap.sh
when: need_bootstrap | failed
- set_fact:
ansible_python_interpreter: "{{ bin_dir }}/python"
- name: Python | Check if we need to install pip
shell: "{{ansible_python_interpreter}} -m pip --version"
register: need_pip
ignore_errors: True
changed_when: false
when: need_bootstrap | failed
- name: Python | Copy get-pip.py
copy: src=get-pip.py dest=~/get-pip.py
when: need_pip | failed
- name: Python | Install pip
shell: "{{ansible_python_interpreter}} ~/get-pip.py"
when: need_pip | failed
- name: Python | Remove get-pip.py
file: path=~/get-pip.py state=absent
when: need_pip | failed
- name: Python | Install pip launcher
copy: src=runner dest={{ bin_dir }}/pip mode=0755
when: need_pip | failed
- name: Install required python modules
pip:
name: "{{ item }}"
with_items: pip_python_modules
-36
View File
@@ -1,36 +0,0 @@
---
- name : reload systemd
shell: systemctl daemon-reload
- name: restart systemd-calico-node
command: /bin/true
notify:
- reload systemd
- restart calico-node
- name: restart calico-node
service:
name: calico-node
state: restarted
- name: restart docker
service: name=docker state=restarted
- name: restart flannel
service: name=flannel state=restarted
notify:
- reload systemd
- stop docker
- delete docker0
- start docker
when: inventory_hostname in groups['kube-node']
- name: stop docker
service: name=docker state=stopped
- name: delete docker0
command: ip link delete docker0
ignore_errors: yes
- name: start docker
service: name=docker state=started

Some files were not shown because too many files have changed in this diff Show More