How to setup kubernetes using minikube

Kubernetes
By Vikrant
July 8, 2018

After hearing too much about the kuberenetes, I thought of start learning about it. Being from the openstack background, I initially planned to use openstack for learning the kubernetes but then I decided to use minikube on MAC with virtual box provider for my initial learning and then gradually moving to use kubernetes on cloud platform like openstack. Starting the kubernetes using minikube is really a cake-walk. Once it’s initialized, usually people are tempted to start the first pod or deployment, I was too but before doing that I thought of checking what is present in minikube by default. In this article, I am going to shed some light on the default resources of minikube.

$ curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.28.0/minikube-darwin-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/

$ minikube version
minikube version: v0.28.0

After installing the minikube, install the kubectl to communicate with minikube.

$ brew install kubernetes-cli
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.5", GitCommit:"f01a2bf98249a4db383560443a59bed0c13575df", GitTreeState:"clean", BuildDate:"2018-03-19T15:59:24Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.0", GitCommit:"fc32d2f3698e36b93322a3465f63a14e9f0eaead", GitTreeState:"clean", BuildDate:"2018-03-26T16:44:10Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

I ran the kubectl get all on the minikube to see all the resources present in kubernetes. I noticed that by default lot of resources are present. All of the resources are present in kube-systems but one service is present in default namespace hence if you are going to run this command without “–all-namespace” switch you would be able to see only one service present by default. Since minikube is running both controller and minion (node) on a single node hence in production when you will be segregating the roles you may not found all of these resources present by default on single node.

w$ kubectl get all --all-namespaces
NAMESPACE     NAME            DESIRED   CURRENT   READY     UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
kube-system   ds/kube-proxy   1         1         1         1            1           <none>          2m

NAMESPACE     NAME                              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
kube-system   deploy/default-http-backend       1         1         1            1           2m
kube-system   deploy/kube-dns                   1         1         1            1           2m
kube-system   deploy/kubernetes-dashboard       1         1         1            1           2m
kube-system   deploy/nginx-ingress-controller   1         1         1            1           2m

NAMESPACE     NAME                                     DESIRED   CURRENT   READY     AGE
kube-system   rs/default-http-backend-59868b7dd6       1         1         1         2m
kube-system   rs/kube-dns-86f4d74b45                   1         1         1         2m
kube-system   rs/kubernetes-dashboard-5498ccf677       1         1         1         2m
kube-system   rs/nginx-ingress-controller-67956bf89d   1         1         1         2m

NAMESPACE     NAME                                           READY     STATUS    RESTARTS   AGE
kube-system   po/default-http-backend-59868b7dd6-55x77       1/1       Running   0          2m
kube-system   po/etcd-minikube                               1/1       Running   0          1m
kube-system   po/kube-addon-manager-minikube                 1/1       Running   0          1m
kube-system   po/kube-apiserver-minikube                     1/1       Running   0          1m
kube-system   po/kube-controller-manager-minikube            1/1       Running   0          2m
kube-system   po/kube-dns-86f4d74b45-wv7j6                   3/3       Running   0          2m
kube-system   po/kube-proxy-x4hhm                            1/1       Running   0          2m
kube-system   po/kube-scheduler-minikube                     1/1       Running   0          1m
kube-system   po/kubernetes-dashboard-5498ccf677-pdw4g       1/1       Running   0          2m
kube-system   po/nginx-ingress-controller-67956bf89d-2chmj   1/1       Running   0          2m
kube-system   po/storage-provisioner                         1/1       Running   0          2m

NAMESPACE     NAME                       TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)         AGE
default       svc/kubernetes             ClusterIP   10.96.0.1       <none>        443/TCP         2m
kube-system   svc/default-http-backend   NodePort    10.96.215.164   <none>        80:30001/TCP    2m
kube-system   svc/kube-dns               ClusterIP   10.96.0.10      <none>        53/UDP,53/TCP   2m
kube-system   svc/kubernetes-dashboard   NodePort    10.96.72.101    <none>        80:30000/TCP    2m

Let’s first undersand the acryonms which we are seeing before “/” in NAME column.

ds = daemonset (Which means that this resource will be running on all nodes in cluster. if any new node is added, it will automatically start on it.)

kube-proxy in this case is running as daemon set. Pod (Covered later in article) corresponding to ds/kube-proxy is “kube-proxy-x4hhm”

$ kubectl describe ds/kube-proxy --namespace kube-system | tail -1
  Normal  SuccessfulCreate  7m    daemonset-controller  Created pod: kube-proxy-x4hhm

deploy = deployment (A Deployment controller provides declarative updates for Pods and ReplicaSets). I am using the kubernetes-dns in the further example because this is the only POD for which the numeber of container is 3 in po (POD) section rest all POD are having one container..

kube-dns deployment definition everything related to RS and POD also. RS should have one replica and POD contains three containers, we can see the image, ports, arguments, Limits, environment variable used to start the containers in a POD.

$ kubectl describe deploy/kube-dns  --namespace kube-system
Name:                   kube-dns
Namespace:              kube-system
CreationTimestamp:      Sat, 07 Jul 2018 18:30:31 +0530
Labels:                 k8s-app=kube-dns
Annotations:            deployment.kubernetes.io/revision=1
Selector:               k8s-app=kube-dns
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  0 max unavailable, 10% max surge
Pod Template:
  Labels:           k8s-app=kube-dns
  Service Account:  kube-dns
  Containers:
   kubedns:
    Image:  k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.8
    Ports:  10053/UDP, 10053/TCP, 10055/TCP
    Args:
      --domain=cluster.local.
      --dns-port=10053
      --config-dir=/kube-dns-config
      --v=2
    Limits:
      memory:  170Mi
    Requests:
      cpu:      100m
      memory:   70Mi
    Liveness:   http-get http://:10054/healthcheck/kubedns delay=60s timeout=5s period=10s #success=1 #failure=5
    Readiness:  http-get http://:8081/readiness delay=3s timeout=5s period=10s #success=1 #failure=3
    Environment:
      PROMETHEUS_PORT:  10055
    Mounts:
      /kube-dns-config from kube-dns-config (rw)
   dnsmasq:
    Image:  k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.8
    Ports:  53/UDP, 53/TCP
    Args:
      -v=2
      -logtostderr
      -configDir=/etc/k8s/dns/dnsmasq-nanny
      -restartDnsmasq=true
      --
      -k
      --cache-size=1000
      --no-negcache
      --log-facility=-
      --server=/cluster.local/127.0.0.1#10053
      --server=/in-addr.arpa/127.0.0.1#10053
      --server=/ip6.arpa/127.0.0.1#10053
    Requests:
      cpu:        150m
      memory:     20Mi
    Liveness:     http-get http://:10054/healthcheck/dnsmasq delay=60s timeout=5s period=10s #success=1 #failure=5
    Environment:  <none>
    Mounts:
      /etc/k8s/dns/dnsmasq-nanny from kube-dns-config (rw)
   sidecar:
    Image:  k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.8
    Port:   10054/TCP
    Args:
      --v=2
      --logtostderr
      --probe=kubedns,127.0.0.1:10053,kubernetes.default.svc.cluster.local,5,SRV
      --probe=dnsmasq,127.0.0.1:53,kubernetes.default.svc.cluster.local,5,SRV
    Requests:
      cpu:        10m
      memory:     20Mi
    Liveness:     http-get http://:10054/metrics delay=60s timeout=5s period=10s #success=1 #failure=5
    Environment:  <none>
    Mounts:       <none>
  Volumes:
   kube-dns-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      kube-dns
    Optional:  true
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   kube-dns-86f4d74b45 (1/1 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  13m   deployment-controller  Scaled up replica set kube-dns-86f4d74b45 to 1

rs = replica set controller (Decides how many replicas for POD should be running)

For the brevity I am truncating the container section from the following output which we have seen in the previous output.

$ kubectl describe rs/kube-dns-86f4d74b45 --namespace kube-system
Name:           kube-dns-86f4d74b45
Namespace:      kube-system
Selector:       k8s-app=kube-dns,pod-template-hash=4290830601
Labels:         k8s-app=kube-dns
                pod-template-hash=4290830601
Annotations:    deployment.kubernetes.io/desired-replicas=1
                deployment.kubernetes.io/max-replicas=2
                deployment.kubernetes.io/revision=1
Controlled By:  Deployment/kube-dns
Replicas:       1 current / 1 desired
Pods Status:    1 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:           k8s-app=kube-dns
                    pod-template-hash=4290830601
  Service Account:  kube-dns

service (To access the pod dns service is created)

$ kubectl describe svc/kube-dns  --namespace kube-system
Name:              kube-dns
Namespace:         kube-system
Labels:            k8s-app=kube-dns
                   kubernetes.io/cluster-service=true
                   kubernetes.io/name=KubeDNS
Annotations:       <none>
Selector:          k8s-app=kube-dns
Type:              ClusterIP
IP:                10.96.0.10
Port:              dns  53/UDP
TargetPort:        53/UDP
Endpoints:         172.17.0.2:53
Port:              dns-tcp  53/TCP
TargetPort:        53/TCP
Endpoints:         172.17.0.2:53
Session Affinity:  None
Events:            <none>

pod (Finally smallest unit in kubernetes is POD). We can see that further POD is using the volumes to provide the secrets and configmaps.

$ kubectl describe po/kube-dns-86f4d74b45-wv7j6  --namespace kube-system
Name:           kube-dns-86f4d74b45-wv7j6
Namespace:      kube-system
Node:           minikube/10.0.2.15
Start Time:     Sat, 07 Jul 2018 18:30:37 +0530
Labels:         k8s-app=kube-dns
                pod-template-hash=4290830601
Annotations:    <none>
Status:         Running
IP:             172.17.0.2
Controlled By:  ReplicaSet/kube-dns-86f4d74b45


SKIPPING THE CONTAINER OUTPUT.

Conditions:
  Type           Status
  Initialized    True
  Ready          True
  PodScheduled   True
Volumes:
  kube-dns-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      kube-dns
    Optional:  true
  kube-dns-token-nfcdr:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  kube-dns-token-nfcdr
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     CriticalAddonsOnly
                 node-role.kubernetes.io/master:NoSchedule
                 node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:          <none>

Let’s check the configmap and secrets present on system because these were not listed in all the resource output which dumped initially to see all resources on system.

$ kubectl get configmap --namespace kube-system
NAME                                 DATA      AGE
extension-apiserver-authentication   6         14h
ingress-controller-leader-nginx      0         14h
kube-proxy                           2         14h
kubeadm-config                       1         14h
nginx-load-balancer-conf             2         14h

$ kubectl get secrets --namespace kube-system
NAME                                             TYPE                                  DATA      AGE
attachdetach-controller-token-xzvzr              kubernetes.io/service-account-token   3         14h
bootstrap-signer-token-pz8m9                     kubernetes.io/service-account-token   3         14h
bootstrap-token-cgd2by                           bootstrap.kubernetes.io/token         7         14h
certificate-controller-token-4fcjb               kubernetes.io/service-account-token   3         14h
clusterrole-aggregation-controller-token-fr29l   kubernetes.io/service-account-token   3         14h
cronjob-controller-token-4zd24                   kubernetes.io/service-account-token   3         14h
daemon-set-controller-token-k92hf                kubernetes.io/service-account-token   3         14h
default-token-k68bf                              kubernetes.io/service-account-token   3         14h
deployment-controller-token-jbc7z                kubernetes.io/service-account-token   3         14h
disruption-controller-token-7dwqt                kubernetes.io/service-account-token   3         14h
endpoint-controller-token-vcfhj                  kubernetes.io/service-account-token   3         14h
generic-garbage-collector-token-lk2m5            kubernetes.io/service-account-token   3         14h
horizontal-pod-autoscaler-token-fnvnd            kubernetes.io/service-account-token   3         14h
job-controller-token-nmkcq                       kubernetes.io/service-account-token   3         14h
kube-dns-token-nfcdr                             kubernetes.io/service-account-token   3         14h
kube-proxy-token-t7sjn                           kubernetes.io/service-account-token   3         14h
kubernetes-dashboard-key-holder                  Opaque                                2         14h
namespace-controller-token-l4gj7                 kubernetes.io/service-account-token   3         14h
node-controller-token-g7czq                      kubernetes.io/service-account-token   3         14h
persistent-volume-binder-token-slsv6             kubernetes.io/service-account-token   3         14h
pod-garbage-collector-token-ljt2j                kubernetes.io/service-account-token   3         14h
pv-protection-controller-token-j5sst             kubernetes.io/service-account-token   3         14h
pvc-protection-controller-token-c4hgm            kubernetes.io/service-account-token   3         14h
replicaset-controller-token-7gkdt                kubernetes.io/service-account-token   3         14h
replication-controller-token-gcfhk               kubernetes.io/service-account-token   3         14h
resourcequota-controller-token-7chh8             kubernetes.io/service-account-token   3         14h
service-account-controller-token-q4z2z           kubernetes.io/service-account-token   3         14h
service-controller-token-qcjsh                   kubernetes.io/service-account-token   3         14h
statefulset-controller-token-w547r               kubernetes.io/service-account-token   3         14h
storage-provisioner-token-d48tx                  kubernetes.io/service-account-token   3         14h
token-cleaner-token-vh9c2                        kubernetes.io/service-account-token   3         14h
ttl-controller-token-89vvh                       kubernetes.io/service-account-token   3         14h

I didn’t see kube-dns in config map which was defined in POD output then I read it carefully, it’s an optional component. Just to verify that it’s not present in any namespace.

$ kubectl get configmap --all-namespaces
NAMESPACE     NAME                                 DATA      AGE
kube-public   cluster-info                         2         14h
kube-system   extension-apiserver-authentication   6         14h
kube-system   ingress-controller-leader-nginx      0         14h
kube-system   kube-proxy                           2         14h
kube-system   kubeadm-config                       1         14h
kube-system   nginx-load-balancer-conf             2         14h

Since kubernetes is based on plugin architecture. If you want to see the provided

# pwd
/etc/kubernetes/addons
# ls -lrt
total 28
-rw-r----- 1 root root  273 Jul  7 12:59 storageclass.yaml
-rw-r----- 1 root root 1161 Jul  7 12:59 ingress-configmap.yaml
-rw-r----- 1 root root 3714 Jul  7 12:59 ingress-dp.yaml
-rw-r----- 1 root root 1011 Jul  7 12:59 ingress-svc.yaml
-rw-r----- 1 root root 1709 Jul  7 12:59 storage-provisioner.yaml
-rw-r----- 1 root root 1510 Jul  7 12:59 dashboard-dp.yaml
-rw-r----- 1 root root 1016 Jul  7 12:59 dashboard-svc.yaml