Introduction

This is the english post on this post.

Although Heapster is deprecated on Kubernetes 1.11, but metrics-server looks like developing and a little inconvenience for me (for example, no dashboard support). So I tried to install Heapster to Kubernetes 1.11.3.

But at 2018-10-29, It doesn’t work easily. In this post, I write the small patch to work correctly in the following environment.

$ uname -a
Linux XXX 4.4.0-137-generic #163-Ubuntu SMP x86/64
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.3", GitCommit:"a4529464e4629c21224b3d52edfe0ea91b072862", GitTreeState:"clean", BuildDate:"2018-09-09T18:02:47Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.4", GitCommit:"bf9a868e8ea3d3a8fa53cbb22f566771b3f8068b", GitTreeState:"clean", BuildDate:"2018-10-25T19:06:30Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
$ docker version
Client:
 Version:      17.03.2-ce
 API version:  1.27
 Go version:   go1.6.2
 Git commit:   f5ec1e2
 Built:        Thu Jul  5 23:07:48 2018
 OS/Arch:      linux/amd64

Server:
 Version:      17.03.2-ce
 API version:  1.27 (minimum version 1.12)
 Go version:   go1.6.2
 Git commit:   f5ec1e2
 Built:        Thu Jul  5 23:07:48 2018
 OS/Arch:      linux/amd64
 Experimental: false

Heapster

Clone

First, clone the Heapster repository.

$ git clone https://github.com/kubernetes/heapster/
$ cd heapster

Then, set Grafana Service Type to NodePort and downgrade container version from 5.0.4 to 4.4.3 as follows. The reason why I downgrade the version is the dashboard on Grafana is not shown in 5.0.4.

diff --git a/deploy/kube-config/influxdb/grafana.yaml b/deploy/kube-config/influxdb/grafana.yaml
index 216bd9a..266f47a 100644
--- a/deploy/kube-config/influxdb/grafana.yaml
+++ b/deploy/kube-config/influxdb/grafana.yaml
@@ -13,7 +13,7 @@ spec:
     spec:
       containers:
       - name: grafana
-        image: k8s.gcr.io/heapster-grafana-amd64:v5.0.4
+        image: k8s.gcr.io/heapster-grafana-amd64:v4.4.3
         ports:
         - containerPort: 3000
           protocol: TCP
@@ -64,7 +64,7 @@ spec:
   # or through a public IP.
   # type: LoadBalancer
   # You could also use NodePort to expose the service at a randomly-generated port
-  # type: NodePort
+  type: NodePort
   ports:
   - port: 80
     targetPort: 3000

In this point, The pods are launched if I kubectl apply under deploy/kube-config/rbac and deploy/kube-config/influxdb , but the following error logs are generated and not worked correctly.

E1028 07:39:05.011439       1 manager.go:101] Error in scraping containers from Kubelet:XX.XX.XX.XX:10255: failed to get all container stats from Kubelet URL "http://XX.XX.XX.XX:10255/stats/container/": Post http://XX.XX.XX.XX:10255/stats/container/: dial tcp XX.XX.XX.XX:10255:
 getsockopt: connection refused

Apply patch

After googling the error message, I found this issue.

It looks the port to access are changed and I need to deal with HTTPS.

So I edit deploy/kube-config/influxdb/heapster.yaml and deploy/kube-config/rbac/heapster-rbac.yaml as below.

diff --git a/deploy/kube-config/influxdb/heapster.yaml b/deploy/kube-config/influxdb/heapster.yaml
index e820ca5..195061a 100644
--- a/deploy/kube-config/influxdb/heapster.yaml
+++ b/deploy/kube-config/influxdb/heapster.yaml
@@ -24,7 +24,7 @@ spec:
         imagePullPolicy: IfNotPresent
         command:
         - /heapster
-        - --source=kubernetes:https://kubernetes.default
+        - --source=kubernetes.summary_api:''?useServiceAccount=true&kubeletHttps=true&kubeletPort=10250&insecure=true
         - --sink=influxdb:http://monitoring-influxdb.kube-system.svc:8086
 ---
 apiVersion: v1
diff --git a/deploy/kube-config/rbac/heapster-rbac.yaml b/deploy/kube-config/rbac/heapster-rbac.yaml
index 6e63803..1f982fb 100644
--- a/deploy/kube-config/rbac/heapster-rbac.yaml
+++ b/deploy/kube-config/rbac/heapster-rbac.yaml
@@ -5,7 +5,7 @@ metadata:
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
-  name: system:heapster
+  name: heapster
 subjects:
 - kind: ServiceAccount
   name: heapster

Moreover, I create deploy/kube-config/rbac/heapster-role.yaml.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: heapster
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - nodes
  - namespaces
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - extensions
  resources:
  - deployments
  verbs:
  - get
  - list
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - nodes/stats
  verbs:
  - get

Create pods

Finally, I apply these configurations.

$ kubectl create -f ./deploy/kube-config/rbac/
clusterrolebinding.rbac.authorization.k8s.io/heapster created
clusterrole.rbac.authorization.k8s.io/heapster created
$ kubectl create -f ./deploy/kube-config/influxdb/
deployment.extensions/monitoring-grafana created
service/monitoring-grafana created
serviceaccount/heapster created
deployment.extensions/heapster created
service/heapster created
deployment.extensions/monitoring-influxdb created
service/monitoring-influxdb created

After updating, I can configure launching Heapster, Grafana, and Influx DB by kubectl get all -n kube-system. And after waiting for minutes, I can see the metrics by kubectl top node.

Thought

When I faced to some miss behavior on Kubernetes, I often find a clue of solution by seeing the log by kubectl logs.

Heapster on Kubernetes 1.11.3

Contents