Kubernetes metrics-server

metrics-server 는 kubernetes의 오토스케일링 파이프라인들에 유용한 자원으로 kubernetes autoscaler를 사용하기 위해선 필수 자원이다.

metrics-server는 kubelet으로부터 metric을 수집하고 Metric API를 통해서 kubernetes api에서 노출된다.

복잡한 말이긴 하지만 kubectl top 명령어를 사용하거나 Horizontal Pod Autoscaler, Vertical Pod Autoscaler 를 사용하기 위해선 필수적인 자원이다.

당연하겠지만 kubernetes cluster에서 설치되어야 한다.

metrics-server 설치

metrics-server를 설치하기 위해선 별도의 yaml 파일을 생성하거나 아래처럼 직접 git 에서 직접 설치가 가능하다.

$ kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

serviceaccount/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
service/metrics-server created
deployment.apps/metrics-server created
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created

$ kubectl get pods -n kube-system | grep metric
metrics-server-5d5d6598c7-kk6g8                            0/1     Running   0          19s

위에 처럼 설치할 경우 kubelet certificate가 필요하다. 그래서 cert validation을 피하기 위해선 --kubelet-insecure-tls flag가 필요하다 

테스트 환경에서 cert validation을 패스하고 insecure로 실행해보자.

~$ mkdir metrics-server
~$ cd metrics-server
~/metrics-server$ wget https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

# components.yaml 파일을 오픈한뒤 아래처럼 deployment 실행 인자에 --kubelet-insecure-tls 를 추가해준다.

---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  selector:
    matchLabels:
      k8s-app: metrics-server
  strategy:
    rollingUpdate:
      maxUnavailable: 0
  template:
    metadata:
      labels:
        k8s-app: metrics-server
    spec:
      containers:
      - args:
        - --cert-dir=/tmp
        - --secure-port=443
        - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
        - --kubelet-use-node-status-port
        - --metric-resolution=15s
        - --kubelet-insecure-tls               <-----------여기 추가
        image: k8s.gcr.io/metrics-server/metrics-server:v0.5.1
        imagePullPolicy: IfNotPresent
        


# 실행 확인


$ kubectl get pods -n kube-system  | grep metric
metrics-server-d69dd899-7ksrz                              1/1     Running   0          11m

$ kubectl logs -f metrics-server-d69dd899-7ksrz -n kube-system
I1014 09:46:17.398903       1 serving.go:325] Generated self-signed cert (/tmp/apiserver.crt, /tmp/apiserver.key)
I1014 09:46:19.086702       1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController
I1014 09:46:19.086745       1 shared_informer.go:240] Waiting for caches to sync for RequestHeaderAuthRequestController
I1014 09:46:19.086797       1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I1014 09:46:19.086808       1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I1014 09:46:19.086836       1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I1014 09:46:19.086858       1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I1014 09:46:19.087228       1 dynamic_serving_content.go:130] Starting serving-cert::/tmp/apiserver.crt::/tmp/apiserver.key
I1014 09:46:19.088055       1 secure_serving.go:197] Serving securely on [::]:4443
I1014 09:46:19.088270       1 tlsconfig.go:240] Starting DynamicServingCertificateController
I1014 09:46:19.187012       1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I1014 09:46:19.187063       1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I1014 09:46:19.187020       1 shared_informer.go:247] Caches are synced for RequestHeaderAuthRequestController

 

kubectl top 동작 확인

$ kubectl top pods
NAME                                              CPU(cores)   MEMORY(bytes)
foo-6579459448-4ddc7                              3m           69Mi
foo-6579459448-4g4mg                              2m           69Mi
foo-6579459448-5pgk2                              2m           68Mi

 

kubectl top 동작 확인 시 아래와 같은 에러가 발생하면 여기 에서 RBAC 을 추가한 이후 다시 확인해보자.

$ kubectl top pod
Error from server (Forbidden): pods.metrics.k8s.io is forbidden: User "front-proxy-client" cannot list resource "pods" in API group "metrics.k8s.io" in the namespace "default"