--- title: "Argocd vs Helmfile: Helmfile" date: 2023-02-17T12:48:51+01:00 draft: false ShowToc: false --- In two previous posts I've described how it's possible to install a couple of applications with [`Applications`]({{< ref "argocd-vs-helmfile-application" >}}) and [`ApplicationSets`]({{< ref "argocd-vs-helmfile-applicationset" >}}), and this one is the last in a row. And here I'm going to install the same applications (`VPA` and `Goldilocks`) with helmfile, and I will tell why I think that it's better than `ArgoCD` So let's start. Here you can find the [initial config](https://git.badhouseplants.net/allanger/helmfile-vs-argo/src/branch/helmfile-main). Let's see what we got here: The main file is `/helmfile.yaml` ```YAML --- {{ readFile "releases.yaml" }} bases: - environments.yaml - repositories.yaml releases: helmfiles: - path: {{.Environment.Name }}/helmfile.yaml ``` You can see several imports here, let's check them one by one: ```YAML # releases.yaml <- here we will define all the charts that are going to be used by helmfile. It's a templating layer, so we don't have to copy-paste a lot of yaml stuff --- templates: # It's supposed to be empty when nothing is installed, but I've decided to show that helmfile also can be used to manage CRDs. With this hooks, you'll be able to install CRDs with helmfile, see diffs when updating and update them while updating releases. But I need to say that I haven't checked how it's wotking on big systems, so consider this thing experimental crd-management-hook: &crd-management-hook hooks: - events: ["preapply"] showlogs: true command: "sh" args: - -c - "helm show crds {{ .Release.Chart }} --version {{ .Release.Version }}| kubectl apply -f -" - events: ["prepare"] showlogs: true command: "sh" args: - -c - "helm show crds {{ .Release.Chart }} --version {{ .Release.Version }} | kubectl diff -f - || true" - events: ["postuninstall"] showlogs: true command: "sh" args: - -c - "helm show crds {{ .Release.Chart }} --version {{ .Release.Version }} | kubectl delete -f -" # environments.yaml --- # Here you can define all you clusters so you can sync helmfiles using the `-e` flag # helmfile -e cluster_test sync environments: cluster_test: kubeContext: kind-kind cluster1: kubeContext: kind-kind cluster2: kubeContext: kind-kind # repositories.yaml --- # it's empty now, because we have nothing installed yet. But here we will add all the helm repos to avoid copy-paste again. repositories: ``` Also there is a more complicated and less obvious import: ```YAML helmfiles: - path: {{.Environment.Name }}/helmfile.yaml ``` It's going to import helmfiles that are not used across all clusters. So if you want to install something to `cluster-2` accross, you will add it to the `/cluster2/helmfile.yaml` and sync the main helmfile. I will show an example later. So we're all set and ready to begin installing new stuff. - Add a new repo to `repositories.yaml` ```YAML repositories: - name: fairwinds-stable url: https://charts.fairwinds.com/stable ``` - Add new charts to `releases.yaml` ```YAML releases: vpa: &vpa name: vpa chart: fairwinds-stable/vpa version: 1.6.0 values: - common/{{ .Release.Name }}/values.yaml inherit: template: crd-management-hook # <- Here we are using the crd-management-hook, so VPA CRDs are managed outside of helm itself goldilocks: &goldilocks name: goldilocks chart: fairwinds-stable/goldilocks version: 6.5.0 ``` And now we only need to say to which clusters we want to install those releases. So to the main `helmfile.yaml` I'm adding this: ```YAML releases: - <<: *vpa installed: true namespace: vpa-system createNamespace: true ``` And to `cluster2/helmfile.yaml` and `cluster_test/helmfile.yaml` I'm adding: ```YAML releases: - <<: *goldilocks installed: true namespace: vpa-system ``` Let's see how the CI could be implemented with this setup. - If branch is not `main` - `helmfile -e cluster_test apply` - `helmfile -e cluster1 diff` - `helmfile -e cluster2 diff` - If branch is `main` - `helmfile -e cluster_test apply` - `helmfile -e cluster1 apply` - `helmfile -e cluster2 apply` {{< details "Diffs will look like that:" >}} ```YAML Comparing release=goldilocks, chart=fairwinds-stable/goldilocks ******************** Release was not present in Helm. Diff will show entire contents as new. ******************** vpa-system, goldilocks-controller, ClusterRole (rbac.authorization.k8s.io) has been added: - + # Source: goldilocks/templates/controller-clusterrole.yaml + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRole + metadata: + name: goldilocks-controller + labels: + app.kubernetes.io/name: goldilocks + helm.sh/chart: goldilocks-6.5.0 + app.kubernetes.io/instance: goldilocks + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: controller + rules: + - apiGroups: + - 'apps' + resources: + - '*' + verbs: + - 'get' + - 'list' + - 'watch' + - apiGroups: + - '' + resources: + - 'namespaces' + - 'pods' + verbs: + - 'get' + - 'list' + - 'watch' + - apiGroups: + - 'autoscaling.k8s.io' + resources: + - 'verticalpodautoscalers' + verbs: + - 'get' + - 'list' + - 'create' + - 'delete' + - 'update' + - apiGroups: + - 'argoproj.io' + resources: + - rollouts + verbs: + - 'get' + - 'list' + - 'watch' vpa-system, goldilocks-controller, ClusterRoleBinding (rbac.authorization.k8s.io) has been added: - + # Source: goldilocks/templates/controller-clusterrolebinding.yaml + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRoleBinding + metadata: + name: goldilocks-controller + labels: + app.kubernetes.io/name: goldilocks + helm.sh/chart: goldilocks-6.5.0 + app.kubernetes.io/instance: goldilocks + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: controller + roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: goldilocks-controller + subjects: + - kind: ServiceAccount + name: goldilocks-controller + namespace: vpa-system vpa-system, goldilocks-controller, Deployment (apps) has been added: - + # Source: goldilocks/templates/controller-deployment.yaml + apiVersion: apps/v1 + kind: Deployment + metadata: + name: goldilocks-controller + labels: + app.kubernetes.io/name: goldilocks + helm.sh/chart: goldilocks-6.5.0 + app.kubernetes.io/instance: goldilocks + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: controller + spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: goldilocks + app.kubernetes.io/instance: goldilocks + app.kubernetes.io/component: controller + template: + metadata: + labels: + app.kubernetes.io/name: goldilocks + app.kubernetes.io/instance: goldilocks + app.kubernetes.io/component: controller + spec: + serviceAccountName: goldilocks-controller + securityContext: + {} + containers: + - name: goldilocks + image: "us-docker.pkg.dev/fairwinds-ops/oss/goldilocks:v4.6.2" + imagePullPolicy: Always + command: + - /goldilocks + - controller + - -v2 + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 10324 + resources: + limits: + cpu: 25m + memory: 32Mi + requests: + cpu: 25m + memory: 32Mi vpa-system, goldilocks-controller, ServiceAccount (v1) has been added: - + # Source: goldilocks/templates/controller-serviceaccount.yaml + apiVersion: v1 + kind: ServiceAccount + metadata: + name: goldilocks-controller + labels: + app.kubernetes.io/name: goldilocks + helm.sh/chart: goldilocks-6.5.0 + app.kubernetes.io/instance: goldilocks + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: controller vpa-system, goldilocks-dashboard, ClusterRole (rbac.authorization.k8s.io) has been added: - + # Source: goldilocks/templates/dashboard-clusterrole.yaml + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRole + metadata: + name: goldilocks-dashboard + labels: + app.kubernetes.io/name: goldilocks + helm.sh/chart: goldilocks-6.5.0 + app.kubernetes.io/instance: goldilocks + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: dashboard + rules: + - apiGroups: + - 'autoscaling.k8s.io' + resources: + - 'verticalpodautoscalers' + verbs: + - 'get' + - 'list' + - apiGroups: + - 'apps' + resources: + - '*' + verbs: + - 'get' + - 'list' + - apiGroups: + - '' + resources: + - 'namespaces' + - 'pods' + verbs: + - 'get' + - 'list' + - apiGroups: + - 'argoproj.io' + resources: + - rollouts + verbs: + - 'get' + - 'list' vpa-system, goldilocks-dashboard, ClusterRoleBinding (rbac.authorization.k8s.io) has been added: - + # Source: goldilocks/templates/dashboard-clusterrolebinding.yaml + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRoleBinding + metadata: + name: goldilocks-dashboard + labels: + app.kubernetes.io/name: goldilocks + helm.sh/chart: goldilocks-6.5.0 + app.kubernetes.io/instance: goldilocks + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: dashboard + roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: goldilocks-dashboard + subjects: + - kind: ServiceAccount + name: goldilocks-dashboard + namespace: vpa-system vpa-system, goldilocks-dashboard, Deployment (apps) has been added: - + # Source: goldilocks/templates/dashboard-deployment.yaml + apiVersion: apps/v1 + kind: Deployment + metadata: + name: goldilocks-dashboard + labels: + app.kubernetes.io/name: goldilocks + helm.sh/chart: goldilocks-6.5.0 + app.kubernetes.io/instance: goldilocks + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: dashboard + spec: + replicas: 2 + selector: + matchLabels: + app.kubernetes.io/name: goldilocks + app.kubernetes.io/instance: goldilocks + app.kubernetes.io/component: dashboard + template: + metadata: + labels: + app.kubernetes.io/name: goldilocks + app.kubernetes.io/instance: goldilocks + app.kubernetes.io/component: dashboard + spec: + serviceAccountName: goldilocks-dashboard + securityContext: + {} + containers: + - name: goldilocks + image: "us-docker.pkg.dev/fairwinds-ops/oss/goldilocks:v4.6.2" + imagePullPolicy: Always + command: + - /goldilocks + - dashboard + - --exclude-containers=linkerd-proxy,istio-proxy + - -v2 + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 10324 + ports: + - name: http + containerPort: 8080 + protocol: TCP + livenessProbe: + httpGet: + path: /health + port: http + readinessProbe: + httpGet: + path: /health + port: http + resources: + limits: + cpu: 25m + memory: 32Mi + requests: + cpu: 25m + memory: 32Mi vpa-system, goldilocks-dashboard, Service (v1) has been added: - + # Source: goldilocks/templates/dashboard-service.yaml + apiVersion: v1 + kind: Service + metadata: + name: goldilocks-dashboard + labels: + app.kubernetes.io/name: goldilocks + helm.sh/chart: goldilocks-6.5.0 + app.kubernetes.io/instance: goldilocks + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: dashboard + spec: + type: ClusterIP + ports: + - port: 80 + targetPort: http + protocol: TCP + name: http + selector: + app.kubernetes.io/name: goldilocks + app.kubernetes.io/instance: goldilocks + app.kubernetes.io/component: dashboard vpa-system, goldilocks-dashboard, ServiceAccount (v1) has been added: - + # Source: goldilocks/templates/dashboard-serviceaccount.yaml + apiVersion: v1 + kind: ServiceAccount + metadata: + name: goldilocks-dashboard + labels: + app.kubernetes.io/name: goldilocks + helm.sh/chart: goldilocks-6.5.0 + app.kubernetes.io/instance: goldilocks + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: dashboard hook[prepare] logs | diff -u -N /var/folders/w1/27ptcr29547f0g8732kmffwm0000gn/T/LIVE-4051758900/apiextensions.k8s.io.v1.CustomResourceDefinition..verticalpodautoscalercheckpoints.autoscaling.k8s.io /var/folders/w1/27ptcr29547f0g8732kmffwm0000gn/T/MERGED-3664876659/apiextensions.k8s.io.v1.CustomResourceDefinition..verticalpodautoscalercheckpoints.autoscaling.k8s.io hook[prepare] logs | --- /var/folders/w1/27ptcr29547f0g8732kmffwm0000gn/T/LIVE-4051758900/apiextensions.k8s.io.v1.CustomResourceDefinition..verticalpodautoscalercheckpoints.autoscaling.k8s.io 2023-02-17 13:15:29 hook[prepare] logs | +++ /var/folders/w1/27ptcr29547f0g8732kmffwm0000gn/T/MERGED-3664876659/apiextensions.k8s.io.v1.CustomResourceDefinition..verticalpodautoscalercheckpoints.autoscaling.k8s.io 2023-02-17 13:15:29 hook[prepare] logs | @@ -0,0 +1,216 @@ hook[prepare] logs | +apiVersion: apiextensions.k8s.io/v1 hook[prepare] logs | +kind: CustomResourceDefinition hook[prepare] logs | +metadata: hook[prepare] logs | + annotations: hook[prepare] logs | + api-approved.kubernetes.io: https://github.com/kubernetes/kubernetes/pull/63797 hook[prepare] logs | + controller-gen.kubebuilder.io/version: v0.4.0 hook[prepare] logs | + creationTimestamp: "2023-02-17T12:15:29Z" hook[prepare] logs | + generation: 1 hook[prepare] logs | + name: verticalpodautoscalercheckpoints.autoscaling.k8s.io hook[prepare] logs | + uid: bd3d2d46-b0c1-48c4-bc72-968cebfd7640 hook[prepare] logs | +spec: hook[prepare] logs | + conversion: hook[prepare] logs | + strategy: None hook[prepare] logs | + group: autoscaling.k8s.io hook[prepare] logs | + names: hook[prepare] logs | + kind: VerticalPodAutoscalerCheckpoint hook[prepare] logs | + listKind: VerticalPodAutoscalerCheckpointList hook[prepare] logs | + plural: verticalpodautoscalercheckpoints hook[prepare] logs | + shortNames: hook[prepare] logs | + - vpacheckpoint hook[prepare] logs | + singular: verticalpodautoscalercheckpoint hook[prepare] logs | + scope: Namespaced hook[prepare] logs | + versions: hook[prepare] logs | + - name: v1 hook[prepare] logs | + schema: hook[prepare] logs | + openAPIV3Schema: hook[prepare] logs | + description: VerticalPodAutoscalerCheckpoint is the checkpoint of the internal hook[prepare] logs | + state of VPA that is used for recovery after recommender's restart. hook[prepare] logs | + properties: hook[prepare] logs | + apiVersion: hook[prepare] logs | + description: 'APIVersion defines the versioned schema of this representation hook[prepare] logs | + of an object. Servers should convert recognized schemas to the latest hook[prepare] logs | + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' hook[prepare] logs | + type: string hook[prepare] logs | + kind: hook[prepare] logs | + description: 'Kind is a string value representing the REST resource this hook[prepare] logs | + object represents. Servers may infer this from the endpoint the client hook[prepare] logs | + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' hook[prepare] logs | + type: string hook[prepare] logs | + metadata: hook[prepare] logs | + type: object hook[prepare] logs | + spec: hook[prepare] logs | + description: 'Specification of the checkpoint. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status.' hook[prepare] logs | + properties: hook[prepare] logs | + containerName: hook[prepare] logs | + description: Name of the checkpointed container. hook[prepare] logs | + type: string hook[prepare] logs | + vpaObjectName: hook[prepare] logs | + description: Name of the VPA object that stored VerticalPodAutoscalerCheckpoint hook[prepare] logs | + object. hook[prepare] logs | + type: string hook[prepare] logs | + type: object hook[prepare] logs | + status: hook[prepare] logs | + description: Data of the checkpoint. hook[prepare] logs | + properties: hook[prepare] logs | + cpuHistogram: hook[prepare] logs | + description: Checkpoint of histogram for consumption of CPU. hook[prepare] logs | + properties: hook[prepare] logs | + bucketWeights: hook[prepare] logs | + description: Map from bucket index to bucket weight. hook[prepare] logs | + type: object hook[prepare] logs | + x-kubernetes-preserve-unknown-fields: true hook[prepare] logs | + referenceTimestamp: hook[prepare] logs | + description: Reference timestamp for samples collected within hook[prepare] logs | + this histogram. hook[prepare] logs | + format: date-time hook[prepare] logs | + nullable: true hook[prepare] logs | + type: string hook[prepare] logs | + totalWeight: hook[prepare] logs | + description: Sum of samples to be used as denominator for weights hook[prepare] logs | + from BucketWeights. hook[prepare] logs | + type: number hook[prepare] logs | + type: object hook[prepare] logs | + firstSampleStart: hook[prepare] logs | + description: Timestamp of the fist sample from the histograms. hook[prepare] logs | + format: date-time hook[prepare] logs | + nullable: true hook[prepare] logs | + type: string hook[prepare] logs | + lastSampleStart: hook[prepare] logs | + description: Timestamp of the last sample from the histograms. hook[prepare] logs | + format: date-time hook[prepare] logs | + nullable: true hook[prepare] logs | + type: string hook[prepare] logs | + lastUpdateTime: hook[prepare] logs | + description: The time when the status was last refreshed. hook[prepare] logs | + format: date-time hook[prepare] logs | + nullable: true hook[prepare] logs | + type: string hook[prepare] logs | + memoryHistogram: hook[prepare] logs | + description: Checkpoint of histogram for consumption of memory. hook[prepare] logs | + properties: hook[prepare] logs | + bucketWeights: hook[prepare] logs | + description: Map from bucket index to bucket weight. hook[prepare] logs | + type: object hook[prepare] logs | + x-kubernetes-preserve-unknown-fields: true hook[prepare] logs | + referenceTimestamp: hook[prepare] logs | + description: Reference timestamp for samples collected within hook[prepare] logs | + this histogram. hook[prepare] logs | + format: date-time hook[prepare] logs | + nullable: true hook[prepare] logs | + type: string hook[prepare] logs | + totalWeight: hook[prepare] logs | + description: Sum of samples to be used as denominator for weights hook[prepare] logs | + from BucketWeights. hook[prepare] logs | + type: number hook[prepare] logs | + type: object hook[prepare] logs | + totalSamplesCount: hook[prepare] logs | + description: Total number of samples in the histograms. hook[prepare] logs | + type: integer hook[prepare] logs | + version: hook[prepare] logs | + description: Version of the format of the stored data. hook[prepare] logs | + type: string hook[prepare] logs | + type: object hook[prepare] logs | + type: object hook[prepare] logs | + served: true hook[prepare] logs | + storage: true hook[prepare] logs | + - name: v1beta2 hook[prepare] logs | + schema: hook[prepare] logs | + openAPIV3Schema: hook[prepare] logs | + description: VerticalPodAutoscalerCheckpoint is the checkpoint of the internal hook[prepare] logs | + state of VPA that is used for recovery after recommender's restart. hook[prepare] logs | + properties: hook[prepare] logs | + apiVersion: hook[prepare] logs | + description: 'APIVersion defines the versioned schema of this representation hook[prepare] logs | + of an object. Servers should convert recognized schemas to the latest hook[prepare] logs | + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' hook[prepare] logs | + type: string hook[prepare] logs | + kind: hook[prepare] logs | + description: 'Kind is a string value representing the REST resource this hook[prepare] logs | + object represents. Servers may infer this from the endpoint the client hook[prepare] logs | + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' hook[prepare] logs | + type: string hook[prepare] logs | + metadata: hook[prepare] logs | + type: object hook[prepare] logs | + spec: hook[prepare] logs | + description: 'Specification of the checkpoint. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status.' hook[prepare] logs | + properties: hook[prepare] logs | + containerName: hook[prepare] logs | + description: Name of the checkpointed container. hook[prepare] logs | + type: string hook[prepare] logs | + vpaObjectName: hook[prepare] logs | + description: Name of the VPA object that stored VerticalPodAutoscalerCheckpoint hook[prepare] logs | + object. hook[prepare] logs | + type: string hook[prepare] logs | + type: object hook[prepare] logs | + status: hook[prepare] logs | + description: Data of the checkpoint. hook[prepare] logs | + properties: hook[prepare] logs | + cpuHistogram: hook[prepare] logs | + description: Checkpoint of histogram for consumption of CPU. hook[prepare] logs | + properties: hook[prepare] logs | + bucketWeights: hook[prepare] logs | + description: Map from bucket index to bucket weight. hook[prepare] logs | + type: object hook[prepare] logs | + x-kubernetes-preserve-unknown-fields: true hook[prepare] logs | + referenceTimestamp: hook[prepare] logs | + description: Reference timestamp for samples collected within hook[prepare] logs | + this histogram. hook[prepare] logs | + format: date-time hook[prepare] logs | + nullable: true hook[prepare] logs | + type: string hook[prepare] logs | + totalWeight: hook[prepare] logs | + description: Sum of samples to be used as denominator for weights hook[prepare] logs | + from BucketWeights. hook[prepare] logs | + type: number hook[prepare] logs | + type: object hook[prepare] logs | + firstSampleStart: hook[prepare] logs | + description: Timestamp of the fist sample from the histograms. hook[prepare] logs | + format: date-time hook[prepare] logs | + nullable: true hook[prepare] logs | + type: string hook[prepare] logs | + lastSampleStart: hook[prepare] logs | + description: Timestamp of the last sample from the histograms. hook[prepare] logs | + format: date-time hook[prepare] logs | + nullable: true hook[prepare] logs | + type: string hook[prepare] logs | + lastUpdateTime: hook[prepare] logs | + description: The time when the status was last refreshed. hook[prepare] logs | + format: date-time hook[prepare] logs | + nullable: true hook[prepare] logs | + type: string hook[prepare] logs | + memoryHistogram: hook[prepare] logs | + description: Checkpoint of histogram for consumption of memory. hook[prepare] logs | + properties: hook[prepare] logs | + bucketWeights: hook[prepare] logs | + description: Map from bucket index to bucket weight. hook[prepare] logs | + type: object hook[prepare] logs | + x-kubernetes-preserve-unknown-fields: true hook[prepare] logs | + referenceTimestamp: hook[prepare] logs | + description: Reference timestamp for samples collected within hook[prepare] logs | + this histogram. hook[prepare] logs | + format: date-time hook[prepare] logs | + nullable: true hook[prepare] logs | + type: string hook[prepare] logs | + totalWeight: hook[prepare] logs | + description: Sum of samples to be used as denominator for weights hook[prepare] logs | + from BucketWeights. hook[prepare] logs | + type: number hook[prepare] logs | + type: object hook[prepare] logs | + totalSamplesCount: hook[prepare] logs | + description: Total number of samples in the histograms. hook[prepare] logs | + type: integer hook[prepare] logs | + version: hook[prepare] logs | + description: Version of the format of the stored data. hook[prepare] logs | + type: string hook[prepare] logs | + type: object hook[prepare] logs | + type: object hook[prepare] logs | + served: true hook[prepare] logs | + storage: false hook[prepare] logs | +status: hook[prepare] logs | + acceptedNames: hook[prepare] logs | + kind: "" hook[prepare] logs | + plural: "" hook[prepare] logs | + conditions: null hook[prepare] logs | + storedVersions: hook[prepare] logs | + - v1 hook[prepare] logs | diff -u -N /var/folders/w1/27ptcr29547f0g8732kmffwm0000gn/T/LIVE-4051758900/apiextensions.k8s.io.v1.CustomResourceDefinition..verticalpodautoscalers.autoscaling.k8s.io /var/folders/w1/27ptcr29547f0g8732kmffwm0000gn/T/MERGED-3664876659/apiextensions.k8s.io.v1.CustomResourceDefinition..verticalpodautoscalers.autoscaling.k8s.io hook[prepare] logs | --- /var/folders/w1/27ptcr29547f0g8732kmffwm0000gn/T/LIVE-4051758900/apiextensions.k8s.io.v1.CustomResourceDefinition..verticalpodautoscalers.autoscaling.k8s.io 2023-02-17 13:15:29 hook[prepare] logs | +++ /var/folders/w1/27ptcr29547f0g8732kmffwm0000gn/T/MERGED-3664876659/apiextensions.k8s.io.v1.CustomResourceDefinition..verticalpodautoscalers.autoscaling.k8s.io 2023-02-17 13:15:29 hook[prepare] logs | @@ -0,0 +1,550 @@ hook[prepare] logs | +apiVersion: apiextensions.k8s.io/v1 hook[prepare] logs | +kind: CustomResourceDefinition hook[prepare] logs | +metadata: hook[prepare] logs | + annotations: hook[prepare] logs | + api-approved.kubernetes.io: https://github.com/kubernetes/kubernetes/pull/63797 hook[prepare] logs | + controller-gen.kubebuilder.io/version: v0.4.0 hook[prepare] logs | + creationTimestamp: "2023-02-17T12:15:29Z" hook[prepare] logs | + generation: 1 hook[prepare] logs | + name: verticalpodautoscalers.autoscaling.k8s.io hook[prepare] logs | + uid: 91c297f7-3d26-43f3-bce8-14885ca00c10 hook[prepare] logs | +spec: hook[prepare] logs | + conversion: hook[prepare] logs | + strategy: None hook[prepare] logs | + group: autoscaling.k8s.io hook[prepare] logs | + names: hook[prepare] logs | + kind: VerticalPodAutoscaler hook[prepare] logs | + listKind: VerticalPodAutoscalerList hook[prepare] logs | + plural: verticalpodautoscalers hook[prepare] logs | + shortNames: hook[prepare] logs | + - vpa hook[prepare] logs | + singular: verticalpodautoscaler hook[prepare] logs | + scope: Namespaced hook[prepare] logs | + versions: hook[prepare] logs | + - additionalPrinterColumns: hook[prepare] logs | + - jsonPath: .spec.updatePolicy.updateMode hook[prepare] logs | + name: Mode hook[prepare] logs | + type: string hook[prepare] logs | + - jsonPath: .status.recommendation.containerRecommendations[0].target.cpu hook[prepare] logs | + name: CPU hook[prepare] logs | + type: string hook[prepare] logs | + - jsonPath: .status.recommendation.containerRecommendations[0].target.memory hook[prepare] logs | + name: Mem hook[prepare] logs | + type: string hook[prepare] logs | + - jsonPath: .status.conditions[?(@.type=='RecommendationProvided')].status hook[prepare] logs | + name: Provided hook[prepare] logs | + type: string hook[prepare] logs | + - jsonPath: .metadata.creationTimestamp hook[prepare] logs | + name: Age hook[prepare] logs | + type: date hook[prepare] logs | + name: v1 hook[prepare] logs | + schema: hook[prepare] logs | + openAPIV3Schema: hook[prepare] logs | + description: VerticalPodAutoscaler is the configuration for a vertical pod hook[prepare] logs | + autoscaler, which automatically manages pod resources based on historical hook[prepare] logs | + and real time resource utilization. hook[prepare] logs | + properties: hook[prepare] logs | + apiVersion: hook[prepare] logs | + description: 'APIVersion defines the versioned schema of this representation hook[prepare] logs | + of an object. Servers should convert recognized schemas to the latest hook[prepare] logs | + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' hook[prepare] logs | + type: string hook[prepare] logs | + kind: hook[prepare] logs | + description: 'Kind is a string value representing the REST resource this hook[prepare] logs | + object represents. Servers may infer this from the endpoint the client hook[prepare] logs | + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' hook[prepare] logs | + type: string hook[prepare] logs | + metadata: hook[prepare] logs | + type: object hook[prepare] logs | + spec: hook[prepare] logs | + description: 'Specification of the behavior of the autoscaler. More info: hook[prepare] logs | + https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status.' hook[prepare] logs | + properties: hook[prepare] logs | + recommenders: hook[prepare] logs | + description: Recommender responsible for generating recommendation hook[prepare] logs | + for this object. List should be empty (then the default recommender hook[prepare] logs | + will generate the recommendation) or contain exactly one recommender. hook[prepare] logs | + items: hook[prepare] logs | + description: VerticalPodAutoscalerRecommenderSelector points to hook[prepare] logs | + a specific Vertical Pod Autoscaler recommender. In the future hook[prepare] logs | + it might pass parameters to the recommender. hook[prepare] logs | + properties: hook[prepare] logs | + name: hook[prepare] logs | + description: Name of the recommender responsible for generating hook[prepare] logs | + recommendation for this object. hook[prepare] logs | + type: string hook[prepare] logs | + required: hook[prepare] logs | + - name hook[prepare] logs | + type: object hook[prepare] logs | + type: array hook[prepare] logs | + resourcePolicy: hook[prepare] logs | + description: Controls how the autoscaler computes recommended resources. hook[prepare] logs | + The resource policy may be used to set constraints on the recommendations hook[prepare] logs | + for individual containers. If not specified, the autoscaler computes hook[prepare] logs | + recommended resources for all containers in the pod, without additional hook[prepare] logs | + constraints. hook[prepare] logs | + properties: hook[prepare] logs | + containerPolicies: hook[prepare] logs | + description: Per-container resource policies. hook[prepare] logs | + items: hook[prepare] logs | + description: ContainerResourcePolicy controls how autoscaler hook[prepare] logs | + computes the recommended resources for a specific container. hook[prepare] logs | + properties: hook[prepare] logs | + containerName: hook[prepare] logs | + description: Name of the container or DefaultContainerResourcePolicy, hook[prepare] logs | + in which case the policy is used by the containers that hook[prepare] logs | + don't have their own policy specified. hook[prepare] logs | + type: string hook[prepare] logs | + controlledResources: hook[prepare] logs | + description: Specifies the type of recommendations that hook[prepare] logs | + will be computed (and possibly applied) by VPA. If not hook[prepare] logs | + specified, the default of [ResourceCPU, ResourceMemory] hook[prepare] logs | + will be used. hook[prepare] logs | + items: hook[prepare] logs | + description: ResourceName is the name identifying various hook[prepare] logs | + resources in a ResourceList. hook[prepare] logs | + type: string hook[prepare] logs | + type: array hook[prepare] logs | + controlledValues: hook[prepare] logs | + description: Specifies which resource values should be controlled. hook[prepare] logs | + The default is "RequestsAndLimits". hook[prepare] logs | + enum: hook[prepare] logs | + - RequestsAndLimits hook[prepare] logs | + - RequestsOnly hook[prepare] logs | + type: string hook[prepare] logs | + maxAllowed: hook[prepare] logs | + additionalProperties: hook[prepare] logs | + anyOf: hook[prepare] logs | + - type: integer hook[prepare] logs | + - type: string hook[prepare] logs | + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ hook[prepare] logs | + x-kubernetes-int-or-string: true hook[prepare] logs | + description: Specifies the maximum amount of resources that hook[prepare] logs | + will be recommended for the container. The default is hook[prepare] logs | + no maximum. hook[prepare] logs | + type: object hook[prepare] logs | + minAllowed: hook[prepare] logs | + additionalProperties: hook[prepare] logs | + anyOf: hook[prepare] logs | + - type: integer hook[prepare] logs | + - type: string hook[prepare] logs | + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ hook[prepare] logs | + x-kubernetes-int-or-string: true hook[prepare] logs | + description: Specifies the minimal amount of resources that hook[prepare] logs | + will be recommended for the container. The default is hook[prepare] logs | + no minimum. hook[prepare] logs | + type: object hook[prepare] logs | + mode: hook[prepare] logs | + description: Whether autoscaler is enabled for the container. hook[prepare] logs | + The default is "Auto". hook[prepare] logs | + enum: hook[prepare] logs | + - Auto hook[prepare] logs | + - "Off" hook[prepare] logs | + type: string hook[prepare] logs | + type: object hook[prepare] logs | + type: array hook[prepare] logs | + type: object hook[prepare] logs | + targetRef: hook[prepare] logs | + description: TargetRef points to the controller managing the set of hook[prepare] logs | + pods for the autoscaler to control - e.g. Deployment, StatefulSet. hook[prepare] logs | + VerticalPodAutoscaler can be targeted at controller implementing hook[prepare] logs | + scale subresource (the pod set is retrieved from the controller's hook[prepare] logs | + ScaleStatus) or some well known controllers (e.g. for DaemonSet hook[prepare] logs | + the pod set is read from the controller's spec). If VerticalPodAutoscaler hook[prepare] logs | + cannot use specified target it will report ConfigUnsupported condition. hook[prepare] logs | + Note that VerticalPodAutoscaler does not require full implementation hook[prepare] logs | + of scale subresource - it will not use it to modify the replica hook[prepare] logs | + count. The only thing retrieved is a label selector matching pods hook[prepare] logs | + grouped by the target resource. hook[prepare] logs | + properties: hook[prepare] logs | + apiVersion: hook[prepare] logs | + description: API version of the referent hook[prepare] logs | + type: string hook[prepare] logs | + kind: hook[prepare] logs | + description: 'Kind of the referent; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds"' hook[prepare] logs | + type: string hook[prepare] logs | + name: hook[prepare] logs | + description: 'Name of the referent; More info: http://kubernetes.io/docs/user-guide/identifiers#names' hook[prepare] logs | + type: string hook[prepare] logs | + required: hook[prepare] logs | + - kind hook[prepare] logs | + - name hook[prepare] logs | + type: object hook[prepare] logs | + updatePolicy: hook[prepare] logs | + description: Describes the rules on how changes are applied to the hook[prepare] logs | + pods. If not specified, all fields in the `PodUpdatePolicy` are hook[prepare] logs | + set to their default values. hook[prepare] logs | + properties: hook[prepare] logs | + minReplicas: hook[prepare] logs | + description: Minimal number of replicas which need to be alive hook[prepare] logs | + for Updater to attempt pod eviction (pending other checks like hook[prepare] logs | + PDB). Only positive values are allowed. Overrides global '--min-replicas' hook[prepare] logs | + flag. hook[prepare] logs | + format: int32 hook[prepare] logs | + type: integer hook[prepare] logs | + updateMode: hook[prepare] logs | + description: Controls when autoscaler applies changes to the pod hook[prepare] logs | + resources. The default is 'Auto'. hook[prepare] logs | + enum: hook[prepare] logs | + - "Off" hook[prepare] logs | + - Initial hook[prepare] logs | + - Recreate hook[prepare] logs | + - Auto hook[prepare] logs | + type: string hook[prepare] logs | + type: object hook[prepare] logs | + required: hook[prepare] logs | + - targetRef hook[prepare] logs | + type: object hook[prepare] logs | + status: hook[prepare] logs | + description: Current information about the autoscaler. hook[prepare] logs | + properties: hook[prepare] logs | + conditions: hook[prepare] logs | + description: Conditions is the set of conditions required for this hook[prepare] logs | + autoscaler to scale its target, and indicates whether or not those hook[prepare] logs | + conditions are met. hook[prepare] logs | + items: hook[prepare] logs | + description: VerticalPodAutoscalerCondition describes the state hook[prepare] logs | + of a VerticalPodAutoscaler at a certain point. hook[prepare] logs | + properties: hook[prepare] logs | + lastTransitionTime: hook[prepare] logs | + description: lastTransitionTime is the last time the condition hook[prepare] logs | + transitioned from one status to another hook[prepare] logs | + format: date-time hook[prepare] logs | + type: string hook[prepare] logs | + message: hook[prepare] logs | + description: message is a human-readable explanation containing hook[prepare] logs | + details about the transition hook[prepare] logs | + type: string hook[prepare] logs | + reason: hook[prepare] logs | + description: reason is the reason for the condition's last transition. hook[prepare] logs | + type: string hook[prepare] logs | + status: hook[prepare] logs | + description: status is the status of the condition (True, False, hook[prepare] logs | + Unknown) hook[prepare] logs | + type: string hook[prepare] logs | + type: hook[prepare] logs | + description: type describes the current condition hook[prepare] logs | + type: string hook[prepare] logs | + required: hook[prepare] logs | + - status hook[prepare] logs | + - type hook[prepare] logs | + type: object hook[prepare] logs | + type: array hook[prepare] logs | + recommendation: hook[prepare] logs | + description: The most recently computed amount of resources recommended hook[prepare] logs | + by the autoscaler for the controlled pods. hook[prepare] logs | + properties: hook[prepare] logs | + containerRecommendations: hook[prepare] logs | + description: Resources recommended by the autoscaler for each hook[prepare] logs | + container. hook[prepare] logs | + items: hook[prepare] logs | + description: RecommendedContainerResources is the recommendation hook[prepare] logs | + of resources computed by autoscaler for a specific container. hook[prepare] logs | + Respects the container resource policy if present in the spec. hook[prepare] logs | + In particular the recommendation is not produced for containers hook[prepare] logs | + with `ContainerScalingMode` set to 'Off'. hook[prepare] logs | + properties: hook[prepare] logs | + containerName: hook[prepare] logs | + description: Name of the container. hook[prepare] logs | + type: string hook[prepare] logs | + lowerBound: hook[prepare] logs | + additionalProperties: hook[prepare] logs | + anyOf: hook[prepare] logs | + - type: integer hook[prepare] logs | + - type: string hook[prepare] logs | + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ hook[prepare] logs | + x-kubernetes-int-or-string: true hook[prepare] logs | + description: Minimum recommended amount of resources. Observes hook[prepare] logs | + ContainerResourcePolicy. This amount is not guaranteed hook[prepare] logs | + to be sufficient for the application to operate in a stable hook[prepare] logs | + way, however running with less resources is likely to hook[prepare] logs | + have significant impact on performance/availability. hook[prepare] logs | + type: object hook[prepare] logs | + target: hook[prepare] logs | + additionalProperties: hook[prepare] logs | + anyOf: hook[prepare] logs | + - type: integer hook[prepare] logs | + - type: string hook[prepare] logs | + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ hook[prepare] logs | + x-kubernetes-int-or-string: true hook[prepare] logs | + description: Recommended amount of resources. Observes ContainerResourcePolicy. hook[prepare] logs | + type: object hook[prepare] logs | + uncappedTarget: hook[prepare] logs | + additionalProperties: hook[prepare] logs | + anyOf: hook[prepare] logs | + - type: integer hook[prepare] logs | + - type: string hook[prepare] logs | + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ hook[prepare] logs | + x-kubernetes-int-or-string: true hook[prepare] logs | + description: The most recent recommended resources target hook[prepare] logs | + computed by the autoscaler for the controlled pods, based hook[prepare] logs | + only on actual resource usage, not taking into account hook[prepare] logs | + the ContainerResourcePolicy. May differ from the Recommendation hook[prepare] logs | + if the actual resource usage causes the target to violate hook[prepare] logs | + the ContainerResourcePolicy (lower than MinAllowed or hook[prepare] logs | + higher that MaxAllowed). Used only as status indication, hook[prepare] logs | + will not affect actual resource assignment. hook[prepare] logs | + type: object hook[prepare] logs | + upperBound: hook[prepare] logs | + additionalProperties: hook[prepare] logs | + anyOf: hook[prepare] logs | + - type: integer hook[prepare] logs | + - type: string hook[prepare] logs | + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ hook[prepare] logs | + x-kubernetes-int-or-string: true hook[prepare] logs | + description: Maximum recommended amount of resources. Observes hook[prepare] logs | + ContainerResourcePolicy. Any resources allocated beyond hook[prepare] logs | + this value are likely wasted. This value may be larger hook[prepare] logs | + than the maximum amount of application is actually capable hook[prepare] logs | + of consuming. hook[prepare] logs | + type: object hook[prepare] logs | + required: hook[prepare] logs | + - target hook[prepare] logs | + type: object hook[prepare] logs | + type: array hook[prepare] logs | + type: object hook[prepare] logs | + type: object hook[prepare] logs | + required: hook[prepare] logs | + - spec hook[prepare] logs | + type: object hook[prepare] logs | + served: true hook[prepare] logs | + storage: true hook[prepare] logs | + subresources: {} hook[prepare] logs | + - name: v1beta2 hook[prepare] logs | + schema: hook[prepare] logs | + openAPIV3Schema: hook[prepare] logs | + description: VerticalPodAutoscaler is the configuration for a vertical pod hook[prepare] logs | + autoscaler, which automatically manages pod resources based on historical hook[prepare] logs | + and real time resource utilization. hook[prepare] logs | + properties: hook[prepare] logs | + apiVersion: hook[prepare] logs | + description: 'APIVersion defines the versioned schema of this representation hook[prepare] logs | + of an object. Servers should convert recognized schemas to the latest hook[prepare] logs | + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' hook[prepare] logs | + type: string hook[prepare] logs | + kind: hook[prepare] logs | + description: 'Kind is a string value representing the REST resource this hook[prepare] logs | + object represents. Servers may infer this from the endpoint the client hook[prepare] logs | + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' hook[prepare] logs | + type: string hook[prepare] logs | + metadata: hook[prepare] logs | + type: object hook[prepare] logs | + spec: hook[prepare] logs | + description: 'Specification of the behavior of the autoscaler. More info: hook[prepare] logs | + https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status.' hook[prepare] logs | + properties: hook[prepare] logs | + resourcePolicy: hook[prepare] logs | + description: Controls how the autoscaler computes recommended resources. hook[prepare] logs | + The resource policy may be used to set constraints on the recommendations hook[prepare] logs | + for individual containers. If not specified, the autoscaler computes hook[prepare] logs | + recommended resources for all containers in the pod, without additional hook[prepare] logs | + constraints. hook[prepare] logs | + properties: hook[prepare] logs | + containerPolicies: hook[prepare] logs | + description: Per-container resource policies. hook[prepare] logs | + items: hook[prepare] logs | + description: ContainerResourcePolicy controls how autoscaler hook[prepare] logs | + computes the recommended resources for a specific container. hook[prepare] logs | + properties: hook[prepare] logs | + containerName: hook[prepare] logs | + description: Name of the container or DefaultContainerResourcePolicy, hook[prepare] logs | + in which case the policy is used by the containers that hook[prepare] logs | + don't have their own policy specified. hook[prepare] logs | + type: string hook[prepare] logs | + maxAllowed: hook[prepare] logs | + additionalProperties: hook[prepare] logs | + anyOf: hook[prepare] logs | + - type: integer hook[prepare] logs | + - type: string hook[prepare] logs | + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ hook[prepare] logs | + x-kubernetes-int-or-string: true hook[prepare] logs | + description: Specifies the maximum amount of resources that hook[prepare] logs | + will be recommended for the container. The default is hook[prepare] logs | + no maximum. hook[prepare] logs | + type: object hook[prepare] logs | + minAllowed: hook[prepare] logs | + additionalProperties: hook[prepare] logs | + anyOf: hook[prepare] logs | + - type: integer hook[prepare] logs | + - type: string hook[prepare] logs | + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ hook[prepare] logs | + x-kubernetes-int-or-string: true hook[prepare] logs | + description: Specifies the minimal amount of resources that hook[prepare] logs | + will be recommended for the container. The default is hook[prepare] logs | + no minimum. hook[prepare] logs | + type: object hook[prepare] logs | + mode: hook[prepare] logs | + description: Whether autoscaler is enabled for the container. hook[prepare] logs | + The default is "Auto". hook[prepare] logs | + enum: hook[prepare] logs | + - Auto hook[prepare] logs | + - "Off" hook[prepare] logs | + type: string hook[prepare] logs | + type: object hook[prepare] logs | + type: array hook[prepare] logs | + type: object hook[prepare] logs | + targetRef: hook[prepare] logs | + description: TargetRef points to the controller managing the set of hook[prepare] logs | + pods for the autoscaler to control - e.g. Deployment, StatefulSet. hook[prepare] logs | + VerticalPodAutoscaler can be targeted at controller implementing hook[prepare] logs | + scale subresource (the pod set is retrieved from the controller's hook[prepare] logs | + ScaleStatus) or some well known controllers (e.g. for DaemonSet hook[prepare] logs | + the pod set is read from the controller's spec). If VerticalPodAutoscaler hook[prepare] logs | + cannot use specified target it will report ConfigUnsupported condition. hook[prepare] logs | + Note that VerticalPodAutoscaler does not require full implementation hook[prepare] logs | + of scale subresource - it will not use it to modify the replica hook[prepare] logs | + count. The only thing retrieved is a label selector matching pods hook[prepare] logs | + grouped by the target resource. hook[prepare] logs | + properties: hook[prepare] logs | + apiVersion: hook[prepare] logs | + description: API version of the referent hook[prepare] logs | + type: string hook[prepare] logs | + kind: hook[prepare] logs | + description: 'Kind of the referent; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds"' hook[prepare] logs | + type: string hook[prepare] logs | + name: hook[prepare] logs | + description: 'Name of the referent; More info: http://kubernetes.io/docs/user-guide/identifiers#names' hook[prepare] logs | + type: string hook[prepare] logs | + required: hook[prepare] logs | + - kind hook[prepare] logs | + - name hook[prepare] logs | + type: object hook[prepare] logs | + updatePolicy: hook[prepare] logs | + description: Describes the rules on how changes are applied to the hook[prepare] logs | + pods. If not specified, all fields in the `PodUpdatePolicy` are hook[prepare] logs | + set to their default values. hook[prepare] logs | + properties: hook[prepare] logs | + updateMode: hook[prepare] logs | + description: Controls when autoscaler applies changes to the pod hook[prepare] logs | + resources. The default is 'Auto'. hook[prepare] logs | + enum: hook[prepare] logs | + - "Off" hook[prepare] logs | + - Initial hook[prepare] logs | + - Recreate hook[prepare] logs | + - Auto hook[prepare] logs | + type: string hook[prepare] logs | + type: object hook[prepare] logs | + required: hook[prepare] logs | + - targetRef hook[prepare] logs | + type: object hook[prepare] logs | + status: hook[prepare] logs | + description: Current information about the autoscaler. hook[prepare] logs | + properties: hook[prepare] logs | + conditions: hook[prepare] logs | + description: Conditions is the set of conditions required for this hook[prepare] logs | + autoscaler to scale its target, and indicates whether or not those hook[prepare] logs | + conditions are met. hook[prepare] logs | + items: hook[prepare] logs | + description: VerticalPodAutoscalerCondition describes the state hook[prepare] logs | + of a VerticalPodAutoscaler at a certain point. hook[prepare] logs | + properties: hook[prepare] logs | + lastTransitionTime: hook[prepare] logs | + description: lastTransitionTime is the last time the condition hook[prepare] logs | + transitioned from one status to another hook[prepare] logs | + format: date-time hook[prepare] logs | + type: string hook[prepare] logs | + message: hook[prepare] logs | + description: message is a human-readable explanation containing hook[prepare] logs | + details about the transition hook[prepare] logs | + type: string hook[prepare] logs | + reason: hook[prepare] logs | + description: reason is the reason for the condition's last transition. hook[prepare] logs | + type: string hook[prepare] logs | + status: hook[prepare] logs | + description: status is the status of the condition (True, False, hook[prepare] logs | + Unknown) hook[prepare] logs | + type: string hook[prepare] logs | + type: hook[prepare] logs | + description: type describes the current condition hook[prepare] logs | + type: string hook[prepare] logs | + required: hook[prepare] logs | + - status hook[prepare] logs | + - type hook[prepare] logs | + type: object hook[prepare] logs | + type: array hook[prepare] logs | + recommendation: hook[prepare] logs | + description: The most recently computed amount of resources recommended hook[prepare] logs | + by the autoscaler for the controlled pods. hook[prepare] logs | + properties: hook[prepare] logs | + containerRecommendations: hook[prepare] logs | + description: Resources recommended by the autoscaler for each hook[prepare] logs | + container. hook[prepare] logs | + items: hook[prepare] logs | + description: RecommendedContainerResources is the recommendation hook[prepare] logs | + of resources computed by autoscaler for a specific container. hook[prepare] logs | + Respects the container resource policy if present in the spec. hook[prepare] logs | + In particular the recommendation is not produced for containers hook[prepare] logs | + with `ContainerScalingMode` set to 'Off'. hook[prepare] logs | + properties: hook[prepare] logs | + containerName: hook[prepare] logs | + description: Name of the container. hook[prepare] logs | + type: string hook[prepare] logs | + lowerBound: hook[prepare] logs | + additionalProperties: hook[prepare] logs | + anyOf: hook[prepare] logs | + - type: integer hook[prepare] logs | + - type: string hook[prepare] logs | + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ hook[prepare] logs | + x-kubernetes-int-or-string: true hook[prepare] logs | + description: Minimum recommended amount of resources. Observes hook[prepare] logs | + ContainerResourcePolicy. This amount is not guaranteed hook[prepare] logs | + to be sufficient for the application to operate in a stable hook[prepare] logs | + way, however running with less resources is likely to hook[prepare] logs | + have significant impact on performance/availability. hook[prepare] logs | + type: object hook[prepare] logs | + target: hook[prepare] logs | + additionalProperties: hook[prepare] logs | + anyOf: hook[prepare] logs | + - type: integer hook[prepare] logs | + - type: string hook[prepare] logs | + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ hook[prepare] logs | + x-kubernetes-int-or-string: true hook[prepare] logs | + description: Recommended amount of resources. Observes ContainerResourcePolicy. hook[prepare] logs | + type: object hook[prepare] logs | + uncappedTarget: hook[prepare] logs | + additionalProperties: hook[prepare] logs | + anyOf: hook[prepare] logs | + - type: integer hook[prepare] logs | + - type: string hook[prepare] logs | + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ hook[prepare] logs | + x-kubernetes-int-or-string: true hook[prepare] logs | + description: The most recent recommended resources target hook[prepare] logs | + computed by the autoscaler for the controlled pods, based hook[prepare] logs | + only on actual resource usage, not taking into account hook[prepare] logs | + the ContainerResourcePolicy. May differ from the Recommendation hook[prepare] logs | + if the actual resource usage causes the target to violate hook[prepare] logs | + the ContainerResourcePolicy (lower than MinAllowed or hook[prepare] logs | + higher that MaxAllowed). Used only as status indication, hook[prepare] logs | + will not affect actual resource assignment. hook[prepare] logs | + type: object hook[prepare] logs | + upperBound: hook[prepare] logs | + additionalProperties: hook[prepare] logs | + anyOf: hook[prepare] logs | + - type: integer hook[prepare] logs | + - type: string hook[prepare] logs | + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ hook[prepare] logs | + x-kubernetes-int-or-string: true hook[prepare] logs | + description: Maximum recommended amount of resources. Observes hook[prepare] logs | + ContainerResourcePolicy. Any resources allocated beyond hook[prepare] logs | + this value are likely wasted. This value may be larger hook[prepare] logs | + than the maximum amount of application is actually capable hook[prepare] logs | + of consuming. hook[prepare] logs | + type: object hook[prepare] logs | + required: hook[prepare] logs | + - target hook[prepare] logs | + type: object hook[prepare] logs | + type: array hook[prepare] logs | + type: object hook[prepare] logs | + type: object hook[prepare] logs | + required: hook[prepare] logs | + - spec hook[prepare] logs | + type: object hook[prepare] logs | + served: true hook[prepare] logs | + storage: false hook[prepare] logs | +status: hook[prepare] logs | + acceptedNames: hook[prepare] logs | + kind: "" hook[prepare] logs | + plural: "" hook[prepare] logs | + conditions: null hook[prepare] logs | + storedVersions: hook[prepare] logs | + - v1 hook[prepare] logs | Comparing release=vpa, chart=fairwinds-stable/vpa ******************** Release was not present in Helm. Diff will show entire contents as new. ******************** vpa-system, vpa-actor, ClusterRole (rbac.authorization.k8s.io) has been added: - + # Source: vpa/templates/clusterroles.yaml + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRole + metadata: + name: vpa-actor + rules: + - apiGroups: + - "" + resources: + - pods + - nodes + - limitranges + verbs: + - get + - list + - watch + - apiGroups: + - "" + resources: + - events + verbs: + - get + - list + - watch + - create + - apiGroups: + - "poc.autoscaling.k8s.io" + resources: + - verticalpodautoscalers + verbs: + - get + - list + - watch + - patch + - apiGroups: + - "autoscaling.k8s.io" + resources: + - verticalpodautoscalers + verbs: + - get + - list + - watch + - patch vpa-system, vpa-actor, ClusterRoleBinding (rbac.authorization.k8s.io) has been added: - + # Source: vpa/templates/clusterrolebindings.yaml + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRoleBinding + metadata: + name: vpa-actor + roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: vpa-actor + subjects: + - kind: ServiceAccount + name: vpa-recommender + namespace: vpa-system vpa-system, vpa-checkpoint-actor, ClusterRole (rbac.authorization.k8s.io) has been added: - + # Source: vpa/templates/clusterroles.yaml + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRole + metadata: + name: vpa-checkpoint-actor + rules: + - apiGroups: + - "poc.autoscaling.k8s.io" + resources: + - verticalpodautoscalercheckpoints + verbs: + - get + - list + - watch + - create + - patch + - delete + - apiGroups: + - "autoscaling.k8s.io" + resources: + - verticalpodautoscalercheckpoints + verbs: + - get + - list + - watch + - create + - patch + - delete + - apiGroups: + - "" + resources: + - namespaces + verbs: + - get + - list vpa-system, vpa-checkpoint-actor, ClusterRoleBinding (rbac.authorization.k8s.io) has been added: - + # Source: vpa/templates/clusterrolebindings.yaml + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRoleBinding + metadata: + name: vpa-checkpoint-actor + roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: vpa-checkpoint-actor + subjects: + - kind: ServiceAccount + name: vpa-recommender + namespace: vpa-system vpa-system, vpa-evictioner, ClusterRole (rbac.authorization.k8s.io) has been added: - + # Source: vpa/templates/clusterroles.yaml + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRole + metadata: + name: vpa-evictioner + rules: + - apiGroups: + - "apps" + - "extensions" + resources: + - replicasets + verbs: + - get + - apiGroups: + - "" + resources: + - pods/eviction + verbs: + - create vpa-system, vpa-metrics-reader, ClusterRole (rbac.authorization.k8s.io) has been added: - + # Source: vpa/templates/clusterroles.yaml + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRole + metadata: + name: vpa-metrics-reader + rules: + - apiGroups: + - "metrics.k8s.io" + resources: + - pods + verbs: + - get + - list vpa-system, vpa-metrics-reader, ClusterRoleBinding (rbac.authorization.k8s.io) has been added: - + # Source: vpa/templates/clusterrolebindings.yaml + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRoleBinding + metadata: + name: vpa-metrics-reader + roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: vpa-metrics-reader + subjects: + - kind: ServiceAccount + name: vpa-recommender + namespace: vpa-system vpa-system, vpa-recommender, Deployment (apps) has been added: - + # Source: vpa/templates/recommender-deployment.yaml + apiVersion: apps/v1 + kind: Deployment + metadata: + name: vpa-recommender + labels: + app.kubernetes.io/component: recommender + helm.sh/chart: vpa-1.6.0 + app.kubernetes.io/name: vpa + app.kubernetes.io/instance: vpa + app.kubernetes.io/version: "0.11.0" + app.kubernetes.io/managed-by: Helm + spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/component: recommender + app.kubernetes.io/name: vpa + app.kubernetes.io/instance: vpa + template: + metadata: + labels: + app.kubernetes.io/component: recommender + app.kubernetes.io/name: vpa + app.kubernetes.io/instance: vpa + spec: + serviceAccountName: vpa-recommender + securityContext: + runAsNonRoot: true + runAsUser: 65534 + containers: + - name: vpa + securityContext: + {} + image: "k8s.gcr.io/autoscaling/vpa-recommender:0.11.0" + imagePullPolicy: Always + args: + - --pod-recommendation-min-cpu-millicores=15 + - --pod-recommendation-min-memory-mb=100 + - --v=4 + livenessProbe: + failureThreshold: 6 + httpGet: + path: /health-check + port: metrics + scheme: HTTP + periodSeconds: 5 + successThreshold: 1 + timeoutSeconds: 3 + readinessProbe: + failureThreshold: 120 + httpGet: + path: /health-check + port: metrics + scheme: HTTP + periodSeconds: 5 + successThreshold: 1 + timeoutSeconds: 3 + ports: + - name: metrics + containerPort: 8942 + protocol: TCP + resources: + limits: + cpu: 200m + memory: 1000Mi + requests: + cpu: 50m + memory: 500Mi vpa-system, vpa-recommender, ServiceAccount (v1) has been added: - + # Source: vpa/templates/recommender-service-account.yaml + apiVersion: v1 + kind: ServiceAccount + automountServiceAccountToken: true + metadata: + name: vpa-recommender + labels: + helm.sh/chart: vpa-1.6.0 + app.kubernetes.io/name: vpa + app.kubernetes.io/instance: vpa + app.kubernetes.io/version: "0.11.0" + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: recommender vpa-system, vpa-status-reader, ClusterRole (rbac.authorization.k8s.io) has been added: - + # Source: vpa/templates/clusterroles.yaml + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRole + metadata: + name: vpa-status-reader + rules: + - apiGroups: + - "coordination.k8s.io" + resources: + - leases + verbs: + - get + - list + - watch vpa-system, vpa-target-reader, ClusterRole (rbac.authorization.k8s.io) has been added: - + # Source: vpa/templates/clusterroles.yaml + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRole + metadata: + name: vpa-target-reader + rules: + - apiGroups: + - '*' + resources: + - '*/scale' + verbs: + - get + - watch + - apiGroups: + - "" + resources: + - replicationcontrollers + verbs: + - get + - list + - watch + - apiGroups: + - apps + resources: + - daemonsets + - deployments + - replicasets + - statefulsets + verbs: + - get + - list + - watch + - apiGroups: + - batch + resources: + - jobs + - cronjobs + verbs: + - get + - list + - watch vpa-system, vpa-target-reader-binding, ClusterRoleBinding (rbac.authorization.k8s.io) has been added: - + # Source: vpa/templates/clusterrolebindings.yaml + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRoleBinding + metadata: + name: vpa-target-reader-binding + roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: vpa-target-reader + subjects: + - kind: ServiceAccount + name: vpa-recommender + namespace: vpa-system vpa-system, vpa-updater, ServiceAccount (v1) has been added: - + # Source: vpa/templates/updater-service-account.yaml + apiVersion: v1 + kind: ServiceAccount + automountServiceAccountToken: true + metadata: + name: vpa-updater + labels: + helm.sh/chart: vpa-1.6.0 + app.kubernetes.io/name: vpa + app.kubernetes.io/instance: vpa + app.kubernetes.io/version: "0.11.0" + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: updater ``` {{< /details >}} Yeah, it's huge, but you can see everything that's going to happen. So I'd say it's good. ## Conclusion It's a short article, because I think the whole setup is super easy, CI is easy too. You still have a full `GitOps` (or almost full) but you also have control. I love this setup and would like to use it for my infrastructure. Why do I think it's better that `ArgoCD`? With `ArgoCD` I either have a lot of `yaml` to install things, or I have complicated setups with `ApplicationSets` that are most probably very special and won't be reused in other companies. I need to care about how `ArgoCD` will handle a lot of applications that are added there only for diffing. I need additional applications installed im my clusters not only as a part of infrastructure itself, but also as a service that I'm providing other teams with. Because I want to manage applications that are being developed by other teams with `Argo`, so I'm mixing a lot of different kinds of applications here. Helmfile lets me separate infra from applications. `ArgoCD` can be only provided as a service, and other teams can use, because it's making k8s easier for those who don't need to understand it so deeply. Also, helmfile lets me use helm-secrets to encrypt values. I can do it with Argo too, but then I need to either have a custom `ArgoCD` image, or support a CMP plugin, that will handle SOPS. You can find an example of PR here: > When helmfile is not GitOps? > To uninstall a helm release, you need to add `isntalled: false` to it. If you just remove a release from helmfile.yaml, it isn't going to be removed. So in such cases it's not GitOps. You can write a hook, that is comparing a previous state of your helmfile to the current one and doing a cleanup, then it's again fully GitOps. But I prefer removing things manually, so to me, it's not a problem. Removing stuff is something that I think should be mostly done by a human being, if it's not a part of your daily work. {{< comments >}}