Init commit
This commit is contained in:
commit
840eeeeef8
GPG Key ID:
0AA46A90E25592AD
|
|
@ -0,0 +1,219 @@
|
|||
# Helmfile vs Flux
|
||||
|
||||
> First, I need to say that I'm not really good at Kustomize, and maybe everything can be done better.
|
||||
|
||||
Let's say we need to install Bitnami Postgres to 2 clusters using helm charts.
|
||||
|
||||
- Passwords should be stored in git encrypted with SOPS
|
||||
- One database needs to be 10Gb in size, and another just 1Gb
|
||||
|
||||
With flux we're creating two manifests:
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: source.toolkit.fluxcd.io/v1beta2
|
||||
kind: HelmRepository
|
||||
metadata:
|
||||
name: bitnami
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 1m0s
|
||||
type: default
|
||||
url: https://charts.bitnami.com/bitnami
|
||||
---
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2beta1
|
||||
kind: HelmRelease
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
name: postgresql
|
||||
namespace: flux-system
|
||||
spec:
|
||||
chart:
|
||||
spec:
|
||||
chart: postgresql
|
||||
sourceRef:
|
||||
kind: HelmRepository
|
||||
name: bitnami
|
||||
namespace: flux-system
|
||||
version: 13.3.1
|
||||
install:
|
||||
crds: Create
|
||||
createNamespace: true
|
||||
interval: 1m0s
|
||||
releaseName: postgresql
|
||||
targetNamespace: database
|
||||
```
|
||||
|
||||
With helmfile it's going to be this:
|
||||
```
|
||||
repositories:
|
||||
- name: bitnami
|
||||
url: https://charts.bitnami.com/bitnami
|
||||
|
||||
releases:
|
||||
- name: postgresql
|
||||
chart: bitnami/postgresql
|
||||
version: 13.3.1
|
||||
namespace: database
|
||||
```
|
||||
|
||||
So it's **30** lines of code against **8**.
|
||||
|
||||
Then we need to add values. With flux we would add something like that to the manifest.
|
||||
```yaml
|
||||
valuesFrom:
|
||||
- kind: ConfigMap
|
||||
name: database-postgresql-values.postgresql.yaml
|
||||
valuesKey: database-postgresql-values.postgresql.yaml
|
||||
- kind: ConfigMap
|
||||
name: database-postgresql-values.spec.postgresql.yaml
|
||||
valuesKey: database-postgresql-values.spec.postgresql.yaml
|
||||
```
|
||||
|
||||
> Spec means special in that case
|
||||
|
||||
And we need to create 2 configmaps (or it can be just one, cause I wanted to reuse common values, but I think kustomize won't let me use the folder that is not in that kustomize context, so I'm not sure how)
|
||||
|
||||
And it's either **4** or **7** lines of code in addition
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
data:
|
||||
database-postgresql-values.spec.postgresql.yaml: |
|
||||
persistence:
|
||||
size: 1Gi
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
name: database-postgresql-values.spec.postgresql.yaml
|
||||
namespace: flux-system
|
||||
---
|
||||
apiVersion: v1
|
||||
data:
|
||||
database-postgresql-values.postgresql.yaml: |
|
||||
architecture: standalone
|
||||
|
||||
auth:
|
||||
database: postgres
|
||||
|
||||
metrics:
|
||||
enabled: false
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
name: database-postgresql-values.postgresql.yaml
|
||||
namespace: flux-system
|
||||
```
|
||||
|
||||
In case it's just one configmap, it would be something like:
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
data:
|
||||
database-postgresql-values.postgresql.yaml: |
|
||||
persistence:
|
||||
size: 1Gi
|
||||
architecture: standalone
|
||||
auth:
|
||||
database: postgres
|
||||
metrics:
|
||||
enabled: false
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
name: database-postgresql-values.postgresql.yaml
|
||||
namespace: flux-system
|
||||
```
|
||||
|
||||
So it's either about **25** or **15** lines of code. Also, it's not just you go to the chart's values and creates a values file, you need to create a whole configmap, so you either need to remember the whole CM spec,cmds to create it, or you need to find a reference somewhere to copy-paste.
|
||||
|
||||
Also, since it's just static manifests, names of configmaps and keys are not set once and reused. So it's error-prone configuration, especially when you can't just debug it, and you need to deploy it to spot a problem. Once you update a configmap name/key, you need not to forget to update the Release manifest.
|
||||
|
||||
With helmfile it's going to be something like that.
|
||||
|
||||
Add it to the helmfile:
|
||||
```yaml
|
||||
values:
|
||||
- ./values/common/values.postgresql.yaml
|
||||
- ./values/cluster-1/values.postgresql.yaml
|
||||
```
|
||||
|
||||
And then values
|
||||
```yaml
|
||||
---
|
||||
architecture: standalone
|
||||
|
||||
auth:
|
||||
database: postgres
|
||||
|
||||
metrics:
|
||||
enabled: false
|
||||
---
|
||||
persistence:
|
||||
size: 1Gi
|
||||
```
|
||||
|
||||
So, additional ~**10** lines of code.
|
||||
|
||||
And the secret thingy
|
||||
|
||||
> I'm pretty syre it can be done better, since I'm not good at Kustomize, so it might be a bad point.
|
||||
|
||||
How am I doing that?
|
||||
|
||||
I'm creating a Kubernetes secret where data is encrypted with Sops. To be honest I encrypt the whole secret, but it's just because I've screwed up the sops config part. But the idea is that there is a k8s secret with encrypted data. I won't count lines here, because probably it should be less that I have, approximately it's about **10** lines of code, because it's the whole manifest. And it has the same problem ConfigMap has, because it's a static manifest.
|
||||
but with my approach it's also required to have a secret-generator, that is decrypting secrets with the ksops plugin
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: viaduct.ai/v1
|
||||
kind: ksops
|
||||
metadata:
|
||||
name: shoebill-secret-gen
|
||||
files:
|
||||
- src/secrets/database-postgresql-secrets.postgresql.yaml
|
||||
```
|
||||
|
||||
And it's **6** additional lines + a system dependency (ksops). But I've been told that I don't have to do it this way, so it might be not required at all.
|
||||
|
||||
Helmfile:
|
||||
|
||||
```yaml
|
||||
secrets:
|
||||
- ./values/{{ .Environment.Name}}/secrets.postgresql.yaml
|
||||
```
|
||||
|
||||
So **2** lines are added to the helmfile, and a secret, that in unencrypted state is **4** lines of code
|
||||
|
||||
And already at this step it about **65** vs **24**. for one chart in one cluster. With the helmfile you don't need to add k8s objects boilerplate around values, you just use them as they are defined in a chart, you don't need to use k8s object boilerplate code around release and repo. And you don't need to check names of secret/configmaps/keys, so they correspond in two static objects.
|
||||
|
||||
Now, let's add a second cluster:
|
||||
|
||||
> Maybe if you use Kustomize with variables, you can avoid it, so I'm not sure
|
||||
|
||||
But probably, it would be just 1-to-1 copypaste of the first cluster with one line changed, so I'd expect to have **130** lines of code. And with helmfile I'd do it this way
|
||||
|
||||
```yaml
|
||||
---
|
||||
environments:
|
||||
cluster-1:
|
||||
cluster-2:
|
||||
---
|
||||
repositories:
|
||||
- name: bitnami
|
||||
url: https://charts.bitnami.com/bitnami
|
||||
|
||||
releases:
|
||||
- name: postgresql
|
||||
chart: bitnami/postgresql
|
||||
version: 13.3.1
|
||||
namespace: database
|
||||
values:
|
||||
- ./values/common/values.postgresql.yaml
|
||||
- ./values/{{ .Environment.Name }}/values.postgresql.yaml
|
||||
secrets:
|
||||
- ./values/{{ .Environment.Name}}/secrets.postgresql.yaml
|
||||
```
|
||||
|
||||
So it's just about **4** lines of code added to the helmfile, and 3 added as new values, so **31** in total.
|
||||
|
||||
Then if you need to change something that is common for there tho clusters (helmchart version, or common values), with helmfile you'll need to update it once, and with flux you'll have to do it as many times, as you have clusters, since it's going to be copy-paste, it's boring and hence error-prone task.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Subproject commit 7ddc6f1d90f387a2889e9e473316e05247857fae
|
||||
|
|
@ -0,0 +1 @@
|
|||
Subproject commit 1543773dc3dee56a05d7ca8bcb990bb7759f6ff2
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
creation_rules:
|
||||
- path_regex: values/cluster-1/secrets.*.yaml
|
||||
key_groups:
|
||||
- age:
|
||||
- age1nrsmsgq0xynqke4sh8qmuxnlqqg7z5ll5stkpe8qy6tqy40cearqhxjy70
|
||||
- path_regex: values/cluster-2/secrets.*.yaml
|
||||
key_groups:
|
||||
- age:
|
||||
- age1qf6709hu4wlg6s5wyy3w0en265k9qjuxesz2tqq8e0xdrfwjrc2qngtfew
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
---
|
||||
environments:
|
||||
cluster-1:
|
||||
cluster-2:
|
||||
---
|
||||
repositories:
|
||||
- name: bitnami
|
||||
url: https://charts.bitnami.com/bitnami
|
||||
|
||||
releases:
|
||||
- name: postgresql
|
||||
chart: bitnami/postgresql
|
||||
version: 13.3.1
|
||||
namespace: database
|
||||
values:
|
||||
- ./values/common/values.postgresql.yaml
|
||||
- ./values/{{ .Environment.Name }}/values.postgresql.yaml
|
||||
secrets:
|
||||
- ./values/{{ .Environment.Name}}/secrets.postgresql.yaml
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
global:
|
||||
postgresql:
|
||||
auth:
|
||||
postgresPassword: ENC[AES256_GCM,data:12gVr3DqV0A=,iv:f2mW2kRjZypW+BPGXKvruV2KTYJ/Dc0OhokwXg3lcHk=,tag:q3j63qCBe1HGtouQL0sE6A==,type:int]
|
||||
sops:
|
||||
kms: []
|
||||
gcp_kms: []
|
||||
azure_kv: []
|
||||
hc_vault: []
|
||||
age:
|
||||
- recipient: age1nrsmsgq0xynqke4sh8qmuxnlqqg7z5ll5stkpe8qy6tqy40cearqhxjy70
|
||||
enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB4WnJvenRtbGI2UEgzN2ZT
|
||||
WlY0RTlDZFNCNEszMG9QZis3cGZRRGo5VlVRCnlJc1pFcmQ4d0YyVVNiLzdRamk1
|
||||
ZGZhM3NzbUJJd29acXo4VVpaSWtwUlkKLS0tIE03ckJnMjh6TmNYOGo0Q1dhU3Jo
|
||||
TG5hYmp4UWh0N2dHQ0F1Sm1DTFB1Z0UKnwunl+cciUDpBedC8ujvx5z8gSOGzDxh
|
||||
NFN9lmqyBroFEjLBZzApuu5HV/JJhKmDeTZjefw01ThXBPKEDSDWsA==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
lastmodified: "2024-02-06T20:43:27Z"
|
||||
mac: ENC[AES256_GCM,data:E2OOuErjEHZcFAAw8NED/o2v63p18sz4aiSzImK48poe/FURtqjR7+vhVUBmoVWR/wRZC0GVQcovllbKW+XOUe+H6HbVKFgRqr/WPSK494cWTMv+FtcwL2R/jQ+Ek11cXpXJVWk+osYRGfZW3Fb0DIsU94gowWpL5fMUwbmuGqc=,iv:Z/kMbm+H4sKDsgmy9fjYaI5jqIWksMOpWQO3A8M0oso=,tag:CifqLrjmAGpAYnzX/QBMCw==,type:str]
|
||||
pgp: []
|
||||
unencrypted_suffix: _unencrypted
|
||||
version: 3.8.1
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
persistence:
|
||||
size: 1Gi
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
global:
|
||||
postgresql:
|
||||
auth:
|
||||
postgresPassword: ENC[AES256_GCM,data:TcvZJ7Kq5YA=,iv:eTCJSuwQrQDtaQifpm7euRq5iLYS7u9lkukG+LGQVug=,tag:zQmIQlypS+ToHla1mE75vA==,type:int]
|
||||
sops:
|
||||
kms: []
|
||||
gcp_kms: []
|
||||
azure_kv: []
|
||||
hc_vault: []
|
||||
age:
|
||||
- recipient: age1qf6709hu4wlg6s5wyy3w0en265k9qjuxesz2tqq8e0xdrfwjrc2qngtfew
|
||||
enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB4ampFRDI5bFFzMUE1TXNv
|
||||
SE5wZnBsSGorM3FkaHdoNUs1SUpWU3VuVUNNCldPZ2lTUUVocFhnOHBxUEUydnFh
|
||||
Z2c3MWJNWWVHQ3dXRzFjNVJSY242SE0KLS0tIHl6alEzOXk3ZHhxTnpSTkpqWDBt
|
||||
VUtpWDhlTklYekRZdzRCQjg4aFdsajAKmscKIHebzjhyRKnhIcWU82FOpJ4m85A4
|
||||
6i+H4aXkCVI6Jpbq8RQzkPTi/VtN35dTr2zp1+b0Z8C3VLwXgPgp3g==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
lastmodified: "2024-02-06T20:43:50Z"
|
||||
mac: ENC[AES256_GCM,data:u32rdU9zqJMMsvbGxiWQ1Vu6kLDdVqbS4xRolBcgA9ux14wtUQSPUnahH+2ynlOZUlGHYK4zModyuRcVQ2dVS1tpFd87Aw5HRQQufFUL3xyCgj/39awvxfi8MCYkvtUi63TubplEr62RL61w0Lfw/DcpL/2hk2gwHCLzk3NMUuY=,iv:VROIWeYK3reZwkU9B3oLoNEZuGbTr0i+iPaVEGJ/lQU=,tag:r/PzoOEPrIgpiXbmIortRg==,type:str]
|
||||
pgp: []
|
||||
unencrypted_suffix: _unencrypted
|
||||
version: 3.8.1
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
persistence:
|
||||
size: 10Gi
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
architecture: standalone
|
||||
|
||||
auth:
|
||||
database: postgres
|
||||
|
||||
metrics:
|
||||
enabled: false
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
# Common key
|
||||
# created: 2024-02-06T21:21:42+01:00
|
||||
# public key: age157q8nk565pe7qavht280e2lvn9ld3mmkxuus4dhlm5asvtk2c5pqseadqm
|
||||
AGE-SECRET-KEY-1J5K6Q9E9DHMYLK2VF96VS0K3G6XSHXLFFRKA97TYPPWSEPTE025SX726HK
|
||||
|
||||
# cluster-1
|
||||
# created: 2024-02-06T21:17:52+01:00
|
||||
# public key: age1nrsmsgq0xynqke4sh8qmuxnlqqg7z5ll5stkpe8qy6tqy40cearqhxjy70
|
||||
AGE-SECRET-KEY-1VEHK4PAXLFRTEA3426N5Z90VNVRNK99K0F2MQDW6TLD4442FU0KSU2U9JP
|
||||
|
||||
# cluster-2
|
||||
# created: 2024-02-06T21:20:37+01:00
|
||||
# public key: age1qf6709hu4wlg6s5wyy3w0en265k9qjuxesz2tqq8e0xdrfwjrc2qngtfew
|
||||
AGE-SECRET-KEY-12Q2K824X6LM04FQW4W8CUC4PA0AUS9W0DZJG9S09U48R3YNRXATQZFKVQS
|
||||
|
||||
Loading…
Reference in New Issue