Edgecase 2023: thursday early morning sessions¶

Fantastic pipelines and where to run them - Fabian Met¶

Pipelines in Jenkins. Later pipelines in a local gitlab installation. Then pipelines in github. Then the security officer found out about pipelines running on github, probably, in the USA. So they had to move back. Rewriting, rewriting, rewriting.

He thought “keep it simple: let’s do it ourselves!” What did they really need?

• Git as input

• Pipelines

• Some output

For the pipelines, they chose tekton. Argocd was also possible, btw.

Kubernetes is very good at orchestration. You can get enterprise solutions, but we’re all moving to as-basic-as-possible to prevent lock-in. But with git we go in the opposite direction: nobody uses plain git, everybody uses github or gitlab and clicks all the enterprise functions. Lock-in! Why do we do that?!?

When you have a kubernetes-based pipeline that you made yourself, you can run it locally. Way nicer than doing commit after commit to test out your pipeline on github/gitlab.

At this point I figured out he was mostly talking about deploy pipelines. I initially thought it was also about software tests, as that’s what I was running in jenkins and now on github… Once he said to not store the pipeline config in the same repository as the code I caught on. Sorry, that’s what you get when a developer goes to a kubernetes conference.

Managing hundreds of clusters at the edge - Tim Jones¶

powerflex is a USA company with solar power installations, electricity storage solutions, EV chargers, etc. Equipment that’s often in remote locations with horrible network connectivity.

Their initial setup was with Rancher k3os, a minimal linux for easily installing K3S. But they ran into problems like lack of custom driver support and no remote reboots. In the end community and company support for k3os was also lacking. Development has stopped in the meantime.

So… they moved to Talos linux (from sidero) as the os/kubernetes combination.

• Small and fast.

• Hardened for remote locations. Very secure. Immutable. You cannot even write to disk. The OS effectively runs from a RAM disk. No shell/ssh access. No regular GNU utils. No nothing. Entire classes of attacks are impossible.

• Simple to manage. Configuration is done through one single yaml file.

• Same image everywhere. It is independent of the hardware, so you can really have multi-cloud that’s similar. Edge locations combined with the regular cloud? All the same.

Installation is simple for the most part. To help with the last 20%, they made Sidero omni:

• Single management plane.

• Fewer deep IT skills required. A fresh node can register itself and Omni can take care of the rest.

• Enterprise grade authentication. Any regular oidc provider will do, like github, google, etc. With omni, all interaction is protected this way.

• Highly available out of the box. Omni itself is of course HA. Edge locations should be treated as “cattle, not pets”, that’s what they really subscribe too. Just fire up a new machine or ship a new box.

• Firewall friendly. You do need a connection to omni, but that’s it. Local image storage is fine, for instance. They have installations in hospitals without any egress (apart from the omni connection).

They’re working on version 2:

• More flexible depoyment options.

• Customizable builds.

• More hardware support.

• Reduced hardware load.

Back to the “powerflex” example. They migrated 450+ clusters, moving on to a 1000. Field technicians can provision boxes in a self-service manner. They’re working on having a warehouse with pre-imaged boxes, ready for shipping out and installing at a moment’s notice.

He showed a demo of omni/talos. In Omni, there’s the option to download a slightly customized version of the regular talos image: the only difference is that the container automatically safely “phones home” to your omni account and register itself.

In response to a question about upgrading/deprecation: they support the current kubernetes release and the three previous versions. That way, most people can stay up-to-date with talos without immediately needing to upgrade kubernetes. Talos itself aims to be as minimal as possible: you should not have to care about the OS. Kubernetes is where your worries should be: the OS should be as invisible and worry-free as possible.

A clarification by myself: I originally understood talos to be like k3os, an easy small linux OS to run k3s. But talos is more an integration of the two. Talos is the minimal OS plus the main kubernetes components.