Steven's Blog
0%

Kubernetes Operator with RedHat Operator SDK

Posted on Edited on

This tutorial is to develop Kubernetes Operators with RedHat Operator SDK.

What is Operator?

RedHat Operator Framework

Please refer to GitHub - Operator Framework

Operator SDK

  1. Create a new project

    1
    2
    3
    $ export GO111MODULE=on
    $ operator-sdk new sample-operator --repo=github.com/stevenchiu30801/sample-operator
    $ cd sample-operator

  2. Add a new Custom Resource Definition

    1
    $ operator-sdk add api --api-version=sample.io/v1alpha1 --kind=SampleCr

  3. Define the spec and status

    1
    2
    3
    4
    5
    // pkg/apis/sample/v1alpha1/samplecr_types.go
    type SampleCrSpec struct {
        TestList []int `json:"testList"`
        Count int `json:"count"`
    }
    1
    2
    3
    4
    5
    # update the generated code for that resource type
    $ operator-sdk generate k8s
    # automatically generate the CRDs
    $ operator-sdk generate crds
    # changes would be applied in deploy/crds/sample.io_samplecr_crd.yaml

  4. Add a new Controller

    1
    $ operator-sdk add controller --api-version=sample.io/v1alpha1 --kind=SampleCr

  5. Design the controller pkg/controller/samplecr/samplecr_controller.go

  6. Register CRD

    1
    $ kubectl create -f deploy/crds/sample.io_samplecr_crd.yaml

  7. Run the operator

    Two ways

    1. Run as a Deployment inside the cluster

      1
      2
      3
      4
      5
      6
      # build image and push it to your registry first!
      $ sed -i 's|REPLACE_IMAGE|<docker/repo:tag>|g'deploy/operator.yaml
      $ kubectl create -f deploy/service_account.yaml
      $ kubectl create -f deploy/role.yaml
      $ kubectl create -f deploy/role_binding.yaml
      $ kubectl create -f deploy/operator.yaml

    2. Run locally outside the cluster

      1
      2
      # during development cycle to deploy and test faster
      $ operator-sdk run --local --namespace=default

  8. Create CR

    1
    2
    # modify CR spec first
    $ kubectl create -f deploy/crds/sample.io_v1alpha1_samplecr_cr.yaml

Some design logic of controller

  • Design control logic in the view of maintaining states of reconciled objects (custom resources)
    • Don’t think in the way of handling requests on orchestrator or manager
    • Since reconciled object would be requeue due to changes of status
  • Try to not maintain soft state in operator
    • Store data in database or volume
    • Use Status to keep object states
  • Use finalizer to handle cleanup of external resources (resources created from Helm) or customize cleanup logic
    • For resources allocated in Go objects, one could use Controller Reference to enable automated cleanup

Reference

Go API Package

In my design case, I use Go Operator but have to deploy resources with Helm (Too many resources to deploy with Go objects)

Here are some API references to do so and also refer to the code

Operator

Finalizer

Others

Update

Operator SDK has its own website and documentation.