Advanced Cluster Management Updates (Part 1 of 2)

It's been almost a year and half since I did the series on Advanced Cluster Management (ACM).  This consisted of four parts

Overview of Advanced Cluster Management for Kubernetes (Part 1)

This article is based on version 2.3.2 of ACM. ACM provides the mechanism to create and manage Openshift clusters at scale. In some cases, even non-Openshift clusters can be managed. I am going to provide a walkthrough on some of the menus in the Multicluster hub object in

My OpenShift BlogKeith Calligan

Applications in Advanced Cluster Management for Kubernetes (Part 2)

Welcome to part 2 of the ACM blog series. The next section of ACM that will be covered are applications. When going to the “Applications” link on the left-side of the ACM screen, you will be taken to the following screen. In the overview tab, a list of current applications

My OpenShift BlogKeith Calligan

Governance/Policies in Advanced Cluster Management for Kubernetes (Part 3)

In this part of my series on ACM, we will cover Governance/Policies. Another name for this is the Governance, Risk, and Compliance framework. When we think of governance, security comes to mind. A big part of ACM policies does have to do with security but it is important to

My OpenShift BlogKeith Calligan

Pushing an Authentication Policy in Advanced Cluster Management for Kubernetes (Part 4)

In part 4 of this series, I am going to show a practical example of pushing a policy in ACM. Many customers ask for a way to control the authentication provider(s) that are supported by OCP. In this example, an htpasswd oauth provider will be created as well as

My OpenShift BlogKeith Calligan

These articles were based on version based on version 2.3.2 of ACM.  We are now at ACM 2.6.3.  There have been many enhancements which include updates to the GUI/layout, the integration of the multi-cloud console into the main OCP web console (separate route to be deprecated soon), the addition of other cloud providers, more support for Gitops approaches to creating clusters/applications/policies), and (lastly) the addition of the CIM (Central Infrastructure Management/Assisted-Service tooling which allows bare-metal hosts to be discovered and provisioned into clusters.

This article will show two things:

1. We already know that applications can be tied to a Github repo based on Channels and Subscriptions, but it hasn't been clear on how to do the same thing with Governance policies.  That is to create a GitHub repo (channel) which contains policy (YAML) definitions that can be pulled into ACM.
2. In part2 of this series, I wanted to show you how to configure the Central Infrastructure Management/Assisted-Service to handle the discovery and provisioning of bare-metal resources that can eventually be consumed to be part of an OCP cluster.  This blog-post outlines how to setup the CIM/Assisted-Service including the bare-metal/cluster YAML definitions.  ACM has a GUI that generates these various YAML definitions (infrastructureenvironment, klusterlet, agentclusterinstall, etc) for you. This part will be covered here with a re-hash of some information based on that previous article.

Referencing a GitHub Repo for Governance Policies

One point that was made previously in regards to policies in ACM is that any object definition in Openshift can become a policy to be pushed to other clusters.

In one of the previous articles (part 4 of ACM series), we worked on creating a policy that updated the Oauth configuration to add an htpasswd identity provider.  This htpasswd-policy used a secret that created a few users with one of the users given the cluster-admin role.

A similar example will be provided here.  I will show the 3 objects that were created to make up this policy (separately).  These objects will be put into a policy definition using the PolicyGenerator.  Lastly, the policy definition will be put into a GitHub repo which serves as a single-source-of-truth and follows GitOps (Infrastructure as Code) principles.

Just to clarify, these YAML defitions create two users (user1 and user2).  Both have passwords of "password".  Finally, the user1 is given cluster-admin privileges while user2 remains a regular user.

This code is also provided at the following Github link

acm-git-demos/htpasswd-policy-generator at main · kcalliga/acm-git-demos

ACM Git Demos. Contribute to kcalliga/acm-git-demos development by creating an account on GitHub.

GitHubkcalliga

OAUTH Configuration

apiVersion: config.openshift.io/v1
kind: OAuth
metadata:
  annotations:
    include.release.openshift.io/ibm-cloud-managed: "true"
    include.release.openshift.io/self-managed-high-availability: "true"
    include.release.openshift.io/single-node-developer: "true"
    release.openshift.io/create-only: "true"
  name: cluster
spec:
  identityProviders:
  - htpasswd:
      fileData:
        name: htpasswd-gmszp
    mappingMethod: claim
    name: htpasswd
    type: HTPasswd

HTPASSWD Secret

apiVersion: v1
data:
  htpasswd: dXNlcjI6JDJ5JDA1JDRkQ2wvMjVNZG1sSWJvT3FRMGs5cGVMY3pGUElsRFRJQkk0ZWpicmE5MXhpM1FyNWpoNjV1DQp1c2VyMTokMnkkMDUkZE9tV2pkM2lvRFhUMEV5dUNsMW12Li43VnJuTjYwSjJkakYuMDdmTXJkZGo1U1RIZFFlaWENCg==
kind: Secret
metadata:
  name: htpasswd-gmszp
  namespace: openshift-config
type: Opaque

Linking Cluster-Admin Role to user1

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: cluster-admin-0
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: user1

Now, let's put each of these objects into a single file.  This combined file is called htpasswd-combined.yaml located in the same path as the above files.  This file has each object separated by ---

In order to generate an ACM Policy, we need to download the kustomize binary and policyGenerator binary.  The steps that follow are based on the following article

https://cloud.redhat.com/blog/generating-governance-policies-using-kustomize-and-gitops

1. Download kustomize binary

curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh"  | bash

This copies the file to your current directory.  I recommend copying to /usr/local/bin.

cp kustomize /usr/local/bin
chmod 755 /usr/local/bin/kustomize

2. Make a directory for policyGenerator binary

mkdir -p ${HOME}/.config/kustomize/plugin/policy.open-cluster-management.io/v1/policygenerator

3.  Download policyGenerator

curl -L \
  -o ${HOME}/.config/kustomize/plugin/policy.open-cluster-management.io/v1/policygenerator/PolicyGenerator \
  https://github.com/stolostron/policy-generator-plugin/releases/download/v1.8.0/linux-amd64-PolicyGenerator

4.  Change permissions to 755 on file

chmod +x ${HOME}/.config/kustomize/plugin/policy.open-cluster-management.io/v1/policygenerator/PolicyGenerator

Now, we need to create a couple of additional files

The first is kustomization.yaml

generators:
  - policy-generator-config.yaml

Next, is policy-generator-config.yaml

apiVersion: policy.open-cluster-management.io/v1
kind: PolicyGenerator
metadata:
  name: push-htpasswd-policy
policyDefaults:
  namespace: policies
  placement:
    clusterSelectors:
      vendor: "OpenShift"
  remediationAction: enforce
policies:
  - name: push-htpasswd-policy
    manifests:
      - path: htpasswd-combined.yaml

This takes the htpasswd-combined YAML file and adds a clusterSelector (vendor: OpenShift) and a remediation action of enforce to the resulting policy.

To build the policy, run the following command.  The policy will be output to the screen.  You can either redirect to a file or copy/paste from here.

kustomize build --enable-alpha-plugins

This resulting policy has been saved on the Github repo as htpasswd-policy.yaml for reference.  This file will be used again in the next part (deploying policy through Github repo)

apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
  name: placement-push-htpasswd-policy
  namespace: policies
spec:
  clusterConditions:
  - status: "True"
    type: ManagedClusterConditionAvailable
  clusterSelector:
    matchExpressions:
    - key: vendor
      operator: In
      values:
      - OpenShift
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
  name: binding-push-htpasswd-policy
  namespace: policies
placementRef:
  apiGroup: apps.open-cluster-management.io
  kind: PlacementRule
  name: placement-push-htpasswd-policy
subjects:
- apiGroup: policy.open-cluster-management.io
  kind: Policy
  name: push-htpasswd-policy
---
apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
  annotations:
    policy.open-cluster-management.io/categories: CM Configuration Management
    policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
    policy.open-cluster-management.io/standards: NIST SP 800-53
  name: push-htpasswd-policy
  namespace: policies
spec:
  disabled: false
  policy-templates:
  - objectDefinition:
      apiVersion: policy.open-cluster-management.io/v1
      kind: ConfigurationPolicy
      metadata:
        name: push-htpasswd-policy
      spec:
        object-templates:
        - complianceType: musthave
          objectDefinition:
            apiVersion: config.openshift.io/v1
            kind: OAuth
            metadata:
              annotations:
                include.release.openshift.io/ibm-cloud-managed: "true"
                include.release.openshift.io/self-managed-high-availability: "true"
                include.release.openshift.io/single-node-developer: "true"
                release.openshift.io/create-only: "true"
              name: cluster
            spec:
              identityProviders:
              - htpasswd:
                  fileData:
                    name: htpasswd-gmszp
                mappingMethod: claim
                name: htpasswd
                type: HTPasswd
        - complianceType: musthave
          objectDefinition:
            apiVersion: v1
            data:
              htpasswd: dXNlcjI6JDJ5JDA1JDRkQ2wvMjVNZG1sSWJvT3FRMGs5cGVMY3pGUElsRFRJQkk0ZWpicmE5MXhpM1FyNWpoNjV1DQp1c2VyMTokMnkkMDUkZE9tV2pkM2lvRFhUMEV5dUNsMW12Li43VnJuTjYwSjJkakYuMDdmTXJkZGo1U1RIZFFlaWENCg==
            kind: Secret
            metadata:
              name: htpasswd-gmszp
              namespace: openshift-config
            type: Opaque
        - complianceType: musthave
          objectDefinition:
            apiVersion: rbac.authorization.k8s.io/v1
            kind: ClusterRoleBinding
            metadata:
              name: cluster-admin-0
            roleRef:
              apiGroup: rbac.authorization.k8s.io
              kind: ClusterRole
              name: cluster-admin
            subjects:
            - apiGroup: rbac.authorization.k8s.io
              kind: User
              name: user1
        remediationAction: enforce
        severity: low

I am now going to refernce another Github repository which will be used to hold any ACM policies that I maintain.  This repo is located at

policy-collection/my-policies at main · kcalliga/policy-collection

ACM Policies. Contribute to kcalliga/policy-collection development by creating an account on GitHub.

GitHubkcalliga

The directory structure of this repo has a folder called my-policies with the htpasswd-policy.yaml contained in it

Let's create the channel and subscription which will let us push this policy automatically from ACM.  The following steps are based on

Chapter 2. Governance Red Hat Advanced Cluster Management for Kubernetes 2.6 | Red Hat Customer Portal

Access Red Hat’s knowledge, guidance, and support through your subscription.

Red Hat Customer Portal

1. Clone the ACM policy collection GitRepo

git clone https://github.com/stolostron/policy-collection.git

2.  Once you clone this repo, CD to policy-collection/deploy directory

cd policy-collection/deploy

3.  I am doing all of this as kubeadmin user so I need to create a rolebinding giving kubeadmin subscription-admin privileges

oc adm policy add-cluster-role-to-user open-cluster-management:subscription-admin system:admin

4.  Run the following command to deploy the htpasswd policy.  This command will push any policy from the my-policies path but this is the only policy currently there.

./deploy.sh -u https://github.com/kcalliga/policy-collection -p my-policies -n policy-namespace -a calligan-policies

Adding the "-a calligan-policies" will make it easier to distinguish the associated channels/subscriptions from others.

4.  After running this command, let's return to the ACM GUI and see if the policy has been pushed.

Go to Governance Menu in ACM

See the push-htpasswd-policy has been pushed

I hope you enjoyed this article.  The second part of this will be coming soon.