8 min read

Openshift Compliance Operator

This post will be based on the documentation available at https://docs.openshift.com/container-platform/4.7/security/compliance_operator/compliance-operator-understanding.html.

Many enterprises are required by law and/or corporate policies to follow the Center for Internet Security (CIS) framework and other best-practices for the secure configuration of target systems. The Openshift Compliance Operator is based on the OpenScap project which should be familiar to anyone who is in charge of IT security. For more information about OpenScap, please visit https://www.open-scap.org/. By installing this operator, you will be able to scan your Openshift cluster and the underlying operating system (CoreOS) which is read-only (hardened) version of RHEL (Red Hat Enterprise Linux) for any compliance violations and in most-cases, the violation can be mitigated by applying a MachineConfig configuration to the operating system.

In this post and accompanying videos, I will:

  1. Show how the Compliance Operator is installed via the OCP web console.
  2. Explain the various custom-resource-defininions/APIs that are available after installing the Compliance Operator.
  3. Explain the profiles that are available out-of-the-box.
  4. Describe Compliance Rules
  5. Describe Compliance Operator Scans
  6. How to Retrieve Operator Scan Results
  7. How to Handle Remediations

The Compliance Operator lets OpenShift Container Platform administrators describe the desired compliance state of a cluster and provides them with an overview of gaps and ways to remediate them. The Compliance Operator assesses compliance of both the Kubernetes API resources of OpenShift Container Platform, as well as the nodes running the cluster. The Compliance Operator uses OpenSCAP, a NIST-certified tool, to scan and enforce security policies provided by the content.

Installing the Compliance Operator

Custom-Resource-Definitions for Compliance Operator

A brief overview of the various custom-resource-defintons will follow.

ComplianceCheckResult- ComplianceCheckResult represent a result of a single compliance “test”

ComplianceRemediation – ComplianceRemediation represents a remediation that can be applied to the cluster to fix the found issues.

ComplianceScan– ComplianceScan represents a scan with a certain configuration that will be applied to objects of a certain entity in the host. These could be nodes that apply to a certain nodeSelector, or the cluster itself.

ComplianceSuite– ComplianceSuite represents a set of scans that will be applied to the cluster. These should help deployers achieve a certain compliance target.

ProfileBundle– ProfileBundle is the Schema for the profilebundles API.

Profile– Profile is the Schema for the profiles API.

Rule– Rule is the Schema for the rules API.

ScanSetBinding– ScanSettingBinding is the Schema for the scansettingbindings API.

ScanSetting– ScanSetting is the Schema for the scansettings API.

TailoredProfile– TailoredProfile is the Schema for the tailoredprofiles API.

Variable– Variable describes a tunable in the XCCDF profile.

Profiles

To list the various profiles that are available, run the following command:

oc get -n openshift-compliance profiles.compliance


To describe each of these policies, run the following command:

oc describe profiles.compliance -n openshift-compliance


Here is some information about the profiles (profiles.compliance CRD) that are available out-of-the-box:

OCP4-CIS– This profile defines a baseline that aligns to the Center for Internet Security;Red Hat OpenShift Container Platform 4 Benchmark™, V0.3, currently unreleased. This profile includes Center for Internet Security;Red Hat OpenShift Container Platform 4 CIS Benchmarks™ content. Note that this part of the profile is meant to run on the Platform that Red Hat OpenShift Container Platform 4 runs on top of.This profile is applicable to OpenShift versions 4.6 and greater.

OCP4-CIS-NODE– This profile defines a baseline that aligns to the Center for Internet Security;Red Hat OpenShift Container Platform 4 Benchmark™, V0.3, currently unreleased. This profile includes Center for Internet Security® Red Hat OpenShift Container Platform 4 CIS Benchmarks™ content. Note that this part of the profile is meant to run on the Operating System that Red Hat OpenShift Container Platform 4 runs on top of. This profile is applicable to OpenShift versions 4.6 and greater.

OCP4-E8– This profile contains configuration checks for Red Hat OpenShift Container Platform that align to the Australian Cyber Security Centre (ACSC) Essential Eight. A copy of the Essential Eight in Linux Environments guide can be found at the ACSC website: https://www.cyber.gov.au/acsc/view-all-content/publications/hardening-linux-workstations-and-server

OCP4-MODERATE– This compliance profile reflects the core set of Moderate-Impact Baseline configuration settings for deployment of Red Hat OpenShift Container Platform into U.S. Defense, Intelligence, and Civilian agencies. Development partners and sponsors include the U.S. National Institute of Standards and Technology (NIST), U.S. Department of Defense, the National Security Agency, and Red Hat. This baseline implements configuration requirements from the following sources: – NIST 800-53 control selections for Moderate-Impact systems (NIST 800-53) For any differing configuration requirements, e.g. password lengths, the stricter security setting was chosen. Security Requirement Traceability Guides (RTMs) and sample System Security Configuration Guides are provided via the scap-security-guide-docs package. This profile reflects U.S. Government consensus content and is developed through the ComplianceAsCode initiative, championed by the National Security Agency. Except for differences in formatting to accommodate publishing processes, this profile mirrors ComplianceAsCode content as minor divergences, such as bugfixes, work through the consensus and release processes.

RHCOS4-E8– This profile contains configuration checks for Red Hat Enterprise Linux CoreOS that align to the Australian Cyber Security Centre (ACSC) Essential Eight. A copy of the Essential Eight in Linux Environments guide can be found at the ACSC website: https://www.cyber.gov.au/acsc/view-all-content/publications/hardening-linux-workstations-and-servers

RHCOS4-MODERATE– This compliance profile reflects the core set of Moderate-Impact Baseline configuration settings for deployment of Red Hat Enterprise Linux CoreOS into U.S. Defense, Intelligence, and Civilian agencies. Development partners and sponsors include the U.S. National Institute of Standards and Technology (NIST), U.S. Department of Defense, the National Security Agency, and Red Hat. This baseline implements configuration requirements from the following sources: – NIST 800-53 control selections for Moderate-Impact systems (NIST 800-53) For any differing configuration requirements, e.g. password lengths, the stricter security setting was chosen. Security Requirement Traceability Guides (RTMs) and sample System Security Configuration Guides are provided via the scap-security-guide-docs package. This profile reflects U.S. Government consensus content and is developed through the ComplianceAsCode initiative, championed by the National Security Agency. Except for differences in formatting to accommodate publishing processes, this profile mirrors ComplianceAsCode content as minor divergences, such as bugfixes, work through the consensus and release processes.

Compliance Rules

Within each profile is a bunch of rules that can check and mitigate any vulnerability. One example is to log login audit events. The name for one of the rules is rhcos4-audit-rules-login-events.

To see the list of all rules, run the following command:

oc get -n openshift-compliance rules.compliance

To describe one of the rules, run the following command:

oc describe -n openshift-compliance rules.compliance <nameofrule>

The description section in the output will show more details about the rule.



Compliance Operator Scans

The scansettings CRD shows the general specifications of the scans that will be run including the persistent volume settings, the node roles that will be checked, and when the scan is scheduled.

To show the default scansetting, run the following command:

oc describe scansettings default



There are 2 scansetting CRDs in a default cluster. One is called “default” (shown above) and the other is default-auto-apply. The difference between the 2 is that the auto-apply scansetting will automatically try to remediate any issue that shows in the scan.

The default scansetting will be used right now so I can show you have to apply a remediation.

A ScanSetBinding attaches the scan settings to one of the compliance profiles. There are no scansetbindings by default.

To create one, use the following template. The following example shows the ocp4 specific scansettings. Another object would need to be created in order to check the nodes. The node one can use the rhcos4-moderate profile.

apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: cis-compliance
profiles:
  - name: ocp4-cis-node
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
  - name: ocp4-cis
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: default
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1

This attaches the ocp-4-cis-node and ocp4-cis profiles to the default scansetting.

To apply, run the following command:

oc create -f <nameoffile>.yaml -n openshift-compliance

For demonstration purposes, I am going to edit the default scansetting to run at a time close to the current time so we can check for results. I am setting for approximately 5 minutes from current time which is 14:50. The schedule parameter is in cron format.

oc edit scansetting default



To check the status of compliancescan, run the following command:

oc get compliancescan -w -n openshift-compliance

Retrieving Operator Scan Results

After a scan is run, a ComplianceSuite CRD will be created.

The results shown in the following examples are OCP specific (based on ocp4 profiles). The scansettingbinding should be updated to use the rhcos4-moderate profile to see operating system scans.

To get a listing of the compliance suites available, run the following command:

oc get compliancesuites

The result shows “inconsistent”. Let’s look into that some more.

To get the listing of the persistent volume claims that hold the report(s), run the following command:

oc get compliancesuites cis-compliance -o json \
| jq ‘.status.scanStatuses[].resultsStorage’


To list some basic info about the PVC, run the following command:

oc get pvc -n openshift-compliance ocp4-cis


To view the report, we will need to create a pod to claim the persistent volume. Use this template:

apiVersion: "v1"
kind: Pod
metadata:
  name: pv-extract
spec:
  containers:
    - name: pv-extract-pod
      image: registry.access.redhat.com/ubi8/ubi
      command: ["sleep", "3000"]
      volumeMounts:
      - mountPath: "/workers-scan-results"
        name: workers-scan-vol
  volumes:
    - name: workers-scan-vol
      persistentVolumeClaim:
        claimName: ocp4-cis
oc create -f <nameoffile>.yaml -n openshift-compliance

Now, copy this report to your local machine

oc cp pv-extract:/workers-scan-results .

Now, delete the pv-extract pod


oc delete pod pv-extract

The reports will be zipped with bzip. You will need to install bzip2 to uncompress.

bzip2 -d <nameoffile>.bz

To view the file, it is best to output to something along the lines of html. You will need to install openscap-utils to run this.

oscap xccdf generate report <nameoffile>.out > report.html

Sample Report


Handling Remediations

The ComplianceRemediation CRD shows how to fix the problem defined in the ComplianceCheckResult.

To see a list of the ComplianceRemediation CRDs, run the following command:

oc get complianceremediation

The results show NotApplied due to the fact that we are only using the default (not the default-auto-deploy scansetting). To remediate automatically, we would need to use the default-auto-deploy scansetting instead of default.

The ComplianceCheckResult will show the results of any scan (collectively across all roles that are scanned).

oc get ComplianceCheckResult

Keith Calligan