Skip to content

Cloud Security

1. Cloud Native Security

Note
  • Followed a layered approach that is based on defense in depth approach.
  • The 4C layers of Cloud Native Security
  • Cloud Datacenter
  • Cluster (Kubernetes)
  • Container
  • Code 
    ???note Cloud Data Center
- Cloud Provider: Their hardware their policy
- In-house infrastructures
  - Physical security
  - Network access
  - Database access (`etcd` for Kubernetes)

???note Cluster (Kubernetes) - RBAC Authorization - Authentication - Encryption - Network Policies - TLS for Kubernetes Ingress 

???note Container
  - Vulnerability Scanner/OS Dependencsy Security
  - Privilege Control/Disabling
  - Container runtime isolation

Note
  • Secure communication over TLS
  • Port range limitation
  • Security of external library dependency
  • Static code analysis
  • Dynamic code security (SQL injection, cross-site scripting, ...) 
    ## 2. Cluster Security: Kubernetes

???note Authentication

- Users: 
  - Normal users (managed by host system/remote management like LDAP)
  - Service accounts (internal to K8s)
- Users can present a valid certificate signed by the cluster's CA (Certificate Authority) 
are considered authenticated.
Note
  • Instantiate from branch k8s-helm of the csc603ngo CloudLab profile
  • Once the experiment is fully deployed, and all Startup Finished running:
  • SSH into the head node and run the followings
$ cat ~/.kube/config
  • Keep a record of the following fields (or do this in separate terminals)
  • certificate-authority-data
  • server
  • client-certificate-data

  • client-key-data

  • Create a new user called student:

$ sudo useradd -s /bin/bash -d /home/student -m -G docker student
$ sudo su - student
  • Try running kubectl commands now:
$ kubectl get nodes
$ kubectl get svc
  • As student, run the following command:
$ mkdir .kube
  • Next, create a file called config inside .kube with the following contents:
  • Replace COPY_FROM_ABOVE with the corresponding values of the corresponding fields above. 
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: COPY_FROM_ABOVE
    server: COPY_FROM_ABOVE
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: student
  name: student@kubernetes
current-context: student@kubernetes
kind: Config
preferences: {}
users:
- name: student
  user:
    client-certificate-data: COPY_FROM_ABOVE
    client-key-data: COPY_FROM_ABOVE
  • Try running kubectl commands now:
$ kubectl get nodes
$ kubectl get svc

Challenge Adding at least another Kubernetes cluster from one of your classmates' experiment to your student account. In other words, one account can have access to multiple K8S cluster. 

???note Namespace overview

- Similar to Linux's namespaces, Kubernetes' namespaces allow the isolation 
of groups of resources withint a single cluster. 
  - Names of isolated resources within the same namespace need to be unique, 
  but not across namespaces. 
- Scope of namespace isolation is only applicable to namespaced objects like 
`Deployments` and `Services`, but not for cluster-wide objects like `StorageClass`, 
`Nodes`, and `PersistentVolumes`. 
- To check which Kubernetes resources are and aren't in a namespace:

~~~bash
# In a namespace
$ kubectl api-resources --namespaced=true

# Not in a namespace
$ kubectl api-resources --namespaced=false
~~~
Note
  • Kubernetes uses the rbac.authorization.k8s.io API group to drive RBAC-based authorization decisions,
  • By default, Kubernetes clusters are often launched with RBAC:
$ kubectl describe pods kube-apiserver -n kube-system | grep authorization
  • RBAC API: Role, ClusterRole, RoleBinding, ClusterRoleBinding.
  • Role: sets permissions within a particular namespace
$ kubectl get role --all-namespaces
$ kubectl describe role system::leader-locking-kube-controller-manager -n kube-system
  • ClusterRole: set permission for non-namespaced resources
$ kubectl get clusterrole --all-namespaces
$ kubectl describe clusterrole cluster-admin
  • RoleBinding binding grants the permissions defined in a role to a user or set of users.
  • ClusterRoleBinding grants that access cluster-wide.

$ kubectl get RoleBinding --all-namespaces
$ kubectl get ClusterRoleBinding

- What `resources` are available and what `verbs` are applicable?
$ kubectl api-resources --sort-by name -o wide 
```

???note RBAC Authorization hands-on
- Run the following to deploy jenkins

~~~bash
$ bash /local/repository/jenkins/deploy_jenkins.sh

  • Setup a single build pipeline
Note
  • Create a new repository called hello_kube with contents copied from the instructor's hello repo's hello_kube branch
  • In Jenkins, create a New Item of type Pipeline and name it hello_kube, click OK.
  • Under Pipeline select Pipeline script from SCM.
  • Once SCM appears, select Git and provide the Repository URL for your hello_kube repo.
  • Under Branch to build, change */master to */main.
  • Make sure that Script Path, enter Jenkinsfile.
  • This is the Jenkinsfile in the hello_kube branch.
  • Click Apply.
  • Click Save.
  • Inside the Pipeline hello_kube, select Build Now.

:::{image} ../fig/csc603/08-cloud-security/rbac-01.png :alt: Missing service account :class: bg-primary mb-1 :height: 800px :align: center :::

  • Let's create a service account, then rerun Build Now. 
$ kubectl create serviceaccount jenkins
  • Has to be in the same namespace
$ kubectl delete serviceaccount jenkins
$ kubectl create namespace jenkins
$ kubectl create serviceaccount jenkins -n jenkins
  • Try Build Now again, does it still work?
???note Kubernetes Ingress

- Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the 
cluster. 
- Traffic routing is controlled by rules defined on the Ingress resource.
- Ingress is managed by Ingress Controllers. 
- There are many diffferent Ingress Controllers
Note
  • Deploy nginx ingress controller using NodePort
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.4.0/deploy/static/provider/baremetal/deploy.yaml
$ kubectl get svc -n ingress-nginx -o wide
  • Go to the URL gotten from running hostname -f and use the port that is mapped to port 80 of ingress-nginx-controller to view:

:::{image} ../fig/csc603/08-cloud-security/ingress-01.png :alt: Ingress controller with no back-end services :class: bg-primary mb-1 :height: 200px :align: center :::

  • Deploy an apache server and observe its opened ClusterIP port (internal to the cluster)
$ kubectl create deployment demo --image=httpd --port=80
$ kubectl expose deployment demo
$ kubectl get svc
  • Visit the URL gotten from running hostname -f and see if you can view the default webpage.
  • No you can't!
  • Now let's redirect things via the ingress controller
kubectl create ingress demo-localhost --class=nginx --rule="$(hostname -f)/*=demo:80"
  • Refresh the URL to the ingress controller above

:::{image} ../fig/csc603/08-cloud-security/ingress-02.png :alt: Ingress controller with back-end services :class: bg-primary mb-1 :height: 200px :align: center :::

  • What about another apps?
kubectl run template-frontend --image=hashicorp/http-echo --labels=app=template,type=frontend -- -listen=:80 -text="Frontend"
kubectl run template-api --image=hashicorp/http-echo --labels=app=template,type=api -- -listen=:80 -text="API"
kubectl expose pod template-api --port=80
kubectl expose pod template-frontend --port=80
kubectl get svc
  • Delete and recreate the demo-localhost rule
  • We need to add an annotation to the rule to provide alternative path options to different service end-points. 
kubectl delete ingress demo-localhost
kubectl create ingress demo-localhost --class=nginx   --rule="$(hostname -f)/demo/=demo:80" --rule="$(hostname -f)/api/=template-api:80" --rule="$(hostname -f)/frontend/=template-frontend:80"
kubectl annotate ingress demo-localhost nginx.ingress.kubernetes.io/rewrite-target="/"
  • Refresh the URL to the ingress controller above with different paths: /demo/, /api/, and /frontend/.
  • What about security?
  • Need to deploy certificate manager
$ bash /local/repository/cert-manager/deploy_cert_manager.sh
  • Check and find the corresponding node port for 443 (HTTPS) of the nginx-controller
$ kubectl get svc -n ingress-nginx
  • Recreate the demo-local ingress with updated TLS rules
kubectl delete ingress demo-localhost
kubectl create ingress demo-localhost --class=nginx   --rule="$(hostname -f)/demo/=demo:80,tls=clemson.cloudlab.us-cert-secret" --rule="$(hostname -f)/api/=template-api:80" --rule="$(hostname -f)/frontend/=template-frontend:80"
kubectl annotate ingress demo-localhost nginx.ingress.kubernetes.io/rewrite-target="/"
  • Confirm that URL to the ingress controller above with different paths: /api/ and /frontend/ still work with the HTTP NodePort, but /demo/ requires HTTPS NodePort

:::{image} ../fig/csc603/08-cloud-security/ingress-03.png :alt: Ingress controller with secured back-end services :class: bg-primary mb-1 :height: 200px :align: center :::

## 3. Container Security

???note Pod Security Standards
- Pod security will influence container security. 
- Three policies:
  - Privileged
  - Baseline
  - Restricted 
- Cummulative and range from highly permissive to highly restrictive
Note
  • Purposely open, entirely unrestricted
  • Enable system- and infrastructure-level workloads
  • Implied understanding that these workloads are run as privileged/trusted users.
  • Absence of restrictions and allow-by-default mechanism
???note Baseline

- Streamline adoption for common containerized workloads
- Prevent known privilege escalations
- Example capabilities (Linux capabilities) enabled in baseline
  - AUDIT_WRITE
  - CHOWN
  - DAC_OVERRIDE
  - FOWNER
  - FSETID
  - KILL
  - MKNOD
  - NET_BIND_SERVICE
  - SETFCAP
  - SETGID
  - SETPCAP
  - SETUID
  - SYS_CHROOT
- [Linux full capability list](https://man7.org/linux/man-pages/man7/capabilities.7.html)
Note
  • Enforce Pod hardening best practices at the expense of compatibility
  • Target security-critical applications and lower-trust users
  • Example policy:
  • Disable privilege escalation
  • Containers must run as non-root
  • Drop ALL capabilities and only allow NET_BIND_SERVICE. ```