Kubernetes Security

This tutorial is adapted from Web Age course Kubernetes for Developers Training.

1.1 Security Overview

Security is critical to production deployments. Kubernetes offers several features to secure your environment:

authentication

authorization

  • ABAC
  • RBAC

Role, ClusterRole, RoleBinding, ClusterRoleBinding

network policies

1.2 API Server

Kubernetes has a built-in API server that provides access to objects, such as nodes, pods, deployments, services, secrets, config maps, and namespaces. These objects are exposed via simple REST API through which basic CRUD operations are performed. API Server acts as the gateway to the Kubernetes platform. Components such as kubelet, scheduler, and controller access the API via the API Server for orchestration and coordination. The distributed key/value database, etcd, is accessible only through the API Server. In the Kubernetes API, most resources are represented and accessed using a string representation of their object name, such as pods for a Pod. Some Kubernetes APIs involve a subresource, such as the logs for a Pod. A request for a Pod’s logs looks like:

GET /api/v1/namespaces/{namespace}/pods/{name}/log

1.3 API & Security

Both the kubectl CLI tool and the web portal talks to the API Server. Before an object is accessed or manipulated within the Kubernetes cluster, the request needs to be authenticated by the API Server. The REST endpoint uses TLS based on the X.509 certificate to secure and encrypt the traffic. The CA certificate and client certificate information is stored in ~/.kube/config.

You can view the file using any text editor or you can also view it by running the following command:

kubectl config view

1.4  ~/.kube/config

Sample ~/.kube/config file

apiVersion: v1
clusters:
- cluster:
    certificate-authority: /Users/test/.minikube/ca.crt
    server: https://192.168.99.100:8443
  name: minikube
contexts:
- context:
    cluster: minikube
    user: minikube
  name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
  user:
    client-certificate: /Users/test/.minikube/client.crt
    client-key: /Users/test/.minikube/client.key

1.5 ~/.kube/config (contd.)

The file ca.crt represents the CA used by the cluster. The client.crt and client.key files map to the user minikube that is the default cluster-admin. Kubectl uses these certificates and keys from the current context to encode the request.

 

1.6 Kubernetes Access Control Layers

When a valid request hits the API Server, it goes through three stages before it is either allowed or denied.

  • Authentication
  • Authorization
  • Admission Controller

1.7 Authentication

After the request gets past TLS, it passes through the authentication phase that involves authentication modules. Authentication modules are configured by the administrator during the cluster creation process. Examples of authentication modules: client certificates, password, plain tokens, bootstrap tokens, and JWT tokens (used for service accounts). Details of authentication modules are available on the Kubernetes website: https://kubernetes.io/docs/reference/access-authn-authz/authentication/

Client certificates are the default and most common scenario. External authentication mechanisms provided by OpenID, Github, or even LDAP can be integrated with Kubernetes through one of the authentication modules.

1.8 Authorization

After authentication, the next step is to determine whether the operation is allowed or not. 

For authorizing a request, Kubernetes looks at three aspects:

  • the username of the requester – extracted from the token embedded in the header
  • the requested action – one of the HTTP verbs like GET, POST, PUT, DELETE mapped to CRUD operations
  • the object affected by the action – one of the valid Kubernetes objects such as a pod or a service.

Kubernetes determines the authorization based on an existing policy. By default, Kubernetes follows the philosophy of closed-to-open, which means an explicit allow policy is required to even access the resources. Like authentication, authorization is configured based on one or more modes/modules, such as:

  • RBAC
  • ABAC

1.9 ABAC Authorization

Attribute-based access control (ABAC) defines an access control paradigm whereby access rights are granted to users through the use of policies that combine attributes. ABAC uses a policy file where one JSON object is listed per line. Each line in the JSON policy file is a policy object.

If you are using the Minikube distribution, you can enable ABAC authorization like this:

minikube start --extra-config=apiserver.AuthorizationMode=ABAC --extra-config=apiserver.AuthorizationPolicyFile=/path/to/your/abac/policy.json

1.10 ABAC – Policy Format

Versioning properties:

  • apiVersion: “abac.authorization.kubernetes.io/v1beta1”
  • kind: “Policy”

spec: property set to a map with the following properties:

Subject-matching properties:

  • user: “userName
  • group: “groupName” | system:authenticated | system:unauthenticated

Resource-matching properties:

  • apiGroup: “*” | “extensions
  • namespace: “*” | “your_custom_namespace”
  • resource: “*” | “pods” | “deployments” | “services“, …

Non-resource-matching properties:

  • nonResourcePath: “/version” | “*”

readonly: true | false, type boolean, when true, means that the Resource-matching policy only applies to get, list, and watch operations, Non-resource-matching policy only applies to get operation.

1.11 ABAC – Examples

Alice can do anything to all resources:

{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user": "alice", "namespace": "*", "resource": "*", "apiGroup": "*"}}

The Kubelet can read any pods:

{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user": "kubelet", "namespace": "*", "resource": "pods", "readonly": true}}

The Kubelet can read and write events:

{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user": "kubelet", "namespace": "*", "resource": "events"}}

Bob can just read pods in namespace “projectCaribou”:

{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user": "bob", "namespace": "projectCaribou", "resource": "pods", "readonly": true}}

Anyone can make read-only requests to all non-resource paths:

{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"group": "system:authenticated", "readonly": true, "nonResourcePath": "*"}}
{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"group": "system:unauthenticated", "readonly": true, "nonResourcePath": "*"}}

1.12  RBAC Authorization

Role-based access control (RBAC) is a method of regulating access to a computer or network resources based on the roles of individual users within your organization.

RBAC authorization uses the rbac.authorization.k8s.io API group to drive authorization decisions, allowing you to dynamically configure policies through the Kubernetes API.

RBAC authorization involves the following resources:

  • Role
  • CluserRole
  • RoleBinding
  • ClusterRoleBinding

RBAC is the default authorization mode. If you want to explicitly specify this mode, you can use the following command with the Minikube distribution:

minikube start --extra-config=apiserver.Authorization.Mode=RBAC

1.13 Role and ClusterRole

An RBAC Role or ClusterRole contains rules that represent a set of permissions.

Role – always sets permissions within a particular namespace. When you create a Role, you have to specify the namespace it belongs in. Treat it as a project-based role where a user will have to access to a specific namespace.

ClusterRole – is a non-namespaced resource. Use it to create admin users who can define permissions on namespaced resources and be granted within an individual namespace(s). It defines permissions on cluster-scoped resources, such as nodes. For example, you can use a ClusterRole to allow a particular user to run kubectl get pods –all-namespaces. The resources have different names (Role and ClusterRole) because a Kubernetes object always has to be either namespaced or not namespaced; it can’t be both.

 

1.14 Role – Example

Here’s an example Role in the “marketing” namespace that can be used to grant read access to pods:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: marketing
  name: marketing-pod-reader
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

 

1.15 ClusterRole – Example

Here is an example of a ClusterRole that can be used to grant read access to nodes:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  # "namespace" omitted since ClusterRoles are not namespaced
  name: nodes-reader
rules:
- apiGroups: [""]
  #
  # at the HTTP level, the name of the resource for accessing Secret
  # objects is "nodes"
  resources: ["nodes"]
  verbs: ["get", "watch", "list"]

 

1.16 RoleBinding and ClusterRoleBinding

A role binding grants the permissions defined in a role to a user or set of users. It holds a list of subjects (users, groups, or service accounts), and a reference to the role being granted. A RoleBinding grants permissions within a specific namespace whereas ClusterRoleBinding grants that access cluster-wide. A RoleBinding may reference any Role in the same namespace. If you want to bind a ClusterRole to all the namespaces in your cluster, you use a ClusterRoleBinding.

1.17 RoleBinding – Example

Here is an example of a RoleBinding that grants the “pod-reader” Role to the user “alice” within the “sales” namespace. This allows “alice” to read pods in the “default” namespace.

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: sales
subjects:
- kind: User
  name: alice
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

1.18 ClusterRoleBinding – Example

The following ClusterRoleBinding allows any user in the group “manager” to read deployments in any namespace.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: read-deployment-global
subjects:
- kind: Group
  name: manager 
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: deployment-reader
  apiGroup: rbac.authorization.k8s.io

1.19 Authorization Modes – Node

A special-purpose authorization mode that grants permissions to kubelets based on the pods they are scheduled to run. To learn more about using the Node authorization mode

1.20 Authorization Modes – ABAC

In Attribute-based access control (ABAC), access rights are granted to users through the use of policies that combine attributes. The policies can use any type of attributes (user attributes, resource attributes, object, environment attributes, etc). To enable ABAC mode, specify –authorization-policy-file=SOME_FILENAME and –authorization-mode=ABAC on startup.

1.21 Admission Controller

After authorization, the request goes through the final stage: Admission Controller. Admission controllers limit requests to create, delete, modify, or connect to (proxy). They do not support read requests. For example, an admission control module may be used to enforce the pulling of images policy each time a pod is created. There are various admission controllers compiled into the kube-apiserver binary. Here are some of them:

  • AlwaysPullImages: When this admission controller is enabled, images are always pulled before starting containers, which means valid credentials are required
  • CertificateApproval: This admission controller observes requests to ‘approve’ CertificateSigningRequest resources

For more details, refer to Kubernetes doc: https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/

1.22 Network Policies

Network policies are the equivalent of a firewall that specify how groups of pods are allowed to communicate with each other and other network endpoints. Each network policy has a podSelector field, which selects a group of pods. When a pod is selected by a network policy, the network policy is applied to it. Each network policy also specifies a list of allowed (ingress and egress) connections. When the network policy is created, all the pods that it applies to are allowed to make or accept the connections listed in it. If no network policies are applied to a pod, then no connections to or from it would be permitted. Network policies require a network plugin that enforces network policies. Although Kubernetes allows the creation of network policies they aren’t enforced unless a plugin is installed and configured. There are various plugins, such as Calico, Cilium, Kube-router, Romana, and, Weave Net.

1.23 Network Policies – Examples

You can apply various network policies, such as:

  • Limit access to services
  • Pod isolation
  • Allow internet access for pods
  • Allow pod-to-pod communication within the same or different namespaces.

You can get various useful network policy recipes available from the following sites:

https://github.com/ahmetb/kubernetes-network-policy-recipes

https://github.com/stackrox/network-policy-examples

1.24 Network Policies – Pod Isolation

Pods are “isolated” if at least one network policy applies to them; if no policies apply, they are “non-isolated”. Network policies are not enforced on non-isolated pods. This behavior exists to make it easier to get a cluster up and running a user who does not understand network policies can run their applications without having to create one. It’s recommended you start by applying a “default-deny-all” network policy. The effect of the default-deny-all policy specification is to isolate all pods, which means that only connections explicitly listed by other network policies will be allowed.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
spec:
  podSelector: {}
  policyTypes:
  - Ingress

Notes

Since network policies are namespaced resources, you will need to create this policy for each namespace. You can do so by running kubectl -n <namespace> create -f <filename> for each namespace.

1.25 Network Policies – Internet Access for Pods

With just the default-deny-all policy in place in every namespace, none of your pods will be able to talk to each other or receive traffic from the Internet. For most applications to work, you will need to allow some pods to receive traffic from outside sources.

The following network policy allows traffic from all sources for pods having the custom networking/allow-internet-access=true label:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: internet-access
spec:
  podSelector:
    matchLabels:
      networking/allow-internet-access: "true"
  policyTypes:
  - Ingress
  ingress:
  - {}

1.26  Network Policies – New Deployments

When you create new deployments, they will not be able to talk to anything by default until you apply a network policy. You can create custom network policies that allow deployments/pods labeled networking/allow-all-connections=true to talk to all other pods in the same namespace.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-ingress-from-new
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          networking/allow-all-connections: "true"
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-ingress-to-new
spec:
  podSelector:
    matchLabels:
      networking/allow-all-connections: "true"
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector: {}

1.27 Summary

In this tutorial, you learned the following:

  • Security Overview
  • Accessing the API
  • Authentication
  • Authorization
  • ABAC and RBAC
  • Admission Controller
  • Network Policies

Kubernetes Architecture

This tutorial is adapted from Web Age course Docker and Kubernetes Administration.

1.1 Architecture Diagram

In this tutorial, we will review various parts of the following architecture diagram:

Kubenetes Architecture

1.2 Components

Cluster – Includes one or more master and worker nodes

Master – Manages nodes and pods

(worker) Node – a physical, virtual or cloud machine

Pod – A group of one or more containers, created and managed by Kubernetes

Container – Are most commonly Docker containers where application processes are run

Volume – A directory of data accessible to containers in a pod. It shares a lifetime with the pod it works with.

Namespace – A virtual cluster. Allows for multiple virtual clusters within a physical one.

1.3 Kubernetes Cluster

A Kubernetes cluster is a set of machines(nodes) used to run containerized applications. To do work a cluster needs to have at least one master node and one worker node. The Master node determines where and what is run on the cluster. Worker nodes contain pods that contain containers. Containers hold execution environments where work can be done. A cluster is configured via the kubectl command-line interface or by the Kubernetes API.

1.4 Master Node

 The Master node manages worker nodes.

The master node includes several components:

Kube-APIServer – traffic enters the cluster here

Kube-Controller-Manager – runs the cluster’s controllers

Etcd – Maintains cluster state, provides key-value persistence

Kube Scheduler – schedules activities to worker nodes

Clusters can have more than one master node

Clusters can have only one active master node

 

1.5 Kube-Control-Manager

The Kube-Control-Manager (part of the Master Node) manages the following controllers:

Node controller

Replication controller

Endpoints controller

Service account controller

Token controller

All these controller operations are compiled into a single application. The controllers are responsible for the configuration and health of the cluster’s components.

1.6 Nodes

A node consists of a physical, virtual, or cloud machine where Kubernetes can run Pods that house containers. Clusters have one or more nodes. Nodes can be configured manually through kubectl. Nodes can also self-configure by sending their information to the Master when they start up. Information about running nodes can be viewed with kubectl.

Notes

Other components found on the worker node include:

kubelet – interacts with the master node, manages containers and pods on the node

kube-proxy – responsible for network configuration

container runtime – responsible for running containers in the pods (typically Docker)

 

1.7 Other Components

Pods – Logical container for runtime containers

Containers – Pods typically contain Docker runtime containers holding OS images and applications. Work is run in containers.

 

1.8 Interacting with Kubernetes

All user interaction goes through the master node’s api-server. kubectl provides a command-line interface to the API. Control of Kubernetes can also be done through the Kubernetes Dashboard (web UI).

 

1.9 Summary

In this tutorial, we covered:

Architecture Diagram

Components

Cluster

Master

Node

Pod

Container

Interaction through API

How to do Unit Testing with Jest in React?

In this tutorial, we will create a new project to illustrate Test Driven Development (TDD) with React and Jest. We will create a simple model that can generate a greeting and integrate it in a Reach application. 

Part 1 – Creating the project

Let’s create a new project.

1. Create a directory C:\LabWork , open a command project.

2. Change into the LabWork directory [create the folder if required]:

cd C:\LabWork

3. Create a new React project using the following command:

npx create-react-app testing-app

4. Enter the new project:

cd testing-app

If desired, start your favorite text editor.

Part 2 – Creating our first test

In the spirit of TDD, let’s create a unit test to verify that our future model works correctly. The unit test will also serve as a specification for our model.

1. Using your preferred text editor, create the file src/Greeter.test.js

2. Enter the following code:

import React from 'react';
import Greeter from './Greeter'

test('provides a greeting', () => {
  const greeter = new Greeter('Ada Lovelace');

  const greeting = greeter.getGreeting();

  expect(greeting).toBe('Hello, Ada Lovelace')
});

3. Save your work.

4. Run the unit test and watch it fail:

npm run test

By default, the test running executes any files in the src directory that end with ‘.test.js’

5. Leave the test running in the command prompt window.

6. Create a new file called src/Greeter.js

7. Enter the following code:

export default class Greeter {
  constructor(userName = '') {
    this.userName = userName;
  }

  getGreeting() {
    if (this.userName) {
      return `Hello, ${this.userName}`;
    } else {
      return `Hello, Anonymous`;
    }
  }

  async getGreetingAsync() {
    return this.getGreeting();
  }
}

8. Save your work.

9. Return to the command prompt. Observer that the test now passes.

(If the test fails, verify your code from step 7)

10. Let’s complete the unit test. Add the following code at the end of src/Greeter.test.js:

test('provides a default greeting', () => {
  const greeter = new Greeter();

  const greeting = greeter.getGreeting();

  expect(greeting).toBe('Hello, Anonymous')
});

test('Can generate a greeting asynchronously', async () => {
  const greeter = new Greeter('Ada Lovelace');

  const greeting = await greeter.getGreetingAsync();

  expect(greeting).toBe('Hello, Ada Lovelace');
})

11. Save the file.

12. Return to the command prompt window. Observe that now all tests are said to pass.

Part 3 – Creating the React application

In the previous part, we performed unit tests but we did not write any React code. Now, let’s add a user interface on top of our model and test it using Jest.

1. Keep the current command prompt window open.

2. Start a second command prompt.

3. Change directory to C:\LabWork\testing-app

cd C:\LabWork\testing-app

4. Start the development server by executing the following:

npm run start

The placeholder React page comes up in the default browser.

5. Create the file src/GreeterComponent.js with the following code:

import React from 'react';
import Greeter from './Greeter'

export default class GreeterComponent extends React.Component {
  constructor(props) {
    super(props)

    this.state = {
      greeter: new Greeter(),
    }
  }

  static getDerivedStateFromProps(props, state) {
    return {
      greeter: new Greeter(props.userName)
    };
  }

  render() {
    return <div id='greet'>Message: {this.state.greeter.getGreeting()}</div>
  }
}

6. Save the file.

7. Open src/App.js and replace the placeholder code with the following:

import React from 'react'
import './App.css'
import Greeter from './GreeterComponent'

export default class App extends React.Component {
  constructor(props) {
    super(props)
    this.state = {
      userName: '',
    }
  }

  updateUserName = (event) => {
    this.setState({
      userName: event.target.value,
    })
  }

  clearName = (event) => {
    this.setState({
      userName: '',
    })
  }

  render() {
    return (
      <div id='app'>
        <h3>Input:</h3>
        <p>
          <label htmlFor='userName'>User Name: </label>
          <input type='text' id='userName' value={this.state.userName} onChange={this.updateUserName}/>
          <button onClick={this.clearName}>Clear</button>
        </p>
        <h3>Message:</h3>
        <Greeter userName={this.state.userName}/>
      </div>
    )
  }
}

8. Save the file.

9. Return to the default web browser, and ensure the following page comes up:

10. Test the basic behavior by entering a name and observe the message down below. Test the clear button.

11. Return to the command-prompt that is running Jest. Notice there are src/App.test.js is now failing.

We are ready to add React tests!

Part 4 – Adding React Test

In this part, we will add automated test to ensure that our application renders as expected.

1. Open src/App.test.js

2. Replace the code with the following:

import React from 'react';
import { render } from '@testing-library/react';
import App from './App';
import { fireEvent } from '@testing-library/react'

test('renders two message elements', () => {
  const app = render(<App />);
  const inputHeadings = app.getAllByText(/message:/i);
  expect(inputHeadings.length).toBe(2);
});

3. Save the file.

4. Ensure that all unit test pass.

5. Let’s add a couple more unit test. Add the following code at the end of src/App.test.js:

test('has a GreetingComponent', () => {
  const app = render(<App />);

  const greetingComponent = app.getByText(/Anonymous/i);

  expect(greetingComponent).toBeInTheDocument();
})

test('changing the input updates the message', () => {
  const app = render(<App />);

  const input = app.getByLabelText(/user name/i);

  fireEvent.change(input, { target: { value: 'Ada Lovelace '}});

  const message1 = app.getByText(/Hello, Ada Lovelace/i);
  expect(message1).toBeInTheDocument();

  const clearButton = app.getByText(/clear/i);

  fireEvent.click(clearButton);

  const message2 = app.getByText(/Hello, Anonymous/i);
  expect(message2).toBeInTheDocument();
})

6. Save the file.

7. Ensure that all test pass.

8. Quit Jest by pressing ‘q’ at the command prompt.

9. Close all.

Part 5 – Review

In this tutorial, we created unit test to ensure our business logic and our rendering is correct. This will allow up to evolve the application with the confidence that and side-effect introduced by changes in the application can be quickly detected.