Containerized Scan Engine

To quickly and easily deploy updates, create the persistent volume (PV) and persistent volume claim (PVC) by using the volume ID and the following configs:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: ebs-pv
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce # meaning this PV can only be claimed once in read-write mode by one Pod
  storageClassName: ""
  persistentVolumeReclaimPolicy: Retain # when PVC is deleted, the PV will be kept around in protection mode meaning no other PVC can claim it, and data in the volume is persisted
  csi:
    driver: <driver in this example it is ebs.csi.aws.com>
    volumeHandle: <reference to volume vol-08a88cac24536707e>

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ebs-claim
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: ""
  resources:
    requests:
      storage: 5Gi

kubectl apply -f ./pvc.yaml
kubectl apply -f ./pv.yaml

Create the following deployment with the node that can run the Containerized Scan Engine:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: scan-engine
spec:
  selector:
    matchLabels:
      app: scan-engine
  template:
    metadata:
      labels:
        app: scan-engine
    spec:
      containers:
        - name: scan-engine
          image: rapid7/insightvm_scan_engine:latest
          volumeMounts:
          - name: persistent-storage
            mountPath: /opt/rapid7/nexpose/nse/conf
            subPath: engine1/conf
          - name: persistent-storage
            mountPath: /opt/rapid7/nexpose/nse/keystores
            subPath: engine1/keystores
          env:
          - name: CONNECT_TO_ADDRESS
            value: <ip-address>
          - name: CONNECT_TO_SHARED_SECRET
            value: <shared secret>
          ports:
          - containerPort: 40814
      volumes:
      - name: persistent-storage
        persistentVolumeClaim:
          claimName: ebs-claim

kubectl apply -f ./scan-engine-deployment.yaml

This exposes the service to accept connections from consoles:

apiVersion: v1
kind: Service
metadata:
  name: scan-engine
  labels:
    app: scan-engine
spec:
  ports:
    - port: 40814
      targetPort: 40814
      protocol: TCP
  selector:
    app: scan-engine

kubectl apply -f ./scan-engine-service.yaml

Manually update the Scan Engine:

kubectl rollout restart deployment/scan-engine

The three major components when creating a Statefulset are:

• scan-engine-group-srv - a headless service that’s used for the network identity of the Pods.
• scan-engine-group-srv - this StatefulSet configures three replicas that will be launched in unique pods .
• VolumeClaimTemplates - for each VolumeClaimTemplate entry defined in a StatefulSet a pod receives one PersistentVolumeClaim, and a PersistentVolume and storage will be dynamically provisioned.

Here is an example of creating a StatefulSet using YAML:

apiVersion: v1
kind: Service
metadata:
  name: scan-engine-group-srv
  labels:
    app: scan-engine-group-srv
spec:
  ports:
  - port: 80
    name: scan-engine-group-web
  selector:
    app: scan-engine-group
 
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: scan-engine-group
spec:
  selector:
    matchLabels:
      app: scan-engine-group # has to match .spec.template.metadata.labels
  serviceName: "scan-engine-group"
  replicas: 3
  template:
    metadata:
      labels:
        app: scan-engine-group # has to match .spec.selector.matchLabels
    spec:
      containers:
      - name: scan-engine-container
        image: rapid7/insightvm_scan_engine:latest
        ports:
        - containerPort: 40814
        env:
          - name: CONNECT_TO_SHARED_SECRET
            value: <CONNECT_TO_SHARED_SECRET>
          - name: CONNECT_TO_ADDRESS
            value: <CONNECT_TO_ADDRESS>
        volumeMounts:
        - name: ajing-cluster-volume
          mountPath: /opt/rapid7/nexpose/nse/conf
          subPath: scan-engine-group/conf
        - name: ajing-cluster-volume
          mountPath: /opt/rapid7/nexpose/nse/keystores
          subPath: scan-engine-group/keystores
  volumeClaimTemplates:
  - metadata:
      name: ajing-cluster-volume
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "standard"
      resources:
        requests:
          storage: 1Gi

After deploying the templates for all three components, three pods will be created named scan-engine-group-0, scan-engine-group-1, scan-engine-group-2.

kubectl apply -f scan-engine-statefulset.yaml

kubectl get pods NAME                  READY   STATUS    RESTARTS   AGE
scan-engine-group-0   1/1     Running   0          6m8s
scan-engine-group-1   1/1     Running   0          5m38s
scan-engine-group-2   1/1     Running   0          3m48s

Each pod receives a PersistentVolume with a StorageClass standard and 1GB of provisioned storage. When a pod is deployed, VolumeMounts mount the PersistentVolume associated with its PersistentVolumeClaim.

Verify that the PVCs and PV are dynamically provisioned:

kubectl get pvc
NAME                                       STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
ajing-cluster-volume-scan-engine-group-0   Bound    pvc-44085d05-bea9-48f2-9753-0c2480db8867   1Gi        RWO            standard       3d1h
ajing-cluster-volume-scan-engine-group-1   Bound    pvc-44eb177c-3f50-4dfd-b8a6-c35823576c23   1Gi        RWO            standard       3d1h
ajing-cluster-volume-scan-engine-group-2   Bound    pvc-29bcabbb-437d-4a55-905c-a53093a59cc9   1Gi        RWO            standard       3d1h

kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                              STORAGECLASS   REASON   AGE
pvc-29bcabbb-437d-4a55-905c-a53093a59cc9   1Gi        RWO            Delete           Bound    default/ajing-cluster-volume-scan-engine-group-2   standard                3d1h
pvc-44085d05-bea9-48f2-9753-0c2480db8867   1Gi        RWO            Delete           Bound    default/ajing-cluster-volume-scan-engine-group-0   standard                3d1h
pvc-44eb177c-3f50-4dfd-b8a6-c35823576c23   1Gi        RWO            Delete           Bound    default/ajing-cluster-volume-scan-engine-group-1   standard                3d1h

Scale the number of pods using the kubectl scale command:

kubectl scale statefulsets scan-engine-group --replicas=<replica_count>

Alternatively, you can update the number of replicas  in the StatefulSet yaml and reapply.

This is the entry point for your cluster. You cannot create a cluster without this. Follow the AWS guide outlining how to create the IAM role. However, be sure to modify the permissions to include the same EKS permissions of the Nexpose target cluster. This ensures that the cluster has access to the AWS resources that it interacts with.

You can create the cluster either on the AWS Console or the CLI. Refer to the AWS Guide  for instructions. Ensure that the role has the ingress and egress roles required for asset access, otherwise you may not be able to scan everything you want to.

The nodegroup needs access permissions for the control plane to interact with it along with AWS. This is the simplest way to scale nodes for the cluster. It is recommended that you use the built-in AWS fargate to create the nodes. Refer to the AWS Guide  for instructions. When creating nodes, be sure that they meet the minimum requirements to run the Containerized Scan Engine. Note that resources are shared between containers in the pod on nodes, so ensure that you scale the node requirements accordingly.

Volume needs to be allocated prior to mounting. Mount points require a PersistentVolumeClaim and a PersistentVolume. These abstraction layers allow for flexible mounting. Refer to the AWS Guide  for more on EKS persistent storage. It is recommended that you use the CSI  standard to retain flexibility. In the AWS examples EBS volumes are used to retain state. In order for the cluster to communicate with Amazon EBS, the EBS CSI driver  is required.

Before you begin:

1. Create your Google Cloud project.

2. Ensure billing is enabled.

3. Enable the Artifact Registry and Google Kubernetes Engine APIs.

4. Create a VPC Network that will be used with the cluster you create.

The following solution uses the Cloud Shell provided on the Google Cloud platform. It is a shell environment that comes preinstalled with the Google Cloud cli as well as the kubectl command line. For more information about the Cloud Shell on Google Cloud.

1. Navigate to the Google Cloud console.
2. Click the Activate Cloud Shell button.
3. Before beginning the set-up of Kubernetes, ensure that you are in the correct project by running this command in the Cloud Shell:

gcloud config set project <PROJECT_ID>

When creating a cluster in GKE there are one of two modes of operation and main cluster configuration choices - Standard and Autopilot. The method selected cannot be changed after the cluster is created. In order to make an informed decision it is necessary to understand the level of flexibility, responsibility and control that is required for the cluster. Refer to the official documentation on Cluster Configuration  to make an informed decision when creating clusters.

Autopilot

GKE Autopilot is a mode of operation that ensures management of the cluster configuration is completed by Google. These configurations include nodes, scaling, security, and more.

Create an auto cluster:

Copy

gcloud container clusters create-auto <cluster-name> --location=<region> --network=<network-name>

If the network is excluded from the command it will be set to default instead.

Make authentications interact with the cluster

Copy

gcloud container clusters get-credentials <cluster-name> --location <region>

Standard

Creating the cluster using the standard method allows for the choice of either a regional or zonal cluster. For more information, refer to Google’s About Regional Clusters  and Creating Zonal Clusters  documentation. For specifics on the requirements needed for the cluster to deploy the Containerized Scan Engine successfully, refer to the base requirements .

1. Create the cluster using GCP’s Create Kubernetes Engine.
2. After the cluster is created, navigate to the Cloud Shell.
3. Make authentications interact with the cluster:

Copy

gcloud container clusters get-credentials <cluster-name> --location <region>

The Containerized Scan Engine is running as root as this is required in order to perform some checks. The Containerized Scan Engine is able to run without root but this will skip some checks. For steps on how to mitigate risk, refer to Running in Rootless Mode  and Linux Post-Installation Steps for Docker Engine  which have been outlined by Docker.

To connect the Containerized Scan Engine to the Security Console on a non-default port, use the CONNECT_TO_PORT environment variable when starting up the container. This port must be exposed as it replaces the existing listening port number in the nsc.xml file.

For example, run this Docker command while mapping the TCP port:

docker run --env CONNECT_TO_ADDRESS=127.0.0.1 CONNECT_TO_PORT=00000 --env CONNECT_TO_SHARED_SECRET=<shared-secret> rapid7/insightvm_scan_engine:latest

Refer to Container Networking  for more information.

1. The Docker networks using docker network and ls.
2. Inspect the bridge network using docker inspect bridge.
3. Connect with the address listed under Gateway.

Log levels can be changed using the docker exec command or using mount points.

Configure log levels using docker exec

This approach allows you to change log levels by opening the Containerized Scan Engine and manually changing the logging.xml file.

1. Run the container.
   • docker run --env CONNECT_TO_ADDRESS=<address> --env CONNECT_TO_SHARED_SECRET=<shared_secret> rapid7/insightvm_scan_engine:latest
2. Navigate to a new terminal window and copy the container ID.
   • docker ps
3. In another terminal window run these commands in order to run vi editor inside the Docker container:
   • docker exec -it <container_id> sh -c "apt-get update && apt-get install -y vim"
   • docker exec -it <container_id> sh -c "apt list --installed | grep vim"
4. Run the following docker exec command to open the Docker container’s interactive shell:
   • docker exec -it <container_id> sh
5. Navigate to the conf directory in the container.
   • cd /opt/rapid7/nexpose/nse/conf
6. Change the log level in the logging.xml file. The container logs will now output the logs at the level you want.
   • vim logging.xml
7. To exit the interactive shell without interrupting the container’s process, enter Ctr + P > Ctr + Q.