Private Data Fabric Helm Charts

Table of Contents

• Overview
• Prerequisites
• Installation Guide
  • 1. Install Cert-Manager
  • 2. Create a PriorityClass
  • 3. Add Registry Certificate on Nodes
  • 4. Install Tenant Operator
  • 5. Generate External Secrets
  • 6. Apply External Secrets and Additional Configurations
  • 7. Deploy the Tenant
  • 8. Update Role Bindings
• Running Spark Applications
  • Option 1: Using Ticket Authentication
  • Option 2: Using JWT Token Authentication
• Troubleshooting
• FAQ

Overview

This repository contains Helm charts for deploying and managing the Private Data Fabric Tenant Operator on Kubernetes clusters. The Tenant Operator enables the creation and management of tenant environments within your Kubernetes infrastructure.

Prerequisites

Before beginning the installation, ensure you have the following:

• Kubernetes Cluster: A running Kubernetes cluster with sufficient permissions
• kubectl: Installed and configured to interact with your Kubernetes cluster
• Helm: Version 3.x or later installed
• Container Registry Access: Access to the container registry containing the required images
• Cluster Admin Privileges: Required for creating cluster-wide resources

Required Images

All necessary container images can be found in the MapR Technologies Docker Hub repository.

Installation Guide

Follow these steps to install and configure the Tenant Operator and related components.

1. Install Cert-Manager

Cert-Manager is required for managing certificates within the cluster.

# Apply Cert-Manager CRDs
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.8.0/cert-manager.crds.yaml

# Add the Jetstack Helm Repository and update
helm repo add jetstack https://charts.jetstack.io
helm repo update

# Install Cert-Manager in the 'cert-manager' namespace
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.8.0

2. Create a PriorityClass

Create a PriorityClass to ensure critical pods are scheduled with higher priority.

# Save the following YAML as 'hpe-critical-priorityclass.yaml'
cat <<EOF > hpe-critical-priorityclass.yaml
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: hpe-critical
value: 1000000
globalDefault: false
description: "Priority class for critical HPE pods"
EOF

# Apply the PriorityClass
kubectl apply -f hpe-critical-priorityclass.yaml

3. Add Registry Certificate on Nodes

Install the registry certificate on all cluster nodes according to your internal security procedures.

Note: The exact steps for this will vary depending on your specific environment. Refer to your internal documentation for guidance.

4. Install Tenant Operator

Install the Tenant Operator using Helm:

helm install tenant-operator tenant-operator-chart/ \
  -n hpe-df-tenant \
  --create-namespace \
  -f tenant-operator-chart/values.yaml

You can customize the installation by modifying the values.yaml file or providing override values with the --set flag.

5. Generate External Secrets

Run the external secret generation script on a cluster node:

# Execute the script from the repository root
./gen-external-secrets.sh

This script generates the necessary external secrets for cluster communication.

6. Apply External Secrets and Additional Configurations

a. Apply External Secrets

# Create the namespace for external cluster information
kubectl create namespace hpe-externalclusterinfo

# Apply the generated external secrets
kubectl apply -f /tmp/mapr-external-secrets.yaml

b. Create the hpe-secure Namespace and LDAP ConfigMap

# Create the hpe-secure namespace
kubectl create namespace hpe-secure

# Create LDAP ConfigMap (replace with your actual LDAP configuration)
cat <<EOF > ldapclient-cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: ldapclient-cm
  namespace: hpe-secure
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: "{}"
data:
  ldap.conf: |
    BASE http://example.com
    URI test
    TLS_CACERTDIR /etc/openldap/certs
    TLS_REQCERT allow
    SASL_NOCANON on
EOF

# Apply the ConfigMap
kubectl apply -f ldapclient-cm.yaml

c. Create an ImagePull Secret

Create a secret for pulling container images in the tenant namespace:

# Replace <your-base64-encoded-docker-config> with your actual encoded docker config
cat <<EOF > imagepull-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: imagepull
  namespace: sampletenant
  labels:
    hpe.com/cluster: none
    hpe.com/component: imagepull
    hpe.com/namespacetype: Tenant
    hpe.com/tenant: sampletenant
    hpe.com/version: 7.0.0
data:
  .dockerconfigjson: "<your-base64-encoded-docker-config>"
type: kubernetes.io/dockerconfigjson
EOF

# Apply the secret
kubectl apply -f imagepull-secret.yaml

To generate the base64-encoded docker config:

# Login to your Docker registry
docker login <your-registry>

# Encode the Docker config file
cat ~/.docker/config.json | base64 -w 0

d. Create a Cluster Role for Persistent Volume Operations

# Create the ClusterRole for PV operations
cat <<EOF > hpe-pvcreate-clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: hpe-pvcreate
rules:
- apiGroups: [""]
  resources: ["persistentvolumes"]
  verbs: ["create", "delete", "get", "list", "watch"]
EOF

# Apply the ClusterRole
kubectl apply -f hpe-pvcreate-clusterrole.yaml

7. Deploy the Tenant

Deploy a tenant using the provided example configuration:

kubectl apply -f tenant-crs/external-full.yaml

You should customize the tenant configuration according to your requirements before applying it.

8. Update Role Bindings

Modify the tenant roles to include necessary permissions:

# Example: Add PV permissions to the tenant role
kubectl edit role hpe-sampletenant-role -n sampletenant

Add the following rules to the role:

- apiGroups: [""]
  resources: ["persistentvolumes"]
  verbs: ["create", "delete", "get", "list", "watch"]
- apiGroups: [""]
  resources: ["secrets", "pods", "configmaps"]
  verbs: ["create", "delete", "get", "list", "watch", "update", "patch"]

Running Spark Applications

You can run Spark applications using one of two authentication methods:

Option 1: Using Ticket Authentication

a. Run the Ticket Creator Script

Navigate to the tenant CLI pod and execute the bundled script:

./ticketcreator.sh

This script generates a Kubernetes secret containing the authentication ticket.

b. Launch Spark Shell with Ticket Authentication

/opt/mapr/spark/spark-3.5.1/bin/spark-shell \
  --master k8s://https://<kubernetes-controller-host>:6443 \
  --conf spark.executor.instances=2 \
  --conf spark.mapr.user.secret=<generated-secret-name> \
  --conf spark.kubernetes.container.image=<spark-image> \
  --conf spark.kubernetes.namespace=<tenant-namespace> \
  --conf spark.kubernetes.container.image.pullPolicy=Always \
  --conf spark.mapr.cluster.configMap=cluster-cm \
  --conf spark.authenticate=false \
  --conf spark.authenticate.enableSaslEncryption=false \
  --conf spark.kubernetes.authenticate.driver.serviceAccountName=hpe-<tenant-namespace>

Replace the following placeholders:

• <kubernetes-controller-host>: The hostname or IP address of your Kubernetes controller
• <generated-secret-name>: The name of the secret generated by the ticketcreator.sh script
• <spark-image>: The full name of the Spark container image (e.g., docker.io/maprtech/spark:3.5.1)
• <tenant-namespace>: The namespace where your tenant is deployed (e.g., sampletenant)

Option 2: Using JWT Token Authentication

a. Generate JWT Token

Use the following curl command to generate your JWT token:

curl -k -d 'client_id=edf-client' \
  -d 'client_secret=<your-client-secret>' \
  -d 'username=<your-username>' \
  -d 'password=<your-password>' \
  -d 'grant_type=password' \
  'https://<sso-node>:6443/realms/master/protocol/openid-connect/token' | \
  jq

Replace the placeholder values with your actual authentication information.

b. Create a Secret with JWT Tokens

# Extract tokens from the JWT response
ACCESS_TOKEN=$(echo $JWT_RESPONSE | jq -r '.access_token')
REFRESH_TOKEN=$(echo $JWT_RESPONSE | jq -r '.refresh_token')

# Create a secret with the tokens
cat <<EOF > jwt-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: <token-secret-name>
  namespace: <tenant-namespace>
type: Opaque
data:
  CLUSTER_NAME: $(echo -n "<your-cluster-name>" | base64 -w 0)
  JWT_REFRESH_TOKEN: $(echo -n "$REFRESH_TOKEN" | base64 -w 0)
  JWT_TOKEN: $(echo -n "$ACCESS_TOKEN" | base64 -w 0)
EOF

# Apply the secret
kubectl apply -f jwt-secret.yaml

c. Launch Spark Shell with JWT Authentication

/opt/mapr/spark/spark-3.5.1/bin/spark-shell \
  --master k8s://https://<kubernetes-controller-host>:6443 \
  --conf spark.executor.instances=2 \
  --conf spark.mapr.jwt.secret=<token-secret-name> \
  --conf spark.kubernetes.container.image=<spark-image> \
  --conf spark.kubernetes.namespace=<tenant-namespace> \
  --conf spark.kubernetes.container.image.pullPolicy=Always \
  --conf spark.mapr.cluster.configMap=cluster-cm \
  --conf spark.authenticate=false \
  --conf spark.authenticate.enableSaslEncryption=false \
  --conf spark.kubernetes.authenticate.driver.serviceAccountName=<tenant-namespace>

Replace the following placeholders:

• <kubernetes-controller-host>: The hostname or IP address of your Kubernetes controller
• <token-secret-name>: The name of the secret containing your JWT tokens
• <spark-image>: The full name of the Spark container image
• <tenant-namespace>: The namespace where your tenant is deployed