bbrundert.com

With the recent announcement of VMware and Bitnami joining forces, I wanted to revisit the kubeapps project on Enterprise PKS earlier today. I followed the community documentation but ran into some smaller issues (see my GitHub comments here) that were coming up in the MongoDB deployment initially.

UPDATE: At first I thought you needed to enable privileged containers in PKS but actually you don’t have to do that! There was a typo in my configuration which led to an unknown flag for the MongoDB deployment. I used the flag “mongodb.securityContext.enable=false” when deploying the Helm chart but it should have been “mongodb.securityContext.enabled=false”. Thanks to Andres from the Bitnami team for catching this! The instructions below have been updated!

Install Helm

Add the bitnami repo:

helm repo add bitnami https://charts.bitnami.com/bitnami

Add a “kubeapps” namespace to deploy into

kubectl create namespace kubeapps

Add a Service Account to Tiller

vi rbac-config-tiller.yaml
---
apiVersion: v1
 kind: ServiceAccount
 metadata:
   name: tiller
   namespace: kube-system
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRoleBinding
 metadata:
   name: tiller
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: cluster-admin
 subjects:
 kind: ServiceAccount
 name: tiller
 namespace: kube-system 
---
kubectl create -f rbac-config-tiller.yaml

Leverage newly created service account for Tiller:

helm init --service-account tiller

Create Service account for kubeapps-operator

kubectl create serviceaccount kubeapps-operator 

kubectl create clusterrolebinding kubeapps-operator \
--clusterrole=cluster-admin \
--serviceaccount=default:kubeapps-operator

kubectl get secret $(kubectl get serviceaccount kubeapps-operator -o jsonpath='{.secrets[].name}') -o jsonpath='{.data.token}' | base64 --decode

Copy the secret for use in the kubeapps dashboard later on.

Since NSX-T brings an out-of-the-box capability for exposing kubeapps to an external IP address, we can use LoadBalancer and skip the port-forwarding section of the documentation. Following what I found in another bug, I set some extra flags for disabling IPv6:

helm install --name kubeapps --namespace kubeapps bitnami/kubeapps \
--set frontend.service.type=LoadBalancer \
--set mongodb.securityContext.enabled=false \
--set mongodb.mongodbEnableIPv6=false

After a few minutes, the deployed services & deployments should be up and running:

Follow then part three of the instructions to access the dashboard.

I have installed VMware PKS in a variety of environments and I typically show something that helps Platform Operators running PKS to dive even deeper into the status of PKS components beyond the pks cli. One of the key lifecycle components in PKS is called BOSH. BOSH deploys Kubernetes masters and workers and performs a number of other tasks. So how do you get access to BOSH in the easiest way?

Step 1)

• Login to the Ops Manager VM via ssh: ssh ubuntu@your.opsmanager.com  

Step 2)

• Open Ops Manager and click on the BOSH Director tile: 
  
• Click on the “Credentials” Tab and search for “BOSH Commandline Credentials”:
  
• You will see an output similar like this one:

  {"credential":"BOSH_CLIENT=ops_manager 
  BOSH_CLIENT_SECRET=ABCDEFGhijklmnopQRSTUVWxyz 
  BOSH_CA_CERT=/var/tempest/workspaces/default/root_ca_certificate 
  BOSH_ENVIRONMENT=192.168.1.100 bosh "}

• Copy and paste that line and reformat it the following way:

  BOSH_CLIENT=ops_manager 
  BOSH_CLIENT_SECRET=ABCDEFGhijklmnopQRSTUVWxyz 
  BOSH_CA_CERT=/var/tempest/workspaces/default/root_ca_certificate 
  BOSH_ENVIRONMENT=192.168.1.100

   

• Easiest way to get started every time is to make it part of your .bashrc configuration by doing the following:
  • edit your .bashrc and append the outputs from above like this:

  • export BOSH_CLIENT=ops_manager 
    export BOSH_CLIENT_SECRET=ABCDEFGhijklmnopQRSTUVWxyz 
    export BOSH_CA_CERT=/var/tempest/workspaces/default/root_ca_certificate 
    export BOSH_ENVIRONMENT=192.168.1.100

  • logout and login again (or just run the export commands on the CLI manually once)

 

• Some example commands on how to interact with BOSH (and a nice cheat sheet at https://github.com/DennyZhang/cheatsheet-bosh-A4):

• bosh deployments
  PKS=$(bosh deployments | grep ^pivotal | awk '{print $1;}')
  bosh -d $PKS vms
  bosh -d $PKS instances
  bosh -d $PKS tasks
  bosh -d $PKS tasks -ar
  bosh -d $PKS task 724
  bosh -d $PKS task 724 --debug
  
  CLUSTER=$(bosh deployments | grep ^service-instance | awk '{print $1;}')
  bosh -d $CLUSTER vms
  bosh -d $CLUSTER vms --vitals
  bosh -d $CLUSTER tasks --recent=9
  bosh -d $CLUSTER task 2009 --debug
  bosh -d $CLUSTER ssh master/0
  bosh -d $CLUSTER ssh worker/0
  bosh -d $CLUSTER logs
  bosh -d $CLUSTER cloud-check

   

• Advanced users: you can also install the BOSH CLI on and admin VM and run from there:
  • Download from https://github.com/cloudfoundry/bosh-cli/releases
  • Copy the certificate from the Ops Manager VM (/var/tempest/workspaces/default/root_ca_certificate) to your admin VM and edit the .bashrc environment variables accordingly