How to deploy the PaNOSC Portal

local deployment

We need to install the following software:

• Helm + Minikube
• Keycloak
• portal microservices
• RemoteDesktop / Jupyter instances

https://github.com/panosc-portal/helm-charts/tree/master/panosc-portal-demo
https://confluence.panosc.eu/pages/viewpage.action?pageId=10879127

Please follow the instructions line by line!

helm

See https://helm.sh/docs/intro/install/

on Ubuntu:

curl https://baltocdn.com/helm/signing.asc | sudo apt-key add -
sudo apt-get install apt-transport-https --yes
echo "deb https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list
sudo apt-get update
sudo apt-get install helm

Minikube

See https://minikube.sigs.k8s.io/docs/start/

on Ubuntu:

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube_latest_amd64.deb
sudo dpkg -i minikube_latest_amd64.deb

kubectl

Install Kubernetes: https://kubernetes.io/docs/tasks/tools/install-kubectl/

on Ubuntu:

sudo apt-get update && sudo apt-get install -y apt-transport-https gnupg2 curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl

test Kubernetes

minikube start
kubectl get po -A
minikube dashboard   #(!!! web interface to minikube)
kubectl create deployment hello-minikube --image=k8s.gcr.io/echoserver:1.4
kubectl expose deployment hello-minikube --type=NodePort --port=8080
kubectl port-forward service/hello-minikube 7080:8080

then check http://localhost:7080/whateveritis

Clean up:

kubectl delete service hello-minikube
kubectl delete deployment hello-minikube

Check the output of the command:

kubectl cluster-info

The IP-address it provides will be later referred as Yourk8sMaster.

Just IP-address is easier to get with minicube ip

Keycloak

Install Keycloak See https://www.keycloak.org/docs/latest/server_installation/#installation

• distribution files (localhost installation)

---

https://www.keycloak.org/downloads #'keycloak-12.0.4.[zip|tar.gz]'

tar xvzf keycloak-12.0.4.tar.gz 
keycloak-12.0.4/bin$ ./standalone.sh  # check if it works
keycloak-12.0.4/bin$ ./add-user.sh    # add the admin user

A Keycloak user in the Management Realm (= admin) can be added either via the Keycloak web-interface (localhost 8080 port, see below) in the Administration Console (leftmost panel) or from the command line with add-user.sh, like so:

What type of user do you wish to add?
 a) Management User (mgmt-users.properties)
 b) Application User (application-users.properties)
(a):

Enter the details of the new user to add.
Using realm 'ManagementRealm' as discovered from the existing property files.
Username : panosc
Password recommendations are listed below. To modify these restrictions edit the add-user.properties configuration file.
 - The password should be different from the username
 - The password should not be one of the following restricted values {root, admin, administrator}
 - The password should contain at least 8 characters, 1 alphabetic character(s), 1 digit(s), 1 non-alphanumeric symbol(s)
Password :
WFLYDM0101: Password should have at least 1 digit.
Are you sure you want to use the password entered yes/no? yes
Re-enter Password :
What groups do you want this user to belong to? (Please enter a comma separated list, or leave blank for none)[  ]:
About to add user 'panosc' for realm 'ManagementRealm'
Is this correct yes/no? yes
Added user 'panosc' to file '/scratch/panosc-dev/keycloak-12.0.4/standalone/configuration/mgmt-users.properties'
Added user 'panosc' to file '/scratch/panosc-dev/keycloak-12.0.4/domain/configuration/mgmt-users.properties'
Added user 'panosc' with groups  to file '/scratch/panosc-dev/keycloak-12.0.4/standalone/configuration/mgmt-groups.properties'
Added user 'panosc' with groups  to file '/scratch/panosc-dev/keycloak-12.0.4/domain/configuration/mgmt-groups.properties'
Is this new user going to be used for one AS process to connect to another AS process?
e.g. for a slave host controller connecting to the master or for a Remoting connection for server to server EJB calls.
yes/no? no

Only after an admin user (named "panosc" here) has been created by either method, we login to the HAL management console (localhost, port 9990) with exactly these admin credentials and do the following steps with panosc realm etc.

(a) keycloak -
http://127.0.0.1:9990 # HAL Management Console
http://127.0.0.1:8080 # Keycloak (create new keycloak admin user+pass when first connected)

on Keycloak Admin console:

• New realm: panosc

In panosc realm:

• new user: <USERNAME>

Concretely:
Users -> Add user -> panosc_keycloak : panosc_pwd

user <USERNAME>:

• EmailVerification:off (to have immediately a fully enabled account)

• user credentials: set password

• temporary:off (to have immediately a fully enabled account)

• new client: <CLIENT_ID>

• new client scope: openid add openid as optional client scope to client <CLIENT_ID>

Clients -> Client Scopes -> Optional Client Scopes: button: Add selected

Concretely:
CLIENT_ID = panosc_portal

testing keycloak

https://www.appsdeveloperblog.com/keycloak-client-credentials-grant-example/

curl --location --request POST 'http://localhost:8080/auth/realms/panosc/protocol/openid-connect/token' --header 'Content-Type: application/x-www-form-urlencoded' --data-urlencode 'client_id=<CLIENT_ID>' --data-urlencode 'scope=openid' --data-urlencode 'grant_type=password' --data-urlencode 'username=<USERNAME>' --data-urlencode 'password=<USERPASSWORD>'

Concretely:

curl --location --request POST 'http://localhost:8080/auth/realms/panosc/protocol/openid-connect/token' --header 'Content-Type: application/x-www-form-urlencoded' --data-urlencode 'client_id=panosc_portal' --data-urlencode 'scope=openid' --data-urlencode 'grant_type=password' --data-urlencode 'username=panosc_keycloak' --data-urlencode 'password=panosc_pwd'

mapping to "external" port 8090

Install socat if necessary.

socat tcp-listen:8090,reuseaddr,fork tcp:localhost:8080 &
ifconfig        # check for <KEYCLOAK_EXTERNAL_IP>

curl --location --request POST 'http://<KEYCLOAK_EXTERNAL_IP>:8090/auth/realms/panosc/protocol/openid-connect/token' --header 'Content-Type: application/x-www-form-urlencoded' --data-urlencode 'client_id=<CLIENT_ID>' --data-urlencode 'scope=openid' --data-urlencode 'grant_type=password' --data-urlencode 'username=' --data-urlencode 'password='

<-YourLoginField-> or username is set here by default to 'preferred_username' (check in keycloak: Client Scopes > profile > Mappers > username > TokenClaimName) also by copying the id_token generated above to https://jwt.ms/#id_token=...

shall look similar to:

https://jwt.ms/#id_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IjdfWnVmMXR2a3dMeFlhSFMzcTZsVWpVWUlHdyIsImtpZCI6IjdfWnVmMXR2a3dMeFlhSFMzcTZsVWpVWUlHdyJ9.eyJhdWQiOiJiMTRhNzUwNS05NmU5LTQ5MjctOTFlOC0wNjAxZDBmYzljYWEiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC9mYTE1ZDY5Mi1lOWM3LTQ0NjAtYTc0My0yOWYyOTU2ZmQ0MjkvIiwiaWF0IjoxNTM2Mjc1MTI0LCJuYmYiOjE1MzYyNzUxMjQsImV4cCI6MTUzNjI3OTAyNCwiYWlvIjoiQVhRQWkvOElBQUFBcXhzdUIrUjREMnJGUXFPRVRPNFlkWGJMRDlrWjh4ZlhhZGVBTTBRMk5rTlQ1aXpmZzN1d2JXU1hodVNTajZVVDVoeTJENldxQXBCNWpLQTZaZ1o5ay9TVTI3dVY5Y2V0WGZMT3RwTnR0Z2s1RGNCdGsrTExzdHovSmcrZ1lSbXY5YlVVNFhscGhUYzZDODZKbWoxRkN3PT0iLCJhbXIiOlsicnNhIl0sImVtYWlsIjoiYWJlbGlAbWljcm9zb2Z0LmNvbSIsImZhbWlseV9uYW1lIjoiTGluY29sbiIsImdpdmVuX25hbWUiOiJBYmUiLCJpZHAiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC83MmY5ODhiZi04NmYxLTQxYWYtOTFhYi0yZDdjZDAxMWRiNDcvIiwiaXBhZGRyIjoiMTMxLjEwNy4yMjIuMjIiLCJuYW1lIjoiYWJlbGkiLCJub25jZSI6IjEyMzUyMyIsIm9pZCI6IjA1ODMzYjZiLWFhMWQtNDJkNC05ZWMwLTFiMmJiOTE5NDQzOCIsInJoIjoiSSIsInN1YiI6IjVfSjlyU3NzOC1qdnRfSWN1NnVlUk5MOHhYYjhMRjRGc2dfS29vQzJSSlEiLCJ0aWQiOiJmYTE1ZDY5Mi1lOWM3LTQ0NjAtYTc0My0yOWYyOTU2ZmQ0MjkiLCJ1bmlxdWVfbmFtZSI6IkFiZUxpQG1pY3Jvc29mdC5jb20iLCJ1dGkiOiJMeGVfNDZHcVRrT3BHU2ZUbG40RUFBIiwidmVyIjoiMS4wIn0=.UJQrCA6qn2bXq57qzGX_-D3HcPHqBMOKDPx4su1yKRLNErVD8xkxJLNLVRdASHqEcpyDctbdHccu6DPpkq5f0ibcaQFhejQNcABidJCTz0Bb2AbdUCTqAzdt9pdgQvMBnVH1xk3SCM6d4BbT4BkLLj10ZLasX7vRknaSjE_C5DI7Fg4WrZPwOhII1dB0HEZ_qpNaYXEiy-o94UJ94zCr07GgrqMsfYQqFR7kn-mn68AjvLcgwSfZvyR_yIK75S_K37vC3QryQ7cNoafDe9upql_6pB2ybMVlgWPs_DmbJ8g0om-sPlwyn74Cc1tW3ze-Xptw_2uVdPgWyqfuWAfq6Q

Install the portal components

clone:

git clone https://github.com/panosc-portal/api-service-client-cli
git clone https://github.com/panosc-portal/api-service
git clone https://github.com/panosc-portal/account-service-client-cli
git clone https://github.com/panosc-portal/cloud-service-client-cli
git clone https://github.com/panosc-portal/cloud-service
git clone https://github.com/panosc-portal/cloud-provider-client-cli
git clone https://github.com/panosc-portal/frontend
git clone https://github.com/panosc-portal/account-service
git clone https://github.com/panosc-portal/cloud-provider-kubernetes
git clone https://github.com/panosc-portal/simple-notebook-client
git clone https://github.com/panosc-portal/desktop-service-web-test-client
git clone https://github.com/panosc-portal/desktop-service
git clone https://github.com/panosc-portal/microservices-integration-test
git clone https://github.com/panosc-portal/helm-charts
git clone https://github.com/panosc-portal/remote-desktop-test-image
git clone https://github.com/panosc-portal/notebook-service-web-test-client
git clone https://github.com/panosc-portal/notebook-service

Install helm components

helm repo add panosc-portal https://panosc-portal.github.io/helm-charts/
helm repo update

kubectl create namespace panosc-portal  #<YourExistentNamespace>

The installation command template:

helm install <YourReleaseName> panosc-portal/panosc-portal-demo \
--set global.kubernetesMasterHostname=<Yourk8sMaster> \      
--set account-service.idp.url=<YourOpenIDDiscoveryEndpoint> \      #<KEYCLOAK_EXTERNAL_IP>
--set account-service.idp.clientId=<YourClientID> \
--set account-service.idp.loginField=<YourLoginField> \
-n <YourExistentNamespace>

eg:

helm install panosc-portal panosc-portal/panosc-portal-demo \
--set global.kubernetesMasterHostname=192.168.49.2 \
--set account-service.idp.url=http://131.169.212.94:8090/auth/realms/panosc/.well-known/openid-configuration \
--set account-service.idp.clientId=PanoscPortal \
--set account-service.idp.loginField=preferred_username \
-n panosc-portal

Now wait until the portal is up and running (check on minikube dashboard)

Test the portal

Install Node.js

See https://github.com/nodesource/distributions/blob/master/README.md

on Ubuntu / Debian

curl -sL https://deb.nodesource.com/setup_15.x | sudo -E bash -
sudo apt-get install -y nodejs

on RedHat

curl -fsSL https://rpm.nodesource.com/setup_15.x | bash -
# remove an version if any: yum remove nodejs -y
yum install nodejs -y

account-service-client-cli

Go to the api-service-client-cli repo and install Node components:

cd api-service-client-cli
npm install

create a config:

In the same api-service-client-cli folder create a file config.json with the following content

{
  "idp": {
    "url": "http://<KEYCLOAK_EXTERNAL_IP>:8090/auth/realms/panosc/protocol/openid-connect/token",
    "clientId": "<CLIENT_ID>"
  }
}

allow connecting external (local) client to kubernetes api service on the default port 3000 (check namespace!): kubectl port-forward -n <YourExistentNamespace> service/api-service 3000:3000

bin/run user-instance:list #provide <USERNAME> and <USERPASSWORD> if requested for obtaining a token
bin/run user-instance:add  #(select e.g plan jupyer_small and name it jupytersmall)