Service Operations
In this section you can find the common operational activities related to keep the service available to our users.
Initial set-up
Notebooks VO
The resources used for the Notebooks deployments are managed with the
vo.notebooks.egi.eu
VO. Operators of the service should join the VO, check the
entry at the
operations portal
and at AppDB.
Clients installation
In order to manage the resources you will need these tools installed on your client machine:
fedcloudclient
for discovering sites and managing tokens,terraform
to create the VMs at the providers,ansible
to configure the VMs and install kubernetes at the providers,terraform-inventory
to get the list of hosts to use from terraform.
Get the configuration repository
All the configuration of the notebooks is stored at a git repository available in GitHub. Start by cloning the repo:
Kubernetes
We use terraform
and ansible
to build the cluster at one of the EGI Cloud
providers. If you are building the cluster for the first time, create a new
directory on your local git repository from the template, add it to the repo,
and get terraform
ready:
Using the fedcloud
you can get set the right environment for interacting with
the OpenStack APIs of a given site:
Whenever you need to get a valid token for the site and VO, you can obtain it with:
First get the network IDs and pool to use for the site:
In this case we will use public00
as the pool for public IPs and
1aaf20b6-47a1-47ef-972e-7b36872f678f
as the network ID. Check with the
provider which is the right network to use. Use these values in the
terraform.tfvars
file:
You may want to check the right flavors for your VMs and adapt other variables
in terraform.tfvars
. To get a list of flavors you can use:
Finally, ensure your public ssh key is also listed in the cloud-init.yaml
file
and then you are ready to deploy the cluster with:
Your VMs are up and running, it's time to get kubernetes configured and running with Ansible.
The following Ansible role needs to be installed first:
and then:
Interacting with the cluster
As the master will be on a private IP, you won't be able to directly interact
with it, but you can still ssh into the VM using the ingress node as a gateway
host (you can get the different hosts with
TF_STATE=./terraform terraform-inventory --inventory
)
Modifying/Destroying the cluster
You should be able to change the number of workers in the cluster and re-apply terraform to start them and then execute the playbook to get them added to the cluster.
Any changes in the master, NFS or ingress VMs should be done carefully as those will probably break the configuration of the Kubernetes cluster and of any application running on top.
Destroying the cluster can be done with a single command:
Notebooks deployments
Once the k8s cluster is up and running, you can deploy a notebooks instance. For
each deployment you should create a file in the deployments
directory
following the template provided:
Each deployment will need a domain name pointing to your ingress host, you can create one at the FedCloud dynamic DNS service.
Then you will need to create an OpenID Connect client for EGI Check-in to
authorise users into the new deployment. You can create a client by going to the
EGI Federation Registry.
You can find more information about registering an OIDC Client in
EGI Check-in guide for SPs
Use the following as redirect URL:
https://<your host domain name>/hub/oauth_callback
.
Then add offline_access
to the list of scopes and submit the request. After
the approval of the Service request, save the client and take note of the
client ID and client secret for later.
Finally, you will also need 3 different random strings generated with
openssl rand -hex 32
that will be used as secrets in the file describing the
deployment.
Go and edit the deployment description file to add this information (search for
# FIXME NEEDS INPUT
in the file to quickly get there)
For deploying the notebooks instance we will also use ansible
:
The first deployment trial may fail due to a timeout caused by the downloading of the container images needed. You can retry after a while to re-deploy.
In the master you can check the status of your deployment (the name of the deployment will be the same as the name of your local deployment file):
Updating a deployment
Just edit the deployment description file and run ansible again. The helm will be upgraded at the cluster.