Katonic MLOps

Node pool requirements

The AKS cluster must have at least two node pools that produce worker nodes with the following specifications and distinct node labels, and it might include an optional GPU pool:

SR NO.	POOL	MIN-MAX	VM	LABELS	TAINTS
1	Platform	3-4	Standard_DS3_v2 or Standard_D8ads_v5	katonic.ai/node-pool=platform	katonic.ai/node-pool=platform:NoSchedule
2	Compute	1-10	Standard_D8s_v3 or Standard_D8ads_v5	katonic.ai/node-pool=compute
3	Deployment	1-10	Standard_D8s_v3 or Standard_D8ads_v5	katonc.ai/node-pool=deployment	katonic.ai/node-pool=deployment:NoSchedule
4	GPU (Optional)	0-5	Standard_NC6s_v3	katonic.ai/node-pool=gpu-{GPU-type}	nvidia.com/gpu=gpu-{GPU-type}:NoSchedule

Note: Instance Type and Region Considerations
The cost of virtual machines varies according to the chosen instance type and region. You are encouraged to select from the mentioned instance types based on your specific requirements and budgetary considerations.

Please note that the following regions are not supported for deployment:

• Brazil South
• Germany Central
• Germany Northeast
• Austria East
• Denmark East
• Italy South
• Italy Central
• East India
• New Zealand North
• New Zealand Southeast
• Central Indonesia
• East Indonesia

Note: For example we can use GPU type as v100, A30, A100

Note: When backup_enabled = True, then compute_nodes.min_count should be set to 2.

If you want Katonic to run with some components deployed as highly available ReplicaSets you must use 2 availability zones. All compute node pools you use must have corresponding ASGs in any AZ used by other node pools. Setting up an isolated node pool in one zone can cause volume affinity issues.

To run the node pools across multiple availability zones, you will need duplicate ASGs in each zone with the same configuration, including the same labels, to ensure pods are delivered to the zone where the required ephemeral volumes are available.

Additional ASGs with distinct katonic.ai/node-pool labels can be added to make other instance types available for Katonic executions.

The Katonic installer can set up all configurations of ASG and zones for the Katonic platform.

Azure Platform-Node Specifications

Platform nodes in platform Azure cloud deployments must fulfil the following hardware specification requirements according to the deployment type:

SR NO.	COMPONENT	SPECIFICATION
1	Node count	Min 3
2	Instance type	Standard_DS3_v2 or Standard_D8ads_v5
3	vCPUs	4
4	Memory	14 GB
5	Boot disk size	128 GB

Azure Compute-Node Specifications

The following instance types are required for compute nodes in Azure cloud deployments for the Katonic platform:

Choose the type that best fits your requirements. Azure Kubernetes Service (AKS) is also supported for application nodes, using the instance types listed below. The Katonic platform requires at least 1 minimum Compute node for the Katonic Data Science version. For specification details for each type, refer to the Azure documentation.

Note: Supported compute node configurations

• Standard_D8ads_v5 (default configuration)
• Standard_D8s_v3
• Standard_D16s_v3
• Standard_D32s_v3
• Standard_D48s_v3
• Standard_D64s_v3
• Boot Disk: 128GB

Azure Deployment-Node Specifications

The following instance types are required for deployment nodes in Azure cloud deployments for the Katonic platform:

Choose the type that best fits your requirements. Azure Kubernetes Service (AKS) is also supported for application nodes, using the instance types listed below. The Katonic platform requires at least 1 minimum deployment node for the teams version. For specification details for each type, refer to the Azure documentation.

Note: Supported deployment node configurations

• Standard_D8ads_v5 (default configuration)
• Standard_D8s_v3
• Standard_D16s_v3
• Standard_D32s_v3
• Standard_D48s_v3
• Standard_D64s_v3
• Boot Disk: 128GB

Azure GPU-Node Specifications

GPU nodes in platform Azure cloud deployments must use one of the following instance types:

Choose the type that best fits your requirements. Azure Kubernetes Service (AKS) is also supported for application nodes, using the instance types listed below. For specification details for each type, refer to the Azure documentation.

Note: Supported GPU node configurations

• NCv3-series (GPU optimized)
• Boot Disk: 512 GB

Additional node pools can be added with distinct katonic.ai/node-pool labels to make other instance types available for Katonic executions.

Prerequisites

To install and configure Katonic in your Azure account you must have:

1. Quay credentials from Katonic.

2. PEM encoded public key certificate for your domain and private key associated with the given certificate.

3. An Azure subscription with enough quota to create:

• At least 4 Standard_D8s_v3 or Standard_D8ads_v5 VMs.
• NC6s_v3 or similar SKU VMs, if you want to use GPU.

4. A Linux operating system (Ubuntu/Debian) based machine with the following Steps:

   a. Create a Resource Group in Azure

   az group create --name <RESOURCE_GROUP> \ --location <ZONE> 

   Note: You can get a list of all available locations by running the following command:

   az account list-locations

   You need to pass the name of the resource group later to the Katonic-installer.

   b. A Linux operating system (Ubuntu/Debian) based machine having 4GB RAM and 2vcpus. Skip this step if you already have the machine with the given specifications.

   Note: After the platform is deployed successfully, the VM can be deleted.

   c. Switch to the root user inside the machine.

   d. Azure CLI's Latest version 2.35.0+ specifically must be installed and logged in to your Azure account using the az login command, with a user that has a contributor role on the subscription.

   Note: To achieve this on Debian-based machines, follow the install Azure CLI v2.35+.

   e. If your Azure has tenants, use the following command to get your subscription ID.

   az account list --output table

Save this as later on you need to pass it to the Katonic-installer.

• The following tools should be installed:
  • Kubectl
  • Docker

To install Katonic Platform MLOps version follow the steps mentioned below:

1. Access the JumpHost and perform az login.

2. Log in to Quay with the credentials described in the requirements section above.

docker login quay.io

3. Retrieve the Katonic installer image from Quay.

docker pull quay.io/katonic/katonic-installer:v4.5

4. Create a directory.

mkdir katonic
cd katonic

5. Adding PEM Encoded Public Key Certificate and Private Key to Directory

Put the PEM encoded public key certificate (having extension.crt) for your domain and private key associated with the given certificate (having extension .key) inside the current directory (katonic).

6. The Katonic Installer can deploy the Katonic Platform MLOps version in two ways:

1. Creating Private AKS and deploying the Katonic Platform MLOps version.
2. Install Katonic Platform MLOps version on existing Private AKS Azure Kubernetes Service.

1. Creating AKS and deploying the Katonic MLOps Platform

Initialize the installer application to generate a template configuration file named katonic.yml.

docker run -it --rm --name generating-yaml -v $(pwd):/install quay.io/katonic/katonic-installer:v4.5 init azure katonic_mlops deploy_kubernetes private

Edit the configuration file with all necessary details about the target cluster, storage systems, and hosting domain. Read the following configuration reference:

SR NO.	PARAMETER	DESCRIPTION	VALUE
1	katonic_platform_version	It has the value by default regarding the Katonic Platform Version.	katonic_mlops
2	deploy_on	Cloud platform on which Katonic is to be deployed.	Azure
3	private_cluster	Set "True" when opting for private cluster	False
4	enable_exposing_genai_applications_to_internet	set "True" if opting for exposing genai applications to internet	False
5	public_domain_for_genai_applications	Public FQDN of domain for genai applications that will be exposed to the internet	(eg. public-chatbots.google.com)
6	internal_loadbalancer	Set "True" when opting for private ip for loadbalancer	False
7	create_k8s_cluster	Is set to false if the Kubernetes cluster is already present. If it is true, the installer will create Kubernetes cluster on provided cloud platform	True
8	kubernetes_version	AKS Version	eg. 1.28.5(1.27 and above versions supported)
9	cluster_name	Name of the cluster	eg. katonic-mlops-platform-v4-5
10	resource_group_name	Azure resource group name	eg. my-resource-group
11	resource_group_location	Azure resource group location	eg. centralindia
12	azure_subscription_id	Azure Subscription ID
13	vnet_name	name of vnet created for private cluster
14	aks_subnet_name	name of subnet created for private cluster
15	platform_nodes.instance_type	Platform node VM size	eg. Standard_D8ads_v5
16	platform_nodes.min_count	Minimum number of platform nodes should be 2 Note: You require a minimum of 3 platform nodes to install Superset or Airbyte	eg. 2
17	platform_nodes.max_count	Maximum number of platform nodes	eg. 4
18	platform_nodes.os_disk_size	Platform Nodes OS Disk Size	eg. 128 GB
19	compute_nodes.instance_type	Compute node VM size	eg. Standard_D8ads_v5
20	compute_nodes.min_count	Minimum number of compute nodes shoul be 1	eg. 1
21	compute_nodes.max_count	Maximum number of compute nodes	eg. 4
22	compute_nodes.os_disk_size	Compute Nodes OS Disk Size	eg. 128 GB
23	deployment_nodes.instance_type	Deployment Node VM size	eg. Standard_D8ads_v5
24	deployment_nodes.min_count	Minimum number of Deployment nodes should be 1	eg. 1
25	deployment_nodes.max_count	Maximum number of Deployment should be greater than Deployment nodes min count nodes.	eg. 4
26	deployment_nodes.os_disk_size	Deployment Nodes OS Disk Size	eg. 128 GB
27	gpu_enabled	Add GPU nodepool	True or False
28	gpu_nodes.instance_type	GPU node VM size	eg. Standard_NC6s_v3
29	gpu_nodes.gpu_type	Enter the type of gpu available on machine	eg v100, k80
30	gpu_nodes.min_count	Minimum number of GPU nodes	eg. 1
31	gpu_nodes.max_count	Maximum number of GPU nodes	eg. 2
32	gpu_nodes.os_disk_size	GPU Nodes OS Disk Size	eg. 512 GB
33	gpu_nodes.gpu_vRAM	Enter Gpu node RAM size
34	gpu_nodes.gpus_per_node	Enter GPU per node count
35	enable_gpu_workspace	Set it true if you want to use GPU Workspace	True or False
36	storage_class_type.Premium_LRS	If you prefer to select "Premium_LRS" as your storage class type instead of "StandardSSD_LRS," please write "True”.	True or False
37	shared_storage.create	Create shared storage	True or False
38	private_bucket_limit	Set the private bucket size.	eg. 10GB
39	minio_storage	Storage for minio	eg 250
40	workspace_timeout_interval	Set timeout interval hours	eg. 1
41	backup_enabled	Backup enabling	True or False
42	backup_schedule	Scheduling of backup	0 0 1 * *
43	backup_expiration	Expiration of backup	2160h0m0s
44	use_custom_domain	Set this to True if you want to host katonic platform on your custom domain. Skip if use_katonic_domain: True	True or False
45	custom_domain_name	Expected a valid domain.	eg. katonic.tesla.com
46	use_katonic_domain	Set this to True if you want to host katonic platform on Katonic MLOps Platform domain. Skip if use_custom_domain: True	True or False
47	katonic_domain_prefix	One word expected with no special characters and all small alphabets	eg. tesla
48	enable_pre_checks	Set this to True if you want to perform the Pre-checks	True / False
49	enable_accelerator	Set "True" to enable accelerators	False
50	enable_playground	Set "True" to enable playground	False
51	AD_Group_Management	Set "True" to enable functionality that provides you ability to sign in using Azure AD	False
52	AD_CLIENT_ID	Client ID of App registered for SSO in client's Azure or Identity Provider
53	AD_CLIENT_ID	Client Secret of App registered for SSO in client's Azure or any other Identity Provider
54	AD_AUTH_URL	Authorization URL endpoint of app registered for SSO.
55	AD_TOKEN_URL	Token URL endpoint of app registered for SSO.
56	quay_username	Username for quay
57	quay_password	Password for quay
58	adminUsername	Email for admin user	eg. john@katonic.ai
59	adminPassword	Password for admin user	at least 1 special character at least 1 upper case letter at least 1 lower case letter minimum 8 characters
60	adminFirstName	Admin first name	eg. john
61	adminLastName	Admin last name	eg. musk

Installing the Katonic Platform MLOps version

docker run -it --rm --name install-katonic -v /root/.azure:/root/.azure -v $(pwd):/inventory quay.io/katonic/katonic-installer:v4.5

This will start a container and deploy the entire platform.

2. Deploying Katonic Platform MLOps version on existing Private AKS

The steps are similar to Installing the Katonic Platform with Azure Kubernetes Service. Just edit the configuration file with all the necessary details about the target cluster, storage systems, and hosting domain. Read the following configuration reference, these are the only parameters required when installing the Katonic MLOps platform on existing AKS.

Prerequisites

You will need to create a kfs named storage class. Please refer to the main documentation of Azure → Dynamic Block Storage for instructions on how to create the storage class.

Initialize the installer application to generate a template configuration file named katonic.yml.

docker run -it --rm --name generating-yaml -v $(pwd):/install quay.io/katonic/katonic-installer:v4.5 init azure katonic_mlops kubernetes_already_exists private

SR NO.	PARAMETER	DESCRIPTION	VALUE
1	katonic_platform_version	It has the value by default regarding the Katonic Platform Version.	katonic_mlops
2	deploy_on	Katonic MLOps platform can be deployed on	Azure
3	cluster_name	Enter cluster name which you deploy	eg. katonic-mlops-platform-v4-5
4	private_cluster	Set "True" when opting for private cluster	False
5	enable_exposing_genai_applications_to_internet	set "True" if opting for exposing genai applications to internet	False
6	public_domain_for_genai_applications	Public FQDN of domain for genai applications that will be exposed to the internet	(eg. public-chatbots.google.com)
7	internal_loadbalancer	Set "True" when opting for private ip for loadbalancer	False
8	resource_group_name	Enter your cluster resource group name	eg. my-resource-group
9	resource_group_location	Enter your cluster resource group name location	eg. centralindia
10	azure_subscription_id	Azure Subscription ID
11	vnet_name	name of subnet created for private cluster
12	aks_subnet_name	aks_subnet_name
13	private_bucket_limit	Set the private bucket size.	eg. 10GB
14	minio_storage	Set the value to amount of storage required in file manager /16	eg. 20Gi
15	workspace_timeout_interval	Set timeout interval hours	eg. 1
16	backup_enabled	Backup enabling	True or False
17	backup_schedule	Scheduling of backup	0 0 1 * *
18	backup_expiration	Expiration of backup	2160h0m0s
19	use_custom_domain	Set this to True if you want to host Katonic platform on your custom domain. Skip if use_katonic_domain: True	True or False
20	custom_domain_name	Expected a valid domain.	eg. katonic.tesla.com
21	use_katonic_domain	Set this to True if you want to host Katonic platform on Katonic MLOps Platform domain. Skip if use_custom_domain: True	True or False
22	katonic_domain_prefix	One word expected with no special characters and all small alphabets	eg. tesla
23	enable_pre_checks	Set this to True if you want to perform the Pre-checks	True / False
24	enable_accelerator	Set "True" to enable accelerators	False
25	enable_playground	Set "True" to enable playground	False
26	AD_Group_Management	Set "True" to enable functionality that provides you ability to sign in using Azure AD	False
27	AD_CLIENT_ID	Client ID of App registered for SSO in client's Azure or Identity Provider
28	AD_CLIENT_SECRET	Client Secret of App registered for SSO in client's Azure or any other Identity Provider
29	AD_AUTH_URL	Authorization URL endpoint of app registered for SSO.
30	AD_TOKEN_URL	Token URL endpoint of app registered for SSO.
31	quay_username	Username for quay
32	quay_password	Password for quay
33	adminUsername	Email for admin user	eg. john@katonic.ai
34	adminPassword	Password for admin user	at least 1 special character at least 1 upper case letter at least 1 lower case letter minimum 8 characters
35	adminFirstName	Admin first name	eg. john
36	adminLastName	Admin last name	eg. musk

Installing the Katonic Platform MLOps version

docker run -it --rm --name install-katonic -v /root/.azure:/root/.azure -v $(pwd):/inventory quay.io/katonic/katonic-installer:v4.5

Installation Verification

The installation process can take up to 45 minutes to fully complete. The installer will output verbose logs, and commands to take kubectl access of deployed cluster and surface any errors it encounters. After installation, you can use the following commands to check whether all applications are in a running state or not.

cd /root/katonic
az aks get-credentials --resource-group $(cat /root/katonic/katonic.yml | grep resource_group_name | awk '{print $2}') --name $(cat /root/katonic/katonic.yml | grep cluster_name | awk '{print $2}')
kubectl get pods --all-namespaces

This will show the status of all pods being created by the installation process. If you see any pods enter a crash loop or hang in a non-ready state, you can get logs from that pod by running:

kubectl logs $POD_NAME --namespace $NAMESPACE_NAME

If the installation completes successfully, you should see a message that says:

TASK [platform-deployment : Credentials to access Katonic MLOps Platform] *******************************ok: [localhost] => {
    "msg": [
        "Platform Domain: $domain_name",
        "Username: $adminUsername",
        "Password: $adminPassword"
    ]
}

However, the application will only be accessible via HTTPS at that FQDN if you have configured DNS for the name to point to an ingress load balancer with the appropriate SSL certificate that forwards traffic to your platform nodes.

Test and troubleshoot

To verify the successful installation of Katonic, perform the following tests:

• If you encounter a 500 or 502 error, take access of your cluster and execute the following command:

  kubectl rollout restart deploy nodelog-deploy -n application

• If you have any file manager-related issues:

  kubectl rollout restart sts minio
  kubectl rollout status sts minio
  kubectl rollout restart deploy minio-console
  kubectl rollout status deploy minio-console

• Login to the Katonic application and ensure that all the navigation panel options are operational. If this test fails, please verify that Keycloak was set up properly.

• Create a new project and launch a Jupyter/JupyterLab workspace. If this test fails, please check that the default environment images have been loaded in the cluster.

• Publish an app with Flask or Shiny. If this test fails, please verify that the environment images have Flask and Shiny installed.

Deleting the Katonic Platform from Azure

To delete Katonic Platform from your Azure, you must delete its resource group.