Skip to main content

Model Registry

Model Registry​

Katonic incorporates a comprehensive model registry feature, empowering users to define and track the stages of their models effectively. With the Katonic Model Registry, teams can collaborate seamlessly, develop models at various lifecycle stages, and manage them in a centralized and organized manner.

The Katonic Model Registry comprises a centralized model store, APIs, and a user interface (UI) to facilitate collaborative management throughout the model's entire lifecycle. It offers essential functionalities such as model lineage, versioning, stage transitions (e.g., from staging to production), and annotations.

Users have full control over lifecycle stage changes, including experimentation, testing, and production, with the flexibility to choose between automatic and manual control. The Katonic Model Registry maintains a complete history of each model, ensuring governance and facilitating the management of change approvals by designated individuals.

Some features include:

  • Central repository: Register Katonic models to a centralized location.

  • Model versioning: Keep track of the version history of models. Model built for a specific task can have several versions.

  • Model stage: Model versions have stages to represent the cycle as a whole. Together with model versioning, older model versions can gradually become phased out while the newest versions are sent to staging first.

  • CI/CD workflow integration: CI-triggered automated build-and-test steps ensure that code changes merged into the repository are reliable. As part of the CD process, the code is then delivered quickly and seamlessly.For better control and governance, record stage transitions, request, review, and approve changes as part of CI/CD pipelines.

    With that, let’s look at how you can register your model in Katonic.

Concepts​

The Model Registry introduces a few concepts that describe and facilitate the full lifecycle of a Model.

  • Model: A model from an experiment or run can be logged with Katonic SDK. Once logged, this model can then be registered with the Model Registry.

  • Registered model: Once registered in the Katonic Model Registry, it has a unique name, version, stage, and more.

  • Model Version: When registered models are updated, it keep track of their versions automatically.

  • Stage: Over the course of the model’s lifecycle, a model evolves—from development to staging to production.

    One can transition a registered model to one of the stages:

    • Staging.
    • Production.
    • Archived.

  • Model stage transitions: New registration events or changes should be recorded as activities that automatically log users, changes, and additional metadata like comments.

  • Description: The user can annotate the model for the team.

  • Activities: Katonic records a registered model’s activities, providing a history of the model’s stages.

Pre-Requisite​

Before you can add a model to the Model Registry, you must log it using the <model_class>.<model_name> methods of the corresponding model type. Once a model has been logged, you can add, modify, update, transition, or delete model in the Model Registry through the UI or the API.

Model Registry Workflow​

1.From the Katonic Platform page, select your experiment in the Experiments section.

2.Once you click on any of your experiment, you will see the runs details on the right side.

Untitled

3.Now click on any of the runs you want to register, it will open up the page containing the model details.

Untitled

4.Click on the Register Model button. You will see a pop-up window; from there you can register the model into new version or create your register model.

Untitled

5.Create new version of the model, select the model's name from the drop-down menu. then click on the Register button.

Untitled

6.Once the model is registered, you can see the details about it.

Untitled

Using the Model Registry​

1.Navigate to the Model Registry page from the Katonic platform. here you can see all your registered models.

Untitled

2.Now click on your registered model, you can see all versions of your registered models and their stages.

Untitled

3.Click on any of the version, you can see version details regarding that model.

Untitled

4.On the version detail page, you can see model version details and the current stage of the model version. Click the Stage drop-down, to transition the model version to one of the other valid stages.

Untitled

5.You can further investigate the version, by clicking on the Source Run link, it will open up the run page, where you can find details like metrics,params, model location path etc.

SDK Workflow​

An alternative way to interact with Model Registry is using the Katonic SDK. In particular, you can register a model during an experiment run or after all your experiment runs.

Train and log Model to the Model Registry​

You can Train and Log your machine learning model, by initiating the model class with your training/test set and experiment name, after that you can call different models to train and log in model registry.

 from katonic.ml.client import set_exp
 from katonic.ml.classification import Classifier
 set_exp(exp_name='exp-name')
 clf = Classifier(X_train, X_test, y_train, y_test, exp_name)
 clf.LogisticRegression()

In the above code snippet, if a registered model with the name doesn’t exist, the method registers a new model and creates Version 1. If a registered model with the name exists, the method creates a new model version.

Fetching an Model from the Model Registry​

After you have registered a model, you can fetch that model using <model_class>.load_model(), or more generally, load_model().

  model = clf.load_model(registry_path)

where registry_path is the path for model in model registry.

example registry path: s3://models/11/328fcc895ec745178524c4d32fb0bca1/artifacts

Transitioning an Model’s Stage​

Over the course of the model’s lifecycle, a model evolves—from development to staging to production. You can transition a registered model to one of the stages: Staging, Production or Archived.

result = clf.stagging( run_id='91ebd975b96f40348523ce924976de0b', model_name='my-ne2-exp_21_gradient_boosting_classifier_tuned', stage="Production")

The accepted values for stage are: Staging|Archived|Production|None.

Listing and Searching MLflow Models​

You can fetch a list of all registered models in the registry with a simple method.

from katonic.ml.util import model_versions

model_versions(model_name='my-ne2-exp_21_gradient_boosting_classifier_tuned')
f runs done : ", len(df_runs))
df_runs

Output:

With hundreds of models, it can be cumbersome to persue the results returned from this call. A more efficient approach would be to search for a specific model name and list its version details using the model_versions() method and provide a filter string

such as "name='sk-learn-random-forest-reg-model'"

from katonic.ml.util import model_versions

model_versions(model_name='my-ne2-exp_21_gradient_boosting_classifier_tuned')
Output:
> name=my-ne2-exp_21_gradient_boosting_classifier_tuned; run_id=91ebd975b96f40348523ce924976de0b; version=1
> name=my-ne2-exp_21_gradient_boosting_classifier_tuned; run_id=91ebd975b96f40348523ce924976de0b; version=2
> name=my-ne2-exp_21_gradient_boosting_classifier_tuned; run_id=91ebd975b96f40348523ce924976de0b; version=3

Archiving a Model​

You can move models' versions out of a Production stage into an Archived stage. At a later point, if that archived model is not needed, you can delete it.

# Archive models version 3 from Production into Archived
result = clf.stagging(
 run_id='91ebd975b96f40348523ce924976de0b',
 model_name='my-ne2-exp_21_gradient_boosting_classifier_tuned',
 stage="Archived"
)

Deleting Models​

You can either delete specific versions of a registered model or you can delete a registered model and all its versions.

Note : Deleting registered models or model versions is irrevocable, so use it judiciously.

# Delete versions 1,2, and 3 of the model
from katonic.ml.util import model_versions, delete_model_version

delete_model_version(
 model_name='my-ne2-exp_21_gradient_boosting_classifier_tuned',
 ver_list=['1',’2’,’3’]
)
# Delete a registered model along with all its versions
client.delete_reg_model(name="my-ne2-exp_21_gradient_boosting_classifier_tuned")

While the above workflow API demonstrates interactions with the Model Registry, two exceptional cases require attention.

  • One is when you have existing ML models saved from training without the use of Katonic SDK. Serialized and persisted on disk in sklearn’s pickled format, you want to register this model with the Model Registry.

  • The second is when you use an ML framework without a built-in model flavour support, for instance, vaderSentiment, and want to register the model.

Registering a Custom Model​

Not everyone will start their model training with Katonic SDK. So, you may have some models trained before the use of SDK. Instead of retraining the models, all you want to do is register your saved models with the Model Registry. This code snippet creates a keras model, which we assume that you had created and saved in native pickle format.

Example:

## import necessary libraries
import tensorflow as tf
from keras import metrics
from katonic.ml.client import MLClient
from katonic.ml.miscellaneous import LogModel
import os

## define experiment and create LogModel instance 
experiment_name= "test_user_model_logging"
lm = LogModel(experiment_name)
exp_id = lm.id

## load the dataset
mnist = tf.keras.datasets.mnist
(X_train_full, y_train_full), (X_test, y_test) = mnist.load_data()

X_valid, X_train = X_train_full[:5000] / 255, X_train_full[5000:] / 255
y_valid, y_train = y_train_full[:5000], y_train_full[5000:]


X_test = X_test / 255

## define the model
LAYERS = [
         tf.keras.layers.Flatten(input_shape=(28, 28), name='inputLayer'),
         tf.keras.layers.Dense(300, activation='relu', name='hiddenLayer1'),
         tf.keras.layers.Dense(100, activation='relu', name='hiddenLayer2'),
         tf.keras.layers.Dense(10, activation='softmax', name='outputLayer')
]

loss_function = 'sparse_categorical_crossentropy'
optimizers = 'SGD'
metric = ['accuracy']

model_clf = tf.keras.models.Sequential(LAYERS)


model_clf.compile(loss=loss_function,
               optimizer=optimizers,
               metrics=metric)

## train the model
EPOCHS = 2
VALIDATION = (X_valid, y_valid)

history = model_clf.fit(X_train, y_train,
           epochs=EPOCHS,
           validation_data=VALIDATION,)

## Save the model in h5 format
model_clf.save('keras.h5')

## Load the model from h5format
like you can see, we can load the h5 format directly to evaluate and log model into registry
model_from_h5_format = tf.keras.models.load_model('keras.h5')

## log model into registry.
current_working_dir = os.getcwd()  
artifact_path = "mnist-keras-model"
model_mertics = {model_from_h5_format.metrics_names[0]:res[0],
model_from_h5_format.metrics_names[1]:res[1]}
lm.model_logging("mnistkeras","keras",model_clf,artifact_path,current_working_dir,model_mertics)

## Print the runs to see details of log model.
df_runs = lm.search_runs(exp_id)
print("Number of runs done : ", len(df_runs))
df_runs.head()