How do Models interact with the Model Runtime?
When you deploy a model into Kodexa you include the model runtime that you want to use. Today, all Kodexa model runtimes have the same interface, but this may change in the future. There are two primary forms of interaction between the model runtime and the model. The first is inference, the second is training. How the model runtime calls your model is based on how you have declared your model in the model.yml file.Inference
The most common starting point with working with a model is learning how inference works. Let’s take a simple example of a model.yml:modelRuntimeRef
which is set to kodexa/base-model-runtime
. This means that the Model Assistant will use the base model runtime to run the model. In fact, it will look up the model runtime with the ref kodexa/base-model-runtime
. Then it will look at the model runtime to determine which action it uses for inference, and build a pipeline including that action. The platform will then schedule that pipeline and then model runtime action will be called.

Model
and then look for a function in that package called infer
.
The model runtime will pass the document that we are processing to the model and then the model will return a document. The model runtime will then pass the document back to the platform for further processing.
Inference with Options
In the previous example, we saw how the model runtime would pass the document to the model. In this example, we will see how the model runtime will pass options to the model. First, let’s add some inference options to our model.yml file:Overriding Entry Points
If you want to override the entry point for your model, you can do so by specifying themodelRuntimeParameters
property in the model.yml file.
Magic Parameters for Training and Inference
When you are training or inferring a model, you can pass in some magic parameters to the model runtime. If you include a parameter in either your train or infer function, it will be passed in by the model runtime.Parameter Name | Train/Infer | Description |
---|---|---|
model_store | Both | An instance of the ModelStoreEndpoint for the model you are using |
model_data | Both | A string representing the path where you in training you can store data and in inference you can pick it up |
pipeline_context | Both | The PipelineContext for the processing pipeline |
training_id | Both | The ID of the ModelTraining that is in use |
additional_training_document | Train | A Document representing the document being tested |
training_options | Training | A dictionary of the training options that have been set for the model |
Inference Options
While in training we pass all the training options as a dictionary, in inference we pass the options as individual parameters. This is because we want to make it easy to use the options in the inference code. Therefore, if you have an inference option calledmy_option
then you will get a parameter called my_option
passed to your inference function.