Integrating a model into the V2 API (CURRENT)¶

Important

V2 of the API (starting on release 1.0.0) is the default, supported version. It is backwards incompatible with the V1 version.

Note

The current version of the DEEPaaS API is V2. The first release supporting this API version was 1.0.0. Please do not be confused with the deepaas release (i.e. 0.5.2, 1.0.0) and the DEEPaaS API version (V1 or V2).

Defining what to load¶

The DEEPaaS API uses Python’s Setuptools entry points that are dynamically loaded to offer the model functionality through the API. This allows you to offer several models using a single DEEPaaS instance, by defining different entry points for the different models.

When the DEEPaaS API is spawned it will look for the deepaas.v2.model entrypoint namespace, loading and adding the names found into the API namespace. In order to define your entry points, your module should leverage setuptools and be ready to be installed in the system. Then, in order to define your entry points, you should add the following to your setup.cfg configuration file:

[entry_points]

deepaas.v2.model =
    my_model = package_name.module

This will define an entry point in the deepaas.v2.model namespace, called my_model. All the required functionality will be fetched from the package_name.module module. This means that module should provide the model-api as described below.

If you provide a class with the required functionality, the entry point will be defined as follows:

[entry_points]

deepaas.v2.model =
    my_model = package_name.module:Class

Again, this will define an entry point in the deepaas.v2.model namespace, called my_model. All the required functionality will be fetched from the package_name.module.Class class, meaning that an object of Class will be created and used as entry point. This also means that Class objects should provide the model-api as described below.

Entry point (model) API¶

Regardless on the way you implement your entry point (i.e. as a module or as an object), you should expose the following functions or methods:

Defining model metadata¶

Your model entry point must implement a get_medatata function that will return some basic metadata information about your model, as follows:

get_metadata(self)¶

Return metadata from the exposed model.

The metadata that is expected should follow the schema that is shown below. This basically means that you should return a dictionary with the following aspect:

{
    "author": "Author name",
    "description": "Model description",
    "license": "Model's license",
    "url": "URL for the model (e.g. GitHub repository)",
    "version": "Model version",
}

The only fields that are mandatory are ‘description’ and ‘name’.

The schema that we are following is the following:

{
    "id": =  fields.Str(required=True,
                        description='Model identifier'),
    "name": fields.Str(required=True,
                       description='Model name'),
    "description": fields.Str(required=True,
                              description='Model description'),
    "license": fields.Str(required=False,
                          description='Model license'),
    "author": fields.Str(required=False,
                         description='Model author'),
    "version": fields.Str(required=False,
                          description='Model version'),
    "url": fields.Str(required=False,
                      description='Model url'),
    "links": fields.List(
        fields.Nested(
            {
                "rel": fields.Str(required=True),
                "href": fields.Url(required=True),
            }
        )
    )
}

Returns: dictionary containing the model’s metadata.

Warming a model¶

You can initialize your model before any prediction or train is done by defining a warm function. This function receives no arguments and returns no result, but it will be call before the API is spawned.

You can use it to implement any loading or initialization that your model may use. This way, your model will be ready whenever a first prediction is done, reducint the waiting time.

warm(self)¶

Warm (initialize, load) the model.

This is called when the model is loaded, before the API is spawned.

If implemented, it should prepare the model for execution. This is useful for loading it into memory, perform any kind of preliminary checks, etc.

Training¶

Regarding training there are two functions to be defined. First of all, you can specify the training arguments to be defined (and published through the API) with the get_train_args function, as follows:

get_train_args(self)¶

Return the arguments that are needed to train the application.

This function should return a dictionary of webargs fields (check here for more information). For example:

from webargs import fields

(...)

def get_train_args():
    return {
        "arg1": fields.Str(
            required=False,  # force the user to define the value
            missing="foo",  # default value to use
            enum=["choice1", "choice2"],  # list of choices
            description="Argument one"  # help string
        ),
    }

Return dict: A dictionary of webargs fields containing the application required arguments.

Then, you must implement the training function (named train) that will receive the defined arguments as keyword arguments:

train(self, **kwargs)¶

Perform a training.

Parameters: kwargs – The keyword arguments that the predict method accepts must be defined by the get_train_args() method so the API is able to pass them down. Usually you would populate these with all the training hyper-parameters
Returns: TBD

Prediction and inference¶

For prediction, there are different functions to be implemented. First of all, as for the training, you can specify the prediction arguments to be defined, (and published through the API) with the get_predict_args as follows:

get_predict_args(self)¶

Return the arguments that are needed to perform a prediction.

This function should return a dictionary of webargs fields (check here for more information). For example:

from webargs import fields

(...)

def get_predict_args():
    return {
        "arg1": fields.Str(
            required=False,  # force the user to define the value
            missing="foo",  # default value to use
            enum=["choice1", "choice2"],  # list of choices
            description="Argument one"  # help string
        ),
    }

Return dict: A dictionary of webargs fields containing the application required arguments.

Do not forget to add an input argument to hold your data. If you want to upload files for inference to the API, you should use a webargs.fields.Field field created as follows:

def get_predict_args():
    return {
        "data": fields.Field(
            description="Data file to perform inference on.",
            required=False,
            missing=None,
            type="file",
            location="form")
     }

You can also predict data stored in an URL by using:

def get_predict_args():
    return {
        "url": fields.Url(
            description="Url of data to perform inference on.",
            required=False,
            missing=None)
     }

Important

do not forget to add the location="form" and type="file" to the argument definition, otherwise it will not work as expected.

Once defined, you will receive an object of the class described below for each of the file arguments you declare. You can open and read the file stored in the filename attribute.

class UploadedFile(name, filename, content_type, original_filename)¶

Class to hold uploaded field metadata when passed to model’s methods

name¶: Name of the argument where this file is being sent.

filename¶: Complete file path to the temporary file in the filesystem,

content_type¶: Content-type of the uploaded file

original_filename¶: Filename of the original file being uploaded.

Then you should define the predict function as indicated below. You will receive all the arguments that have been parsed as keyword arguments:

predict(self, **kwargs)¶

Prediction from incoming keyword arguments.

Parameters: kwargs – The keyword arguments that the predict method accepts must be defined by the get_predict_args() method so the API is able to pass them down.
Returns: The response must be a str or a dict.

By default, the return values from these two functions will be casted into a string, and will be returned in the following JSON response:

{
   "status": "OK",
   "predictions": "<model response as string>"
}

However, it is recommended that you specify a custom response schema. This way the API exposed will be richer and it will be easier for developers to build applications against your API, as they will be able to discover the response schemas from your endpoints.

In order to define a custom response, the response attribute is used:

Returning different content types¶

Sometimes it is useful to return something different than a JSON file. For such cases, you can define an additional argument accept defining the content types that you are able to return as follows:

def get_predict_args():
    return {
        'accept': fields.Str(description="Media type(s) that is/are acceptable for the response.",
                             missing='application/zip',
                             validate=validate.OneOf(['application/zip', 'image/png', 'application/json']))
     }

Find here a comprehensive list of possible content types. Then the predict function will have to return the raw bytes of a file according to the user selection. For example:

def predict(**args):
    # Run your prediction

    # Return file according to user selection
    if args['accept'] == 'image/png':
        return open(img_path, 'rb')

    elif args['accept'] == 'application/json':
        return {'some': 'json'}

    elif args['accept'] == 'application/zip':
        return open(zip_path, 'rb')

If you want to return several content types at the same time (let’s say a JSON and an image), the easiest way it to return a zip file with all the files.

Using classes¶

Apart from using a module, you can base your entrypoints on classes. If you want to do so, you may find useful to inhering from the deepaas.model.v2.base.BaseModel abstract class:

class BaseModel¶

Base class for all models to be used with DEEPaaS.

Note that it is not needed for DEEPaaS to inherit from this abstract base class in order to expose the model functionality, but the entrypoint that is configured should expose the same API.

abstract get_metadata()¶

Return metadata from the exposed model.

The metadata that is expected should follow the schema that is shown below. This basically means that you should return a dictionary with the following aspect:

{
    "author": "Author name",
    "description": "Model description",
    "license": "Model's license",
    "url": "URL for the model (e.g. GitHub repository)",
    "version": "Model version",
}

The only fields that are mandatory are ‘description’ and ‘name’.

The schema that we are following is the following:

{
    "id": =  fields.Str(required=True,
                        description='Model identifier'),
    "name": fields.Str(required=True,
                       description='Model name'),
    "description": fields.Str(required=True,
                              description='Model description'),
    "license": fields.Str(required=False,
                          description='Model license'),
    "author": fields.Str(required=False,
                         description='Model author'),
    "version": fields.Str(required=False,
                          description='Model version'),
    "url": fields.Str(required=False,
                      description='Model url'),
    "links": fields.List(
        fields.Nested(
            {
                "rel": fields.Str(required=True),
                "href": fields.Url(required=True),
            }
        )
    )
}

Returns: dictionary containing the model’s metadata.

abstract get_predict_args()¶

Return the arguments that are needed to perform a prediction.

This function should return a dictionary of webargs fields (check here for more information). For example:

from webargs import fields

(...)

def get_predict_args():
    return {
        "arg1": fields.Str(
            required=False,  # force the user to define the value
            missing="foo",  # default value to use
            enum=["choice1", "choice2"],  # list of choices
            description="Argument one"  # help string
        ),
    }

Return dict: A dictionary of webargs fields containing the application required arguments.

abstract get_train_args()¶

Return the arguments that are needed to train the application.

This function should return a dictionary of webargs fields (check here for more information). For example:

from webargs import fields

(...)

def get_train_args():
    return {
        "arg1": fields.Str(
            required=False,  # force the user to define the value
            missing="foo",  # default value to use
            enum=["choice1", "choice2"],  # list of choices
            description="Argument one"  # help string
        ),
    }

Return dict: A dictionary of webargs fields containing the application required arguments.

abstract predict(**kwargs)¶

Prediction from incoming keyword arguments.

Parameters: kwargs – The keyword arguments that the predict method accepts must be defined by the get_predict_args() method so the API is able to pass them down.
Returns: The response must be a str or a dict.

schema = None¶

Must contain a valid schema for the model’s predictions or None.

A valid schema is either a marshmallow.Schema subclass or a dictionary schema that can be converted into a schema.

In order to provide a consistent API specification we use this attribute to define the schema that all the prediction responses will follow, therefore: - If this attribute is set we will validate them against it. - If it is not set (i.e. schema = None), the model’s response will

be converted into a string and the response will have the following form:
{
    "status": "OK",
    "predictions": "<model response as string>"
}

As previously stated, there are two ways of defining an schema here. If our response have the following form:

{
    "status": "OK",
    "predictions": [
        {
            "label": "foo",
            "probability": 1.0,
        },
        {
            "label": "bar",
            "probability": 0.5,
        },
    ]
}

We should define or schema as schema as follows:

Using a schema dictionary. This is the most straightforward way. In order to do so, you must use the marshmallow Python module, as follows:

from marshmallow import fields

schema = {
    "status": fields.Str(
                description="Model predictions",
                required=True
    ),
    "predictions": fields.List(
        fields.Nested(
            {
                "label": fields.Str(required=True),
                "probability": fields.Float(required=True),
            },
        )
    )
}

Using a marshmallow.Schema subclass. Note that the schema must be the class that you have created, not an object:

import marshmallow
from marshmallow import fields

class Prediction(marshmallow.Schema):
    label = fields.Str(required=True)
    probability = fields.Float(required=True)

class Response(marshmallow.Schema):
    status = fields.Str(
        description="Model predictions",
        required=True
    )
    predictions = fields.List(fields.Nested(Prediction))

schema = Response

abstract train(**kwargs)¶

Perform a training.

Parameters: kwargs – The keyword arguments that the predict method accepts must be defined by the get_train_args() method so the API is able to pass them down. Usually you would populate these with all the training hyper-parameters
Returns: TBD

abstract warm()¶

Warm (initialize, load) the model.

This is called when the model is loaded, before the API is spawned.

If implemented, it should prepare the model for execution. This is useful for loading it into memory, perform any kind of preliminary checks, etc.

Warning

The API uses multiprocessing for handling tasks. Therefore if you use decorators around your methods, please follow best practices and use functools.wraps so that the methods are still pickable. Beware also of using global variables that might not be shared between processes.