Train a model remotely¶
This is a step by step guide on how to train a general model from the DEEP Open Catalog marketplace with your own dataset.
Requirements
Before being able to run your model at the Pilot Infrastructure you should first fulfill the following prerequisites:
- DEEP-IAM registration
For this example we are going to use DEEP-Nextcloud for storing data. This means you also have to:
- Register at DEEP-Nextcloud (i.e. login with your DEEP-IAM credentials)
- Obtain Nextcloud security parameters for rclone. This will provide you with
<your_nextcloud_username>
and<your_nextcloud_password>
.
1. Choose a model¶
The first step is to choose a model from the DEEP Open Catalog marketplace. For educational purposes we are going to use a general model to identify images. Some of the model dependent details can change if using another model, but this tutorial will provide a general overview of the workflow to follow when using any of the models in the Marketplace. A demo showing the different steps in this HowTo has also be recorded and you can find it our YouTube channel.
2. Upload your files to Nextcloud¶
We login to DEEP-Nextcloud with DEEP-IAM credentials and upload there the files needed for training. We create the folders that you need in order to store the inputs needed for the training and to retrieve the output. These folders will be visible from within the container. In this example we just need two folders (models
and data
):
A folder called models where the training weights will be stored after the training
A folder called data that contains two different folders:
- The folder images containing the input images needed for the training
- The folder dataset_files containing a couple of scripts: train.txt indicating the relative path to the training images and classes.txt indicating which are the categories for the training
The folder structure and their content will of course depend on the model to be used. This structure is just an example in order to complete the workflow for this tutorial.
3. Deploy with the Training Dashboard¶
- Go to the Training Dashboard
- Login with DEEP-IAM credentials
- Find the Train an image classifier module, click Train module
- Fill the webform, e.g. provide <your_nextcloud_username> for
rclone_user
and <your_nextcloud_password> forrclone_password
(see Nextcloud configuration for rclone) - Click Submit
- Look for Access and choose DEEPaaS, you will be redirected to
http://deepaas_endpoint
If you prefer deploying using a more advanced deploying via the command-line-interface check the deploy with CLI guide.
See the Dashboard guide for more details.
4. Go to the API, train the model¶
Now comes the fun!
Go to http://deepaas_endpoint/ui
and look for the train
POST method. Modify the training parameters you wish to
change and execute. If some kind of monitorization tool is available for the module, you will be able to follow the training
progress at http://monitor_endpoint
.
In the Dashboard you will be able to view the training history of that deployment:
5. Testing the training¶
Once the training has finished, you can directly test it by clicking on the predict
POST method. There you can either upload the image your want to classify or give an URL to it.