Architecture overview
There are several different components in the DEEP-HybridDataCloud project that are relevant for the users. Later on you will see how each different type of user can take advantage of the different components.
The Marketplace
The DEEP Marketplace is a catalogue of modules that every user can have access to. Modules can be:
Trainable: Those are modules that an user can train on their own data to create a new service. Like training an image classifier with a plants dataset to create a plant classifier service. Look for the
trainable
tag in the marketplace to find those modules.Trained (inference-ready): Those are modules that have been pre-trained for a specific task (like the plant classifier mentioned earlier). Look for the
inference
andpre-trained
tags in the marketplace to find those modules.
Some modules can both be trainable and trained. For example the image classifier can be trained to create other image classifiers but can also be deployed for inference as it comes pre-trained with a general-purpose image classifier.
The DEEPaaS API
The DEEPaaS API is a key component for making the modules accessible to everybody (including non-experts), as it provides a consistent and easy to use way to access the model’s functionality. It is available for both inference and training.
Advanced users that want to create new modules can make them compatible with the API to make them available to the whole community. This can be easily done, since it only requires minor changes in user’s code.
The data storage resources
Storage is essential for user that want to create new services by training modules on their custom data. For the moment we support hosting data in DEEP-Nextcloud (up to 2 Terabytes by default), as well as integration with popular cloud storage options like Google Drive, Dropbox, Amazon S3 and many more.
We are currently exploring more advanced features such as data caching (see OneData), in cooperation with the eXtreme-DataCloud project.
The Dashboard
The DEEP dashboard. allow users to access computing resources to deploy, perform inference, and train modules hosted at the DEEP Open Catalog, as well as deploying external Docker images hosted in Dockerhub. The Dashboard simplifies the deployment and hides some of the technical parts that most users do not need to worry about.