It takes more than models.
Building the whole pipeline used for model generation and utilization in production requires multiple capabilities. Data ingestion, cleansing, labeling, model training infrastructure, model performance monitoring, and feedback collection are must-haves. Furthermore, these elements need to be automated to account for frequently changing data.
Evaluate the data strategy.
Design and build the ingestion and model deployment pipeline.
Monitor the model's performance and close feedback loop.
Build an integrated machine learning pipeline.
Creating machine learning models is only part of the work that needs to be done for a fully-fledged ML solution. Equally important is defining how the data should be ingested, processed, transformed, and stored for model training. Then later, determine how that model is deployed, and feedback of model performance is collected.
Data ingestion
Review existing data ingestion routines and provide insights into how can it be modified for effective usage in machine learning tasks.
Model deployment
Package and deploy the model into the operational environment. Recognizing that model outputs are not intuitive, set a requirement to properly collect all appropriate data used in decisioning for troubleshooting or monitoring model performance. Then, implement fully automated pipelines that generate, package, and later deploy models into production for consumption.
Performance monitoring
Track performance to verify that a model is performing according to metrics received in the modeling phase. Any bias shift in data may degrade performance. Be sure to build out feedback collection and performance degradation detection workflows, along with automatic triggers for model retraining and redeployment.
White paper
Go beyond velocity with advanced product metrics
Configuring a comprehensive framework of metrics that drive results
Download
