PhD defense: Towards Continuous Development of MLOps Practices

. .

Candidate: Meenu Mary John, Malmö University Venue: Hörsal C, Niagara, Malmö University Opponent: Dr. Marie Christin Platenius-Mohr, PhD, Research Team Manager, Automation and Software Architecture, ABB AG Corporate Research, Germany Committee: – Tommi Mikkonen, Professor of Software Engineering, Software and communications engineering, University of Jyväskylä, Finland -Maria Paasivaara, Professor of Software Engineering, Software Engineering, Lappeenranta-Lahti University of Technology, Finland – Stefan Biffl, Associate Professor of Software Engineering, Institute of Information Systems Engineering, Vienna University of Technology, Austria

Abstract:

Context: With digitalisation, companies that specialise in software-intensive embedded systems are transforming from a business reliant on hardware and products to a business that utilises software, data and AI (especially Machine Learning (ML) and Deep Learning (DL)). These technologies allow companies to extend their product offerings, develop new products or services, and create revenue opportunities. Despite the advancements, the majority of ML/DL deployments fail within companies. This highlights the need to optimise the end-to-end process of developing, deploying, and evolving ML/DL models, and a strong understanding and collaboration between researchers and practitioners in both the fields of ML/DL and Software Engineering.

Objective: The research focuses on two main themes:(a) Establishing systematic and structured frameworks for the development, deployment and evolution of models, and (b) Exploring how the evolving (changing) needs of software-intensive embedded systems companies utilising ML/DL technologies shape the MLOps (Machine Learning Operationalisation) practises they needed and measures its maturity over time. Based on the themes mentioned above, we have three primary objectives: (a) Identifying the need for MLOps, (b) Developing the frameworks for MLOps adoption, and (c) Standardising and measuring the MLOps practices.

Method: The research is conducted in collaboration with companies and employ various research approaches, including case study, action research and multi-vocal literature review. Also, we employ different techniques, including interviews and observations.

Results: First, it identifies the challenges faced and activities conducted by practitioners in companies when developing, deploying and evolving models. Second, it derives a conceptual framework that presents three parallel and concurrent activities that companies utilise when developing, deploying and evolving models.  Third, it presents a framework based on current literature to accelerate the end-to-end deployment process and advance knowledge on integrating, deploying and operationalising ML/DL models. Fourth, it develops a generic framework with five architectural alternatives for deploying ML/DL models at the edge.  Five, it explores how MLOps, as a practice, brings together data scientist teams and operations to ensure the continuous delivery and evolution of models. Sixth, it presents the MLOps framework, maps companies to the MLOps maturity model, and validates the MLOps framework and maturity model with other companies. It also presents critical trade-offs that practitioners made when adopting MLOps. Seventh, it presents an MLOps taxonomy that helps companies determine their maturity stage and provide tailored MLOps practices to advance.

Conclusion: The thesis shows well-structured approach to improve end-to end ML lifecycle. Through the research, we seek to enable and advance not only experts but also non-experts to effectively approach the development, deployment, and evolution of ML/DL models in the current embedded systems industry. This is relevant considering the shortage of highly skilled data scientists.