Deploying and Scaling Machine Learning in Production: A Q&A with Modzy COO and Co-Founder Josh Elliot
In the latest episode of DMI’s podcast Beyond DigitaI, Amy Barger, DMI’s Vice President of Digital Business Solutions, and Josh Elliot, COO and Co-Founder of Modzy, discuss the deployment and scaling of machine learning in production. Below are edited excerpts from the conversation.
Q: Josh, what led you to launch Modzy?
A: Throughout my career, I have led large teams solving problems using a variety of technologies. More recently, during those projects, the teams and I started to see organizations making significant investments in data preparation, data science, and even the beginnings of artificial intelligence capabilities. We also noticed that operationalizing those investments was difficult for most organizations. This caused them to fail to launch or create the value that they hoped for. Many leaders were becoming disenfranchised with this amazing technology and we knew we could help. That was the motivation behind starting Modzy, making it super easy to deploy and run artificial intelligence and machine learning anywhere, whether in the enterprise or at the edge, so that those organizations could create and capture the value from their investments.
Q: AI at scale requires a drastically different approach to, say, traditional software development, or DevOps. This is where MLOps comes in, right?
A: Yes, many organizations are starting to expand their use of AI beyond experimentation and running just a few machine learning models. We’re starting to see some of the larger tech players growing to hundreds, thousands, or even millions of models in production. When you get to those levels, you really need to start thinking about a solution to manage the complexity that’s introduced, and that’s where machine learning ops comes in.
Q: How would you define MLOps?
A: I would describe it as a set of processes, practices and tools that automate the deployment, management, running and monitoring of all of your machine learning and AI in production at scale. With an MLOps solution, you can enable faster deployments, better quality control, and even increase your organization’s ability to rapidly respond to changing business or mission needs.
Q: What challenges are your customers facing with their AI/ML implementations?
A: It depends on the maturity of the organization, but almost every organization wants to be an AI-enabled company. That said, we’re seeing teams struggle with the complexity and lengthy development cycles for building those AI-enabled solutions. First, there are talent gaps in the market. We’re seeing employees being asked to do things that are really outside of their expertise. For example, we’ve got data scientists that are being asked to write production code that’ll scale — certainly not something that they’re trained to do or likely want to do.
You also have complexities that come with operating in production environments, whether that’s in the cloud, on premise, or at the edge. Another challenge is that processes are super time-intensive because organizations are just building custom pipelines to support each individual use case or model. That’s a huge waste of time, honestly. Then, you have also think about governance risks and controls that organizations need as they mature their organization’s AI pipelines.
Q: How does an organization ramp up to realize the benefits of AI while also managing the many risk factors?
A: I think much of the focus today is on development at the expense of what happens next. There certainly are both intrinsic and extrinsic risks at every turn, but also opportunities to manage those risks. Take use case selection or the application that you’re going to actually enable with AI. There are a number of model impact assessment frameworks that exist to help guide organizations through this process. The key is understanding the scope of the stakeholders and the magnitude of impact to each of them if something were to go wrong with your machine learning algorithm.
Organizations also need to pay attention to data-related risks, including both the quality and limitations of that data during model deployment. To manage these risks, they need to put in place testing steps that inspect that data being used to train the machine learning models. Another risk is model performance drift. With MLOps, organizations can certainly put in place mechanisms that alert the data scientists when a model might be experiencing concept or data drift. The last risk area I would highlight is that there are a lot of new rules and regulations starting to form around machine learning and artificial intelligence. Organizations should make sure they are aware of what those new rules and regulations are and how they may affect their industry.
Q: Let’s go deeper on the value that an MLOps solution can add to an organization.
A: For many organizations, the value in adopting an MLOps solution or approach is really in accelerating those development cycles with a solution that handles all the plumbing required of these pipelines. An MLOps platform is also going to ensure that your models will run and scale in a consistent way, and it’ll provide an API-based tool that’s going to make it super easy to connect with your existing CICD pipelines, your data storage solutions, your enterprise software and more.
A couple more areas where machine learning ops solutions can really benefit organizations: Being able to track the lifecycle from the beginning to the end. When that performance does diminish an MLOps solution can help you retrain those models. Machine learning solutions can also help you understand the pedigree of those models, the data used to train those models, and the approval steps that go into putting those models into production.
Q: What advice would you give to CEOs, CIOs, or other stakeholders that are just getting started on their AI journey?
A: I love this question. I think organizations really have an opportunity right now to rethink how they adopt artificial intelligence across their enterprise. What I mean by that is that organizations should be looking for solutions that support composable business and technical architectures that easily integrate into existing workflows and tech stacks without causing major disruptions or creating additional friction.
I also think MLOps solutions aren’t just for extremely mature organizations. They will accelerate value creation through faster model deployment, better quality control through standards and APIs, and the ability to quickly respond to changing business needs. Some successful proofs of concepts and artificial intelligence can also lead organizations towards an AI center of excellence type of operating model. As you operationalize those technologies, your organization can develop its talent and best practices and then move out into the business once the critical mass exists.
Want to learn more about MLOps? You can hear the full conversation here. You can also connect with our team of more than 125+ certified data analytics experts to discuss an MLOps solution for your organization.
