Model Orchestration Platform for Improved Cross-Team Collaboration
Our client faced multiple business challenges, with distributed teams operating in a disconnected digital and data environment, following suboptimal internal processes, and little or no cross-team resource interchangeability. Severe bottlenecks in developing crucial simulations, statistical, and machine learning models, led to increased resource management overhead and longer time-to-market.
We rolled out a solution that turned into a centralized platform for all teams, offering them model orchestration, management, monitoring, and ease of deployment in a consistent way. The Model Orchestration platform provided a standard set of tools that are reusable, scalable, and fault-tolerant. Its implementation improved productivity, collaboration and enabled faster iteration of ideas to production.
80% Reduction of Model Crashes
40% Quicker Diagnosis of Root-Causes
24% Decrease in Model Execution Times
An international energy company that operates numerous power plants and leverages advanced technologies and innovations to manage large-scale power systems and infrastructure. The company sources oil, natural gas, and global commodities and trades electricity, emissions certificates, natural gas, LNG, and coal. They operate a sophisticated ecosystem of power plants and manage their own power infrastructure business.
In order to modernize and harmonize our client’s modelling setup, we took a two-pronged approach. First, we consulted and advised their internal teams on best practices for software development and governance. As a second step, we designed and implemented a modelling platform, which would be used across different departments, serving various use cases, each with its own technical and functional requirements.
I. Process consultancy
We began by performing a detailed analysis of our client’s needs and workflows when developing, orchestrating and collaborating on different models. We proceeded to create a comprehensive gap analysis, which outlined our findings and suggested improvements. We implemented several CI/CD pipelines following industry standards and best practices, which allowed the in-house engineers to collaborate in model creation and efficiently utilize the Modelling Platform’s capabilities.
As a result we:
- Harmonized the technological stack and tools used by teams across departments;
- Implemented a proper Gitflow for version control;
- Set up centralized Azure DevOps pipelines for building, testing and dockerizing models;
- Set up a central Container Registry for image distribution.
II. Modelling platform
To enhance the cross-team collaboration, we implemented a central modelling platform to be used as an orchestration, execution, and monitoring tool, tailored to satisfy all current use cases. The platform is based on Azure technologies and allows for the deployment, orchestration, monitoring, and debugging of various models with different technical and functional requirements. It is designed with extensibility in mind, so it can be easily adapted whenever new requirements and use cases surface in the future.
As a result, our client’s internal collaboration was significantly improved, inter-department projects were vastly facilitated and overall model quality was enhanced.
Unlocked business value:
- 80% reduction of model crashes. Improved model stability & IT infrastructure;
- 40% quicker diagnosis of root-causes;
- 24% decrease in average model execution times;
- Faster Time-to-Market. Model deployment/release timelines reduced from avg of 1-2 hours to 10 min;
- Robust processes and collaboration.
We have a global client base that includes Fortune 500 companies, innovative startups and industry leaders in Information Technology, E-Commerce, Insurance, Healthcare, Finance and Energy & Utilities.