Measuring Your DevOps Success
By Seshachari, June 20, 2016
Organizations looking to implement DevOps today will likely need to make the business case to the organization’s leaders. Highly visible, useful metrics will become an essential part of DevOps success.
Transforming an organization to the DevOps model requires an organization to go over 4 Key aspects –
- Organization & Processes (Collaboration within and outside the team, incentives and structures, security and agile)
- People (Competency, Knowledge, Change Management, and Culture)
- Learning & Development (Enterprise based, Product based, team based, metrics and measurement)
- IT landscape & enablement (Transformation, rationalization, and agile architecture).
CAMS i.e. Culture, Automation, Measurement, and Sharing helps teams embrace the DevOps culture by improving collaboration, communication, and coordination while reducing the risk of failed deployments among different functions in the organization, & coupling with the best in breed tools like Electric cloud – Continuous Delivery solution, automating and accelerating build/test/deploy processes, New Relic – Application performance monitoring, & other fast deployment tools.
Establishing key performance indicators (KPIs) will enable to determine if you’re keeping pace with customer needs – Shorter Development Cycle, Increased Release Velocity, Improved Defect Detection, Reduced Deployment Failures and Rollbacks Reduced.
- Change volume – How many user stories and new lines of code are being deployed?
- Lead Time – How long does it take for a new requirement/story to go from the time when it hits development to when it successfully gets deployed into production?
- Deployment Frequency – How often is the team deploying into production?
- Build duration – How long are builds taking to complete?
- Percentage of Failed Deployments – What is the percentage of deployments which have caused an outage or negative user reaction?
- Cycle Time – How is the team trending in terms of cycle time, over the life-cycle of a project?
- Mean Time to Recovery (MTTR) – When failure does occur, how long does it take the team to recover from the issue?
- Velocity and Estimation – Will they finish what is promised?
- Customer Ticket Volume – “This is a basic indicator of customer satisfaction,” and an insightful metric to track
- Percentage of change in User Volume – “Number of new users signing up, interacting with [the team’s] service and generating traffic.”
- Availability – Were any SLAs violated, and what is the overall up-time for the product or service?
- Performance (Response Time) – “This metric should remain stable irrespective of %change in user volume or any new deployment”
- Revenue/profit impact of outages/performance issues
- Number and cost of resources
It’s mandatory for any organization to have a baseline i.e. organizations can build a business case, apply targets & goals to all projects, monitor and measure the progress made – Celebrate success!