1. No category

METRICS RESEARCH –
ENABLING ACTIONABLE SOFTWARE
METRICS IN MODERN COMPANIES
ESTABLISHING CUTTING EDGE METRICS RESEARCH AND DEVELOPMENT ENVIRONMENT
MIROSLAW STARON
WILHELM MEDING
KENT NIESEL
ANDERS HENRIKSSON
CHRISTOFFER HÖGLUND
OLA SÖDER
VARD ANTINYAN, …
METRICS
Who am I?
• Background
– Associate professor, University of Gothenburg
– Head of Software Engineering for Embedded and Automotive Systems research group
(SEAS) at Chalmers | University of Gothenburg
– > 130 papers published in the area of metrics, modelling (UML) and requirements
– Collaborations with Ericsson, Volvo Cars, AB Volvo, Saab, Sony, Grundfos, Telelogic,
Axis, ABB, …
– Wilhelm Meding, Kent Niesel, Ola Söder, Christoffer Höglund, Anders Henriksson, …
• Research interest
–
–
–
–
Metrology in Software Engineering
Corporate Performance Measurement
Product metrics and indicators
Visualizations and dashboards
2
METRICS
Outline for today
• Software Metrics theme
• Global trends in software metrics
• Metrics theme contributions
to the global trends
• Wrap-up
3
METRICS
Software Center Metrics Program
Goals and wanted position
• Goal
– Rapidly empower the company (at all levels) to become excellent in
measuring
• Wanted position
– Evolving existing best (metrics) practices
– Enhance release readiness assessments
– Increase robustness of measurement programs
4
METRICS
Metrics research – organizational context
Predictions,
simulations,
self-evolvning
Insight
Org. cataloguing
management
Statistics
Elevating
organizational
performance
Decision
support
Collection
Visualization
Papers/research
contribution
SW development
and management
Metrics research
Data analyses
5
METRICS
Global Trends in Software Metrics
• Increased importance of software as an element of a global supply chain
– New metrics
• Empowered teams embracing diversity
– New dissemination patterns
• Status reporting is getting replaced by the need to have deeper insight
– New visualization patterns
• High level KPIs are complemented with PIs and low level trends
– New status reporting
• Shift from providing information to triggering decisions
– New intraction patterns
• Availability of large data sets
– New analysis methods
Based on Staron M, Meding W, Niesel K, Söder O, ” Evolution of the Role of Measurement Systems in Industrial Decision
Support”, book chapter in ”Handbook of Research in Global Supply Chain Management”
6
METRICS
Increased importance of software as an
element of a global supply chain
• SW as an element of a global supply chain
– Systems of systems
– Internet of things
– Suppliers of ECUs with/without SW
• New metrics
– Release readiness
– Innovation
– IPR
7
METRICS
Infrastructure and common language set-up:
Simplistic measurement system
• Measurement system is a set of
interrelate and interacting elements
necessary to achieve metrological
confirmation and continual control of
measurement processes
(JCGM 200:2008)
• Can be realized using simple tools as
MS Excel, Task Scheduler, etc
• Can be standardized using
ISO 15939
8
METRICS
Pro-active measurement:
Self-healing measurement infrastructure
• Advanced algorithms to automatically
recover metrics
• Increased throughput of metric teams
• Increased data quality
• Reduced maintenance cost
M. Staron, W. Meding, M. Tichy, J. Bjurhede, H. Giese, ”Evolving
Measurement Systems into Self-Healing
Systems for Improved Availability”
9
METRICS
Robustness of measurement programs
• Motivation
– Measurement awareness checklist
from the workshop in 2013
• Goal
– Quantifying awareness of metrics
• Impact
– Cross-company benchmarking
– Exchange of good practices
– Development of formal robustness model
of measurement programs
10
METRICS
Robustness of measurement programs
Visualization of results (excerpt)
• Large variability
• Several categories
have ”leading”
companies
• A number of good
examples have been
identified
11
METRICS
Automatic Ranking of Textual
Requirements Based on Quality
• Problem
– About 200 ECUs and 10 000 textual requirements
– How to automatically identify requirements that are difficult to
understand, implement, or test as early as possible
• Objective
– Design a method and tool for automatic assessment of requirements
Identify
Properties
Measure
Calibrate
Evaluate
with
designers
Name
LD_Req-1175 v9
LD_Req-30085 v1
LD_Req-20677 v5
LD_Req-28823 v3
LD_Req-13284 v6
LD_Req-11659 v1
LD_Req-12235 v4
LD_Req-11349 v4
LD_Req-9847 v24
LD_Req-9916 v4
1Name
Risk index
Maintenance
107 service
Valve Lash103
& Unit Injector Pre-Load Monitor Evaluation
Main-beam96
activation in Vehicle Mode Running/Prerunning, With AHS
Display Customer1
95
Maintenance
Display Standard
93 Maintenance
VehicleMode_ctrl
81
Standard Oil
79Degradation Monitor Evaluation
Standard Tachograph
79
Monitor Evaluation
Engine Belt78
Monitor Evaluation
Air Filter Monitor
78
Evaluation
12
Requirements quality model
We designed
measures
for
Measuring the Internal
quality properties of
requirements
which affects
the external quality
properties
METRICS
Evaluate the ranking accuracy
• Quality index = Complexity + Dependency + Ambiguity
• Evaluation
– Volvo GTT project 1 – 87% prediction accuracy
– Volvo GTT project 2 – 70%
– Saab project 1 - 70%
– Volvo CC project - ongoing
xID
x0400000002CEE415
x0400000002860EDF
x04000000011AE00E
x04000000011AE09E
x0400000003062477
x04000000011ADFF4
Risky 15 by tool x04000000011AE0F1
x04000000031EC161
x0400000001511DB4
x0400000002E51331
x0400000002CEE406
x040000000288B2C1
x0400000002FB0BD4
x0400000001BFEA10
x0400000002B66036
Conditions
Uncertainty
Dependency
References
Structuredness
Tool
Andreas Mattias
48 1 31 0 52
78
4
4
41 0 13 3 43
67
5
3
34 1 51 0 60
62
4
4
34 1 51 0 60
62
5
4
34 1 51 0 60
62
4
4
33 1 51 0 60
60
4
4
33 1 87 0 96
60
4
4
18 1 11 4 12
58
4
3
39 0 28 0 51
55
4
5
34 6 3 0 36
53
4
4
25 1 21 0 25
49
3
4
39 0 17 0 46
49
4
3
19 1 21 0 15
47
1
1
24 4 18 0 32
46
4
3
25 2 12 0 26
42
3
3
Average comments
Effectivenes %
4
4 disagr. of raters
4
4,5
4
4
4
3,5
87%
4,5
4
3,5
3,5
1 Faulse positive
3,5
14
3 Faulse positive
METRICS
Future of metrics
1. Costs will decrease, speed will increase
a. Delivery speed will increase
b. Update speed will increase
c. Stakeholders will get in control of the entire delivery chain
2. The number of metrics will continue to grow
a. New products, New usage scenarios
b. New markets, New challenges/problems
3. New analysis methods will appear
a. Big data
b. Software analytics
c. Clouds
4. Complexity of metrics will grow
a. Advanced analyses
b. Instant access
c. More stakeholders
15
VOLVO CARS METRICS SEMINAR
2015-04-30, PVH
9.00 – 12.00
SUCCESSFUL MEASUREMENT PROGRAMS – HOW TO
CREATING NEW PROCESS AND PRODUCT KPI-S
SOFTWARE ARCHITECTURE EVOLUTION MEASUREMENT
SOFTWARE DEFECT PREDICTION
AUTOMATED REQUIREMENTS RISK ASSESSMENT
MIROSLAW STARON
miroslaw.staron@gu.se
17