1. No category

Automated Software Testing to Discover Energy
Inefficiencies in Mobile Apps
Sudipta Chattopadhyay
Linköping University, Sweden
Joint work with Abhijeet Banerjee, Lee Kee Chong and Abhik Roychoudhury
National University of Singapore
Context
Developer
Write energy-efficient !
software
Programming abstractions
Desktops, handheld devices etc.
2
Tools and
techniques
Overview
Tools and techniques to write energy-efficient software
Energy-aware !
Software Testing
Energy-hungry !
Code Patterns
Desktop machines, handheld devices etc.
3
Energy-aware !
Coding Guidelines
Overview
Tools and techniques to write energy-efficient software
Energy-aware !
Software Testing
Energy-hungry !
Code Patterns
Desktop machines, handheld devices etc.
4
Energy-aware !
Coding Guidelines
Android Devices
Smartphone Market
Smartphone Sales
Mobile App Testing Market Size
Billion Dollars
$2.30
$0.80
$0.70
$0.20
2013
2014
2015
2016
2017
Data obtained from IDC, Gartner and ABI Research
Energy Inefficiency
•
How do we quantify energy inefficiency?
•
•
•
High energy consumption, what is high?!
High energy consumption
•
High utilization of hardware components
•
Low utilization of hardware components
Ratio Energy/Utilization
Energy Inefficiency
Cause/Source
Hardware components
Resource leak
Suboptimal resource
binding
Sleep state transition
Wakelock bug
Tail Energy hotspot
Background Service
Vacuous background
service
Expensive
background service
Defective Functionality
Immortality bug
Loop energy hotspot
Energy Inefficiency
Wasted Energy
Resource
acquired
Resource
first used
Suboptimal resource binding
Wasted Energy
Service started
Never used
Vacuous background service
A Broader Categorization
Cause/Source
Energy Bugs
Energy Hotspots
Hardware components
Resource leak
Suboptimal resource
binding
Sleep state transition
Wakelock bug
Tail Energy hotspot
Background Service
Vacuous background
service
Expensive
background service
Defective Functionality
Immortality bug
Loop energy hotspot
Device does !
not return to idle
High energy consumption !
+ low utilization
Measurement
Yokogawa Digital
Power Meter
LG Optimus
smartphone
Our framework
Measurement
•
Measuring Energy/Utilization ratio for an application
Energy/Utilization
Time
App is not executing
(PRE)
App is executing
events (EXC)
Recovery
(REC)
Idle
(POST)
Energy Inefficiency
Comparable
Energy/Utilization
Time
App is not executing
(PRE)
App is executing
events (EXC)
Recovery
(REC)
Idle
(POST)
Energy Inefficiency
Comparable
Energy/Utilization
Time
Not Comparable
Energy/Utilization
Energy Bugs
Time
App is not executing
(PRE)
App is executing
events (EXC)
Recovery
(REC)
Idle
(POST)
Energy Inefficiency
Normal behavior
Energy/Utilization
Time
App is not executing
(PRE)
App is executing
events (EXC)
Recovery
(REC)
Idle
(POST)
Energy Inefficiency
Normal behavior
Energy/Utilization
Time
Energy/Utilization
Abnormal !
(high energy!
low utilization)
Energy Hotspots
Time
App is not executing
(PRE)
App is executing
events (EXC)
Recovery
(REC)
Idle
(POST)
Test Generation
Our framework
Detecting
Energy
Inefficiency
Guiding to execute
energy-inefficient
scenarios
Test Generation
Our framework
Detecting
Energy
Inefficiency
Guiding to execute
energy-inefficient
scenarios
Measurement
Yokogawa Digital
Power Meter
Record
Energy
Trace
LG Optimus
smartphone
Send events
Record Utilization
Measurement
Yokogawa Digital
Power Meter
Record
Energy
Trace
Global !
clock
LG Optimus
smartphone
Send events
Record Utilization
Utilization
Energy consumption of different components is not even (GPS < CPU)
100% CPU does not consume same energy as GPS being on
Detecting Energy Bugs
Comparable
Energy/Utilization
Time
Not Comparable
Energy/Utilization
Statistical !
dis-similarity !
between !
PRE and POST
App is not executing
(PRE)
Time
App is executing
events (EXC)
Recovery
(REC)
Idle
(POST)
Detecting Energy Hotspots
Normal behavior
Energy/Utilization
Time
Energy/Utilization
Detecting !
discords in !
time-series!
data
App is not executing
(PRE)
Abnormal !
(high energy!
low utilization)
Time
App is executing
events (EXC)
Recovery
(REC)
Idle
(POST)
Test Generation
Our framework
Detecting
Energy
Inefficiency
Guiding to execute
energy-inefficient
scenarios
Guided Exploration
•
Energy-inefficient execution
•
Which fragments are energy-inefficient?
•
What is an appropriate coverage metric?
A Broader Categorization
Cause/Source
Energy Bugs
Energy Hotspots
Hardware components
Resource leak
Suboptimal resource
binding
Sleep state transition
Wakelock bug
Tail Energy hotspot
Background Service
Vacuous background
service
Expensive
background service
Defective Functionality
Immortality bug
Loop energy hotspot
Invoked via System Calls
Test Generation
Our framework
Detecting
Energy
Inefficiency
Android App
+
System call pool
To smartphone
Event
trace
Guided
exploration
Guiding to execute
energy-inefficient
scenarios
Event flow
graph
Event trace
database
Test Generation
Android App
+
System call pool
To smartphone
Event
trace
Event flow
graph
Guided
exploration
Event trace
database
An ordering between trace T1 and T2
T1 > T2
Buggy
trace
s1
s1
s1
s2
s2
s2
s3
More system calls
!
T1 > T2
!
Executed
trace
T1 < T2
s1
s5
s1
s1
s5
s2
s6
s2
s2
s6
s4
s3
s3
Similarity with buggy trace
Covering more system calls
Evaluation
Travel/Transportation
Lifestyle/Health
Books/News
Productivity
Photography/Media
Entertainment
Puzzle
Tools
Category of Android Apps Evaluated
Summary of Evaluation
App
Aripuca
Feasible!
traces
502
Energy
Bugs
Yes
Energy !
Hotspots
Type
Reported!
before
No
Vacuous
background
service
No
No
Montreal
Transit
64
No
Yes
Suboptimal
resource
binding and
more
Sensor
Test
2800
Yes
No
Immortality Bug
No
Yes
Vacuous
background
service, suboptimal
resource binding
No
760
KFMB AM
26
Yes
All Results are in the paper
(10 energy bugs and 3 energy hotspots found
out of 30 tested apps)
Summary of Evaluation
Code !
coverage
21
Lines of Code
Aagtl
System call!
coverage
100
Android Battery Dog
100
17
463
Aripuca
100
15
4353
Kitchen Timer
100
30
1101
Montreal Transit
89
11
10925
NPR News
100
24
6513
OmniDroid
83
36
6130
Pedometer
100
56
849
Vanilla Music Player
86
20
4081
App
11612
To cover all system calls, exploring only a small part of the program suffices
A substantial portion of the code is used for provide user feedback, compatibility
over different OS
Case Studies
Aripuca (Energy Bug)
Reason: Vacuous Background Service
Fix:
serviceConnection.getService().stopLocationUpdates();
serviceConnection.getService().stopSensorUpdates();
Case Studies
Aripuca (Energy Bug)
Reason: Vacuous Background Service
Case Studies
Montreal Transit (Energy hotspot for <5 sec)
Loading Ads
Main Thread
Location service
Location service
(for loading ads)
User exits
GPS released
Case Studies
Montreal Transit (Energy hotspot for <5 sec)
Loading Ads
Main Thread
Location service
Main Thread
Location service
Location service
User exits
GPS released
(for loading ads)
Loading Ads by Asynchronous thread
Share location
for loading ads
User exits
GPS released
Case Studies
Montreal Transit (Energy hotspot for <5 sec)
Montreal Transit (After fixing)
Summary
•
•
Categorization of energy inefficiency
•
Energy bugs
•
Energy hotspots
A guided exploration of event traces
•
•
Targeting system call coverage
Evaluation with Android apps
•
Energy bugs and hotspots exist in several Android apps
Open Problems
•
Event Flow Graph may require intelligent inputs
•
Games
•
Confirmation of bugs/hotspots require repeated
execution
•
Debugging and optimization is manual