1. technology and computing

STANDARDISING CREW COMMUNICATION SKILLS TRAINING
USING SPEECH ANALYSIS TECHNOLOGY: APPLICATION FOR
AIR TRANSPORT OPERATORS.
Dr. Brian Vaughan, Cofounder & CTO, Vocavio
About Vocavio
Vocavio has developed speech signal processing
technology to generate insights into communication
effectiveness and team dynamics.
Our first product, CrewFactors, measures excellence
in team communication skills to enhance safety and
operational systems. This cloud based solution uses
patent pending speech analysis technology to
measure communication effectiveness in highly
skilled teams. This technology can be used alongside
existing simulation training systems to gain valuable
insights into communication and team dynamics.
Instructors and crew benefit by having access to
evidence based metrics that augment the simulation
training debrief and ultimately accelerates training.
Team Communication
The ability to work together with partners on a team
is an essential skill for successful communication in
stressful environments, such as aviation. This in turn
facilitates the successful exchange of information,
improves team effectiveness and cohesiveness, and
generally leads to a more successful outcome. While
there are a number of ways that communicative ability can be assessed, these can be somewhat subjec-
Copyright © 2015 Vocavio Ltd.
tive; the challenge then is how to objectively assess
it. Human communication is a multi-faceted,
multi-modal exchange of information utilising
gestures, gaze, facial expression, body language and
tone of voice. However, pilots are limited by the
physical layout of the cockpit and by the need to stay
fully aware of the instrumentation: thus pilots rely on
speech as the primary method of communication,
over and above the normal visual modalities.
Numerous methods and training protocols exist to
train crew in the non-technical aspects of flying a
plane: Multi Crew Cooperation (MCC), Jet Orientation Course (JOC) and Crew Resource Management
(CRM). Assessed communication skills commonly
include Situational Awareness, Decision Making,
Adaptability, Teamwork, Leadership, Assertiveness
and Mission Analysis (REF). These methods may
however suffer from subjectivity, as different instructors and training schools may assess differently.
Consequently, this presents a real challenge forstandardising communication skills across an air transport operator. So how to objectify the subjective? In
Trinity College Dublin, two speech scien- tists, Dr.
Brian Vaughan and Dr. Céline De Looze, have developed a speech analysis engine (called ‘Vocavio’) that
provides objective insights and metrics about
individuals’ communication performance.
1
This technology is utilised by Vocavios first product,
CrewFactors, to augment subjective CRM practices
with a layer of objective metrics that can be used by
both instructors and pilots. The technology is based
on the research hypothesis that cooperation and coordination, within a team/crew environment, would be
evident in the speech of the team members. By
analysing how the prosodic parameters (the pitch,
rhythm,and tone of speech) of two people interact
over the course of a conversation, a process known
as prosodic accommodation, insights about a crews
(and an individual’s) communication performance
can be gleaned. This is due to the fact that these
prosodic accommodation processes are an important
socio-communicative aspect of human interaction,
being necessary for both information exchange and
the establishment and maintenance of good relationships and rapport.
The Analysis System
The Vocavio speech analysis system is a cloud-based
system that analyses the recorded speech of two
people and produces detailed metrics. The CrewFactors product utilises the Vocavio engine to provide
detailed, metrics via a by a web-based visualisation
system. This turns the raw speech data into information, presented as intuitive and understandable
figures and charts. The performance metrics are
accessed via a secure website for use on an iPad by
an instructor in a debriefing.
1.
Record
pilot audio
5.
Deliver
insights
report
4.
Generate &
visualise
metrics
Testing the System
Through working with a number of civil and commercial aviation partners it was established that the
Vocavio speech analysis system metrics were strongly correlated with standard aviation communication
metrics carried out using a number of different data
sources over a period of time: a concentrated validation process involving human factors aviation
psychologists and commercial partners was complimented by subsequent validation work with CRM
instructors and various commercial aviation
partners.
The CrewFactors system turns the raw speech data
from the Vocavio analysis engine into usable,
concise and understandable information via a web
based visualisation framework. The metrics that
CrewFactors produces from the vocal analysis are:
global team score and communication performance
over time (based on the levels of prosodic accommodation between the two partners), team balance
(based on each individual’s effort to adapt to their
partner’s communicative style) and global amount of
time talking, listening and overlapping.
Result / Validation
This research has demonstrated that the CrewFactors
metrics provide strong objective metrics of CRM to
augment the instructors own assessment, with
strong, clear visuals that can be used in a de-brief to
give pilots and instructors an unambiguous and
concise set of results to discuss and work with.
Value for Objective Evaluation
2.
Upload to
speech
engine
3.
Process
speech
signals
Powered by Vocavio, a crew communications
train-ing system called CrewFactors has been developed, which provides both crew and their training
instructors with objective insights into communication and coordination skills. This web-based system
can be easily operated by training instructors and
integrated into existing crew training programmes.
Two team partners perform a task. Their vocal interaction is recorded, processed, and a report is automatically generated by the system.
Overview of the CrewFactors System, incorporating
the Vocavio speech analysis engine.
Copyright © 2015 Vocavio Ltd.
2
CrewFactors metrics include team score, communication performance over time, balance, talking, listening
and overlapping quotients (See Appendix 1). Along
with time aligned stressful events, these metrics
provide information about an individual’s behavioural
changes in critical situations (e.g. stress recovery).
Figure 1 illustrates behavioural maintenance (a) and
change (b) (blue curve) at a stressful event (orange
dotted line). As can be seen, on the one hand, team#2
drops slightly in terms of their communication and
recovers at a steady rate over the rest of the task; on
the other hand, team #3’s communication drops
significantly at a stress event, but increases and recovers very quickly immediately afterwards for a better
performance outcome.
Value for Standardisation
By providing objective insights about an individual’s
communication performance, Crewfactors, utilising
Vocavio technology, can be used to assist an air transport operator in standardising crew communication
skills. Once standards are defined (using statistical
modelling techniques), evaluations of new entrants,
skill monitoring and data gathering and comparison
become more accessible, reliable and cost effective.
Moreover, objective communication metrics can be
used to focus communication training on the areas
that individual pilots, and groups of pilots, are not
strong in.
Contact us at:
enquiry@vocavio.com
Vocavio, Trinity Technology Campus,
Grand Canal Quay,
Dublin 2. Ireland
Figure 1: Levels of communication effectiveness
(blue line) over time obtained for two crews. The
colour range indicates poor to excellent communication skills. The orange dotted lines indicate an instructor led stress event during a training scenario.
Copyright © 2015 Vocavio Ltd.
3