1. technology and computing
2. programming languages
3. c and c++

DEPENDENCY PARSING，FRAMENET,
SEMANTIC ROLE LABELING, SEMANTIC
PARSING
Heng Ji
jih@rpi.edu
September 17, 2014
Acknowledgement: FrameNet slides from Charles Fillmore;
Semantic Parsing Slides from Rohit Kate and Yuk Wah Wong
Outline
• Dependency Parsing
• Formal definition
• Dynamic programming
• Supervised Classification
• Semantic Role Labeling
• Propbank
• Automatic SRL
• FrameNet
• Semantic Parsing
Semantic Parsing
• “Semantic Parsing” is, ironically, a semantically
ambiguous term
• Semantic role labeling
• Finding generic relations in text
• Transforming a natural language sentence into its meaning
representation
Semantic Parsing
• Semantic Parsing: Transforming natural language (NL)
sentences into computer executable complete meaning
representations (MRs) for domain-specific applications
• Realistic semantic parsing currently entails domain
dependence
• Example application domains
• ATIS: Air Travel Information Service
• CLang: Robocup Coach Language
• Geoquery: A Database Query Application
ATIS: Air Travel Information Service
• Interface to an air travel
database [Price, 1990]
• Widely-used benchmark for
spoken language
understanding
May I see all the flights
from Cleveland to
Dallas?
Semantic
Parsing
Air-Transportation
Show: (Flight-Number)
Origin: (City "Cleveland")
Destination: (City "Dallas")
NA 1439, TQ 23,
…
Query
CLang: RoboCup Coach Language
• In RoboCup Coach competition teams compete to
coach simulated players [http://www.robocup.org]
• The coaching instructions are given in a computer
language called CLang [Chen et al. 2003]
If the ball is in our
goal area then player
1 should intercept it.
Semantic Parsing
(bpos (goal-area our) (do our {1} intercept))
CLang
Simulated soccer field
Geoquery: A Database Query Application
• Query application for U.S. geography database containing
about 800 facts [Zelle & Mooney, 1996]
Which rivers run
through the states
bordering Texas?
Arkansas, Canadian, Cimarron,
Gila, Mississippi, Rio Grande …
Answer
Semantic Parsing
answer(traverse(next_to(stateid(‘texas’))))
Query
What is the meaning of “meaning”?
• Representing the meaning of natural language is
ultimately a difficult philosophical question
• Many attempts have been made to define generic formal
semantics of natural language
• Can they really be complete?
• What can they do for us computationally?
• Not so useful if the meaning of Life is defined as Life’
• Our meaning representation for semantic parsing does
something useful for an application
• Procedural Semantics: The meaning of a sentence is a
formal representation of a procedure that performs some
action that is an appropriate response
• Answering questions
• Following commands
Meaning Representation Languages
• Meaning representation language (MRL) for an
application is assumed to be present
• MRL is designed by the creators of the application to
suit the application’s needs independent of natural
language
• CLang was designed by RoboCup community to send
formal coaching instructions to simulated players
• Geoquery’s MRL was based on the Prolog database
Engineering Motivation for Semantic
Parsing
• Applications of domain-dependent semantic parsing
•
•
•
•
Natural language interfaces to computing systems
Communication with robots in natural language
Personalized software assistants
Question-answering systems
• Machine learning makes developing semantic parsers
for specific applications more tractable
• Training corpora can be easily developed by tagging
natural-language glosses with formal statements
Cognitive Science Motivation for Semantic
Parsing
• Most natural-language learning methods require
supervised training data that is not available to a child
• No POS-tagged or treebank data
• Assuming a child can infer the likely meaning of an
utterance from context, NL-MR pairs are more cognitively
plausible training data
Distinctions from Other NLP Tasks:
Deeper Semantic Analysis
• Information extraction involves shallow semantic analysis
Show the
long
email Alice sent
me
yesterday
Sender
Sent-to
Type
Time
Alice
Me
Long
7/10/2010
Distinctions from Other NLP Tasks:
Deeper Semantic Analysis
• Semantic role labeling also involves shallow semantic
analysis
sender
Show the
long
email Alice sent
theme
recipient
me
yesterday
Distinctions from Other NLP Tasks:
Deeper Semantic Analysis
• Semantic parsing involves deeper semantic analysis to
understand the whole sentence for some application
Show the
long email Alice sent me yesterday
Semantic Parsing
Distinctions from Other NLP Tasks:
Final Representation
• Part-of-speech tagging, syntactic parsing, SRL etc.
generate some intermediate linguistic representation,
typically for latter processing; in contrast, semantic
parsing generates a final representation
sentence
noun phrase
sentence
verb phrase
noun phrase
verb determiner adjective
Show
noun
noun phrase
verb phrase
noun
verb pronoun
the long email Alice sent me
noun
yesterday
Distinctions from Other NLP Tasks:
Computer Readable Output
• The output of some NLP tasks, like question-answering,
summarization and machine translation, are in NL and
meant for humans to read
• Since humans are intelligent, there is some room for
incomplete, ungrammatical or incorrect output in these
tasks; credit is given for partially correct output
• In contrast, the output of semantic parsing is in formal
language and is meant for computers to read; it is critical
to get the exact output, strict evaluation with no partial
credit
Distinctions from Other NLP Tasks
• Shallow semantic processing
• Information extraction
• Semantic role labeling
• Intermediate linguistic representations
• Part-of-speech tagging
• Syntactic parsing
• Semantic role labeling
• Output meant for humans
• Question answering
• Summarization
• Machine translation
Relations to Other NLP Tasks:
Word Sense Disambiguation
• Semantic parsing includes performing word sense
disambiguation
State
State?
Which rivers run through the states bordering Mississippi?
Semantic Parsing
answer(traverse(next_to(stateid(‘mississippi’))))
River?
Relations to Other NLP Tasks:
Syntactic Parsing
• Semantic parsing inherently includes syntactic parsing but
as dictated by the semantics
MR: bowner(player(our,2))
A semantic derivation:
bowner(player(our,2))
player(our,2)
our
our
bowner(_)
player(_,_)
2
player
2
null
bowner(_)
has
null
the
null
ball
Relations to Other NLP Tasks:
Syntactic Parsing
• Semantic parsing inherently includes syntactic parsing but
as dictated by the semantics
MR: bowner(player(our,2))
A semantic derivation:
S-bowner(player(our,2))
NP-player(our,2)
PRP$-our
our
VP-bowner(_)
NN-player(_,_) CD-2
player
2
null
VB-bowner(_)
has
null
the
null
ball
Relations to Other NLP Tasks:
Machine Translation
• The MR could be looked upon as another NL [Papineni et
al., 1997; Wong & Mooney, 2006]
Which rivers run through the states bordering Mississippi?
answer(traverse(next_to(stateid(‘mississippi’))))
Relations to Other NLP Tasks:
Natural Language Generation
• Reversing a semantic parsing system becomes a natural
language generation system [Jacobs, 1985; Wong &
Mooney, 2007a]
Which rivers run through the states bordering Mississippi?
Semantic Parsing NL Generation
answer(traverse(next_to(stateid(‘mississippi’))))
Relations to Other NLP Tasks
• Tasks being performed within semantic parsing
• Word sense disambiguation
• Syntactic parsing as dictated by semantics
• Tasks closely related to semantic parsing
• Machine translation
• Natural language generation
Dependency Grammars
• In CFG-style phrase-structure grammars the main focus is
on constituents.
• But it turns out you can get a lot done with just binary
relations among the words in an utterance.
• In a dependency grammar framework, a parse is a tree
where
• the nodes stand for the words in an utterance
• The links between the words represent dependency relations
between pairs of words.
• Relations may be typed (labeled), or not.
Dependency Relations
Dependency Parse
They hid the letter on the shelf
Dependency Parsing
• The dependency approach has a number of advantages
over full phrase-structure parsing.
• Deals well with free word order languages where the constituent
structure is quite fluid
• Parsing is much faster than CFG-bases parsers
• Dependency structure often captures the syntactic relations needed
by later applications
• CFG-based approaches often extract this same information from trees
anyway.
Dependency Parsing
• There are two modern approaches to dependency parsing
• Optimization-based approaches that search a space of trees for the
tree that best matches some criteria
• Shift-reduce approaches that greedily take actions based on the
current word and state.
Phrase Structure Tree
Dependency Grammar
• Syntactic structure consists of lexical items,
linked by binary asymmetric relations called
dependencies
• Interested in grammatical relations between
individual words (governing & dependent words)
• Does not propose a recursive structure
• Rather a network of relations
• These relations can also have labels
Draw the dependency tree
• Red figures on the screens indicated falling stocks
Dependency Tree
Dependency Tree Example
• Phrasal nodes are missing in the dependency structure
when compared to constituency structure.
Dependency Tree with Labels
Comparison
• Dependency structures explicitly represent
• Head-dependent relations (directed arcs)
• Functional categories (arc labels)
• Possibly some structural categories (parts-of-speech)
• Phrase structure explicitly represent
• Phrases (non-terminal nodes)
• Structural categories (non-terminal labels)
• Possibly some functional categories (grammatical functions)
Learning DG over PSG
• Dependency Parsing is more straightforward
• Parsing can be reduced to labeling each token wi with wj
• Direct encoding of predicate-argument structure
• Fragments are directly interpretable
• Dependency structure independent of word order
• Suitable for free word order languages (like Indian languages)
Dependency Tree
• Formal definition
• An input word sequence w1…wn
• Dependency graph D = (W,E) where
• W is the set of nodes i.e. word tokens in the input seq.
• E is the set of unlabeled tree edges (wi, wj) (wi, wj є W).
• (wi, wj) indicates an edge from wi (parent) to wj (child).
• Task of mapping an input string to a dependency graph
satisfying certain conditions is called dependency parsing
Well-formedness
• A dependency graph is well-formed iff
• Single head: Each word has only one head.
• Acyclic: The graph should be acyclic.
• Connected: The graph should be a single tree with all the words in
the sentence.
• Projective: If word A depends on word B, then all words between A
and B are also subordinate to B (i.e. dominated by B).
Dependency Parsing
• Dependency based parsers can be broadly categorized
into
• Grammar driven approaches
• Parsing done using grammars.
• Data driven approaches
• Parsing by training on annotated/un-annotated data.
Dependency Parsing
• Dependency based parsers can be broadly categorized
into
• Grammar driven approaches
• Parsing done using grammars.
• Data driven approaches
• Parsing by training on annotated/un-annotated data.
• These approaches are not mutually exclusive
Covington’s Incremental Algorithm
• Incremental parsing in O(n2) time by trying to
link each new word to each preceding one
[Covington 2001]:
PARSE(x = (w1, . . . ,wn))
1. for i = 1 up to n
2. for j = i − 1 down to 1
3.
LINK(wi , wj )
Covington’s Incremental
Algorithm
2
• Incremental parsing in O(n ) time by trying to
link each new word to each preceding one
[Covington 2001]:
PARSE(x = (w1, . . . ,wn))
1. for i = 1 up to n
2. for j = i − 1 down to 1
3.
LINK(wi , wj )
• Different conditions, such as Single-Head and
Projectivity, can be incorporated into the LINK
operation.
Dynamic Programming
• Basic Idea: Treat dependencies as constituents.
• Use, e.g. , CYK parser (with minor modifications)
Dynamic Programming Approaches
• Original version [Hays 1964] (grammar driven)
• Link grammar [Sleator and Temperley 1991] (grammar driven)
• Bilexical grammar [Eisner 1996] (data driven)
• Maximum spanning tree [McDonald 2006] (data driven)
Eisner 1996
• Two novel aspects:
• Modified parsing algorithm
• Probabilistic dependency parsing
• Time requirement: O(n3)
• Modification: Instead of storing subtrees, store
spans
• Span: Substring such that no interior word links to
any word outside the span.
• Idea: In a span, only the boundary words are
active, i.e. still need a head or a child
• One or both of the boundary words can be active
Example
_ROOT_
Red
figures
on
the
screen
indicated
falling
stocks
Example
_ROOT_
Red
figures
on
the
screen
indicated
falling
stocks
Spans:
Red
figures
indicated
falling
stocks
Assembly of correct parse
_ROOT_
Red
figures
on
the
screen
indicated
falling
stocks
Start by combining adjacent words to minimal spans
Red
figures
figures
on
on
the
Assembly of correct parse
_ROOT_
Red
figures
on
the
screen
indicated
falling
stocks
Combine spans which overlap in one word; this word must
be governed by a word in the left or right span.
on the
+
the
screen
→
on
the
screen
Assembly of correct parse
Red
_ROOT_
figures
on
the
screen
indicated
falling
stocks
Combine spans which overlap in one word; this word must
be governed by a word in the left or right span.
figures
on
+
on
the
screen
→
figures
on
the
screen
Assembly of correct parse
_ROOT_
Red
figures
on
the
screen
indicated
falling
stocks
Combine spans which overlap in one word; this word must
be governed by a word in the left or right span.
Invalid span
Red
figures
on
the
screen
Assembly of correct parse
_ROOT_
Red
figures
on
the
screen
indicated
falling
stocks
Combine spans which overlap in one word; this word must
be governed by a word in the left or right span.
indicated
falling
+
falling
stocks
→
indicated
falling
stocks
Classifier-Based Parsing
• Data-driven deterministic parsing:
• Deterministic parsing requires an oracle.
• An oracle can be approximated by a classifier.
• A classifier can be trained using treebank data.
• Learning algorithms:
• Support vector machines (SVM) [Kudo and Matsumoto 2002,
Yamada and Matsumoto 2003,Isozaki et al. 2004, Cheng et al. 2004, Nivre
et al. 2006]
• Memory-based learning (MBL) [Nivre et al. 2004, Nivre and
Scholz 2004]
• Maximum entropy modeling (MaxEnt) [Cheng et al. 2005]
Feature Models
• Learning problem:
• Approximate a function from parser states, represented by feature
vectors to parser actions, given a training set of gold standard
derivations.
• Typical features:
• Tokens and POS tags of :
• Target words
• Linear context (neighbors in S and Q)
• Structural context (parents, children, siblings in G)
• Can not be used in dynamic programming algorithms.
Feature Models
Dependency Parsers for download
• MST parser by Ryan McDonald
• Malt parser by Joakim Nivre
• Stanford parser
Outline
• Dependency Parsing
• Formal definition
• Dynamic programming
• Supervised Classification
• Semantic Role Labeling
• Propbank
• Automatic SRL
• FrameNet
• Semantic Parsing
62
What is PropBank:
From Sentences to Propositions
Powell met Zhu Rongji
battle
wrestle
join
debate
Powell and Zhu Rongji met
Powell met with Zhu Rongji
Powell and Zhu Rongji had
a meeting
consult
Proposition: meet(Powell, Zhu Rongji)
meet(Somebody1, Somebody2)
...
When Powell met Zhu Rongji on Thursday they discussed the return of the spy plane.
meet(Powell, Zhu)
discuss([Powell, Zhu], return(X, plane))
63
Capturing semantic roles
SUBJ
•
Faisal broke [ ARG1 Tom’s chair].
•
SUBJchair] was broken by Lucian.
[ARG1 Monica’s
•
[ARG1 Nitin’s chair] broke into pieces when it fell
down.
SUBJ
64
A TreeBanked Sentence
S
NP
DT
JJ
VP
NN
The Supreme Court
VBD
NP
gave
NNS
states
NP
VBG
working
NN
Leeway
“Non Terminal” (130969 ARGs)
“Terminal”(4246 ARGs)
65
The Same Sentence, PropBanked
S
NP/ARG0
DT
JJ
VP
NN
The Supreme Court
VBD
gave
NNS
VBG
states
Predicate
NP/ARG1
NP/ARG2
NN
working Leeway/ARG0
Predicate
66
Core Arguments
•
•
•
•
•
Arg0 = agent
Arg1 = direct object / theme / patient
Arg2 = indirect object / benefactive / instrument /
attribute / end state
Arg3 = start point / benefactive / instrument /
attribute
Arg4 = end point
67
Secondary ArgMs
•
•
•
•
•
•
•
•
•
•
LOC - where at?
DIR - where to?
MNR - how?
PRP -why?
REC - himself, themselves, each other
PRD -this argument refers to or modifies another
ADV –others
TMP - when?
TPC – topic
ADV –others
68
Distributions of Argument Types
14187
2518
5103
7346
26308
ARG0
91519
ARG1
ARG2
10100
4502
74
11120
ARG3
18703
ARG5
CORE Types
ARG4
EXT
8876
2837
5600
73
2553
3944
DIR
LOC
21670
116628
TMP
REC
PRD
ARGM Types
69
How to Use PropBank: Train a Semantic
Role Labeling System
•
(CONLL 04, CONLL 05 and) Our Goal: Given a list of
(3073) target verbs the system should be able to tag the
possible nodes with semantic role labels (Ji et al., 2005)
70
Predicate Features: Lexical
•
Head Word, Head Pos of (-2,-1,0,1,2) window of Predicate
•
Predicate is a Transitive verb or not
•
Predicate Voice (Passive or not)
• Verb itself: must be in its past particle form
• Passive Context
--Immediately following the verb "be"
--Postmodifying a noun in a reduced relative clause,
"The building damaged by fire".
• Encoding
Conjunction feature of Predicate POS_Passive Context
71
Predicate Sub-Categorization Feature
S
NP/ARG0
DT
JJ
VP
NN
VBD
The Supreme Court gave
NP/ARG1
NP/ARG2
NNS
states
VBG
NN
working Leeway/ARG0
VP->VBD-NP-NP
 The Phrase structure rule expanding the predicate’s grandparent
72
Predicate Pivot Features
S
NP/ARG0
DT
JJ
VP
NN
The Supreme Court
VBD
gave
NP/ARG1
NP/ARG2
NNS
states
VBG
NN
working Leeway/ARG0
vbd_NNS_VBG_NN, v_NNS_VBG_NN, gave_NNS_VBG_NN,
vbd_NP_NP, v_NP_NP, gave_NP_NP
 Consider the predicate as a “Pivot”, and its grandparent’s children
are defined in relation to it.
73
Argument Features: Lexical/Syntactic
•
Head Word, Head Pos, Phrase Type of (-2,-1,0,1,2) window words,
Begin Word, Last Word, Left Sister, Right Sister, Parent of
Argument
(Head of PP replaced by head word of NP inside it)
•
•
•
•
•
•
•
•
Head Pos, Phrase Type of GrandParent
Suffix1, Suffix2, Suffix3
Preceding, Succeeding Node’s Label
Length (Span)
Level from Leaves
Beginning Letter of Phrase Type (for generalization)
Punctuation before/after
If it includes a Preposition or not, and Prep POS
74
Intervene Feature:
Path
S
NP/ARG0
DT
JJ
VP
NN
The Supreme Court
VBD
…
gave
Path: NP S VP VBD gave PartialPath: NP S
InterPath: S VP VBD
PredPathArgPathLens: 3_1
CollapsedPath (delete nodes between clauses): NP S VP VBD gave
PathSOnly(Replace all the non-clause nodes with "*“): NP S * * gave
PathSRemain(only keep clause nodes): NP S gave
PathNGrams: NP|S|VP, S|VP|VBD, VP|VBD|gave, VBD|gave|*, gave|*|*
PathLen: 4
ArgPhraseType, PathLen: NP, 4
PredHeadWord, PathSRemain: gave, NP S gave
75
Intervene Feature:
Position
S
NP/ARG0
DT
JJ
VP
NN
The Supreme Court
VBD
…
gave
Directionality: Left
SameClause: true Dominate Phrase Type: S
Adjacency (Adjacent/not-Adjacent): Adjacent
ArgPhraseType,Adjacency: NP,Adjacent
PositionInClause (Begin, Inside, End): Begin
RelPosition (Distance between Spans): 0
RelPosition,Directionality: 0, Left
RelPosition,Transitive: 0, True
RelPosition,PredHeadPos,PassiveContext: 0, VBD, False
RelPosition,ArgPhraseType: 0, NP
RelPosition,PredPivotv: 0, v_NNS_VBG_NN
76
Intervene Features: Pivot
S
NP/ARG0
DT
JJ
VP
NN
The Supreme Court
VBD
gave
PP
NP/ARG1
NP/ARG2
…
np_v_NP_NP_PP, CUR_v_NP_NP_PP, CUR_gave_NP_NP_PP
 Consider the predicate and candidate argument as “Pivots”, and other
constituents are defined in relation to them
77
Other Features
•
•
•
•
•
•
PredHeadWord, ArgHeadWord
PredHeadWord, ArgPhraseType
ArgPreposition,Transitive
Frequency of VP, NP, SBAR, CC,
“,”, “:”, “”” in the sentence
…
…
Hmm. Haven’t I heard
that word “frame” before?
Yes, it’s intended to
be seen as a
variation on the
word as it’s been
used in various
branches of the
cognitive sciences
in recent decades.
“Frames” Traditions
Let’s locate “our” notion of frame within the various
traditions in the cognitive sciences that use the words
frame or schema (joined, sometimes, with stereotype,
script, scenario or idealized cognitive model) which
appear to be dealing with essentially the same concept.
These terms are used to denote structured sets of
expectations that play a central role in how human beings
create or interpret their experiences.
customer
The noun customer is typically defined as ‘someone who
buys something in a shop or business.’ That includes
everyone I know over the age of 5.
Suppose you overhear somebody say
Sue tends to be rude to customers.
What situation do you imagine?
chicken (mass noun)
• The noun chicken, as a count noun, is the name of a well-known
domestic bird. As a mass noun it is defined as ‘the meat of a chicken
or chickens’.
• What’s wrong with the following sentence?
The fox that lives near our farm
chicken. (compare: likes chickens)
likes
• The image you might get is of a fox eating fried chicken, holding a
knife and a fork, and a napkin, in its paws.
• The products of the lexical construction that yields
mass noun uses of chicken, lamb, duck, turkey,
etc., refer to meats prepared as part of human
cuisine.
*The wolf that lives near our ranch prefers lamb.
Invoking and Evoking Frames
•
•
People invoke (summon
up from their memory)
frames, to make sense of
their experience,
linguistic or otherwise.
a cognitive act
•
•
Words evoke categories
and knowledge
structures that shape
interpreters’
understanding of a text.
a cognitive experience
Warning: this not a standard use of these words.
So,
We need to describe words in terms of the
“framal” background.
If we don’t understand the frame, we don’t
understand
the word,
or why the language needs this word,
or why the speaker chose to use it.
The ideal dictionary should let you
1.
2.
3.
4.
5.
6.
Look up a word
Get a link to a description of the relevant frame, for each of its
meanings, and see the names of the frame’s components
See a display of its combinatory affordances, its valence
possibilities, both semantic and syntactic
Find a collection of example sentences illustrating all of its main
combinatory patterns
Find a list of other words that evoke the same frame
Link to other semantically related frames
Frame examples: Risk
Taking_a_risk:
Protagonist, Action, Harm, Asset
1.
2.
3.
4.
I’m going to risk a swim in the sea.
You’ll risk bankruptcy if you make that investment.
You’re risking your reputation by doing that.
You’re taking a big risk.
Being_at_risk:
Protagonist, Harm, Asset
1.
2.
3.
4.
Buildings in California risk destruction by earthquakes.
Newborns in this hospital run the risk of hypothermia.
We risk our lives every day.
I am at risk of stroke.
Frame examples: Explanation
Communication.explanation:
Speaker, Addressee, Mystery, Account
1.
2.
3.
4.
The coach explained the problem to the team.
The coach explained that they hadn’t learned the maneuvers.
What’s your explanation of these facts?
The defense lawyer gave an inadequate explanation.
Cognition.explanation:
Mystery, Account
1.
2.
What can explain these facts?
A history of unrestricted logging explains the erosion pattern we see here.
Compare: explain & reveal
In their cognitive senses, as opposed to their meanings
as verbs of speaking, the verbs explain and reveal are
near-inverses of each other, where the Mystery and the
Account in the former correspond to Evidence and
Conclusion in the latter.
1.
2.
A history of unrestrained logging
explains
the erosion pattern we see here.
(explains, accounts for)
The erosion pattern we see here
reveals
a history of unrestrained logging.
(reveals, shows, suggests)
Reading the Pictures
 The boxes refer to five-part scenarios consisting of an initial state, a transition, an
intermediary state, another transition, and a final state.
 The writing under the pictures abbreviates particular role names and gives verbs that
evoke instances of the scenario.
 The bold borders indicate a profiling of some portion of the event.
state
transition
state
transition
state
X; return, go back, come back
He returned to Hong Kong.
He returned Tuesday evening after a week’s trip to Australia.
He returned to his home for a few days.
The verb RETURN profiles the time of arrival, but it evokes
the entire frame; other information in the sentences can fill
in some of the details of the larger scenario.
A, X; return, replace, put back
I returned your books this morning.
I returned to your desk the books that I had borrowed last week.
After the earthquake we replaced all the books on the shelf.
frame elements
• Participants and Sub-events .
• Avenger
the one who enacts revenge
• Offender
the original offender
• Injured_party
the offender’s victim
• Injury
the offender’s act
• Punishment
the avenger’s act
grammar
• Components of linguistic form for expressing the
FEs (defining valence).
• Subject
• Direct Object
• Prepositional marking
(by, for, with, on, at, against)
• Subordinate clause marking
(for DOING, by DOING)
Outline
• Dependency Parsing
• Formal definition
• Dynamic programming
• Supervised Classification
• Semantic Role Labeling
• Propbank
• Automatic SRL
• FrameNet
• Semantic Parsing
Learning Semantic Parsers
Training Sentences &
Meaning Representations
Semantic Parser Learner
Sentences
Meaning Representations
Semantic Parser
Data Collection for ATIS
• Air travel planning scenarios (Hirschman, 1992)
You have 3 days for job hunting, and you have arranged job interviews in 2 different
cities! Start from City-A and plan the flight and ground transportation itinerary to City-B
and City-C, and back home to City-A.
• Use of human wizards:
• Subjects were led to believe they were talking to a fully automated
systems
• Human transcription and error correction behind the scene
• A group at SRI responsible for database reference
answers
• Collected more than 10,000 utterances and 1,000
sessions for ATIS-3
Sample Session in ATIS
• may i see all the flights from cleveland to , dallas
• can you show me the flights that leave before noon ,
only
• could you sh- please show me the types of aircraft
used on these flights
Air-Transportation
Show: (Aircraft)
Origin: (City "Cleveland")
Destination: (City "Dallas")
Departure-Time: (< 1200)
Chanel (Kuhn & De Mori, 1995)
• Consists of a set of decision trees
• Each tree builds part of a meaning representation
• Some trees decide whether a given attribute should be
displayed in query results
• Some trees decide the semantic role of a given substring
• Each correpsonds to a query constraint
Chanel (Kuhn & De Mori, 1995)
show me TIME flights from CITY1 to
CITY2 and how much they cost
Tree 1:
Display aircraft_code?
Tree 23:
Display fare_id?
CITY tree:
For each CITY: origin, dest, or stop?
Tree 114:
Display booking_class?
TIME tree:
For each TIME: arrival or departure?
Display Attributes: {flight_id, fare_id}
Constraints: {from_airport = CITY1,
to_airport = CITY2, departure_time = TIME}
Statistical Parsing (Miller et al., 1996)
• Find most likely meaning M0, given words W and history H
(M': pre-discourse meaning, T: parse tree)
• Three successive stages: parsing, semantic
interpretation, and discourse
• Parsing model similar to Seneff (1992)
• Requires annotated parse trees for training
Statistical Parsing (Miller et al., 1996)
/wh-question
flight
/np
arrival
/vp
flight-constraints
/rel-clause
departure
/vp
flight
/corenp
time
/wh-head
/aux
/det
flight
/np-head
When
do
the
flights
departure
/pp
departure departure
/comp /vp-head /prep
that
leave
from
city
/npr
Boston
location
/pp
arrival location
/vp-head /prep
arrive
in
city
/npr
Atlanta
Recent Approaches
• Different levels of supervision
• Ranging from fully supervised to unsupervised
• Advances in machine learning
• Structured learning
• Kernel methods
• Grammar formalisms
• Combinatory categorial grammars
• Sychronous grammars
• Unified framework for handling various phenomena
• Spontaneous speech
• Discourse
• Perceptual context
• Generation
Combinatory Categorial Grammar
• Highly structured lexical entries
• A few general parsing rules (Steedman, 2000; Steedman
& Baldridge, 2005)
• Each lexical entry is a word paired with a category
Texas := NP
borders := (S \ NP) / NP
Mexico := NP
New Mexico := NP
Parsing Rules (Combinators)
• Describe how adjacent categories are combined
• Functional application:
⇒ A
A/B
B
B
A\B
⇒
(>)
(<)
A
Texas
borders
New Mexico
NP
(S \ NP) / NP
NP
>
S \ NP
<
S
CCG for Semantic Parsing
• Extend categories with semantic types
Texas := NP : texas
borders := (S \ NP) / NP : λx.λy.borders(y, x)
• Functional application with semantics:
A/B:f
B:a
B:a
A\B:f
⇒
⇒
A : f(a)
A : f(a)
(>)
(<)
Sample CCG Derivation
Texas
NP
texas
borders
(S \ NP) / NP
λx.λy.borders(y, x)
New Mexico
NP
new_mexico
>
S \ NP
λy.borders(y, new_mexico)
<
S
borders(texas, new_mexico)
Another Sample CCG Derivation
Texas
NP
texas
borders
(S \ NP) / NP
λx.λy.borders(y, x)
New Mexico
NP
mexico
>
S \ NP
λy.borders(y, mexico)
<
S
borders(texas, mexico)
Probabilistic CCG for Semantic
Parsing
Zettlemoyer
& Collins (2005)
• L (lexicon) =
Texas := NP : texas
borders := (S \ NP) / NP : λx.λy.borders(y, x)
Mexico := NP : mexico
New Mexico := NP : new_mexico
• w (feature weights)
• Features:
• fi(x, d): Number of times lexical item i is used in derivation d
• Log-linear model: Pw(d | x)  exp(w . f(x, d))
• Best derviation: d* = argmaxd w . f(x, d)
• Consider all possible derivations d for the sentence x given the
lexicon L
Learning Probabilistic CCG
Training Sentences &
Logical Forms
Lexical Generation
Lexicon L
Parameter Estimation
Feature weights w
Sentences
Logical Forms
CCG Parser
Lexical Generation
• Input:
Texas borders New Mexico
borders(texas, new_mexico)
• Output lexicon:
Texas := NP : texas
borders := (S \ NP) / NP : λx.λy.borders(y, x)
New Mexico := NP : new_mexico
Lexical Generation
Input logical form:
borders(texas, new_mexico)
Input sentence:
Texas borders New Mexico
Output substrings:
Texas
borders
New
Mexico
Texas borders
borders New
New Mexico
Texas borders New
…

Output categories:
NP : texas
NP : new _mexico
(S \ NP) / NP : λx.λy.borders(y, x)
(S \ NP) / NP : λx.λy.borders(x, y)
…
Category Rules
Input Trigger
Output Category
constant c
NP : c
arity one predicate p
N : λx.p(x)
arity one predicate p
S \ NP : λx.p(x)
arity two predicate p
(S \ NP) / NP : λx.λy.p(y, x)
arity two predicate p
(S \ NP) / NP : λx.λy.p(x, y)
arity one predicate p
N / N : λg.λx.p(x)  g(x)
arity two predicate p and
constant c
N / N : λg.λx.p(x, c)  g(x)
arity two predicate p
(N \ N) / NP : λx.λg.λy.p(y, x)  g(x)
arity one function f
NP / N : λg.argmax/min(g(x), λx.f(x))
arity one function f
S / NP : λx.f(x)
Parameter Estimation
• Maximum conditional likelihood
• Derivations d are not annotated, treated as hidden
variables
• Stochastic gradient ascent (LeCun et al., 1998)
• Keep only those lexical items that occur in the highest
scoring derivations of training set
Results
• Test for correct logical forms
• Precision: # correct / total # parsed sentences
• Recall: # correct / total # sentences
• For Geoquery, 96% precision, 79% recall
• Low recall due to incomplete lexical generation:
• Through which states does the Mississippi run?
Context-Free Semantic Grammar
QUERY
QUERY  What is CITY
What
is
CITY
CITY  the capital CITY
CITY  of STATE
STATE  Ohio
the
capital
of
CITY
STATE
Ohio
Sample SCFG Production
Natural language
Formal language
QUERY  What is CITY / answer(CITY)
Sample SCFG Derivation
QUERY
QUERY
Sample SCFG Derivation
QUERY
What
is
QUERY
CITY
answer
(
CITY
QUERY  What is CITY / answer(CITY)
)
Sample SCFG Derivation
QUERY
What
is
the
QUERY
answer
CITY
capital
capital
CITY
of
STATE
Ohio
What is the capital of Ohio
(
CITY
(
loc_2
)
CITY
(
)
STATE
stateid
(
)
'ohio'
)
answer(capital(loc_2(stateid('ohio'))))
Another Sample SCFG Derivation
QUERY
What
is
the
QUERY
answer
CITY
capital
capital
CITY
of
RIVER
Ohio
What is the capital of Ohio
(
CITY
(
loc_2
)
CITY
(
riverid
)
RIVER
(
)
'ohio'
)
answer(capital(loc_2(riverid('ohio'))))
Probabilistic SCFG for Semantic Parsing
• S (start symbol) = QUERY
QUERY  What is CITY / answer(CITY)
CITY  the capital CITY / capital(CITY)
• L (lexicon) =
CITY  of STATE / loc_2(STATE)
STATE  Ohio / stateid('ohio')
• w (feature weights)
• Features:
• fi(x, d): Number of times production i is used in derivation d
• Log-linear model: Pw(d | x)  exp(w . f(x, d))
• Best derviation: d* = argmaxd w . f(x, d)
Learning Probabilistic SCFG
Unambiguous CFG for
Meaning Representations
Training Sentences &
Meaning Representations
Lexical Acquisition
Lexicon L
Parameter Estimation
Feature weights w
Sentences
Meaning Representations
SCFG Parser
Lexical Acquisition
• SCFG productions are extracted from word alignments
between training sentences and their meaning
representations
The
goalie
should
always
stay
in
our
half
( ( true ) ( do our { 1 } ( pos ( half our ) ) ) )
Extracting SCFG Productions
The
goalie
should
always
stay
in
our
half
RULE  (CONDITION DIRECTIVE)
CONDITION  (true)
DIRECTIVE  (do TEAM {UNUM} ACTION)
TEAM  our
UNUM  1
ACTION  (pos REGION)
REGION  (half TEAM)
TEAM  our
Extracting SCFG Productions
The
goalie
should
always
stay
in
TEAM
half
RULE  (CONDITION DIRECTIVE)
CONDITION  (true)
DIRECTIVE  (do TEAM {UNUM} ACTION)
TEAM  our
UNUM  1
ACTION  (pos REGION)
REGION  (half TEAM)
TEAM  our / our
Extracting SCFG Productions
The
goalie
should
always
stay
in
TEAM
half
RULE  (CONDITION DIRECTIVE)
CONDITION  (true)
DIRECTIVE  (do TEAM {UNUM} ACTION)
TEAM  our
UNUM  1
ACTION  (pos REGION)
REGION  (half TEAM)
Extracting SCFG Productions
The
goalie
should
always
stay
in
REGION
RULE  (CONDITION DIRECTIVE)
CONDITION  (true)
DIRECTIVE  (do TEAM {UNUM} ACTION)
TEAM  our
UNUM  1
ACTION  (pos REGION)
REGION  TEAM half / (half TEAM)
Output SCFG Productions
TEAM  our / our
REGION  TEAM half / (half TEAM)
ACTION  stay in REGION / (pos REGION)
UNUM  goalie / 1
RULE  [the] UNUM should always ACTION / ((true) (do
our {UNUM} ACTION))
• Phrases can be non-contiguous
Wong & Mooney (2007b)
Handling Logical Forms with Variables
FORM  state / λx.state(x)
FORM  by area / λx.λy.area(x, y)
FORM  [the] smallest FORM FORM
/ λx.smallest(y, (FORM(x), FORM(x, y))
QUERY  what is FORM
/ answer(x, FORM(x))
• Operators for variable binding
Unambiguous Supervision for Learning
Semantic Parsers
• The training data for semantic parsing consists of
hundreds of natural language sentences unambiguously
paired with their meaning representations
Unambiguous Supervision for Learning
Semantic Parsers
• The training data for semantic parsing consists of
hundreds of natural language sentences unambiguously
paired with their meaning representations
Which rivers run through the states bordering Texas?
answer(traverse(next_to(stateid(‘texas’))))
What is the lowest point of the state with the largest area?
answer(lowest(place(loc(largest_one(area(state(all)))))))
What is the largest city in states that border California?
answer(largest(city(loc(next_to(stateid( 'california'))))))
……
Shortcomings of Unambiguous
Supervision
• It requires considerable human effort to annotate each
sentence with its correct meaning representation
• Does not model the type of supervision children receive
when they are learning a language
• Children are not taught meanings of individual sentences
• They learn to identify the correct meaning of a sentence from
several meanings possible in their perceptual context
“Mary is on the phone”
???
Ambiguous Supervision for Learning
Semantic
Parsers
• A computer system
simultaneously exposed to perceptual contexts
and natural language utterances should be able to learn the
underlying language semantics
• We consider ambiguous training data of sentences associated with
multiple potential meaning representations
• Siskind (1996) uses this type “referentially uncertain” training data to
learn meanings of words
• We use ambiguous training data to learn meanings of sentences
• Capturing meaning representations from perceptual contexts is a
difficult unsolved problem
• Our system directly works with symbolic meaning representations
“Mary is on the phone”
???
???
“Mary is on the phone”
Ironing(Mommy, Shirt)
???
“Mary is on the phone”
Ironing(Mommy, Shirt)
Working(Sister, Computer)
???
“Mary is on the phone”
Ironing(Mommy, Shirt)
Carrying(Daddy, Bag)
Working(Sister, Computer)
???
“Mary is on the phone”
Ambiguous Training Example
Ironing(Mommy, Shirt)
Carrying(Daddy, Bag)
Working(Sister, Computer)
Talking(Mary, Phone)
Sitting(Mary, Chair)
???
“Mary is on the phone”
Next Ambiguous Training Example
Ironing(Mommy, Shirt)
Working(Sister, Computer)
Talking(Mary, Phone)
Sitting(Mary, Chair)
???
“Mommy is ironing shirt”
Sample Ambiguous Corpus
gave(daisy, clock, mouse)
Daisy gave the clock to the mouse.
ate(mouse, orange)
ate(dog, apple)
Mommy saw that Mary gave the
hammer to the dog.
saw(mother,
gave(mary, dog, hammer))
broke(dog, box)
The dog broke the box.
gave(woman, toy, mouse)
gave(john, bag, mouse)
John gave the bag to the mouse.
The dog threw the ball.
threw(dog, ball)
runs(dog)
saw(john, walks(man, dog))
Parsing in Watson
Watson English Slot Grammar (ESG) parser
Deep parser which explores the syntactic and
logical structure to generate semantic clues
Slots
subj
obj
WS(arg)
features
Subj(n) John(1)
noun pron
Top
sold(2,1,4)
verb
Ndet
a(3)
det indef
ndet
John sold a fish
Fig: Slot filling for John sold a fish
fish(4)
noun
Fig: SlotObj(n)
Grammer
Analysis Structure