One common way probabilities are assigned to rules in a PCFG is to use a treebank. When will some rules have a probability of 0.0? What is the problem if some rule has a probability of 0.0? How can it be overcome?
Question:
One common way probabilities are assigned to rules in a PCFG is to use a treebank. When will some rules have a probability of 0.0? What is the problem if some rule has a probability of 0.0? How can it be overcome?
Answer:
When the tree bank
has no instance of a rule among its derivation trees, then the rule will have a
probability of 0.0.
The problem is that
the probability for any parse tree using a rule with probability 0.0 will be 0.0
since the probability of the parse tree is obtained as a product of all rules
in the parse. [Refer here for more]
The same smoothing methods
used to handle zero counts in language models can be used here. For example,
Laplace smoothing which starts with a count of 1 for all rules.
*************************
Related questions:
One common way probabilities are assigned to rules in a PCFG is to use a treebank. When will some rules have a probability of 0.0? What is the problem if some rule has a probability of 0.0? How can it be overcome?
No comments:
Post a Comment