Natural language processing question bank 02

Why is it better to compute likelihood and probability values in log space?

Question:

Why is it better to computer likelihood and probability values in log space?

Answer:

Computing likelihood and probability values in log space solves the underflow problem (refer below). Moreover, when computing in log space, we can replace the multiplication operations by addition operations, which are usually faster than multiplication operations on a modern computer.

What is Underflow?

Joint probability distribution often involves multiplying small individual probabilities. Many probabilistic algorithms involve multiplying probabilities of individual data points that takes the result very close to zero. This is called as underflow.

*************************

Related questions:

• Why is it better to compute likelihood and probability values in log space?

• One common way probabilities are assigned to rules in a PCFG is to use a treebank. When will some rules have a probability of 0.0? What is the problem if some rule has a probability of 0.0? How can it be overcome?

• You are building an ngram model of a corpus. Should you stem the words and do the counts or leave them in the surface form? Give pros and cons and include what characteristics of the corpus might influence your decision.

• List down the features of PCFGs

• How many parameters are required to fully define a Hidden Markov Model?  

• Differentiate between generative and discriminative models.