Lecture 0. Basic concepts in Probability Theory

Probability theory enables us to understand uncertainty (variability) in the data. This knowledge helps us make predictions, make better decisions and evaluate risks.

The purpose of the theory of probability is to model random experiments through a probability space. Some useful tools in scientific research are the concepts of random variable, cumulative distribution function, and probability distribution. Some widely known discrete probability distributions are the Bernoulli, Binomial, Negative Binomial, Poisson, etc. distributions, and continuous probability distributions are the Uniform, Normal, Student's t, Chi-square, etc. distributions.

 An area of probability theory focuses its interest in calculating magnitudes (expectation, variance, etc..) associated with a given probability model, and this requires assuming what the correct probability model is. However, in many real situations, the best we can say is that the correct measure of probability is one of those that make up a set of possible probability measures. This set is called statistical model and an inportant area of current research is that of model selection.

The aim of statistical inference is to establish assertions about the characteristics of the true underlying probability measure in the statistical model. These assertions are based on the available statistical information about the model and the observed responses (data). Several kinds of inference are, in principle, possible: (i) estimation or prediction i.e., predicting the value of an unknown response by an estimator. (ii) Construction of a confidence interval i.e., create a subset on sample space (as small as possible) having a high probability of containing an unknown response value. (iii) Hypotheses test i.e., answering if a given observed response is or not a plausible value of the specified probability distribution.

  Different answers to the question on  how to combine the information (the data and the statistical models) for any of these kinds of inferences, lead to different procedimental approaches, and in particular, to those based on the likelihood principle and those based on Bayes' theorem. 

The likelihood-based inferences utilise the likelihood function. In this approach, it is the ratio of the likelihood function for different parameter values that drives the inference. A maximum likelihood estimator (MLE) is a set of statistical model parameters that maximizes the likelihood function.

Bayesian inference incorporates the prior probability distribution as an additional element for use in the inferences about the unknown value of the parameters. Bayesian inference provides a complete framework to infer full (posterior) probability distributions for the model parameters, and to select models.

In all cases, it is important to be cautious checking/validating the assumptions, because their failure to hold could invalidate our inferences.