A Comparison of the Bayesian regression and the Ordinary Least Squares regression

Abstract

The ordinary least squares regression model assumes that there are enough data to make inference about the parameters. For Bayesian regression however, the data are supplemented with additional information in the form of a prior probability distribution. The prior belief about the parameters is combined with the data's likelihood function according to Bayes theorem to yield the posterior belief about the parameters. The prior can take different functional forms depending on the domain and the information that is available a priori. This study uses simulated data to compare models fitted using the classical regression approach and those that obtained by the Bayesian regression technique.