A Syntactic Justification for Occam's razor

Woodward, John and Evans, Andrew and Dempster, Paul (2008) A Syntactic Justification for Occam's razor. In: 2008 Midwest, A New Kind of Science Conference. UNSPECIFIED, USA.

[img]
Preview
PDF (occamNKS)
occamNKS.pdf - Accepted Version
Available under License Creative Commons Attribution-NonCommercial-NoDerivs.

Download (274kB)

Abstract

Informally, Occam’s razor states, “Given two hypotheses which equally agree with the observed data, choose the simpler”, and has become a central guiding heuristic in the empirical sciences and in particular machine learning. We criticize previous arguments for the validity of Occam’s razor. The nature of hypotheses spaces is explored and we observe a correlation between the complexity of a concept yielded by a hypothesis and the frequency with which it is represented when the hypothesis space is uniformly sampled. We argue that there is not a single best hypothesis but a set of hypotheses which give rise to the same predictions (i.e. the hypotheses are semantically equivalent), whereas Occam’s razor suggests there is a single best hypothesis. We prefer one set of hypotheses over another set because it is the larger set (and therefore the most probable) and the larger set happens to contain the simplest consistent hypothesis. This gives the appearance that simpler hypotheses generalize better. Thus, the contribution of this paper is the justification of Occam’s razor by a simple counting argument.

Item Type:
Contribution in Book/Report/Proceedings
ID Code:
133357
Deposited By:
Deposited On:
20 May 2019 13:40
Refereed?:
Yes
Published?:
Published
Last Modified:
20 Sep 2020 06:45