Tutorial About Probabilistic Learning Models

Benjamin Adrian, Gunnar Grimnes, Jörn Hees, Matthias Sperber

Abstract

Introduction

Classification in general is the problem of deciding for a given input to which class it belongs. Usually classification can be subdivided into a learning phase (aka training phase) and a classification phase (aka test phase). (TODO: offline, online, reinforcement, ... learning)

Relational Classification

Nearest Neighbor (1-NN or NN)

Nearest Neighbor classifiers are classifiers of the most simple kind. In the training phase they simply record the class for each sample. Later in the classification phase they calculate the distances of the query to all samples in their records and return the class of the sample which is closest to the query.

$k$NN

The $k$-Nearest Neighbor classifier is a generalization of the simple NN, which does not immediately return the single "best match" sample's class, but inspects the nearest $k$ samples to the query and returns a class depending on a merging function, such as:

• most often observed class
• classes weighted by inverse distances

Naive Bayes

Naive Bayes Classificators are

TracMath macro processor has detected an error. Please fix the problem before continuing.

The command:

'/usr/bin/pdflatex -interaction=nonstopmode a111c55d9e535d1f2efdb89ba85f72dc19ef7127.tex'

failed with the following output:

"This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2019/dev/Debian) (preloaded format=pdflatex)\n restricted \\write18 enabled.\nentering extended mode\n(./a111c55d9e535d1f2efdb89ba85f72dc19ef7127.tex\nLaTeX2e <2018-12-01>\n(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls\nDocument Class: article 2018/09/03 v1.4i Standard LaTeX document class\n(/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo))\n(/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty)\n\n! LaTeX Error: File `cmap.sty' not found.\n\nType X to quit or <RETURN> to proceed,\nor enter new name. (Default extension: sty)\n\nEnter file name: \n! Emergency stop.\n<read *> \n         \nl.4 \\usepackage\n               {type1ec}^^M\n!  ==> Fatal error occurred, no output PDF file produced!\nTranscript written on a111c55d9e535d1f2efdb89ba85f72dc19ef7127.log.\n"

Maximum Entropy

Multi Layer Perceptrons

Support Vector Machines

Sequential Classification

Hidden Markov Model

Conditional Random Field

Appendix

Mathematical Foundations

Bayes Rule

TracMath macro processor has detected an error. Please fix the problem before continuing.

The command:

'/usr/bin/pdflatex -interaction=nonstopmode c36e0fe50fe8286ec9d8338e59e1fb19adf1b31c.tex'

failed with the following output:

"This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2019/dev/Debian) (preloaded format=pdflatex)\n restricted \\write18 enabled.\nentering extended mode\n(./c36e0fe50fe8286ec9d8338e59e1fb19adf1b31c.tex\nLaTeX2e <2018-12-01>\n(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls\nDocument Class: article 2018/09/03 v1.4i Standard LaTeX document class\n(/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo))\n(/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty)\n\n! LaTeX Error: File `cmap.sty' not found.\n\nType X to quit or <RETURN> to proceed,\nor enter new name. (Default extension: sty)\n\nEnter file name: \n! Emergency stop.\n<read *> \n         \nl.4 \\usepackage\n               {type1ec}^^M\n!  ==> Fatal error occurred, no output PDF file produced!\nTranscript written on c36e0fe50fe8286ec9d8338e59e1fb19adf1b31c.log.\n"