Statistical foundations of machine learning

(1) Foundations of statistical modelling, (2) parametric estimation, (3) nonparametric estimation and resampling, (4) supervised learning ( model selection, variable selection), (5) algorithms for regression (neural networks, local learning, (6) classification algorithms (KNN, Naive- Bayes, SVM), (vii) applications of machine learning (data mining, text mining, web mining)

Statistical machine learning is the discipline which aims at extracting knowledge and inferring predictive models from observed data. The course will focus on the statistical notions (like bias, variance, regression, validation) which are necessary to create, identify and assess a predictive model. This course aims to find a good balance between theory and practice by situating most of the theoretical notions in a real context with the help of illustrative case studies (from biology, finance, medicine) and real datasets.

Required knowledge and skills

• Basic notions of probability and estimation (bias, variance)
• Linear algebra and numerical analysis (linear systems, eigenvalues)
• Least-squares
• Programming

Courses requiring this course

Lectures, exercises and practical assignments. All the assignments involve practical work using the software package R.

Contribution to the teaching profile

• Analysis and modelling of information

• Collect, analyse, discuss and interpret data

• Learning of new concepts

• Design a modelling procedure

• Critical analysis of the results wrt state-of-the-art

• Operation knowledge of English

• Conceive a structural solution and algorithms to solve a problem

• Implement a prototype

Pour les étudiants MA-BINF:

• Maîtriser les approches mathématiques, statistiques et informatiques sur lesquelles se fondent les études bioinformatiques et de modélisation

• Mettre en œuvre une démarche scientifique depuis la conception d'un projet jusqu'à la validation des résultats scientifiques obtenus pour résoudre des problèmes complexes.

• Comprendre l'abstraction et son rôle dans l'élaboration d'une théorie ou d'un modèle

• Comprendre comment se dégage un concept à partir d'observations.

• Apprendre à interagir avec des chercheurs de différents domaines (informatique, biologie,

  bioingénierie, physique, etc.) et à travailler et communiquer en équipe.

References, bibliography, and recommended reading

• G. Bontempi "Statistical foundations of machine learning" (handbook)

• L. Wasserman (2004) All of statistics: a coincise course in statistical inference. Springer.

• R. O. Duda, P.E. Hart, D. G. Stork (2001) Pattern Classification. Wiley.

• T. Hastie, R. Tibshirani, J. Friedman (2001) The elements of statistical learning: data mining, inference, and prediction. Springer.

Course notes

• Syllabus
• Université virtuelle

Additional information

The class is alternatively taught every second year in ULB (Pr. G. Bontempi) and VUB (Pr. B. Manderick).

In 2020-21 the teacher is Pr. Bontempi (ULB). 

Contacts

Email: gbonte@ulb.ac.be

Office: Campus La Plaine, NO8-107

Postal address: Département d'Informatique, Bld de Triomphe, CP 212

Campus

Plaine

Method(s) of evaluation

• Other

Project (in R language) and written exam on theoretical aspects of the course.

Mark calculation method (including weighting of intermediary marks)

• 10/20 (project)
• 10/20 (oral assessment)

Language(s) of evaluation

• english
• (if applicable french )