Machine Learning

Winter Term 2011/12

Course: Machine Learning (MICS, 3rd semester; 2h/week, 4 ECTS)

ORGANISATION

Prof. Dr. Christoph Schommer with Sviatlana Danilava, Mihail Minev, and Jayanta Poray

OBJECTIVE

Originally, Machine Learning has been a research field of Articial Intelligence but has nowadays established as constant in the computation and simulation of human intelligence. In literature, Machine Learning refers to adapt to new circumstances and to detect and extrapolate patterns". Furthermore, it is concerned with the question on how to construct computer programs that automatically improve with experience". In this course, we deepen the central aspect of learning, which is applied in many situations and which plays a major part both for Information Retrieval and Knowledge Discovery/Data Mining. The course is split into 3 parts, which is composed of reading including a discussion, a paper review, and exercises.

Introduction and Overview.
Markov Models & Conditional Random Fields.
Selected Models of Artificial Neural Networks.
Support Vector Machines.
Wrap-up; presentation of potential Master Theses.

AIMS OF QUALIFICATION

The course is offered in the third semester of the Master program MICS; it is part of the specialisation Adaptive Computing and counts for 4 ECTS (= 2h per week). The course is a mixture of a reading, paper reading and review, and an exercise. No examination is written at the end. The course is a continuation of Knowledge Discovery and Data Mining. We concern more in detail algorithms with respect to the four machine learning applications: clustering, classication, time series, and association discovery.

LITERATURE

S. J. Russell, P. Norvig. Artificial Intelligence: A Modern Approach, Prentice Hall, 1995.
T. M. Mitchell. Machine Learning. McGraw-Hill, New York, 1997.
L. V. Fausett. Fundamentals of Neural Networks: Architectures, Algorithms and Applications. Prentice Hall, US Edition, 1993.
D. E. Rumelhart, J. L. McClelland and the PDP Research Group. Parallel Distributed Processing Vol. 1 Foundations. MIT Press. 1986.
J. L. McClelland, D. E. Rumelhart and the PDP Research Group (1986). Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. 2: Psychological and Biological Models. Cambridge, MA: MIT Press.
J. Hertz, A. Krogh, R. Palmer. Introduction to the Theory of Neural Computation. Addison Wesley Publicing Company. 1991.
C. M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 1995.
S. Haykin: Neural Networks. A Comprehensive Foundation, Prentice Hall Publishers, 1998.
W. N. Venables, B.D. Ripley: Modern Applied Statistics with S. Springer- Verlag, 2002.
S. Pinker, J. Mehler. Connections and Symbols. Cambridge MA: MIT Press.
J. L. Elman, E. A. Bates, M. H. Johnson, A. Karmilo-Smith, D. Parisi, K. Plunkett. Rethinking Innateness: A connectionist perspective on development. Cambridge MA: MIT Press.
G. F. Marcus. The Algebraic Mind: Integrating Connectionism and Cognitive Science (Learning, Development, and Conceptual Change). Cambridge, MA: MIT Press.
V. Kecman. Learning and Soft Computing: Support Vector Machines, Neural Networks, and Fuzzy Logic Models.The MIT Press. 2001.

Winter Term 2010/11

GENERAL

The course is offered with 2h/week; it counts for 4 ECTS.
In Winter 2010/11, we have studied the principles of Artificial Neural Networks.
The discussions have based on the script of Dr. Thomas Stibor, given at the TU Darmstadt.
The Organisation of the course has been as follows: reading (40%) - exercises (20%) - project (40% / presentation & implementation).

ORGANISATION

Prof. Dr. Christoph Schommer with Jayanta Poray

COURSE CONTENTS

Theoretical Concepts

Introduction and Overview
Gradients: Lagrangian Multipliers - Discrimination Function - Gradients, Direction of greatest increase. - Steepest Descent Algorithm
Single Layer Networks: Activation Function - Bayes' Theorem & Decision Theorem - Perceptron: Training, Error Reduction, Algorithm - Linear Separability
Multi-Layer Networks:Backpropation - Demonstration - Backpropagation Network - Presentation of the projects - Hopfield Neural Networks - An Auto-associative Memory
Winner-take-all Networks: SOMs - k-means algorithm - Practical: Connectionism with respect to Natural Language Understanding. - Waltz and Pollack: Massively Parallel Parsing

Exercises

Introduction to ENCOG Library

Encog is an advanced neural network and machine learning framework. It contains classes to create a wide variety of networks, as well as support classes to normalize and process data for these neural networks.
Encog trains using multithreaded resilient propagation. Encog can also make use of a GPU to further speed processing time. A GUI based workbench is also provided to help model and train neural networks.

Implementation Project

Find the project description here: ML_Projects_2010.pdf

Winter Term 2009/10

ORGANISATION

Prof. Dr. Christoph Schommer with Maria Biryukov, Sascha Kaufmann, and Jayanta Poray

OBJECTIVE

The course is a mixture of lecture, exercise, and seminar presentation (30h in total) and counts for 4 ECTS. The course takes place on mondays – beginning by September 14, 2009 – from 15h45 to 17h15 in Room A14.

CONTENT

Course Overview.
Supervised Learning.
Supervised Learning.
Exercises – Supervised Learning: Exercises-
Unsupervised Learning.
Unsupervised Learning.
Unsupervised Learning: Exercises
Reinforcement Learning.
Reinforcement Learning.
Sub-symbolic Learning.
Exercises – Sub-symbolic Learning.
Seminar – individual scientific paper publication.
Seminar – individual scientific paper publication.
Summary of the course or Seminar – individual scientific paper publication.

"Machine Learning" is mentioned on: Courses | Knowledge Discovery and Data Mining