A Theory of Learning and Generalization

Book excerpt: A Theory of Learning and Generalization provides a formal mathematical theory for addressing intuitive questions of the type: How does a machine learn a new concept on the basis of examples? How can a neural network, after sufficient training, correctly predict the output of a previously unseen input? How much training is required to achieve a specified level of accuracy in the prediction? How can one "identify" the dynamical behaviour of a nonlinear control system by observing its input-output behaviour over a finite interval of time? This is the first book to treat the problem of machine learning in conjunction with the theory of empirical processes, the latter being a well-established branch of probability theory. The treatment of both topics side by side leads to new insights, as well as new results in both topics. An extensive references section and open problems will help readers to develop their own work in the field.