Analysis of Learning in Support Vector Machines

Introduction

Analysis of Learning in Support Vector Machines is a EPSRC funded project. This project is led by Manfred Opper working with Dörthe Malzahn (the postdoctoral RA). The goal of the project is to apply techniques from statistical mechanics to study analytically the learning and generalization performance of SVMs in typical, rather than worst case, scenarios. The method will not only yield exact results for various quantities of the trained SVM, but will also provide insights into SVM learning. Investigation of model selection for learning with noisy training data will provide invaluable insight for the application of SVMs to real world tasks from which new approaches may emerge. Finally, we will study the ability of SVM training algorithm to deal with large data sets as it is crucial for future applications.

Research Funding


EPSRC: 113,302 GBP

Duration

The project started in the March, 2000, and will last for 2 years.

Project Background

Support-vector machines (SVMs) constitute a new promising paradigm in neural computation and machine learning and are presently one of the most active topics in these research areas. SVMs are nonparametric approaches to supervised learning which avoid the necessity of an a priori specification of the number if adjustable parameters.

Previous theoretical approaches to the generalization ability of SVMs rely mainly on worst case, distribution independent bounds. The tightness of these estimates in less pessimistic, typical cases is not clear and the choice of a suitable SVM model based on them could lead to suboptimal performance.

A more detailed information on SVMs is here.

Objectives

Understand how training noise affects the generalization performance of SVMs and how SVM models can be optimized.
Analyse the use of model selection criteria for finding near optimal SVM parameters.
Devise and analyse sequential versions of SVM learning algorithms for learning large datasets.

Achievements

Learning curves for Gaussian processes regression: A framework for good approximations.

Further Information

We attempt to produce online versions of all relevant publications arising from this project. You can see a list of all the available technical reports by following the link below: