University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Automatic relevance determination with ensembles of Bayesian MLPs.

Fu, Yu. (2008) Automatic relevance determination with ensembles of Bayesian MLPs. Doctoral thesis, University of Surrey (United Kingdom)..

Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (18MB) | Preview


The problem of controlling model complexity and data complexity are fundamental issues in neural network learning. Some researchers have used Bayesian-learning on neural networks to control the model complexity. The Bayesian-based technique, Automatic Relevance Determination (ARD), can effectively control the complexity of data, and automatically determine the relevance of input features by controlling the distribution of corresponding groups of weights in a network. However, we found that the relevance determination made by a single ARD model is not stable and accurate. Neural network ensemble techniques were used in our research to improve the accuracy of feature relevance determination. The accuracy of the ensemble feature relevance determination was evaluated using two synthetic datasets in which the relevance of each individual input feature was pre-determined. The results showed that ensemble feature relevance determination can effectively separate relevant features, redundancies and irrelevant features from each other, and provide useful suggestions of the boundaries between these relevance levels. Thus, the features selected, based on the ensemble feature relevance determination, benefits not only non-linear models such as neural networks, but also linear models such as the linear regression model, by enabling them to classify the samples in several real-world datasets more accurately than by using all the available input features, extracted principal components and independent components from the datasets. We also found that an ensemble of ARD models is good at selecting group relevance features, but not at ranking the relevance for each individual input feature, because the relevance rank determination of an input feature can be affected by any redundancies in the data which are highly correlated with it.

Item Type: Thesis (Doctoral)
Divisions : Theses
Authors :
Fu, Yu.
Date : 2008
Contributors :
Depositing User : EPrints Services
Date Deposited : 09 Nov 2017 12:15
Last Modified : 16 Jan 2019 19:00

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800