next up previous
Next: Bibliography Up: Information Discriminant Analysis (IDA) Previous: Introduction

Example

Application of these functions is illustrated on data set Satellite from the UCI machine learning repository. This data set consists of 6435 samples (each sample is 36-D). The first 4435 samples are used for training, with the remaining samples used for testing. The number of classes is 6. For example, running [pc mu T et] = test_satellite(2,10,1,'lda','tr'); from MATLABTMcommand prompt returns the following:

The input arguments represent:

In case you care, here is the function test_satellite.m posted. Note that the function is specifically tailored to Satellite data. Also note that it is assumed that a file Satimage.mat resides in the same directory as test_satellite.m. This file should contain a variable Data in the form of 6435 x 37 matrix. The last column of this matrix contains class label indicators (integers), e.g. {0, 1  ... , 5}. Type help test_satellite or take a look at the source code for further information on this function. Final note: running test_satellite.m will call classify.m which is a function from MATLABTMStatistics Toolbox. If the toolbox is not installed, these functions (linear and quadratic classifier) can be easily written.

Figure 1: 2-D features corresponding to the training samples of the Satellite data. Colors indicate class memberships.
Image features

Fig. 1 shows the resulting 2-D feature plot. The performances (percent correct) of the linear and quadratic classifiers are 65.90% and 72.45%, respectively. Table 1 shows results for various dimensions of the feature space. IDA is initialized with two methods: LDA and CHE (Chernoff method of Loog and Duin [2]). For m > 5, LDA does not yield a feature extraction matrix, therefore a random matrix was used instead. These results are separated from LDA-initialized results by a horizonal line (see Table 1). The boxed values represent the best results for each classifier-method combination.

Table 1: Performances (% correct) of IDA, the method of Loog and Duin [2] (CHE) and LDA. The options in the parentheses are the number of runs (random restarts), the initialization method (Chernoff,IDA), and the optimization method (trust-region, conjugate-gradient).
m IDA, (10,`che','tr') IDA, (10,`lda','tr') IDA, (10,`lda','cg') CHE LDA
  (L) (Q) (L) (Q) (L) (Q) (L) (Q) (L) (Q)
1 59.20 67.30 59.20 67.30 59.20 67.30 71.45 73.45 52.35 56.50
2 65.90 72.45 65.90 72.45 65.90 72.45 80.75 81.10 75.95 78.35
3 82.15 84.75 82.15 84.75 82.15 84.75 82.00 84.55 82.30 84.10
4 82.30 85.20 82.30 85.15 82.30 85.15 82.20 84.25 82.75 \fbox{84.70}
5 82.25 83.65 82.25 83.65 82.25 83.65 82.25 84.10 \fbox{82.85} 84.50
6 81.80 83.40 81.65 83.25 81.80 83.40 82.05 83.50    
7 82.00 84.15 81.65 84.20 81.65 84.15 82.45 84.25    
8 82.30 83.85 81.55 83.60 81.80 84.60 82.55 84.00    
9 82.65 84.20 82.15 84.15 82.25 84.15 82.70 84.05    
10 82.75 84.15 82.50 84.25 82.60 84.50 82.95 84.35    
11 82.85 83.75 82.15 84.10 82.55 83.95 82.75 84.30    
12 82.80 83.95 82.65 84.25 82.40 84.30 83.00 84.50    
13 82.75 84.10 82.55 84.35 82.60 84.75 82.80 84.35    
14 82.80 83.95 82.70 84.55 82.75 84.45 82.75 84.60    
15 82.80 84.50 82.50 84.85 82.80 84.35 82.80 84.90    
16 82.85 84.50 82.40 84.95 82.90 84.50 82.75 84.55    
17 83.10 84.70 83.05 84.65 83.10 84.70 82.90 84.85    
18 83.20 84.95 83.05 84.85 83.20 84.95 83.00 85.15    
19 \fbox{83.30} 85.10 \fbox{83.20} 85.05 \fbox{83.30} 85.10 83.00 85.10    
20 83.10 84.95 82.90 84.95 82.85 85.00 82.85 \fbox{85.25}    
21 82.85 85.10 82.90 85.15 82.85 85.10 82.65 84.95    
22 83.00 85.20 82.75 85.20 83.00 85.20 82.75 85.05    
23 83.05 85.00 83.05 85.15 83.05 85.00 82.85 85.05    
24 82.95 85.00 82.75 84.95 82.95 85.00 82.75 84.90    
25 82.75 84.85 82.90 85.00 82.75 84.85 82.80 84.95    
26 82.90 84.70 82.90 84.65 82.90 84.65 82.80 84.90    
27 82.85 84.75 83.00 85.00 83.10 84.95 \fbox{83.05} 84.85    
28 82.80 85.00 82.90 84.95 82.80 85.00 82.80 84.80    
29 82.95 85.20 82.85 85.15 82.85 84.95 82.90 84.85    
30 82.90 85.00 82.90 85.05 82.90 85.00 82.90 85.15    
31 82.65 \fbox{85.25} 82.65 \fbox{85.35} 82.65 \fbox{85.25} 82.95 85.10    
32 82.85 85.05 82.85 85.05 82.85 85.05 82.95 84.90    
33 82.90 84.90 82.90 84.90 82.90 84.90 82.80 84.85    
34 82.70 84.85 82.70 84.85 82.70 84.85 82.70 84.90    
35 82.80 84.85 82.80 84.85 82.80 84.85 82.85 84.90    


next up previous
Next: Bibliography Up: Information Discriminant Analysis (IDA) Previous: Introduction
Zoran Nenadic 2007-10-04