Example

Application of these functions is illustrated on data set Satellite from the UCI machine learning repository. This data set consists of 6435 samples (each sample is 36-D). The first 4435 samples are used for training, with the remaining samples used for testing. The number of classes is 6. For example, running [pc mu T et] = test_satellite(2,10,1,'lda','tr'); from MATLAB^TMcommand prompt returns the following:

• The classification accuracy pc, determined by the linear and quadratic Bayesian classifiers.
• The value of the $\mu$-measure mu in the optimal feature space.
• The optimal feature extraction matrix T.
• The elapsed time et.

The input arguments represent:

• The size of the feature space m = 2 (passed to (2)).
• The number of optimization runs Nruns = 10 (passed to (2)).
• Plot indicator PlotFlag = 1 to enable the plotting of features.
• Initial condition (initial choice of feature extraction matrix) is set to linear discriminant analysis (LDA) matrix (passed to (2)).
• Choice of optimization method is set to trust-region method (functions (2) and (1) will be called).

In case you care, here is the function test_satellite.m posted. Note that the function is specifically tailored to Satellite data. Also note that it is assumed that a file Satimage.mat resides in the same directory as test_satellite.m. This file should contain a variable Data in the form of 6435 x 37 matrix. The last column of this matrix contains class label indicators (integers), e.g. {0, 1  ^... , 5}. Type help test_satellite or take a look at the source code for further information on this function. Final note: running test_satellite.m will call classify.m which is a function from MATLAB^TMStatistics Toolbox. If the toolbox is not installed, these functions (linear and quadratic classifier) can be easily written.

Figure 1: 2-D features corresponding to the training samples of the Satellite data. Colors indicate class memberships.

Fig. 1 shows the resulting 2-D feature plot. The performances (percent correct) of the linear and quadratic classifiers are 65.90% and 72.45%, respectively. Table 1 shows results for various dimensions of the feature space. IDA is initialized with two methods: LDA and CHE (Chernoff method of Loog and Duin [2]). For m > 5, LDA does not yield a feature extraction matrix, therefore a random matrix was used instead. These results are separated from LDA-initialized results by a horizonal line (see Table 1). The boxed values represent the best results for each classifier-method combination.

Table 1: Performances (% correct) of IDA, the method of Loog and Duin [2] (CHE) and LDA. The options in the parentheses are the number of runs (random restarts), the initialization method (Chernoff,IDA), and the optimization method (trust-region, conjugate-gradient).

m	IDA, (10,`che','tr')		IDA, (10,`lda','tr')		IDA, (10,`lda','cg')		CHE		LDA
	(L)	(Q)	(L)	(Q)	(L)	(Q)	(L)	(Q)	(L)	(Q)
1	59.20	67.30	59.20	67.30	59.20	67.30	71.45	73.45	52.35	56.50
2	65.90	72.45	65.90	72.45	65.90	72.45	80.75	81.10	75.95	78.35
3	82.15	84.75	82.15	84.75	82.15	84.75	82.00	84.55	82.30	84.10
4	82.30	85.20	82.30	85.15	82.30	85.15	82.20	84.25	82.75
5	82.25	83.65	82.25	83.65	82.25	83.65	82.25	84.10		84.50
6	81.80	83.40	81.65	83.25	81.80	83.40	82.05	83.50
7	82.00	84.15	81.65	84.20	81.65	84.15	82.45	84.25
8	82.30	83.85	81.55	83.60	81.80	84.60	82.55	84.00
9	82.65	84.20	82.15	84.15	82.25	84.15	82.70	84.05
10	82.75	84.15	82.50	84.25	82.60	84.50	82.95	84.35
11	82.85	83.75	82.15	84.10	82.55	83.95	82.75	84.30
12	82.80	83.95	82.65	84.25	82.40	84.30	83.00	84.50
13	82.75	84.10	82.55	84.35	82.60	84.75	82.80	84.35
14	82.80	83.95	82.70	84.55	82.75	84.45	82.75	84.60
15	82.80	84.50	82.50	84.85	82.80	84.35	82.80	84.90
16	82.85	84.50	82.40	84.95	82.90	84.50	82.75	84.55
17	83.10	84.70	83.05	84.65	83.10	84.70	82.90	84.85
18	83.20	84.95	83.05	84.85	83.20	84.95	83.00	85.15
19		85.10		85.05		85.10	83.00	85.10
20	83.10	84.95	82.90	84.95	82.85	85.00	82.85
21	82.85	85.10	82.90	85.15	82.85	85.10	82.65	84.95
22	83.00	85.20	82.75	85.20	83.00	85.20	82.75	85.05
23	83.05	85.00	83.05	85.15	83.05	85.00	82.85	85.05
24	82.95	85.00	82.75	84.95	82.95	85.00	82.75	84.90
25	82.75	84.85	82.90	85.00	82.75	84.85	82.80	84.95
26	82.90	84.70	82.90	84.65	82.90	84.65	82.80	84.90
27	82.85	84.75	83.00	85.00	83.10	84.95		84.85
28	82.80	85.00	82.90	84.95	82.80	85.00	82.80	84.80
29	82.95	85.20	82.85	85.15	82.85	84.95	82.90	84.85
30	82.90	85.00	82.90	85.05	82.90	85.00	82.90	85.15
31	82.65		82.65		82.65		82.95	85.10
32	82.85	85.05	82.85	85.05	82.85	85.05	82.95	84.90
33	82.90	84.90	82.90	84.90	82.90	84.90	82.80	84.85
34	82.70	84.85	82.70	84.85	82.70	84.85	82.70	84.90
35	82.80	84.85	82.80	84.85	82.80	84.85	82.85	84.90