A19 - Probabilistic Classification

Linear discriminant analysis is a classification technique by which one creates a discriminant function from predictor variables.

In discriminant analysis, there is a dependent variable (Y) which is the group and the independent variables (X) which are the object features that might describe the group. If the groups are linearly separable, then we can use linear discriminant analysis. This method suggests that the groups can be separated by a linear combination of features that describe the objects.

In this activity, i used LDA to classify pillows (chocolate-coated snack) and kwek-kwek (orange, flour-coated quail egg) based on the features such as pixel area, length-to-width ratio, average red component (NCC), and average green component (NCC).

Images of the two samples were taken using Olympus Stylus 770SW. The images are then white balanced using reference white algorithm with the white tissue as the reference. This is to maintain uniform tissue color in each of the images. The images are then cut such that a single image contains a single sample. Features are then extracted from each of the cut images in the same manner as the previous activity.

The data set is again divided into training and test sets, with the first four images of each sample comprising the training set while the last four images for the test set.

Following the discussion and equations from Pattern_Recognition_2.pdf by Dr. S. Marcos, I computed the following values.

An object is assigned to a class with the highest f value. As can be seen from the table below, most of group 1 (pillows) have higher f1 values while those in group 2 (kwek-kwek) have higher f2 values. 100% classification is obtained.

---
//code

TRpillow=fscanfMat('TrainingSet-Pillows.txt');
TSpillow=fscanfMat('TestSet-Pillows.txt');

TRkwek=fscanfMat('TrainingSet-Kwekkwek.txt');
TSkwek=fscanfMat('TestSet-Kwekkwek.txt');

TRpillow=TRpillow';
TSpillow=TSpillow';
TRkwek=TRkwek';
TSkwek=TSkwek';

TRpilmean=mean(TRpillow,1);
TRkwekmean=mean(TRkwek,1);

globalmean=(TRpilmean+TRkwekmean)/2;
globalmean=mtlb_repmat(globalmean,4,1);

TRpillow=TRpillow-globalmean;
TRkwek=TRkwek-globalmean;

c_pil=((TRpillow')*TRpillow)/4;
c_kwek=((TRkwek')*TRkwek)/4;

C=((4*c_pil)+(4*c_kwek))/8;
P = [0.5;0.5];
for i=1:4
f1(i) = (TRpilmean*inv(C)*(TSkwek(i,:)'))-((0.5)*(TRpilmean)*inv(C)*(TRpilmean')+log(P(1)));
f2(i) = (TRkwekmean*inv(C)*(TSkwek(i,:)'))-((0.5)*(TRkwekmean)*inv(C)*(TRkwekmean')+log(P(2)));
end

//end code

---
Thanks Jeric Tugaff for the tips and discussions and Cole Fabros for the images of the sample.

---
Rating 8.5/10 since I implemented and understood the technique correctly but was late in posting this blog entry.

A18 - Pattern Recognition

Pattern recognition, as a subtopic of machine learning, aims to classify data according to a statistical information extracted from its patterns.

A pattern is usually a group of measurements or observations extracted from the data set. In essence, a pattern is a set of quantifiable features. These features are then arranged into an ordered set to define a feature vector. Feature vectors, then, define the grouping of the data into classes.

In pattern recognition, the specific goal is to decide if a given feature vector belongs to one of the several classes.

In this activity, we aim to classify a set of images into one of the four classes - kwek-kwek (orange, flour-coated quail egg), squid balls, piatos potato chips, pillows (chocolate coated snack).

Images of the four samples were taken using Olympus Stylus 770SW. The images are then white balanced using reference white algorithm with the white tissue as the reference. This is to maintain uniform tissue color in each of the images. The images are then cut such that a single image contains a single sample.

FEATURE VECTOR
For each of the individual sample, the features extracted are as follows:
1. pixel area
2. length-to-width ratio
3. average red component (NCC)
4. average green component (NCC)

To get the pixel area, the sample images were first binarized to separate ROI from the background. Closing operation is then performed on the binarized images to reduce effect of poor thresholding. The pixel area is then equal to the sum of the (binarized then closed) image.

Length-to-width ratio is computed as the maximum filled pixel coordinate along the x-axis minus the minimum filled pixel coordinate along x-axis divided by its y-axis counterpart.

The average red component and green component is obtained from the ROI only using Normalized Chromaticity Coordinates.

The following table summarizes the features extracted for all the sample images.

MINIMUM DISTANCE CLASSIFICATION
The data set is divided into two - training and test sets. The training set is composed of the first four images from each classes while the test set are the last four images for each sample.
(Note: Pixel Area was first normalized before subjecting to minimum distance classification.)

To facilitate classification of the test images into one of the four classes, we use minimum distance classification.

This is done by getting the mean feature vector from the training set for each class. Therefore, we get the mean of the training set by getting the mean pixel area, mean L/W, mean r, mean g for each of the four classes.

Classification is done by getting the Euclidean distance of an unknown feature vector from the mean feature vector of each class. The unknown feature vector then belongs to the class with which it has the smallest distance.

In this activity, we use the test set to check for the validity of minimum distance classification as a pattern recognition algorithm. The following results were obtained.

From the matrix above, it is shown that among the 16 samples of the test set, only 1 was misclassified as belonging to another class.

---
Thanks Benj for the discussions with regards to this activity and Cole for the sample images.

---
Rating: 8.5/10 since I implemented the task correctly but was late in posting this blog entry.

A16 - Color Image Segmentation

In image segmentation, a region of interest (ROI) is picked out from the rest of the image such that further processing can be done on it. Selection rules are based on features unique to the ROI.

In this activity, we use color as a feature for segmenting images. To do so, we first transform the color image's RGB into rgI. rgI is part of the normalized chromaticity coordinates or NCC color space. This color space separates chromaticity (r g) and brightness (I) information.

Per pixel, let I = R+G+B. Then the normalized chromaticity coordinates are

r = R/I
g = G/I
b = B/I

---



*http://images.replacements.com/images/images5/china/H/homer_laughlin_fiesta_shamrock_green_mug_P0000201549S0044T2.jpg

---
Parametric Probability Distribution

Segmentation based on color can be performed by determining the probability that a pixel belongs to a color distribution of interest. This implies that the color histogram of the region of interest must first be extracted. To do so, one crops a subregion of the ROI and compute the histogram from it. The histogram when normalized by the number of pixels is already the probability distribution function (PDF) of the color. To tag a pixel as belonging to a region of interest or not is to find its probability of belonging to the color of the ROI. Since our space is r and g we can have a joint probability p(r) p(g) function to test the likelihood of pixel membership to the ROI. We can assume a Gaussian distribution independently along r and g, that is, from the r-g values of the cropped pixel we compute the mean μr , μg and standard deviation σr , σg from the pixel samples. The probability that a pixel with chromaticity r or g belongs to the ROI is then

The joint probability is just the product of p(r) and p(g).

Shown below is the cropped region of the colored image, wherein we get the mean and standard deviation for both r and g so as to get the probability that a certain pixel belongs to the ROI.



We perform this over all pixels and plot the probability that a pixel belongs to our ROI.
Here's the distribution



And here's the segmented image, wherein the white parts corresponds to the pixels belonging to the ROI according to our Gaussian PDF.



---

Nonparametric Segmentation (Histogram Backprojection)

In non-parametric estimation, the histogram itself is used to tag the membership of pixels. Histogram backprojection is one such technique where based on the color histogram, a pixel location is given a value equal to its histogram value in chromaticity space.

Here's the histogram used

and the corresponding segmented image



Nonparametric image segmentation produced better results as compared to the parametric segmentation. Also, in nonparametric segmentation, we never assumed normality of distribution of the r and g, thus, this method is more accurate as compared to parametric segmentation.

---
Reference:
Activity 16 Lecture Handouts by Dr. Maricor Soriano.

---
Thanks to Ed for the valuable discussions and arguments with regards to the technique.

---
I give myself a 10 for I implemented image segmentation quite well. :)