Presentation Thesis Uniovi

Introduction How to Apply NMF What NMF Model? NMF vs. FCM Bi-clustering Olfactory Images Results & Conclusions

Bi-clustering Olfactory ImagesUsing Non-negative Matrix Factorizing

Milad Avazbeigi

European Centre for Soft Computing & University of Oviedo

July 22, 2011

Milad Avazbeigi Bi-clustering Olfactory Images


Table of Contents

1 Introduction

2 How to Apply NMF

3 What NMF Model?

4 NMF vs. FCM

5 Bi-clustering Olfactory Images

6 Results & Conclusions



Matrix Factorizing

Factorizing a data matrix is extensively studied in NumericalLinear Algebra. There are different methods for factorizing adata matrix:

Matrix Factorizing Methods1 Principal Components Analysis (PCA)2 Vector Quantization (VQ)3 Non-negative Matrix Factorization (NMF)



Formulation of PCA, QA and NMF I

Given a matrix V , the goal is to find matrix factors W and Hsuch that:

V ≈WH

or

Viµ ≈ (WH)iµ =

K∑a=1

WiaHaµ

whereV is a n×m matrix where m is the number of examples andn is related to the number of dimensions of each data set.W : n×KH: K ×m



Formulation of PCA, QA and NMF II

Viµ ≈ (WH)iµ · · ·... V

...· · ·

n×m

=

· · ·... W

...· · ·

n×K

×

· · ·... H

...· · ·

K×m

Columnar RepresentationV ≈WH can be rewritten column by column as v ≈Wh:

...v...

n×1

=

· · ·... W

...· · ·

n×K

×

...h...

K×1



Formulation of PCA, QA and NMF III

Columns as ImagesIn other words, each data vector v (For example an image)is approximated by a linear combination of the columns ofW , weighted by the components of h.Since every column of W has n elements, they also couldbe interpreted as images.



Differences Among VQ, PCA and NMF:Vector Quantization

Each column of H is constrained to be a unary vector, withone element equal to unity and the other elements equal tozero.In other words, every image (column of V ) is approximatedby a single basis image (column of W ) in the factorizationV ≈WH.



Differences Among VQ, PCA and NMF:Principal Components Analysis

PCA constrains the columns of W to be orthonormal andthe rows of H to be orthogonal to each other.PCA relaxes the unary constraint of V Q, allowing a dis-tributed representation in which each image is approximatedby a linear combination of all the basis images, or eigenim-ages.Many of the eigenimages do not have an obvious visual in-terpretation. This is because PCA allows the entries of Wand H to be of arbitrary sign.



Differences Among VQ, PCA and NMF:Non-negative Matrix Factorization I

NMF does not allow negative entries in the matrix factorsW and H.Non-negativity is a very useful property in comparison withPCA that allows arbitrary signs of pixels and loses the inter-pretability of the results specially in images data that haveonly positive values.Unlike the unary constraint of V Q, non-negativity constraintspermit the combination of multiple basis images to repre-sent an image. But only additive combinations are allowed,because the non-zero elements of W and H are all positive.In contrast to PCA, no subtractions can occur.



Differences Among VQ, PCA and NMF:Non-negative Matrix Factorization II

For these reasons, the non-negativity constraints are com-patible with the intuitive notion of combining parts to form awhole, which is how NMF learns a parts-based representa-tion.



Parts-based Representation

There is psychological and physiological evidence for parts-based representations in the brain, and certain computa-tional theories of object recognition rely on such represen-tations.But little is known about how brains or computers mightlearn the parts of objects.Non-negative matrix factorization that is able to learn partsof images and semantic features of text provides us thechance to explore this property. Non-negativity constraintslead to a parts-based representation because they allowonly additive, not subtractive, combinations.This is in contrast to other methods, such as principal com-ponents analysis (PCA) and vector quantization (VQ), thatlearn holistic, not parts-based, representations.



Applications of NMF

NMF has been shown to be useful in a variety of appliedsettings, including:

ChemometricsPattern recognitionMultimedia data analysisText miningDNA gene expression analysisProtein interaction



Uniqueness of NMF Solution

The factorization is not unique.As an example, a matrix and its inverse can be used totransform the two factorization matrices:

WH = WBB−1H

If the two new matrices W̃ = WB and H̃ = B−1H are non-negative they form another parametrization of the factoriza-tion.



Table of Contents

1 Introduction

2 How to Apply NMF

3 What NMF Model?

4 NMF vs. FCM





How to Apply NMF for Bi-clustering Images

In order to apply NMF for bi-clustering some images, thefollowing steps should be passed:

Data Prepration1 Segmentation of Images.2 Removal of Background Pixels.3 Vectorizing Images to form Matrix V in NMF.4 Normalization of V to have only positive values.



Segmentation and Removal of Background

The first step is the segmentation of the images and deter-mining background pixels.After segmentation, those pixels that are active in all imagesare selected and the rest of them are removed to reduce thedimensions.The data set has 470 images in which every image is a80×44-pixel image. The number of active pixels is equal to1780 out of 3520 pixels.



Segmentation and Removal of Background: Example

0 10 20 30 40 50 60 70 800

5

10

15

20

25

30

35

40

45

−100

−90

−80

−70

−60

−50

−40

−30

−20

−10

0



Vectorizing Images

Since, we are working with 2-D images, the set of all imagesforms a 3-D data set.In order to apply NMF, it is mandatory to re-form the imagesin such a way that all the data are presented in a 2-D matrix(V ).



Vectorizing Images: Example

020406080100120140160180200

0

10

20

30

40

50

60

70

80

90

100



Analysis of olfactory images

Analysis of olfactory images is so important since it is believedthat it would help us to understand the olfactory coding. A codeis a set of rules by which information is transposed from oneform to another.In the case of olfactory coding, it would describe the ways inwhich information about odorant molecules is transposed intoneural responses. The idea is that when we understood thatcode, we might be able to predict:

The odorant molecule from the neural response,The neural response from the odorant molecule and,The perception of the odorant molecule from the neural re-sponse.



Table of Contents

1 Introduction

2 How to Apply NMF

3 What NMF Model?

4 NMF vs. FCM





NMF Models

In order to realize what method is appropriate for the givenproblem, some methods are compared together and finally thebest one with the minimum error is used. The methods are asfollows:

convexnmf: Convex and semi-nonnegative matrix factor-izations.orthnmf: Orthogonal nonnegative matrix t-factorizations.nmfnnls: Sparse non-negative matrix factorizations via al-ternating non-negativity constrained least squares.nmfrule: Basic NMF.nmfmult: NMF with Multiplicative Update Formula.nmfals: NMF with Auxiliary Constraints.



Design of Experiments

In order to make a reasonable comparison among thesemethods, every algorithm is executed 500 times and theaverage of all results is finally reported.Since the number of clusters, K in NMF formula, is not obvi-ous, every algorithm is executed 10 times for each numberof clusters (1 to 50 clusters). · · ·

... V...

· · ·

n×m

=

· · ·... W

...· · ·

n×K

×

· · ·... H

...· · ·

K×m



Results of Experiments

Method RMSE Maximum Number of Iterationsconvexnmf 0.1260 1000

orthnmf 0.1320 1000nmfnnls 0.1159 1000nmfrule 0.1162 1000nmfmult 0.1657 1000nmfals 0.1178 1000

V ∗ ≈WH√∑iµ(Viµ − (WH)iµ)2

m× n



Original Images, Normalized Images and NMF Images

Estimated #39

20 40

20

40

60

80

Normalized #39

20 40

20

40

60

80

Main #39

20 40

20

40

60

80

Estimated #181

20 40

20

40

60

80

Normalized #181

20 40

20

40

60

80

Main #181

20 40

20

40

60

80

Estimated #230

20 40

20

40

60

80

Normalized #230

20 40

20

40

60

80

Main #230

20 40

20

40

60

80



RMSE of nmfnnls

0 5 10 15 20 25 30 35 40 450.08

0.09

0.1

0.11

0.12

0.13

0.14

0.15

0.16

RM

SE

#Clusters

RMSE



Table of Contents

1 Introduction

2 How to Apply NMF

3 What NMF Model?

4 NMF vs. FCM





Fuzzy c-Means

Fuzzy c-means (FCM) is a method of clustering which allowsone piece of data to belong to two or more clusters. It is basedon minimization of the following objective function:

Jm =

N∑i=1

C∑j=1

umij ||xi − cj ||2, 1 ≤ m <∞



Cluster Validity Index

In order to compare two clustering method it is mandatoryto use a measure which is usually called “cluster validityindex”.In this research, a cluster validity index is used that is basedon the relative degree of sharing of two clusters as theweighted sum of the relative degrees of sharing for all data.



Relative degree of sharing of two fuzzy clusters I

Definition 1.

The relative degree of sharing of two fuzzy clusters F̃p and F̃qat xj is defined as:

Srel(xj : F̃p, F̃q) =µF̃p

(xj) ∧ µF̃q(xj)

1c

∑ci=1 µF̃i

(xj)

The numerator is the degree of sharing of F̃p and F̃q at xj andthe denominator is the average membership value of xj over cfuzzy clusters. fuzzy clusters. The relative degree of sharing oftwo fuzzy clusters is defined as the weighted summation ofSrel(xj : F̃p, F̃q) for all data in X.



Relative degree of sharing of two fuzzy clusters II

Definition 2. Relative degree of sharing of two fuzzy clusters

The relative degree of sharing of fuzzy clusters F̃p and F̃q isdefined as:

Srel(F̃p, F̃q) =

n∑j=1

Srel(xj : F̃p, F̃q)h(xj)

where h(xj) = −∑c

i=1 µF̃i(xj)logaµF̃i

(xj).

Here, h(xj) is the entropy of datum xj and µF̃i(xj) is the

membership value with which xj belongs to cluster F̃i. h(xj)measures how vaguely the datum xj is classified over cdifferent clusters. h(xj) is introduced to assign a weight forvague data. Vague data are given more weight than clearlyclassified data.



Validity Index

Definition 3. Validity Index

Let F̃p and F̃q be two fuzzy clusters belonging to a fuzzypartition (U, V ) and c be the number of clusters. Let Srel(F̃p, F̃q)be the degree of sharing of two fuzzy clusters. Then the indexV is defined as:

V (U, V : X) =2

c(c− 1)

c∑p 6=q

Srel(F̃p, F̃q) =

2

c(c− 1)

c∑p 6=q

n∑j=1

[c.[µF̃p(xj) ∧ µF̃q

(xj)]h(xj)]



Comparison of FCM and NMF According to CVI

K NMF FCM K NMF FCM2 1700.24 1734.29 24 599199.23 2643326.183 5296.82 5649.79 25 662361.84 2994109.884 10662.24 13219.41 26 727368.23 3363723.265 16886.72 25474.80 27 778396.07 3753364.516 26472.04 43810.20 28 869837.14 4193670.347 36242.72 68989.94 29 935658.81 4647376.758 48749.37 102089.70 30 1021121.80 5132955.039 63069.27 144948.65 31 1080491.94 5657771.12

10 81978.00 195586.80 32 1173188.39 6222877.99

Table: Cluster Validity Index Values for Different K’s



NMF VS. FCM

0 5 10 15 20 25 30 35 40 450

2

4

6

8

10

12

14

16

18x 10

6

#Clusters

Clu

ste

r V

alid

ity I

nd

ex

FCM

NMF



Visualization of Clusters for K=2

FCM

Cluster #1

10 20 30 40

10

20

30

40

50

60

70

80

Cluster #2

10 20 30 40

10

20

30

40

50

60

70

80

NMF

Cluster #1

10 20 30 40

10

20

30

40

50

60

70

80

Cluster #2

10 20 30 40

10

20

30

40

50

60

70

80




FCM

Cluster #1

10 20 30 40

10

20

30

40

50

60

70

80

Cluster #2

10 20 30 40

10

20

30

40

50

60

70

80

Cluster #3

10 20 30 40

10

20

30

40

50

60

70

80

NMF

Cluster #1

10 20 30 40

10

20

30

40

50

60

70

80

Cluster #2

10 20 30 40

10

20

30

40

50

60

70

80

Cluster #3

10 20 30 40

10

20

30

40

50

60

70

80




Cluster #1

20 40

20

40

60

80

Cluster #2

20 40

20

40

60

80

Cluster #3

20 40

20

40

60

80

Cluster #4

20 40

20

40

60

80

Cluster #5

20 40

20

40

60

80

Cluster #6

20 40

20

40

60

80

Cluster #7

20 40

20

40

60

80

Cluster #8

20 40

20

40

60

80

Cluster #9

20 40

20

40

60

80

Cluster #10

20 40

20

40

60

80

Cluster #11

20 40

20

40

60

80

Cluster #12

20 40

20

40

60

80

Cluster #1

20 40

20

40

60

80

Cluster #2

20 40

20

40

60

80

Cluster #3

20 40

20

40

60

80

Cluster #4

20 40

20

40

60

80

Cluster #5

20 40

20

40

60

80

Cluster #6

20 40

20

40

60

80

Cluster #7

20 40

20

40

60

80

Cluster #8

20 40

20

40

60

80

Cluster #9

20 40

20

40

60

80

Cluster #10

20 40

20

40

60

80

Cluster #11

20 40

20

40

60

80

Cluster #12

20 40

20

40

60

80



Table of Contents

1 Introduction

2 How to Apply NMF

3 What NMF Model?

4 NMF vs. FCM





Images and Pixels

Decomposing V into two matrices W and H is the final goalof a matrix factorization.W is usually called the basis matrix and H is called thecoefficient matrix.In a data matrix V that the rows represent pixels and thecolumns images, the basis matrix W can be used to dividethe m images into k image-clusters and the coefficient ma-trix H to divide the n pixels into k pixel-clusters.



Feature Selection I

Step 1. First all the rows of W are normalized. A row has thedata related to the membership of a pixel to different clusters(images). Normalization of row i is done by the followingequation:

W (i, k) =W (i, k)∑Kj W (i, j)

Step 2. Using the following equation, the data are scaled suchthat a pixel (feature) with a smaller membership value, wouldhave a bigger negative value. The results are then stored inmatric LogW .

LogW (i, k) = log2(W (i, k))×W (i, k)



Feature Selection II

Step 3. Feature scores are computed using LogW from step 2.Score(i) represents the score of i-th feature (pixel) and K is thenumber of clusters.

Score(i) = 1 +1

log2(K)×

K∑j=1

LogW (i, j)

Step 4. After finding the scores of all features (pixels), the moreeffective features can be filtered. In this research the first 30%of pixels are considered as the most effective ones.



Score Examples

Two extreme examples are presented here to show how theScore works.

Example 1: F1 = [1, 0, 0, 0, 0]. Score of feature F1 will beequal to 1 since:

Score(F1) = 1 +1

log25× (1× log21) = 1

Example 2: F2 = [0.2, 0.2, 0.2, 0.2, 0.2]. Score of feature F2

will be equal to 0 since:

Score(F2) = 1+1

log25× (0.2× log20.2+ · · ·+0.2× log20.2)

= 1 +log20.2

log25= 0



Bi-clustering Concept

We have an m× n matrix D, and assume that its entriescorrespond to pair wise dissimilarities between m row objectsOr and n column objects Oc, which, taken together (as aunion), comprise a set O of N = m+ n objects. The problemsare the assessment of cluster tendency:

P1) amongst the row objects Or;P2) amongst the column objects Oc;P3) amongst the union of the row and column objectsOr⋃Oc and;

P4) amongst the union of the row and column objects thatcontain at least one object of each type (co-clusters).



Bi-clustering Example

Problem Kcr 2cc 2cr

⋃c 3

cco 1cpure 3-1 = 2

0 2 4 6 8 10 12−2

0

2

4

6

8

10

12



Visual Assessment Tendency (VAT) - STEP 0

Let R be an n×n dissimilarity matrix corresponding to the setO = {o1, . . . , on}. R satisfies the following (metric) conditionsfor all 1 ≤ i, j ≤ n:

Rij ≥ 0

Rij = RjiRij = 0

R can be displayed as an intensity image I which is called adissimilarity image.

R =

0 0.73 0.19 0.71 0.16

0.73 0 0.59 0.12 0.780.19 0.59 0 0.55 0.190.71 0.12 0.55 0 0.740.16 0.78 0.19 0.74 0

I = ∅, J = 1, 2, 3, 4, 5




R =

0 0.73 0.19 0.71 0.16

0.73 0 0.59 0.12 0.780.19 0.59 0 0.55 0.190.71 0.12 0.55 0 0.740.16 0.78 0.19 0.74 0

Iold = ∅, Jold = 1, 2, 3, 4, 5

Inew = 2, Jnew = 1, 3, 4, 5




R =

0 0.73 0.19 0.71 0.16

0.73 0 0.59 0.12 0.780.19 0.59 0 0.55 0.190.71 0.12 0.55 0 0.740.16 0.78 0.19 0.74 0

Iold = 2, Jold = 1, 3, 4, 5

Inew = 2, 4, Jnew = 1, 3, 5




R =

0 0.73 0.19 0.71 0.16

0.73 0 0.59 0.12 0.780.19 0.59 0 0.55 0.190.71 0.12 0.55 0 0.740.16 0.78 0.19 0.74 0

Iold = 2, 4, Jold = 1, 3, 5

Inew = 2, 4, 3, Jnew = 1, 5




R =

0 0.73 0.19 0.71 0.16

0.73 0 0.59 0.12 0.780.19 0.59 0 0.55 0.190.71 0.12 0.55 0 0.740.16 0.78 0.19 0.74 0

Iold = 2, 4, 3, Jold = 1, 5

Inew = 2, 4, 3, 5, Jnew = 1




R =

0 0.73 0.19 0.71 0.16

0.73 0 0.59 0.12 0.780.19 0.59 0 0.55 0.190.71 0.12 0.55 0 0.740.16 0.78 0.19 0.74 0

Iold = 2, 4, 3, 5, Jold = 1

Inew = 2, 4, 3, 5, 1, Jnew = ∅



Ordered Dissimilarity Matrix

Rordered =

2 4 3 5 12 [2, 2] [2, 4] [2, 3] [2, 5] [2, 1]4 [4, 2] [4, 4] [4, 3] [4, 5] [4, 1]3 [3, 2] [3, 4] [3, 3] [3, 5] [3, 1]5 [5, 2] [5, 4] [5, 3] [5, 5] [5, 1]1 [1, 2] [1, 4] [1, 3] [1, 5] [1, 1]



VAT Visualization: From R to Rordered

1 2 3 4 5

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

Step #1

2 4

2

4

Step #2

2 4

2

4

Step #3

2 4

2

4

Step #4

2 4

2

4

Step #5

2 4

2

4

Step #6

2 4

2

4

Step #7

2 4

2

4

Step #8

2 4

2

4

Step #9

2 4

2

4

Step #10

2 4

2

4

Step #11

2 4

2

4

Step #12

2 4

2

4

Step #13

2 4

2

4

Step #14

2 4

2

4

Step #15

2 4

2

4

Step #16

2 4

2

4

Step #17

2 4

2

4

Step #18

2 4

2

4

Step #19

2 4

2

4

Step #20

2 4

2

4

Step #21

2 4

2

4

Step #22

2 4

2

4

Step #23

2 4

2

4

Step #24

2 4

2

4

Step #25

2 4

2

4

1 2 3 4 5

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5



co-Visual Assessment Tendency (co-VAT)

[Dr DDT Dc

]=

d(o1, o1) · · · d(o1, om)...

. . ....

d(om, o1) · · · d(om, om)

d(o1, om+1) · · · d(om, om+1)

.... . .

...d(o1, om+n) · · · d(om, om+n)

d(o1, om+1) · · · d(om, om+1)

.... . .

...d(o1, om+n) · · · d(om, om+n)

d(om+1, om+1) · · · d(om+1, om+n)

.... . .

...d(om+n, om+1) · · · d(om+n, om+n)



Vectorized Images as D

Image 1 Image 2 ... Image 469 Image 470Pixel 1Pixel 2

.

.

.

.

.

.Pixel 1779Pixel 1780



co-VAT Results

Dr for P1

2 4 6 8 10 12 14 16 18 20

5

10

15

20

Dc for P2

200 400 600 800 1000 1200 1400 1600

200

400

600

800

1000

1200

1400

1600

Dunion

for P3

200 400 600 800 1000 1200 1400 1600 1800

200

400

600

800

1000

1200

1400

1600

1800

coVAT Image: P4

200 400 600 800 1000 1200 1400 1600

5

10

15

20



Number of Clusters?

coVAT Image: P4

200 400 600 800 1000 1200 1400 1600

2

4

6

8

10

12

14

16

18

20

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4



Number of Clusters?

200 400 600 800 1000 1200 1400 1600

2

4

6

8

10

12

14

16

18

20

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1



Stability Analysis: CVI

2 4 6 8 10 12 141720

1740

1760

1780

1800

1820

1840

1860

Mean

Cluster #

CVI



Stability Analysis: CVI

2 4 6 8 10 12 1420

30

40

50

60

70

80

90

100

Sta

ndard

Devia

tion

Cluster #

CVI



Stability Analysis: RMSE

2 4 6 8 10 12 140.095

0.1

0.105

0.11

0.115

0.12

0.125

0.13

0.135

0.14

0.145

Mean

Cluster #

RMSE



Stability Analysis: RMSE

2 4 6 8 10 12 140

0.5

1

1.5

2

2.5

3

3.5

4

4.5x 10

−5

Sta

ndard

Devia

tion

Cluster #

RMSE



Number of Clusters: co-VAT & Stability Analysis

Number of Clusters? · · ·... V

...· · ·

n×m

=

· · ·... W

...· · ·

n×K

×

· · ·... H

...· · ·

K×m

K = 6



Table of Contents

1 Introduction

2 How to Apply NMF

3 What NMF Model?

4 NMF vs. FCM





Clusters for K = 6

Cluster #1

10 20 30 40

10

20

30

40

50

60

70

80

Cluster #2

10 20 30 40

10

20

30

40

50

60

70

80

Cluster #3

10 20 30 40

10

20

30

40

50

60

70

80

Cluster #4

10 20 30 40

10

20

30

40

50

60

70

80

Cluster #5

10 20 30 40

10

20

30

40

50

60

70

80

Cluster #6

10 20 30 40

10

20

30

40

50

60

70

80



Heat Map for K = 6, Feature Selection: 30%

1 2 3 4 5 6

6

5

4

3

2

1

−2

−1

0

1

−1.2

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

−5

−4

−3

−2

−1

0

1



Cross Validation

C#1

20 40

20

40

60

80

C#2

20 40

20

40

60

80

C#3

20 40

20

40

60

80

C#4

20 40

20

40

60

80

C#5

20 40

20

40

60

80

C#6

20 40

20

40

60

80

C#1

20 40

20

40

60

80

C#2

20 40

20

40

60

80

C#3

20 40

20

40

60

80

C#4

20 40

20

40

60

80

C#5

20 40

20

40

60

80

C#6

20 40

20

40

60

80

C#1

20 40

20

40

60

80

C#2

20 40

20

40

60

80

C#3

20 40

20

40

60

80

C#4

20 40

20

40

60

80

C#5

20 40

20

40

60

80

C#6

20 40

20

40

60

80

C#1

20 40

20

40

60

80

C#2

20 40

20

40

60

80

C#3

20 40

20

40

60

80

C#4

20 40

20

40

60

80

C#5

20 40

20

40

60

80

C#6

20 40

20

40

60

80

C#1

20 40

20

40

60

80

C#2

20 40

20

40

60

80

C#3

20 40

20

40

60

80

C#4

20 40

20

40

60

80

C#5

20 40

20

40

60

80

C#6

20 40

20

40

60

80



Analysis of olfactory images

Analysis of olfactory images is so important since it is believedthat it would help us to understand the olfactory coding. A codeis a set of rules by which information is transposed from oneform to another.In the case of olfactory coding, it would describe the ways inwhich information about odorant molecules is transposed intoneural responses. The idea is that when we understood thatcode, we might be able to predict:

The odorant molecule from the neural response,The neural response from the odorant molecule and,The perception of the odorant molecule from the neural re-sponse.



Conclusions & Future Works I

The purpose of this research is to bi-cluster olfactory im-ages and pixels simultaneously and finally find pixels (re-gions of images) that respond in a similar manner for a se-lection of chemicals.In order to find these regions, Non-negative Matrix Factor-ization (NMF) is applied.Since the introduction of NMF, NMF has been applied suc-cessfully in different areas including Chemometrics, FaceRecognition, Multimedia data analysis, Text mining DNAgene expression analysis, Protein interaction and many oth-ers.To the knowledge of the writers of this report, no work hasbeen reported yet related to the bi-clustering of olfactoryimages.



Conclusions & Future Works II

Sparsity: After comparison of different NMF’s models, itcame out that sparse models have a better performance.This is probably because of the sparse nature of the dataset. In the future, the sparsity of the data set could be eval-uated and more sparse NMF methods should be used.FCM vs. NMF: FCM is compared with NMF. NMF has showna superior performance in all the experiments. In the largenumber of clusters, NMF can find very heterogeneous pat-terns while fcm completely fails to do so. It seems that ap-plication of classical methods such as FCM is not a goodchoice for clustering olfactory images.



Conclusions & Future Works III

Cluster Tendency: Since the problem is an unsupervisedclustering, the tendency of clustering should be evaluatedbefore clustering. This means that before applying any clus-tering method one should ask if there is any cluster or no.Visual Assessment Cluster Tendency (VAT) is used to eval-uate the cluster tendency. Finally, VAT and Stability Analy-sis together are used to determine the number of clusters. Itshould be mentioned that after obtaining clusters, the inter-pretability of clusters must be evaluated by an expert relatedto the context of the problem.Stability: Since NMF and also FCM produce non-global so-lutions, it is probable to obtain different clusters in every run.The stability of NMF should be evaluated more extensivelyusing methods such as cross-validation.



Conclusions & Future Works IV

After obtaining clusters (W ) and H by NMF, V ≈ WH canbe rewritten column by column as v ≈ Wh where v and hare the corresponding columns of V and H:

...v...

n×1

=

· · ·... W

...· · ·

n×K

×

...h...

K×1

This additivity property of NMF allow us to realize the mosteffective regions in images that finally leads to a better un-derstanding of olfaction. Development of new additive clus-tering methods such as NMF could be considered as a fu-ture work.



Conclusions & Future Works V: Validation


Presentation Thesis Uniovi

Documents

Transcript of Presentation Thesis Uniovi