FLAME clustering


Fuzzy clustering by Local Approximation of MEmberships is a data clustering algorithm that defines clusters in the dense parts of a dataset and performs cluster assignment solely based on the neighborhood relationships among objects. The key feature of this algorithm is that the neighborhood relationships among neighboring objects in the feature space are used to constrain the memberships of neighboring objects in the fuzzy membership space.

Description of the FLAME algorithm

The FLAME algorithm is mainly divided into three steps:
  1. Extraction of the structure information from the dataset:
  2. # Construct a neighborhood graph to connect each object to its K-Nearest Neighbors ;
  3. # Estimate a density for each object based on its proximities to its KNN;
  4. # Objects are classified into 3 types:
  5. ## Cluster Supporting Object : object with density higher than all its neighbors;
  6. ## Cluster Outliers: object with density lower than all its neighbors, and lower than a predefined threshold;
  7. ## the rest.
  8. Local/Neighborhood approximation of fuzzy memberships:
  9. # Initialization of fuzzy membership:
  10. ## Each CSO is assigned with fixed and full membership to itself to represent one cluster;
  11. ## All outliers are assigned with fixed and full membership to the outlier group;
  12. ## The rest are assigned with equal memberships to all clusters and the outlier group;
  13. # Then the fuzzy memberships of all type 3 objects are updated by a converging iterative procedure called Local/Neighborhood Approximation of Fuzzy Memberships, in which the fuzzy membership of each object is updated by a linear combination of the fuzzy memberships of its nearest neighbors.
  14. Cluster construction from fuzzy memberships in two possible ways:
  15. # One-to-one object-cluster assignment, to assign each object to the cluster in which it has the highest membership;
  16. # One-to-multiple object-clusters assignment, to assign each object to the cluster in which it has a membership higher than a threshold.

    The optimization problem in FLAME

The Local/Neighborhood Approximation of Fuzzy Memberships is a procedure to minimize the Local/Neighborhood Approximation Error defined as the following:
where is the set of all type 3 objects, is the fuzzy membership vector of object, is the set of nearest neighbors of, and with are the coefficients reflecting the relative proximities of the nearest neighbors.
The NAE can be minimized by solving the following linear equations with unique solution which is the unique global minimum of NAE with value zero:
where is the number of CSOs plus one. The following iterative procedure can be used to solve these linear equations:

A simple illustration on a 2-Dimension testing dataset