Vector space model


Vector space model or term vector model is an algebraic model for representing text documents as vectors of identifiers, such as, for example, index terms. It is used in information filtering, information retrieval, indexing and relevancy rankings. Its first use was in the SMART Information Retrieval System.

Definitions

Documents and queries are represented as vectors.
Each dimension corresponds to a separate term. If a term occurs in the document, its value in the vector is non-zero. Several different ways of computing these values, also known as weights, have been developed. One of the best known schemes is tf-idf weighting.
The definition of term depends on the application. Typically terms are single words, keywords, or longer phrases. If words are chosen to be the terms, the dimensionality of the vector is the number of words in the vocabulary.
Vector operations can be used to compare documents with queries.

Applications

s of documents in a keyword search can be calculated, using the assumptions of document similarities theory, by comparing the deviation of angles between each document vector and the original query vector where the query is represented as a vector with same dimension as the vectors that represent the other documents.
In practice, it is easier to calculate the cosine of the angle between the vectors, instead of the angle itself:
Where is the intersection of the document and the query vectors, is the norm of vector d2, and is the norm of vector q. The norm of a vector is calculated as such:
Using the cosine the similarity between document dj and query q can be calculated as:
As all vectors under consideration by this model are element wise nonnegative, a cosine value of zero means that the query and document vector are orthogonal and have no match. See cosine similarity for further information.

Term frequency-inverse document frequency weights

In the classic vector space model proposed by Salton, Wong and Yang the term-specific weights in the document vectors are products of local and global parameters. The model is known as term frequency-inverse document frequency model. The weight vector for document d is, where
and
The vector space model has the following advantages over the Standard Boolean model:
  1. Simple model based on linear algebra
  2. Term weights not binary
  3. Allows computing a continuous degree of similarity between queries and documents
  4. Allows ranking documents according to their possible relevance
  5. Allows partial matching
Most of these advantages are a consequence of the difference in the density of the document collection representation between Boolean and term frequency-inverse document frequency approaches. When using Boolean weights, any document lies in a vertex in a n-dimensional hypercube. Therefore, the possible document representations are and the maximum Euclidean distance between pairs is. As documents are added to the document collection, the region defined by the hypercube's vertices become more populated and hence denser. Unlike Boolean, when a document is added using term frequency-inverse document frequency weights, the inverse document frequencies of the terms in the new document decrease while that of the remaining terms increase. In average, as documents are added, the region where documents lie expands regulating the density of the entire collection representation. This behavior models the original motivation of Salton and his colleagues that a document collection represented in a low density region could yield better retrieval results.

Limitations

The vector space model has the following limitations:
  1. Long documents are poorly represented because they have poor similarity values
  2. Search keywords must precisely match document terms; word substrings might result in a "false positive match"
  3. Semantic sensitivity; documents with similar context but different term vocabulary won't be associated, resulting in a "false negative match".
  4. The order in which the terms appear in the document is lost in the vector space representation.
  5. Theoretically assumes terms are statistically independent.
  6. Weighting is intuitive but not very formal.
Many of these difficulties can, however, be overcome by the integration of various tools, including mathematical techniques such as singular value decomposition and lexical databases such as WordNet.

Models based on and extending the vector space model

Models based on and extending the vector space model include:
The following software packages may be of interest to those wishing to experiment with vector models and implement search services based upon them.

Free open source software