Search and vector indexes

Parent: Indexing · Published reference · snapshot 2026-08-31

Full-text/search indexes and approximate-nearest-neighbour vector indexes: how they differ from B-tree indexes, their parameters and their limits. Child pack of Indexing.

These notes link each claim to its source. A source may be a research report hosted on this site rather than the primary document. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.

Definitions

MongoDB Search Indexes are a specific type of index supported for search queries within Atlas. [source]

Structure and components

| Compound Index Components | Compound Index Behavior | |---|---| | Ascending indexes Descending indexes | Only indexes documents that contain a value for at least one of the keys. | | Ascending indexes Descending indexes [Geospatial indexes](https://www.mongodb.com/docs/manual/geospatial-queries/#std-label-index-feature-geospatial) | Only indexes a document when it contains a value for one of the `geospatial` fields. Does not index documents in the ascending or descending indexes. | | Ascending indexes Descending indexes [Text indexes](https://www.mongodb.com/docs/manual/core/indexes/index-ty [source]
However, if the query used a single-field text index only on the `description` field, the query would examine **three** index keys. There are three documents in the collection where the `description` field contains the string `green`. [source]
When creating HNSW, you can specify the maximum number of connections in a layer (`m`) and the number of candidate vectors considered when building the graph (`ef_construction`). More connections and more candidate vectors will improve recall but will increase build time and memory. If you don't specify the parameters, the default values are `m = 16` and `ef_construction = 64`. [source]
With vector indexes, the query will search a subset of the vectors that is expected to contain the nearest neighbors (but may not contain all of them). This is computationally efficient, but the results are not guaranteed to be the nearest neighbors. [source]
1. **Vectors**: A vector is a list of numbers that represents the essential characteristics of data. In AI, vectors are created using models that capture relationships between elements of the data. * **Example in NLP**: A sentence like "The cat sat on the mat" can be converted into a vector by a language model such as BERT or GPT. The vector might look something like `[0.34, 0.67, -0.23, 0.88, ...]`. This vector contains semantic information about the sentence. * **Example in Image Recognition**: An image of a car could be transformed into a vector that represents its visual features, such as [source]
Like [database indexes](/database/reading-data/indexes/.md), search indexes are a data structure that is built in advance to enable efficient querying. Search indexes are defined as part of your Convex [schema](/database/schemas.md). [source]
Like [database indexes](/database/reading-data/indexes/.md), vector indexes are a data structure that is built in advance to enable efficient querying. Vector indexes are defined as part of your Convex [schema](/database/schemas.md). [source]
Only documents that contain a vector of the size and in the field specified by a vector index will be included in the index and returned by the vector search. [source]
- To learn how to control the ranking of `$text` query results, see[Assign Weights to $text Query Results on Self-Managed Deployments.](https://www.mongodb.com/docs/manual/core/indexes/index-types/index-text/control-text-search-results/#std-label-specify-weights) - You can include a wildcard text index as part of a compound text index. To learn more about compound text indexes, see [Create a Compound Text Index.](https://www.mongodb.com/docs/manual/core/indexes/index-types/index-text/create-text-index/#std-label-compound-text-index-example) - To see examples of `$text` queries, see[`$text`.](h [source]
Text indexes are diacritic insensitive. The text index does not distinguish between characters that contain diacritical marks and their non-marked counterparts, such as `é`, `ê`, and `e`. More specifically, the text index strips the markings categorized as diacritics in the [Unicode 8.0 Character Database Prop List](http://www.unicode.org/Public/8.0.0/ucd/PropList.txt). [source]
| Parameter | Type | Description | |---|---|---| | `weights` | document | Optional. For [text](https://www.mongodb.com/docs/manual/core/indexes/index-types/index-text/#std-label-index-type-text) indexes, a document that contains field and weight pairs. The weight is an integer ranging from 1 to 99,999 and denotes the significance of the field relative to the other indexed fields in terms of the score. You can specify weights for some or all the indexed fields. See[Assign Weights to $text Query Results on Self-Managed Deployments](https://www.mongodb.com/docs/manual/core/indexes/index-types/ind [source]
.mongodb.com/docs/manual/core/indexes/index-types/index-text/#std-label-index-type-text) indexes, the language that determines the list of stop words and the rules for the stemmer and tokenizer. See[$text Query Languages on Self-Managed Deployments](https://www.mongodb.com/docs/manual/reference/text-search-languages/#std-label-text-search-languages) for the available languages and[Specify Language for Text Indexes on Self-Managed MongoDB](https://www.mongodb.com/docs/manual/core/indexes/index-types/index-text/specify-text-index-language/) for more information and examples. The default value is [source]

How it works

* When the connection is in MVCC mode (`journal_mode=mvcc`). * On `WITHOUT ROWID` tables. * For custom index methods that have no backing B-tree, such as FTS and vector indexes. * When `PRAGMA query_only` is enabled. [source]
1. Create the index **after** the table has some data 2. Choose an appropriate number of lists - a good place to start is `rows / 1000` for up to 1M rows and `sqrt(rows)` for over 1M rows. 3. When querying, specify an appropriate number of probes (higher is better for recall, lower is better for speed) - a good place to start is `sqrt(lists)` [source]
LibSQL implements [DiskANN](https://turso.tech/blog/approximate-nearest-neighbor-search-with-diskann-in-libsql) algorithm in order to speed up approximate nearest neighbors queries for tables with vector columns. [source]
The `fullTextIndex` preview feature provides support for introspection and migration of full text indexes in MySQL and MongoDB. This can be configured using the `@@fulltext` attribute. Existing full text indexes in the database are added to your Prisma schema after introspecting with `db pull`, and new full text indexes added in the Prisma schema are created in the database when using Prisma Migrate. [source]
Label-based filtering is a technique that allows you to filter the results of an ANN search based on a label while using the DiskANN index. Other filters are supported, but will use pgvector's post-filtering (i.e. after the ANN search). [source]
The [`.withSearchIndex`](/api/interfaces/server.QueryInitializer.md#withsearchindex) method defines which search index to query and how Convex will use that search index to select documents. The first argument is the name of the index and the second is a *search filter expression*. A search filter expression is a description of which documents Convex should consider when running the query. [source]
Having a very specific search filter expression will make your query faster and less likely to hit Convex's limits because Convex will use the search index to efficiently cut down on the number of results to consider. [source]
* [Text search](/search/overview.md) returns all documents that include a word for which at least one word in the searched string is a prefix. It does not sort the results by relevance. * [Vector search](/search/vector-search.md) returns results sorted by cosine similarity, but doesn't use an efficient vector index in its implementation. * There is no support for [cron jobs](/scheduling/cron-jobs.md), you should trigger your functions manually from the test. [source]
After you create a wildcard text index, when you insert or update documents, the index updates to include any new string field values. As a result, wildcard text indexes negatively impact performance for inserts and updates. [source]
Text indexes are always [sparse](https://www.mongodb.com/docs/manual/core/index-sparse/#std-label-index-type-sparse). When you create a text index, MongoDB ignores the `sparse` option. [source]

Parameters and configuration

| Name | Type | Description | | --------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ [source]
| Name | Type | Description | | --------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ [source]
| Name | Type | Description | | --------------------- | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------ [source]
| Name | Type | Description | | -------------------- | ----------------------------------------- | -------------------------------------------------------------------------------------------------------------- [source]
| Name | Type | Description | | --------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ [source]
| Name | Type | Description | | --------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ [source]
* This is the field which will be indexed for full text search. * It must be of type `string`. [source]

How-to and procedures

Add an index for each distance function you want to use. [source]
Define a staged index on this table. [source]
To drop a [text](https://www.mongodb.com/docs/manual/core/indexes/index-types/index-text/#std-label-index-type-text) index, specify the index name instead of the index specification document. [source]
Define a search index on this table. [source]
To learn about search indexes, see [Search](https://docs.convex.dev/text-search). [source]
Define a staged search index on this table. [source]
Use PyMongo to create indexes including single field, compound, text, geospatial, and MongoDB Search indexes for optimization. [source]
To create a wildcard text index, set the index key to the wildcard specifier (`$**`) and set the index value to `text`: [source]
To override the default version and specify a different version for your text index, set the `textIndexVersion` option when you create an index: [source]
To enable the `fullTextIndex` preview feature, add the `fullTextIndex` feature flag to the `generator` block of the `schema.prisma` file: [source]
Define an index on this table. [source]
To learn about full text search, see [Indexes](https://docs.convex.dev/text-search). [source]
Partition the MongoDB Search index to support more index objects. [source]
Create a wildcard text index using the `$**` specifier to enable $text queries on all string fields in a collection. [source]
Use a compound text index with an equality match to limit the number of entries scanned during $text queries for better performance. [source]
To perform similarity search, you need to create a table with a vector column and an `HNSW` or `IVFFlat` index on this column for better performance: [source]
Define a vector index on this table. [source]
To learn about vector indexes, see [Vector Search](https://docs.convex.dev/vector-search). [source]
Define a staged vector index on this table. [source]
To add a search index onto a table, use the [`searchIndex`](/api/classes/server.TableDefinition.md#searchindex) method on your table's schema. For example, if you want an index which can search for messages matching a keyword in a channel, your schema could look like: [source]
To add a vector index onto a table, use the [`vectorIndex`](/api/classes/server.TableDefinition.md#vectorindex) method on your table's schema. Every vector index has a unique name and a definition with: [source]
Create a text index on the `content`, `users.comments`, and `users.profiles` fields. Set the index `name` to `InteractionsTextIndex`: [source]
You can create a text index that contains every document field with string data in a collection. These text indexes are called **wildcard text indexes**. Wildcard text indexes support `$text`[queries](https://www.mongodb.com/docs/manual/core/text-search/on-prem/#std-label-text-search-on-prem) on unknown, arbitrary, or dynamically generated fields. [source]
Create a wildcard text index on the `blog` collection: [source]
To specify a language for the text index, see [Specify Language for Text Indexes on Self-Managed MongoDB.](https://www.mongodb.com/docs/manual/core/indexes/index-types/index-text/specify-text-index-language/#std-label-specify-text-index-language) [source]

Examples and snippets

``` — `defineTable({` … [source]
```sql theme={null} — `CREATE INDEX ON wiki_documents USING hnsw (embedding vector_l2_ops);` … [source]
```sql theme={null} — `CREATE INDEX ON wiki_documents USING hnsw (embedding vector_ip_ops);` … [source]
```sql theme={null} — `CREATE INDEX ON wiki_documents USING hnsw (embedding vector_cosine_ops);` … [source]
```sql theme={null} — `CREATE INDEX ON wiki_documents USING hnsw (embedding vector_cosine_ops) WITH (m = 16, ef_construction = 100);` … [source]
```sql theme={null} — `CREATE INDEX ON wiki_documents USING ivfflat (embedding vector_l2_ops) WITH (lists = 100);` … [source]
```sql theme={null} — `CREATE INDEX ON wiki_documents USING ivfflat (embedding vector_ip_ops) WITH (lists = 100);` … [source]
```sql theme={null} — `CREATE INDEX ON wiki_documents USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100);` … [source]
```sql theme={null} — `CREATE INDEX movies_idx ON movies (libsql_vector_idx(embedding));` … [source]
```sql theme={null} — `CREATE INDEX movies_idx ON movies (libsql_vector_idx(embedding))` … [source]
``` — `movieEmbeddings: defineTable({` … [source]
```ts — `// CREATE INDEX ON items USING hnsw (embedding vector_l2_ops);` … [source]
```ts — `// CREATE INDEX ON items USING hnsw (embedding halfvec_l2_ops);` … [source]
```ts — `// CREATE INDEX ON items USING hnsw (embedding sparsevec_l2_ops);` … [source]
```ts — `// CREATE INDEX "hamming_index" ON "items" USING hnsw ("embedding" bit_hamming_ops);` … [source]
```sql theme={null} — `-- 1. Shared data table` … [source]
```sql theme={null} — `-- 1. Tenant-aware table` … [source]
```sql theme={null} — `-- 1. Create a label column` … [source]
```sql theme={null} — `CREATE INDEX ON wiki_documents USING hnsw ((embedding::halfvec(3)) halfvec_l2_ops);` … [source]
```prisma title="schema.prisma" showLineNumbers — `generator client {` … [source]
```prisma title="schema.prisma" showLineNumbers — `generator js {` … [source]
```sql theme={null} — `SET hnsw_ef = 100;` … [source]
```sql theme={null} — `SET ivfflat_probes = 100;` … [source]
``` — `import { defineSchema, defineTable } from "convex/server";` … [source]
``` — `foods: defineTable({` … [source]

Measurements and reference values

With approximate indexes, filtering is applied after the index is scanned. If a condition matches 10% of rows, with HNSW and the default hnsw\.ef\_search of 40, only 4 rows will match on average. [source]
| | Free/Starter | Professional | Business | Enterprise | Notes | | -------------- | ------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- | ------------------------- | ------------------------- | ---- [source]

Problems, failure modes and limitations

- You can create multiple wildcard indexes in a collection. - A wildcard index can cover the same fields as other indexes in the collection. - Wildcard indexes omit the `_id` field by default. To include the`_id` field in the wildcard index, you must explicitly include it in the`wildcardProjection` document by specifying`{ "_id" : 1 }` . - Wildcard indexes are [sparse indexes](https://www.mongodb.com/docs/manual/core/index-sparse/#std-label-index-type-sparse) and only contain entries for documents that have the indexed field, even if the index field contains a null value. - Wildcard indexes ar [source]
For large tables, index backfill can be slow. Staging an index allows you to push the schema and enable the index later. [source]
If `staged` is `true`, the index will be staged and will not be enabled until the staged flag is removed. Staged indexes do not block push completion. Staged indexes cannot be used in queries. [source]
Decide whether they should be replaced with a `.withIndex` condition — per [this section](/understanding/best-practices/.md#only-use-collect-with-a-small-number-of-results), if you are filtering over a large (1000+) or potentially unbounded number of documents, you should use an index. If not using a `.withIndex` / `.withSearchIndex` condition, consider replacing them with a filter in code for more readability and flexibility. [source]
| Name | Type | Description [source]
In a small collection like the one used in the preceding example, there isn't a noticeable difference in performance between single-field and compound text indexes. However, in larger collections, increased index entry scans can noticeably hinder performance. For best performance, create text indexes that limit the number of index entries scanned to best fit your equality matches. [source]
Stored source fields can be defined in a search index to improve query performance and avoid full document lookups. [source]
* Vector index works only for tables **with** `ROWID` or with singular `PRIMARY KEY`. Composite `PRIMARY KEY` without `ROWID` is not supported [source]
Search indexes work best with English or other Latin-script languages. Text is tokenized using Tantivy's [`SimpleTokenizer`](https://docs.rs/tantivy/latest/tantivy/tokenizer/struct.SimpleTokenizer.html), which splits on whitespace and punctuation. We also limit terms to 32 characters in length and lowercase them. [source]
Search indexes count against the [limit of 32 indexes per table](/database/reading-data/indexes/.md#limits). [source]
Vector indexes count towards the [limit of 32 indexes per table](/database/reading-data/indexes/.md#limits). In addition you can have up to 4 vector indexes per table. [source]

Comparisons and alternatives

From a syntax perspective, the vector index differs from ordinary application-defined B-Tree indices in that it must wrap the vector column into a `libsql_vector_idx` marker function like this [source]
A compound text index on the `department` and `description` fields limits the index keys scanned to only documents within the specified `department`. The compound text index provides improved performance compared to a single-field text index on the `description` field. [source]
An HNSW index creates a multilayer graph. It has slower build times and uses more memory than IVFFlat, but has better query performance (in terms of speed-recall tradeoff). There’s no training step like IVFFlat, so the index can be created without any data in the table. [source]
An IVFFlat index divides vectors into lists, and then searches a subset of those lists that are closest to the query vector. It has faster build times and uses less memory than HNSW, but has lower query performance (in terms of speed-recall tradeoff). [source]
In addition, vector databases have indexes that can make searching large collections of vectors even more efficient. These indexes are different from those you are familiar with because they are based on machine learning algorithms. These algorithms find close vectors very efficiently, even with millions of vectors. But they have accuracy and memory use tradeoffs. Because of the accuracy tradeoffs (also known as **recall tradeoffs**, since it looks like the database “forgot” some of the data), the algorithms used in the indexes are called **ANN - approximate nearest neighbors** (as opposed to [source]
Keep in mind that vector indexes are unlike other database indexes. They are used to perform efficient **approximate** nearest neighbor searches. [source]
When using vector indexes, you can control the trade-off between speed and recall by specifying the index type and parameters. [source]
Filtering in code instead of using the `.filter` syntax has the same performance, and is generally easier code to write. Conditions in `.withIndex` or `.withSearchIndex` are more efficient than `.filter` or filtering in code, so almost all uses of `.filter` should either be replaced with a `.withIndex` or `.withSearchIndex` condition, or written as TypeScript code. [source]

Changes and history

Version 3 text indexes are the current standard, supporting multiple languages via the `default_language` option. [source]

Facts and statements

export const posts = pgTable( 'posts', { id: serial('id').primaryKey(), title: text('title').notNull(), body: text('body').notNull(), bodySearch: tsvector('body_search') .notNull() .generatedAlwaysAs((): SQL => sql`to_tsvector('english', ${posts.body})`), }, (t) => [ index('idx_body_search').using('gin', t.bodySearch), ] ); ``` </CodeTab> ```sql CREATE TABLE "posts" ( "id" serial PRIMARY KEY NOT NULL, "title" text NOT NULL, "body" text NOT NULL, "body_search" "tsvector" GENERATED ALWAYS AS (to_tsvector('english', "posts"."body")) STORED NOT NULL ); --> statement-breakpoint CREATE INDEX "idx_bo [source]
export const posts = pgTable( 'posts', { id: serial('id').primaryKey(), title: text('title').notNull(), body: text('body').notNull(), search: tsvector('search') .notNull() .generatedAlwaysAs( (): SQL => sql`setweight(to_tsvector('english', ${posts.title}), 'A') || setweight(to_tsvector('english', ${posts.body}), 'B')`, ), }, (t) => [ index('idx_search').using('gin', t.search), ], ); ``` </CodeTab> ```sql CREATE TABLE "posts" ( "id" serial PRIMARY KEY NOT NULL, "title" text NOT NULL, "body" text NOT NULL, "search" "tsvector" GENERATED ALWAYS AS (setweight(to_tsvector('english', "posts"."title") [source]
* An index created for a table with existing data will be automatically populated with this data * All updates to the base table will be **automatically** reflected in the index * You can rebuild index from scratch using `REINDEX movies_idx` command * You can drop index with `DROP INDEX movies_idx` command * You can create [partial](https://www.sqlite.org/partialindex.html) vector index with a custom filtering rule: [source]
Indexes speed up queries by allowing efficient lookups on specific fields. Use `.withIndex()` in your queries to leverage them. [source]
`pg_bigm` supports full-text search indexes: [source]
The `fts` index method creates a full-text search index powered by Tantivy. FTS indexes support tokenizer configuration through the `WITH` clause. [source]
Index fields must be queried in the same order they are defined. If you need to query by `field2` then `field1`, create a separate index with that field order. [source]
**Best practice:** Always include all index fields in the index name (e.g., `"by_field1_and_field2"`). [source]
| Name | Type | | ---------------- | ---------------------------------------------- | | `IndexName` | extends `string` | | `FirstFieldPath` | extends `any` | | `RestFieldPaths` | extends `ExtractFieldPaths`<`DocumentType`>\[] | [source]
| Name | Type | Description | | -------- | ----------------------------------------- | --------------------------------------------------------------- | | `name` | `IndexName` | The name of the index. | | `fields` | \[`FirstFieldPath`, ...RestFieldPaths\[]] | The fields to index, in order. Must specify at least one field. | [source]
| Name | Type | | -------------- | ----------------------- | | `IndexName` | extends `string` | | `SearchField` | extends `any` | | `FilterFields` | extends `any` = `never` | [source]
| Name | Type | | -------------- | ----------------------- | | `IndexName` | extends `string` | | `VectorField` | extends `any` | | `FilterFields` | extends `any` = `never` | [source]
An object with parameters for performing a vector search against a vector index. [source]
4. \[Optional] A boolean `staged` flag  * If set to `true`, the index will be backfilled asynchronously from the deploy similar to [staged database indexes](/database/reading-data/indexes/.md#staged-indexes). This is useful for large tables where the index backfill time is significant. Defaults to `false`. [source]
A [TableDefinition](/api/classes/server.TableDefinition.md) with this search index included. [source]
The library supports various index types including single field, compound, multikey, text, and MongoDB Search indexes. [source]
export const test = cockroachTable( "test", { id: int4("id").primaryKey().generatedAlwaysAsIdentity(), content: string("content"), contentSearch: tsVector("content_search", { dimensions: 3, }).generatedAlwaysAs( (): SQL => sql`to_tsvector('english', ${test.content})` ), }, (t) => [ index("idx_content_search").using("gin", t.contentSearch) ] ); ``` ```sql {4} CREATE TABLE "test" ( "id" int4 PRIMARY KEY GENERATED ALWAYS AS IDENTITY (INCREMENT BY 1 MINVALUE 1 MAXVALUE 2147483647 START WITH 1 CACHE 1), "content" string, "content_search" tsvector GENERATED ALWAYS AS (to_tsvector('english', "test"." [source]
As for now, Drizzle doesn't support `tsvector` type natively, so you need to convert your data in the `text` column on the fly. To enhance the performance, you can create a `GIN` index on your column like this: [source]
export const guides = pgTable( 'guides', { id: serial('id').primaryKey(), title: text('title').notNull(), description: text('description').notNull(), url: text('url').notNull(), embedding: vector('embedding', { dimensions: 1536 }), }, (table) => [ index('embeddingIndex').using('hnsw', table.embedding.op('vector_cosine_ops')), ] ); ``` </CodeTab> ```sql CREATE TABLE IF NOT EXISTS "guides" ( "id" serial PRIMARY KEY NOT NULL, "title" text NOT NULL, "description" text NOT NULL, "url" text NOT NULL, "embedding" vector(1536) ); --> statement-breakpoint CREATE INDEX IF NOT EXISTS "embeddingIndex" ON [source]
export const test = pgTable( "test", { id: integer("id").primaryKey().generatedAlwaysAsIdentity(), content: text("content"), contentSearch: tsVector("content_search", { dimensions: 3, }).generatedAlwaysAs( (): SQL => sql`to_tsvector('english', ${test.content})` ), }, (t) => [ index("idx_content_search").using("gin", t.contentSearch) ] ); ``` ```sql {4} CREATE TABLE "test" ( "id" integer PRIMARY KEY GENERATED ALWAYS AS IDENTITY (sequence name "test_id_seq" INCREMENT BY 1 MINVALUE 1 MAXVALUE 2147483647 START WITH 1 CACHE 1), "content" text, "content_search" tsvector GENERATED ALWAYS AS (to_tsvec [source]
* Create a table with one or more vector columns (e.g. `FLOAT32`) * Provide vector values in binary format or convert text representation to binary using the appropriate conversion function (e.g. `vector32(...)`) * Calculate vector similarity between vectors in the table or from the query itself using dedicated vector functions (e.g. `vector_distance_cos`) * Create a special vector index to speed up nearest neighbors queries (use the `libsql_vector_idx(column)` expression in the `CREATE INDEX` statement to create vector index) * Query the index with the special `vector_top_k(idx_name, q_vector [source]
At the moment vector index must be queried **explicitly** with special `vector_top_k(idx_name, q_vector, k)` [table-valued function](https://www.sqlite.org/vtab.html#table_valued_functions). The function accepts index name, query vector and amount of neighbors to return. This function searches for `k` approximate nearest neighbors and returns `ROWID` of these rows or `PRIMARY KEY` if base index [does not have ROWID](https://www.sqlite.org/withoutrowid.html). [source]
```text 200 theme={null} PRAGMA foreign_keys=OFF; BEGIN TRANSACTION; CREATE TABLE IF NOT EXISTS mytable ( content TEXT, embedding FLOAT32(1536) ); CREATE TABLE IF NOT EXISTS libsql_vector_index (type TEXT, name TEXT, vector_type TEXT, block_size INTEGER, dims INTEGER, distance_ops TEXT); INSERT INTO libsql_vector_index VALUES('diskann','mytable_idx','float32',128,1536,'cosine'); CREATE INDEX mytable_idx USING diskann_cosine_ops ON mytable (embedding); COMMIT; ``` </CodeGroup> [source]
| Flag | Description | | ---------------------------------------- | ----------------------------------------------------------------------------------- | | `--experimental-views` | Enable views (`CREATE VIEW` / `CREATE MATERIALIZED VIEW`) | | `--experimental-custom-types` | Enable custom types (`CREATE TYPE` / `DROP TYPE` / `CREATE DOMAIN` / `DROP DOMAIN`) | | `--experimental-encryption` | Enable at-rest database encrypt [source]
<Info> Sparse vectors can be indexed with the experimental sparse vector index method (enable `--experimental-index-method`). See [CREATE INDEX](/sql-reference/statements/create-index). </Info> [source]
▸ **index**<`IndexName`, `FirstFieldPath`, `RestFieldPaths`>(`name`, `indexConfig`): [`TableDefinition`](/api/classes/server.TableDefinition.md)<`DocumentType`, [`Expand`](/api/modules/server.md#expand)<`Indexes` & `Record`<`IndexName`, \[`FirstFieldPath`, ...RestFieldPaths\[], `"_creationTime"`]>>, `SearchIndexes`, `VectorIndexes`> [source]
[`TableDefinition`](/api/classes/server.TableDefinition.md)<`DocumentType`, [`Expand`](/api/modules/server.md#expand)<`Indexes` & `Record`<`IndexName`, \[`FirstFieldPath`, ...RestFieldPaths\[], `"_creationTime"`]>>, `SearchIndexes`, `VectorIndexes`> [source]
▸ **index**<`IndexName`, `FirstFieldPath`, `RestFieldPaths`>(`name`, `fields`): [`TableDefinition`](/api/classes/server.TableDefinition.md)<`DocumentType`, [`Expand`](/api/modules/server.md#expand)<`Indexes` & `Record`<`IndexName`, \[`FirstFieldPath`, ...RestFieldPaths\[], `"_creationTime"`]>>, `SearchIndexes`, `VectorIndexes`> [source]
▸ **index**<`IndexName`, `FirstFieldPath`, `RestFieldPaths`>(`name`, `indexConfig`): [`TableDefinition`](/api/classes/server.TableDefinition.md)<`DocumentType`, `Indexes`, `SearchIndexes`, `VectorIndexes`> [source]
▸ **searchIndex**<`IndexName`, `SearchField`, `FilterFields`>(`name`, `indexConfig`): [`TableDefinition`](/api/classes/server.TableDefinition.md)<`DocumentType`, `Indexes`, [`Expand`](/api/modules/server.md#expand)<`SearchIndexes` & `Record`<`IndexName`, { `searchField`: `SearchField` ; `filterFields`: `FilterFields` }>>, `VectorIndexes`> [source]
[`TableDefinition`](/api/classes/server.TableDefinition.md)<`DocumentType`, `Indexes`, [`Expand`](/api/modules/server.md#expand)<`SearchIndexes` & `Record`<`IndexName`, { `searchField`: `SearchField` ; `filterFields`: `FilterFields` }>>, `VectorIndexes`> [source]
▸ **searchIndex**<`IndexName`, `SearchField`, `FilterFields`>(`name`, `indexConfig`): [`TableDefinition`](/api/classes/server.TableDefinition.md)<`DocumentType`, `Indexes`, `SearchIndexes`, `VectorIndexes`> [source]
▸ **vectorIndex**<`IndexName`, `VectorField`, `FilterFields`>(`name`, `indexConfig`): [`TableDefinition`](/api/classes/server.TableDefinition.md)<`DocumentType`, `Indexes`, `SearchIndexes`, [`Expand`](/api/modules/server.md#expand)<`VectorIndexes` & `Record`<`IndexName`, { `vectorField`: `VectorField` ; `dimensions`: `number` ; `filterFields`: `FilterFields` }>>> [source]
[`TableDefinition`](/api/classes/server.TableDefinition.md)<`DocumentType`, `Indexes`, `SearchIndexes`, [`Expand`](/api/modules/server.md#expand)<`VectorIndexes` & `Record`<`IndexName`, { `vectorField`: `VectorField` ; `dimensions`: `number` ; `filterFields`: `FilterFields` }>>> [source]
▸ **vectorIndex**<`IndexName`, `VectorField`, `FilterFields`>(`name`, `indexConfig`): [`TableDefinition`](/api/classes/server.TableDefinition.md)<`DocumentType`, `Indexes`, `SearchIndexes`, `VectorIndexes`> [source]
| Name | Type | Description | | -------------- | -------------------------------- [source]
This will be an object mapping index names to the search index config. [source]
This will be an object mapping index names to the vector index config. [source]
1. `vectorField` string  * The name of the field indexed for vector search. 2. `dimensions` number  * The fixed size of the vectors index. If you're using embeddings, this dimension should match the size of your embeddings (e.g. `1536` for OpenAI). 3. \[Optional] `filterFields` array  * The names of additional fields that are indexed for fast filtering within your vector index. 4. \[Optional] `staged` boolean  * If set to `true`, the index will be backfilled asynchronously from the deploy similar to [staged database indexes](/database/reading-data/indexes/.md#st [source]
Wildcard text indexes are distinct from [wildcard indexes](https://www.mongodb.com/docs/manual/core/indexes/index-types/index-wildcard/#std-label-wildcard-index-core). Wildcard text indexes support queries that use the [`$text`](https://www.mongodb.com/docs/manual/reference/operator/query/text/#mongodb-query-op.-text) operator, while wildcard indexes do not. [source]
Text indexes tokenize and stem the terms in the indexed fields for the index entries. The index uses simple [language-specific](https://www.mongodb.com#std-label-text-index-supported-languages) suffix stemming. For each document in the collection, the text index stores one index entry for each unique stemmed term in each indexed field. [source]
If an existing or newly inserted document lacks a text index field (or the field is null or an empty array), MongoDB does not add a text index entry for the document. [source]
For data hosted on MongoDB, you can support full-text search with MongoDB Search indexes. To learn more, see [Create a MongoDB Search Index.](https://www.mongodb.com/docs/atlas/atlas-search/create-index/) [source]
Vector Search Indexes support queries on vector embeddings. To create Vector Search Indexes, see [Index Fields for Vector Search.](https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-type/) [source]
`listIndexes` does not return information on [MongoDB Search indexes](https://www.mongodb.com/docs/atlas/atlas-search/atlas-search-overview/#fts-indexes). Instead, use `$listSearchIndexes`. [source]
Prisma ORM allows configuration of database indexes, unique constraints and primary key constraints. Full text indexes in MySQL and MongoDB are available through the `fullTextIndex` preview feature using the `@@fulltext` attribute. [source]
* `topic`: `"current_storage_usage"` * `timestamp`: Unix epoch timestamp in milliseconds * `total_document_size_bytes`: number, total size in bytes of all documents stored in database tables * `total_index_size_bytes`: number, total size in bytes of all database indexes * `total_vector_storage_bytes`: number, total size in bytes of vector index storage * `total_text_storage_bytes`: number, total size in bytes of text index storage * `total_file_storage_bytes`: number, total size in bytes of file storage * `total_backup_storage_bytes`: number, total size in bytes of snapshot/backup storage * `t [source]
| Name | Type | | ----------- | ---------------------------------------- | | `IndexName` | extends `string` \| `number` \| `symbol` | [source]
The following example demonstrates adding a `@@fulltext` index to the `title` and `content` fields of a `Post` model: [source]
On MongoDB, you can use the `@@fulltext` index attribute (via the `fullTextIndex` preview feature) with the `sort` argument to add fields to your full-text index in ascending or descending order. The following example adds a `@@fulltext` index to the `title` and `content` fields of the `Post` model, and sorts the `title` field in descending order: [source]
You should see the two apparel rows first - a good sanity check that the index works. [source]
`pgvector` supports two types of indexes: [source]
While querying, you can specify the size of the candidate list that will be searched (`hnsw_ef`): [source]
Vectors with up to 2,000 dimensions can be indexed. [source]
LibSQL introduces a custom index type that helps speed up nearest neighbors queries against a fixed distance function (cosine similarity by default). [source]
▸ \*\* indexes\*\*(): { `indexDescriptor`: `string` ; `fields`: `string`\[] }\[] [source]
Returns indexes defined on this table. Intended for the advanced use cases of dynamically deciding which index to use for a query. If you think you need this, please chime in on ths issue in the Convex JS GitHub repo. <https://github.com/get-convex/convex-js/issues/49> [source]
{ `indexDescriptor`: `string` ; `fields`: `string`\[] }\[] [source]
<https://docs.convex.dev/database/reading-data/indexes> [source]
Search queries must always search for some text within the index's `searchField`. This query can optionally add equality filters for any `filterFields` specified in the index. [source]
1. A name.  * Must be unique per table. [source]
Search and AI Recommendations skills provide guidance on configuring search indexes and integrating AI capabilities into applications. [source]
Search and AI Recommendations provides guidance for implementing MongoDB Atlas Search and AI-powered recommendations, including configuring search indexes. [source]
MongoDB Vector Search — > Use MongoDB Vector Search to create vector indexes and perform vector search, including semantic search and hybrid search, on your vector embeddings in MongoDB. [source]
MongoDB Vector Search enables the creation of vector indexes to perform semantic and hybrid search on vector embeddings within MongoDB. [source]
The dateFacet field type includes date values for faceting in a search index. [source]
The numberFacet field type includes numeric values of int32, int64, and double data types in the search index for faceted search. [source]
MongoDB Search Indexes are supported for search functionality within the C Driver. [source]
MongoDB Search and MongoDB Vector Search indexes can be created and managed using the .NET/C# driver. [source]
Text queries can be performed by creating text indexes and sorting results by relevance. [source]
MongoDB\Collection::createSearchIndex() creates a Search or Vector Search index, available for Atlas deployments with a minimum M10 cluster tier. [source]
MongoDB\Collection::listSearchIndexes() retrieves index information for all search indexes in a collection with customizable options. [source]
Updating a MongoDB Search index via MongoDB\Collection::updateSearchIndex() requires an Atlas cluster tier of at least M10. [source]
Indexes should be used with Mongoid to improve query performance, including compound, geospatial, and Search indexes. [source]
Atlas Search Indexes are supported for search capabilities within the driver. [source]
Text indexes enhance search performance on string fields and support $text queries with case and diacritic insensitivity. [source]
MongoDB 7.0 release notes cover improvements in sharding, storage, internal operations, Atlas Search index management, and aggregation enhancements. [source]
> Use MongoDB Vector Search to create vector indexes and perform vector search, including semantic search and hybrid search, on your vector embeddings in MongoDB. [source]
After that, `pgvector.Vector(1536)` is a column type in your contract, vector operators appear in the query builder, and `migration plan` knows how to create vector indexes. See [Using extensions](https://www.prisma.io/docs/orm/extensions/using-extensions). [source]
If you use the [full-text index](https://www.prisma.io/docs/orm/v7/prisma-schema/data-model/indexes#full-text-indexes-mysql-and-mongodb) feature in your app, you can now remove `fullTextIndex` from the `previewFeatures` in your Prisma schema: [source]
- [`External tables`](https://www.prisma.io/docs/orm/v7/prisma-schema/data-model/externally-managed-tables): How to declare and use externally managed tables in Prisma ORM - [`Indexes`](https://www.prisma.io/docs/orm/v7/prisma-schema/data-model/indexes): How to configure index functionality and add full text indexes - [`Models`](https://www.prisma.io/docs/orm/v7/prisma-schema/data-model/models): Learn about the concepts for building your data model with Prisma: Models, scalar types, enums, attributes, functions, IDs, default values and more - [`Multi-schema`](https://www.prisma.io/docs/orm/v7/ [source]
How to configure index functionality and add full text indexes [source]
Before you can generate embeddings or create text search indexes for documents, you need to prepare them. The preparation steps depend on the document type and the retrieval method you choose. For example, if you use PDFs or HTML documents, you will need to extract the text from them. If you used scanned documents, you will need to use OCR to extract the text. [source]
DiskANN index support for PostgreSQL with pgvectorscale [source]
The `pgvectorscale` extension adds diskANN index support for pgvector. This extension is useful in cases where `pgvector`'s `hnsw` index does not fit into available memory and as a result the ANN search does not perform as expected. [source]
* StreamingDiskANN index - disk-backed HNSW variant. * Statistical Binary Quantization (SBQ) * Label-based filtering combined with DiskANN index. [source]
* Create a label column in your table. It has to be an array of `smallint`s. Other types will revert to using the post-filtering. * Create a diskann index that uses both the embedding and the label column. * Use the `&&` (array intersection) operator in search queries. * Optional, but recommended: Use a separate table and joins to translate smallint labels to meaningful descriptions. [source]
* DiskANN index supports `cosine`, `l2` and `inner_product` distance metrics, not the entire pgvector's set of distance metrics. * Label-based filtering is only supported for `smallint` arrays and the `&&` operator. Other types will revert to using the post-filtering. * DiskANN is best suited for datasets where `hnsw` index would be too large to fit into memory. For smaller datasets, `hnsw` is still a good choice. [source]
* Exact and approximate nearest neighbor search (with optional HNSW and IVFFlat indexes) * Single-precision, half-precision, binary, and sparse vectors * L2 distance, inner product, cosine distance, L1 distance, Hamming distance, and Jaccard distance * Any language with a Postgres client Plus ACID compliance, point-in-time recovery, JOINs, and all of the other great features of Postgres [source]
Without vector indexes (also called **flat indexes** by other vector stores), queries sequentially scan through all vectors for the given query, and compute the distance to the query vector. This is computationally expensive, but guarantees to find the nearest neighbors (also called **exact nearest neighbors** or **KNN**). [source]
<Card title="Pgvectorscale" href="./pgvectorscale" icon="vector-square"> DiskANN index support for pgvector </Card> [source]
DiskANN index support for pgvector [source]
<Note> Vector index works only for column with one of the vector types described above </Note> [source]
LibSQL vector index optionally can accept settings which must be specified as variadic parameters of the `libsql_vector_idx` function as strings in the format `key=value`: [source]
| Setting key | Value type | Description | | -------------------- | ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- [source]
<Note> Vector index for column of type `T1` with `max_neighbors=M` and `compress_neighbors=T2` will approximately use $\texttt{N} (Storage(\texttt {T1}) + \texttt{M} \cdot Storage(\texttt{T2}))$ storage bytes for `N` rows. </Note> [source]
<Note> By default, similarity searches use a linear scan over the table. An experimental sparse vector index method is available behind the `--experimental-index-method` flag; without it, consider limiting search to a subset of rows with a WHERE clause for large datasets. </Note> [source]
[`TableDefinition`](/api/classes/server.TableDefinition.md)<`DocumentType`, `Indexes`, `SearchIndexes`, `VectorIndexes`> [source]
A [TableDefinition](/api/classes/server.TableDefinition.md) with this vector index included. [source]
▸ `Protected` **self**(): [`TableDefinition`](/api/classes/server.TableDefinition.md)<`DocumentType`, `Indexes`, `SearchIndexes`, `VectorIndexes`> [source]
Query by running a full text search against a search index. [source]
| Name | Type | Description | | ----------- | -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | | `fieldName` | `FieldName` | The name of the field to compare. This must be listed in the [source]
The configuration for a full text search index. [source]
| Name | Type | | -------------------------- | -------------------------------------------------------- [source]
The configuration for a vector index. [source]
A type describing the configuration of a search index. [source]
A type describing all of the search indexes in a table. [source]
A type describing the configuration of a vector index. [source]
A type describing all of the vector indexes in a table. [source]
The search indexes in a table for a given [GenericTableInfo](/api/modules/server.md#generictableinfo). [source]
The names of search indexes in a table for a given [GenericTableInfo](/api/modules/server.md#generictableinfo). [source]
Extract the config of a search index from a [GenericTableInfo](/api/modules/server.md#generictableinfo) by name. [source]
The vector indexes in a table for a given [GenericTableInfo](/api/modules/server.md#generictableinfo). [source]
The names of vector indexes in a table for a given [GenericTableInfo](/api/modules/server.md#generictableinfo). [source]
Extract the config of a vector index from a [GenericTableInfo](/api/modules/server.md#generictableinfo) by name. [source]
Ƭ **DataModelFromSchemaDefinition**<`SchemaDef`>: `MaybeMakeLooseDataModel`<{ \[TableName in keyof SchemaDef\["tables"] & string]: SchemaDef\["tables"]\[TableName] extends TableDefinition\<infer DocumentType, infer Indexes, infer SearchIndexes, infer VectorIndexes> ? Object : never }, `SchemaDef`\[`"strictTableNameTypes"`]> [source]
* To search a vector index, use the `vectorSearch` field. Read on about [Vector Search](/search/vector-search.md). [source]
| | Value | | ------------------------ | ----- | | Search indexes per table | 4 | | Filters per search index | 16 | | Terms per search query | 16 | | Filters per search query | 8 | | Maximum term length | 32 B | | Maximum result set | 1024 | [source]
| | Value | | ------------------------ | ------------------------ | | Vector indexes per table | 4 | | Filters per vector index | 16 | | Terms per search query | 16 | | Vectors to search by | 1 | | Dimension fields | 1 (value between 2-4096) | | Filters per search query | 64 | | Maximum term length | 32 B | | Maximum result set | 256 (defaults to 10) | [source]
1. Define a search index. 2. Run a search query. [source]
Search indexes are built and queried using Convex's multi-segment search algorithm on top of [Tantivy](https://github.com/quickwit-oss/tantivy), a powerful, open-source, full-text search library written in Rust. [source]
3. \[Optional] A list of `filterField`s  * These are additional fields that are indexed for fast equality filtering within your search index. [source]
This is just a normal [database read](/database/reading-data/.md) that begins by querying the search index! [source]
Search expressions are issued against a search index, filtering and ranking documents by their relevance to the search expression's query. Internally, Convex will break up the query into separate words (called *terms*) and approximately rank documents matching these terms. [source]
1. First, querying the search index using the search filter expression in `withSearchIndex`. 2. Then, filtering the results one-by-one using any additional `filter` expressions. [source]
Additionally, search queries can scan up to 1024 results from the search index. [source]
1. Define a vector index. 2. Run a vector search from within an [action](/functions/actions.md). [source]
* Exactly 1 vector index field.  * The field must be of type `v.array(v.float64())` (or a union in which one of the possible types is `v.array(v.float64())`) * Exactly 1 dimension field with a value between 2 and 4096. * Up to 16 filter fields. [source]
Text indexes support `$text` queries on fields containing string content. [source]
[MongoDB Search](https://www.mongodb.com/docs/atlas/atlas-search/) offers advanced full-text search capabilities, including [configurable dynamic indexing](https://www.mongodb.com/docs/search/index/define-field-mappings/#std-label-fts-configure-dynamic-mappings). We recommend using [MongoDB Search indexes](https://www.mongodb.com/docs/search/index/manage-indexes/#std-label-fts-manage-indexes) instead of text indexes. [source]
The wildcard text index supports `$text` queries on all fields in the collection. Consider the following queries: [source]
This page describes the behavior of [version 3](https://www.mongodb.com/docs/manual/core/indexes/index-types/index-text/text-index-versions/#std-label-text-index-versions) text indexes. [source]
Text indexes are case insensitive. The text index does not distinguish between capitalized and lower-case characters, such as `e` and `E`. [source]
Text indexes support case foldings as specified in [Unicode 8.0 Character Database Case Folding](http://www.unicode.org/Public/8.0.0/ucd/CaseFolding.txt): [source]
[Previous text index versions](https://www.mongodb.com/docs/manual/core/indexes/index-types/index-text/text-index-versions/#std-label-text-index-versions) are only case insensitive for non-diacritic Latin characters `[A-z]`. Previous text index versions treat all other characters as distinct. [source]
[Previous versions](https://www.mongodb.com/docs/manual/core/indexes/index-types/index-text/text-index-versions/#std-label-text-index-versions) of the text index treat characters with diacritics as distinct. [source]
For tokenization, text indexes use the delimiters categorized under `Dash`, `Hyphen`, `Pattern_Syntax`, `Quotation_Mark`, `Terminal_Punctuation`, and `White_Space` in the [Unicode 8.0 Character Database Prop List](http://www.unicode.org/Public/8.0.0/ucd/PropList.txt). [source]
MongoDB supports `$text` queries for various languages. Text indexes use simple language-specific suffix stemming. Text indexes also drop language-specific stop words such as `the`, `an`, `a`, and `and` in English. For a list of the supported languages, see [$text Query Languages on Self-Managed Deployments.](https://www.mongodb.com/docs/manual/reference/text-search-languages/#std-label-text-search-languages) [source]
| Text Index Version | Description | |---|---| | Version 3 | MongoDB 3.2 introduces version 3 of text indexes. Version 3 is the default version for text indexes created in MongoDB 3.2 and later. | | Version 2 | MongoDB 2.6 introduces version 2 of text indexes. Version 2 is the default version for text indexes created in MongoDB 2.6 to 3.0. | | Version 1 | MongoDB 2.4 introduces version 1 of text indexes. MongoDB 2.4 only supports version 1. | [source]
The following command creates a version 2 text index on the `content` field: [source]
For self-managed (non-Atlas) deployments, MongoDB provides a `text` index type that supports searching for string content in a collection. To learn more about self-managed text indexes, see [Text Indexes on Self-Managed Deployments.](https://www.mongodb.com/docs/manual/core/indexes/index-types/index-text/#std-label-index-feature-text) [source]
b.com/docs/manual/core/indexes/index-types/index-text/specify-text-index-language/#std-label-specify-language-field-text-index-example) for an example. | | `textIndexVersion` | integer | Optional. The `text` index version number. Users can use this option to override the default version number. For available versions, see [Text Index Versions on Self-Managed Deployments.](https://www.mongodb.com/docs/manual/core/indexes/index-types/index-text/text-index-versions/#std-label-text-index-versions) | [source]
▸ **withSearchIndex**<`IndexName`>(`indexName`, `searchFilter`): [`OrderedQuery`](/api/interfaces/server.OrderedQuery.md)<`TableInfo`> [source]
Ƭ **NamedVectorIndex**<`TableInfo`, `IndexName`>: [`VectorIndexes`](/api/modules/server.md#vectorindexes)<`TableInfo`>\[`IndexName`] [source]

Related concepts

vector index — is a hyponym of Search and vector indexes; pgvector / Atlas Vector Search / Convex vector index — an index type, not the Pinecone sense
text index — is a hyponym of Search and vector indexes; full-text/search indexes (Atlas Search, Convex search index, Postgres FTS)
index build — is a dependent of Search and vector indexes
index definition — is a part of Search and vector indexes
GIN index — is a hyponym of Search and vector indexes; Postgres
withIndex — is a instance of Search and vector indexes; Convex index API
B-tree — is a hyponym of Search and vector indexes
wildcard index — is a hyponym of Search and vector indexes
clustered index — is a hyponym of Search and vector indexes
index overhead — is a problem of Search and vector indexes
compound index — is a hyponym of Search and vector indexes; MongoDB 'compound' = SQL 'composite'/'multicolumn'
multikey index — is a hyponym of Search and vector indexes
geospatial index — is a hyponym of Search and vector indexes
sparse index — is a hyponym of Search and vector indexes
unique constraint — is a near-synonym of Search and vector indexes; backed by a unique index in every engine here

Children

No children recorded.

← the whole tree · 3D view· how to read this page