Integrate Azure Vector Search

Implementing Vector Search with Azure Cognitive Search (PART I)

What is Vector Search?

Vector search is a technique that enables search engines to understand the semantic meaning of queries and documents. Instead of relying solely on keyword matches, it uses mathematical representations (vectors) to capture the context and relationships between words, phrases, or even images. This allows for more accurate and relevant search results, especially when queries are phrased differently or contain synonyms.

Why Use Vector Search?

• Semantic Understanding: Finds results based on meaning, not just exact words.
• Handles Misspellings & Synonyms: Returns relevant results even with typos or different terminology.
• Improved Ranking: Orders results by contextual relevance, not just keyword frequency.

How Does It Work?

1. Vectorization: Text or data is converted into numerical vectors using embedding models.
2. Indexing: These vectors are stored in a specialized index for fast retrieval.
3. Querying: User queries are also vectorized, and the system finds the closest matches using similarity metrics.

Let's explore how vector search compares to traditional keyword search and how it can be implemented using Azure Cognitive Search. In this first part of the blog post, we will focus on understanding the concepts, setting up the necessary resources in Azure and creating vector search index.

Traditional Keyword Search :

In traditional keyword-based search, the system looks for exact keyword matches in the document and ranks results based on how often those keywords appear. The more frequently a keyword shows up, especially in prominent places like titles or headings, the higher the document ranks.

Lets take an example of searching for shoes on an E-commerce Site

Product Descriptions:

1. Top-rated running shoes for marathon training
2. Men’s lightweight running shoes for optimal performance
3. Best sneakers for jogging and daily wear
4. Top-rated sneakers for running

Query Results

1. Search: sneakers -> Matches 3 and 4, because they include the exact word sneakers.
2. Search: running -> Matches 1, 2, and 4, as these explicitly mention running.
3. Search: Best running shoes -> Returns all four, because each document contains best, running, or shoes and will return results accordingly
4. Search: sneakars -> It will not return anything as the spelling mistake is there.

Vector Search / Semantic Search:

Enhancing search engines to return more relevant results based on the meaning of queries rather than just keyword matching. Now when we are searching for Best Running Shoes it should have returned products with following order based on semantic meaning of product description:

#1 – Top‑rated running shoes for marathon training – strongest match: running shoes, top-rated (synonym for best).

#2 – Men’s lightweight running shoes for optimal performance – very close match (running shoes, performance-related).

#3 – Top‑rated sneakers for running – matches top-rated + running even though it says sneakers.

#4 – Best sneakers for jogging and daily wear – includes best and sneakers aligned with running/casual wear.

Before diving deeper into the implementation of vector search lets understand few terminologies related to vector search:

Vectorization/ Vector Embeddings:

Vectorization is simply turning things like words, sentences, pictures, or even whole documents into lists of numbers that computers can work with, so one piece of text becomes something like [0.32, 0.42, …], and another text with a similar meaning ends up with numbers that are close to it. It's like placing ideas on a map: sneakers and shoes sit near each other because they're alike, while heels is farther away. This makes it easy for a computer to group similar items, find synonyms, or rank search results by meaning rather than just exact words. There are different models which we can use to do vector embedding for queries and also for storing data in vector format.

Text-embedding-ada-002

This is text embedding model provided by Open AI. It is used to transform text into fixed length vectors. It can be used for various applications like Semantic Search, Clustering and Classification or Recommendation system. In this blog post we will be using this model to generate vector embeddings.

Distance and Similarity Matrix:

A distance matrix tells you how different two items are. The smaller the value, the more similar the items. A similarity matrix tells you how similar two items are. The higher the value, the more similar the items.

1. Cosine Similarity : Measures the angle between two vectors, focusing on direction over length. Its great for text comparisons.
2. Euclidean Distance : The straight-line distance between points. Useful but less effective in high dimensions.
3. Dot Product : Captures both direction and magnitude. It is commonly used inside models.

k-NN and ANN

1. k-Nearest Neighbors (k-NN) : Finds exact closest vectors by comparing to every point—accurate but slow for large data.
2. Approximate Nearest Neighbors (ANN) : Sacrifices a bit of accuracy for huge speed gains using smart indexing. We will be using algorithm hnsw (Hierarchical Navigable Small World) for ANN.

Compressor

It is used to compress vector data to save storage space and improve search performance. Product Quantization or Scalar Quantization are two techinques which can be used to compress vector data. In this blog post we will be using Scalar Quantization to compress the vector data.

Vectorizer

A vectorizer is a component that converts text into vectors using a specific model. It defines how the text will be transformed into numerical representations. In this blog post, we will use the Azure OpenAI service to create a vectorizer that uses the text-embedding-ada-002 model.

Let's now see how all this concepts can be implemented in Azure Cognitive Search to perform vector search.

Pre-requisite

1. You must have an Azure account and active subscription.
2. Your account must be approved for Azure OpenAI

Create Azure Open AI resource in Azure Portal

1. Go to azure portal and Search for Azure OpenAI.
2. Create a new resource of Azure OpenAI Service.
3. Follow the prompt as mentioned in below image and wait till deployment is completed.
4. Go to the AzureFoundry Azure AI Playground.
5. Go to Deployment Model and deploy Text-embedding-ada-002 model by following the prompt as shown below:
6. Once deployment is completed we can consume the API to generate the text embeddings using the api-endpoint and api-key.
7. You can test the API using any API testing tool like Postman. Here’s an example of how to call the API to get embeddings for a text:

1POST https://<your-resource-name>.openai.azure.com/openai/deployments/text-embedding-ada-002/embeddings?api-version=2023-05-15
2Content-Type: application/json  
3api-key: <your-api-key>
4Body :
5{
6  "input": "Best running shoes for marathon training",
7  "model": "text-embedding-ada-002"
8}

8. The response will contain the vector embeddings for the input text, which you can use for vector search.

9. We will use these embeddings to store vector data in Azure Cognitive Search index for vector field and querying vector fields.

Create Azure Search Index

1. On Azure Portal create the Search Service Resource
   • Search for Azure Cognitive Search in the Azure Portal.
   • Click on Create and fill in the required details like Subscription, Resource Group, Name, Region, and Pricing tier.
   • Click on Review + Create to deploy the search service.
2. Once the deployment is complete, navigate to the newly created Azure Cognitive Search resource. You will find the resource url there.
3. In Azure Search Service Resource go to Settings > Keys to get the admin key. You will need this key to interact with the search service from your application.

Creating Azure Search Index Using API

You can create an Azure Cognitive Search index programmatically using the REST API or SDKs. In this blog post we will see how we can create the index using Rest API.

Create Index Definition

1. Define fields for the index. For vector search, you need to include a field for the vector embeddings. Vector fields should be of type Collection(Edm.Single) as it will store the list of vectors. Here’s an example of an index definition:

1{
2  "name": "products-index",
3  "fields": [
4    {
5      "name": "Id",
6      "type": "Edm.String",
7      "key": true,
8      "filterable": true
9    },
10    {
11      "name": "Title",
12      "type": "Edm.String",
13      "searchable": true
14    },
15    {
16      "name": "Description",
17      "type": "Edm.String",
18      "searchable": true
19    },
20    {
21      "name": "TitleVector",
22      "type": "Collection(Edm.Single)",
23      "searchable": true,
24      "dimensions": 1536,
25      "vectorSearchProfile": "vector-profile-blog"
26    },
27    {
28      "name": "DescriptionVector",
29      "type": "Collection(Edm.Single)",
30      "searchable": true,
31      "dimensions": 1536,
32      "vectorSearchProfile": "vector-profile-blog"
33    }
34  ]
35}

2. Add the vector search profile to the index definition. This profile defines how the vectors will be indexed and searched. Replace the apiKey and resourceUri with your actual Azure OpenAI API key and endpoint.

1"vectorSearch": {
2    "algorithms": [
3      {
4        "name": "hnsw-1",
5        "kind": "hnsw",
6        "hnswParameters": {
7          "m": 4,
8          "efConstruction": 400,
9          "efSearch": 500,
10          "metric": "cosine"
11        }
12      }
13    ],
14    "compressions": [
15      {
16        "name": "default-compression",
17        "kind": "scalarQuantization"
18      }
19    ],
20    "vectorizers": [
21      {
22        "name": "vector-search-blog-vectorizer",
23        "kind": "azureOpenAI",
24        "azureOpenAIParameters": {
25          "resourceUri": "https://<your-openai-service-name>.openai.azure.com/",
26          "deploymentId": "text-embedding-ada-002",
27          "modelName": "text-embedding-ada-002",
28          "apiKey": "************"
29        }
30      }
31    ],
32    "profiles": [
33      {
34        "name": "vector-profile-blog",
35        "algorithm": "hnsw-1"
36      }
37    ]
38  }

3. Add a semantic configuration to the index to enable semantic search capabilities. This configuration defines how the search engine should interpret and rank results based on semantic meaning.

1"semantic": {
2    "configurations": [
3      {
4        "name": "semantic-blog",
5        "prioritizedFields": {
6          "titleField": {
7            "fieldName": "Title"
8          },
9          "prioritizedContentFields": [
10            {
11              "fieldName": "Title"
12            },
13            {
14              "fieldName": "Description"
15            }
16          ],
17          "prioritizedKeywordsFields": [
18            {
19              "fieldName": "Title"
20            }
21          ]
22        }
23      }
24    ]
25  }

4. Use the Azure Cognitive Search REST API to create the index with the defined fields and vector search profile.

1POST: https://<your-search-service-name>.search.windows.net/indexes?api-version=2024-07-01
2Content-Type: application/json      
3api-key: <your-api-key>
4Body: Add Index Definition Json 

5. After creating the index, you can verify from the Azure Portal to ensure that the index has been created successfully.

Conclusion

In this first part of the blog post, we have explored the concepts of vector search, how it differs from traditional keyword search, and how to set up the necessary resources in Azure. We also created an Azure Cognitive Search index with vector fields and semantic configuration. In the next part, we will see how to add data into the Azure Cognitive Search index and perform vector search queries using the API.

References

• Azure Cognitive Search Documentation
• Create and Deploy Azure OpenAI Resource