KA Coders Blog

Hybrid Queries in Azure Vector Search: Combining Keyword, Vector, and Semantic Search

In the previous article , we have configured Azure Cognitive Search and Azure OpenAI to implement a vector search index. Let's take it a step farther and investigate vector index querying. Additionally, we will demonstrate how combining vector and keyword search with semantic re-ranking may improve the results.

We'll continue working with the product-index created in the previous article , it will store data related to footwear.

📝 Step 1: Insert Data into `products-index`

products-index will store Id, Title, Description, TitleVector, and DescriptionVector fields.

Generate the vector embedding for Title and Description fields using Azure OpenAI API.

1POST: https://<your-openai-service-name>.openai.azure.com/openai/deployments/text-embedding-ada-002/embeddings?api-version=2023-05-15
2Content-Type: application/json
3api-key: <your-api-key>
4Body:
5{
6    "input" :"Nike Air Zoom Pegasus 40"
7}

Add the documents to product-index along with vector fields

1POST: https://<your-search-service-name>.search.windows.net/indexes/products-index/docs/index?api-version=2024-07-01
2Content-Type: application/json
3api-key: <your-api-key>
4Body: 
5{
6    "value": [
7    {
8        "@search.action": "upload",
9        "Id": "1",
10        "Title": "Nike Air Zoom Pegasus 40",
11        "Description":"Men's running shoes with breathable mesh, lightweight cushioning, and durable outsole.",
12        "TitleVector":[vector-embedding],
13        "DescriptionVector":[vector-embedding],
14    },
15    {
16        "@search.action": "upload",
17        "Id": "2",
18        "Title":"Adidas Ultraboost Light",
19        "Description":"Women's running shoes with responsive boost midsole and adaptive Primeknit upper.",
20        "TitleVector":[vector-embedding],
21        "DescriptionVector":[vector-embedding],
22    },
23    {
24        "@search.action": "upload",
25        "Id":"3",
26        "Title":"Puma Smash v2",        
27        "Description":"Unisex casual sneakers with classic suede design and comfortable rubber sole." ,
28        "TitleVector":[vector-embedding],
29        "DescriptionVector":[vector-embedding],       
30    },
31    {
32        "@search.action": "upload",
33        "Id":"4",
34        "Title":"Skechers Arch Fit",
35        "Description":"Men's walking shoes featuring arch support and a lightweight flexible design." ,
36        "TitleVector":[vector-embedding],
37        "DescriptionVector":[vector-embedding],
38    },
39    {
40        "@search.action": "upload",
41        "Id":"5",
42        "Title":"Bata Formal Leather Shoes",
43        "Description":"Classic men's leather shoes for formal wear with cushioned insole for comfort." ,
44        "TitleVector":[vector-embedding],
45        "DescriptionVector":[vector-embedding],
46    },
47    {
48        "@search.action": "upload",
49        "Id":"6",
50        "Title":"Crocs Classic Clog",
51        "Description":"Unisex waterproof clogs with slip-on comfort, lightweight build, and multiple colors.",
52        "TitleVector":[vector-embedding],
53        "DescriptionVector":[vector-embedding],
54    },
55    {
56        "@search.action": "upload",
57        "Id":"7",
58        "Title":"Reebok Nano X3",
59        "Description":"Training shoes built for crossfit with stable sole and breathable upper." ,
60        "TitleVector":[vector-embedding],
61        "DescriptionVector":[vector-embedding],
62    },
63    {
64        "@search.action": "upload",
65        "Id":"8",
66        "Title":"Woodland Outdoor Boots",
67        "Description":"Rugged men's boots with genuine leather, high ankle support, and durable grip sole." ,
68        "TitleVector":[vector-embedding],
69        "DescriptionVector":[vector-embedding],
70    },
71    {
72        "@search.action": "upload",
73        "Id":"9",
74        "Title":"Converse Chuck Taylor All Star",
75        "Description":"Unisex canvas sneakers with timeless design and rubber sole, perfect for casual wear.",
76        "TitleVector":[vector-embedding],
77        "DescriptionVector":[vector-embedding],
78    },
79    {
80        "@search.action": "upload",
81        "Id":"10",
82        "Title":"New Balance 574",
83        "Description":"Classic retro running shoes with suede-mesh upper and EVA cushioning." ,
84        "TitleVector":[vector-embedding],
85        "DescriptionVector":[vector-embedding],
86    }]
87}

🔍 Step 2: Querying the Index

Keyword Search:

Querying with plain text will return documents that contains the search text keywords. Below is the example of keyword search:
Input Query: women's comfortable sneakers

1POST:https://<your-search-service-name>.search.windows.net/indexes/<index-name>/docs/Search?api-version=2024-07-01
2Content-Type: application/json      
3api-key: <your-api-key>
4Body:{
5    "count": true, // Display results count
6    "select": "Title, Description ",   // Field names for retreival 
7    "search": "women's comfortable sneakers" // search text
8}

Results : It will return the documents that has exact match with women's or comfortable or sneakers in the docuemnt. in title or description field.

1"value": [
2        {
3            // match-reason: Women's in Description field
4            "@search.score": 0.8394282,
5            "Id": "2",
6            "Title": "Adidas Ultraboost Light",
7            "Description": "Women's running shoes with responsive boost midsole and adaptive Primeknit upper."
8        },
9        {
10            // match-reason: Comfortable and sneakers in Description field
11            "@search.score": 0.77041245,
12            "Id": "3",
13            "Title": "Puma Smash v2",
14            "Description": "Unisex casual sneakers with classic suede design and comfortable rubber sole."
15        },
16        {
17            // match-reason: Sneakers in Description field
18            "@search.score": 0.16044298,
19            "Id": "9",
20            "Title": "Converse Chuck Taylor All Star",
21            "Description": "Unisex canvas sneakers with timeless design and rubber sole, perfect for casual wear."
22        }
23    ]

✅ Pros: Works well for exact term matches.
❌ Cons: May miss semantically similar results (e.g., casual shoes ≠ sneakers).

Vector Search:

Input Query: footwear for everyday wear
Generate the vector embedding for the the input query by using Azure OpenAI API. (Refer the OpenAI API we used while inserting documents to the product-index to generate vector embedding for title and description fields).
Use following request to get the results usign vector embeddings.

1POST:https://<your-search-service-name>.search.windows.net/indexes/<index-name>/docs/Search?api-version=2024-07-01
2Content-Type: application/json    
3api-key: <your-api-key>
4Body: {
5    "count": true,
6    "select": "Title, Description ",
7    "vectorQueries": [
8        {
9            "kind": "vector",
10            "vector":[..] //Vector Embeddings,
11            "exhaustive": true,
12            "fields": "DescriptionVector,TitleVector",          
13            "k": 2  //Number of results with distinct search score to return
14        }
15    ]
16}

Results: It will return the documents that are semantically similar to the input query.

1{
2    "@odata.context": "https://vector-search-blog.search.windows.net/indexes('products-index')/$metadata#docs(*)",
3    "@odata.count": 4,
4    "value": [
5        {
6            "@search.score": 0.01666666753590107,
7            "Title": "Puma Smash v2",
8            "Description": "Unisex casual sneakers with classic suede design and comfortable rubber sole."
9        },
10        {
11            "@search.score": 0.01666666753590107,
12            "Title": "Crocs Classic Clog",
13            "Description": "Unisex waterproof clogs with slip-on comfort, lightweight build, and multiple colors."
14        },
15        {
16            "@search.score": 0.016393441706895828,
17            "Title": "Woodland Outdoor Boots",
18            "Description": "Rugged men's boots with genuine leather, high ankle support, and durable grip sole."
19        },
20        {
21            "@search.score": 0.016393441706895828,
22            "Title": "Converse Chuck Taylor All Star",
23            "Description": "Unisex canvas sneakers with timeless design and rubber sole, perfect for casual wear."
24        }
25    ]
26}

In our previous blog post, we have also set up the vectorizer for our index. Hence, we can pass the vector query as raw text. Vectorizer will convert the input query to the corresponding vector.

1// Body for the search request 
2{
3    "count": true,
4    "select": "Title, Description ",
5    "vectorQueries": [
6        {
7            "kind": "text", // Input text query to be vectorized
8            "text": "footwear for everyday wear", // Input text query
9            "exhaustive": true,
10            "fields": "DescriptionVector,TitleVector",          
11            "k": 2
12        }
13    ]
14}

The above request will return the same results as the previous vector search example, where we passed the corresponding vector embeddings for the search text.

✅ Pros: Retrieves results based on meaning, not just keywords.
❌ Cons: Less Precise Without Re-Ranking. Raw vector similarity may bring “fuzzy” matches.

Semantic Search with Keyword Search:

In our previous blog post, we have set up the semantic configuration for the product-index. Semantic configurations let Azure re-rank results based on their relevance to the query's intent. This is particularly useful when combined with keyword search.

Input Query: footwear for everyday wear

1// Body for the search request
2{
3  "count":true,
4  "search": "footwear everyday wear",
5  "queryType": "semantic",
6  "semanticConfiguration": "semantic-blog" 
7}

Without semantic search , keyword search will return the following results:

1"value": [
2        {
3            "@search.score": 0.8394282,
4            "Id": "5",
5            "Title": "Bata Formal Leather Shoes",
6            "Description": "Classic men's leather shoes for formal wear with cushioned insole for comfort."
7        },
8        {
9            "@search.score": 0.6099695,
10            "Id": "9",
11            "Title": "Converse Chuck Taylor All Star",
12            "Description": "Unisex canvas sneakers with timeless design and rubber sole, perfect for casual wear."
13        }
14    ]

With semantic search enabled, the results are re-ranked to prioritize relevance:

1"value": [
2        {
3            "@search.score": 0.6099695,
4            "@search.rerankerScore": 2.402677059173584,
5            "Id": "9",
6            "Title": "Converse Chuck Taylor All Star",
7            "Description": "Unisex canvas sneakers with timeless design and rubber sole, perfect for casual wear."
8        },
9        {
10            "@search.score": 0.8394282,
11            "@search.rerankerScore": 1.9392085075378418,
12            "Id": "5",
13            "Title": "Bata Formal Leather Shoes",
14            "Description": "Classic men's leather shoes for formal wear with cushioned insole for comfort."
15        }
16    ]

✅ Pros: AI-powered re-ranking, relevant snippets and Q&A-style answers.
❌ Cons: Might miss matches if the kewords aren’t present at all.

Hybrid Search (Keyword + Vector + Semantic) :

Input Query : footwear for everyday wear

1// Body for the search request
2{
3  "count":true,
4  "search": "footwear for everyday wear",
5  "queryType": "semantic",
6  "semanticConfiguration": "semantic-blog",
7  "select": "Title, Description,Id ",
8    "vectorQueries": [
9        {
10            "kind": "text",
11            "text": "footwear for everyday wear",
12            "exhaustive": true,
13            "fields": "DescriptionVector,TitleVector",          
14            "k": 1
15        }
16    ]
17}

🔎 How it works:

Vector search → Finds semantically similar documents.
Keyword search → Adds lexical filtering.
Semantic search → Re-ranks results.

Results:

1"value": [
2        {
3            "@search.score": 0.03306011110544205,
4            "@search.rerankerScore": 2.526563882827759,
5            "Id": "9",
6            "Title": "Converse Chuck Taylor All Star",
7            "Description": "Unisex canvas sneakers with timeless design and rubber sole, perfect for casual wear."
8        },
9        {
10            "@search.score": 0.01666666753590107,
11            "@search.rerankerScore": 2.271080255508423,
12            "Id": "8",
13            "Title": "Woodland Outdoor Boots",
14            "Description": "Rugged men's boots with genuine leather, high ankle support, and durable grip sole."
15        },
16        {
17            "@search.score": 0.016129031777381897,
18            "@search.rerankerScore": 2.238867998123169,
19            "Id": "7",
20            "Title": "Reebok Nano X3",
21            "Description": "Training shoes built for crossfit with stable sole and breathable upper."
22        },
23        {
24            "@search.score": 0.01666666753590107,
25            "@search.rerankerScore": 2.134078025817871,
26            "Id": "5",
27            "Title": "Bata Formal Leather Shoes",
28            "Description": "Classic men's leather shoes for formal wear with cushioned insole for comfort."
29        }
30    ]

✅ Best of all worlds → High recall, high precision, and user-friendly results.

🎯 Conclusion:

Keyword search = Good for precision,but fails to handle typos and semantic meaning.
Vector search = Great for semantic coverage, but noisy alone.
Semantic re-ranking = Boosts relevance, but needs filtered results to work well.
Hybrid search = The winning formula — broad recall from vectors, precision from keywords, and smart ranking from semantics.

If you’re building modern search in Azure, consider combining keyword, vector, and semantic search for the best user experience. Hybrid queries unlock the full power of Azure Cognitive Search, delivering both precision and relevance for your applications.

Azure Vector Search - Part II