Search Tutorial

Introduction

This tutorial covers the GroundX search API.

If you’d rather have an agent search and answer for you instead of wiring this up in application code, see Ingest, Search, and Answer With an Agent.

Under the hood, the GroundX Search API does the following:

• Analyzes your search query and improves it if necessary, then runs a semantic search that avoids the hallucination trap most RAG apps fall into with vectorized search.

• Searches the GroundX buckets where your content and its extra search data are stored. GroundX’s extra search data keeps your content in its original context. The Search API returns more than raw text chunks: a response bundled with intelligible text chunks, extra search data, automatically generated document and section summaries, source URL, and a readable, contextualized, performant version of the text chunk ready for LLM use.

• Uses a proprietary re-ranker that scores every chunk on how well it answers the original question, so the most trustworthy results are on top.

• Merges all of this into a simple text block you can send to the LLM of your choice, for accurate, contextualized, hallucination-free responses from your content.

LLM Integration

GroundX handles the heavy lifting in the background. Here’s the flow:

1. Make an API search request.

Example:

1
import os
2
from groundx import GroundX
3
4
client = GroundX(
5
    api_key=os.environ["GROUNDX_API_KEY"],
6
)
7
8
bucket_id = 0
9
query = "How many workers walk to work in the Bay Area?"
10
11
result = client.search.content(
12
    id=bucket_id, 
13
    query=query,
14
)
15
16
llm_text = result.search.text

2. Retrieve the search.text property and pass it on to the LLM of your choice.

Example:

1
import os
2
from openai import OpenAI
3
openai_client = OpenAI(
4
    api_key=os.environ["OPENAI_API_KEY"],
5
)
6
7
completion = openai_client.chat.completions.create(
8
  model="gpt-4o-mini",
9
  messages=[
10
    {
11
        "role": "system", 
12
        "content": """Get response from this data:
13
===
14
%s
15
===
16
"""
17
      % (llm_text),
18
    },
19
    {
20
        "role": "user", 
21
        "content": query,
22
    }
23
  ]
24
)

3. Get a response from the LLM using your retrieved data.

Example using:

1
print(completion.choices[0].message.content)

The end result: retrievals that outperform traditional vector systems and boost the accuracy of LLM completions.

Getting Started

The example above used placeholder values. Here’s the same flow with more detail on each parameter.

To make an API request, do the following:

Ingest content

Make sure you’ve already uploaded the content you want to search to a GroundX bucket. See the documentation on content ingestion for more information.

Set up environment

Set up your environment with the GroundX SDK.

1
from groundx import GroundX
2
3
client = GroundX(
4
    api_key="YOUR_API_KEY",
5
)

Bucket or Group ID

Get the ID of the group or bucket you want to search.

Example:

1
bucket_id = 3839

Search Query String

Create a search query string — a question or the keywords you want to search for.

If the string is more than 30 words long, GroundX automatically rewrites it with keywords using its internal LLM.

GroundX processes the string against your content and its extra search data to retrieve the most relevant results.

Example:

1
query = "How many workers walk to work in the Bay Area?"

Number of Results

Optionally, set the number of results returned by the search request. By default, search queries return up to 20 results.

Example:

1
n = 10

API Request

Make the API search request, including the ID and search query string in the request body.

In addition to id, query, and n, the search request accepts a few more optional parameters:

For search rate limits (concurrent requests, requests per minute) and error-handling guidance, see the Direct SDK/API Quickstart’s Errors and Rate Limits section.

Example:

1
response = client.search.content(
2
    id=bucket_id, 
3
    query=query,
4
    n=n,
5
)
6
7
print(response.search.text)

API Response

The request returns a Search object.

The Search schema includes the following details.

Response sample:

1
{
2
    "count": 10,
3
    "query": "How many workers walk to work in the Bay Area?",
4
    "results":  [
5
         {
6
            "bucketId": 6839,
7
            "documentId": "81111b9e-ff13-41ef-a89a-af667a4b1cac",
8
            "chunkId": "3af6a2c1-9e4d-4c2a-8f36-1e2f0a5d7b90",
9
            "processId": "2b7e6a2c-1e4d-4c2a-8f36-1e2f0a5d7b90",
10
            "score": 357.09656,
11
            "fileSummary": "This document is a detailed report from the Healthy Communities Data and Indicators Project by UCSF and CDPH...",
12
            "sectionSummary": "Given a row from the provided TSV table, this row represents data about the commuting habits of workers in a specific region in California over two time periods: 2005-2007 and 2008...",
13
            "searchData": {
14
                "documentSummary": "This document is a detailed report from the Healthy Communities Data and Indicators Project by UCSF and CDPH...",
15
                "sectionSummary": "Given a row from the provided TSV table, this row represents data about the commuting habits of workers in a specific region in California over two time periods: 2005-2007 and 2008...",
16
                "fullTitle": "Percent of population aged 16 years or older whose commute to work is 10 or more minutes/day by walking or biking",
17
                "publisher": "California Department of Public Health"
18
            },
19
            "sourceUrl": "https://upload.groundx.ai/file/hciwalkbicycle778narrativeandexamples3-26-14.pdf",
20
            "suggestedText": "In the Bay Area between 2005-2007, 70,287 out of 3,122,743 workers, or 2.25%, walked for 10 or more minutes per day. This number increased to 81,737 out of 3,202,761 workers, or 2.55%, in the 2008-2010 period. In Butte, 1,266 out of 85,042 workers, or 1.49%, walked for 10 or more minutes per day between 2005-2007. This increased to 1,678...",
21
            "text": "\t|\t\t|\t2005-2007\t|\t\t|\t\t|\t2008-2010\n\t|\tWorkers who\t|\t\t|\t\t|\tWorkers who\n\t|\twalk ≥10\t|\t\t|\t\t|\twalk ≥10\nRegion name\t|\tmin / day\t|\tTotal...",
22
            "multimodalUrl": "https://upload.groundx.ai/file/hciwalkbicycle778narrativeandexamples3-26-14-table2.jpg",
23
            "boundingBoxes": [
24
                {
25
                    "pageNumber": 2,
26
                    "topLeftX": 72.5,
27
                    "topLeftY": 140.2,
28
                    "bottomRightX": 520.8,
29
                    "bottomRightY": 310.6
30
                }
31
            ],
32
            "pages": [
33
                {
34
                    "number": 2,
35
                    "imageUrl": "https://upload.groundx.ai/file/hciwalkbicycle778narrativeandexamples3-26-14-page2.jpg",
36
                    "width": 1700,
37
                    "height": 2200
38
                }
39
            ]
40
            }
41
    ],
42
    "score": 369.19244,
43
    "text": "fullTitle: Percent of population aged 16 years or older whose commute to work is 10 or more minutes/day by walking or biking\npublisher: California Department of Public Health\nIn the Bay Area between 2005-2007, 70,287 out of 3,122,743 workers, or 2.25%, walked for 10 or more minutes per day. This number increased to 81,737 out of 3,202,761 workers, or 2.55%, in the 2008-2010 period. In Butte, 1,266 out of 85,042 workers, or 1.49%, walked for 10 or more minutes per day between 2005-2007. This increased to 1,678 out of 81,321 workers, or 2.06%, in 2008-2010..."
44
}

Recommended data

The most significant property in the search response is text: a string containing the automatically rewritten text chunks and the extra search data added to the original content. This content, generated by GroundX’s internal LLM, is intelligible, contextualized, and machine-understandable — pass it straight to your LLM.

Advanced data handling

We recommend passing search.text directly to your LLM, but you can also add your own logic around the search.results property.

results is a list of the original text chunks that matched the search query, ordered by a score based on semantic search and re-ranking. Each result includes the suggested text mentioned above — the more intelligible, performant version of the text chunk.

Each result also carries automatically generated extra search data, like summaries of the document and section the chunk is found in. As of SDK 3.5.4, the document-level and section-level summaries are also available directly on each result as the top-level fileSummary and sectionSummary fields, in addition to the documentSummary/ sectionSummary pair inside searchData — use the top-level fields when you just need the summary text and don’t need the rest of searchData.

The GroundX URL, document ID, and bucket ID of the source content are provided too.

Each result also carries a unique chunkId, and — when the chunk is a visual element such as a table or chart — a multimodalUrl pointing to an image clipping of that element. For source-attribution UIs, boundingBoxes gives the coordinates of where the chunk appears on its source page(s), and pages gives the rendered page image(s) (with pixel dimensions) those coordinates are relative to.

Use verbosity (see API Request above) to control whether results is returned at all, and whether each result includes the full searchData object.

Final Details

You’re now ready to integrate GroundX with your LLM to generate accurate, hallucination-free responses from your own content.

Next Steps

• For tips on turning search results into consistent, high-quality LLM responses, see Prompting and Integration.
• Want an agent to run search and answer for you instead of wiring this up in application code? See Ingest, Search, and Answer With an Agent (also linked in the Introduction above).