Package edu.internet2.middleware.grouper.mcp

Class GrouperMcpDocSearchIndex

java.lang.Object
edu.internet2.middleware.grouper.mcp.GrouperMcpDocSearchIndex
public class GrouperMcpDocSearchIndex extends Object
Manages an in-memory Lucene index for MCP document search (RAG). Reads document content from institution-managed database tables via configurable SQL queries, chunks the content, and indexes it for full-text search.

Also indexes the Grouper data field/row dictionary as a built-in source (grouperDataDictionary). Search results from data dictionary entries are filtered by privacy realm access at query time.

The index is built lazily on first search and rebuilt periodically based on the configured reindex interval.

Thread-safe: builds a new index then swaps the reference atomically.

  • Field Details

    • DATA_DICTIONARY_SOURCE_CONFIG_ID

      public static final String DATA_DICTIONARY_SOURCE_CONFIG_ID
      source config id for the built-in data dictionary source
      See Also:

    • chunkSizeCharsOverrideForTesting

      public static int chunkSizeCharsOverrideForTesting
      override chunk size for testing, or -1 for default

    • chunkOverlapCharsOverrideForTesting

      public static int chunkOverlapCharsOverrideForTesting
      override chunk overlap for testing, or -1 for default

  • Constructor Details

    • GrouperMcpDocSearchIndex

      public GrouperMcpDocSearchIndex()

  • Method Details

    • search

      public static List<GrouperMcpDocSearchIndex.DocSearchResult> search(String queryString, int maxResults, String filterSourceConfigId, Subject subject)
      search the document index, filtering by privacy realm access for the subject. uses keyword search (escaped query) for backward compatibility.
      Parameters:
      queryString - the search query
      maxResults - max results to return
      filterSourceConfigId - optional source config id to filter by, or null for all
      subject - the authenticated subject for privacy realm filtering (may be null for no filtering)
      Returns:
      list of search results ordered by relevance

    • search

      public static List<GrouperMcpDocSearchIndex.DocSearchResult> search(String queryString, int maxResults, String filterSourceConfigId, Subject subject, String searchType)
      search the document index, filtering by privacy realm access for the subject
      Parameters:
      queryString - the search query
      maxResults - max results to return
      filterSourceConfigId - optional source config id to filter by, or null for all
      subject - the authenticated subject for privacy realm filtering (may be null for no filtering)
      searchType - "keyword" (default, escaped query) or "lucene" (raw Lucene query syntax)
      Returns:
      list of search results ordered by relevance

    • rebuildIfNeeded

      public static void rebuildIfNeeded()
      rebuild the index if it is stale or not yet built. runs the build in a background thread so the calling thread is not blocked. if a build is already in progress, this is a no-op.

    • forceRebuild

      public static void forceRebuild()
      force rebuild the index now (synchronously, blocks the calling thread)

    • convertHtmlToMarkdown

      public static String convertHtmlToMarkdown(String html)
      convert HTML to markdown using flexmark
      Parameters:
      html - the HTML string
      Returns:
      the markdown string, or empty string if input is blank

    • chunkContent

      public static List<String> chunkContent(String content)
      chunk content into pieces of approximately CHUNK_SIZE_CHARS characters with CHUNK_OVERLAP_CHARS overlap, splitting on paragraph boundaries.
      Parameters:
      content - the full content text
      Returns:
      list of chunks

    • chunkContent

      public static List<String> chunkContent(String content, int chunkSizeChars, int overlapChars)
      chunk content into pieces of approximately chunkSizeChars characters with overlapChars overlap, splitting on paragraph boundaries.
      Parameters:
      content - the full content text
      chunkSizeChars - target chunk size in characters
      overlapChars - overlap between chunks in characters
      Returns:
      list of chunks

    • retrieveChunks

      public static List<GrouperMcpDocSearchIndex.DocSearchResult> retrieveChunks(String sourceConfigId, String name, String url, List<Integer> chunkIndexes, Subject subject)
      retrieve specific chunks by exact match on sourceConfigId, name or url, and chunkIndexes. filters by privacy realm access for the subject.
      Parameters:
      sourceConfigId - the source config id (required)
      name - the document name (optional, at least one of name or url required)
      url - the document url (optional, at least one of name or url required)
      chunkIndexes - the chunk indexes to retrieve (required, non-empty)
      subject - the authenticated subject for privacy realm filtering (may be null)
      Returns:
      list of results ordered by chunkIndex request order

    • isDataDictionaryEnabled

      public static boolean isDataDictionaryEnabled()
      check if the data dictionary indexing is enabled and has any content
      Returns:
      true if enabled and there are data field or row configs

    • hasAnySourcesForSubject

      public static boolean hasAnySourcesForSubject(Subject subject)
      check if there are any doc search sources available for the given subject. considers SQL-based sources and data dictionary access.
      Parameters:
      subject - the subject to check, or null to skip privacy checks
      Returns:
      true if any sources are available

    • listNames

      public static GrouperMcpDocSearchIndex.ListNamesResult listNames(String sourceConfigId, Subject subject, int maxNames)
      list distinct document names for a given sourceConfigId, filtered by privacy realm access.
      Parameters:
      sourceConfigId - the source config id
      subject - the subject for privacy filtering
      maxNames - maximum names to return
      Returns:
      list of distinct names, and whether the list was truncated

    • getConfigIds

      public static Set<String> getConfigIds()
      get the configured SQL-based doc search source config ids
      Returns:
      set of config ids

    • getDocumentationForAiClient

      public static String getDocumentationForAiClient(String configId)
      get the documentation for AI client for a given config id
      Parameters:
      configId - the config id
      Returns:
      the documentation string