Search Implementation

This documentation page outlines the implementation details of the search functionality used in the documentation site.

Overview

Image of the search component:

How It Works

The search system follows a simple flow:

1. Build Time: The generate-search-index.js script runs during prebuild to scan all MDX files and create a search index
2. Storage: The index is stored as app/search-index.json
3. API Endpoint: An API endpoint serves the search index to the frontend
4. Frontend: The SearchComponent fetches the index via the API endpoint
5. Middleware: Requests go through Next.js middleware for processing
6. Search: Client-side search runs against the loaded index with real-time results

Files

scripts/generate-search-index.js

1. Recursively scans the content directory for .mdx and .md files
2. Reads each file and extracts frontmatter using gray-matter
3. Cleans content by removing code blocks, inline code, and markdown formatting
4. Generates URL-friendly slugs from file paths
5. Limits preview to 200 characters and searchable content to 3000 characters
6. Writes the complete index to app/search-index.json

app/search-index.json

1. Stores the generated search index as a JSON array
2. Contains entries with: slug, title, description, content, keywords, category, fullContent
3. Served as a static file accessible at /search-index.json
4. Loaded by the frontend on component mount

app/api/search-index/route.ts

1. Defines a GET API route at /api/search-index
2. Reads and returns the contents of app/search-index.json

components/SearchComponent.tsx

1. Fetches /api/search-index when component mounts
2. Listens for user input with 200ms debouncing
3. Splits search query into words and scores each document
4. Ranks results by score (title matches > keywords > description > content)
5. Highlights matched terms in yellow
6. Displays top 10 results in a dropdown

Setup

Add to package.json:

{
  "scripts": {
    "prebuild": "node scripts/generate-search-index.js",
    "build": "next build"
  }
}

The search index is automatically regenerated before every build.