MCP NVIDIA NIM
TypeScript MCP Server for Resilient NVIDIA Cloud Model Orchestration
High-resilience Model Context Protocol (MCP) server orchestrating NVIDIA Cloud AI models, featuring a 40 RPM sliding-window FIFO rate limiter, zero-downtime fallback chains, and an automated CI/CD test matrix across Node runtimes.
Overview
`mcp-nvidia-nim` is an open-source Model Context Protocol (MCP) server built in TypeScript that connects autonomous AI agents and coding assistants (Google Antigravity, Claude Desktop, Cursor) directly to NVIDIA's high-capacity NIM Cloud APIs.
Designed for production workloads and agentic pairing, it exposes specialized tools for deep architectural reasoning (Nemotron Ultra 550B), code review (Poolside Laguna XS, Mistral Large), multimodal computer vision (Llama 3.2 90B Vision), and structured document parsing.
Architecture & Resilience Engineering
- Sliding-Window FIFO Rate Limiter: Implements a strict in-memory FIFO queue enforcing a 40 requests-per-minute threshold (matching NVIDIA's free cloud quota), preventing rate-limit drops during parallel agent tool invocations.
- Dynamic Fallback Chains: Detects upstream HTTP 404, 403, 500, or network timeouts and immediately fails over to hot-standby models without terminating the caller's agent loop.
- Proactive Health Probing & Guardrails: Offers synthetic diagnostic probes with built-in quota floors (`minRpmReserve = 15`) and 10-minute TTL caching so diagnostic pings never starve developer workflows.
- Rigorous CI/CD & Governance: Validated across a GitHub Actions matrix (Node 20, 22, 24) with strict TypeScript types, 26 automated tests, and semantic versioning.