This page covers the requirements to run a GPU node

​
Hardware requirements

  • At least 4 CPU cores
  • At least 8GB RAM
  • A Nvidia GPU with at least 8GB VRAM

​
Software requirements

  • Linux (Ubuntu recommended)
  • Python version 3.10+
  • Nvidia drivers, version > 450. nvidia-smi must work

​
Runtime requirements

  • Ability to serve inference requests of moderate length on average under 8 seconds.

​
API keys

  • A valid Galadriel API key (can get it from the dashboard)
Retrieval-Augmented GenerationQuickstart