Changelog

galadriel-node 0.0.16

🚀 New Features

  • Multiple backends were set up and the node will now auto-reconnect to the optimal one. This will optimize the latency of the backend-node connection.

  • Add LMDeploy support which based on our tests, serves inference responses around 15% faster

😎 Improvements

  • Redesigned logging to be more useful. The logs are now colored and contain timestamps. Also, every command accepts --debug flag and exceptions print stack traces to give more visibility and improve debugging

🐛 Bug Fix

  • Code cleanups

galadriel-node 0.0.15

🚀 New Features

  • Report node health

😎 Improvements

  • Reuse client for llm inference

🐛 Bug Fix

  • Some code cleanup

galadriel-node 0.0.14

🚀 New Features

  • Report GPU count

😎 Improvements

  • Increase minimum reconnecting backoff time, from 4 sec to 24 sec
  • Make required benchmarks model specific

🐛 Bug Fix

  • Make the node always try to reconnect