alternative to ChatGPT

TensorRT-LLM

Loading the README

About

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

PythonLocal model runtimesOther

Owner: NVIDIA Corporation organisation
Stars: 15k
Forks: 2.8k
Latest: v1.2.1
Last push: 2026-10-06
Terms: Open source
Self-hosted: Yes
Deploy with: Container image
Created: 2023, 3 years ago
Active contributors: 125+ commit authors on the default branch in the last 90 days

Read from GitHub on 2026-10-06.Entry last edited on 2026-10-06.

Compare TensorRT-LLM and Ollama →

#blackwell#cuda#llm-serving#moe#pytorch

github.com/NVIDIA/TensorRT-LLM ↗Website ↗Edit this entry Changes to TensorRT-LLM (RSS)

Other tools in Local model runtimes

OllamaGo→
llama.cppC++→
vLLMPython→
LocalAIGo→
exoPython→
SGLangPython→
llamafileC++→
MLC LLMPython→
KTransformersPython→
XinferencePython→
LMDeployPython→
mistral.rsRust→
llama-swapGo→
LemonadeC++→
GPUStackPython→
RamaLamaPython→
TabbyAPIPython→