Inference Frameworks

LLM and Stable Diffusion inference frameworks for the XH2 AI accelerator

Inference Framework Guide

LLM Inference Frameworks

LLM inference frameworks for the XH2: vLLM, C++ , llama.cpp and Python ecosystems

houmo-HLIECpp_xh2_2_0_0-win_x64.exe
HLIE C++ inference framework (Windows x64) with multi-engine, cross-platform serving.
162 MB Windows x64
Download
houmo-HLIELLama-xh2_2_1_0_windows11_x86_64_tar.gz
Adapted from the llama.cpp ecosystem for efficient on-device inference (Windows 11 x64).
26 MB Windows 11 x64
Download
HLIEvLLM-linux_1_4_0rc0.zip
Adapted from the vLLM ecosystem for stable, high-concurrency LLM serving.
52 MB Linux
Download
houmo-HLIECpp_xh2_2_1_0_linux_x86_64.run
HLIE C++ inference framework (Linux x86_64) with multi-engine, cross-platform serving.
74 MB Linux x86_64
Download
houmo-HLIELLama-xh2_2.1.0_linux_x86_64.tar.gz
LLM inference framework based on the llama.cpp ecosystem (Linux x86_64).
44 MB Linux x86_64
Download
houmo-HLIELLama-xh2_2_0_1_linux_x86_64_tar.gz
Adapted from the llama.cpp ecosystem for efficient on-device inference.
37 MB Linux x86_64
Download
houmo-HLIELLama-xh2_2_1_0_linux_x86_64_tar.gz
LLM inference framework based on the llama.cpp ecosystem (Linux x86_64, v2.1.0).
44 MB Linux x86_64
Download
houmo-HLIELLama-xh2_2_1_1_linux_x86_64_tar.gz
LLM inference framework based on the llama.cpp ecosystem (Linux x86_64, v2.1.1).
44 MB Linux x86_64
Download
houmo-Python-xh2_3_0_1_linux_x86_64_tar.gz
Python-based LLM inference framework (Linux x86_64, v3.0.1).
4.9 GB Linux x86_64
Download

Stable Diffusion Inference Framework

C++ Stable Diffusion framework with SD3 text-to-image support

houmo-HLIESDCpp-xh2_1_0_0_windows11_x86_64_tar.gz
C++ Stable Diffusion inference framework (Windows 11 x64) with SD3 text-to-image support.
6.6 MB Windows 11 x64
Download
houmo-HLIESDCpp-xh2_1_0_0_linux_x86_64_tar.gz
C++ Stable Diffusion inference framework (Linux x86_64) with SD3 text-to-image support.
9.9 MB Linux x86_64
Download

Download Notes