Models

RedHatAI/starcoder2-3b-quantized.w8a16

Text Generation • 1B • Updated Oct 9, 2024 • 14

RedHatAI/starcoder2-7b-quantized.w8a16

Text Generation • 2B • Updated Oct 9, 2024 • 13

RedHatAI/starcoder2-3b-quantized.w8a8

Text Generation • 3B • Updated Oct 9, 2024 • 8

RedHatAI/starcoder2-7b-quantized.w8a8

Text Generation • 7B • Updated Oct 9, 2024 • 8

RedHatAI/starcoder2-15b-quantized.w8a8

Text Generation • 16B • Updated Oct 9, 2024 • 9

RedHatAI/gemma-2-2b-it-FP8

3B • Updated Oct 9, 2024 • 231 • 1

nm-testing/Mixtral-8x7B-Instruct-v0.1-W4A16-quantized

6B • Updated Oct 9, 2024 • 565

nm-testing/Mixtral-8x7B-Instruct-v0.1-W8A16-quantized

12B • Updated Oct 9, 2024 • 560

nm-testing/Phi-3-mini-128k-instruct-FP8

4B • Updated Oct 9, 2024 • 733

RedHatAI/Meta-Llama-3.1-405B-Instruct-quantized.w4a16

Text Generation • 58B • Updated Oct 10, 2024 • 26 • 12

nm-testing/Meta-Llama-3-8B-Instruct-fp8-compressed

8B • Updated Oct 9, 2024 • 6

RedHatAI/gemma-2-9b-it-quantized.w8a16

Text Generation • 4B • Updated Oct 9, 2024 • 16 • 1

RedHatAI/gemma-2-2b-it-quantized.w8a16

Text Generation • 2B • Updated Oct 9, 2024 • 47 • 1

RedHatAI/gemma-2-2b-quantized.w8a16

Text Generation • 2B • Updated Oct 9, 2024 • 11

nm-testing/Mixtral-8x7B-Instruct-v0.1-W4A16-channel-quantized

6B • Updated Oct 9, 2024 • 574

RedHatAI/Phi-3-small-128k-instruct-quantized.w8a16

Text Generation • 3B • Updated Oct 9, 2024 • 7

RedHatAI/gemma-2-9b-it-quantized.w4a16

Text Generation • 3B • Updated Oct 9, 2024 • 66 • 2

RedHatAI/gemma-2-2b-it-quantized.w4a16

Text Generation • 1B • Updated Oct 9, 2024 • 117 • 1

RedHatAI/SmolLM-1.7B-Instruct-quantized.w8a16

Text Generation • 0.6B • Updated Oct 9, 2024 • 8

RedHatAI/Mistral-Nemo-Instruct-2407-quantized.w4a16

Text Generation • 3B • Updated Oct 9, 2024 • 123 • 4

RedHatAI/gemma-2-2b-it-quantized.w8a8

Text Generation • 3B • Updated Oct 9, 2024 • 30

RedHatAI/gemma-2-9b-it-quantized.w8a8

Text Generation • 10B • Updated Oct 9, 2024 • 18 • 2

RedHatAI/Meta-Llama-3.1-405B-Instruct-quantized.w8a8

Text Generation • 406B • Updated Dec 3, 2024 • 24 • 2

RedHatAI/Meta-Llama-3.1-405B-Instruct-quantized.w8a16

Text Generation • 105B • Updated Oct 9, 2024 • 21 • 1

nm-testing/Meta-Llama-3-8B-Instruct-fp8-hf_compat

8B • Updated Aug 30, 2024 • 130

RedHatAI/gemma-2-27b-it-quantized.w8a16

Text Generation • 9B • Updated Oct 9, 2024 • 7

RedHatAI/SmolLM-135M-Instruct-quantized.w8a16

Text Generation • 83.4M • Updated Oct 9, 2024 • 34

RedHatAI/SmolLM-360M-Instruct-quantized.w8a8

Text Generation • 0.4B • Updated Oct 9, 2024 • 9

RedHatAI/SmolLM-135M-Instruct-quantized.w8a8

Text Generation • 0.2B • Updated Oct 9, 2024 • 8 • 1

nm-testing/SmolLM-135M-Instruct-quantized.w4a16

Text Generation • 71.6M • Updated Oct 9, 2024 • 7