Hugging Face's logo

Hugging Face Party @ PyTorch Conference
community
Activity Feed Request to join this org

AI & ML interests

None defined yet.
csabakecskemeti 
posted an update 9 days ago
view post
Post
3128
Just sharing a result of a homelab infrastructure experiment:

I've managed to setup a distributed inference infra at home using a DGX Spark (128GB unified gddr6) and a linux workstation with an RTX 6000 Pro (96GB gddr7) connected via 100Gbps RoCEv2. The model I've used (https://lnkd.in/gx6J7YuB) is about 140GB so could not fit either of the GPU. Full setup and tutorial soon on devquasar.com



Screen recording:
https://lnkd.in/gKM9H5GJ
·
Johannes 
authored a paper 10 days ago
Aurelien-Morgan 
posted an update 26 days ago
csabakecskemeti 
posted an update about 1 month ago
csabakecskemeti 
posted an update about 1 month ago
view post
Post
2073
Looking for some help to test an INT8 Deepseek 3.2:
SGLang supports Channel wise INT8 quants on CPUs with AMX instructions (Xeon 5 and above AFAIK)
https://lmsys.org/blog/2025-07-14-intel-xeon-optimization/

Currently uploading an INT8 version of Deepseek 3.2 Speciale:
DevQuasar/deepseek-ai.DeepSeek-V3.2-Speciale-Channel-INT8

I cannot test this I'm on AMD
"AssertionError: W8A8Int8LinearMethod on CPU requires that CPU has AMX support"
(I assumed it can fall back to some non optimized kernel but seems not)

If anyone with the required resources (Intel Xeon 5/6 + ~768-1TB ram) can help to test this that would be awesome.

If you have hints how to make this work on AMD Threadripper 7000 Pro series please guide me.

Thanks all!
·
csabakecskemeti 
posted an update about 2 months ago
view post
Post
303
Recently there are so much activity on token efficient formats, I've also build a package (inspired by toon).

Deep-TOON

My goal was to token efficiently handle json structures with complex embeddings.

So this is what I've built on the weekend. Feel free try:

https://pypi.org/project/deep-toon/0.1.0/
JingzeShi 
posted an update about 2 months ago
csabakecskemeti 
posted an update 3 months ago
view post
Post
2619
Christmas came early this year
·
mbrack 
authored a paper 3 months ago
lysandre 
posted an update 4 months ago
view post
Post
7443
We're kick-starting the process of Transformers v5, with @ArthurZ and @cyrilvallez !

v5 should be significant: we're using it as a milestone for performance optimizations, saner defaults, and a much cleaner code base worthy of 2025.

Fun fact: v4.0.0-rc-1 came out on Nov 19, 2020, nearly five years ago!
  • 6 replies
·
teknium 
authored 2 papers 4 months ago
jeffboudier 
posted an update 4 months ago
view post
Post
3096
Quick 30s demo of the new Hub > Azure AI integration to deploy HF models in your own Azure account. Now with Py and CLI!

GG @alvarobartt @kramp @pagezyhf
clem 
posted an update 5 months ago
JingzeShi 
authored a paper 5 months ago
JingzeShi 
posted an update 5 months ago
ybelkada 
authored 2 papers 5 months ago
jeffboudier 
posted an update 6 months ago
view post
Post
562
AMD summer hackathons are here!
A chance to get hands-on with MI300X GPUs and accelerate models.
🇫🇷 Paris - Station F - July 5-6
🇮🇳 Mumbai - July 12-13
🇮🇳 Bengaluru - July 19-20

Hugging Face and GPU Mode will be on site and on July 6 in Paris @ror will share lessons learned while building new kernels to accelerate Llama 3.1 405B on ROCm

Register to Paris event: https://lu.ma/fmvdjmur?tk=KeAbiP
All dates: https://lu.ma/calendar/cal-3sxhD5FdxWsMDIz
mbrack 
authored a paper 7 months ago