A clustering-based knowledge distillation pipeline that transforms large language models (e.g., Apertus 70B) into efficient, specialized smaller models through unsupervised domain clustering and LoRA ...