Efficient Model Compression: A Teacher - Student Distillation Framework for Cost-Efficient Customer Support Automation
Department of Artificial Intelligence and Data Science, Ajeenkya DY Patil School of Engineering, Pune, India · India
Abstract
Customer-support automation needs intent classifiers that approach the accuracy of frontier large language models (LLMs) without paying their per-ticket cost and latency. This paper is a literature-based analytical study of three compression techniques-knowledge distillation, parameter-efficient fine-tuning (LoRA/QLoRA) and post-training quantization (GPTQ/AWQ)-together with intent-classification benchmarks and cost-aware LLM cascading, centred on ten primary studies published between 2015 and 2025. A criterion-driven analysis reveals that all three techniques are mainly evaluated individually on general benchmark tests, and no one of the studied papers evaluates the joint effect of the techniques on support-ticket classification. To address the detected deficiency, we suggest the concept of a teacher-student-quantization framework with a confidence escalation strategy and describe the evaluation methodology for it. Our contribution here is conceptual: no dataset has been created, no model has been trained, and no performance metrics have been evaluated yet.
Keywords
How to Cite This Article
Rights
BODHIVRUKSHA JOURNAL OF DIVERSE DISCIPLINE