Distilling Large Embeddings via Hyperspherical Householder Quantization
Large embedding models have become the backbone of modern retrieval systems, offering strong semantic representations at the cost of substantial storage and computation. While …
Detoxification for LLM: From Dataset Itself
Existing detoxification methods for large language models mainly focus on post-training stage or inference time, while few tackle the source of toxicity, namely, the dataset …
wei-shao
QUITO-X: A New Perspective on Context Compression from the Information Bottleneck Theory
Generative LLM have achieved remarkable success in various industrial applications, owing to their promising In-Context Learning capabilities. However, the issue of long context in …
MDPO: Customized Direct Preference Optimization with a Metric-based Sampler for Question and Answer Generation
With the extensive use of large language models, automatically generating QA datasets for domain-specific fine-tuning has become crucial. However, considering the multifaceted …
QUITO: Accelerating Long-Context Reasoning through Query-Guided Context Compression
In-context learning (ICL) capabilities are foundational to the success of large language models (LLMs). Recently, context compression has attracted growing interest since it can …
wenshan-wang