소규모 언어 모델은 약 70억 개 미만의 매개변수로 작동하는 트랜스포머 기반 자연어 처리 시스템을 의미합니다. 이 임계값은 형식적인 경계라기보다는 클라우드 추론 인프라 없이 소비자 하드웨어, 모바일 장치 및 임베디드 시스템에 배포할 수 있다는 실질적인 제약 조건에 의해 정의됩니다.
이 분야는 최첨단 모델의 계산 및 경제적 비용에 대한 직접적인 대응으로 등장했습니다. 수십억 개 이상의 매개변수를 가진 아키텍처는 광범위한 일반 기능을 보여주지만, 메모리 사용량, 추론 지연 시간 및 에너지 소비량으로 인해 온디바이스 배포, 개인 정보 보호에 민감한 애플리케이션, 저대역폭 또는 오프라인 운영 환경과 구조적으로 호환되지 않습니다.
핵심 연구 프로그램은 지식 증류(더 큰 교사 모델의 출력 분포를 기반으로 더 작은 학생 모델을 학습시키는 것), 구조적 및 비구조적 가지치기, INT4 및 INT8 표현으로의 적극적인 가중치 양자화, 그리고 최소한의 추가 컴퓨팅 비용으로 압축된 기본 모델을 도메인별 작업에 맞게 조정하는 LoRA 및 QLoRA와 같은 매개변수 효율적인 미세 조정 방법을 결합하여 소형 모델과 최첨단 모델 간의 성능 격차를 해소하는 데 주력하고 있습니다.
아래에 색인된 논문 및 특허는 모델 압축 기술, 양자화 알고리즘, 증류 프로토콜, 효율적인 변환기 아키텍처, 온디바이스 추론 최적화 및 도메인별 미세 조정 파이프라인을 다룹니다.
본 자료는 소규모 언어 모델(SLM)에 관한 전 세계 영어 논문 및 특허를 엄선하여 정리한 것으로, 다양한 온라인 과학 저널에 게재된 자료들을 포함합니다. 주요 주제는 소규모 언어 모델(SLM), 온디바이스 언어 모델, 에지 언어 모델, 컴팩트 트랜스포머, 70억 미만 파라미터 모델, 언어 모델 압축, 지식 증류 자연어 처리, 구조적 가지치기 언어 모델, 비구조적 가지치기 언어 모델, 가중치 양자화 언어 모델, INT4 양자화 자연어 처리, INT8 양자화 자연어 처리, 파라미터 효율적 미세 조정, LoRA 미세 조정, QLoRA 미세 조정, 어댑터 튜닝 언어 모델, 온디바이스 추론, 에지 추론 자연어 처리, 투기적 디코딩, 모델 증류 트랜스포머, GGUF 양자화 형식, 전문가 혼합 컴팩트 모델입니다.
Parameter-free method for efficient and accurate llm inference acceleration via speculative decoding
Patent published on the 2026-05-07 in WO under Ref WO2026092843 by MARZOLLO MICHELE [DE] (Marzollo Michele [de], Mueller Lorenz [de], Zhuang Jiawei [de], Roemer Niklas [de], Cavigelli Lukas [de])
Abstract: In some examples, apparatus and methods are provided for selecting a draft token sequence for verification by using a large language model, LLM. Different sources of statistics on text data (prompt, generated output, large dataset of text data) can be utilized in order to choose candidates to use for speculative decoding via look-ups.[...]
Our summary: This method accelerates LLM inference without parameters by using speculative decoding. It selects draft token sequences for verification through statistical analysis of text data. The approach utilizes various sources of statistics to optimize candidate selection for decoding.
speculative decoding, LLM inference, token sequence selection, text data statistics
Patent
Automated synthesis of planar linkage mechanisms with diverse joint types via spring-connected link models and contrastive graph learning
Published on 2026-03-28 by @OXFORD
Abstract: AbstractThe automated synthesis of planar linkage mechanisms has long been a challenge in mechanism design, requiring both geometric feasibility and motion accuracy. Recent advances in data-driven and neural network–based methods have shown promise in automating linkage synthesis, improving efficiency and scalability compared to traditional analytical or optimization-based techniques. Nevertheless, existing data-driven approaches remain limited in handling diverse joint configurations and ofte[...]
Our summary: This study presents a framework for automating the synthesis of planar linkage mechanisms using deep learning and physics-based modeling. It employs a spring-connected link model for diverse joint configurations and utilizes contrastive graph learning for efficient linkage retrieval. The method demonstrates improved accuracy and optimization stability compared to traditional approaches.
mechanism synthesis, deep learning, contrastive graph learning, optimization stability
Publication
Enhancing Whisper Fine-Tuning with Discrete Wavelet Transform-Based LoRA Initialization
Published on 2026-01-29 by Liang Lan, Molin Fang, Yuxuan Chen, Daliang Wang, Wenyong Wang @MDPI
Abstract: In low-resource automatic speech recognition (ASR) scenarios, parameter-efficient fine-tuning (PEFT) has become a crucial approach for adapting large pre-trained speech models. Although low-rank adaptation (LoRA) offers clear advantages in efficiency, stability, and deployment friendliness, its performance remains constrained because random initialization fails to capture the time–frequency structural characteristics of speech signals. To address this limitation, this work proposes[...]
Our summary: This work introduces a structured initialization mechanism combining LoRA with discrete wavelet transform for fine-tuning in low-resource ASR. The proposed DWTLoRA method enhances convergence speed, stability, and accuracy by aligning with speech signal characteristics. Experimental results show DWTLoRA outperforms standard LoRA and other PEFT methods in character error rate and training efficiency.
Fine-Tuning, Discrete Wavelet Transform, Low-Rank Adaptation, Automatic Speech Recognition
Publication
A Systematic Evaluation and Adaptation of Large Language Models for Very Short-Term Power Load Forecasting
Published on 2026-01-26 by Yansheng Chen, Miao Chen, Chenchao Hu, Jinxi Wu, Ruilin Qin @MDPI
Abstract: Power load forecasting is critical for ensuring grid security and stability and optimizing energy resource allocation. The high integration of renewable energy poses significant challenges to traditional methods in data-scarce scenarios. Recently, Large Language Models (LLMs) have shown considerable potential in processing time-series data, yet their effectiveness in very short-term power load forecasting lacks systematic evaluation. This paper proposes a targeted prompt engineering framework an[...]
Our summary: This study evaluates the effectiveness of Large Language Models (LLMs) for very short-term power load forecasting. It proposes a targeted prompt engineering framework and introduces Ele-LLM, a model utilizing Low-Rank Adaptation for improved performance. Experimental results indicate Ele-LLM significantly outperforms traditional forecasting methods, demonstrating the potential of LLMs in this domain.
Large Language Models, Power Load Forecasting, Prompt Engineering, Low-Rank Adaptation
Publication
A Comparative Study and Introduction of a New Heat Source Model for the Macro-Scale Numerical Simulation of Selective Laser Melting Technology
Published on 2026-01-25 by Hao Zhang, Shuai Wang, Junjie Wang, Zhiqiang Yan @MDPI
Abstract: Selective Laser Melting (SLM), as a common metal additive manufacturing (AM) technology, achieves high-precision complex part formation by layer-by-layer melting of metal powder using a laser. However, the dynamic behavior of the melt pool during the SLM process is influenced by the heat source model, which is crucial for suppressing porosity defects and optimizing process parameters, directly determining the reliability of numerical simulations. To address the issue of traditional surface heat [...]
Our summary: This study introduces a new dynamic heat source model for Selective Laser Melting (SLM) to improve numerical simulations. It compares various heat source models and demonstrates that the dynamic model achieves greater accuracy in predicting melt pool dimensions. The results indicate significant improvements in both melt pool width and depth compared to traditional models.
Heat Source Model, Selective Laser Melting, Finite Element Analysis, Temperature Field
Publication
Influence and Optimization of Process Parameters on Surface Roughness of Selective Laser Melting of 316L Stainless Steel
Published on 2026-01-20 by Pin Dong, Kamonpong Jamkamon, Suppawat Chuvaree @MDPI
Abstract: To achieve better surface quality in selective laser melting (SLM), this study used 316L stainless steel powder and conducted a systematic design experiment to investigate the influence mechanism of process parameters on the surface roughness of the top and vertical surfaces. Response surface methodology (RSM) was then used for parameter optimization. The results showed that scanning speed has the greatest impact on surface roughness, followed by laser power, while scanning spacing has the least[...]
Our summary: This study investigates the impact of process parameters on the surface roughness of 316L stainless steel in selective laser melting. Scanning speed significantly affects surface quality, with optimal conditions identified for minimal roughness. The findings validate the effectiveness of the response surface methodology used for parameter optimization.
Selective Laser Melting, Surface Roughness, Process Parameters, Response Surface Methodology
Publication
Effect of Selectively Etched Al-Rich and Si-Rich Microstructures on the Adhesion of Polyimide Coatings to SLM AlSi10Mg
Published on 2026-01-18 by Jianzhu Li, Shuo Yang, Yujie Li @MDPI
Abstract: Interfacial adhesion between selective laser-melted (SLM) AlSi10Mg and polyimide (PI) insulating coatings is often limited by mismatched physicochemical properties. To improve adhesion, Al-rich and Si-rich microstructured surfaces were fabricated on the XY plane (perpendicular to the build direction) and the Z plane (parallel to the build direction) by acidic and alkaline etching, exploiting the characteristic microstructure of SLM AlSi10Mg. Surface topography, chemical composition, and wettabil[...]
Our summary: Selectively etched microstructures on AlSi10Mg improve adhesion to polyimide coatings. Al-rich surfaces showed cohesive failure, while Si-rich surfaces exhibited mixed failure modes. Shear and pull-off tests demonstrated significant increases in mechanical performance compared to polished surfaces.
adhesion, microstructures, polyimide, selective laser melting
Publication
Monolithic microelectromechanical systems based spatial light modulators including multiple arrays, each array configured to modulate different wavele
Patent published on the 2026-01-02 in WO under Ref WO2026006343 by SILICON LIGHT MACHINES CORP [US] (Hamann Stephen [us], Payne Alexander [us], Hunter James [us], Liu Tianbo [us], Eng Lars [us])
Abstract: A monolithic spatial light modulator (SLM) is provided. Generally, the SLM includes a substrate with a number of substrate electrodes in a surface thereof, multiple MEMS-based linear arrays formed on the surface of the substrate, and a drive circuit monolithically integrated in the substrate below the surface of the substrate. Each linear array includes multiple ribbons suspended above the surface of the substrate, each ribbon having a light reflective surface facing away from the surface of the[...]
Our summary: The content describes a monolithic spatial light modulator (SLM) that includes multiple MEMS-based linear arrays. Each array is designed to modulate different, non-overlapping wavelength ranges. The SLM integrates a drive circuit with substrate electrodes to control the electrostatically displaceable ribbons for light modulation.
MEMS, spatial light modulators, wave modulation, monolithic integration
Patent
