Product Design, Manufacturing & Innovation Resources

최신 논문 – 소형 언어 모델(SLM) 관련 특허

Small Language Models

팁: 아래 선택 항목 외에도, 저희가 제공하는 두 개의 전체 데이터베이스를 검색하고 필터링할 수 있습니다.

> 무료 간행물 검색 도구 < 저자, 주제, 키워드, 날짜 또는 저널별로 검색.

> 무료 특허 검색 도구 < 유럽 ​​특허청의 영문 특허를 참조하십시오.

Small language models
Small language models enable efficient natural language processing on consumer and embedded devices.

소규모 언어 모델은 약 70억 개 미만의 매개변수로 작동하는 트랜스포머 기반 자연어 처리 시스템을 의미합니다. 이 임계값은 형식적인 경계라기보다는 클라우드 추론 인프라 없이 소비자 하드웨어, 모바일 장치 및 임베디드 시스템에 배포할 수 있다는 실질적인 제약 조건에 의해 정의됩니다.

이 분야는 최첨단 모델의 계산 및 경제적 비용에 대한 직접적인 대응으로 등장했습니다. 수십억 개 이상의 매개변수를 가진 아키텍처는 광범위한 일반 기능을 보여주지만, 메모리 사용량, 추론 지연 시간 및 에너지 소비량으로 인해 온디바이스 배포, 개인 정보 보호에 민감한 애플리케이션, 저대역폭 또는 오프라인 운영 환경과 구조적으로 호환되지 않습니다.

핵심 연구 프로그램은 지식 증류(더 큰 교사 모델의 출력 분포를 기반으로 더 작은 학생 모델을 학습시키는 것), 구조적 및 비구조적 가지치기, INT4 및 INT8 표현으로의 적극적인 가중치 양자화, 그리고 최소한의 추가 컴퓨팅 비용으로 압축된 기본 모델을 도메인별 작업에 맞게 조정하는 LoRA 및 QLoRA와 같은 매개변수 효율적인 미세 조정 방법을 결합하여 소형 모델과 최첨단 모델 간의 성능 격차를 해소하는 데 주력하고 있습니다.

아래에 색인된 논문 및 특허는 모델 압축 기술, 양자화 알고리즘, 증류 프로토콜, 효율적인 변환기 아키텍처, 온디바이스 추론 최적화 및 도메인별 미세 조정 파이프라인을 다룹니다.

본 자료는 소규모 언어 모델(SLM)에 관한 전 세계 영어 논문 및 특허를 엄선하여 정리한 것으로, 다양한 온라인 과학 저널에 게재된 자료들을 포함합니다. 주요 주제는 소규모 언어 모델(SLM), 온디바이스 언어 모델, 에지 언어 모델, 컴팩트 트랜스포머, 70억 미만 파라미터 모델, 언어 모델 압축, 지식 증류 자연어 처리, 구조적 가지치기 언어 모델, 비구조적 가지치기 언어 모델, 가중치 양자화 언어 모델, INT4 양자화 자연어 처리, INT8 양자화 자연어 처리, 파라미터 효율적 미세 조정, LoRA 미세 조정, QLoRA 미세 조정, 어댑터 튜닝 언어 모델, 온디바이스 추론, 에지 추론 자연어 처리, 투기적 디코딩, 모델 증류 트랜스포머, GGUF 양자화 형식, 전문가 혼합 컴팩트 모델입니다.

Teaching small language models petro-technical knowledge through language reasoning and fine-tuning

Patent published on the 2026-10-15 in US under Ref US20260319121 by SCHLUMBERGER TECH CORPORATION [US] (Paydayesh Mehdi [gb], Shamsa Amir [ca])

Abstract: A method for automatically generating an answer to a petro-technical query in an oil and gas domain includes receiving input data including question-answer pairs related to the oil and gas domain. The method also includes training a small language model (SLM) based upon the input data to produce a trained model. The method also includes fine-tuning the trained model to produce a fine-tuned model. The method also includes receiving the petro-technical query in the oil and gas domain. The method a[...]


Our summary: This method trains a small language model using question-answer pairs from the oil and gas domain. It fine-tunes the model to improve its ability to generate answers to petro-technical queries. The process enables automatic response generation for specific queries in the oil and gas sector.

small language models, petro-technical knowledge, fine-tuning, query generation

Patent

Method of laser-based additive manufacturing with pulse scan patterns for magnesium and its alloys

Patent published on the 2026-10-08 in US under Ref US20260307150 by OSSEOLABS CO LTD [TH] (Promoppatum Patcharapit [th], Sukpasert Montian [th], Ahuja Vikram [th], Harrysson Ola [us])

Abstract: This invention introduces an advanced pulse scanning technique for Selective Laser Melting (SLM) of magnesium, addressing critical manufacturing challenges. The method employs iterative optimization to regulate heat input through precise pulse patterns, preventing overheating and material defects. By alternating between powered and unpowered laser phases, the technique reduces material evaporation, minimizes internal stresses, and enhances microstructural uniformity. This approach enables more s[...]


Our summary: This invention presents a pulse scanning technique for Selective Laser Melting of magnesium. It optimizes heat input to prevent overheating and defects by alternating laser phases. This method improves fabrication stability and microstructural uniformity for magnesium components.

laser-based manufacturing, pulse scanning, magnesium alloys, Selective Laser Melting

Patent

Memory-efficient neural dialog artificial intelligence

Patent published on the 2026-10-08 in US under Ref US20260310927 by HSU JOSEPH BERNARD [US] (Hsu Joseph Bernard [us])

Abstract: [0000] A hybrid generative AI artificial intelligence system addresses cloud-centric AI inefficiencies through dynamic hierarchical memory indexing and selective cloud escalation. The system implements a three-object architecture comprising user profile management, thread index management, and dialog process control that enables client-side processing with precision-targeted cloud resource utilization. A small language model functions as a local decision engine, performing heuristic inference an[...]


Our summary: A hybrid generative AI system enhances cloud-centric AI efficiency through dynamic memory indexing. It features a three-object architecture for user profile, thread index, and dialog control. The system reduces response time by 84% and cloud dependency by 80% while ensuring cognitive coherence.

neural networks, cloud computing, memory indexing, dialog systems

Patent

Template guided speculative decoding

Patent published on the 2026-10-06 in US under Ref US12754958 by INTUIT INC [US] (Ben Shoham Ofir [il], Bar Eliyahu Natalie [il], Mendelson Shon [il], Antebi Sagiv [il], Ardazi Shai [il])

Abstract: Aspects of the present disclosure involve accelerated machine learning based sequence generation. Aspects include generating a first portion of a sequence based on an input query using a speculative decoding process involving a draft machine learning model and a target machine learning model with a larger number of parameters than the draft model. Aspects include determining, based on the input query and the first portion, that a next portion of the sequence is predicted to correspond to a templ[...]


Our summary: This disclosure involves a method for accelerated sequence generation using speculative decoding with two machine learning models. It includes generating a sequence portion based on an input query and determining if the next portion corresponds to a template. The next portion is generated through a template-based process and verified using a larger target model before appending to the sequence.

speculative decoding, sequence generation, machine learning models, template-based process

Patent

Secure explanation generation by large language models for sensitive search targets

Patent published on the 2026-10-06 in US under Ref US12755173 by INTUIT INC [US] (Eliyahu Natalie Bar [il], Mendelson Shon [il], Hazan Itay [il], Bechler Sigalit [il], Haas Noa [il], Bitton Ron [il])

Abstract: [0000] A method includes generating, by a large language model (LLM), a multitude of match explanations for a multitude of search targets. The multitude of match explanations is clustered to obtain a multitude of explanation clusters. An explanation template is generated for an explanation cluster. The explanation template includes a static portion and a dynamic portion. A multitude of explanation templates is obtained in this manner. A target match for a search target received from a user appli[...]


Our summary: The method generates match explanations for search targets using a large language model. These explanations are clustered to create templates with static and dynamic portions. A small language model populates match explanations based on user search targets and encoded templates.

explanation generation, large language model, sensitive search targets, clustering

Patent

Speculative decoding using expanded token matching

Patent published on the 2026-09-03 in US under Ref US20260260065 by MICROSOFT TECH LICENSING LLC [US] (Pathak Sayan Dev [us], Kesavamoorthy Harini [us], Basoglu Christopher Hakan [us], Vikram Ayush [us], Shah Amy Parag [us], Kelkar Rakesh [us])

Abstract: [0000] A speculative decoding technique generates tokens of a response using a verifying model and drafting model. In operation, the verifying model receives a set of draft tokens produced by the drafting model. The verifying model then verifies whether the draft tokens are correct by comparing probability information generated by the verifying model with probability information generated by the drafting model. The verifying model then determines whether any draft token that is rejected by the v[...]


Our summary: Speculative decoding generates response tokens using a verifying and drafting model. The verifying model checks draft tokens against probability information from both models. This iterative process expands accepted tokens, reducing resource use and latency.

speculative decoding, token matching, verifying model, drafting model

Patent

Automated synthesis of planar linkage mechanisms with diverse joint types via spring-connected link models and contrastive graph learning

Published on 2026-03-28 by @OXFORD

Abstract: AbstractThe automated synthesis of planar linkage mechanisms has long been a challenge in mechanism design, requiring both geometric feasibility and motion accuracy. Recent advances in data-driven and neural network–based methods have shown promise in automating linkage synthesis, improving efficiency and scalability compared to traditional analytical or optimization-based techniques. Nevertheless, existing data-driven approaches remain limited in handling diverse joint configurations and ofte[...]


Our summary: This study presents a framework for automating the synthesis of planar linkage mechanisms using deep learning and physics-based modeling. It employs a spring-connected link model for diverse joint configurations and utilizes contrastive graph learning for efficient linkage retrieval. The method demonstrates improved accuracy and optimization stability compared to traditional approaches.

mechanism synthesis, deep learning, contrastive graph learning, optimization stability

Publication

Enhancing Whisper Fine-Tuning with Discrete Wavelet Transform-Based LoRA Initialization

Published on 2026-01-29 by Liang Lan, Molin Fang, Yuxuan Chen, Daliang Wang, Wenyong Wang @MDPI

Abstract: In low-resource automatic speech recognition (ASR) scenarios, parameter-efficient fine-tuning (PEFT) has become a crucial approach for adapting large pre-trained speech models. Although low-rank adaptation (LoRA) offers clear advantages in efficiency, stability, and deployment friendliness, its performance remains constrained because random initialization fails to capture the time&amp;ndash;frequency structural characteristics of speech signals. To address this limitation, this work proposes[...]


Our summary: This work introduces a structured initialization mechanism combining LoRA with discrete wavelet transform for fine-tuning in low-resource ASR. The proposed DWTLoRA method enhances convergence speed, stability, and accuracy by aligning with speech signal characteristics. Experimental results show DWTLoRA outperforms standard LoRA and other PEFT methods in character error rate and training efficiency.

Fine-Tuning, Discrete Wavelet Transform, Low-Rank Adaptation, Automatic Speech Recognition

Publication

다룬 주제: 소형 언어 모델, 자연어 처리, 트랜스포머 기반 시스템, 파라미터 효율성, 지식 증류, 모델 압축, 구조적 가지치기, 비구조적 가지치기, 가중치 양자화, INT4, INT8, 미세 조정 방법, 온디바이스 배포, 추론 지연 시간, 에너지 소비, 개인 정보 보호에 민감한 애플리케이션, 저대역폭 작업, 오프라인 운영 환경, IEEE 80211, ISO/IEC 30170, ISO/IEC 27001, ISO/IEC 25010 및 NIST SP 800-53.

사용된 용어집

Natural Language Processing (NLP): 인공지능 분야 중 컴퓨터와 인간 언어 간의 상호작용에 초점을 맞춘 분야로, 기계가 자연어 텍스트나 음성을 이해하고 해석하며 생성할 수 있도록 합니다. 언어 번역, 감정 분석, 음성 인식과 같은 작업을 포함합니다.

Small Language Models (SLM): 소형 신경망은 자연어 처리 작업을 위해 설계되었으며, 일반적으로 대형 모델에 비해 매개변수 수가 적고 계산 요구 사항이 낮지만, 제한된 범위 내에서 일관성 있는 텍스트를 생성하고 문맥을 이해할 수 있는 기능을 갖추고 있습니다.

역사적 맥락

1965
1970
1970
1974-11-15
1980
1980
1980
1964
1968
1970
1970
1975
1980
1980
1980

(날짜를 알 수 없거나 관련이 없는 경우, 예를 들어 "유체역학"의 경우, 주목할 만한 등장 시기를 대략적으로 추정하여 제공합니다.)

10년차 기념일

고화질 이미지 및 다운로드는 등록된 회원에게만 100% 무료로 제공됩니다.

> 로그인 <