Product Design, Manufacturing & Innovation Resources

最新の出版物:小型言語モデル(SLM)に関する特許

Small Language Models

ヒント:以下の選択肢に加えて、2つのデータベース全体を検索およびフィルタリングできます。

> 無料の出版物検索ツール < 著者、トピック、キーワード、日付、または雑誌名で検索できます。

> 無料特許検索ツール < 欧州特許庁の英語による特許情報。

Small language models
Small language models enable efficient natural language processing on consumer and embedded devices.

小規模言語モデルとは、約70億パラメータ未満で動作するトランスフォーマーベースの自然言語処理システムを指します。この閾値は、形式的な境界というよりも、クラウド推論インフラストラクチャを持たない消費者向けハードウェア、モバイルデバイス、組み込みシステムへの展開可能性という実際的な制約によって定義されます。

この分野は、最先端規模のモデルにおける計算コストと経済コストへの直接的な対応として出現しました。10億個以上のパラメータを持つアーキテクチャは幅広い汎用性を示す一方で、そのメモリ使用量、推論遅延、およびエネルギー消費量により、デバイス上での展開、プライバシーに配慮したアプリケーション、低帯域幅またはオフラインの運用環境とは構造的に互換性がありません。

中心となる研究プログラムは、知識蒸留(より大きな教師の出力分布に対してより小さな生徒モデルを訓練する)、構造化および非構造化剪定、INT4およびINT8表現への積極的な重み量子化、そして圧縮された基本モデルを最小限の追加計算コストでドメイン固有のタスクに適応させるLoRAやQLoRAなどのパラメータ効率の高い微調整手法を組み合わせることで、コンパクトモデルと最先端モデルの間の能力ギャップを埋めています。

以下に索引付けされた出版物および特許は、モデル圧縮技術、量子化アルゴリズム、蒸留プロトコル、効率的なトランスフォーマーアーキテクチャ、デバイス上での推論最適化、およびドメイン固有の微調整パイプラインを扱っています。

これは、多数の科学オンラインジャーナルの中から、小型言語モデル (SLM) に関する英語の世界中の出版物と特許の最新のセレクションです。小型言語モデル、SLM、オンデバイス言語モデル、エッジ言語モデル、コンパクトトランスフォーマー、7B 未満のパラメータモデル、言語モデル圧縮、知識蒸留 NLP、構造化剪定言語モデル、非構造化剪定言語モデル、重み量子化言語モデル、INT4 量子化 NLP、INT8 量子化 NLP、パラメータ効率の高い微調整、LoRA 微調整、QLoRA 微調整、アダプタチューニング言語モデル、オンデバイス推論、エッジ推論 NLP、投機的デコード、モデル蒸留トランスフォーマー、GGUF 量子化フォーマット、および専門家の混合コンパクトモデルに分類され、焦点を絞っています。

Teaching small language models petro-technical knowledge through language reasoning and fine-tuning

Patent published on the 2026-10-15 in US under Ref US20260319121 by SCHLUMBERGER TECH CORPORATION [US] (Paydayesh Mehdi [gb], Shamsa Amir [ca])

Abstract: A method for automatically generating an answer to a petro-technical query in an oil and gas domain includes receiving input data including question-answer pairs related to the oil and gas domain. The method also includes training a small language model (SLM) based upon the input data to produce a trained model. The method also includes fine-tuning the trained model to produce a fine-tuned model. The method also includes receiving the petro-technical query in the oil and gas domain. The method a[...]


Our summary: This method trains a small language model using question-answer pairs from the oil and gas domain. It fine-tunes the model to improve its ability to generate answers to petro-technical queries. The process enables automatic response generation for specific queries in the oil and gas sector.

small language models, petro-technical knowledge, fine-tuning, query generation

Patent

Method of laser-based additive manufacturing with pulse scan patterns for magnesium and its alloys

Patent published on the 2026-10-08 in US under Ref US20260307150 by OSSEOLABS CO LTD [TH] (Promoppatum Patcharapit [th], Sukpasert Montian [th], Ahuja Vikram [th], Harrysson Ola [us])

Abstract: This invention introduces an advanced pulse scanning technique for Selective Laser Melting (SLM) of magnesium, addressing critical manufacturing challenges. The method employs iterative optimization to regulate heat input through precise pulse patterns, preventing overheating and material defects. By alternating between powered and unpowered laser phases, the technique reduces material evaporation, minimizes internal stresses, and enhances microstructural uniformity. This approach enables more s[...]


Our summary: This invention presents a pulse scanning technique for Selective Laser Melting of magnesium. It optimizes heat input to prevent overheating and defects by alternating laser phases. This method improves fabrication stability and microstructural uniformity for magnesium components.

laser-based manufacturing, pulse scanning, magnesium alloys, Selective Laser Melting

Patent

Memory-efficient neural dialog artificial intelligence

Patent published on the 2026-10-08 in US under Ref US20260310927 by HSU JOSEPH BERNARD [US] (Hsu Joseph Bernard [us])

Abstract: [0000] A hybrid generative AI artificial intelligence system addresses cloud-centric AI inefficiencies through dynamic hierarchical memory indexing and selective cloud escalation. The system implements a three-object architecture comprising user profile management, thread index management, and dialog process control that enables client-side processing with precision-targeted cloud resource utilization. A small language model functions as a local decision engine, performing heuristic inference an[...]


Our summary: A hybrid generative AI system enhances cloud-centric AI efficiency through dynamic memory indexing. It features a three-object architecture for user profile, thread index, and dialog control. The system reduces response time by 84% and cloud dependency by 80% while ensuring cognitive coherence.

neural networks, cloud computing, memory indexing, dialog systems

Patent

Template guided speculative decoding

Patent published on the 2026-10-06 in US under Ref US12754958 by INTUIT INC [US] (Ben Shoham Ofir [il], Bar Eliyahu Natalie [il], Mendelson Shon [il], Antebi Sagiv [il], Ardazi Shai [il])

Abstract: Aspects of the present disclosure involve accelerated machine learning based sequence generation. Aspects include generating a first portion of a sequence based on an input query using a speculative decoding process involving a draft machine learning model and a target machine learning model with a larger number of parameters than the draft model. Aspects include determining, based on the input query and the first portion, that a next portion of the sequence is predicted to correspond to a templ[...]


Our summary: This disclosure involves a method for accelerated sequence generation using speculative decoding with two machine learning models. It includes generating a sequence portion based on an input query and determining if the next portion corresponds to a template. The next portion is generated through a template-based process and verified using a larger target model before appending to the sequence.

speculative decoding, sequence generation, machine learning models, template-based process

Patent

Secure explanation generation by large language models for sensitive search targets

Patent published on the 2026-10-06 in US under Ref US12755173 by INTUIT INC [US] (Eliyahu Natalie Bar [il], Mendelson Shon [il], Hazan Itay [il], Bechler Sigalit [il], Haas Noa [il], Bitton Ron [il])

Abstract: [0000] A method includes generating, by a large language model (LLM), a multitude of match explanations for a multitude of search targets. The multitude of match explanations is clustered to obtain a multitude of explanation clusters. An explanation template is generated for an explanation cluster. The explanation template includes a static portion and a dynamic portion. A multitude of explanation templates is obtained in this manner. A target match for a search target received from a user appli[...]


Our summary: The method generates match explanations for search targets using a large language model. These explanations are clustered to create templates with static and dynamic portions. A small language model populates match explanations based on user search targets and encoded templates.

explanation generation, large language model, sensitive search targets, clustering

Patent

Speculative decoding using expanded token matching

Patent published on the 2026-09-03 in US under Ref US20260260065 by MICROSOFT TECH LICENSING LLC [US] (Pathak Sayan Dev [us], Kesavamoorthy Harini [us], Basoglu Christopher Hakan [us], Vikram Ayush [us], Shah Amy Parag [us], Kelkar Rakesh [us])

Abstract: [0000] A speculative decoding technique generates tokens of a response using a verifying model and drafting model. In operation, the verifying model receives a set of draft tokens produced by the drafting model. The verifying model then verifies whether the draft tokens are correct by comparing probability information generated by the verifying model with probability information generated by the drafting model. The verifying model then determines whether any draft token that is rejected by the v[...]


Our summary: Speculative decoding generates response tokens using a verifying and drafting model. The verifying model checks draft tokens against probability information from both models. This iterative process expands accepted tokens, reducing resource use and latency.

speculative decoding, token matching, verifying model, drafting model

Patent

Automated synthesis of planar linkage mechanisms with diverse joint types via spring-connected link models and contrastive graph learning

Published on 2026-03-28 by @OXFORD

Abstract: AbstractThe automated synthesis of planar linkage mechanisms has long been a challenge in mechanism design, requiring both geometric feasibility and motion accuracy. Recent advances in data-driven and neural network–based methods have shown promise in automating linkage synthesis, improving efficiency and scalability compared to traditional analytical or optimization-based techniques. Nevertheless, existing data-driven approaches remain limited in handling diverse joint configurations and ofte[...]


Our summary: This study presents a framework for automating the synthesis of planar linkage mechanisms using deep learning and physics-based modeling. It employs a spring-connected link model for diverse joint configurations and utilizes contrastive graph learning for efficient linkage retrieval. The method demonstrates improved accuracy and optimization stability compared to traditional approaches.

mechanism synthesis, deep learning, contrastive graph learning, optimization stability

Publication

Enhancing Whisper Fine-Tuning with Discrete Wavelet Transform-Based LoRA Initialization

Published on 2026-01-29 by Liang Lan, Molin Fang, Yuxuan Chen, Daliang Wang, Wenyong Wang @MDPI

Abstract: In low-resource automatic speech recognition (ASR) scenarios, parameter-efficient fine-tuning (PEFT) has become a crucial approach for adapting large pre-trained speech models. Although low-rank adaptation (LoRA) offers clear advantages in efficiency, stability, and deployment friendliness, its performance remains constrained because random initialization fails to capture the time&amp;ndash;frequency structural characteristics of speech signals. To address this limitation, this work proposes[...]


Our summary: This work introduces a structured initialization mechanism combining LoRA with discrete wavelet transform for fine-tuning in low-resource ASR. The proposed DWTLoRA method enhances convergence speed, stability, and accuracy by aligning with speech signal characteristics. Experimental results show DWTLoRA outperforms standard LoRA and other PEFT methods in character error rate and training efficiency.

Fine-Tuning, Discrete Wavelet Transform, Low-Rank Adaptation, Automatic Speech Recognition

Publication

取り上げるトピック: 小規模言語モデル、自然言語処理、Transformer ベースシステム、パラメータ効率、知識蒸留、モデル圧縮、構造化枝刈り、非構造化枝刈り、重み量子化、INT4、INT8、微調整手法、デバイス内展開、推論遅延、エネルギー消費、プライバシーに敏感なアプリケーション、低帯域幅操作、オフライン操作コンテキスト、IEEE 80211、ISO/IEC 30170、ISO/IEC 27001、ISO/IEC 25010、および NIST SP 800-53。

用語集

Natural Language Processing (NLP): 人工知能の一分野であり、コンピュータと人間の言語との相互作用に焦点を当て、機械が自然言語のテキストや音声を理解、解釈、生成することを可能にする。言語翻訳、感情分析、音声認識などのタスクが含まれる。

Small Language Models (SLM): 自然言語処理タスク向けに設計されたコンパクトなニューラルネットワークは、一般的に、より大規模なモデルと比較してパラメータ数が少なく、計算要件も軽減されているという特徴を持ちながら、限られた範囲内で一貫性のあるテキストを生成し、文脈を理解する能力を備えている。

歴史的背景

1965
1970
1970
1974-11-15
1980
1980
1980
1964
1968
1970
1970
1975
1980
1980
1980

(日付が不明または関連性がない場合、例えば「流体力学」などでは、その注目すべき出現時期の概算値が提示されます。)

10年目 周年

フルサイズの画像とダウンロードは、登録会員のみが100%無料で利用できます。