漢德百科全書 | 汉德百科全书
IT-Times
知识蒸馏（knowledge distillation）是人工智能领域的一项模型训练技术。该技术透过类似于教师—学生的方式，令规模较小、结构较为简单的人工智能模型从已经经过充足训练的大型、复杂模型身上学习其掌握的知识。该技术可以让小型简单模型快速有效学习到大型复杂模型透过漫长训练才能得到的结果，从而改善模型的效率、减少运算开销，因此亦被称为模型蒸馏（model distillation）。
In machine learning, knowledge distillation or model distillation is the process of transferring knowledge from a large model to a smaller one. While large models (such as very deep neural networks or ensembles of many models) have more knowledge capacity than small models, this capacity might not be fully utilized. It can be just as computationally expensive to evaluate a model even if it utilizes little of its knowledge capacity. Knowledge distillation transfers knowledge from a large model to a smaller one without loss of validity. As smaller models are less expensive to evaluate, they can be deployed on less powerful hardware (such as a mobile device).
Model distillation is not to be confused with model compression, which describes methods to decrease the size of a large model itself, without training a new model. Model compression generally preserves the architecture and the nominal parameter count of the model, while decreasing the bits-per-parameter.
Knowledge distillation has been successfully used in several applications of machine learning such as object detection, acoustic models, and natural language processing. Recently, it has also been introduced to graph neural networks applicable to non-grid data.
Games
Science and technology
History
N 2000 - 2100 AD
IT-Times
Production Engineering/Manufacturing Technologies
IT-Times
Artificial Intelligence
IT-Times
CNC
IT-Times
CRM/EAM/ERP/SRM/SCM/HCM/QM/XM/WFM
IT-Times
CAD/CAE/CAM/EDA/PDM/PLM
IT-Times
Industrial Robot
IT-Times
PLC/DCS/FCS/SCADA/MES
United States
| Technology | Location | |
|---|---|---|
| National Additive Manufacturing Innovation Institute (NAMII) | 3D printing | Youngstown, Ohio |
| Digital Manufacturing and Design Innovation Institute (DMDII) | Digital manufacturing | Chicago, Illinois |
| Lightweight Materials Manufacturing Innovation Institute (ALMMII) | Lightweight materials | Detroit, Michigan |
| Next Generation Power Electronics Institute (PowerAmerica) | Wide-bandgap semiconductors | Raleigh, North Carolina |
| Institute for Advanced Composites Manufacturing Innovation (IACMI) | Composite materials | Knoxville, Tennessee |
| American Institute for Manufacturing Integrated Photonics (AIM Photonics) | Photonic integrated circuits | Rochester, New York |
| Flexible Hybrid Electronics Manufacturing Innovation Institute | Flexible electronics | San Jose, California |
| Advanced Functional Fabrics of America (AFFOA) | Textiles | Cambridge, Massachusetts |
| Smart Manufacturing Innovation Institute | Smart manufacturing | Los Angeles, California |
Medical, Pharmaceutical, Rehabilitation
Medical equipment
Automobile
Graphcore智能处理单元（IPU）是一个高度并行的处理器，专门为机器学习和人工智能应用而设计。基于IPU的系统，如IPU-POD™或Bow™ Pod，连接到一个主机，该主机可以在一个或多个IPU上执行代码。
Die Graphcore Intelligence Processing Unit (IPU) ist ein hochparalleler Prozessor, der speziell für maschinelles Lernen und Anwendungen der künstlichen Intelligenz entwickelt wurde. Ein IPU-basiertes System, wie z. B. ein IPU-POD™ oder ein Bow™ Pod, wird mit einem Host-Computer verbunden, der Code auf einer oder mehreren IPUs ausführen kann.