第1节 · 深度学习简介
🎯 了解深度学习基本概念与发展脉络。
📖 笔记
Day01 笔记：深度学习简介
Day 01 - 深度学习简介
学习目标
-
理解深度学习的基本概念
-
了解深度学习的常用模型
-
了解深度学习的预备数学知识
-
掌握安装pytorch的方法
-
理解张量的概念
-
掌握张量的基本创建方式
-
掌握创建线性和随机张量
-
掌握创建全 0/1 张量
-
掌握张量元素的类型转换
-
掌握张量 和 numpy 之间的相互转换
学习目标
- 理解深度学习的基本概念
- 深度学习（Deep Learning）是机器学习的分支，是一种以人工神经网络为架构，对数据进行特征学习的算法。
- 深度学习中的“深度”是指在网络中使用多层。
- 了解深度学习的常用模型
- 全连接神经网络（DNN / MLP）： 适用于结构化数据（如表格数据）、简单分类与回归任务，比如商品销量预测、房价预测等。
- 卷积神经网络（CNN）： 适用于处理图像与视觉任务，比如图像分类、目标检测、图像分割、人脸识别等。
- 循环神经网络（RNN）： 适用于顺序数据相关的任务，比如自然语言处理（NLP）、语音识别、时间序列预测。
- Transformer： 当前主流模型，覆盖文本与图像等多领域，比如机器翻译、文本生成、对话系统、多模态任务。
- Diffusion（扩散模型）： 当前图像/视频生成的主导架构（如 Stable Diffusion、Sora）。
-
了解深度学习的预备数学知识
- 线性代数
- 标量只有一个数值，称作0维/0D张量
- 向量为标量组成的列表，称作一维张量/1D张量
- 矩阵为多个向量组成的二维数组，称作二维张量/2D张量
- \(L_2\)范数是向量元素平方和的平方根：
- 导数和微分
- 导数是一个比值极限，可以将导数\(f'(x)\)解释为\(f(x)\)相对于\(x\)的 瞬时 变化率
- 线性代数
-
掌握安装pytorch的方法
- 安装miniconda/anaconda
- 创建虚拟环境dl_project
- 安装pytorch
- 验证pytorch安装
- 理解张量的概念
- 张量（Tensor）是深度学习中的数据容器，张量可以描述任意数量轴的n维数组，n≥0
- 掌握张量的基本创建方式
- torch.tensor 根据指定数据创建张量
- torch.Tensor 根据形状创建张量, 其也可用来创建指定数据的张量
- torch.IntTensor、torch.FloatTensor、torch.DoubleTensor 创建指定类型的张量
- 掌握创建线性和随机张量
- torch.arange() 创建一个线性张量，一维张量
- torch.linspace() 创建一个等差数列，一维张量
- torch.rand() 创建一个随机数，形状任意
- torch.randn() 创建一个随机数, 形状任意
- torch.randint() 创建一个随机数, 形状任意
- 掌握创建全 0/1 张量
- torch.ones 和 torch.ones_like 创建全1张量
- torch.zeros 和 torch.zeros_like 创建全0张量
- torch.full 和 torch.full_like 创建全为指定值张量
- 掌握张量元素的类型转换
- data.type(torch支持的数据类型)
- data.to(torch支持的数据类型)
- data.half/double/float/short/int/long()
- 掌握张量 和 numpy 之间的相互转换
- 张量 转 numpy数组对象：张量对象.numpy()
- numpy数组对象 转 张量：torch.tensor(numpy数组对象)
- 标量张量 转 python数值：张量对象.item()
# 导包
import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
import matplotlib.pyplot as plt
# 设置中文字体
plt.rcParams['font.sans-serif'] = ['SimHei'] # 微软雅黑
plt.rcParams['axes.unicode_minus'] = False # 解决负号显示问题
# 设置设备
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
课程大纲
-
- 预备数学知识
-
- pytorch张量操作
-
- ANN人工神经网络 Artifical neural network
-
- CNN卷积神经网络 Convolutional neural network
-
- RNN循环神经网络 Recurrent neural network
1.什么是深度学习[知道]
深度学习（Deep Learning）是机器学习的分支，是一种以人工神经网络为架构，对数据进行特征学习的算法。深度学习中的形容词“深度”是指在网络中使用多层。
深度学习核心思想是==通过模仿人脑的神经网络来处理和分析复杂的数据，从大量数据中自动提取复杂特征==，擅长处理高维数据，如图像、语音和文本。
2.深度学习特点[知道]
-
多层非线性变换：神经网络模型由多个层次组成，每一层都应用非线性激活函数对输入数据进行变换。较低的层级通常捕捉到简单的特征（如边缘、颜色等），而更高的层级则可以识别更复杂的模式（如物体或面部识别）。
-
自动特征提取：与传统机器学习算法不同，深度学习能够从原始数据中自动提取特征，不需要人工特征工程。
-
大数据和计算能力：神经网络模型通常需要大量的标注数据和强大的计算资源（如GPU）来进行训练。大数据和高性能计算使得深度学习在图像识别、自然语言处理等领域取得了显著突破。
-
可解释性差：神经网络模型内部的运作机制相对不透明，被称为“黑箱”，难以解释模型的决策逻辑或过程。
3.常见的深度学习模型[了解]
- 全连接神经网络（DNN / MLP）
- 输入输出均为定长向量，多用于结构化数据的分类与回归。
-
在图像/文本任务中已很少单独使用，多作为基础组件。
-
卷积神经网络（CNN）
- 利用卷积核提取局部特征，是图像分类、检测、分割的经典主力。
-
工程部署中仍是最常用、最高效的视觉模型体系。
-
循环神经网络（RNN）
- 处理序列数据（时间序列、NLP），LSTM/GRU 能建模长依赖。
-
在大多数任务中已被 Transformer 取代。
-
Diffusion Models（扩散模型）
- 当前图像/视频生成的主导架构（如 Stable Diffusion、Sora）。
-
生成质量、稳定性和自由度全面优于 GAN。
-
Transformer（当今最主流架构）
- 通过自注意力捕获全局关系，是 NLP 的事实标准。
-
在视觉领域衍生出 ViT、Swin Transformer，并成为 LLM / VLM 的核心架构。
-
图神经网络（GNN）
- 处理图结构数据，如分子结构、社交网络、知识图谱。
-
典型模型：GCN、GAT、GraphSAGE，在药物发现与推荐系统中应用广泛。
-
深度强化学习（DRL）
- 将神经网络与强化学习结合，用于序列决策与控制。
- 典型方法：DQN、PPO；应用于 AlphaGo、机器人、自主驾驶等。
4.深度学习应用场景[了解]
-
==计算机视觉（Computer Vision）==
-
图像分类：将图像分为不同的类别。常用于人脸识别、物体检测等。
- 自动标注社交媒体照片、医疗影像中的病变检测。
- 目标检测（Object Detection）：在图像或视频中定位并分类多个对象。
- 自动驾驶中的行人检测、监控视频中的入侵检测。
-
面部识别：通过分析面部特征进行身份验证或分类。
- 手机解锁、安防监控系统。
-
图像生成：基于输入生成新的图像，如风格转换、图像超分辨率等。
- 艺术风格迁移、老旧照片修复。
-
==自然语言处理（Natural Language Processing, NLP）==
-
机器翻译：使用深度学习模型将一种语言的文本自动翻译成另一种语言。
- Google翻译、实时语音翻译。
-
情感分析：分析文本中的情感倾向，如正面、负面或中性。
- 社交媒体监控、产品评论分析。
-
文本生成：生成符合语法和语义的自然语言文本。
- 自动写作助手、新闻生成。
-
语音识别：将语音转化为文字。
- 智能助手（如Siri、Alexa）、自动字幕生成。
-
聊天机器人（Chatbot）：通过深度学习理解用户输入并生成合理的回应。
- 客服机器人、虚拟助手（如GPT类模型）。
-
==推荐系统（Recommendation Systems）==
-
电影、音乐推荐：根据用户历史的评分和行为，推荐相关的电影、音乐或电视剧。
- Netflix、Spotify的个性化推荐。
- 电商推荐：根据用户的购买历史和浏览习惯推荐商品。
- 亚马逊、淘宝的商品推荐系统。
- 社交媒体推荐：分析用户的社交行为，推荐相关内容或朋友。
- Facebook、Instagram的内容推荐。
...
5.深度学习发展史[了解]
-
早期探索
-
20世纪40年代：沃尔特·皮茨（Walter Pitts）和沃伦·麦卡洛克（Warren McCulloch）等开始模仿生物神经系统来构建计算模型，如McCulloch-Pitts神经元
- 1958年：弗兰克·罗森布拉特（Frank Rosenblatt）提出感知器概念，能够进行简单的二分类任务
- 1960年代末：出现了多层感知器（MLP），但当时由于计算能力和数据量的限制，这些模型的应用受到很大限制
- 图灵测试：如果一台机器能够与人类展开对话（透过电传设备）而不被辨别出其机器身份，那么称这台机器具有智慧
-
挑战与瓶颈
-
1986年：反向传播算法（Backpropagation）的提出标志着神经网络研究的一个重要突破。杰弗里·辛顿（Geoffrey Hinton）和大卫·鲁梅尔哈特（David Rumelhart）等人提出了反向传播算法，使得多层神经网络（即深层网络）能够通过梯度下降优化参数，解决复杂的非线性问题。
- 虽然神经网络方法在一些领域表现不错，但由于计算资源的限制以及对复杂数据（如图像和语音）的处理能力较弱，深度学习未能广泛应用。此时，支持向量机（SVM）、决策树等传统机器学习方法成为主流。
-
复兴与突破
-
2006年：杰弗里·辛顿和其团队提出了深度信念网络（DBN），标志着深度学习的复兴。他们引入了无监督预训练的技术，使得深层网络能够有效训练。这为深度学习的发展奠定了基础。
- 2012年：深度学习的一个重要突破是AlexNet的出现。亚历克斯·克里泽夫斯基（Alex Krizhevsky）在ImageNet图像分类竞赛中使用了一个深度卷积神经网络，显著提升了图像分类的精度，比传统方法提高了20%以上。AlexNet的成功标志着深度学习在计算机视觉领域的成功应用。
- 2014年：生成对抗网络（GANs）由伊恩·古德费洛（Ian Goodfellow）等人提出，开启了生成模型的新时代，能够生成非常逼真的图像、音频和视频。
- 2015年：ResNet（残差网络）由何凯明（Kaiming He）等提出，解决了深度网络中的梯度消失和梯度爆炸问题，允许训练极深的网络（如50层、152层），极大推动了深度学习在图像识别任务中的应用。
- 爆发期
- **2016年：**Google AlphaGo 战胜李世石（人工智能第三次浪潮），AlphaGo 展现了深度强化学习（Deep Reinforcement Learning）在解决复杂问题上的巨大潜力，将其推向了公众视野。
- **2017年：**自然语言处理NLP的Transformer框架出现，奠定了后续预训练语言模型（如 BERT 和 GPT）的基础。
- **2018年：**BERT和GPT的出现，基于Transformer架构的预训练语言模型的代表。
- **2022年：**ChatGPT的出现，进入到大模型AIGC发展的阶段，开启了 AI 与人交互的新模式，使人们可以更容易地使用 AI 并从中受益。
6.扩展-预备的数学知识-理解
线性代数
- 标量
标量只有一个数值，称作零维张量/0D张量
下面的代码将实例化两个标量，并执行一些熟悉的算术运算，即加法、乘法、除法和指数。
import torch
x = torch.tensor(3.0)
y = torch.tensor(2.0)
x + y, x-y, x * y, x / y, x**y
(tensor(5.), tensor(1.), tensor(6.), tensor(1.5000), tensor(9.))
- 向量
向量可以被视为标量值组成的列表，称作一维张量/1D张量
通过一维张量表示向量。
x = torch.arange(5)
x
tensor([0, 1, 2, 3, 4])
在数学中，向量\(\mathbf{x}\)可以写为：
\[\mathbf{x} =\begin{bmatrix}x_{1} \\x_{2} \\ \vdots \\x_{n}\end{bmatrix},\]
其中\(x_1,\ldots,x_n\)是向量的元素。在代码中，通过张量的索引来访问任一元素。
x[3]
tensor(3)
- 矩阵
多个向量组成的二维数组，称作二维张量/2D张量
通常用粗体、大写字母来表示
（例如，\(\mathbf{X}\)、\(\mathbf{Y}\)和\(\mathbf{Z}\)），
表示具有两个轴的张量。
数学上，使用\(\mathbf{A} \in \mathbb{R}^{m \times n}\)
来表示矩阵\(\mathbf{A}\)，其由\(m\)行和\(n\)列的实值标量组成。
可以将任意矩阵\(\mathbf{A} \in \mathbb{R}^{m \times n}\)视为一个表格，
其中每个元素\(a_{ij}\)属于第\(i\)行第\(j\)列：
\[\mathbf{A}=\begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \\ \end{bmatrix}.\]
对于任意\(\mathbf{A} \in \mathbb{R}^{m \times n}\)，
\(\mathbf{A}\)的形状是（\(m\),\(n\)）或\(m \times n\)。
通过指定形状\(m \times n\)来创建矩阵
A = torch.arange(20).reshape(5, 4)
A
tensor([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15],
[16, 17, 18, 19]])
可以通过行索引（\(i\)）和列索引（\(j\)）来访问矩阵中的标量元素\(a_{ij}\)，
例如\([\mathbf{A}]_{ij}\)。
A[1,2]
tensor(6)
矩阵的转置(transpose) : 交换矩阵的行和列。
通常用\(\mathbf{a}^\top\)来表示矩阵的转置。
A.T
tensor([[ 0, 4, 8, 12, 16],
[ 1, 5, 9, 13, 17],
[ 2, 6, 10, 14, 18],
[ 3, 7, 11, 15, 19]])
- 张量
深度学习中的数据容器，可以描述任意数量轴的n维数组, n≥0
张量是描述具有任意数量轴的\(n\)维数组的通用方法。
例如，向量是一维张量，矩阵是二维张量。
张量用特殊字体的大写字母表示（例如，\(\mathsf{X}\)、\(\mathsf{Y}\)和\(\mathsf{Z}\)），
它们的索引机制（例如\(x_{ijk}\)和\([\mathsf{X}]_{1,2i-1,3}\)）与矩阵类似。
比如图像数据，通常为3维张量，HWC格式的3个轴对应于高度height、宽度width，以及一个通道channel轴，用于表示颜色通道（红色、绿色和蓝色）。
32323
X = torch.arange(24).reshape(2, 3, 4)
X
tensor([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]],
[[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]]])
- 张量的维度和形状
张量的维度用来表示张量具有的轴的数量。张量的某个轴的维数就是这个轴/维的长度。
| 概念 | 0D: 体重60 | 1D: 股价[10, 9, 8, 7, 6] | 2D: 多个股价[[10, 9, 8, 7, 6], [10, 9, 8, 7, 6]] | 3D: 单个图片HWC, 形状(32, 32, 3) | |
|---|---|---|---|---|---|
| 形状 Shape | 张量在每个轴的长度，描述了张量的结构。 | () |
(5) |
(2, 5) |
(32, 32, 3) |
| 每个维度的长度 Length | 张量在某一维度的长度，表示该维度/轴的元素数。 | None | 5 |
2, 5 |
32, 32, 3 |
| 维度/轴数 Dimension/Axes | 张量的维度数，即张量形状的长度，表示数据有几个轴。 | 0 | 1 | 2 | 3 |
# 0D 张量（0轴）
tensor_0d = torch.tensor(3)
print("0D 张量:")
print(tensor_0d)
print(f"维度/轴数: {tensor_0d.ndimension()}") # 输出: 0 （0 个轴）
print(f"形状: {tensor_0d.shape}") # 输出: torch.Size([])
print()
0D 张量:
tensor(3)
维度/轴数: 0
形状: torch.Size([])
# 1D 张量（一轴）
tensor_1d = torch.tensor([1, 2, 3, 4])
print("1D 张量:")
print(tensor_1d)
print(f"维度/轴数: {tensor_1d.ndimension()}") # 输出: 1 （1 个轴）
print(f"形状: {tensor_1d.shape}") # 输出: torch.Size([4])
print(f"第一轴的长度: {tensor_1d.size(0)}") # 输出: 4 （第一轴的长度）
print()
1D 张量:
tensor([1, 2, 3, 4])
维度/轴数: 1
形状: torch.Size([4])
第一轴的长度: 4
# 2D 张量（两轴）
tensor_2d = torch.tensor([[1, 2, 3], [4, 5, 6]])
print("2D 张量:")
print(tensor_2d)
print(f"维度/轴数: {tensor_2d.ndimension()}") # 输出: 2 （2 个轴）
print(f"形状: {tensor_2d.shape}") # 输出: torch.Size([2, 3])
print(f"第一轴的长度: {tensor_2d.size(0)}") # 输出: 2 （第一轴的长度）
print(f"第二轴的长度: {tensor_2d.size(1)}") # 输出: 3 （第二轴的长度）
print()
2D 张量:
tensor([[1, 2, 3],
[4, 5, 6]])
维度/轴数: 2
形状: torch.Size([2, 3])
第一轴的长度: 2
第二轴的长度: 3
# 3D 张量（三轴）
tensor_3d = torch.randn(2, 3, 4) # 随机生成一个形状为 (2, 3, 4) 的张量
print("3D 张量:")
print(tensor_3d)
print(f"维度/轴数: {tensor_3d.ndimension()}") # 输出: 3 （3 个轴）
print(f"形状: {tensor_3d.shape}") # 输出: torch.Size([2, 3, 4])
print(f"第一轴的长度: {tensor_3d.size(0)}") # 输出: 2 （第一轴的长度）
print(f"第二轴的长度: {tensor_3d.size(1)}") # 输出: 3 （第二轴的长度）
print(f"第三轴的长度: {tensor_3d.size(2)}") # 输出: 4 （第三轴的长度）
3D 张量:
tensor([[[-0.6098, -1.5741, -0.1081, -1.1542],
[ 0.8110, -1.1747, -0.0876, -0.7189],
[ 0.4491, -0.4631, -0.6066, -0.2520]],
[[-1.0018, 0.8333, 1.6792, 1.0797],
[-1.0365, 0.1009, 0.3324, -0.0785],
[ 0.8173, -1.6884, -0.3801, 0.4057]]])
维度/轴数: 3
形状: torch.Size([2, 3, 4])
第一轴的长度: 2
第二轴的长度: 3
第三轴的长度: 4
- 范数
向量的范数是表示一个向量有多大。
\(L_2\)范数是向量元素平方和的平方根：
($\(\|\mathbf{x}\|_2 = \sqrt{\sum_{i=1}^n x_i^2},\)$)
其中，在\(L_2\)范数中常常省略下标\(2\)，也就是说\(\|\mathbf{x}\|\)等同于\(\|\mathbf{x}\|_2\)。
在代码中，我们可以按如下方式计算向量的\(L_2\)范数。
注意：深度学习中的L2范数，是指平方项的和，没有开根号。
u = torch.tensor([3, 4.])
torch.norm(u)
tensor(5.)
\(L_1\)范数，它表示为向量元素的绝对值之和：
($\(\|\mathbf{x}\|_1 = \sum_{i=1}^n \left|x_i \right|.\)$)
与\(L_2\)范数相比，\(L_1\)范数受异常值的影响较小。
torch.abs(u).sum()
tensor(7.)
\(L_p\)范数：
\[\|\mathbf{x}\|_p = \left(\sum_{i=1}^n \left|x_i \right|^p \right)^{1/p}\]
[矩阵]\(\mathbf{X} \in \mathbb{R}^{m \times n}\) 的Frobenius范数（Frobenius norm）是矩阵元素平方和的平方根：
$\(\|\mathbf{X}\|_F = \sqrt{\sum_{i=1}^m \sum_{j=1}^n x_{ij}^2}\)$
torch.norm(torch.ones((9, 9)))
tensor(9.)
导数和微分
假设有一个函数\(f: \mathbb{R} \rightarrow \mathbb{R}\)，其输入和输出都是标量。
\(f\)的导数被定义为:
$\(f'(x) = \lim_{h \rightarrow 0} \frac{f(x+h) - f(x)}{h}\)$
如果\(f'(a)\)存在，则称\(f\)在\(a\)处是可微（differentiable）的。
如果\(f\)在一个区间内的每个数上都是可微的，则此函数在此区间中是可微的。
我们可以将导数\(f'(x)\)解释为\(f(x)\)相对于\(x\)的瞬时（instantaneous）变化率。
注意：
- 导函数\(f'(x)\)表示函数\(f(x)\)在一个区间的导数。
- 导数\(f'(a)\)表示导函数\(f'(x)\)在\(a\)处的数值。
- 深度学习中，导数也称作梯度， 导数可能是 导函数 或 导数，根据语义区分。
为了更好地解释导数，让我们做一个实验。
(定义\(u=f(x)=3x^2-4x\))如下：
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
from matplotlib_inline import backend_inline
def f(x):
return 3 * x ** 2 - 4 * x # 6x - 4
通过令\(x=1\)并让\(h\)接近\(0\)，(\(\frac{f(x+h)-f(x)}{h}\)的数值结果接近\(2\))
def numerical_lim(f, x, h):
return (f(x + h) - f(x)) / h
h = 0.1
for i in range(6):
print(f'h={h:.6f}, numerical limit={numerical_lim(f, 1, h):.6f}')
h *= 0.1
h=0.100000, numerical limit=2.300000
h=0.010000, numerical limit=2.030000
h=0.001000, numerical limit=2.003000
h=0.000100, numerical limit=2.000300
h=0.000010, numerical limit=2.000030
h=0.000001, numerical limit=2.000003
导数符号，
给定\(y=f(x)\)，其中\(x\)和\(y\)分别是函数\(f\)的自变量和因变量。以下表达式是等价的：
\[f'(x) = y' = \frac{dy}{dx} = \frac{df}{dx} = \frac{d}{dx} f(x) = Df(x) = D_x f(x),\]
其中符号\(\frac{d}{dx}\)和\(D\)是微分运算符，表示微分操作。
常见函数求微分：
- \(C' = 0\)（\(C\)是一个常数）
- \((x^n)' = nx^{n-1}\)（\(n\)是任意实数）
- \((e^x)' = e^x\)
- \((\ln(x))' = 1/x\)
# 创建 x 数据
x = np.arange(0, 3, 0.1)
# 绘制 f(x) 和切线
plt.plot(x, f(x), label="f(x) = 3 * x ^ 2 - 4 * x") # 绘制函数 f(x)
plt.plot(x, 2 * x - 3, label='Tangent line (x=1): f(x) = 2 * x - 3', linestyle='--') # 绘制切线
# 添加标签、标题和图例
plt.xlabel('x')
plt.ylabel('y')
plt.title('Plot of f(x) and its Tangent Line at x=1')
plt.legend()
plt.grid()
plt.show()
7. 安装PyTorch
- 安装miniconda
下载地址 https://www.anaconda.com/download/success
- 初始化conda
# conda init --all
- 创建虚拟环境dl_project
# conda create -n dl_project python=3.10
# conda activate dl_project
- 安装pytorch
选择CPU版本/GPU版本，
参考网址：https://pytorch.org/get-started/locally/
如果没有GPU，直接安装CPU版本的pytorch
# CPU
# pip3 install torch torchvision
如果有英伟达的GPU，首先检查支持的CUDA版本，然后安装对应CUDA版本的pytorch
- 建议：如果GPU显存 >= 4G，则安装GPU版本的pytorch，否则直接安装CPU版本的pytorch
# 首先查看本机支持的CUDA版本,向下兼容
nvidia-smi
# GPU, 比如CUDA12.6，要根据CUDA版本选择对应的安装命令，参考
# pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu126
- 验证pytorch安装
import torch
print("pytorch版本: ", torch.__version__)
print("能否使用GPU: ", torch.cuda.is_available())
if torch.cuda.is_available():
print(f"GPU型号: {torch.cuda.get_device_name()}") # 获取 GPU 名称
pytorch版本: 2.10.0.dev20251114+cu128
能否使用GPU: True
GPU型号: NVIDIA GeForce RTX 5070 Laptop GPU
8. 什么是PyTorch？
PyTorch是Facebook开发的深度学习框架，具有以下特点：
- 类似于NumPy的张量计算
- PyTorch中的基本数据结构是张量（Tensor），它与NumPy中的数组类似，但PyTorch的张量具有GPU加速的能力（通过CUDA），这使得深度学习模型能够高效地在GPU上运行。
- 自动微分系统
- PyTorch提供了强大的自动微分功能（autograd模块），能够自动计算模型中每个参数的梯度。
- 自动微分使得梯度计算过程变得简洁和高效，并且支持复杂的模型和动态计算图。
- 深度学习库
- PyTorch提供了一个名为torch.nn的子模块，用于构建神经网络。它包括了大量的预构建的层（如全连接层、卷积层、循环神经网络层等），损失函数（如交叉熵、均方误差等），以及优化算法（如SGD、Adam等）。
- torch.nn.Module是PyTorch中构建神经网络的基础类，用户可以通过继承该类来定义自己的神经网络架构。
- 动态计算图
- PyTorch使用动态计算图机制，允许在运行时构建和修改模型结构，具有更高的灵活性，适合于研究人员进行实验和模型调试。
- GPU加速（CUDA支持）
- PyTorch提供对GPU的良好支持，能够在NVIDIA的CUDA设备上高效地进行计算。用户只需要将数据和模型转移到GPU上，PyTorch会自动优化计算过程。
- 通过简单的
tensor.to(device)方法，可以轻松地将模型和数据从CPU转移到GPU或从一个GPU转移到另一个GPU。 - 跨平台支持
- PyTorch支持在多种硬件平台（如CPU、GPU、TPU等）上运行，并且支持不同操作系统（如Linux、Windows、macOS）以及分布式计算环境（如多GPU、分布式训练）。
9. 什么是PyTorch张量？
张量是PyTorch中的核心数据抽象
-
PyTorch中的张量就是元素为同一种数据类型的多维数组，类似NumPy数组。
-
PyTorch中，张量以"类"的形式封装起来，对张量的一些运算、处理的方法（数值计算、矩阵操作、自动求导）被封装在类中。
day01_01_张量的基本创建方式
- torch.tensor()
- torch.Tensor()
- 张量的数据类型有
- 张量中默认的数据类型是float32(torch.FloatTensor)
- torch.tensor 根据指定数据创建张量
"""
案例：
演示 张量的基本创建方式
张量：
pytorch框架 是 最常用的深度学习框架， 后续 ANN, CNN, RNN, 底层数据操作，都是使用 张量
张量：存储 同一种数据类型的元素的容器，元素必须是数值
张量的基本创建方式：
torch.tensor 根据指定数据创建张量
torch.Tensor 根据形状创建张量, 其也可用来创建指定数据的张量
torch.IntTensor、torch.FloatTensor、torch.DoubleTensor 创建指定类型的张量
tensor 与 Tensor的区别：
Tensor 多了一个功能：可以指定形状
需要掌握：
torch.tensor(数据，元素类型)
"""
# 导包
import torch
# 1.定义函数，演示 torch.tensor, 根据数据来创建张量
def demo01():
# 场景1，创建0维 / 0D张量, 也就是标量
t1 = torch.tensor(6)
print("0维张量：")
print(f"t1: {t1}, type: {type(t1)}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
print('-'*60)
# 场景2，创建1维张量，也就是向量
t2 = torch.tensor([1,2,3,4,5], dtype=torch.float32)
print("1维张量：")
print(f"t2: {t2}, type: {type(t2)}, shape: {t2.shape}, dtype元素类型：{t2.dtype}")
print('-'*60)
# 场景3，创建2维张量，也就是矩阵
t3 = torch.tensor([[1,2,3],[4,5,6]])
print("2维张量：")
print(f"t3: {t3}, type: {type(t3)}, shape: {t3.shape}, dtype元素类型：{t3.dtype}")
print('-'*60)
# 创建3维张量，也就是立方体
t4 = torch.tensor([[[1,2,3],[4,5,6]],[[7,8,9],[10,11,12]]], device='cuda')
print("3维张量：")
print(f"t4: {t4}, type: {type(t4)}, shape: {t4.shape}, dtype元素类型：{t4.dtype}")
# 打印张量的属性，type, shape, dtype, device
print(f"t4: type: {t4.type()}, shape: {t4.shape}, dtype元素类型：{t4.dtype}, device: {t4.device}")
# # 场景4，创建指定形状的张量，2*4
# t5 = torch.tensor(2,4)
# print(f"t5: {t5}, type: {type(t5)}, shape: {t5.shape}, dtype元素类型：{t5.dtype}")
# 测试
if __name__ == '__main__':
demo01()
0维张量：
t1: 6, type: <class 'torch.Tensor'>, shape: torch.Size([]), dtype元素类型：torch.int64
------------------------------------------------------------
1维张量：
t2: tensor([1., 2., 3., 4., 5.]), type: <class 'torch.Tensor'>, shape: torch.Size([5]), dtype元素类型：torch.float32
------------------------------------------------------------
2维张量：
t3: tensor([[1, 2, 3],
[4, 5, 6]]), type: <class 'torch.Tensor'>, shape: torch.Size([2, 3]), dtype元素类型：torch.int64
------------------------------------------------------------
3维张量：
t4: tensor([[[ 1, 2, 3],
[ 4, 5, 6]],
[[ 7, 8, 9],
[10, 11, 12]]], device='cuda:0'), type: <class 'torch.Tensor'>, shape: torch.Size([2, 2, 3]), dtype元素类型：torch.int64
t4: type: torch.cuda.LongTensor, shape: torch.Size([2, 2, 3]), dtype元素类型：torch.int64, device: cuda:0
- torch.Tensor 根据形状创建张量, 其也可用来创建指定数据的张量
"""
案例：
演示 张量的基本创建方式
张量：
pytorch框架 是 最常用的深度学习框架， 后续 ANN, CNN, RNN, 底层数据操作，都是使用 张量
张量：存储 同一种数据类型的元素的容器，元素必须是数值
张量的基本创建方式：
torch.tensor 根据指定数据创建张量
torch.Tensor 根据形状创建张量, 其也可用来创建指定数据的张量
torch.IntTensor、torch.FloatTensor、torch.DoubleTensor 创建指定类型的张量
tensor 与 Tensor的区别：
Tensor 多了一个功能：可以指定形状
需要掌握：
torch.tensor(数据，元素类型)
"""
# 导包
import torch
# 2.定义函数，演示 torch.Tensor, 根据数据来创建张量
def demo02():
# 场景1，创建0维 / 0D张量, 也就是标量
t1 = torch.Tensor(6)
print("0维张量：")
print(f"t1: {t1}, type: {type(t1)}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
print('-'*60)
# 场景2，创建1维张量，也就是向量
t2 = torch.Tensor([1,2,3,4,5])
print("1维张量：")
print(f"t2: {t2}, type: {type(t2)}, shape: {t2.shape}, dtype元素类型：{t2.dtype}")
print('-'*60)
# 场景3，创建2维张量，也就是矩阵
t3 = torch.Tensor([[1,2,3],[4,5,6]])
print("2维张量：")
print(f"t3: {t3}, type: {type(t3)}, shape: {t3.shape}, dtype元素类型：{t3.dtype}")
print('-'*60)
# 创建3维张量，也就是立方体
t4 = torch.Tensor([[[1,2,3],[4,5,6]],[[7,8,9],[10,11,12]]])
print("3维张量：")
print(f"t4: {t4}, type: {type(t4)}, shape: {t4.shape}, dtype元素类型：{t4.dtype}")
# 打印张量的属性，type, shape, dtype, device
print(f"t4: type: {t4.type()}, shape: {t4.shape}, dtype元素类型：{t4.dtype}, device: {t4.device}")
# 场景4，创建指定形状的张量，2*4
t5 = torch.Tensor(2, 4)
print(f"t5: {t5}, type: {type(t5)}, shape: {t5.shape}, dtype元素类型：{t5.dtype}")
# 测试
if __name__ == '__main__':
demo02()
0维张量：
t1: tensor([0., 0., 0., 0., 0., 0.]), type: <class 'torch.Tensor'>, shape: torch.Size([6]), dtype元素类型：torch.float32
------------------------------------------------------------
1维张量：
t2: tensor([1., 2., 3., 4., 5.]), type: <class 'torch.Tensor'>, shape: torch.Size([5]), dtype元素类型：torch.float32
------------------------------------------------------------
2维张量：
t3: tensor([[1., 2., 3.],
[4., 5., 6.]]), type: <class 'torch.Tensor'>, shape: torch.Size([2, 3]), dtype元素类型：torch.float32
------------------------------------------------------------
3维张量：
t4: tensor([[[ 1., 2., 3.],
[ 4., 5., 6.]],
[[ 7., 8., 9.],
[10., 11., 12.]]]), type: <class 'torch.Tensor'>, shape: torch.Size([2, 2, 3]), dtype元素类型：torch.float32
t4: type: torch.FloatTensor, shape: torch.Size([2, 2, 3]), dtype元素类型：torch.float32, device: cpu
t5: tensor([[0., 0., 0., 0.],
[0., 0., 0., 0.]]), type: <class 'torch.Tensor'>, shape: torch.Size([2, 4]), dtype元素类型：torch.float32
- torch.IntTensor、torch.FloatTensor、torch.DoubleTensor 创建指定类型的张量
"""
案例：
演示 张量的基本创建方式
张量：
pytorch框架 是 最常用的深度学习框架， 后续 ANN, CNN, RNN, 底层数据操作，都是使用 张量
张量：存储 同一种数据类型的元素的容器，元素必须是数值
张量的基本创建方式：
torch.tensor 根据指定数据创建张量
torch.Tensor 根据形状创建张量, 其也可用来创建指定数据的张量
torch.IntTensor、torch.FloatTensor、torch.DoubleTensor 创建指定类型的张量
tensor 与 Tensor的区别：
Tensor 多了一个功能：可以指定形状
需要掌握：
torch.tensor(数据，元素类型)
"""
# 导包
import torch
# 3.定义函数，演示 torch.IntTensor, FloatTensor, DoubleTensor, 根据数据来创建张量
def demo03():
# 场景1，创建一个2*3的张量，使用torch.FloatTensor
t1 = torch.FloatTensor([[1,2,3],[4,5,6]])
print("2维张量：")
print(f"t1: {t1}, type: {type(t1)}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
# 测试
if __name__ == '__main__':
demo03()
2维张量：
t1: tensor([[1., 2., 3.],
[4., 5., 6.]]), type: <class 'torch.Tensor'>, shape: torch.Size([2, 3]), dtype元素类型：torch.float32
张量的属性
- type: 对象的类型
- shape: 张量的形状
- dtype: 数据类型
- device: 存储位置(CPU/GPU)
- requires_grad: 是否需要梯度
# 张量属性
t = torch.tensor(
[[1, 2, 3], [4, 5, 6], [7, 10, 9]],
dtype=torch.float32, # 使用浮点数（可参与梯度计算）
requires_grad=True # 开启梯度计算
)
print("对象类型:", type(t))
print("形状:", t.shape)
print("元素类型:", t.dtype)
print("设备:", t.device)
print("开启梯度计算/自动微分:", t.requires_grad)
对象类型: <class 'torch.Tensor'>
形状: torch.Size([3, 3])
元素类型: torch.float32
设备: cpu
开启梯度计算/自动微分: True
day01_02_创建线性和随机张量
- torch.arange()
- torch.linspace()
- torch.rand()
- torch.randn()
- torch.randint()
"""
案例：
演示 pytorch中 如何创建 线性 和 随机张量
涉及到的API:
torch.arange() 和 torch.linspace() 创建线性张量
torch.manual_seed() 随机种子设置
torch.rand/randn() 创建随机浮点类型张量
torch.randint(low, high, size=()) 创建随机整数类型张量
要掌握的：
torch.arange()
torch.linspace()
torch.manual_seed()
torch.randint(low, high, size=())
"""
# 导包
import torch
# 1.定义函数，演示 创建线性张量
def demo01():
# 1.使用 torch.arange() 创建 线性张量
# 参数1：起始值(包含)；参数2：结束值(不包含)；参数3：步长
t1 = torch.arange(0,10,2) # [0,2,4,...,8]
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
print('-'*60)
# 2.使用 torch.linspace() 创建 线性张量
# 参数1：起始值(包含)；参数2：结束值(包含)；参数3：元素个数
t1 = torch.linspace(0, 9, 10) # [0,2,4,...,8]
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
# 2.定义函数，演示 创建随机张量
def demo02():
# 0.设置随机种子
torch.manual_seed(6)
# 1.使用torch.rand来创建一个随机张量，形状3*4，从U[0,1]随机采样
t1 = torch.rand(3,4)
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
print('-' * 60)
# 2.使用torch.randn来创建一个随机张量，形状3*4，从标准正态分布N(0,1)随机采样
t1 = torch.randn(3, 4)
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
print('-' * 60)
# 3.使用torch.randint来创建一个随机张量，形状3*4*5, 从[0,10]随机采样
t1 = torch.randint(0, 10, (3, 4, 5))
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
# 测试
if __name__ == '__main__':
demo01()
demo02()
t1: tensor([0, 2, 4, 6, 8]), shape: torch.Size([5]), dtype元素类型：torch.int64
------------------------------------------------------------
t1: tensor([0., 1., 2., 3., 4., 5., 6., 7., 8., 9.]), shape: torch.Size([10]), dtype元素类型：torch.float32
t1: tensor([[0.5722, 0.5539, 0.9868, 0.6080],
[0.2347, 0.4492, 0.6743, 0.7480],
[0.5601, 0.1674, 0.3333, 0.4648]]), shape: torch.Size([3, 4]), dtype元素类型：torch.float32
------------------------------------------------------------
t1: tensor([[ 1.4557, 0.6403, 1.9355, -1.1402],
[-0.1656, 0.3920, 1.0946, -0.4448],
[-1.0702, -2.5711, 0.6639, 0.3103]]), shape: torch.Size([3, 4]), dtype元素类型：torch.float32
------------------------------------------------------------
t1: tensor([[[7, 1, 3, 3, 2],
[3, 4, 2, 2, 1],
[4, 9, 3, 9, 8],
[0, 5, 7, 3, 2]],
[[4, 4, 5, 0, 3],
[4, 6, 6, 5, 4],
[5, 5, 5, 1, 8],
[1, 0, 7, 1, 9]],
[[8, 4, 0, 9, 1],
[4, 3, 4, 1, 6],
[8, 6, 9, 9, 1],
[3, 7, 1, 5, 6]]]), shape: torch.Size([3, 4, 5]), dtype元素类型：torch.int64
day01_03_创建全0_1_指定值张量
- torch.ones()
- torch.ones_like()
- torch.zeros()
- torch.zeros_like()
- torch.full()
- torch.full_like()
"""
案例：
演示 创建 全0，全1，指定值的张量
涉及到的API:
torch.ones 和 torch.ones_like 创建全1张量
torch.zeros 和 torch.zeros_like 创建全0张量
torch.full 和 torch.full_like 创建全为指定值张量
需要掌握：
torch.zeros, 用于初始化模型的偏置参数
"""
# 导包
import torch
# 1.场景1： 演示 torch.ones 和 torch.ones_like 创建全1张量
t1 = torch.ones(2,3, dtype=torch.int64)
print(f"t1: {t1}, shape: {t1.shape}, dtype: {t1.dtype}")
print('-'*60)
# 演示 ones_like
t2 = torch.ones_like(t1)
print(f"t2: {t2}, shape: {t2.shape}, dtype: {t2.dtype}")
t1: tensor([[1, 1, 1],
[1, 1, 1]]), shape: torch.Size([2, 3]), dtype: torch.int64
------------------------------------------------------------
t2: tensor([[1, 1, 1],
[1, 1, 1]]), shape: torch.Size([2, 3]), dtype: torch.int64
# 2.场景1： 演示 torch.zeros 和 torch.zeros_like 创建全1张量
t1 = torch.zeros(2,3, dtype=torch.int64)
print(f"t1: {t1}, shape: {t1.shape}, dtype: {t1.dtype}")
print('-'*60)
# 演示 zeros_like
t2 = torch.zeros_like(t1)
print(f"t2: {t2}, shape: {t2.shape}, dtype: {t2.dtype}")
t1: tensor([[0, 0, 0],
[0, 0, 0]]), shape: torch.Size([2, 3]), dtype: torch.int64
------------------------------------------------------------
t2: tensor([[0, 0, 0],
[0, 0, 0]]), shape: torch.Size([2, 3]), dtype: torch.int64
# 3.场景1： 演示 torch.full 和 torch.full_like 创建固定值张量
t1 = torch.full((2,3),0)
print(f"t1: {t1}, shape: {t1.shape}, dtype: {t1.dtype}")
print('-'*60)
# 演示 full_like
t2 = torch.full_like(t1,fill_value=6)
print(f"t2: {t2}, shape: {t2.shape}, dtype: {t2.dtype}")
t1: tensor([[0, 0, 0],
[0, 0, 0]]), shape: torch.Size([2, 3]), dtype: torch.int64
------------------------------------------------------------
t2: tensor([[6, 6, 6],
[6, 6, 6]]), shape: torch.Size([2, 3]), dtype: torch.int64
day01_04_张量元素的类型转换
- .to()
- .type()
- .half/double/float/short/int/long()
"""
案例：
演示 张量元素的类型转换
涉及到的API：
data.to(torch.float32)
data.type(torch.DoubleTensor)
data.half/double/float/short/int/long()
需要掌握的：
data.to(torch.float32)
data.type(torch.DoubleTensor)
"""
# 导包
import torch
# 1.场景1：创建指定元素类型的张量
# 参数1：数据；参数2：元素类型dtype
t1 = torch.tensor([[1,2,3],[4,5,6]], dtype=torch.float32)
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型: {t1.dtype}")
print('-'*60)
# 2.场景2：先创建张量，再转换它的元素类型
# 参数1：数据
t2 = torch.tensor([[1,2,3],[4,5,6]])
print(f"t2: {t2}, shape: {t2.shape}, dtype元素类型: {t2.dtype}")
print('-'*60)
# 2.1 演示 .to 转换元素类型
t3 = t2.to(torch.float32)
print(f"t3: {t3}, shape: {t3.shape}, dtype元素类型: {t3.dtype}, device: {t3.device}")
t3 = t2.to(device="cuda")
print(f"t3: {t3}, shape: {t3.shape}, dtype元素类型: {t3.dtype}, device: {t3.device}")
print('-'*60)
# 2.2 演示 .type 转换元素类型
t3 = t2.type(torch.float32)
print(f"t3: {t3}, shape: {t3.shape}, dtype元素类型: {t3.dtype}")
# 2.3 演示data.half/double/float/short/int/long()
t3 = t2.double()
print(f"t3: {t3}, shape: {t3.shape}, dtype元素类型: {t3.dtype}")
t1: tensor([[1., 2., 3.],
[4., 5., 6.]]), shape: torch.Size([2, 3]), dtype元素类型: torch.float32
------------------------------------------------------------
t2: tensor([[1, 2, 3],
[4, 5, 6]]), shape: torch.Size([2, 3]), dtype元素类型: torch.int64
------------------------------------------------------------
t3: tensor([[1., 2., 3.],
[4., 5., 6.]]), shape: torch.Size([2, 3]), dtype元素类型: torch.float32, device: cpu
t3: tensor([[1, 2, 3],
[4, 5, 6]], device='cuda:0'), shape: torch.Size([2, 3]), dtype元素类型: torch.int64, device: cuda:0
------------------------------------------------------------
t3: tensor([[1., 2., 3.],
[4., 5., 6.]]), shape: torch.Size([2, 3]), dtype元素类型: torch.float32
t3: tensor([[1., 2., 3.],
[4., 5., 6.]], dtype=torch.float64), shape: torch.Size([2, 3]), dtype元素类型: torch.float64
- .to()
# 同时指定device和dtype
t1_gpu = t1.to(device=device, dtype=torch.float32)
print(t1_gpu)
tensor([[1., 2., 3.],
[4., 5., 6.]], device='cuda:0')
day01_05_张量和numpy之间相互转换
- 张量 转 numpy数组对象：张量对象.numpy()
- numpy数组对象 转 张量：torch.tensor(numpy数组对象)
- 标量张量 转 python数值：张量对象.item()
"""
案例：
演示 张量 和 numpy 之间的相互转换，以及 标量张量 转换为 python数值
涉及到的API:
张量 转 numpy数组对象：
张量对象.numpy() 共享内存，浅拷贝
张量对象.numpy().copy() 不共享内存，深拷贝
numpy数组对象 转 张量：
torch.from_numpy(numpy数组对象) 共享内存，浅拷贝
torch.tensor(numpy数组对象) 不共享内存，深拷贝
标量张量 转 python数值：
张量对象.item()
需要掌握：
张量 转 numpy数组对象：张量对象.numpy() 共享内存，浅拷贝
numpy数组对象 转 张量：torch.tensor(numpy数组对象) 不共享内存，深拷贝
标量张量 转 python数值：张量对象.item()
"""
# 导包
import torch
# 1.定义函数，演示 张量 转 numpy数组对象
def demo01():
# 0.设置随机种子
torch.manual_seed(6)
# 1.创建一个张量，3*6
t1 = torch.randint(0,10,(3,6))
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
print('-'*60)
# 2.张量 -> numpy数组对象
t2 = t1.numpy() # 共享内存，浅拷贝
print(f"t2: {t2}, shape: {t2.shape}, dtype元素类型：{t2.dtype}")
print('-'*60)
# 演示 共享内存
t2[0,0] = 100
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
print(f"t2: {t2}, shape: {t2.shape}, dtype元素类型：{t2.dtype}")
print('-' * 60)
# 演示 不共享内存
t3 = t2.copy() # 不共享内存，深拷贝
print(f"t3: {t3}, shape: {t3.shape}, dtype元素类型：{t3.dtype}")
print('-' * 60)
t3[0,0] = 200
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
print(f"t3: {t3}, shape: {t3.shape}, dtype元素类型：{t3.dtype}")
# 测试
if __name__ == '__main__':
demo01()
t1: tensor([[0, 1, 3, 2, 4, 7],
[7, 6, 4, 8, 7, 9],
[5, 5, 7, 2, 3, 9]]), shape: torch.Size([3, 6]), dtype元素类型：torch.int64
------------------------------------------------------------
t2: [[0 1 3 2 4 7]
[7 6 4 8 7 9]
[5 5 7 2 3 9]], shape: (3, 6), dtype元素类型：int64
------------------------------------------------------------
t1: tensor([[100, 1, 3, 2, 4, 7],
[ 7, 6, 4, 8, 7, 9],
[ 5, 5, 7, 2, 3, 9]]), shape: torch.Size([3, 6]), dtype元素类型：torch.int64
t2: [[100 1 3 2 4 7]
[ 7 6 4 8 7 9]
[ 5 5 7 2 3 9]], shape: (3, 6), dtype元素类型：int64
------------------------------------------------------------
t3: [[100 1 3 2 4 7]
[ 7 6 4 8 7 9]
[ 5 5 7 2 3 9]], shape: (3, 6), dtype元素类型：int64
------------------------------------------------------------
t1: tensor([[100, 1, 3, 2, 4, 7],
[ 7, 6, 4, 8, 7, 9],
[ 5, 5, 7, 2, 3, 9]]), shape: torch.Size([3, 6]), dtype元素类型：torch.int64
t3: [[200 1 3 2 4 7]
[ 7 6 4 8 7 9]
[ 5 5 7 2 3 9]], shape: (3, 6), dtype元素类型：int64
"""
案例：
演示 张量 和 numpy 之间的相互转换，以及 标量张量 转换为 python数值
涉及到的API:
张量 转 numpy数组对象：
张量对象.numpy() 共享内存，浅拷贝
张量对象.numpy().copy() 不共享内存，深拷贝
numpy数组对象 转 张量：
torch.from_numpy(numpy数组对象) 共享内存，浅拷贝
torch.tensor(numpy数组对象) 不共享内存，深拷贝
标量张量 转 python数值：
张量对象.item()
需要掌握：
张量 转 numpy数组对象：张量对象.numpy() 共享内存，浅拷贝
numpy数组对象 转 张量：torch.tensor(numpy数组对象) 不共享内存，深拷贝
标量张量 转 python数值：张量对象.item()
"""
# 导包
import torch
# 2.定义函数，演示 numpy数组对象 转 张量
def demo02():
# 0.设置随机种子
torch.manual_seed(6)
# 1.创建一个张量，3*6
t1 = torch.randint(0,10,(3,6))
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
print('-'*60)
# 2.张量 -> numpy数组对象
t2 = t1.numpy() # 共享内存，浅拷贝
print(f"t2: {t2}, shape: {t2.shape}, dtype元素类型：{t2.dtype}")
print('-'*60)
# 3.numpy数值对象 -> 张量
# t3 = torch.from_numpy(t2) # 共享内存，浅拷贝
t3 = torch.tensor(t2) # 不共享内存，深拷贝
print(f"t3: {t3}, shape: {t3.shape}, dtype元素类型：{t3.dtype}")
print('-' * 60)
# 演示 共享内存，浅拷贝
t3[0,0] = 100
print(f"t2: {t2}, shape: {t2.shape}, dtype元素类型：{t2.dtype}")
print(f"t3: {t3}, shape: {t3.shape}, dtype元素类型：{t3.dtype}")
# 演示 .item(), 转换标量张量 为 python数值
a = torch.tensor([6])
print(f"a: {a}, type: {type(a)}")
b = a.item()
print(f"b: {b}, type: {type(b)}")
# 测试
if __name__ == '__main__':
demo02()
t1: tensor([[0, 1, 3, 2, 4, 7],
[7, 6, 4, 8, 7, 9],
[5, 5, 7, 2, 3, 9]]), shape: torch.Size([3, 6]), dtype元素类型：torch.int64
------------------------------------------------------------
t2: [[0 1 3 2 4 7]
[7 6 4 8 7 9]
[5 5 7 2 3 9]], shape: (3, 6), dtype元素类型：int64
------------------------------------------------------------
t3: tensor([[0, 1, 3, 2, 4, 7],
[7, 6, 4, 8, 7, 9],
[5, 5, 7, 2, 3, 9]]), shape: torch.Size([3, 6]), dtype元素类型：torch.int64
------------------------------------------------------------
t2: [[0 1 3 2 4 7]
[7 6 4 8 7 9]
[5 5 7 2 3 9]], shape: (3, 6), dtype元素类型：int64
t3: tensor([[100, 1, 3, 2, 4, 7],
[ 7, 6, 4, 8, 7, 9],
[ 5, 5, 7, 2, 3, 9]]), shape: torch.Size([3, 6]), dtype元素类型：torch.int64
a: tensor([6]), type: <class 'torch.Tensor'>
b: 6, type: <class 'int'>
day01_06.张量的基本运算
"""
案例：
演示 张量的基本运算，也就是 + - * /
涉及到的API：
add(), sub(), mul(), div(), neg() # 加减乘除，取负
add_(), sub_(), mul_(), div_(), neg_() # 功能同上，但会修改原有数据, 功能类似pandas中的inplace=True
需要掌握：
1.+, -, *, /, - # 加减乘除，取负
2.张量 和 标量 的运算: 就是 张量中的每个元素 和 这个标量 进行运算
"""
# 导包
import torch
# 1.创建一个张量，2*3
t1 = torch.tensor([[1,2,3],[4,5,6]])
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型: {t1.dtype}")
# 2.演示 加法，add()
t2 = t1.add(2)
t1.add_(2)
t1=t1.add(2)
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型: {t1.dtype}")
# 需要掌握 +
t2 = t1 + 2
print(f"t1 + 2: {t2}, shape: {t2.shape}, dtype元素类型: {t2.dtype}")
# 3.演示 减法，sub()
t2 = t1.sub(2)
print(f"t1 - 2: {t2}, shape: {t2.shape}, dtype元素类型: {t2.dtype}")
t1.sub_(2)
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型: {t1.dtype}")
t2 = t1 + (- 2)
print(f"t2: {t2}, shape: {t2.shape}, dtype元素类型: {t2.dtype}")
# 4.演示 取负
t2 = -t1
print(f"t2: {t2}, shape: {t2.shape}, dtype元素类型: {t2.dtype}")
t1: tensor([[1, 2, 3],
[4, 5, 6]]), shape: torch.Size([2, 3]), dtype元素类型: torch.int64
t1: tensor([[ 5, 6, 7],
[ 8, 9, 10]]), shape: torch.Size([2, 3]), dtype元素类型: torch.int64
t1 + 2: tensor([[ 7, 8, 9],
[10, 11, 12]]), shape: torch.Size([2, 3]), dtype元素类型: torch.int64
t1 - 2: tensor([[3, 4, 5],
[6, 7, 8]]), shape: torch.Size([2, 3]), dtype元素类型: torch.int64
t1: tensor([[3, 4, 5],
[6, 7, 8]]), shape: torch.Size([2, 3]), dtype元素类型: torch.int64
t2: tensor([[1, 2, 3],
[4, 5, 6]]), shape: torch.Size([2, 3]), dtype元素类型: torch.int64
t2: tensor([[-3, -4, -5],
[-6, -7, -8]]), shape: torch.Size([2, 3]), dtype元素类型: torch.int64
常见面试题
1. 深度学习与传统机器学习的主要区别
- 特征工程：
- 传统机器学习依赖人来做特征工程（feature engineering），需要特定的领域知识。
- 深度学习可以自动从原始数据中提取特征，无需人参与。
- 多层非线性网络：
- 传统机器学习模型相对简单，如线性回归、SVM等。
- 深度学习模型通常由多层非线性神经网络组成，参数量大，表达能力更强。
- 数据需求：
- 传统机器学习在小数据集上也能表现较好。
- 深度学习通常需要大量数据才能发挥优势。
- 计算资源：
- 深度学习需要更多计算资源（GPU/TPU）。
- 传统机器学习对硬件要求较低，通常CPU就可以。
- 可解释性差：
- 深度学习模型类似“黑箱”，难以解释模型内部的具体决策过程。这对某些应用场景来说是一个挑战。
- 传统机器学习：模型结构更简单，可解释性更强。
2. 常见的深度学习模型及应用场景
- 全连接神经网络（DNN / MLP）
- 应用：适用于结构化数据（如表格数据）、简单分类与回归任务，比如商品销量预测、房价预测等。
- 卷积神经网络（CNN）
- 应用：处理图像与视觉任务，比如图像分类、目标检测、图像分割、人脸识别等。
- 循环神经网络（RNN）
- 应用：顺序数据相关的任务，比如自然语言处理（NLP）、语音识别、时间序列预测。
- Transformer
- 应用：当前主流模型，覆盖文本与图像等多领域，比如机器翻译、文本生成、对话系统、多模态任务。
- Diffusion（扩散模型）
- 应用：当前图像/视频生成的主导架构（如 Stable Diffusion、Sora）。
3. 数学中的矩阵是深度学习中的几维张量？
- 数学中的矩阵对应深度学习中的 2维张量。
4. 数组 [3, 4] 的 L2 范数
- L2 范数（欧几里得范数）计算公式为平方和再开根号：
$
|[3,4]|_2 = \sqrt{3^2 + 4^2} = \sqrt{9 + 16} = \sqrt{25} = 5
$
代码作业
- 1.使用torch.tensor()、torch.arange()和torch.randn()分别创建一个形状为(3, 4)的张量，并打印。
- 2.创建一个形状为(5,10)的全0张量 和 全1张量，并打印。
- 3.使用torch.randint()创建一个形状为(3, 4)的随机整数张量,并将元素类型转换为torch.float32，并打印。
- 4.使用torch.rand()创建一个形状为(3, 4)的张量，然后转换为numpy数组，然后更改其中的一个元素为42，然后再转换为张量，并打印。
- 5.创建一个标量张量，并转换为 python数值，并打印。
扩展-常用国内 PyPI 镜像
- 阿里云 https://mirrors.aliyun.com/pypi/simple/
- 腾讯云 https://mirrors.cloud.tencent.com/pypi/simple/
- 网易云 https://mirrors.163.com/pypi/simple/
- 清华大学 https://pypi.tuna.tsinghua.edu.cn/simple/
- 中国科技大学 https://pypi.mirrors.ustc.edu.cn/simple/
- 豆瓣 https://pypi.doubanio.com/simple/
使用方法：
- pip install 包名 -i 镜像地址
比如：pip install numpy -i https://mirrors.aliyun.com/pypi/simple/
💻 代码
Day01 代码：张量创建与类型转换
4 个文件📄 01_张量的基本创建方式.py 3.6KB
""" 演示 张量的创建的基本方式 张量 pytorch中存储数据的一种容器，元素必须是数值，元素类型必须相同 设计到的API： torch.tensor 根据指定数据创建张量 torch.Tensor 根据形状创建张量, 也可用来创建指定数据的张量 torch.IntTensor、torch.FloatTensor、torch.DoubleTensor 创建指定类型的张量 注意： torch.Tensor比torch.tensor多一个功能，可以指定形状来创建张量 要掌握的： torch.tensor(数据,数据类型torch.float32) """ # 导包 import torch # 1.定义函数，演示torch.tensor创建张量 def demo01(): # 1.创建0D/维张量 t1 = torch.tensor(1,dtype=torch.float32) print(f"0D张量：{t1}，维度：{t1.ndim}, shape: {t1.shape}, type: {type(t1)}, dtype: {t1.dtype}") # 2.创建1D/维张量 t2 = torch.tensor([1.,]) print(f"1D张量：{t2}，维度：{t2.ndim}, shape: {t2.shape}, type: {type(t2)}, dtype: {t2.dtype}") # 3.创建2D/维张量 t3 = torch.tensor([[1,2],[3,4]]) print(f"2D张量：{t3}，维度：{t3.ndim}, shape: {t3.shape}, type: {type(t3)}, dtype: {t3.dtype}") # 4.创建3D/维张量 t4 = torch.tensor([[[1,2],[3,4]],[[5,6],[7,8.]]]) print(f"3D张量：{t4}，维度：{t4.ndim}, shape: {t4.shape}, type: {type(t4)}, dtype: {t4.dtype}") # 2.定义函数，演示torch.Tensor创建张量 def demo02(): # 1.创建0维张量/0D张量，也就是标量 t1 = torch.Tensor(1) print(f"0D-t1:{t1}, shape: {t1.shape}, type: {type(t1)}, dtype: {t1.dtype}") # 2.创建1维张量/1D张量 t2 = torch.Tensor([1,2,3]) print(f"1D-t2:{t2}, shape: {t2.shape}, type: {type(t2)}, dtype: {t2.dtype}") # 3.创建2维张量/2D张量 t3 = torch.Tensor([[1,2,3],[4,5,6]]) # 形状(2,3) print(f"2D-t3:{t3}, shape: {t3.shape}, type: {type(t3)}, dtype: {t3.dtype}") # 4.创建3维张量/3D张量 t4 = torch.Tensor([[[1,2,3],[4,5,6]]]) # 形状(1,2,3) print(f"3D-t4:{t4}, shape: {t4.shape}, type: {type(t4)}, dtype: {t4.dtype}") # 5.创建指定形状的张量，(3,4) t5 = torch.Tensor(3,4) print(f"指定形状-t5:{t5}, shape: {t5.shape}") # 3.定义函数，演示torch.IntTensor(Int32)、torch.FloatTensor(Float32)、torch.DoubleTensor(Float64)创建张量 def demo03(): # 1.创建0维张量/0D张量，也就是标量 t1 = torch.IntTensor(1) print(f"0D-t1:{t1}, shape: {t1.shape}, type: {type(t1)}, dtype: {t1.dtype}") # 2.创建1维张量/1D张量 t2 = torch.IntTensor([1,2,3]) print(f"1D-t2:{t2}, shape: {t2.shape}, type: {type(t2)}, dtype: {t2.dtype}") # 3.创建2维张量/2D张量 t3 = torch.FloatTensor([[1,2,3],[4,5,6]]) # 形状(2,3) print(f"2D-t3:{t3}, shape: {t3.shape}, type: {type(t3)}, dtype: {t3.dtype}") # 4.创建3维张量/3D张量 t4 = torch.DoubleTensor([[[1,2,3],[4,5,6]]]) # 形状(1,2,3) print(f"3D-t4:{t4}, shape: {t4.shape}, type: {type(t4)}, dtype: {t4.dtype}") # 4.定义函数，演示张量的属性 # type:对象类型，shape:形状，dtype:数据类型，device:设备，requires_grad:是否开启梯度计算 def demo04(): # 1.创建一个张量，(2,3) # t1 = torch.tensor([[1,2,3],[4,5,6]],dtype=torch.float32,device="cuda",requires_grad=True) t1 = torch.tensor([[1,2,3],[4,5,6]], device="cuda") print(f"t1:{t1}, type: {type(t1)}, shape: {t1.shape}, dtype: {t1.dtype}, device: {t1.device}, requires_grad: {t1.requires_grad}") # dtype: int32 # device:cpu # requires_grad: False ... # 测试 if __name__ == '__main__': demo01() # demo02() # demo03() # demo04()
📄 02_创建线性和随机张量.py 1.6KB
""" 演示 创建 线性 和 随机张量 设计到的API: torch.arange() 和 torch.linspace() 创建线性张量 torch.manual_seed() 设置随机种子 torch.rand/randn() 创建随机浮点类型张量 torch.randint(low, high, size=()) 创建随机整数类型张量 需要掌握的： torch.arange() torch.linspace() torch.manual_seed() torch.randint(low, high, size=()) """ # 导包 import torch # 1.定义函数，使用torch.arange() 和 torch.linspace()创建线性张量 def demo01(): # 1.使用torch.arange()创建一个从0到9的线性张量 t1 = torch.arange(0,10) print(f"t1:{t1}, shape: {t1.shape}, dtype: {t1.dtype}") # 2.使用torch.linspace()创建一个从0到9的线性张量 t2 = torch.linspace(0,9,10) print(f"t2:{t2}, shape: {t2.shape}, dtype: {t2.dtype}") # 2.定义函数，演示 创建随机张量 def demo02(): # 0.设置随机种子 # torch.random.manual_seed(28) torch.manual_seed(28) # 1.使用torch.rand创建一个随机张量，(3,5) t1 = torch.rand(3,5) print(f"t1:{t1}, shape: {t1.shape}, dtype: {t1.dtype}") # (3,5) # 2.使用torch.randn创建一个随机张量，(3,5) t2 = torch.randn(3,5) print(f"t2:{t2}, shape: {t2.shape}, dtype: {t2.dtype}") # 3.使用torch.randint创建一个随机整数张量，(3,5) t3 = torch.randint(0,10,(6,8)) print(f"t3:{t3}, shape: {t3.shape}, dtype: {t3.dtype}") # 4.打印当前随机种子 print(f"当前随机种子:{torch.initial_seed()}") # 测试 if __name__ == '__main__': # demo01() demo02()
📄 03_创建全0_1_指定值张量.py 1.9KB
""" 演示 创建 全0、全1、指定值 张量 涉及到的API: torch.zeros 和 torch.zeros_like 创建全0张量 torch.ones 和 torch.ones_like 创建全1张量 torch.full 和 torch.full_like 创建全为指定值张量 要掌握： torch.zeros 用处： 网络参数初始化，偏置参数初始化 """ # 导包 import torch # 1.定义函数，演示创建全0张量 def demo01(): # 1.使用torcn.zeros创建全0张量，(3,4) t1 = torch.zeros(3, 4) print(f"t1:{t1}, shape: {t1.shape}, dtype: {t1.dtype}") # 2.创建一个随机张量，(3,5) t2 = torch.randn(3, 5) print(f"t2:{t2}, shape: {t2.shape}, dtype: {t2.dtype}") # 3.使用torch.zeros_like创建相同形状的全0张量 t3 = torch.zeros_like(t2) # (3,5) print(f"t3:{t3}, shape: {t3.shape}, dtype: {t3.dtype}") # 2.定义函数，演示创建全1张量 def demo02(): # 1.使用torcn.ones创建全1张量，(3,4) t1 = torch.ones(3, 4) print(f"t1:{t1}, shape: {t1.shape}, dtype: {t1.dtype}") # 2.创建一个随机张量，(3,5) t2 = torch.randn(3, 5) print(f"t2:{t2}, shape: {t2.shape}, dtype: {t2.dtype}") # 3.使用torch.ones_like创建相同形状的全1张量 t3 = torch.ones_like(t2) # (3,5) print(f"t3:{t3}, shape: {t3.shape}, dtype: {t3.dtype}") # 3.定义函数，演示创建指定值张量 def demo03(): # 1.使用torcn.full创建指定值张量，(3,4) t1 = torch.full((3, 4), 3.14) print(f"t1:{t1}, shape: {t1.shape}, dtype: {t1.dtype}") # 2.创建一个随机张量，(3,5) t2 = torch.randn(3, 5) print(f"t2:{t2}, shape: {t2.shape}, dtype: {t2.dtype}") # 3.使用torch.full_like创建相同形状的指定值张量 t3 = torch.full_like(t2, 10) # (3,5) print(f"t3:{t3}, shape: {t3.shape}, dtype: {t3.dtype}") # 测试 if __name__ == '__main__': demo01() demo02() demo03()
📄 04_张量元素的类型转换.py 1.3KB
""" 演示 张量元素类型的转换 设计到的API: data.to(torch.float32) data.type(torch.DoubleTensor) data.half/double/float/short/int/long() 需要掌握： data.to(dtype,device) """ # 导包 import torch # 设置设备 device = torch.device("cuda" if torch.cuda.is_available() else "cpu") print(f"device:{device}") # 1.定义函数，演示 张量的元素类型转换 def demo01(): # 1.创建张量时，指定元素类型 t1 = torch.tensor([[1,2,3],[4,5,6]], dtype=torch.float32) print(f"t1:{t1}, shape: {t1.shape}, dtype: {t1.dtype}, device: {t1.device}") # 2.使用data.to实现张量元素类型转换 t2 = t1.to(dtype=torch.float64) print(f"t2:{t2}, shape: {t2.shape}, dtype: {t2.dtype}, device: {t2.device}") t2 = t2.to(device) print(f"t2:{t2}, shape: {t2.shape}, dtype: {t2.dtype}, device: {t2.device}") # 3.使用data.type实现张量元素类型转换 t3 = t1.type(dtype=torch.float64) print(f"t3:{t3}, shape: {t3.shape}, dtype: {t3.dtype}, device: {t3.device}") # 4.使用data.double实现张量元素类型转换 t4 = t1.double() print(f"t4:{t4}, shape: {t4.shape}, dtype: {t4.dtype}, device: {t4.device}") # 测试 if __name__ == '__main__': demo01()
📎 资料
每日一图
- 📎 day01.png87.4KB
ℹ️ 离线版不含附件原文件，需要请到在线网站对应课时下载。
第2节 · PyTorch 框架使用
🎯 掌握张量运算、矩阵乘法、索引、形状变换与拼接。
📖 笔记
Day02 笔记：PyTorch 框架使用
Day 02 - PyTorch框架使用
学习目标
- 掌握张量的基本运算
- 掌握张量的按元素相乘和矩阵乘法
- 掌握张量的常用运算函数
- 掌握张量的索引操作
- 掌握张量的形状操作
- 掌握张量的拼接
- 实现自动微分模块入门案例
- 了解梯度下降法求最小值案例
# 导包
import torch
import numpy as np
import matplotlib.pyplot as plt
# 设置中文字体
plt.rcParams['font.sans-serif'] = ['SimHei'] # 微软雅黑
plt.rcParams['axes.unicode_minus'] = False # 解决负号显示问题
# 设置设备
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
day02_01_张量的按元素相乘和矩阵乘法
- 按元素相乘(Hadamard)： A * B，相同形状的张量对应位置的元素相乘。要求两个张量的形状相同。
- 矩阵乘法: A @ B, 第一个矩阵A的行与第二个矩阵B的列的对应元素相乘，比如矩阵A.shape: (n, m)，矩阵B.shape: (m, p), 两个矩阵乘法运算 shape 为: (n, p)。对于高维矩阵，要求A.shape[-1] = B.shape[-2]
两个矩阵的按元素乘法称为Hadamard积（Hadamard product）（数学符号\(\odot\)）
矩阵\(\mathbf{A} \in \mathbb{R}^{m \times n}\)(第\(i\)行和第\(j\)列的元素是\(a_{ij}\))
和\(\mathbf{B} \in \mathbb{R}^{m \times n}\)(第\(i\)行和第\(j\)列的元素是\(b_{ij}\))的Hadamard积为：
$$
\mathbf{A} \odot \mathbf{B} =
\begin{bmatrix}
a_{11} b_{11} & a_{12} b_{12} & \dots & a_{1n} b_{1n} \
a_{21} b_{21} & a_{22} b_{22} & \dots & a_{2n} b_{2n} \
\vdots & \vdots & \ddots & \vdots \
a_{m1} b_{m1} & a_{m2} b_{m2} & \dots & a_{mn} b_{mn}
\end{bmatrix}.
$$
假设两个 2D 张量 A 和 B，形状都是 (3,4)：
张量 A：
\[A =
\begin{bmatrix}
1 & 2 & 3 & 4 \\
5 & 6 & 7 & 8 \\
9 & 10 & 11 & 12
\end{bmatrix}\]
张量 B：
\[B =
\begin{bmatrix}
2 & 0 & 1 & 3 \\
1 & 2 & 0 & 4 \\
0 & 1 & 2 & 1
\end{bmatrix}\]
按元素相乘（矩阵形式，显示计算过程）
\[C = A * B =
\begin{bmatrix}
1*2 & 2*0 & 3*1 & 4*3 \\
5*1 & 6*2 & 7*0 & 8*4 \\
9*0 & 10*1 & 11*2 & 12*1
\end{bmatrix} =
\begin{bmatrix}
2 & 0 & 3 & 12 \\
5 & 12 & 0 & 32 \\
0 & 10 & 22 & 12
\end{bmatrix}\]
"""
案例：
演示 张量的按元素相乘和矩阵乘法
按元素相乘：
要求：两个张量的形状一样，A为(m,n)，B为(m,n)
运算：对应位置的元素直接相乘，得到一个新的张量，形状也一样
API:
t1 * t2
torch.mul(t1, t2)
t1.mul(t2)
注意：tensor.x, torch.x(tensor,)，比如torch.add(t1, t2)表示加法
矩阵乘法：
要求：第一个张量的列数 = 第二个张量的行数,
t1的形状是 (m, n), t2的形状是 (n, p)，结果张量的形状是 (m, p)
运算：第一个张量的行向量 和 第二个张量的列向量 做点积，也就是先相乘，再求和
API:
t1 @ t2
t1.matmul(t2)
torch.matmul(t1, t2)
t1.dot(t2) 只能用于一维张量的点积运算
torch.dot(t1, t2) 只能用于一维张量的点积运算
需要掌握的：
t1 * t2
t1 @ t2
"""
# 导包
import torch
# 0.设置随机种子
torch.random.manual_seed(6)
# 1.创建两个张量, 3*4
t1 = torch.randint(1, 10, (3,4))
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
t2 = torch.randint(1, 10, (3,4))
print(f"t2: {t2}, shape: {t2.shape}, dtype元素类型：{t2.dtype}")
print("-"*60)
# 2.按元素相乘, t1*t2
t3 = t1*t2
print(f"t1*t2: {t3}, shape: {t3.shape}, dtype元素类型：{t3.dtype}")
print("-"*60)
t1: tensor([[9, 7, 7, 3],
[8, 1, 3, 4],
[2, 1, 6, 8]]), shape: torch.Size([3, 4]), dtype元素类型：torch.int64
t2: tensor([[8, 6, 8, 6],
[4, 3, 2, 5],
[7, 4, 6, 9]]), shape: torch.Size([3, 4]), dtype元素类型：torch.int64
------------------------------------------------------------
t1*t2: tensor([[72, 42, 56, 18],
[32, 3, 6, 20],
[14, 4, 36, 72]]), shape: torch.Size([3, 4]), dtype元素类型：torch.int64
------------------------------------------------------------
矩阵乘法（matrix-matrix multiplication）
要求：A.shape[-1]=B.shape[-2]
有两个矩阵\(\mathbf{A} \in \mathbb{R}^{n \times k}\)和\(\mathbf{B} \in \mathbb{R}^{k \times m}\)：
\[\mathbf{A}=\begin{bmatrix}
a_{11} & a_{12} & \cdots & a_{1k} \\
a_{21} & a_{22} & \cdots & a_{2k} \\
\vdots & \vdots & \ddots & \vdots \\
a_{n1} & a_{n2} & \cdots & a_{nk} \\
\end{bmatrix},\quad
\mathbf{B}=\begin{bmatrix}
b_{11} & b_{12} & \cdots & b_{1m} \\
b_{21} & b_{22} & \cdots & b_{2m} \\
\vdots & \vdots & \ddots & \vdots \\
b_{k1} & b_{k2} & \cdots & b_{km} \\
\end{bmatrix}.\]
用行向量\(\mathbf{a}^\top_{i} \in \mathbb{R}^k\)表示矩阵\(\mathbf{A}\)的第\(i\)行，用列向量\(\mathbf{b}_{j} \in \mathbb{R}^k\)作为矩阵\(\mathbf{B}\)的第\(j\)列。
$$\mathbf{A}=
\begin{bmatrix}
\mathbf{a}^\top_{1} \
\mathbf{a}^\top_{2} \
\vdots \
\mathbf{a}^\top_n \
\end{bmatrix},
\quad \mathbf{B}=\begin{bmatrix}
\mathbf{b}{1} & \mathbf{b} \} & \cdots & \mathbf{b}_{m
\end{bmatrix}.
$$
要生成矩阵积\(\mathbf{C} = \mathbf{A}\mathbf{B}\)，将每个元素\(c_{ij}\)计算为点积\(\mathbf{a}^\top_i \mathbf{b}_j\):
\[\mathbf{C} = \mathbf{AB} = \begin{bmatrix}
\mathbf{a}^\top_{1} \\
\mathbf{a}^\top_{2} \\
\vdots \\
\mathbf{a}^\top_n \\
\end{bmatrix}
\begin{bmatrix}
\mathbf{b}_{1} & \mathbf{b}_{2} & \cdots & \mathbf{b}_{m} \\
\end{bmatrix}
= \begin{bmatrix}
\mathbf{a}^\top_{1} \mathbf{b}_1 & \mathbf{a}^\top_{1}\mathbf{b}_2& \cdots & \mathbf{a}^\top_{1} \mathbf{b}_m \\
\mathbf{a}^\top_{2}\mathbf{b}_1 & \mathbf{a}^\top_{2} \mathbf{b}_2 & \cdots & \mathbf{a}^\top_{2} \mathbf{b}_m \\
\vdots & \vdots & \ddots &\vdots\\
\mathbf{a}^\top_{n} \mathbf{b}_1 & \mathbf{a}^\top_{n}\mathbf{b}_2& \cdots& \mathbf{a}^\top_{n} \mathbf{b}_m
\end{bmatrix}.
\]
假设两个 2D 张量 A 和 B，形状分别为 (3,4) 和 (4,3)：
张量 A：
\[A =
\begin{bmatrix}
1 & 2 & 3 & 4 \\
5 & 6 & 7 & 8 \\
9 & 10 & 11 & 12
\end{bmatrix}
\]
张量 B：
\[
B =
\begin{bmatrix}
1 & 0 & 2 \\
0 & 1 & 2 \\
1 & 0 & 1 \\
0 & 1 & 0
\end{bmatrix}
\]
矩阵乘法计算过程
矩阵乘法 C = A @ B，C 的形状为 (3,3)，每个元素 C[i,j] 为 A 的第 i 行与 B 的第 j 列对应元素乘积之和：
\[
C = A @ B=
\begin{bmatrix}
1*1 + 2*0 + 3*1 + 4*0 & * & * \\
5*1 + 6*0 + 7*1 + 8*0 & * & * \\
9*1 + 10*0 + 11*1 + 12*0 & * & *
\end{bmatrix} =
\begin{bmatrix}
4 & 6 & 7 \\
12 & 14 & 25 \\
20 & 22 & 43
\end{bmatrix}
\]
对应计算结果
\[
C =
\begin{bmatrix}
4 & 6 & 7 \\
12 & 14 & 25 \\
20 & 22 & 43
\end{bmatrix}
\]
说明：
- 矩阵乘法的每个元素是 行向量与列向量的点积。
- A 的列数必须等于 B 的行数，结果矩阵 C 的形状为 (A行数, B列数)。
# 3.创建两个张量，做矩阵乘法, (3,4) @ (4,3) --> (3,3)
t1 = torch.randint(1, 10, (3,4))
t2 = torch.randint(1, 10, (4,3))
# 演示 矩阵乘法
t3 = t1 @ t2
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
print(f"t2: {t2}, shape: {t2.shape}, dtype元素类型：{t2.dtype}")
print(f"t1 @ t2: {t3}, shape: {t3.shape}, dtype元素类型：{t3.dtype}")
t1: tensor([[8, 5, 6, 7],
[6, 1, 1, 8],
[5, 4, 9, 3]]), shape: torch.Size([3, 4]), dtype元素类型：torch.int64
t2: tensor([[9, 9, 6],
[3, 8, 6],
[5, 7, 5],
[3, 2, 7]]), shape: torch.Size([4, 3]), dtype元素类型：torch.int64
t1 @ t2: tensor([[138, 168, 157],
[ 86, 85, 103],
[111, 146, 120]]), shape: torch.Size([3, 3]), dtype元素类型：torch.int64
# 演示3维张量的矩阵乘法
import torch
# 创建张量
t1 = torch.arange(30).reshape(3,2,5)
t2 = torch.arange(30).reshape(3,5,2)
# 矩阵乘法
t3 = torch.matmul(t1, t2)
print(f"t1: {t1}")
print(f"t2: {t2}")
print(f"t3=t1@t2: {t3.shape}")
t1: tensor([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9]],
[[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]],
[[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29]]])
t2: tensor([[[ 0, 1],
[ 2, 3],
[ 4, 5],
[ 6, 7],
[ 8, 9]],
[[10, 11],
[12, 13],
[14, 15],
[16, 17],
[18, 19]],
[[20, 21],
[22, 23],
[24, 25],
[26, 27],
[28, 29]]])
t3=t1@t2: torch.Size([3, 2, 2])
点积（Dot Product）
给定两个向量\(\mathbf{x},\mathbf{y}\in\mathbb{R}^d\)，
点积（dot product）\(\mathbf{x}^\top\mathbf{y}\)
（或\(\langle\mathbf{x},\mathbf{y}\rangle\)）
是相同位置的按元素乘积的和：\(\mathbf{x}^\top \mathbf{y} = \sum_{i=1}^{d} x_i y_i\)。
# 4. 演示 点积
# 把t1,t2转为一维张量，做点积运算
t1 = t1.reshape(-1)
t2 = t2.reshape(-1)
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
print(f"t2: {t2}, shape: {t2.shape}, dtype元素类型：{t2.dtype}")
t3 = t1.dot(t2)
print(f"t1.dot(t2): {t3}, shape: {t3.shape}, dtype元素类型：{t3.dtype}")
t1: tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]), shape: torch.Size([30]), dtype元素类型：torch.int64
t2: tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]), shape: torch.Size([30]), dtype元素类型：torch.int64
t1.dot(t2): 8555, shape: torch.Size([]), dtype元素类型：torch.int64
day02_02_张量的常用运算函数
- sum(), mean(), max(), min()
"""
案例：
演示 张量的常用运算函数
涉及到的API：
sum(), mean(), max(), min() -> 有dim参数，可以指定维度/轴进行操作，会改变张量的形状
pow()/**, sqrt(), exp(), log(), log2(), log10() -> 无dim参数，对张量中的所有元素进行运算，不会改变张量的形状
需要掌握的：
sum(), mean(), max(), min(), **
"""
# 导包
import torch
# 0.设置随机种子
torch.manual_seed(6)
# 1.创建一个张量, 3*4*5
t1 = torch.randint(1, 10, (3,4,5))
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
print("-"*60)
# 2.演示 sum
print(f"t1.sum(): {t1.sum()}") # 标量
# 指定dim=0
print(f"t1.sum(dim=0): {t1.sum(dim=0)}, shape: {t1.sum(dim=0).shape}") # 张量
# 指定dim=1
print(f"t1.sum(dim=1): {t1.sum(dim=1)}, shape: {t1.sum(dim=1).shape}") # 张量
print("-"*60)
# 指定dim=2
print(f"t1.sum(dim=2): {t1.sum(dim=2)}, shape: {t1.sum(dim=2).shape}") # 张量
t1: tensor([[[9, 7, 7, 3, 8],
[1, 3, 4, 2, 1],
[6, 8, 8, 6, 8],
[6, 4, 3, 2, 5]],
[[7, 4, 6, 9, 8],
[5, 6, 7, 6, 1],
[1, 8, 5, 4, 9],
[3, 9, 9, 6, 3]],
[[8, 6, 5, 7, 5],
[3, 2, 7, 7, 8],
[2, 2, 3, 6, 4],
[7, 8, 8, 2, 5]]]), shape: torch.Size([3, 4, 5]), dtype元素类型：torch.int64
------------------------------------------------------------
t1.sum(): 322
t1.sum(dim=0): tensor([[24, 17, 18, 19, 21],
[ 9, 11, 18, 15, 10],
[ 9, 18, 16, 16, 21],
[16, 21, 20, 10, 13]]), shape: torch.Size([4, 5])
t1.sum(dim=1): tensor([[22, 22, 22, 13, 22],
[16, 27, 27, 25, 21],
[20, 18, 23, 22, 22]]), shape: torch.Size([3, 5])
------------------------------------------------------------
t1.sum(dim=2): tensor([[34, 11, 36, 20],
[34, 25, 27, 30],
[31, 27, 17, 30]]), shape: torch.Size([3, 4])
# 3.演示 mean
t1 = t1.to(torch.float32)
print(f"t1.mean(): {t1.mean()}") # 标量
# 指定dim=0
print(f"t1.mean(dim=0): {t1.mean(dim=0)}") # 张量
# 指定dim=1
print(f"t1.mean(dim=1): {t1.mean(dim=1)}") # 张量
print("-"*60)
# 4.演示 max
t1 = t1.to(torch.float32)
print(f"t1.max(): {t1.max()}") # 标量
# 指定dim=0
print(f"t1.max(dim=0): {t1.max(dim=0)}") # 张量
# 指定dim=1
print(f"t1.max(dim=1): {t1.max(dim=1)}") # 张量
print("-"*60)
# 5.演示 min
t1 = t1.to(torch.float32)
print(f"t1.min(): {t1.min()}") # 标量
# 指定dim=0
print(f"t1.min(dim=0): {t1.min(dim=0)}") # 张量
# 指定dim=1
print(f"t1.min(dim=1): {t1.min(dim=1)}") # 张量
# 6.演示 pow()/**
# 求2次方
print(f"t1.pow(2): {t1.pow(2)}") # 标量
# 求10次方
print(f"t1.pow(10): {t1.pow(10)}") # 标量
t1.mean(): 5.366666793823242
t1.mean(dim=0): tensor([[8.0000, 5.6667, 6.0000, 6.3333, 7.0000],
[3.0000, 3.6667, 6.0000, 5.0000, 3.3333],
[3.0000, 6.0000, 5.3333, 5.3333, 7.0000],
[5.3333, 7.0000, 6.6667, 3.3333, 4.3333]])
t1.mean(dim=1): tensor([[5.5000, 5.5000, 5.5000, 3.2500, 5.5000],
[4.0000, 6.7500, 6.7500, 6.2500, 5.2500],
[5.0000, 4.5000, 5.7500, 5.5000, 5.5000]])
------------------------------------------------------------
t1.max(): 9.0
t1.max(dim=0): torch.return_types.max(
values=tensor([[9., 7., 7., 9., 8.],
[5., 6., 7., 7., 8.],
[6., 8., 8., 6., 9.],
[7., 9., 9., 6., 5.]]),
indices=tensor([[0, 0, 0, 1, 0],
[1, 1, 1, 2, 2],
[0, 0, 0, 0, 1],
[2, 1, 1, 1, 0]]))
t1.max(dim=1): torch.return_types.max(
values=tensor([[9., 8., 8., 6., 8.],
[7., 9., 9., 9., 9.],
[8., 8., 8., 7., 8.]]),
indices=tensor([[0, 2, 2, 2, 0],
[0, 3, 3, 0, 2],
[0, 3, 3, 0, 1]]))
------------------------------------------------------------
t1.min(): 1.0
t1.min(dim=0): torch.return_types.min(
values=tensor([[7., 4., 5., 3., 5.],
[1., 2., 4., 2., 1.],
[1., 2., 3., 4., 4.],
[3., 4., 3., 2., 3.]]),
indices=tensor([[1, 1, 2, 0, 2],
[0, 2, 0, 0, 0],
[1, 2, 2, 1, 2],
[1, 0, 0, 0, 1]]))
t1.min(dim=1): torch.return_types.min(
values=tensor([[1., 3., 3., 2., 1.],
[1., 4., 5., 4., 1.],
[2., 2., 3., 2., 4.]]),
indices=tensor([[1, 1, 3, 1, 1],
[2, 0, 2, 2, 1],
[2, 1, 2, 3, 2]]))
t1.pow(2): tensor([[[81., 49., 49., 9., 64.],
[ 1., 9., 16., 4., 1.],
[36., 64., 64., 36., 64.],
[36., 16., 9., 4., 25.]],
[[49., 16., 36., 81., 64.],
[25., 36., 49., 36., 1.],
[ 1., 64., 25., 16., 81.],
[ 9., 81., 81., 36., 9.]],
[[64., 36., 25., 49., 25.],
[ 9., 4., 49., 49., 64.],
[ 4., 4., 9., 36., 16.],
[49., 64., 64., 4., 25.]]])
t1.pow(10): tensor([[[3.4868e+09, 2.8248e+08, 2.8248e+08, 5.9049e+04, 1.0737e+09],
[1.0000e+00, 5.9049e+04, 1.0486e+06, 1.0240e+03, 1.0000e+00],
[6.0466e+07, 1.0737e+09, 1.0737e+09, 6.0466e+07, 1.0737e+09],
[6.0466e+07, 1.0486e+06, 5.9049e+04, 1.0240e+03, 9.7656e+06]],
[[2.8248e+08, 1.0486e+06, 6.0466e+07, 3.4868e+09, 1.0737e+09],
[9.7656e+06, 6.0466e+07, 2.8248e+08, 6.0466e+07, 1.0000e+00],
[1.0000e+00, 1.0737e+09, 9.7656e+06, 1.0486e+06, 3.4868e+09],
[5.9049e+04, 3.4868e+09, 3.4868e+09, 6.0466e+07, 5.9049e+04]],
[[1.0737e+09, 6.0466e+07, 9.7656e+06, 2.8248e+08, 9.7656e+06],
[5.9049e+04, 1.0240e+03, 2.8248e+08, 2.8248e+08, 1.0737e+09],
[1.0240e+03, 1.0240e+03, 5.9049e+04, 6.0466e+07, 1.0486e+06],
[2.8248e+08, 1.0737e+09, 1.0737e+09, 1.0240e+03, 9.7656e+06]]])
# 5. 演示 pow, sqrt, exp, log, log2, log10
print(f"t1.pow(2): {t1.pow(2)}") # 每个元素的平方
print(f"torch.pow(t1, 2): {torch.pow(t1, 2)}") # 每个元素的平方
print(f"t1**2: {t1**2}") # 每个元素的平方
print(f"t1.pow(3): {t1.pow(3)}") # 每个元素的立方
print(f"torch.pow(t1, 3): {torch.pow(t1, 3)}") # 每个元素的立方
print(f"t1**3: {t1**3}")
print(f"t1.sqrt(): {t1.sqrt()}") # 每个元素的平方根
print(f"torch.sqrt(t1): {torch.sqrt(t1)}") # 每个元素的平方根
print(f"t1**0.5: {t1**0.5}") # 每个元素的平方根
print(f"t1**(1/3): {t1**(1/3)}") # 每个元素的立方根
print(f"t1.exp(): {t1.exp()}") # 每个元素的e指数
print(f"torch.exp(t1): {torch.exp(t1)}") # 每个元素的e指数
print(f"t1.log(): {t1.log()}") # 每个元素的自然对数
print(f"t1.log2(): {t1.log2()}") # 每个元素的以2为底的对数
print(f"t1.log10(): {t1.log10()}") # 每个元素的以10为底的对数
print(f"torch.log(t1): {torch.log(t1)}")
t1.pow(2): tensor([[[81., 49., 49., 9., 64.],
[ 1., 9., 16., 4., 1.],
[36., 64., 64., 36., 64.],
[36., 16., 9., 4., 25.]],
[[49., 16., 36., 81., 64.],
[25., 36., 49., 36., 1.],
[ 1., 64., 25., 16., 81.],
[ 9., 81., 81., 36., 9.]],
[[64., 36., 25., 49., 25.],
[ 9., 4., 49., 49., 64.],
[ 4., 4., 9., 36., 16.],
[49., 64., 64., 4., 25.]]])
torch.pow(t1, 2): tensor([[[81., 49., 49., 9., 64.],
[ 1., 9., 16., 4., 1.],
[36., 64., 64., 36., 64.],
[36., 16., 9., 4., 25.]],
[[49., 16., 36., 81., 64.],
[25., 36., 49., 36., 1.],
[ 1., 64., 25., 16., 81.],
[ 9., 81., 81., 36., 9.]],
[[64., 36., 25., 49., 25.],
[ 9., 4., 49., 49., 64.],
[ 4., 4., 9., 36., 16.],
[49., 64., 64., 4., 25.]]])
t1**2: tensor([[[81., 49., 49., 9., 64.],
[ 1., 9., 16., 4., 1.],
[36., 64., 64., 36., 64.],
[36., 16., 9., 4., 25.]],
[[49., 16., 36., 81., 64.],
[25., 36., 49., 36., 1.],
[ 1., 64., 25., 16., 81.],
[ 9., 81., 81., 36., 9.]],
[[64., 36., 25., 49., 25.],
[ 9., 4., 49., 49., 64.],
[ 4., 4., 9., 36., 16.],
[49., 64., 64., 4., 25.]]])
t1.pow(3): tensor([[[729., 343., 343., 27., 512.],
[ 1., 27., 64., 8., 1.],
[216., 512., 512., 216., 512.],
[216., 64., 27., 8., 125.]],
[[343., 64., 216., 729., 512.],
[125., 216., 343., 216., 1.],
[ 1., 512., 125., 64., 729.],
[ 27., 729., 729., 216., 27.]],
[[512., 216., 125., 343., 125.],
[ 27., 8., 343., 343., 512.],
[ 8., 8., 27., 216., 64.],
[343., 512., 512., 8., 125.]]])
torch.pow(t1, 3): tensor([[[729., 343., 343., 27., 512.],
[ 1., 27., 64., 8., 1.],
[216., 512., 512., 216., 512.],
[216., 64., 27., 8., 125.]],
[[343., 64., 216., 729., 512.],
[125., 216., 343., 216., 1.],
[ 1., 512., 125., 64., 729.],
[ 27., 729., 729., 216., 27.]],
[[512., 216., 125., 343., 125.],
[ 27., 8., 343., 343., 512.],
[ 8., 8., 27., 216., 64.],
[343., 512., 512., 8., 125.]]])
t1**3: tensor([[[729., 343., 343., 27., 512.],
[ 1., 27., 64., 8., 1.],
[216., 512., 512., 216., 512.],
[216., 64., 27., 8., 125.]],
[[343., 64., 216., 729., 512.],
[125., 216., 343., 216., 1.],
[ 1., 512., 125., 64., 729.],
[ 27., 729., 729., 216., 27.]],
[[512., 216., 125., 343., 125.],
[ 27., 8., 343., 343., 512.],
[ 8., 8., 27., 216., 64.],
[343., 512., 512., 8., 125.]]])
t1.sqrt(): tensor([[[3.0000, 2.6458, 2.6458, 1.7321, 2.8284],
[1.0000, 1.7321, 2.0000, 1.4142, 1.0000],
[2.4495, 2.8284, 2.8284, 2.4495, 2.8284],
[2.4495, 2.0000, 1.7321, 1.4142, 2.2361]],
[[2.6458, 2.0000, 2.4495, 3.0000, 2.8284],
[2.2361, 2.4495, 2.6458, 2.4495, 1.0000],
[1.0000, 2.8284, 2.2361, 2.0000, 3.0000],
[1.7321, 3.0000, 3.0000, 2.4495, 1.7321]],
[[2.8284, 2.4495, 2.2361, 2.6458, 2.2361],
[1.7321, 1.4142, 2.6458, 2.6458, 2.8284],
[1.4142, 1.4142, 1.7321, 2.4495, 2.0000],
[2.6458, 2.8284, 2.8284, 1.4142, 2.2361]]])
torch.sqrt(t1): tensor([[[3.0000, 2.6458, 2.6458, 1.7321, 2.8284],
[1.0000, 1.7321, 2.0000, 1.4142, 1.0000],
[2.4495, 2.8284, 2.8284, 2.4495, 2.8284],
[2.4495, 2.0000, 1.7321, 1.4142, 2.2361]],
[[2.6458, 2.0000, 2.4495, 3.0000, 2.8284],
[2.2361, 2.4495, 2.6458, 2.4495, 1.0000],
[1.0000, 2.8284, 2.2361, 2.0000, 3.0000],
[1.7321, 3.0000, 3.0000, 2.4495, 1.7321]],
[[2.8284, 2.4495, 2.2361, 2.6458, 2.2361],
[1.7321, 1.4142, 2.6458, 2.6458, 2.8284],
[1.4142, 1.4142, 1.7321, 2.4495, 2.0000],
[2.6458, 2.8284, 2.8284, 1.4142, 2.2361]]])
t1**0.5: tensor([[[3.0000, 2.6458, 2.6458, 1.7321, 2.8284],
[1.0000, 1.7321, 2.0000, 1.4142, 1.0000],
[2.4495, 2.8284, 2.8284, 2.4495, 2.8284],
[2.4495, 2.0000, 1.7321, 1.4142, 2.2361]],
[[2.6458, 2.0000, 2.4495, 3.0000, 2.8284],
[2.2361, 2.4495, 2.6458, 2.4495, 1.0000],
[1.0000, 2.8284, 2.2361, 2.0000, 3.0000],
[1.7321, 3.0000, 3.0000, 2.4495, 1.7321]],
[[2.8284, 2.4495, 2.2361, 2.6458, 2.2361],
[1.7321, 1.4142, 2.6458, 2.6458, 2.8284],
[1.4142, 1.4142, 1.7321, 2.4495, 2.0000],
[2.6458, 2.8284, 2.8284, 1.4142, 2.2361]]])
t1**(1/3): tensor([[[2.0801, 1.9129, 1.9129, 1.4422, 2.0000],
[1.0000, 1.4422, 1.5874, 1.2599, 1.0000],
[1.8171, 2.0000, 2.0000, 1.8171, 2.0000],
[1.8171, 1.5874, 1.4422, 1.2599, 1.7100]],
[[1.9129, 1.5874, 1.8171, 2.0801, 2.0000],
[1.7100, 1.8171, 1.9129, 1.8171, 1.0000],
[1.0000, 2.0000, 1.7100, 1.5874, 2.0801],
[1.4422, 2.0801, 2.0801, 1.8171, 1.4422]],
[[2.0000, 1.8171, 1.7100, 1.9129, 1.7100],
[1.4422, 1.2599, 1.9129, 1.9129, 2.0000],
[1.2599, 1.2599, 1.4422, 1.8171, 1.5874],
[1.9129, 2.0000, 2.0000, 1.2599, 1.7100]]])
t1.exp(): tensor([[[8.1031e+03, 1.0966e+03, 1.0966e+03, 2.0086e+01, 2.9810e+03],
[2.7183e+00, 2.0086e+01, 5.4598e+01, 7.3891e+00, 2.7183e+00],
[4.0343e+02, 2.9810e+03, 2.9810e+03, 4.0343e+02, 2.9810e+03],
[4.0343e+02, 5.4598e+01, 2.0086e+01, 7.3891e+00, 1.4841e+02]],
[[1.0966e+03, 5.4598e+01, 4.0343e+02, 8.1031e+03, 2.9810e+03],
[1.4841e+02, 4.0343e+02, 1.0966e+03, 4.0343e+02, 2.7183e+00],
[2.7183e+00, 2.9810e+03, 1.4841e+02, 5.4598e+01, 8.1031e+03],
[2.0086e+01, 8.1031e+03, 8.1031e+03, 4.0343e+02, 2.0086e+01]],
[[2.9810e+03, 4.0343e+02, 1.4841e+02, 1.0966e+03, 1.4841e+02],
[2.0086e+01, 7.3891e+00, 1.0966e+03, 1.0966e+03, 2.9810e+03],
[7.3891e+00, 7.3891e+00, 2.0086e+01, 4.0343e+02, 5.4598e+01],
[1.0966e+03, 2.9810e+03, 2.9810e+03, 7.3891e+00, 1.4841e+02]]])
torch.exp(t1): tensor([[[8.1031e+03, 1.0966e+03, 1.0966e+03, 2.0086e+01, 2.9810e+03],
[2.7183e+00, 2.0086e+01, 5.4598e+01, 7.3891e+00, 2.7183e+00],
[4.0343e+02, 2.9810e+03, 2.9810e+03, 4.0343e+02, 2.9810e+03],
[4.0343e+02, 5.4598e+01, 2.0086e+01, 7.3891e+00, 1.4841e+02]],
[[1.0966e+03, 5.4598e+01, 4.0343e+02, 8.1031e+03, 2.9810e+03],
[1.4841e+02, 4.0343e+02, 1.0966e+03, 4.0343e+02, 2.7183e+00],
[2.7183e+00, 2.9810e+03, 1.4841e+02, 5.4598e+01, 8.1031e+03],
[2.0086e+01, 8.1031e+03, 8.1031e+03, 4.0343e+02, 2.0086e+01]],
[[2.9810e+03, 4.0343e+02, 1.4841e+02, 1.0966e+03, 1.4841e+02],
[2.0086e+01, 7.3891e+00, 1.0966e+03, 1.0966e+03, 2.9810e+03],
[7.3891e+00, 7.3891e+00, 2.0086e+01, 4.0343e+02, 5.4598e+01],
[1.0966e+03, 2.9810e+03, 2.9810e+03, 7.3891e+00, 1.4841e+02]]])
t1.log(): tensor([[[2.1972, 1.9459, 1.9459, 1.0986, 2.0794],
[0.0000, 1.0986, 1.3863, 0.6931, 0.0000],
[1.7918, 2.0794, 2.0794, 1.7918, 2.0794],
[1.7918, 1.3863, 1.0986, 0.6931, 1.6094]],
[[1.9459, 1.3863, 1.7918, 2.1972, 2.0794],
[1.6094, 1.7918, 1.9459, 1.7918, 0.0000],
[0.0000, 2.0794, 1.6094, 1.3863, 2.1972],
[1.0986, 2.1972, 2.1972, 1.7918, 1.0986]],
[[2.0794, 1.7918, 1.6094, 1.9459, 1.6094],
[1.0986, 0.6931, 1.9459, 1.9459, 2.0794],
[0.6931, 0.6931, 1.0986, 1.7918, 1.3863],
[1.9459, 2.0794, 2.0794, 0.6931, 1.6094]]])
t1.log2(): tensor([[[3.1699, 2.8074, 2.8074, 1.5850, 3.0000],
[0.0000, 1.5850, 2.0000, 1.0000, 0.0000],
[2.5850, 3.0000, 3.0000, 2.5850, 3.0000],
[2.5850, 2.0000, 1.5850, 1.0000, 2.3219]],
[[2.8074, 2.0000, 2.5850, 3.1699, 3.0000],
[2.3219, 2.5850, 2.8074, 2.5850, 0.0000],
[0.0000, 3.0000, 2.3219, 2.0000, 3.1699],
[1.5850, 3.1699, 3.1699, 2.5850, 1.5850]],
[[3.0000, 2.5850, 2.3219, 2.8074, 2.3219],
[1.5850, 1.0000, 2.8074, 2.8074, 3.0000],
[1.0000, 1.0000, 1.5850, 2.5850, 2.0000],
[2.8074, 3.0000, 3.0000, 1.0000, 2.3219]]])
t1.log10(): tensor([[[0.9542, 0.8451, 0.8451, 0.4771, 0.9031],
[0.0000, 0.4771, 0.6021, 0.3010, 0.0000],
[0.7782, 0.9031, 0.9031, 0.7782, 0.9031],
[0.7782, 0.6021, 0.4771, 0.3010, 0.6990]],
[[0.8451, 0.6021, 0.7782, 0.9542, 0.9031],
[0.6990, 0.7782, 0.8451, 0.7782, 0.0000],
[0.0000, 0.9031, 0.6990, 0.6021, 0.9542],
[0.4771, 0.9542, 0.9542, 0.7782, 0.4771]],
[[0.9031, 0.7782, 0.6990, 0.8451, 0.6990],
[0.4771, 0.3010, 0.8451, 0.8451, 0.9031],
[0.3010, 0.3010, 0.4771, 0.7782, 0.6021],
[0.8451, 0.9031, 0.9031, 0.3010, 0.6990]]])
torch.log(t1): tensor([[[2.1972, 1.9459, 1.9459, 1.0986, 2.0794],
[0.0000, 1.0986, 1.3863, 0.6931, 0.0000],
[1.7918, 2.0794, 2.0794, 1.7918, 2.0794],
[1.7918, 1.3863, 1.0986, 0.6931, 1.6094]],
[[1.9459, 1.3863, 1.7918, 2.1972, 2.0794],
[1.6094, 1.7918, 1.9459, 1.7918, 0.0000],
[0.0000, 2.0794, 1.6094, 1.3863, 2.1972],
[1.0986, 2.1972, 2.1972, 1.7918, 1.0986]],
[[2.0794, 1.7918, 1.6094, 1.9459, 1.6094],
[1.0986, 0.6931, 1.9459, 1.9459, 2.0794],
[0.6931, 0.6931, 1.0986, 1.7918, 1.3863],
[1.9459, 2.0794, 2.0794, 0.6931, 1.6094]]])
day02_03_张量的索引操作
- 简单行列索引: t1[0, 0], t1[0, :]
- 范围索引: t1[0:3, 1:4]
- 多维索引: t3[1,2,3]
"""
案例：
演示 张量的索引操作
涉及到的API:
简单行列索引
列表索引
范围索引
多维索引
布尔索引
需要掌握的：
简单行列索引: t1[0, 0], t1[0, :]
范围索引: t1[0:3, 1:4]
(多维索引: t3[1,2,3])
"""
# 导包
import torch
# 0.设置随机种子
torch.manual_seed(60)
# 1.创建一个张量, 3*4
t1 = torch.randint(0,10,(3,4))
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
print("-"*60)
# 2.简单行列索引
# 比如获取0,0位置的元素
print(f"t1[0,0]: {t1[0,0]}")
print("-"*60)
# 获取第1行
print(f"t1[0]: {t1[0]}")
print(f"t1[0,:]: {t1[0,:]}")
print("-"*60)
# 获取第1列t1[:,0]
print(f"t1[:,0]: {t1[:,0]}")
print("-"*60)
t1: tensor([[1, 7, 0, 8],
[3, 1, 6, 5],
[1, 0, 3, 9]]), shape: torch.Size([3, 4]), dtype元素类型：torch.int64
------------------------------------------------------------
t1[0,0]: 1
------------------------------------------------------------
t1[0]: tensor([1, 7, 0, 8])
t1[0,:]: tensor([1, 7, 0, 8])
------------------------------------------------------------
t1[:,0]: tensor([1, 3, 1])
------------------------------------------------------------
# 3.演示 列表索引
# 获取(0,1), (1,2)位置的元素
print(f"t1[[0,1],[1,2]]: {t1[[0, 1], [1, 2]]}")
# 获取(0,1), (2,2)位置的元素
print(f"t1[[0,2],[1,2]]: {t1[[0, 2], [1, 2]]}")
# 获取 第一行，第1，2，3列 的元素
print(f"t1[0,[0,1,2]]: {t1[0,[0,1,2]]}")
print("-"*60)
# 4.演示 范围索引，起始位置:结束位置(不包含):步长
# 获取前2行
print(f"t1[0:2,:]: {t1[0:2,:]}")
# 获取第2行到第3行t1[1:3]
print(f"t1[1:3]: {t1[1:3]}")
# 获取第2列到第3列t1[:,1:3]
print(f"t1[:,1:3]: {t1[:,1:3]}")
print("-"*60)
t1 = torch.randint(0,10,(5,6))
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
# 获取奇数行，偶数列 的所有元素 t1[0::2,1::2]
print(f"t1[0::2,1::2]: {t1[0::2,1::2]}")
t1[[0,1],[1,2]]: tensor([7, 6])
t1[[0,2],[1,2]]: tensor([7, 3])
t1[0,[0,1,2]]: tensor([1, 7, 0])
------------------------------------------------------------
t1[0:2,:]: tensor([[1, 7, 0, 8],
[3, 1, 6, 5]])
t1[1:3]: tensor([[3, 1, 6, 5],
[1, 0, 3, 9]])
t1[:,1:3]: tensor([[7, 0],
[1, 6],
[0, 3]])
------------------------------------------------------------
t1: tensor([[4, 1, 6, 2, 7, 6],
[2, 0, 5, 8, 6, 9],
[8, 6, 0, 3, 3, 0],
[0, 6, 7, 8, 8, 2],
[5, 6, 1, 9, 8, 2]]), shape: torch.Size([5, 6]), dtype元素类型：torch.int64
t1[0::2,1::2]: tensor([[1, 2, 6],
[6, 3, 0],
[6, 9, 2]])
# 5. 演示 多维索引
# 创建一个多维张量，3*4*5
t3 = torch.randint(0,10,(3,4,5))
print(f"t3: {t3}, shape: {t3.shape}, dtype元素类型：{t3.dtype}")
# 获取0轴上索引为1的所有元素 t3[1]
print(f"t3[1]: {t3[1]}")
# 获取索引0，0，0的元素 t3[0,0,0]
print(f"t3[0,0,0]: {t3[0,0,0]}")
# 获取索引1，0，2的元素 t3[1，0，2]
print(f"t3[1，0，2]: {t3[1,0,2]}")
# 6. 演示 布尔索引
# 获取第一行中大于5的元素
print(f"t1[0]>5: {t1[0]>5}")
print(f"t1[0,t1[0]>5]: {t1[0,t1[0]>5]}")
# 获取第2列中大于5的元素
print(f"t1[:,1]>5: {t1[:,1]>5}")
print(f"t1[t1[:,1]>5,1]: {t1[t1[:,1]>5,1]}")
# 获取第2列中大于5的元素对应的行
print(f"t1[t1[:,1]>5,:]: {t1[t1[:,1]>5,:]}")
# 获取第2列中小于5的元素对应的行
print(f"t1[t1[:,1]<5,:]: {t1[t1[:,1]<5,:]}")
t3: tensor([[[3, 7, 1, 6, 7],
[2, 0, 3, 2, 1],
[7, 9, 8, 7, 0],
[7, 8, 2, 7, 2]],
[[0, 0, 7, 8, 0],
[2, 1, 0, 0, 2],
[9, 3, 8, 0, 5],
[2, 4, 9, 1, 7]],
[[7, 7, 7, 3, 5],
[6, 0, 6, 6, 7],
[0, 4, 8, 9, 5],
[1, 1, 0, 2, 5]]]), shape: torch.Size([3, 4, 5]), dtype元素类型：torch.int64
t3[1]: tensor([[0, 0, 7, 8, 0],
[2, 1, 0, 0, 2],
[9, 3, 8, 0, 5],
[2, 4, 9, 1, 7]])
t3[0,0,0]: 3
t3[1，0，2]: 7
t1[0]>5: tensor([False, False, True, False, True, True])
t1[0,t1[0]>5]: tensor([6, 7, 6])
t1[:,1]>5: tensor([False, False, True, True, True])
t1[t1[:,1]>5,1]: tensor([6, 6, 6])
t1[t1[:,1]>5,:]: tensor([[8, 6, 0, 3, 3, 0],
[0, 6, 7, 8, 8, 2],
[5, 6, 1, 9, 8, 2]])
t1[t1[:,1]<5,:]: tensor([[4, 1, 6, 2, 7, 6],
[2, 0, 5, 8, 6, 9]])
day02_04_张量的形状操作
- reshape() 不改变张量内容的前提下，改变张量的形状，不改变元素的顺序
- unsqueeze() 在指定的轴/维度上增加一个维度（长度1），升维
- squeeze() 删除所有长度为1的维度，降维
- transpose() 交换维度，只能交换两个维度
- permute() 交换维度，可以交换多个维度
- view() 改变形状，要求是连续张量，张量中元素的顺序没有改变，没有经过transpose或permute等操作
- contiguous() 将非连续张量转换为连续张量
- is_contiguous() 判断张量是否是连续张量
"""
案例：
演示 张量的形状操作
涉及到的API:
reshape() 不改变张量内容的前提下，改变张量的形状，不改变元素的顺序
unsqueeze() 在指定的轴/维度上增加一个维度（长度1），升维
squeeze() 删除所有长度为1的维度，降维
transpose() 交换维度，只能交换两个维度
permute() 交换维度，可以交换多个维度
view() 改变形状，要求是连续张量，张量中元素的顺序没有改变，
没有经过transpose或permute等操作
contiguous() 将非连续张量转换为连续张量
is_contiguous() 判断张量是否是连续张量
需要掌握的：
reshape()
unsqueeze()
permute()
view()
"""
# 导包
import torch
# 0.设置随机种子
torch.manual_seed(6)
# # 1.创建一个张量，3*4
t1 = torch.randint(0,10,(3,4))
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
print("-"*60)
# # 2.演示 reshape
# 改为4*3
t2 = t1.reshape(4,3,1,1,1,1,1)
print(f"t2: {t2}, shape: {t2.shape}, dtype元素类型：{t2.dtype}")
# 改为2*6
t2 = t1.reshape(2,6)
print(f"t2: {t2}, shape: {t2.shape}, dtype元素类型：{t2.dtype}")
# # 改为8*2
# t2 = t1.reshape(8,-1)
# print(f"t2: {t2}, shape: {t2.shape}, dtype元素类型：{t2.dtype}")
# 3.演示 unsqueeze
# 在0轴上增加一个维度
t3 = t1.unsqueeze(dim=0) # (1,3,4)
print(f"t3: {t3}, shape: {t3.shape}, dtype元素类型：{t3.dtype}")
# 在1轴上增加一个维度
t3 = t1.unsqueeze(dim=1) # (3,1,4)
print(f"t3: {t3}, shape: {t3.shape}, dtype元素类型：{t3.dtype}")
# 在2轴上增加一个维度
t3 = t1.unsqueeze(dim=2) # (3,4,1)
print(f"t3: {t3}, shape: {t3.shape}, dtype元素类型：{t3.dtype}")
# 在3轴上增加一个维度
t3 = t3.unsqueeze(dim=3) # (3,4,1,1)
print(f"t3: {t3}, shape: {t3.shape}, dtype元素类型：{t3.dtype}")
# 4.演示 squeeze
# 降维，删除所有长度为1的维度/轴
t4 = t3.squeeze() # (3,4,1,1) -> (3,4)
print(f"t4: {t4}, shape: {t4.shape}, dtype元素类型：{t4.dtype}")
t1: tensor([[0, 1, 3, 2],
[4, 7, 7, 6],
[4, 8, 7, 9]]), shape: torch.Size([3, 4]), dtype元素类型：torch.int64
------------------------------------------------------------
t2: tensor([[[[[[[0]]]]],
[[[[[1]]]]],
[[[[[3]]]]]],
[[[[[[2]]]]],
[[[[[4]]]]],
[[[[[7]]]]]],
[[[[[[7]]]]],
[[[[[6]]]]],
[[[[[4]]]]]],
[[[[[[8]]]]],
[[[[[7]]]]],
[[[[[9]]]]]]]), shape: torch.Size([4, 3, 1, 1, 1, 1, 1]), dtype元素类型：torch.int64
t2: tensor([[0, 1, 3, 2, 4, 7],
[7, 6, 4, 8, 7, 9]]), shape: torch.Size([2, 6]), dtype元素类型：torch.int64
t3: tensor([[[0, 1, 3, 2],
[4, 7, 7, 6],
[4, 8, 7, 9]]]), shape: torch.Size([1, 3, 4]), dtype元素类型：torch.int64
t3: tensor([[[0, 1, 3, 2]],
[[4, 7, 7, 6]],
[[4, 8, 7, 9]]]), shape: torch.Size([3, 1, 4]), dtype元素类型：torch.int64
t3: tensor([[[0],
[1],
[3],
[2]],
[[4],
[7],
[7],
[6]],
[[4],
[8],
[7],
[9]]]), shape: torch.Size([3, 4, 1]), dtype元素类型：torch.int64
t3: tensor([[[[0]],
[[1]],
[[3]],
[[2]]],
[[[4]],
[[7]],
[[7]],
[[6]]],
[[[4]],
[[8]],
[[7]],
[[9]]]]), shape: torch.Size([3, 4, 1, 1]), dtype元素类型：torch.int64
t4: tensor([[0, 1, 3, 2],
[4, 7, 7, 6],
[4, 8, 7, 9]]), shape: torch.Size([3, 4]), dtype元素类型：torch.int64
# 5. 演示 transpose, permute
# 使用 transpose 交换维度
t5 = t1.transpose(0,1) # (3,4) -> (4,3)
print(f"t5: {t5}, shape: {t5.shape}, dtype元素类型：{t5.dtype}")
# 使用 permute 交换维度，第一个位置为新的张量的0轴
t6 = t1.permute(1,0)
print(f"t6: {t6}, shape: {t6.shape}, dtype元素类型：{t6.dtype}")
# 使用 permute 交互多个维度, 图片数据HWC(0,1,2) -> CHW (2,0,1)
t1 = torch.randint(0,10,(2,3,4))
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
t7 = t1.permute(2,0,1) # (4,2,3)
print(f"t7: {t7}, shape: {t7.shape}, dtype元素类型：{t7.dtype}")
# 6. 演示 view
# t8 = t1.view(-1)
t9 = t7.contiguous()
print(f"t9: {t9}, shape: {t9.shape}, dtype元素类型：{t9.dtype}")
t8 = t7.contiguous().view(4,6)
print(f"t8: {t8}, shape: {t8.shape}, dtype元素类型：{t8.dtype}")
print(t7.is_contiguous())
print(t8.is_contiguous())
print(t9.is_contiguous())
t5: tensor([[0, 4, 4],
[1, 7, 8],
[3, 7, 7],
[2, 6, 9]]), shape: torch.Size([4, 3]), dtype元素类型：torch.int64
t6: tensor([[0, 4, 4],
[1, 7, 8],
[3, 7, 7],
[2, 6, 9]]), shape: torch.Size([4, 3]), dtype元素类型：torch.int64
t1: tensor([[[5, 5, 7, 2],
[3, 9, 8, 4],
[6, 1, 6, 4]],
[[2, 4, 9, 9],
[0, 7, 1, 2],
[3, 3, 2, 0]]]), shape: torch.Size([2, 3, 4]), dtype元素类型：torch.int64
t7: tensor([[[5, 3, 6],
[2, 0, 3]],
[[5, 9, 1],
[4, 7, 3]],
[[7, 8, 6],
[9, 1, 2]],
[[2, 4, 4],
[9, 2, 0]]]), shape: torch.Size([4, 2, 3]), dtype元素类型：torch.int64
t9: tensor([[[5, 3, 6],
[2, 0, 3]],
[[5, 9, 1],
[4, 7, 3]],
[[7, 8, 6],
[9, 1, 2]],
[[2, 4, 4],
[9, 2, 0]]]), shape: torch.Size([4, 2, 3]), dtype元素类型：torch.int64
t8: tensor([[5, 3, 6, 2, 0, 3],
[5, 9, 1, 4, 7, 3],
[7, 8, 6, 9, 1, 2],
[2, 4, 4, 9, 2, 0]]), shape: torch.Size([4, 6]), dtype元素类型：torch.int64
False
True
True
day02_05_张量的拼接
- torch.cat() 按指定维度拼接张量，不改变维度数量，除了要拼接的维度，其他维度的形状必须相同 (m, n) + (k, n) -> (m+k, n). numpy.concatenate()
- torch.stack() 在新的维度上堆叠张量，会改变维度数量，所有张量的形状必须相同 (m, n) + (m, n) -> (2, m, n). numpy.stack()
"""
演示：
演示 张量的拼接操作
涉及到的API:
torch.cat()
按指定维度拼接张量，不改变维度数量，除了要拼接的维度，其他维度的形状必须相同 (m, n) + (k, n) -> (m+k, n)
torch.stack()
在新的维度上拼接张量，增加了一个新的维度，会改变维度数量，所有张量的形状必须相同 (m, n) + (m, n) -> (2, m, n)
"""
import torch
# 0.设置随机种子
torch.manual_seed(6)
# 1.创建随机张量
t1 = torch.randint(1,10,(2,3))
print(f"t1: {t1}, t1.shape: {t1.shape}")
t2 = torch.randint(1,10,(2,3))
print(f"t2: {t2}, t2.shape: {t2.shape}")
# 2.演示 torch.cat()
# 按0轴拼接
t3 = torch.cat((t1,t2), dim=0) # (2,3) + (2,3) -> (4,3)
print(f"t3: {t3}, t3.shape: {t3.shape}")
# 按1轴拼接
t3 = torch.cat((t1,t2), dim=1) # (2,3) + (2,3) -> (2,6)
print(f"t3: {t3}, t3.shape: {t3.shape}")
# 按2轴拼接
# t3 = torch.cat((t1,t2), dim=2) # (2,3) + (2,3) -> (2,3,?)
# print(f"t3: {t3}, t3.shape: {t3.shape}")
t1: tensor([[9, 7, 7],
[3, 8, 1]]), t1.shape: torch.Size([2, 3])
t2: tensor([[3, 4, 2],
[1, 6, 8]]), t2.shape: torch.Size([2, 3])
t3: tensor([[9, 7, 7],
[3, 8, 1],
[3, 4, 2],
[1, 6, 8]]), t3.shape: torch.Size([4, 3])
t3: tensor([[9, 7, 7, 3, 4, 2],
[3, 8, 1, 1, 6, 8]]), t3.shape: torch.Size([2, 6])
# 3.演示 torch.stack()
# 按0轴拼接
t3 = torch.stack((t1,t2), dim=0) # (2,3) + (2,3) -> (2,2,3)
print(f"t3: {t3}, t3.shape: {t3.shape}")
# 按1轴拼接
t3 = torch.stack((t1,t2), dim=1) # (2,3) + (2,3) -> (2,2,3)
print(f"t3: {t3}, t3.shape: {t3.shape}")
# 按2轴拼接
t3 = torch.stack((t1,t2), dim=2) # (2,3) + (2,3) -> (2,3,2)
print(f"t3: {t3}, t3.shape: {t3.shape}")
# 按3轴拼接
# t3 = torch.stack((t1,t2), dim=3) # (2,3) + (2,3) -> (2,3,?,2)
# print(f"t3: {t3}, t3.shape: {t3.shape}")
t3: tensor([[[9, 7, 7],
[3, 8, 1]],
[[3, 4, 2],
[1, 6, 8]]]), t3.shape: torch.Size([2, 2, 3])
t3: tensor([[[9, 7, 7],
[3, 4, 2]],
[[3, 8, 1],
[1, 6, 8]]]), t3.shape: torch.Size([2, 2, 3])
t3: tensor([[[9, 3],
[7, 4],
[7, 2]],
[[3, 1],
[8, 6],
[1, 8]]]), t3.shape: torch.Size([2, 3, 2])
day02_06_自动微分模块入门案例
-
requires_grad=True, pytorch提供自动微分功能
-
梯度 = 损失函数对模型参数的导数;
- 损失函数的梯度，无需手动计算，由自动微分模块在反向传播过程中自动计算，进一步由梯度来更新模型参数(权重w和偏置b);
- 梯度下降法更新权重：
w新 = w旧 - 学习率 * 梯度 - 梯度/导数的理解
- 数值：在某一个点上，对函数求导得到的值就是该点的梯度/导数
- 类比：梯度就是上山下山最快的方向
- 在平面内，梯度就是某一点上的斜率
- y = 2x^2 某一点x=1的梯度，就是这一点上的斜率
- 反向传播过程中如何计算梯度
- 反向传播利用链式法则不断的从后向前求导，求出来的值就是梯度
训练神经网络时，常用反向传播算法来计算梯度（反向追溯计算图，利用链式法则计算梯度）。为了计算这些梯度，PyTorch内置了名为 torch.autograd 的自动微分模块。它支持任意计算图的自动梯度计算：
我们使用 backward 方法、grad 属性来实现梯度的计算和访问。
# 自动微分示例
x = torch.tensor([2.0, 3.0], requires_grad=True)
y = x**2+2
# y' = 2x, 4
print("y:", y)
y.sum().backward() # 反向传播
print("x的梯度:", x.grad)
y: tensor([ 6., 11.], grad_fn=<AddBackward0>)
x的梯度: tensor([4., 6.])
"""
案例
演示 自动微分模块如何计算梯度/导数
介绍：
梯度：损失函数对模型参数的导数/梯度，损失函数的导函数的数值，自变量是模型参数w, b
损失函数的梯度/导数，不需要手动计算，由自动微分模块 在backward反向传播中自动计算的，从而进一步更新模型参数
梯度下降法更新模型参数公式：
w新 = w旧 - 学习率 * 梯度
"""
# 导包
import torch
# 1.定义变量，记录初始权重
w = torch.tensor([10,20], requires_grad=True, dtype=torch.float32) # 权重
print(f"初始权重w: {w}, requires_grad: {w.requires_grad}")
print(f"初始权重 w.data: {w.data}")
print(f"梯度 w.grad: {w.grad}")
# 2.定义损失函数
loss = 0.5*w**2 # loss' = w, 10
# w = [w1,w2] => loss = [0.5*w1^2, 0.5*w2^2]
print(f"loss: {loss}")
# 3.反向传播，计算梯度
# 因为神经网络训练中计算的损失是每个批次的平均损失
loss.mean().backward() # w, 10
# loss.mean() = 0.5*(w1^2 + w2^2)/2
# d(loss.mean())/dw1 = 0.5*w1
# d(loss.mean())/dw2 = 0.5*w2
# d(loss.mean())/(dw1,dw2) = [0.5*w1, 0.5*w2] = [5,10]
# 4.打印梯度, w.grad
print(f"梯度 w.grad: {w.grad}")
# 5.更新参数，演示一步梯度下降
w.data = w.data - 0.01 * w.grad
print(f"更新后权重 w: {w}")
初始权重w: tensor([10., 20.], requires_grad=True), requires_grad: True
初始权重 w.data: tensor([10., 20.])
梯度 w.grad: None
loss: tensor([ 50., 200.], grad_fn=<MulBackward0>)
梯度 w.grad: tensor([ 5., 10.])
更新后权重 w: tensor([ 9.9500, 19.9000], requires_grad=True)
day02_07_梯度下降法求最小值案例
- 演示 梯度下降法求最小值, 循环实现 计算梯度, 更新参数
- 定义点 w=10 requires_grad=True dtlosspe=torch.float32
- 定义函数 loss = w**2 + 20
- 利用梯度下降法 循环迭代1000 求最优解
- 3.1 正向计算(前向传播)
- 3.2 梯度清零 w.grad.zero_()
- 3.3 反向传播
- 3.4 梯度更新 w.data = w.data - 0.01 * w.grad
演示梯度下降法
# 目标函数 & 导数
def f(x):
return x**2 + 20
def grad(x):
return 2 * x
# 准备绘图
x_vals = np.linspace(-10, 10, 200)
y_vals = f(x_vals)
plt.figure(figsize=(10,6))
plt.title("梯度下降法演示-二维曲线")
plt.plot(x_vals,y_vals, label="y = $w^{2}$ + 20")
plt.grid()
plt.legend()
plt.tight_layout()
plt.show()
# 梯度下降参数
lr = 0.1 # learning rate
x = 7 # 初始点
steps = 20 # 迭代次数
xs = [x] # 保存每一步的 x
ys = [f(x)] # 保存每一步的 y
# 梯度下降过程
for i in range(steps):
g = grad(x)
x = x - lr * g
xs.append(x)
ys.append(f(x))
# 绘制函数曲线
plt.figure(figsize=(10,6))
plt.plot(x_vals, y_vals, label=r"y = $w^{2}$ + 20")
# 绘制梯度下降路径
plt.scatter(xs, ys, color='red', s=30, label="梯度下降路径")
plt.plot(xs, ys, color='red', linestyle='--')
# 最终点标注
plt.scatter(xs[-1], ys[-1], color='green', s=60, label="最低点")
plt.grid()
plt.legend()
plt.title("梯度下降法演示-二维曲线")
plt.xlabel("w")
plt.ylabel("y")
plt.tight_layout()
plt.show()
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib.animation import FuncAnimation, PillowWriter
# -----------------------------
# 1. 定义损失函数和梯度
# -----------------------------
def f(x, y):
return x**2 + y**2
def grad_f(x, y):
return np.array([2*x, 2*y])
# -----------------------------
# 2. 创建网格用于绘制 3D 曲面
# -----------------------------
x = np.linspace(-5, 5, 200)
y = np.linspace(-5, 5, 200)
X, Y = np.meshgrid(x, y)
Z = f(X, Y)
# -----------------------------
# 3. 梯度下降
# -----------------------------
lr = 0.1
steps = 20
point = np.array([4.0, 4.0])
trajectory = [point.copy()]
for _ in range(steps):
grad = grad_f(point[0], point[1])
point -= lr * grad
trajectory.append(point.copy())
trajectory = np.array(trajectory)
Z_traj = f(trajectory[:,0], trajectory[:,1])
# -----------------------------
# 4. 绘制静态图（带箭头）
# -----------------------------
fig = plt.figure(figsize=(16,12))
ax = fig.add_subplot(111, projection='3d')
ax.set_title("梯度下降法演示：3D损失曲面 + 下降路径 + 梯度反方向箭头")
# 曲面
ax.plot_surface(X, Y, Z, alpha=0.75, cmap="viridis")
# 下降路径
ax.plot(trajectory[:,0], trajectory[:,1], Z_traj,
color='red', marker='o', label="下降路径")
# 添加地面方向箭头（-梯度方向）
for i in range(len(trajectory)):
x, y = trajectory[i]
dx, dy = -grad_f(x, y) # 梯度反方向
ax.quiver(x, y, f(x,y), dx, dy, 0,
length=0.5, color="black")
ax.set_xlabel("w1")
ax.set_ylabel("w2")
ax.set_zlabel("Loss")
ax.legend()
plt.tight_layout()
plt.show()
# -----------------------------
# 5. 生成 GIF 动画
# -----------------------------
fig = plt.figure(figsize=(12,9))
ax = fig.add_subplot(111, projection='3d')
def update(frame):
ax.cla()
ax.plot_surface(X, Y, Z, alpha=0.75, cmap="viridis")
# 当前路径
ax.plot(trajectory[:frame+1,0], trajectory[:frame+1,1],
Z_traj[:frame+1], color='red')
# 当前点
x, y = trajectory[frame]
z = f(x,y)
ax.scatter(x, y, z, color="red", s=80)
# 地面方向箭头
dx, dy = -grad_f(x, y)
ax.quiver(x, y, z, dx, dy, 0, color="black", length=0.5)
ax.set_title("梯度下降过程 (Step {})".format(frame))
ax.set_xlabel("w1")
ax.set_ylabel("w2")
ax.set_zlabel("Loss")
ax.set_zlim(0, 50)
ani = FuncAnimation(fig, update, frames=len(trajectory), interval=300)
# 保存 GIF
gif_path = "./data/gradient_descent.gif"
ani.save(gif_path, writer=PillowWriter(fps=3))
gif_path
'./data/gradient_descent.gif'
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
# 定义函数及梯度
def f(x, y):
return x**2 + y**2
def grad_f(x, y):
return np.array([2*x, 2*y])
# 创建网格
x = np.linspace(-5, 5, 100)
y = np.linspace(-5, 5, 100)
X, Y = np.meshgrid(x, y)
Z = f(X, Y)
# 梯度下降初始化
lr = 0.1 # 学习率
steps = 25
point = np.array([4.0, 4.0]) # 起点
trajectory = [point.copy()]
# 梯度下降迭代
for _ in range(steps):
grad = grad_f(point[0], point[1])
point -= lr * grad
trajectory.append(point.copy())
trajectory = np.array(trajectory)
Z_traj = f(trajectory[:,0], trajectory[:,1])
# 绘图
# Plot surface
fig = plt.figure(figsize=(16,12))
ax = fig.add_subplot(111, projection='3d')
ax.set_title("梯度下降法演示——三维曲面")
ax.plot_surface(X, Y, Z, alpha=0.75, cmap='viridis', label="z = $w_1^{2}$ + $w_2^{2}$") # No colors specified
ax.set_xlabel(r"$w_1$")
ax.set_ylabel(r"$w_2$")
ax.legend()
plt.tight_layout()
fig = plt.figure(figsize=(16,12))
ax = fig.add_subplot(111, projection='3d')
ax.set_title("梯度下降法演示——三维曲面")
ax.plot_surface(X, Y, Z, alpha=0.75, cmap='viridis', label="z = $w_1^{2}$ + $w_2^{2}$") # 半透明曲面
ax.plot(trajectory[:,0], trajectory[:,1], Z_traj, color='r', marker='o', label='梯度下降路径')
ax.set_xlabel(r"$w_1$")
ax.set_ylabel(r"$w_2$")
# ax.set_zlabel(r"z = $w_1^{2}$ + $w_2^{2}$")
ax.legend()
plt.tight_layout()
plt.show()
- 演示自动微分模块, 循环实现 计算梯度, 更新参数
- 定义点 w=10 requires_grad=True dtlosspe=torch.float32
- 定义函数 loss = w**2 + 20
- 利用梯度下降法 循环迭代1000 求最优解
- 3.1 正向计算(前向传播)
- 3.2 梯度清零 w.grad.zero_()
- 3.3 反向传播
- 3.4 梯度更新 w.data = w.data - 0.01 * w.grad
"""
案例：
演示 自动微分模块案例 循环实现，计算梯度，更新参数
题目：
# 求 loss = w**2 + 20 的极小值点 并打印loss是最小值时 w的值(梯度)
解题步骤：
# 1 定义点 w=10 requires_grad=True dtype=torch.float32
# 2 定义函数 loss = w**2 + 20
# 3 利用梯度下降法 循环迭代500 求最优解
# 3.1 正向计算(前向传播)
# 3.2 梯度清零 w.grad.zero_()
# 3.3 反向传播
# 3.4 梯度更新 w.data = w.data - 0.01 * w.grad
"""
# 求 loss = w**2 + 20 的极小值点 并打印loss是最小值时 w的值(梯度)
import torch
# 1 定义点 w=10 requires_grad=True dtype=torch.float32
# 参数1：初始值；参数2：是否自动微分/求导；参数3：元素类型
w = torch.tensor([10,20], requires_grad=True, dtype=torch.float32)
print(f"初始值w: {w}")
# 2 定义函数 loss = w**2 + 20
loss = w ** 2 + 20 # 求导 2w [2w1, 2w2]
print(f"初始值loss: {loss}")
# 3 利用梯度下降法 循环迭代100 求最优解
print(f"开始时,权重初始值w: {w}")
print("(0.01 * w.grad): 无")
print(f"初始值loss: {loss}")
# 记录损失均值
loss_list = []
# 循环迭代500次
for i in range(500):
# 3.1 正向计算(前向传播)
# 求损失值
loss = w ** 2 + 20
# 3.2 梯度清零 w.grad.zero_(), 判断非空
if w.grad is not None:
w.grad.zero_()
# 3.3 反向传播 .backward()
loss.mean().backward()
# 3.4 梯度更新 w.data = w.data - 0.01 * w.grad
w.data = w.data - 0.01 * w.grad
# print(f"第{i+1}次迭代, 梯度: {w.grad}")
print(f"第{i+1}次迭代, 权重w: {w.data}, (0.01 * w.grad): {(0.01 * w.grad).data}, loss.mean(): {loss.mean().data}")
# 添加到loss_list
loss_list.append(loss.mean().item())
# 打印最终的结果
print(f"最终结果, 权重w: {w}, 梯度: {w.grad}, 损失均值loss.mean(): {loss.mean().data}")
初始值w: tensor([10., 20.], requires_grad=True)
初始值loss: tensor([120., 420.], grad_fn=<AddBackward0>)
开始时,权重初始值w: tensor([10., 20.], requires_grad=True)
(0.01 * w.grad): 无
初始值loss: tensor([120., 420.], grad_fn=<AddBackward0>)
第1次迭代, 权重w: tensor([ 9.9000, 19.8000]), (0.01 * w.grad): tensor([0.1000, 0.2000]), loss.mean(): 270.0
第2次迭代, 权重w: tensor([ 9.8010, 19.6020]), (0.01 * w.grad): tensor([0.0990, 0.1980]), loss.mean(): 265.0249938964844
第3次迭代, 权重w: tensor([ 9.7030, 19.4060]), (0.01 * w.grad): tensor([0.0980, 0.1960]), loss.mean(): 260.14898681640625
第4次迭代, 权重w: tensor([ 9.6060, 19.2119]), (0.01 * w.grad): tensor([0.0970, 0.1941]), loss.mean(): 255.37002563476562
第5次迭代, 权重w: tensor([ 9.5099, 19.0198]), (0.01 * w.grad): tensor([0.0961, 0.1921]), loss.mean(): 250.68617248535156
第6次迭代, 权重w: tensor([ 9.4148, 18.8296]), (0.01 * w.grad): tensor([0.0951, 0.1902]), loss.mean(): 246.0955047607422
第7次迭代, 权重w: tensor([ 9.3207, 18.6413]), (0.01 * w.grad): tensor([0.0941, 0.1883]), loss.mean(): 241.59619140625
第8次迭代, 权重w: tensor([ 9.2274, 18.4549]), (0.01 * w.grad): tensor([0.0932, 0.1864]), loss.mean(): 237.18643188476562
第9次迭代, 权重w: tensor([ 9.1352, 18.2703]), (0.01 * w.grad): tensor([0.0923, 0.1845]), loss.mean(): 232.86441040039062
第10次迭代, 权重w: tensor([ 9.0438, 18.0876]), (0.01 * w.grad): tensor([0.0914, 0.1827]), loss.mean(): 228.62840270996094
第11次迭代, 权重w: tensor([ 8.9534, 17.9068]), (0.01 * w.grad): tensor([0.0904, 0.1809]), loss.mean(): 224.47671508789062
第12次迭代, 权重w: tensor([ 8.8638, 17.7277]), (0.01 * w.grad): tensor([0.0895, 0.1791]), loss.mean(): 220.4076385498047
第13次迭代, 权重w: tensor([ 8.7752, 17.5504]), (0.01 * w.grad): tensor([0.0886, 0.1773]), loss.mean(): 216.41952514648438
第14次迭代, 权重w: tensor([ 8.6875, 17.3749]), (0.01 * w.grad): tensor([0.0878, 0.1755]), loss.mean(): 212.51080322265625
第15次迭代, 权重w: tensor([ 8.6006, 17.2012]), (0.01 * w.grad): tensor([0.0869, 0.1737]), loss.mean(): 208.6798095703125
第16次迭代, 权重w: tensor([ 8.5146, 17.0292]), (0.01 * w.grad): tensor([0.0860, 0.1720]), loss.mean(): 204.92507934570312
第17次迭代, 权重w: tensor([ 8.4294, 16.8589]), (0.01 * w.grad): tensor([0.0851, 0.1703]), loss.mean(): 201.2450408935547
第18次迭代, 权重w: tensor([ 8.3451, 16.6903]), (0.01 * w.grad): tensor([0.0843, 0.1686]), loss.mean(): 197.6382598876953
第19次迭代, 权重w: tensor([ 8.2617, 16.5234]), (0.01 * w.grad): tensor([0.0835, 0.1669]), loss.mean(): 194.10325622558594
第20次迭代, 权重w: tensor([ 8.1791, 16.3581]), (0.01 * w.grad): tensor([0.0826, 0.1652]), loss.mean(): 190.63861083984375
第21次迭代, 权重w: tensor([ 8.0973, 16.1946]), (0.01 * w.grad): tensor([0.0818, 0.1636]), loss.mean(): 187.24290466308594
第22次迭代, 权重w: tensor([ 8.0163, 16.0326]), (0.01 * w.grad): tensor([0.0810, 0.1619]), loss.mean(): 183.91476440429688
第23次迭代, 权重w: tensor([ 7.9361, 15.8723]), (0.01 * w.grad): tensor([0.0802, 0.1603]), loss.mean(): 180.65286254882812
第24次迭代, 权重w: tensor([ 7.8568, 15.7136]), (0.01 * w.grad): tensor([0.0794, 0.1587]), loss.mean(): 177.45587158203125
第25次迭代, 权重w: tensor([ 7.7782, 15.5564]), (0.01 * w.grad): tensor([0.0786, 0.1571]), loss.mean(): 174.322509765625
第26次迭代, 权重w: tensor([ 7.7004, 15.4009]), (0.01 * w.grad): tensor([0.0778, 0.1556]), loss.mean(): 171.25149536132812
第27次迭代, 权重w: tensor([ 7.6234, 15.2469]), (0.01 * w.grad): tensor([0.0770, 0.1540]), loss.mean(): 168.2415771484375
第28次迭代, 权重w: tensor([ 7.5472, 15.0944]), (0.01 * w.grad): tensor([0.0762, 0.1525]), loss.mean(): 165.29156494140625
第29次迭代, 权重w: tensor([ 7.4717, 14.9434]), (0.01 * w.grad): tensor([0.0755, 0.1509]), loss.mean(): 162.40025329589844
第30次迭代, 权重w: tensor([ 7.3970, 14.7940]), (0.01 * w.grad): tensor([0.0747, 0.1494]), loss.mean(): 159.56649780273438
第31次迭代, 权重w: tensor([ 7.3230, 14.6461]), (0.01 * w.grad): tensor([0.0740, 0.1479]), loss.mean(): 156.78912353515625
第32次迭代, 权重w: tensor([ 7.2498, 14.4996]), (0.01 * w.grad): tensor([0.0732, 0.1465]), loss.mean(): 154.06703186035156
第33次迭代, 权重w: tensor([ 7.1773, 14.3546]), (0.01 * w.grad): tensor([0.0725, 0.1450]), loss.mean(): 151.3990936279297
第34次迭代, 权重w: tensor([ 7.1055, 14.2111]), (0.01 * w.grad): tensor([0.0718, 0.1435]), loss.mean(): 148.7842559814453
第35次迭代, 权重w: tensor([ 7.0345, 14.0690]), (0.01 * w.grad): tensor([0.0711, 0.1421]), loss.mean(): 146.22145080566406
第36次迭代, 权重w: tensor([ 6.9641, 13.9283]), (0.01 * w.grad): tensor([0.0703, 0.1407]), loss.mean(): 143.7096405029297
第37次迭代, 权重w: tensor([ 6.8945, 13.7890]), (0.01 * w.grad): tensor([0.0696, 0.1393]), loss.mean(): 141.24781799316406
第38次迭代, 权重w: tensor([ 6.8255, 13.6511]), (0.01 * w.grad): tensor([0.0689, 0.1379]), loss.mean(): 138.83499145507812
第39次迭代, 权重w: tensor([ 6.7573, 13.5146]), (0.01 * w.grad): tensor([0.0683, 0.1365]), loss.mean(): 136.47018432617188
第40次迭代, 权重w: tensor([ 6.6897, 13.3794]), (0.01 * w.grad): tensor([0.0676, 0.1351]), loss.mean(): 134.15240478515625
第41次迭代, 权重w: tensor([ 6.6228, 13.2456]), (0.01 * w.grad): tensor([0.0669, 0.1338]), loss.mean(): 131.8807830810547
第42次迭代, 权重w: tensor([ 6.5566, 13.1132]), (0.01 * w.grad): tensor([0.0662, 0.1325]), loss.mean(): 129.6543426513672
第43次迭代, 权重w: tensor([ 6.4910, 12.9820]), (0.01 * w.grad): tensor([0.0656, 0.1311]), loss.mean(): 127.47222137451172
第44次迭代, 权重w: tensor([ 6.4261, 12.8522]), (0.01 * w.grad): tensor([0.0649, 0.1298]), loss.mean(): 125.33351135253906
第45次迭代, 权重w: tensor([ 6.3619, 12.7237]), (0.01 * w.grad): tensor([0.0643, 0.1285]), loss.mean(): 123.23737335205078
第46次迭代, 权重w: tensor([ 6.2982, 12.5965]), (0.01 * w.grad): tensor([0.0636, 0.1272]), loss.mean(): 121.18295288085938
第47次迭代, 权重w: tensor([ 6.2353, 12.4705]), (0.01 * w.grad): tensor([0.0630, 0.1260]), loss.mean(): 119.16940307617188
第48次迭代, 权重w: tensor([ 6.1729, 12.3458]), (0.01 * w.grad): tensor([0.0624, 0.1247]), loss.mean(): 117.19593048095703
第49次迭代, 权重w: tensor([ 6.1112, 12.2223]), (0.01 * w.grad): tensor([0.0617, 0.1235]), loss.mean(): 115.26172637939453
第50次迭代, 权重w: tensor([ 6.0501, 12.1001]), (0.01 * w.grad): tensor([0.0611, 0.1222]), loss.mean(): 113.36602020263672
第51次迭代, 权重w: tensor([ 5.9896, 11.9791]), (0.01 * w.grad): tensor([0.0605, 0.1210]), loss.mean(): 111.50802612304688
第52次迭代, 权重w: tensor([ 5.9297, 11.8593]), (0.01 * w.grad): tensor([0.0599, 0.1198]), loss.mean(): 109.68702697753906
第53次迭代, 权重w: tensor([ 5.8704, 11.7407]), (0.01 * w.grad): tensor([0.0593, 0.1186]), loss.mean(): 107.90225219726562
第54次迭代, 权重w: tensor([ 5.8117, 11.6233]), (0.01 * w.grad): tensor([0.0587, 0.1174]), loss.mean(): 106.15299224853516
第55次迭代, 权重w: tensor([ 5.7535, 11.5071]), (0.01 * w.grad): tensor([0.0581, 0.1162]), loss.mean(): 104.43855285644531
第56次迭代, 权重w: tensor([ 5.6960, 11.3920]), (0.01 * w.grad): tensor([0.0575, 0.1151]), loss.mean(): 102.75823974609375
第57次迭代, 权重w: tensor([ 5.6391, 11.2781]), (0.01 * w.grad): tensor([0.0570, 0.1139]), loss.mean(): 101.11134338378906
第58次迭代, 权重w: tensor([ 5.5827, 11.1653]), (0.01 * w.grad): tensor([0.0564, 0.1128]), loss.mean(): 99.49722290039062
第59次迭代, 权重w: tensor([ 5.5268, 11.0537]), (0.01 * w.grad): tensor([0.0558, 0.1117]), loss.mean(): 97.91523742675781
第60次迭代, 权重w: tensor([ 5.4716, 10.9431]), (0.01 * w.grad): tensor([0.0553, 0.1105]), loss.mean(): 96.36473083496094
第61次迭代, 权重w: tensor([ 5.4168, 10.8337]), (0.01 * w.grad): tensor([0.0547, 0.1094]), loss.mean(): 94.84506225585938
第62次迭代, 权重w: tensor([ 5.3627, 10.7254]), (0.01 * w.grad): tensor([0.0542, 0.1083]), loss.mean(): 93.35565185546875
第63次迭代, 权重w: tensor([ 5.3091, 10.6181]), (0.01 * w.grad): tensor([0.0536, 0.1073]), loss.mean(): 91.89586639404297
第64次迭代, 权重w: tensor([ 5.2560, 10.5119]), (0.01 * w.grad): tensor([0.0531, 0.1062]), loss.mean(): 90.46513366699219
第65次迭代, 权重w: tensor([ 5.2034, 10.4068]), (0.01 * w.grad): tensor([0.0526, 0.1051]), loss.mean(): 89.06288146972656
第66次迭代, 权重w: tensor([ 5.1514, 10.3027]), (0.01 * w.grad): tensor([0.0520, 0.1041]), loss.mean(): 87.68852233886719
第67次迭代, 权重w: tensor([ 5.0999, 10.1997]), (0.01 * w.grad): tensor([0.0515, 0.1030]), loss.mean(): 86.34152221679688
第68次迭代, 权重w: tensor([ 5.0489, 10.0977]), (0.01 * w.grad): tensor([0.0510, 0.1020]), loss.mean(): 85.02132415771484
第69次迭代, 权重w: tensor([4.9984, 9.9967]), (0.01 * w.grad): tensor([0.0505, 0.1010]), loss.mean(): 83.72740173339844
第70次迭代, 权重w: tensor([4.9484, 9.8968]), (0.01 * w.grad): tensor([0.0500, 0.1000]), loss.mean(): 82.459228515625
第71次迭代, 权重w: tensor([4.8989, 9.7978]), (0.01 * w.grad): tensor([0.0495, 0.0990]), loss.mean(): 81.21629333496094
第72次迭代, 权重w: tensor([4.8499, 9.6998]), (0.01 * w.grad): tensor([0.0490, 0.0980]), loss.mean(): 79.99809265136719
第73次迭代, 权重w: tensor([4.8014, 9.6028]), (0.01 * w.grad): tensor([0.0485, 0.0970]), loss.mean(): 78.80413055419922
第74次迭代, 权重w: tensor([4.7534, 9.5068]), (0.01 * w.grad): tensor([0.0480, 0.0960]), loss.mean(): 77.63392639160156
第75次迭代, 权重w: tensor([4.7059, 9.4117]), (0.01 * w.grad): tensor([0.0475, 0.0951]), loss.mean(): 76.48701477050781
第76次迭代, 权重w: tensor([4.6588, 9.3176]), (0.01 * w.grad): tensor([0.0471, 0.0941]), loss.mean(): 75.36292266845703
第77次迭代, 权重w: tensor([4.6122, 9.2244]), (0.01 * w.grad): tensor([0.0466, 0.0932]), loss.mean(): 74.26119995117188
第78次迭代, 权重w: tensor([4.5661, 9.1322]), (0.01 * w.grad): tensor([0.0461, 0.0922]), loss.mean(): 73.18140411376953
第79次迭代, 权重w: tensor([4.5204, 9.0409]), (0.01 * w.grad): tensor([0.0457, 0.0913]), loss.mean(): 72.12310028076172
第80次迭代, 权重w: tensor([4.4752, 8.9505]), (0.01 * w.grad): tensor([0.0452, 0.0904]), loss.mean(): 71.08584594726562
第81次迭代, 权重w: tensor([4.4305, 8.8610]), (0.01 * w.grad): tensor([0.0448, 0.0895]), loss.mean(): 70.06924438476562
第82次迭代, 权重w: tensor([4.3862, 8.7723]), (0.01 * w.grad): tensor([0.0443, 0.0886]), loss.mean(): 69.0728759765625
第83次迭代, 权重w: tensor([4.3423, 8.6846]), (0.01 * w.grad): tensor([0.0439, 0.0877]), loss.mean(): 68.09632110595703
第84次迭代, 权重w: tensor([4.2989, 8.5978]), (0.01 * w.grad): tensor([0.0434, 0.0868]), loss.mean(): 67.13920593261719
第85次迭代, 权重w: tensor([4.2559, 8.5118]), (0.01 * w.grad): tensor([0.0430, 0.0860]), loss.mean(): 66.20112609863281
第86次迭代, 权重w: tensor([4.2133, 8.4267]), (0.01 * w.grad): tensor([0.0426, 0.0851]), loss.mean(): 65.28173065185547
第87次迭代, 权重w: tensor([4.1712, 8.3424]), (0.01 * w.grad): tensor([0.0421, 0.0843]), loss.mean(): 64.38062286376953
第88次迭代, 权重w: tensor([4.1295, 8.2590]), (0.01 * w.grad): tensor([0.0417, 0.0834]), loss.mean(): 63.4974479675293
第89次迭代, 权重w: tensor([4.0882, 8.1764]), (0.01 * w.grad): tensor([0.0413, 0.0826]), loss.mean(): 62.63184356689453
第90次迭代, 权重w: tensor([4.0473, 8.0946]), (0.01 * w.grad): tensor([0.0409, 0.0818]), loss.mean(): 61.783470153808594
第91次迭代, 权重w: tensor([4.0068, 8.0137]), (0.01 * w.grad): tensor([0.0405, 0.0809]), loss.mean(): 60.95197677612305
第92次迭代, 权重w: tensor([3.9668, 7.9336]), (0.01 * w.grad): tensor([0.0401, 0.0801]), loss.mean(): 60.13703918457031
第93次迭代, 权重w: tensor([3.9271, 7.8542]), (0.01 * w.grad): tensor([0.0397, 0.0793]), loss.mean(): 59.33831024169922
第94次迭代, 权重w: tensor([3.8878, 7.7757]), (0.01 * w.grad): tensor([0.0393, 0.0785]), loss.mean(): 58.55548095703125
第95次迭代, 权重w: tensor([3.8490, 7.6979]), (0.01 * w.grad): tensor([0.0389, 0.0778]), loss.mean(): 57.7882194519043
第96次迭代, 权重w: tensor([3.8105, 7.6209]), (0.01 * w.grad): tensor([0.0385, 0.0770]), loss.mean(): 57.03623962402344
第97次迭代, 权重w: tensor([3.7724, 7.5447]), (0.01 * w.grad): tensor([0.0381, 0.0762]), loss.mean(): 56.29921340942383
第98次迭代, 权重w: tensor([3.7346, 7.4693]), (0.01 * w.grad): tensor([0.0377, 0.0754]), loss.mean(): 55.57685852050781
第99次迭代, 权重w: tensor([3.6973, 7.3946]), (0.01 * w.grad): tensor([0.0373, 0.0747]), loss.mean(): 54.86888122558594
第100次迭代, 权重w: tensor([3.6603, 7.3206]), (0.01 * w.grad): tensor([0.0370, 0.0739]), loss.mean(): 54.17499542236328
第101次迭代, 权重w: tensor([3.6237, 7.2474]), (0.01 * w.grad): tensor([0.0366, 0.0732]), loss.mean(): 53.494911193847656
第102次迭代, 权重w: tensor([3.5875, 7.1750]), (0.01 * w.grad): tensor([0.0362, 0.0725]), loss.mean(): 52.82836151123047
第103次迭代, 权重w: tensor([3.5516, 7.1032]), (0.01 * w.grad): tensor([0.0359, 0.0717]), loss.mean(): 52.175071716308594
第104次迭代, 权重w: tensor([3.5161, 7.0322]), (0.01 * w.grad): tensor([0.0355, 0.0710]), loss.mean(): 51.5347900390625
第105次迭代, 权重w: tensor([3.4809, 6.9619]), (0.01 * w.grad): tensor([0.0352, 0.0703]), loss.mean(): 50.907249450683594
第106次迭代, 权重w: tensor([3.4461, 6.8922]), (0.01 * w.grad): tensor([0.0348, 0.0696]), loss.mean(): 50.292198181152344
第107次迭代, 权重w: tensor([3.4117, 6.8233]), (0.01 * w.grad): tensor([0.0345, 0.0689]), loss.mean(): 49.68937683105469
第108次迭代, 权重w: tensor([3.3775, 6.7551]), (0.01 * w.grad): tensor([0.0341, 0.0682]), loss.mean(): 49.09855651855469
第109次迭代, 权重w: tensor([3.3438, 6.6875]), (0.01 * w.grad): tensor([0.0338, 0.0676]), loss.mean(): 48.51949691772461
第110次迭代, 权重w: tensor([3.3103, 6.6207]), (0.01 * w.grad): tensor([0.0334, 0.0669]), loss.mean(): 47.95196533203125
第111次迭代, 权重w: tensor([3.2772, 6.5545]), (0.01 * w.grad): tensor([0.0331, 0.0662]), loss.mean(): 47.39571762084961
第112次迭代, 权重w: tensor([3.2445, 6.4889]), (0.01 * w.grad): tensor([0.0328, 0.0655]), loss.mean(): 46.850547790527344
第113次迭代, 权重w: tensor([3.2120, 6.4240]), (0.01 * w.grad): tensor([0.0324, 0.0649]), loss.mean(): 46.316219329833984
第114次迭代, 权重w: tensor([3.1799, 6.3598]), (0.01 * w.grad): tensor([0.0321, 0.0642]), loss.mean(): 45.79252624511719
第115次迭代, 权重w: tensor([3.1481, 6.2962]), (0.01 * w.grad): tensor([0.0318, 0.0636]), loss.mean(): 45.279258728027344
第116次迭代, 权重w: tensor([3.1166, 6.2332]), (0.01 * w.grad): tensor([0.0315, 0.0630]), loss.mean(): 44.77619934082031
第117次迭代, 权重w: tensor([3.0854, 6.1709]), (0.01 * w.grad): tensor([0.0312, 0.0623]), loss.mean(): 44.28315353393555
第118次迭代, 权重w: tensor([3.0546, 6.1092]), (0.01 * w.grad): tensor([0.0309, 0.0617]), loss.mean(): 43.79991912841797
第119次迭代, 权重w: tensor([3.0240, 6.0481]), (0.01 * w.grad): tensor([0.0305, 0.0611]), loss.mean(): 43.32630157470703
第120次迭代, 权重w: tensor([2.9938, 5.9876]), (0.01 * w.grad): tensor([0.0302, 0.0605]), loss.mean(): 42.86210632324219
第121次迭代, 权重w: tensor([2.9639, 5.9277]), (0.01 * w.grad): tensor([0.0299, 0.0599]), loss.mean(): 42.40715026855469
第122次迭代, 权重w: tensor([2.9342, 5.8685]), (0.01 * w.grad): tensor([0.0296, 0.0593]), loss.mean(): 41.96125030517578
第123次迭代, 权重w: tensor([2.9049, 5.8098]), (0.01 * w.grad): tensor([0.0293, 0.0587]), loss.mean(): 41.52421569824219
第124次迭代, 权重w: tensor([2.8758, 5.7517]), (0.01 * w.grad): tensor([0.0290, 0.0581]), loss.mean(): 41.09588623046875
第125次迭代, 权重w: tensor([2.8471, 5.6942]), (0.01 * w.grad): tensor([0.0288, 0.0575]), loss.mean(): 40.67607879638672
第126次迭代, 权重w: tensor([2.8186, 5.6372]), (0.01 * w.grad): tensor([0.0285, 0.0569]), loss.mean(): 40.264625549316406
第127次迭代, 权重w: tensor([2.7904, 5.5808]), (0.01 * w.grad): tensor([0.0282, 0.0564]), loss.mean(): 39.861358642578125
第128次迭代, 权重w: tensor([2.7625, 5.5250]), (0.01 * w.grad): tensor([0.0279, 0.0558]), loss.mean(): 39.46611785888672
第129次迭代, 权重w: tensor([2.7349, 5.4698]), (0.01 * w.grad): tensor([0.0276, 0.0553]), loss.mean(): 39.07874298095703
第130次迭代, 权重w: tensor([2.7075, 5.4151]), (0.01 * w.grad): tensor([0.0273, 0.0547]), loss.mean(): 38.699073791503906
第131次迭代, 权重w: tensor([2.6805, 5.3609]), (0.01 * w.grad): tensor([0.0271, 0.0542]), loss.mean(): 38.326961517333984
第132次迭代, 权重w: tensor([2.6537, 5.3073]), (0.01 * w.grad): tensor([0.0268, 0.0536]), loss.mean(): 37.96225357055664
第133次迭代, 权重w: tensor([2.6271, 5.2543]), (0.01 * w.grad): tensor([0.0265, 0.0531]), loss.mean(): 37.60480499267578
第134次迭代, 权重w: tensor([2.6009, 5.2017]), (0.01 * w.grad): tensor([0.0263, 0.0525]), loss.mean(): 37.25446701049805
第135次迭代, 权重w: tensor([2.5748, 5.1497]), (0.01 * w.grad): tensor([0.0260, 0.0520]), loss.mean(): 36.911102294921875
第136次迭代, 权重w: tensor([2.5491, 5.0982]), (0.01 * w.grad): tensor([0.0257, 0.0515]), loss.mean(): 36.57456970214844
第137次迭代, 权重w: tensor([2.5236, 5.0472]), (0.01 * w.grad): tensor([0.0255, 0.0510]), loss.mean(): 36.24473571777344
第138次迭代, 权重w: tensor([2.4984, 4.9967]), (0.01 * w.grad): tensor([0.0252, 0.0505]), loss.mean(): 35.92146682739258
第139次迭代, 权重w: tensor([2.4734, 4.9468]), (0.01 * w.grad): tensor([0.0250, 0.0500]), loss.mean(): 35.60462951660156
第140次迭代, 权重w: tensor([2.4487, 4.8973]), (0.01 * w.grad): tensor([0.0247, 0.0495]), loss.mean(): 35.294097900390625
第141次迭代, 权重w: tensor([2.4242, 4.8483]), (0.01 * w.grad): tensor([0.0245, 0.0490]), loss.mean(): 34.989742279052734
第142次迭代, 权重w: tensor([2.3999, 4.7998]), (0.01 * w.grad): tensor([0.0242, 0.0485]), loss.mean(): 34.69144821166992
第143次迭代, 权重w: tensor([2.3759, 4.7518]), (0.01 * w.grad): tensor([0.0240, 0.0480]), loss.mean(): 34.39908981323242
第144次迭代, 权重w: tensor([2.3522, 4.7043]), (0.01 * w.grad): tensor([0.0238, 0.0475]), loss.mean(): 34.112548828125
第145次迭代, 权重w: tensor([2.3286, 4.6573]), (0.01 * w.grad): tensor([0.0235, 0.0470]), loss.mean(): 33.83170700073242
第146次迭代, 权重w: tensor([2.3054, 4.6107]), (0.01 * w.grad): tensor([0.0233, 0.0466]), loss.mean(): 33.55645751953125
第147次迭代, 权重w: tensor([2.2823, 4.5646]), (0.01 * w.grad): tensor([0.0231, 0.0461]), loss.mean(): 33.28668212890625
第148次迭代, 权重w: tensor([2.2595, 4.5190]), (0.01 * w.grad): tensor([0.0228, 0.0456]), loss.mean(): 33.022274017333984
第149次迭代, 权重w: tensor([2.2369, 4.4738]), (0.01 * w.grad): tensor([0.0226, 0.0452]), loss.mean(): 32.76313400268555
第150次迭代, 权重w: tensor([2.2145, 4.4290]), (0.01 * w.grad): tensor([0.0224, 0.0447]), loss.mean(): 32.50914764404297
第151次迭代, 权重w: tensor([2.1924, 4.3847]), (0.01 * w.grad): tensor([0.0221, 0.0443]), loss.mean(): 32.260215759277344
第152次迭代, 权重w: tensor([2.1704, 4.3409]), (0.01 * w.grad): tensor([0.0219, 0.0438]), loss.mean(): 32.0162353515625
第153次迭代, 权重w: tensor([2.1487, 4.2975]), (0.01 * w.grad): tensor([0.0217, 0.0434]), loss.mean(): 31.77711296081543
第154次迭代, 权重w: tensor([2.1273, 4.2545]), (0.01 * w.grad): tensor([0.0215, 0.0430]), loss.mean(): 31.542749404907227
第155次迭代, 权重w: tensor([2.1060, 4.2120]), (0.01 * w.grad): tensor([0.0213, 0.0425]), loss.mean(): 31.313047409057617
第156次迭代, 权重w: tensor([2.0849, 4.1698]), (0.01 * w.grad): tensor([0.0211, 0.0421]), loss.mean(): 31.087915420532227
第157次迭代, 权重w: tensor([2.0641, 4.1281]), (0.01 * w.grad): tensor([0.0208, 0.0417]), loss.mean(): 30.867267608642578
第158次迭代, 权重w: tensor([2.0434, 4.0869]), (0.01 * w.grad): tensor([0.0206, 0.0413]), loss.mean(): 30.65100860595703
第159次迭代, 权重w: tensor([2.0230, 4.0460]), (0.01 * w.grad): tensor([0.0204, 0.0409]), loss.mean(): 30.43905258178711
第160次迭代, 权重w: tensor([2.0028, 4.0055]), (0.01 * w.grad): tensor([0.0202, 0.0405]), loss.mean(): 30.231313705444336
第161次迭代, 权重w: tensor([1.9827, 3.9655]), (0.01 * w.grad): tensor([0.0200, 0.0401]), loss.mean(): 30.027711868286133
第162次迭代, 权重w: tensor([1.9629, 3.9258]), (0.01 * w.grad): tensor([0.0198, 0.0397]), loss.mean(): 29.82815933227539
第163次迭代, 权重w: tensor([1.9433, 3.8866]), (0.01 * w.grad): tensor([0.0196, 0.0393]), loss.mean(): 29.632579803466797
第164次迭代, 权重w: tensor([1.9239, 3.8477]), (0.01 * w.grad): tensor([0.0194, 0.0389]), loss.mean(): 29.44089126586914
第165次迭代, 权重w: tensor([1.9046, 3.8092]), (0.01 * w.grad): tensor([0.0192, 0.0385]), loss.mean(): 29.25301742553711
第166次迭代, 权重w: tensor([1.8856, 3.7711]), (0.01 * w.grad): tensor([0.0190, 0.0381]), loss.mean(): 29.068885803222656
第167次迭代, 权重w: tensor([1.8667, 3.7334]), (0.01 * w.grad): tensor([0.0189, 0.0377]), loss.mean(): 28.888412475585938
第168次迭代, 权重w: tensor([1.8480, 3.6961]), (0.01 * w.grad): tensor([0.0187, 0.0373]), loss.mean(): 28.711532592773438
第169次迭代, 权重w: tensor([1.8296, 3.6591]), (0.01 * w.grad): tensor([0.0185, 0.0370]), loss.mean(): 28.53817367553711
第170次迭代, 权重w: tensor([1.8113, 3.6225]), (0.01 * w.grad): tensor([0.0183, 0.0366]), loss.mean(): 28.368263244628906
第171次迭代, 权重w: tensor([1.7932, 3.5863]), (0.01 * w.grad): tensor([0.0181, 0.0362]), loss.mean(): 28.201736450195312
第172次迭代, 权重w: tensor([1.7752, 3.5504]), (0.01 * w.grad): tensor([0.0179, 0.0359]), loss.mean(): 28.03852081298828
第173次迭代, 权重w: tensor([1.7575, 3.5149]), (0.01 * w.grad): tensor([0.0178, 0.0355]), loss.mean(): 27.878555297851562
第174次迭代, 权重w: tensor([1.7399, 3.4798]), (0.01 * w.grad): tensor([0.0176, 0.0351]), loss.mean(): 27.721771240234375
第175次迭代, 权重w: tensor([1.7225, 3.4450]), (0.01 * w.grad): tensor([0.0174, 0.0348]), loss.mean(): 27.56810760498047
第176次迭代, 权重w: tensor([1.7053, 3.4105]), (0.01 * w.grad): tensor([0.0172, 0.0344]), loss.mean(): 27.417503356933594
第177次迭代, 权重w: tensor([1.6882, 3.3764]), (0.01 * w.grad): tensor([0.0171, 0.0341]), loss.mean(): 27.269893646240234
第178次迭代, 权重w: tensor([1.6713, 3.3427]), (0.01 * w.grad): tensor([0.0169, 0.0338]), loss.mean(): 27.12522315979004
第179次迭代, 权重w: tensor([1.6546, 3.3093]), (0.01 * w.grad): tensor([0.0167, 0.0334]), loss.mean(): 26.98343276977539
第180次迭代, 权重w: tensor([1.6381, 3.2762]), (0.01 * w.grad): tensor([0.0165, 0.0331]), loss.mean(): 26.84446144104004
第181次迭代, 权重w: tensor([1.6217, 3.2434]), (0.01 * w.grad): tensor([0.0164, 0.0328]), loss.mean(): 26.708255767822266
第182次迭代, 权重w: tensor([1.6055, 3.2110]), (0.01 * w.grad): tensor([0.0162, 0.0324]), loss.mean(): 26.57476234436035
第183次迭代, 权重w: tensor([1.5894, 3.1789]), (0.01 * w.grad): tensor([0.0161, 0.0321]), loss.mean(): 26.443923950195312
第184次迭代, 权重w: tensor([1.5735, 3.1471]), (0.01 * w.grad): tensor([0.0159, 0.0318]), loss.mean(): 26.315689086914062
第185次迭代, 权重w: tensor([1.5578, 3.1156]), (0.01 * w.grad): tensor([0.0157, 0.0315]), loss.mean(): 26.190006256103516
第186次迭代, 权重w: tensor([1.5422, 3.0844]), (0.01 * w.grad): tensor([0.0156, 0.0312]), loss.mean(): 26.06682586669922
第187次迭代, 权重w: tensor([1.5268, 3.0536]), (0.01 * w.grad): tensor([0.0154, 0.0308]), loss.mean(): 25.94609832763672
第188次迭代, 权重w: tensor([1.5115, 3.0231]), (0.01 * w.grad): tensor([0.0153, 0.0305]), loss.mean(): 25.827770233154297
第189次迭代, 权重w: tensor([1.4964, 2.9928]), (0.01 * w.grad): tensor([0.0151, 0.0302]), loss.mean(): 25.711795806884766
第190次迭代, 权重w: tensor([1.4814, 2.9629]), (0.01 * w.grad): tensor([0.0150, 0.0299]), loss.mean(): 25.598133087158203
第191次迭代, 权重w: tensor([1.4666, 2.9333]), (0.01 * w.grad): tensor([0.0148, 0.0296]), loss.mean(): 25.48672866821289
第192次迭代, 权重w: tensor([1.4520, 2.9039]), (0.01 * w.grad): tensor([0.0147, 0.0293]), loss.mean(): 25.377544403076172
第193次迭代, 权重w: tensor([1.4374, 2.8749]), (0.01 * w.grad): tensor([0.0145, 0.0290]), loss.mean(): 25.270530700683594
第194次迭代, 权重w: tensor([1.4231, 2.8461]), (0.01 * w.grad): tensor([0.0144, 0.0287]), loss.mean(): 25.165647506713867
第195次迭代, 权重w: tensor([1.4088, 2.8177]), (0.01 * w.grad): tensor([0.0142, 0.0285]), loss.mean(): 25.062850952148438
第196次迭代, 权重w: tensor([1.3948, 2.7895]), (0.01 * w.grad): tensor([0.0141, 0.0282]), loss.mean(): 24.962100982666016
第197次迭代, 权重w: tensor([1.3808, 2.7616]), (0.01 * w.grad): tensor([0.0139, 0.0279]), loss.mean(): 24.863353729248047
第198次迭代, 权重w: tensor([1.3670, 2.7340]), (0.01 * w.grad): tensor([0.0138, 0.0276]), loss.mean(): 24.76657485961914
第199次迭代, 权重w: tensor([1.3533, 2.7067]), (0.01 * w.grad): tensor([0.0137, 0.0273]), loss.mean(): 24.67171859741211
第200次迭代, 权重w: tensor([1.3398, 2.6796]), (0.01 * w.grad): tensor([0.0135, 0.0271]), loss.mean(): 24.578750610351562
第201次迭代, 权重w: tensor([1.3264, 2.6528]), (0.01 * w.grad): tensor([0.0134, 0.0268]), loss.mean(): 24.487632751464844
第202次迭代, 权重w: tensor([1.3131, 2.6263]), (0.01 * w.grad): tensor([0.0133, 0.0265]), loss.mean(): 24.398330688476562
第203次迭代, 权重w: tensor([1.3000, 2.6000]), (0.01 * w.grad): tensor([0.0131, 0.0263]), loss.mean(): 24.310802459716797
第204次迭代, 权重w: tensor([1.2870, 2.5740]), (0.01 * w.grad): tensor([0.0130, 0.0260]), loss.mean(): 24.225017547607422
第205次迭代, 权重w: tensor([1.2741, 2.5483]), (0.01 * w.grad): tensor([0.0129, 0.0257]), loss.mean(): 24.140941619873047
第206次迭代, 权重w: tensor([1.2614, 2.5228]), (0.01 * w.grad): tensor([0.0127, 0.0255]), loss.mean(): 24.058536529541016
第207次迭代, 权重w: tensor([1.2488, 2.4976]), (0.01 * w.grad): tensor([0.0126, 0.0252]), loss.mean(): 23.977771759033203
第208次迭代, 权重w: tensor([1.2363, 2.4726]), (0.01 * w.grad): tensor([0.0125, 0.0250]), loss.mean(): 23.89861297607422
第209次迭代, 权重w: tensor([1.2239, 2.4479]), (0.01 * w.grad): tensor([0.0124, 0.0247]), loss.mean(): 23.821029663085938
第210次迭代, 权重w: tensor([1.2117, 2.4234]), (0.01 * w.grad): tensor([0.0122, 0.0245]), loss.mean(): 23.744991302490234
第211次迭代, 权重w: tensor([1.1996, 2.3991]), (0.01 * w.grad): tensor([0.0121, 0.0242]), loss.mean(): 23.670467376708984
第212次迭代, 权重w: tensor([1.1876, 2.3751]), (0.01 * w.grad): tensor([0.0120, 0.0240]), loss.mean(): 23.597423553466797
第213次迭代, 权重w: tensor([1.1757, 2.3514]), (0.01 * w.grad): tensor([0.0119, 0.0238]), loss.mean(): 23.525836944580078
第214次迭代, 权重w: tensor([1.1639, 2.3279]), (0.01 * w.grad): tensor([0.0118, 0.0235]), loss.mean(): 23.455673217773438
第215次迭代, 权重w: tensor([1.1523, 2.3046]), (0.01 * w.grad): tensor([0.0116, 0.0233]), loss.mean(): 23.386903762817383
第216次迭代, 权重w: tensor([1.1408, 2.2816]), (0.01 * w.grad): tensor([0.0115, 0.0230]), loss.mean(): 23.319503784179688
第217次迭代, 权重w: tensor([1.1294, 2.2587]), (0.01 * w.grad): tensor([0.0114, 0.0228]), loss.mean(): 23.25344467163086
第218次迭代, 权重w: tensor([1.1181, 2.2362]), (0.01 * w.grad): tensor([0.0113, 0.0226]), loss.mean(): 23.188701629638672
第219次迭代, 权重w: tensor([1.1069, 2.2138]), (0.01 * w.grad): tensor([0.0112, 0.0224]), loss.mean(): 23.125247955322266
第220次迭代, 权重w: tensor([1.0958, 2.1917]), (0.01 * w.grad): tensor([0.0111, 0.0221]), loss.mean(): 23.063053131103516
第221次迭代, 权重w: tensor([1.0849, 2.1697]), (0.01 * w.grad): tensor([0.0110, 0.0219]), loss.mean(): 23.002099990844727
第222次迭代, 权重w: tensor([1.0740, 2.1480]), (0.01 * w.grad): tensor([0.0108, 0.0217]), loss.mean(): 22.942358016967773
第223次迭代, 权重w: tensor([1.0633, 2.1266]), (0.01 * w.grad): tensor([0.0107, 0.0215]), loss.mean(): 22.883804321289062
第224次迭代, 权重w: tensor([1.0526, 2.1053]), (0.01 * w.grad): tensor([0.0106, 0.0213]), loss.mean(): 22.826416015625
第225次迭代, 权重w: tensor([1.0421, 2.0842]), (0.01 * w.grad): tensor([0.0105, 0.0211]), loss.mean(): 22.770172119140625
第226次迭代, 权重w: tensor([1.0317, 2.0634]), (0.01 * w.grad): tensor([0.0104, 0.0208]), loss.mean(): 22.715045928955078
第227次迭代, 权重w: tensor([1.0214, 2.0428]), (0.01 * w.grad): tensor([0.0103, 0.0206]), loss.mean(): 22.661014556884766
第228次迭代, 权重w: tensor([1.0112, 2.0223]), (0.01 * w.grad): tensor([0.0102, 0.0204]), loss.mean(): 22.608060836791992
第229次迭代, 权重w: tensor([1.0011, 2.0021]), (0.01 * w.grad): tensor([0.0101, 0.0202]), loss.mean(): 22.55615997314453
第230次迭代, 权重w: tensor([0.9910, 1.9821]), (0.01 * w.grad): tensor([0.0100, 0.0200]), loss.mean(): 22.505292892456055
第231次迭代, 权重w: tensor([0.9811, 1.9623]), (0.01 * w.grad): tensor([0.0099, 0.0198]), loss.mean(): 22.4554386138916
第232次迭代, 权重w: tensor([0.9713, 1.9427]), (0.01 * w.grad): tensor([0.0098, 0.0196]), loss.mean(): 22.406574249267578
第233次迭代, 权重w: tensor([0.9616, 1.9232]), (0.01 * w.grad): tensor([0.0097, 0.0194]), loss.mean(): 22.358684539794922
第234次迭代, 权重w: tensor([0.9520, 1.9040]), (0.01 * w.grad): tensor([0.0096, 0.0192]), loss.mean(): 22.311744689941406
第235次迭代, 权重w: tensor([0.9425, 1.8850]), (0.01 * w.grad): tensor([0.0095, 0.0190]), loss.mean(): 22.265743255615234
第236次迭代, 权重w: tensor([0.9331, 1.8661]), (0.01 * w.grad): tensor([0.0094, 0.0188]), loss.mean(): 22.220653533935547
第237次迭代, 权重w: tensor([0.9237, 1.8474]), (0.01 * w.grad): tensor([0.0093, 0.0187]), loss.mean(): 22.176464080810547
第238次迭代, 权重w: tensor([0.9145, 1.8290]), (0.01 * w.grad): tensor([0.0092, 0.0185]), loss.mean(): 22.13315200805664
第239次迭代, 权重w: tensor([0.9053, 1.8107]), (0.01 * w.grad): tensor([0.0091, 0.0183]), loss.mean(): 22.090702056884766
第240次迭代, 权重w: tensor([0.8963, 1.7926]), (0.01 * w.grad): tensor([0.0091, 0.0181]), loss.mean(): 22.049097061157227
第241次迭代, 权重w: tensor([0.8873, 1.7746]), (0.01 * w.grad): tensor([0.0090, 0.0179]), loss.mean(): 22.008319854736328
第242次迭代, 权重w: tensor([0.8784, 1.7569]), (0.01 * w.grad): tensor([0.0089, 0.0177]), loss.mean(): 21.968353271484375
第243次迭代, 权重w: tensor([0.8697, 1.7393]), (0.01 * w.grad): tensor([0.0088, 0.0176]), loss.mean(): 21.929183959960938
第244次迭代, 权重w: tensor([0.8610, 1.7219]), (0.01 * w.grad): tensor([0.0087, 0.0174]), loss.mean(): 21.890792846679688
第245次迭代, 权重w: tensor([0.8524, 1.7047]), (0.01 * w.grad): tensor([0.0086, 0.0172]), loss.mean(): 21.853166580200195
第246次迭代, 权重w: tensor([0.8438, 1.6877]), (0.01 * w.grad): tensor([0.0085, 0.0170]), loss.mean(): 21.816287994384766
第247次迭代, 权重w: tensor([0.8354, 1.6708]), (0.01 * w.grad): tensor([0.0084, 0.0169]), loss.mean(): 21.78014373779297
第248次迭代, 权重w: tensor([0.8270, 1.6541]), (0.01 * w.grad): tensor([0.0084, 0.0167]), loss.mean(): 21.744720458984375
第249次迭代, 权重w: tensor([0.8188, 1.6375]), (0.01 * w.grad): tensor([0.0083, 0.0165]), loss.mean(): 21.709999084472656
第250次迭代, 权重w: tensor([0.8106, 1.6212]), (0.01 * w.grad): tensor([0.0082, 0.0164]), loss.mean(): 21.67597007751465
第251次迭代, 权重w: tensor([0.8025, 1.6050]), (0.01 * w.grad): tensor([0.0081, 0.0162]), loss.mean(): 21.64261817932129
第252次迭代, 权重w: tensor([0.7945, 1.5889]), (0.01 * w.grad): tensor([0.0080, 0.0160]), loss.mean(): 21.60993194580078
第253次迭代, 权重w: tensor([0.7865, 1.5730]), (0.01 * w.grad): tensor([0.0079, 0.0159]), loss.mean(): 21.577892303466797
第254次迭代, 权重w: tensor([0.7786, 1.5573]), (0.01 * w.grad): tensor([0.0079, 0.0157]), loss.mean(): 21.546493530273438
第255次迭代, 权重w: tensor([0.7709, 1.5417]), (0.01 * w.grad): tensor([0.0078, 0.0156]), loss.mean(): 21.515716552734375
第256次迭代, 权重w: tensor([0.7631, 1.5263]), (0.01 * w.grad): tensor([0.0077, 0.0154]), loss.mean(): 21.485553741455078
第257次迭代, 权重w: tensor([0.7555, 1.5110]), (0.01 * w.grad): tensor([0.0076, 0.0153]), loss.mean(): 21.45599365234375
第258次迭代, 权重w: tensor([0.7480, 1.4959]), (0.01 * w.grad): tensor([0.0076, 0.0151]), loss.mean(): 21.427017211914062
第259次迭代, 权重w: tensor([0.7405, 1.4810]), (0.01 * w.grad): tensor([0.0075, 0.0150]), loss.mean(): 21.39862060546875
第260次迭代, 权重w: tensor([0.7331, 1.4662]), (0.01 * w.grad): tensor([0.0074, 0.0148]), loss.mean(): 21.37078857421875
第261次迭代, 权重w: tensor([0.7257, 1.4515]), (0.01 * w.grad): tensor([0.0073, 0.0147]), loss.mean(): 21.343509674072266
第262次迭代, 权重w: tensor([0.7185, 1.4370]), (0.01 * w.grad): tensor([0.0073, 0.0145]), loss.mean(): 21.3167724609375
第263次迭代, 权重w: tensor([0.7113, 1.4226]), (0.01 * w.grad): tensor([0.0072, 0.0144]), loss.mean(): 21.290569305419922
第264次迭代, 权重w: tensor([0.7042, 1.4084]), (0.01 * w.grad): tensor([0.0071, 0.0142]), loss.mean(): 21.2648868560791
第265次迭代, 权重w: tensor([0.6972, 1.3943]), (0.01 * w.grad): tensor([0.0070, 0.0141]), loss.mean(): 21.239715576171875
第266次迭代, 权重w: tensor([0.6902, 1.3804]), (0.01 * w.grad): tensor([0.0070, 0.0139]), loss.mean(): 21.215045928955078
第267次迭代, 权重w: tensor([0.6833, 1.3666]), (0.01 * w.grad): tensor([0.0069, 0.0138]), loss.mean(): 21.190866470336914
第268次迭代, 权重w: tensor([0.6764, 1.3529]), (0.01 * w.grad): tensor([0.0068, 0.0137]), loss.mean(): 21.16716766357422
第269次迭代, 权重w: tensor([0.6697, 1.3394]), (0.01 * w.grad): tensor([0.0068, 0.0135]), loss.mean(): 21.14394187927246
第270次迭代, 权重w: tensor([0.6630, 1.3260]), (0.01 * w.grad): tensor([0.0067, 0.0134]), loss.mean(): 21.121177673339844
第271次迭代, 权重w: tensor([0.6564, 1.3127]), (0.01 * w.grad): tensor([0.0066, 0.0133]), loss.mean(): 21.098865509033203
第272次迭代, 权重w: tensor([0.6498, 1.2996]), (0.01 * w.grad): tensor([0.0066, 0.0131]), loss.mean(): 21.07699966430664
第273次迭代, 权重w: tensor([0.6433, 1.2866]), (0.01 * w.grad): tensor([0.0065, 0.0130]), loss.mean(): 21.055564880371094
第274次迭代, 权重w: tensor([0.6369, 1.2737]), (0.01 * w.grad): tensor([0.0064, 0.0129]), loss.mean(): 21.034561157226562
第275次迭代, 权重w: tensor([0.6305, 1.2610]), (0.01 * w.grad): tensor([0.0064, 0.0127]), loss.mean(): 21.013973236083984
第276次迭代, 权重w: tensor([0.6242, 1.2484]), (0.01 * w.grad): tensor([0.0063, 0.0126]), loss.mean(): 20.993793487548828
第277次迭代, 权重w: tensor([0.6179, 1.2359]), (0.01 * w.grad): tensor([0.0062, 0.0125]), loss.mean(): 20.974018096923828
第278次迭代, 权重w: tensor([0.6118, 1.2235]), (0.01 * w.grad): tensor([0.0062, 0.0124]), loss.mean(): 20.954635620117188
第279次迭代, 权重w: tensor([0.6056, 1.2113]), (0.01 * w.grad): tensor([0.0061, 0.0122]), loss.mean(): 20.935638427734375
第280次迭代, 权重w: tensor([0.5996, 1.1992]), (0.01 * w.grad): tensor([0.0061, 0.0121]), loss.mean(): 20.91701889038086
第281次迭代, 权重w: tensor([0.5936, 1.1872]), (0.01 * w.grad): tensor([0.0060, 0.0120]), loss.mean(): 20.89876937866211
第282次迭代, 权重w: tensor([0.5877, 1.1753]), (0.01 * w.grad): tensor([0.0059, 0.0119]), loss.mean(): 20.880884170532227
第283次迭代, 权重w: tensor([0.5818, 1.1636]), (0.01 * w.grad): tensor([0.0059, 0.0118]), loss.mean(): 20.863353729248047
第284次迭代, 权重w: tensor([0.5760, 1.1519]), (0.01 * w.grad): tensor([0.0058, 0.0116]), loss.mean(): 20.846174240112305
第285次迭代, 权重w: tensor([0.5702, 1.1404]), (0.01 * w.grad): tensor([0.0058, 0.0115]), loss.mean(): 20.829334259033203
第286次迭代, 权重w: tensor([0.5645, 1.1290]), (0.01 * w.grad): tensor([0.0057, 0.0114]), loss.mean(): 20.81283187866211
第287次迭代, 权重w: tensor([0.5589, 1.1177]), (0.01 * w.grad): tensor([0.0056, 0.0113]), loss.mean(): 20.796655654907227
第288次迭代, 权重w: tensor([0.5533, 1.1065]), (0.01 * w.grad): tensor([0.0056, 0.0112]), loss.mean(): 20.78080177307129
第289次迭代, 权重w: tensor([0.5477, 1.0955]), (0.01 * w.grad): tensor([0.0055, 0.0111]), loss.mean(): 20.7652645111084
第290次迭代, 权重w: tensor([0.5423, 1.0845]), (0.01 * w.grad): tensor([0.0055, 0.0110]), loss.mean(): 20.75003433227539
第291次迭代, 权重w: tensor([0.5368, 1.0737]), (0.01 * w.grad): tensor([0.0054, 0.0108]), loss.mean(): 20.735111236572266
第292次迭代, 权重w: tensor([0.5315, 1.0629]), (0.01 * w.grad): tensor([0.0054, 0.0107]), loss.mean(): 20.720481872558594
第293次迭代, 权重w: tensor([0.5262, 1.0523]), (0.01 * w.grad): tensor([0.0053, 0.0106]), loss.mean(): 20.70614242553711
第294次迭代, 权重w: tensor([0.5209, 1.0418]), (0.01 * w.grad): tensor([0.0053, 0.0105]), loss.mean(): 20.69209098815918
第295次迭代, 权重w: tensor([0.5157, 1.0314]), (0.01 * w.grad): tensor([0.0052, 0.0104]), loss.mean(): 20.67831802368164
第296次迭代, 权重w: tensor([0.5105, 1.0211]), (0.01 * w.grad): tensor([0.0052, 0.0103]), loss.mean(): 20.66482162475586
第297次迭代, 权重w: tensor([0.5054, 1.0108]), (0.01 * w.grad): tensor([0.0051, 0.0102]), loss.mean(): 20.651588439941406
第298次迭代, 权重w: tensor([0.5004, 1.0007]), (0.01 * w.grad): tensor([0.0051, 0.0101]), loss.mean(): 20.638622283935547
第299次迭代, 权重w: tensor([0.4954, 0.9907]), (0.01 * w.grad): tensor([0.0050, 0.0100]), loss.mean(): 20.62591552734375
第300次迭代, 权重w: tensor([0.4904, 0.9808]), (0.01 * w.grad): tensor([0.0050, 0.0099]), loss.mean(): 20.613460540771484
第301次迭代, 权重w: tensor([0.4855, 0.9710]), (0.01 * w.grad): tensor([0.0049, 0.0098]), loss.mean(): 20.60125160217285
第302次迭代, 权重w: tensor([0.4806, 0.9613]), (0.01 * w.grad): tensor([0.0049, 0.0097]), loss.mean(): 20.58928680419922
第303次迭代, 权重w: tensor([0.4758, 0.9517]), (0.01 * w.grad): tensor([0.0048, 0.0096]), loss.mean(): 20.577560424804688
第304次迭代, 权重w: tensor([0.4711, 0.9422]), (0.01 * w.grad): tensor([0.0048, 0.0095]), loss.mean(): 20.56606674194336
第305次迭代, 权重w: tensor([0.4664, 0.9327]), (0.01 * w.grad): tensor([0.0047, 0.0094]), loss.mean(): 20.55480194091797
第306次迭代, 权重w: tensor([0.4617, 0.9234]), (0.01 * w.grad): tensor([0.0047, 0.0093]), loss.mean(): 20.54376220703125
第307次迭代, 权重w: tensor([0.4571, 0.9142]), (0.01 * w.grad): tensor([0.0046, 0.0092]), loss.mean(): 20.532939910888672
第308次迭代, 权重w: tensor([0.4525, 0.9050]), (0.01 * w.grad): tensor([0.0046, 0.0091]), loss.mean(): 20.522335052490234
第309次迭代, 权重w: tensor([0.4480, 0.8960]), (0.01 * w.grad): tensor([0.0045, 0.0091]), loss.mean(): 20.511940002441406
第310次迭代, 权重w: tensor([0.4435, 0.8870]), (0.01 * w.grad): tensor([0.0045, 0.0090]), loss.mean(): 20.501752853393555
第311次迭代, 权重w: tensor([0.4391, 0.8782]), (0.01 * w.grad): tensor([0.0044, 0.0089]), loss.mean(): 20.49176788330078
第312次迭代, 权重w: tensor([0.4347, 0.8694]), (0.01 * w.grad): tensor([0.0044, 0.0088]), loss.mean(): 20.48198127746582
第313次迭代, 权重w: tensor([0.4303, 0.8607]), (0.01 * w.grad): tensor([0.0043, 0.0087]), loss.mean(): 20.472389221191406
第314次迭代, 权重w: tensor([0.4260, 0.8521]), (0.01 * w.grad): tensor([0.0043, 0.0086]), loss.mean(): 20.462989807128906
第315次迭代, 权重w: tensor([0.4218, 0.8436]), (0.01 * w.grad): tensor([0.0043, 0.0085]), loss.mean(): 20.453777313232422
第316次迭代, 权重w: tensor([0.4176, 0.8351]), (0.01 * w.grad): tensor([0.0042, 0.0084]), loss.mean(): 20.444746017456055
第317次迭代, 权重w: tensor([0.4134, 0.8268]), (0.01 * w.grad): tensor([0.0042, 0.0084]), loss.mean(): 20.435895919799805
第318次迭代, 权重w: tensor([0.4093, 0.8185]), (0.01 * w.grad): tensor([0.0041, 0.0083]), loss.mean(): 20.427221298217773
第319次迭代, 权重w: tensor([0.4052, 0.8103]), (0.01 * w.grad): tensor([0.0041, 0.0082]), loss.mean(): 20.418720245361328
第320次迭代, 权重w: tensor([0.4011, 0.8022]), (0.01 * w.grad): tensor([0.0041, 0.0081]), loss.mean(): 20.41038703918457
第321次迭代, 权重w: tensor([0.3971, 0.7942]), (0.01 * w.grad): tensor([0.0040, 0.0080]), loss.mean(): 20.4022216796875
第322次迭代, 权重w: tensor([0.3931, 0.7863]), (0.01 * w.grad): tensor([0.0040, 0.0079]), loss.mean(): 20.394214630126953
第323次迭代, 权重w: tensor([0.3892, 0.7784]), (0.01 * w.grad): tensor([0.0039, 0.0079]), loss.mean(): 20.386371612548828
第324次迭代, 权重w: tensor([0.3853, 0.7706]), (0.01 * w.grad): tensor([0.0039, 0.0078]), loss.mean(): 20.378681182861328
第325次迭代, 权重w: tensor([0.3815, 0.7629]), (0.01 * w.grad): tensor([0.0039, 0.0077]), loss.mean(): 20.37114715576172
第326次迭代, 权重w: tensor([0.3776, 0.7553]), (0.01 * w.grad): tensor([0.0038, 0.0076]), loss.mean(): 20.36376190185547
第327次迭代, 权重w: tensor([0.3739, 0.7477]), (0.01 * w.grad): tensor([0.0038, 0.0076]), loss.mean(): 20.356521606445312
第328次迭代, 权重w: tensor([0.3701, 0.7402]), (0.01 * w.grad): tensor([0.0037, 0.0075]), loss.mean(): 20.34942626953125
第329次迭代, 权重w: tensor([0.3664, 0.7328]), (0.01 * w.grad): tensor([0.0037, 0.0074]), loss.mean(): 20.34247398376465
第330次迭代, 权重w: tensor([0.3628, 0.7255]), (0.01 * w.grad): tensor([0.0037, 0.0073]), loss.mean(): 20.33565902709961
第331次迭代, 权重w: tensor([0.3591, 0.7183]), (0.01 * w.grad): tensor([0.0036, 0.0073]), loss.mean(): 20.3289794921875
第332次迭代, 权重w: tensor([0.3555, 0.7111]), (0.01 * w.grad): tensor([0.0036, 0.0072]), loss.mean(): 20.322433471679688
第333次迭代, 权重w: tensor([0.3520, 0.7040]), (0.01 * w.grad): tensor([0.0036, 0.0071]), loss.mean(): 20.316015243530273
第334次迭代, 权重w: tensor([0.3485, 0.6969]), (0.01 * w.grad): tensor([0.0035, 0.0070]), loss.mean(): 20.30972671508789
第335次迭代, 权重w: tensor([0.3450, 0.6900]), (0.01 * w.grad): tensor([0.0035, 0.0070]), loss.mean(): 20.30356216430664
第336次迭代, 权重w: tensor([0.3415, 0.6831]), (0.01 * w.grad): tensor([0.0034, 0.0069]), loss.mean(): 20.297523498535156
第337次迭代, 权重w: tensor([0.3381, 0.6762]), (0.01 * w.grad): tensor([0.0034, 0.0068]), loss.mean(): 20.291601181030273
第338次迭代, 权重w: tensor([0.3347, 0.6695]), (0.01 * w.grad): tensor([0.0034, 0.0068]), loss.mean(): 20.285799026489258
第339次迭代, 权重w: tensor([0.3314, 0.6628]), (0.01 * w.grad): tensor([0.0033, 0.0067]), loss.mean(): 20.280113220214844
第340次迭代, 权重w: tensor([0.3281, 0.6561]), (0.01 * w.grad): tensor([0.0033, 0.0066]), loss.mean(): 20.274538040161133
第341次迭代, 权重w: tensor([0.3248, 0.6496]), (0.01 * w.grad): tensor([0.0033, 0.0066]), loss.mean(): 20.269073486328125
第342次迭代, 权重w: tensor([0.3215, 0.6431]), (0.01 * w.grad): tensor([0.0032, 0.0065]), loss.mean(): 20.26371955871582
第343次迭代, 权重w: tensor([0.3183, 0.6367]), (0.01 * w.grad): tensor([0.0032, 0.0064]), loss.mean(): 20.258472442626953
第344次迭代, 权重w: tensor([0.3151, 0.6303]), (0.01 * w.grad): tensor([0.0032, 0.0064]), loss.mean(): 20.253328323364258
第345次迭代, 权重w: tensor([0.3120, 0.6240]), (0.01 * w.grad): tensor([0.0032, 0.0063]), loss.mean(): 20.248287200927734
第346次迭代, 权重w: tensor([0.3089, 0.6177]), (0.01 * w.grad): tensor([0.0031, 0.0062]), loss.mean(): 20.243345260620117
第347次迭代, 权重w: tensor([0.3058, 0.6116]), (0.01 * w.grad): tensor([0.0031, 0.0062]), loss.mean(): 20.238502502441406
第348次迭代, 权重w: tensor([0.3027, 0.6054]), (0.01 * w.grad): tensor([0.0031, 0.0061]), loss.mean(): 20.23375701904297
第349次迭代, 权重w: tensor([0.2997, 0.5994]), (0.01 * w.grad): tensor([0.0030, 0.0061]), loss.mean(): 20.22910499572754
第350次迭代, 权重w: tensor([0.2967, 0.5934]), (0.01 * w.grad): tensor([0.0030, 0.0060]), loss.mean(): 20.224546432495117
第351次迭代, 权重w: tensor([0.2937, 0.5875]), (0.01 * w.grad): tensor([0.0030, 0.0059]), loss.mean(): 20.220077514648438
第352次迭代, 权重w: tensor([0.2908, 0.5816]), (0.01 * w.grad): tensor([0.0029, 0.0059]), loss.mean(): 20.2156982421875
第353次迭代, 权重w: tensor([0.2879, 0.5758]), (0.01 * w.grad): tensor([0.0029, 0.0058]), loss.mean(): 20.211406707763672
第354次迭代, 权重w: tensor([0.2850, 0.5700]), (0.01 * w.grad): tensor([0.0029, 0.0058]), loss.mean(): 20.207199096679688
第355次迭代, 权重w: tensor([0.2822, 0.5643]), (0.01 * w.grad): tensor([0.0029, 0.0057]), loss.mean(): 20.203075408935547
第356次迭代, 权重w: tensor([0.2793, 0.5587]), (0.01 * w.grad): tensor([0.0028, 0.0056]), loss.mean(): 20.199033737182617
第357次迭代, 权重w: tensor([0.2765, 0.5531]), (0.01 * w.grad): tensor([0.0028, 0.0056]), loss.mean(): 20.195072174072266
第358次迭代, 权重w: tensor([0.2738, 0.5476]), (0.01 * w.grad): tensor([0.0028, 0.0055]), loss.mean(): 20.191192626953125
第359次迭代, 权重w: tensor([0.2710, 0.5421]), (0.01 * w.grad): tensor([0.0027, 0.0055]), loss.mean(): 20.18738555908203
第360次迭代, 权重w: tensor([0.2683, 0.5367]), (0.01 * w.grad): tensor([0.0027, 0.0054]), loss.mean(): 20.183658599853516
第361次迭代, 权重w: tensor([0.2656, 0.5313]), (0.01 * w.grad): tensor([0.0027, 0.0054]), loss.mean(): 20.180002212524414
第362次迭代, 权重w: tensor([0.2630, 0.5260]), (0.01 * w.grad): tensor([0.0027, 0.0053]), loss.mean(): 20.176422119140625
第363次迭代, 权重w: tensor([0.2604, 0.5207]), (0.01 * w.grad): tensor([0.0026, 0.0053]), loss.mean(): 20.172910690307617
第364次迭代, 权重w: tensor([0.2578, 0.5155]), (0.01 * w.grad): tensor([0.0026, 0.0052]), loss.mean(): 20.16946792602539
第365次迭代, 权重w: tensor([0.2552, 0.5104]), (0.01 * w.grad): tensor([0.0026, 0.0052]), loss.mean(): 20.166095733642578
第366次迭代, 权重w: tensor([0.2526, 0.5053]), (0.01 * w.grad): tensor([0.0026, 0.0051]), loss.mean(): 20.162792205810547
第367次迭代, 权重w: tensor([0.2501, 0.5002]), (0.01 * w.grad): tensor([0.0025, 0.0051]), loss.mean(): 20.1595516204834
第368次迭代, 权重w: tensor([0.2476, 0.4952]), (0.01 * w.grad): tensor([0.0025, 0.0050]), loss.mean(): 20.156375885009766
第369次迭代, 权重w: tensor([0.2451, 0.4902]), (0.01 * w.grad): tensor([0.0025, 0.0050]), loss.mean(): 20.15326499938965
第370次迭代, 权重w: tensor([0.2427, 0.4853]), (0.01 * w.grad): tensor([0.0025, 0.0049]), loss.mean(): 20.15021514892578
第371次迭代, 权重w: tensor([0.2402, 0.4805]), (0.01 * w.grad): tensor([0.0024, 0.0049]), loss.mean(): 20.14722442626953
第372次迭代, 权重w: tensor([0.2378, 0.4757]), (0.01 * w.grad): tensor([0.0024, 0.0048]), loss.mean(): 20.14429473876953
第373次迭代, 权重w: tensor([0.2355, 0.4709]), (0.01 * w.grad): tensor([0.0024, 0.0048]), loss.mean(): 20.14142417907715
第374次迭代, 权重w: tensor([0.2331, 0.4662]), (0.01 * w.grad): tensor([0.0024, 0.0047]), loss.mean(): 20.13861083984375
第375次迭代, 权重w: tensor([0.2308, 0.4616]), (0.01 * w.grad): tensor([0.0023, 0.0047]), loss.mean(): 20.135852813720703
第376次迭代, 权重w: tensor([0.2285, 0.4569]), (0.01 * w.grad): tensor([0.0023, 0.0046]), loss.mean(): 20.133148193359375
第377次迭代, 权重w: tensor([0.2262, 0.4524]), (0.01 * w.grad): tensor([0.0023, 0.0046]), loss.mean(): 20.1304988861084
第378次迭代, 权重w: tensor([0.2239, 0.4479]), (0.01 * w.grad): tensor([0.0023, 0.0045]), loss.mean(): 20.12790298461914
第379次迭代, 权重w: tensor([0.2217, 0.4434]), (0.01 * w.grad): tensor([0.0022, 0.0045]), loss.mean(): 20.125356674194336
第380次迭代, 权重w: tensor([0.2195, 0.4389]), (0.01 * w.grad): tensor([0.0022, 0.0044]), loss.mean(): 20.122861862182617
第381次迭代, 权重w: tensor([0.2173, 0.4345]), (0.01 * w.grad): tensor([0.0022, 0.0044]), loss.mean(): 20.12041664123535
第382次迭代, 权重w: tensor([0.2151, 0.4302]), (0.01 * w.grad): tensor([0.0022, 0.0043]), loss.mean(): 20.11802101135254
第383次迭代, 权重w: tensor([0.2130, 0.4259]), (0.01 * w.grad): tensor([0.0022, 0.0043]), loss.mean(): 20.115673065185547
第384次迭代, 权重w: tensor([0.2108, 0.4216]), (0.01 * w.grad): tensor([0.0021, 0.0043]), loss.mean(): 20.11336898803711
第385次迭代, 权重w: tensor([0.2087, 0.4174]), (0.01 * w.grad): tensor([0.0021, 0.0042]), loss.mean(): 20.111114501953125
第386次迭代, 权重w: tensor([0.2066, 0.4133]), (0.01 * w.grad): tensor([0.0021, 0.0042]), loss.mean(): 20.108901977539062
第387次迭代, 权重w: tensor([0.2046, 0.4091]), (0.01 * w.grad): tensor([0.0021, 0.0041]), loss.mean(): 20.106735229492188
第388次迭代, 权重w: tensor([0.2025, 0.4050]), (0.01 * w.grad): tensor([0.0020, 0.0041]), loss.mean(): 20.104612350463867
第389次迭代, 权重w: tensor([0.2005, 0.4010]), (0.01 * w.grad): tensor([0.0020, 0.0041]), loss.mean(): 20.102529525756836
第390次迭代, 权重w: tensor([0.1985, 0.3970]), (0.01 * w.grad): tensor([0.0020, 0.0040]), loss.mean(): 20.10049057006836
第391次迭代, 权重w: tensor([0.1965, 0.3930]), (0.01 * w.grad): tensor([0.0020, 0.0040]), loss.mean(): 20.09848976135254
第392次迭代, 权重w: tensor([0.1945, 0.3891]), (0.01 * w.grad): tensor([0.0020, 0.0039]), loss.mean(): 20.09653091430664
第393次迭代, 权重w: tensor([0.1926, 0.3852]), (0.01 * w.grad): tensor([0.0019, 0.0039]), loss.mean(): 20.094608306884766
第394次迭代, 权重w: tensor([0.1907, 0.3813]), (0.01 * w.grad): tensor([0.0019, 0.0039]), loss.mean(): 20.09272575378418
第395次迭代, 权重w: tensor([0.1888, 0.3775]), (0.01 * w.grad): tensor([0.0019, 0.0038]), loss.mean(): 20.09088134765625
第396次迭代, 权重w: tensor([0.1869, 0.3737]), (0.01 * w.grad): tensor([0.0019, 0.0038]), loss.mean(): 20.089073181152344
第397次迭代, 权重w: tensor([0.1850, 0.3700]), (0.01 * w.grad): tensor([0.0019, 0.0037]), loss.mean(): 20.087299346923828
第398次迭代, 权重w: tensor([0.1832, 0.3663]), (0.01 * w.grad): tensor([0.0019, 0.0037]), loss.mean(): 20.08556365966797
第399次迭代, 权重w: tensor([0.1813, 0.3626]), (0.01 * w.grad): tensor([0.0018, 0.0037]), loss.mean(): 20.083860397338867
第400次迭代, 权重w: tensor([0.1795, 0.3590]), (0.01 * w.grad): tensor([0.0018, 0.0036]), loss.mean(): 20.082191467285156
第401次迭代, 权重w: tensor([0.1777, 0.3554]), (0.01 * w.grad): tensor([0.0018, 0.0036]), loss.mean(): 20.080554962158203
第402次迭代, 权重w: tensor([0.1759, 0.3519]), (0.01 * w.grad): tensor([0.0018, 0.0036]), loss.mean(): 20.07895278930664
第403次迭代, 权重w: tensor([0.1742, 0.3483]), (0.01 * w.grad): tensor([0.0018, 0.0035]), loss.mean(): 20.077381134033203
第404次迭代, 权重w: tensor([0.1724, 0.3449]), (0.01 * w.grad): tensor([0.0017, 0.0035]), loss.mean(): 20.07583999633789
第405次迭代, 权重w: tensor([0.1707, 0.3414]), (0.01 * w.grad): tensor([0.0017, 0.0034]), loss.mean(): 20.07433319091797
第406次迭代, 权重w: tensor([0.1690, 0.3380]), (0.01 * w.grad): tensor([0.0017, 0.0034]), loss.mean(): 20.072853088378906
第407次迭代, 权重w: tensor([0.1673, 0.3346]), (0.01 * w.grad): tensor([0.0017, 0.0034]), loss.mean(): 20.07140350341797
第408次迭代, 权重w: tensor([0.1656, 0.3313]), (0.01 * w.grad): tensor([0.0017, 0.0033]), loss.mean(): 20.06998062133789
第409次迭代, 权重w: tensor([0.1640, 0.3280]), (0.01 * w.grad): tensor([0.0017, 0.0033]), loss.mean(): 20.068588256835938
第410次迭代, 权重w: tensor([0.1623, 0.3247]), (0.01 * w.grad): tensor([0.0016, 0.0033]), loss.mean(): 20.067224502563477
第411次迭代, 权重w: tensor([0.1607, 0.3214]), (0.01 * w.grad): tensor([0.0016, 0.0032]), loss.mean(): 20.065887451171875
第412次迭代, 权重w: tensor([0.1591, 0.3182]), (0.01 * w.grad): tensor([0.0016, 0.0032]), loss.mean(): 20.0645751953125
第413次迭代, 权重w: tensor([0.1575, 0.3150]), (0.01 * w.grad): tensor([0.0016, 0.0032]), loss.mean(): 20.063289642333984
第414次迭代, 权重w: tensor([0.1559, 0.3119]), (0.01 * w.grad): tensor([0.0016, 0.0032]), loss.mean(): 20.062030792236328
第415次迭代, 权重w: tensor([0.1544, 0.3088]), (0.01 * w.grad): tensor([0.0016, 0.0031]), loss.mean(): 20.0607967376709
第416次迭代, 权重w: tensor([0.1528, 0.3057]), (0.01 * w.grad): tensor([0.0015, 0.0031]), loss.mean(): 20.059585571289062
第417次迭代, 权重w: tensor([0.1513, 0.3026]), (0.01 * w.grad): tensor([0.0015, 0.0031]), loss.mean(): 20.058401107788086
第418次迭代, 权重w: tensor([0.1498, 0.2996]), (0.01 * w.grad): tensor([0.0015, 0.0030]), loss.mean(): 20.057239532470703
第419次迭代, 权重w: tensor([0.1483, 0.2966]), (0.01 * w.grad): tensor([0.0015, 0.0030]), loss.mean(): 20.05609893798828
第420次迭代, 权重w: tensor([0.1468, 0.2936]), (0.01 * w.grad): tensor([0.0015, 0.0030]), loss.mean(): 20.05498504638672
第421次迭代, 权重w: tensor([0.1454, 0.2907]), (0.01 * w.grad): tensor([0.0015, 0.0029]), loss.mean(): 20.053890228271484
第422次迭代, 权重w: tensor([0.1439, 0.2878]), (0.01 * w.grad): tensor([0.0015, 0.0029]), loss.mean(): 20.052818298339844
第423次迭代, 权重w: tensor([0.1425, 0.2849]), (0.01 * w.grad): tensor([0.0014, 0.0029]), loss.mean(): 20.05176544189453
第424次迭代, 权重w: tensor([0.1410, 0.2821]), (0.01 * w.grad): tensor([0.0014, 0.0028]), loss.mean(): 20.050735473632812
第425次迭代, 权重w: tensor([0.1396, 0.2792]), (0.01 * w.grad): tensor([0.0014, 0.0028]), loss.mean(): 20.049724578857422
第426次迭代, 权重w: tensor([0.1382, 0.2765]), (0.01 * w.grad): tensor([0.0014, 0.0028]), loss.mean(): 20.048736572265625
第427次迭代, 权重w: tensor([0.1368, 0.2737]), (0.01 * w.grad): tensor([0.0014, 0.0028]), loss.mean(): 20.047767639160156
第428次迭代, 权重w: tensor([0.1355, 0.2710]), (0.01 * w.grad): tensor([0.0014, 0.0027]), loss.mean(): 20.046815872192383
第429次迭代, 权重w: tensor([0.1341, 0.2682]), (0.01 * w.grad): tensor([0.0014, 0.0027]), loss.mean(): 20.04588508605957
第430次迭代, 权重w: tensor([0.1328, 0.2656]), (0.01 * w.grad): tensor([0.0013, 0.0027]), loss.mean(): 20.044971466064453
第431次迭代, 权重w: tensor([0.1315, 0.2629]), (0.01 * w.grad): tensor([0.0013, 0.0027]), loss.mean(): 20.044076919555664
第432次迭代, 权重w: tensor([0.1301, 0.2603]), (0.01 * w.grad): tensor([0.0013, 0.0026]), loss.mean(): 20.04319953918457
第433次迭代, 权重w: tensor([0.1288, 0.2577]), (0.01 * w.grad): tensor([0.0013, 0.0026]), loss.mean(): 20.042339324951172
第434次迭代, 权重w: tensor([0.1275, 0.2551]), (0.01 * w.grad): tensor([0.0013, 0.0026]), loss.mean(): 20.04149627685547
第435次迭代, 权重w: tensor([0.1263, 0.2525]), (0.01 * w.grad): tensor([0.0013, 0.0026]), loss.mean(): 20.040672302246094
第436次迭代, 权重w: tensor([0.1250, 0.2500]), (0.01 * w.grad): tensor([0.0013, 0.0025]), loss.mean(): 20.03986358642578
第437次迭代, 权重w: tensor([0.1238, 0.2475]), (0.01 * w.grad): tensor([0.0013, 0.0025]), loss.mean(): 20.0390682220459
第438次迭代, 权重w: tensor([0.1225, 0.2450]), (0.01 * w.grad): tensor([0.0012, 0.0025]), loss.mean(): 20.038291931152344
第439次迭代, 权重w: tensor([0.1213, 0.2426]), (0.01 * w.grad): tensor([0.0012, 0.0025]), loss.mean(): 20.03752899169922
第440次迭代, 权重w: tensor([0.1201, 0.2402]), (0.01 * w.grad): tensor([0.0012, 0.0024]), loss.mean(): 20.036781311035156
第441次迭代, 权重w: tensor([0.1189, 0.2378]), (0.01 * w.grad): tensor([0.0012, 0.0024]), loss.mean(): 20.036048889160156
第442次迭代, 权重w: tensor([0.1177, 0.2354]), (0.01 * w.grad): tensor([0.0012, 0.0024]), loss.mean(): 20.03533363342285
第443次迭代, 权重w: tensor([0.1165, 0.2330]), (0.01 * w.grad): tensor([0.0012, 0.0024]), loss.mean(): 20.034629821777344
第444次迭代, 权重w: tensor([0.1154, 0.2307]), (0.01 * w.grad): tensor([0.0012, 0.0023]), loss.mean(): 20.0339412689209
第445次迭代, 权重w: tensor([0.1142, 0.2284]), (0.01 * w.grad): tensor([0.0012, 0.0023]), loss.mean(): 20.03326416015625
第446次迭代, 权重w: tensor([0.1131, 0.2261]), (0.01 * w.grad): tensor([0.0011, 0.0023]), loss.mean(): 20.032604217529297
第447次迭代, 权重w: tensor([0.1119, 0.2239]), (0.01 * w.grad): tensor([0.0011, 0.0023]), loss.mean(): 20.031953811645508
第448次迭代, 权重w: tensor([0.1108, 0.2216]), (0.01 * w.grad): tensor([0.0011, 0.0022]), loss.mean(): 20.03131866455078
第449次迭代, 权重w: tensor([0.1097, 0.2194]), (0.01 * w.grad): tensor([0.0011, 0.0022]), loss.mean(): 20.03069496154785
第450次迭代, 权重w: tensor([0.1086, 0.2172]), (0.01 * w.grad): tensor([0.0011, 0.0022]), loss.mean(): 20.03008460998535
第451次迭代, 权重w: tensor([0.1075, 0.2150]), (0.01 * w.grad): tensor([0.0011, 0.0022]), loss.mean(): 20.02948760986328
第452次迭代, 权重w: tensor([0.1064, 0.2129]), (0.01 * w.grad): tensor([0.0011, 0.0022]), loss.mean(): 20.028900146484375
第453次迭代, 权重w: tensor([0.1054, 0.2108]), (0.01 * w.grad): tensor([0.0011, 0.0021]), loss.mean(): 20.028324127197266
第454次迭代, 权重w: tensor([0.1043, 0.2086]), (0.01 * w.grad): tensor([0.0011, 0.0021]), loss.mean(): 20.027759552001953
第455次迭代, 权重w: tensor([0.1033, 0.2066]), (0.01 * w.grad): tensor([0.0010, 0.0021]), loss.mean(): 20.02720832824707
第456次迭代, 权重w: tensor([0.1022, 0.2045]), (0.01 * w.grad): tensor([0.0010, 0.0021]), loss.mean(): 20.02666664123535
第457次迭代, 权重w: tensor([0.1012, 0.2024]), (0.01 * w.grad): tensor([0.0010, 0.0020]), loss.mean(): 20.026134490966797
第458次迭代, 权重w: tensor([0.1002, 0.2004]), (0.01 * w.grad): tensor([0.0010, 0.0020]), loss.mean(): 20.025615692138672
第459次迭代, 权重w: tensor([0.0992, 0.1984]), (0.01 * w.grad): tensor([0.0010, 0.0020]), loss.mean(): 20.025104522705078
第460次迭代, 权重w: tensor([0.0982, 0.1964]), (0.01 * w.grad): tensor([0.0010, 0.0020]), loss.mean(): 20.02460479736328
第461次迭代, 权重w: tensor([0.0972, 0.1945]), (0.01 * w.grad): tensor([0.0010, 0.0020]), loss.mean(): 20.02411651611328
第462次迭代, 权重w: tensor([0.0963, 0.1925]), (0.01 * w.grad): tensor([0.0010, 0.0019]), loss.mean(): 20.023635864257812
第463次迭代, 权重w: tensor([0.0953, 0.1906]), (0.01 * w.grad): tensor([0.0010, 0.0019]), loss.mean(): 20.02316665649414
第464次迭代, 权重w: tensor([0.0943, 0.1887]), (0.01 * w.grad): tensor([0.0010, 0.0019]), loss.mean(): 20.022705078125
第465次迭代, 权重w: tensor([0.0934, 0.1868]), (0.01 * w.grad): tensor([0.0009, 0.0019]), loss.mean(): 20.022254943847656
第466次迭代, 权重w: tensor([0.0925, 0.1849]), (0.01 * w.grad): tensor([0.0009, 0.0019]), loss.mean(): 20.02181053161621
第467次迭代, 权重w: tensor([0.0915, 0.1831]), (0.01 * w.grad): tensor([0.0009, 0.0018]), loss.mean(): 20.021377563476562
第468次迭代, 权重w: tensor([0.0906, 0.1813]), (0.01 * w.grad): tensor([0.0009, 0.0018]), loss.mean(): 20.020950317382812
第469次迭代, 权重w: tensor([0.0897, 0.1794]), (0.01 * w.grad): tensor([0.0009, 0.0018]), loss.mean(): 20.02053451538086
第470次迭代, 权重w: tensor([0.0888, 0.1777]), (0.01 * w.grad): tensor([0.0009, 0.0018]), loss.mean(): 20.020126342773438
第471次迭代, 权重w: tensor([0.0879, 0.1759]), (0.01 * w.grad): tensor([0.0009, 0.0018]), loss.mean(): 20.019725799560547
第472次迭代, 权重w: tensor([0.0871, 0.1741]), (0.01 * w.grad): tensor([0.0009, 0.0018]), loss.mean(): 20.019332885742188
第473次迭代, 权重w: tensor([0.0862, 0.1724]), (0.01 * w.grad): tensor([0.0009, 0.0017]), loss.mean(): 20.01894760131836
第474次迭代, 权重w: tensor([0.0853, 0.1707]), (0.01 * w.grad): tensor([0.0009, 0.0017]), loss.mean(): 20.018569946289062
第475次迭代, 权重w: tensor([0.0845, 0.1689]), (0.01 * w.grad): tensor([0.0009, 0.0017]), loss.mean(): 20.018199920654297
第476次迭代, 权重w: tensor([0.0836, 0.1673]), (0.01 * w.grad): tensor([0.0008, 0.0017]), loss.mean(): 20.017839431762695
第477次迭代, 权重w: tensor([0.0828, 0.1656]), (0.01 * w.grad): tensor([0.0008, 0.0017]), loss.mean(): 20.017484664916992
第478次迭代, 权重w: tensor([0.0820, 0.1639]), (0.01 * w.grad): tensor([0.0008, 0.0017]), loss.mean(): 20.017135620117188
第479次迭代, 权重w: tensor([0.0811, 0.1623]), (0.01 * w.grad): tensor([0.0008, 0.0016]), loss.mean(): 20.016796112060547
第480次迭代, 权重w: tensor([0.0803, 0.1607]), (0.01 * w.grad): tensor([0.0008, 0.0016]), loss.mean(): 20.016460418701172
第481次迭代, 权重w: tensor([0.0795, 0.1591]), (0.01 * w.grad): tensor([0.0008, 0.0016]), loss.mean(): 20.016132354736328
第482次迭代, 权重w: tensor([0.0787, 0.1575]), (0.01 * w.grad): tensor([0.0008, 0.0016]), loss.mean(): 20.015811920166016
第483次迭代, 权重w: tensor([0.0779, 0.1559]), (0.01 * w.grad): tensor([0.0008, 0.0016]), loss.mean(): 20.0154972076416
第484次迭代, 权重w: tensor([0.0772, 0.1543]), (0.01 * w.grad): tensor([0.0008, 0.0016]), loss.mean(): 20.01519012451172
第485次迭代, 权重w: tensor([0.0764, 0.1528]), (0.01 * w.grad): tensor([0.0008, 0.0015]), loss.mean(): 20.0148868560791
第486次迭代, 权重w: tensor([0.0756, 0.1513]), (0.01 * w.grad): tensor([0.0008, 0.0015]), loss.mean(): 20.014591217041016
第487次迭代, 权重w: tensor([0.0749, 0.1498]), (0.01 * w.grad): tensor([0.0008, 0.0015]), loss.mean(): 20.014301300048828
第488次迭代, 权重w: tensor([0.0741, 0.1483]), (0.01 * w.grad): tensor([0.0007, 0.0015]), loss.mean(): 20.014015197753906
第489次迭代, 权重w: tensor([0.0734, 0.1468]), (0.01 * w.grad): tensor([0.0007, 0.0015]), loss.mean(): 20.013736724853516
第490次迭代, 权重w: tensor([0.0727, 0.1453]), (0.01 * w.grad): tensor([0.0007, 0.0015]), loss.mean(): 20.013463973999023
第491次迭代, 权重w: tensor([0.0719, 0.1439]), (0.01 * w.grad): tensor([0.0007, 0.0015]), loss.mean(): 20.013195037841797
第492次迭代, 权重w: tensor([0.0712, 0.1424]), (0.01 * w.grad): tensor([0.0007, 0.0014]), loss.mean(): 20.01293182373047
第493次迭代, 权重w: tensor([0.0705, 0.1410]), (0.01 * w.grad): tensor([0.0007, 0.0014]), loss.mean(): 20.012676239013672
第494次迭代, 权重w: tensor([0.0698, 0.1396]), (0.01 * w.grad): tensor([0.0007, 0.0014]), loss.mean(): 20.01242446899414
第495次迭代, 权重w: tensor([0.0691, 0.1382]), (0.01 * w.grad): tensor([0.0007, 0.0014]), loss.mean(): 20.012176513671875
第496次迭代, 权重w: tensor([0.0684, 0.1368]), (0.01 * w.grad): tensor([0.0007, 0.0014]), loss.mean(): 20.011934280395508
第497次迭代, 权重w: tensor([0.0677, 0.1354]), (0.01 * w.grad): tensor([0.0007, 0.0014]), loss.mean(): 20.011695861816406
第498次迭代, 权重w: tensor([0.0670, 0.1341]), (0.01 * w.grad): tensor([0.0007, 0.0014]), loss.mean(): 20.011463165283203
第499次迭代, 权重w: tensor([0.0664, 0.1327]), (0.01 * w.grad): tensor([0.0007, 0.0013]), loss.mean(): 20.011234283447266
第500次迭代, 权重w: tensor([0.0657, 0.1314]), (0.01 * w.grad): tensor([0.0007, 0.0013]), loss.mean(): 20.011011123657227
最终结果, 权重w: tensor([0.0657, 0.1314], requires_grad=True), 梯度: tensor([0.0664, 0.1327]), 损失均值loss.mean(): 20.011011123657227
# 绘制损失曲线
iterations = range(500)
plt.figure(figsize=(12,8))
plt.plot(iterations,loss_list)
plt.xlabel("迭代次数")
plt.ylabel("损失均值loss.mean")
plt.grid()
plt.show()
常见面试题
1. 矩阵A形状(4,5),矩阵B形状(5,4),A和B的矩阵乘法结果形状是什么？
- 矩阵A 和 矩阵B 的矩阵乘法结果形状为 (4,4)
2. pytorch中求均值、求指数、求对数的API是什么？
- 均值：
torch.mean()或tensor.mean() - 指数：
torch.exp()或tensor.exp() - 对数：
torch.log()或tensor.log()
3. reshape和view有什么区别？
- reshape 和 view 都可以改变张量的形状，并且不改变元素顺序。
- 主要区别：
- 1.连续性要求上，view 只能作用于 连续张量，非连续张量则会报错；reshape 可以处理 连续或非连续张量，必要时会复制数据。
- 2.共享内存上，view 返回的张量 共享原张量内存，修改会影响原张量；reshape 不一定共享内存，有时会复制数据。
4. pytorch中哪个属性来设置开启梯度计算，哪个属性来获取梯度，哪个方法来计算梯度？
- 开启梯度计算，使用
requires_grad=True - 获取梯度，使用
tensor.grad - 计算梯度，使用
loss.backward()
代码作业
- 1.使用torch.tensor()创建两个形状为(3,4)的张量t1和t2，计算(t1+t2-2)*2, 并打印。
- 2.使用torch.arange创建一个形状为(3,4)的张量t1,创建一个形状为(3,4)的随机张量t2，计算t1*t2,然后将t2转换形状为(4,3)，注意不要改变t2中元素顺序，计算t1@t2,并打印。
- 3.使用torch.randint()创建一个形状为(3,4)的随机整数张量,求所有元素的均值 和 第1个轴的均值，并打印。(注意元素类型转换)
- 4.使用torch.randint()创建一个形状为(3,4)的张量，获取第2个轴的 索引为1的所有数据，然后获取第二个轴的 索引为1的 大于5的元素 对应的 第一个轴的数据，并打印。
- 5.创建两个形状为(3,4)的随机张量t1和t2，都转换形状为(4,3), 然后按0轴进行拼接，得到形状为(8,3)的张量，并打印。
- 6.创建一个形状为(3)的随机张量w，开启requires_grad, 计算loss = w**2 + 20, 使用loss.mean().backward()获取梯度，并打印梯度w.grad。（尝试手动计算梯度）
💻 代码
Day02 代码：张量基本操作
7 个文件📄 01_张量和numpy之间的相互转换.py 3.0KB
""" 演示 张量 和 numpy数组 之间的相互转换，以及 只有一个数值的张量 转换为 python数值 涉及到的API: 张量 转 numpy数组对象： 张量对象.numpy() 共享内存，浅拷贝，两个变量一起变 张量对象.numpy().copy() 不共享内存，深拷贝，两个变量不一起变 numpy数组对象 转 张量： torch.from_numpy(numpy数组对象) 共享内存，浅拷贝 torch.tensor(numpy数组对象) 不共享内存，深拷贝 一个数值的张量 转 python数值： 张量对象.item() 需要掌握： 张量 转 numpy数组对象： 张量对象.numpy() numpy数组对象 转 张量： torch.tensor(numpy数组对象) 一个数值的张量 转 python数值： 张量对象.item() """ # 导包 import torch # 1.定义函数，演示 张量 和 numpy数组之间 的相互转换 def demo01(): # 张量 -> numpy数组对象 # 0.设置随机种子 torch.manual_seed(5) # 1.创建一个随机张量，(3,4) t1 = torch.randn(3,4) print(f"t1:{t1},ndim: {t1.ndim}, shape: {t1.shape}, type: {type(t1)}, dtype: {t1.dtype}") # 2.使用张量对象.numpy(),将张量转为numpy数组 n1 = t1.numpy() print(f"n1:{n1}, shape: {n1.shape},type: {type(n1)}, dtype: {n1.dtype}") # 3.演示 共享内存/浅拷贝，两个变量一起变/心灵感灵，改变n1 n1[0,0] = 100 print(f"n1:{n1}, shape: {n1.shape}, dtype: {n1.dtype}") print(f"t1:{t1}, shape: {t1.shape}, dtype: {t1.dtype}") # 4.使用张量对象.numpy().copy(),将张量转为numpy数组 n2 = t1.numpy().copy() print(f"n2:{n2}, shape: {n2.shape}, dtype: {n2.dtype}") # # 5.演示 不共享内存/深拷贝，两个变量不一起变，改变n2 n2[0,0] = 200 print(f"n2:{n2}, shape: {n2.shape}, dtype: {n2.dtype}") print(f"t1:{t1}, shape: {t1.shape}, dtype: {t1.dtype}") # numpy数组对象 -> 张量 # 6.使用torch.from_numpy(numpy数组对象)，将numpy数组转为张量，共享内存，浅拷贝，心灵感应 t2 = torch.from_numpy(n2) print(f"t2:{t2}, shape: {t2.shape}, type: {type(t2)}, dtype: {t2.dtype}") t2[0,0] = 10 print(f"t2:{t2}, shape: {t2.shape}, dtype: {t2.dtype}") print(f"n2:{n2}, shape: {n2.shape}, dtype: {n2.dtype}") # 7.使用torch.tensor(numpy数组对象)，将numpy数组转为张量,不共享内存/深拷贝 t3 = torch.tensor(n2) print(f"t3:{t3}, shape: {t3.shape}, type:{type(t3)}, dtype: {t3.dtype}") t3[0,0] = 20 print(f"t3:{t3}, shape: {t3.shape}, dtype: {t3.dtype}") print(f"n2:{n2}, shape: {n2.shape}, dtype: {n2.dtype}") # 8.演示 一个数值的张量 转 python数值，张量对象.item() t4 = torch.tensor([[[[10]]]]) print(f"t4:{t4}, ndim: {t4.ndim}, shape: {t4.shape}, dtype: {t4.dtype}") a4 = t4.item() print(f"a4:{a4}, type: {type(a4)}") # 测试 if __name__ == '__main__': demo01()
📄 02_张量的基本运算.py 1.9KB
""" 演示 张量的基本运算，也就是 + - * / - 涉及到的API: add、sub、mul、div、neg add_、sub_、mul_、div_、neg_（其中带下划线的版本会修改原数据）,inplace=True 需要掌握： + - * / 注意： 1.张量 和 标量的运算：就是张量的每个元素和这个标量进行运算 2.pytorch中可以通用的API：Tensor.X = torch.X(Tensor,) """ # 导包 import torch # 1.定义函数，演示 张量的基本运算 def demo01(): # 0.设置随机种子 torch.manual_seed(5) # 1.定义一个张量，(3,4) t1 = torch.randint(0,10,(3,4)) print(f"t1:{t1}, shape: {t1.shape}, dtype: {t1.dtype}") # 2.演示 + # t2 = t1 + 1 # t2 = t1.add(1) # t2 = torch.add(t1,1) # t2 = t1.add_(1) # inplace=True # print(f"t2:{t2}, shape: {t2.shape}, dtype: {t2.dtype}") # print(f"t1:{t1}, shape: {t1.shape}, dtype: {t1.dtype}") # 3.演示 - # t2 = t1 - 1 # t2 = t1.sub(1) # t2 = torch.sub(t1,1) # t2 = t1.sub_(1) # print(f"t2:{t2}, shape: {t2.shape}, dtype: {t2.dtype}") # print(f"t1:{t1}, shape: {t1.shape}, dtype: {t1.dtype}") # 4.演示 * # t2 = t1 * 2 # t2 = t1.mul(2) # t2 = torch.mul(t1,2) # t2 = t1.mul_(2) # print(f"t2:{t2}, shape: {t2.shape}, dtype: {t2.dtype}") # print(f"t1:{t1}, shape: {t1.shape}, dtype: {t1.dtype}") # # 5.演示 / # t2 = t1 / 2 # t2 = t1.div(2) # t2 = torch.div(t1,2) # t2 = t1.div_(2) # print(f"t2:{t2}, shape: {t2.shape}, dtype: {t2.dtype}") # print(f"t1:{t1}, shape: {t1.shape}, dtype: {t1.dtype}") # 6.演示 取负 - t2 = - t1 # t2 = t1.neg() # t2 = torch.neg(t1) # t2 = t1.neg_() print(f"t2:{t2}, shape: {t2.shape}, dtype: {t2.dtype}") print(f"t1:{t1}, shape: {t1.shape}, dtype: {t1.dtype}") ... # 测试 if __name__ == '__main__': demo01()
📄 03_张量的按元素相乘和矩阵乘法.py 2.2KB
""" 演示 按元素相乘 和 矩阵乘法，对两个张量进行运算 按元素相乘: 按元素相乘（Hadamard）指的是两个相同形状的张量对应位置的元素相乘，使用 mul和运算符 * 实现 要求：两个张量的形状一样，A:(m,n),B(m,n) API: t1*t2 t1.mul(t2) torch.mul(t1,t2) 矩阵乘法： 要求：A(m,n),B必须是(n,p),也就是第一个矩阵A.shape[-1]=B.shape[-2] 如果是3D及以上张量，还要求A.shape[:-2]=B.shape[:-2] 运算：A的行向量 和 B的列向量 做点积，也就是先做按元素相乘再求和 (m,n)@(n,p)=(m,p) API: t1@t2 t1.matmul(t2) torch.matmul(t1,t2) t1.dot(t2),用于1D张量/向量的点积运算 需要掌握： t1*t2 t1@t2 """ # 导包 import torch # 1.定义函数，演示 张量的按元素相乘 def demo01(): # 0.设置随机种子 torch.manual_seed(5) # 1.创建两个张量，(3,4) t1 = torch.randint(0,10,(3,4)) print(f"t1:\n{t1}, shape: {t1.shape}, dtype: {t1.dtype}") t2 = torch.randint(0,10,(3,4)) print(f"t2:\n{t2}, shape: {t2.shape}, dtype: {t2.dtype}") # 2. 演示 按元素相乘 # t3 = t1*t2 # t3 = t1.mul(t2) t3 = torch.mul(t1,t2) print(f"t3:\n{t3}, shape: {t3.shape}, dtype: {t3.dtype}") # 2.定义函数，演示 张量的矩阵乘法 def demo02(): # 0.设置随机种子 torch.manual_seed(5) # 1.创建两个张量，(3,4)@(4,3)=(3,3) t1 = torch.randint(0,10,(3,4)) print(f"t1:\n{t1}, shape: {t1.shape}, dtype: {t1.dtype}") t2 = torch.randint(0,10,(4,5)) print(f"t2:\n{t2}, shape: {t2.shape}, dtype: {t2.dtype}") # 2. 演示 矩阵乘法 # t3 = t1@t2 t3 = t1.matmul(t2) # t3 = torch.matmul(t1,t2) print(f"t3:\n{t3}, shape: {t3.shape}, dtype: {t3.dtype}") # 3.演示 点积dot print(f"t1[0,:3]:\n{t1[0,:3]}") print(f"t2[0,:3]:\n{t2[0,:3]}") # t4 = t1[0,:3].dot(t2[0,:3]) t4 = torch.dot(t1[0,:3],t2[0,:3]) print(f"t4:\n{t4}, ndim: {t4.ndim}, shape: {t4.shape}, dtype: {t4.dtype}") ... # 测试 if __name__ == '__main__': # demo01() demo02()
📄 04_张量的常用运算函数.py 2.5KB
""" 演示 张量的常用运算函数,对同一个张量进行运算 涉及到的API: sum(), mean(), max(), min() -> 有dim参数，可以指定维度/轴进行操作，会改变张量的形状 pow()/**, sqrt()/**0.5, exp(), log(), log2(), log10() -> 无dim参数，对张量中的所有元素进行运算，不会改变张量的形状 要掌握的： sum(), mean(), max(), min() ** 求n次方 """ # 导包 import torch # 1.定义函数，演示 常用的运算函数 def demo01(): # 0.设置随机种子 torch.manual_seed(5) # 1.创建一个3D张量，(3,4,5) t1 = torch.randint(0,10,(3,4,5)) print(f"t1: \n{t1}, shape: {t1.shape}, dtype: {t1.dtype}") # 2.演示 sum(), mean(), max(), min() # 2.1 演示 sum() # 没有dim参数，则返回标量 # print(f"t1.sum(): {t1.sum()}, shape: {t1.sum().shape}, ndim: {t1.sum().ndim}") # print(f"t1.sum(dim=0): \n{t1.sum(dim=0)}, shape: {t1.sum(dim=0).shape}, ndim: {t1.sum(dim=0).ndim}") # dim=0: (3,4,5) -> (4,5) # print(f"t1.sum(dim=1): \n{t1.sum(dim=1)}, shape: {t1.sum(dim=1).shape}, ndim: {t1.sum(dim=1).ndim}") # dim=1: (3,4,5) -> (3,5) # 2.2 演示 mean() # t1 = t1.to(torch.float32) # print(f"t1.mean(): {t1.mean()}, shape: {t1.mean().shape}, ndim: {t1.mean().ndim}") # print(f"t1.mean(dim=0): {t1.mean(dim=0)}, shape: {t1.mean(dim=0).shape}, ndim: {t1.mean(dim=0).ndim}") # # (4,5) # print(f"t1.mean(dim=1): {t1.mean(dim=1)}, shape: {t1.mean(dim=1).shape}, ndim: {t1.mean(dim=1).ndim}") # # (3,5) # # 2.3 演示 max() # print(f"t1.max(): {t1.max()}, shape: {t1.max().shape}, ndim: {t1.max().ndim}") # print(f"t1.max(dim=0): {t1.max(dim=0)}, shape: {t1.max(dim=0)[0].shape}") # # (4,5) # print(f"t1.max(dim=1): {t1.max(dim=1)}, shape: {t1.max(dim=1)[0].shape}") # # (3,5) # # 2.4 演示 min() # print(f"t1.min(): {t1.min()}, shape: {t1.min().shape}, ndim: {t1.min().ndim}") # print(f"t1.min(dim=0): {t1.min(dim=0)}, shape: {t1.min(dim=0)[0].shape}") # # (4,5) # print(f"t1.min(dim=1): {t1.min(dim=1)}, shape: {t1.min(dim=1)[0].shape}") # (3,5) # 3.演示 pow()/**, sqrt(), exp(), log(), log2(), log10() # 3.1 演示 ** print(f"t1**0.5: {t1**0.5}, shape: {(t1**0.5).shape}, ndim: {(t1**0.5).ndim}") # (3,4,5) # # 3.2 演示 sqrt # print(f"t1.sqrt(): {t1.sqrt()}, shape: {(t1.sqrt()).shape}, ndim: {(t1.sqrt()).ndim}") ... # 测试 if __name__ == '__main__': demo01()
📄 05_张量的索引操作.py 2.3KB
""" 案例： 演示 张量的索引操作 涉及到的API: 简单行列索引 列表索引 范围索引 多维索引 布尔索引 需要掌握的： 简单行列索引 t1[0,0], t1[0] 范围索引 t1[:2], t1[:2,2:4], 起始(默认0):结束(默认最后):步长(默认1) """ # 导包 import torch # 1.定义函数，演示 张量的索引操作 def demo01(): # 0.设置随机种子 torch.manual_seed(5) # 1.定义一个张量，(3,4) t1 = torch.randint(0,10,(3,4)) print(f"t1:\n{t1}, shape: {t1.shape}, dtype: {t1.dtype}") # 2.演示 简单行列索引 # print(f"t1[0,0]: {t1[0,0]}, ndim: {t1[0,0].ndim},shape: {t1[0,0].shape}") # print(f"t1[1]: {t1[1]},ndim: {t1[1].ndim},shape: {t1[1].shape}") # print(f"t1[:,2]: {t1[:,2]},ndim: {t1[:,2].ndim},shape: {t1[:,2].shape}") # 3.演示 列表索引 # # 获取(0,1),(1,2)位置的元素 # print(f"t1[[0,1],[1,2]]: {t1[[0,1],[1,2]]}, ndim: {t1[[0,1],[1,2]].ndim},shape: {t1[[0,1],[1,2]].shape}") # # 获取(0,2),(1,3)位置的元素 # print(f"t1[[0,1],[2,3]]: {t1[[0,1],[2,3]]}, ndim: {t1[[0,1],[2,3]].ndim},shape: {t1[[0,1],[2,3]].shape}") # 4.演示 范围索引 # # 获取前两行 # print(f"t1[:2]: {t1[:2]}, ndim: {t1[:2].ndim},shape: {t1[:2].shape}") # # 获取前两列 # print(f"t1[:,:2]: {t1[:,:2]}, ndim: {t1[:,:2].ndim},shape: {t1[:,:2].shape}") # # 获取偶数行 # t2 = t1[1::2] # print(f"t2: {t2}, ndim: {t2.ndim},shape: {t2.shape}") # # 获取偶数行的奇数列 # t3 = t1[1::2,::2] # print(f"t3: {t3}, ndim: {t3.ndim},shape: {t3.shape}") # 5.演示 多维索引 t4 = torch.randint(0,10,(3,4,5)) print(f"t4:\n{t4}, shape: {t4.shape}, dtype: {t4.dtype}") # # 获取(0,0,0)位置的元素 # t5 = t4[0,0,0] # print(f"t5: {t5}, ndim: {t5.ndim},shape: {t5.shape}") # # 获取0轴索引1，1轴偶数索引，2轴前3个 的数据 # t6 = t4[1,::2,:3] # (2,3) # print(f"t6: {t6}, ndim: {t6.ndim},shape: {t6.shape}") # 6.演示 布尔索引 # 获取第二行的大于5的所有元素 print(t4[1]>5) # (3,5) t5 = t4[1,t4[1]>5] print(f"t5: {t5}, ndim: {t5.ndim},shape: {t5.shape}") ... # 测试 if __name__ == '__main__': demo01()
📄 06_张量的形状操作.py 3.3KB
""" 演示 张量的形状操作 涉及到的API: reshape() 不改变张量内容/张量的连续性的前提下，改变张量的形状，不改变元素的顺序，从左到右从上到小的顺序 unsqueeze() 在指定的轴/维度上增加一个维度（长度1），升维 squeeze() 删除所有长度为1的维度，降维 transpose() 交换维度/轴，只能交换两个维度/轴 permute() 交换维度/轴，可以交换多个维度/轴 view() 改变形状，要求是连续张量，张量中元素的顺序没有改变， 没有经过transpose或permute等操作 contiguous() 将非连续张量转换为连续张量 is_contiguous() 判断张量是否是连续张量 需要掌握： reshape() 改变形状，不改变张量内容，元素个数守恒 unsqueeze() 在指定的轴/维度上增加一个维度/轴 permute() 交换多个维度/轴, 改变了张量中元素顺序 view() CV计算机视觉中用的较多,要求连续张量 """ # 导包 import torch # 1.定义函数，演示 张量的形状操作，改变形状 def demo01(): # 0.设置随机种子 torch.manual_seed(28) # 1.创建一个张量，(3,4) t1 = torch.randint(0,10,(3,4)) print(f"t1:\n{t1}, shape: {t1.shape}") # 2.使用reshape()方法，改变张量的形状，(4,3) # t2 = t1.reshape(4,3) # t2 = t1.reshape(2,6) # t2 = t1.reshape(-1,3) # -1表示自动计算该维度的大小，元素个数守恒 # print(f"t2:\n{t2}, shape: {t2.shape}") # 3.使用unsqueeze()方法，升维，添加一个新的维度/轴 # t2 = t1.unsqueeze(0) # (1,3,4) t2 = t1.unsqueeze(1).unsqueeze(0) # (1,3,1,4) print(f"t2:\n{t2}, shape: {t2.shape}") # 4.使用squeeze()方法，降维，删除所有长度为1的维度 # t3 = t2.squeeze() # (3,4) t3 = t2.squeeze(dim=1) print(f"t3:\n{t3}, shape: {t3.shape}") # 2.定义函数，演示 交换维度,view def demo02(): # 0.设置随机种子 torch.manual_seed(28) # 1.创建一个张量，(3,4,5) t1 = torch.randint(0,10,(3,4,5)) # HWC print(f"t1:\n{t1}, shape: {t1.shape}") # (3,4,5) print(f"t1是否是连续张量：{t1.is_contiguous()}") # # 2.使用permute()方法，交换维度 HWC(0,1,2) -> CHW(2,0,1),改变了张量的连续性，也就是改变了张量的元素顺序 t2 = t1.permute(2,0,1) # (5,3,4) print(f"t2:\n{t2}, shape: {t2.shape}") # 2.1 使用is_contiguous()方法，判断张量是否是连续张量 print(f"t2是否是连续张量：{t2.is_contiguous()}") # 3.使用transpose()方法，交换维度 # (3,4,5) -> (5,4,3) # t3 = t1.transpose(2,0) # (5,4,3) # print(f"t3:\n{t3}, shape: {t3.shape}") # 4.使用view()方法，改变张量的形状，不改变张量的连续性/元素顺序，要求是连续张量 # t4 = t2.view(10,2,3) # # 4.1 使用contiguous()方法，将非连续张量转换为连续张量 # t4 = t2.contiguous() t4 = t2.contiguous().view(10,2,3) print(f"t4是否是连续张量：{t4.is_contiguous()}") # # # t4 = t2.reshape(3,4,5) print(f"t4:\n{t4}, shape: {t4.shape}") # 测试 if __name__ == '__main__': # demo01() demo02()
📄 07_张量的拼接.py 1.5KB
""" 演示 张量的拼接 涉及到的API: torch.cat() 在指定的维度/轴拼接张量，不改变张量的维度2D/3D，会改变拼接的维度/轴的长度，除了要拼接的维度，其他维度形状必须相同 (m,n)+(k,n)->(m+k,n) torch.stack() 在一个新的维度/轴堆叠张量，增加一个新的维度，比如2D->3D,要求输入的张量的形状必须相同 (m,n)+(m,n)->(2,m,n) 需要掌握的： torch.cat torch.stack """ # 导包 import torch # 1.定义函数，演示 张量的拼接 def demo01(): # 0.设置随机种子 torch.manual_seed(28) # 1.定义两个张量，(3,4),(3,4) t1 = torch.randint(0,10,(3,4)) t2 = torch.randint(0,10,(3,4)) # t4 = torch.randint(0,10,(5,4)) print(f"t1:{t1}, shape: {t1.shape}") print(f"t2:{t2}, shape: {t2.shape}") # 2.使用torch.cat(),拼接t1,t2 t3 = torch.cat([t1,t2],dim=0) # (6,4) print(f"t3:{t3}, shape: {t3.shape}") t3 = torch.cat([t1, t2], dim=1) # (3,8) print(f"t3:{t3}, shape: {t3.shape}") # t3 = torch.cat([t1, t2], dim=2) # (5,7) # print(f"t3:{t3}, shape: {t3.shape}") # 3.使用torch.stack(),堆叠t1,t2 t4 = torch.stack([t1, t2], dim=0) # (2,3,4) print(f"t4:{t4}, shape: {t4.shape}") # t4 = torch.stack([t1, t2], dim=1) # (3,2,4) # print(f"t4:{t4}, shape: {t4.shape}") # t4 = torch.stack([t1, t2], dim=2) # (3,4,2) # print(f"t4:{t4}, shape: {t4.shape}") ... # 测试 if __name__ == '__main__': demo01()
📎 资料
每日一图
- 📎 day02.png92.7KB
ℹ️ 离线版不含附件原文件，需要请到在线网站对应课时下载。
第3节 · 神经网络基础（一）：自动微分与梯度下降
🎯 掌握 autograd、梯度下降与线性回归实现。
📖 笔记
Day03 笔记：神经网络基础（一）
Day 03 - 神经网络基础1
学习目标
- 掌握detach()函数的使用
- 实现pytorch模拟线性回归
- 了解pytorch中进行模型训练的工作流
- 理解神经网络的 概念 和 构成
- 理解神经元的 加权求和 和 激活函数
- 掌握常用激活函数
- 了解常见的参数初始化方法
# 导包
import torch
import numpy as np
import matplotlib.pyplot as plt
# 设置中文字体
plt.rcParams['font.sans-serif'] = ['SimHei'] # 微软雅黑
plt.rcParams['axes.unicode_minus'] = False # 解决负号显示问题
import matplotlib
matplotlib.use('TKAgg')
# 设置设备
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
day03_01_自动微分_detach()函数介绍
"""
案例：
演示 detach() 函数的功能，解决 开启梯度计算 requires_grad=True 的张量，无法转为numpy对象的问题
介绍：
一个张量 一旦开启了梯度计算/自动微分 requires_grad=True，就无法转为 numpy对象，需要通过detach()函数来解决
需要掌握：
n1 = t1.detach().numpy()
"""
# 导包
import torch
import numpy as np # pip install numpy
# 1.定义张量，开启梯度计算
# 参数1：数据；参数2：是否开启梯度计算；参数3：数据类型
t1 = torch.tensor([1,2,3], requires_grad=True, dtype=torch.float32)
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
# # 2.转为numpy数组对象, 报错
# n1 = t1.numpy()
# print(f"n1: {n1}, shape: {n1.shape}, dtype元素类型：{n1.dtype}")
# 3.演示使用detach()函数
t2 = t1.detach()
print(f"t2.requires_grad: {t2.requires_grad}")
print(f"t2: {t2}, shape: {t2.shape}, dtype元素类型：{t2.dtype}")
n1 = t2.numpy()
print(f"n1: {n1}, shape: {n1.shape}, dtype元素类型：{n1.dtype}")
# 测试 是否共享内存
n1[0] = 100
print(f"n1: {n1}, shape: {n1.shape}, dtype元素类型：{n1.dtype}")
print(f"t1: {t1}, shape: {t1.shape}, dtype元素类型：{t1.dtype}")
print(f"t2: {t2}, shape: {t2.shape}, dtype元素类型：{t2.dtype}")
t1: tensor([1., 2., 3.], requires_grad=True), shape: torch.Size([3]), dtype元素类型：torch.float32
t2.requires_grad: False
t2: tensor([1., 2., 3.]), shape: torch.Size([3]), dtype元素类型：torch.float32
n1: [1. 2. 3.], shape: (3,), dtype元素类型：float32
n1: [100. 2. 3.], shape: (3,), dtype元素类型：float32
t1: tensor([100., 2., 3.], requires_grad=True), shape: torch.Size([3]), dtype元素类型：torch.float32
t2: tensor([100., 2., 3.]), shape: torch.Size([3]), dtype元素类型：torch.float32
day03_02_自动微分真实应用场景
- 1.前向传播，计算 预测值z
- 2.计算损失，基于loss_fn损失函数，结合 预测值z 和 真实值y, 来计算损失值loss
- 3.反向传播loss.backward()，过程中调用自动微分模块，来自动计算loss对模型参数w, b的梯度/导数
- 4.更新参数，梯度下降法，w新 = w旧 - 学习率 * 梯度
# 0.设置随机种子
torch.manual_seed(3)
# 1.定义x， 表示：特征，输入数据
x = torch.ones(2,5)
print(f"x: {x}")
# 2.定义y, 表示：真实值
y = torch.zeros(2,1)
print(f"y: {y}")
# 3.初始化 权重w 和 偏置b, x @ w + b, (2, 1)
w = torch.randn(5,1, requires_grad=True, dtype=torch.float32)
print(f"w: {w}")
b = torch.randn(1, requires_grad=True, dtype=torch.float32)
print(f"b: {b}")
# 4.前向传播，计算 预测值z
z = x @ w + b # (2, 1)
# z = torch.matmul(x, w) + b
print(f"z: {z}, shape: {z.shape}")
# 5.计算损失
loss_fn = torch.nn.MSELoss()
loss = loss_fn(z, y)
print(f"loss:",loss)
# 6.自动微分，执行反向传播，获取梯度：loss对w,b的梯度，注意损失函数MSELoss内部会自动求均值，不需要loss.mean()
loss.backward()
# 7.打印loss对w, b的梯度/导数
print(f"w.grad: {w.grad}")
print(f"b.grad: {b.grad}")
# 后续就是：梯度下降法来更新模型参数，w新 = w旧 - 学习率 * 梯度
x: tensor([[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.]])
y: tensor([[0.],
[0.]])
w: tensor([[ 0.8033],
[ 0.1748],
[ 0.0890],
[-0.6137],
[ 0.0462]], requires_grad=True)
b: tensor([-1.3683], requires_grad=True)
z: tensor([[-0.8687],
[-0.8687]], grad_fn=<AddBackward0>), shape: torch.Size([2, 1])
loss: tensor(0.7547, grad_fn=<MseLossBackward0>)
w.grad: tensor([[-1.7374],
[-1.7374],
[-1.7374],
[-1.7374],
[-1.7374]])
b.grad: tensor([-1.7374])
# w = [w1, w2, w3, w4, w5] = [[ 0.8033], [ 0.1748], [ 0.0890], [-0.6137], [ 0.0462]]
# b = [b1] = [-1.3683]
# x = [[1., 1., 1., 1., 1.],
# [1., 1., 1., 1., 1.]]
# z = x @ w + b1 = [w1 + w2 + w3 + w4 + w5 + b1, w1 + w2 + w3 + w4 + w5 + b1] = [z1, z2] = [[-0.8687], [-0.8687]]
# mse_value = (SUM(预测值 - 真实值)^2)/2 = ((z1)^2 + (z2)^2)/2 = (2*z1^2)/2 = z1^2 = (w1 + w2 + w3 + w4 + w5 + b1)^2
# mse_grad | [w1, w2, w3, w4, w5] = [2*z1*1, 2*z1*1, 2*z1*1, 2*z1*1, 2*z1*1, 2*z1*1] = [[-1.7374], [-1.7374], [-1.7374], [-1.7374], [-1.7374]]
# mse_grad | [b1] = [2*z1*1]
day03_03_pytorch框架_模拟线性回归
我们使用 PyTorch 的各个组件来构建线性回归模型。在pytorch中进行模型训练的工作流分为四个步骤：
- 准备数据集
- 构建模型
- 设置损失函数和优化器
- 模型训练
要使用的API：
- 使用 PyTorch 的 nn.MSELoss() 实现 平方损失函数
- 使用 PyTorch 的 data.DataLoader 实现 数据加载器
- 使用 PyTorch 的 optim.SGD 实现 优化器，执行梯度下降来更新模型参数
- 使用 PyTorch 的 nn.Linear 实现 线性回归模型
"""
案例：
演示 pytorch 框架 模拟线性回归
工作流：
1.准备数据集
构建数据加载器：numpy对象 -> 张量 -> 数据集对象TensorDataset -> 数据加载器DataLoader
2.构建模型
3.设置损失函数和优化器
4.模型训练
5.模型测试
"""
# 导包
import torch
# numpy对象 -> 张量 -> 数据集对象TensorDataset -> 数据加载器DataLoader
from torch.utils.data import TensorDataset # 构造数据集对象
from torch.utils.data import DataLoader # 数据加载器，按批次加载数据
from torch import nn # 提供 MSE损失函数 和 线性层，线性层用来模拟线性回归模型
from torch import optim # 提供各种优化器，用于更新模型参数，公式为：w ← w - lr * grad
from sklearn.datasets import make_regression # 创建线性回归的示例数据集, pip install scikit-learn
import matplotlib.pyplot as plt # 绘图 pip install matplotlib
# 设置matplotlib中文字体
plt.rcParams['font.sans-serif'] = ['SimHei'] # 用来正常显示中文标签
plt.rcParams['axes.unicode_minus'] = False # 用来正常显示负号
# import matplotlib
# matplotlib.use('TKAgg')
# 1.准备数据集
# 定义函数，构造数据集
def create_dataset():
# 创建数据集对象
x, y, coef = make_regression(
n_samples=100, # 样本数量,100
n_features=1, # 特征数量,1
n_targets=1, # 目标数量,1
noise=10, # 噪声
coef=True, # 是否返回系数
bias=15.6, # 偏置
random_state=60,
)
# 转换为张量
x = torch.tensor(x, dtype=torch.float32)
y = torch.tensor(y, dtype=torch.float32)
# 返回结果
return x, y, coef
# 2.定义函数，实现模型训练
def train(x, y, coef):
# 1.构建数据加载器：numpy对象 -> 张量 -> 数据集对象TensorDataset -> 数据加载器DataLoader
dataset = TensorDataset(x, y)
# 参数1：数据集对象；参数2：批次大小；参数3：是否打乱数据，训练时打乱，测试时不打乱；参数4：是否删除最后一个批次不满的批次
dataloader = DataLoader(dataset, batch_size=16, shuffle=True, drop_last=False) # 100个样本，16，6*16+4
# 2.构建模型
model = nn.Linear(in_features=1, out_features=1) # 自动进行参数初始化，权重w 和 偏置b
# 3.设置损失函数和优化器
loss_fn = nn.MSELoss()
# 参数1：待更新的模型参数；参数2：学习率
optimizer = optim.SGD(model.parameters(), lr=0.01)
# 4.模型训练
# 定义训练轮数
epochs = 100
# 记录 每轮的平均损失
total_train_loss_list = []
# 遍历轮数epochs
for epoch in range(epochs):
# 初始化 当前轮次的总损失值
total_train_loss = 0.0
# 初始化 当前轮次的样本数量
total_samples = 0
# 遍历数据加载器，按批次获取数据
for train_x, train_y in dataloader:
# 1.前向传播，计算 预测值z
y_pred = model(train_x) # (batch_size, 1)
# print(f"y_pred: {y_pred}, shape: {y_pred.shape}")
# 2.计算损失
# print(f"train_y.shape: {train_y.shape}")
# print(f"y_pred.shape: {y_pred.shape}")
loss = loss_fn(y_pred, train_y.reshape(-1,1))
# print(f"loss: {loss}, shape: {loss.shape}")
# 更新当前轮次的总损失值
batch_size = train_x.shape[0]
total_train_loss += loss.item() * batch_size
total_samples += batch_size
# 3.梯度清零
optimizer.zero_grad()
# 4.反向传播
loss.backward()
# 5.更新参数
optimizer.step()
# 计算当前轮次epoch的平均损失值
epoch_train_loss = total_train_loss / total_samples
total_train_loss_list.append(epoch_train_loss)
# print(f"{epoch}轮 | "
# f"训练损失：{epoch_train_loss:.4f}")
# 打印最终的训练损失
print(f"损失列表为: {total_train_loss_list}")
# 打印最终的模型参数
print(f"模型参数为：w: {model.weight}, b: {model.bias}")
# 直接复制
# 6.可视化
# 绘制损失函数曲线total_train_loss_list
plt.figure(figsize=(15, 8))
plt.subplot(1, 2, 1)
plt.title("训练损失曲线")
x_epoch = range(epochs)
plt.plot(x_epoch, total_train_loss_list, label="训练损失")
plt.grid()
plt.legend()
# 添加x轴标签
plt.xlabel("epoch")
# 添加y轴标签
plt.ylabel("loss")
# 绘制 预测值 与 真实值 的对比图
plt.subplot(1, 2, 2)
plt.title("预测值与真实值的对比")
plt.scatter(x, y, label="真实数据点")
# 计算理论的真实y值
y_true = [v * coef.item() + 15.6 for v in x]
plt.plot(x, y_true, label="真实值")
# 计算预测值
# y_pred = [v*56.4556 + 14.1016 for v in x]
# 关闭自动微分
with torch.no_grad():
y_pred = model(x).detach().numpy()
plt.plot(x, y_pred, label="预测值")
# 添加x轴标签
plt.xlabel("x")
# 添加y轴标签
plt.ylabel("y")
# 添加图例
plt.legend()
# 添加网格
plt.grid()
plt.show()
# 测试
if __name__ == '__main__':
# 1.准备数据集
x, y, coef = create_dataset()
# print(f"x: {x}, shape: {x.shape}")
# print(f"y: {y}, shape: {y.shape}")
print(f"coef: {coef}, shape: {coef.shape}")
# plt.scatter(x, y)
# x = torch.linspace(x.min(), x.max(), 1000)
# y1 = torch.tensor([v * coef + 14.5 for v in x])
# plt.plot(x, y1, label='真实数据')
# plt.grid()
# plt.legend()
# plt.show()
# 2.模型训练
train(x, y, coef)
coef: 56.4340604104979, shape: ()
损失列表为: [3733.35861328125, 2663.7029418945312, 1976.53541015625, 1419.078828125, 1029.4727087402343, 777.937568359375, 573.7079736328125, 429.68401977539065, 334.38183227539065, 268.46607788085936, 218.1030987548828, 185.41197875976562, 156.88349487304689, 137.52262084960938, 123.24655395507813, 113.31351531982422, 107.17456970214843, 102.98481964111328, 99.79693511962891, 97.20145446777343, 95.16941184997559, 94.06178344726563, 93.06755065917969, 92.35160705566406, 91.98932189941407, 91.70067932128906, 91.55541526794434, 91.63355728149413, 91.50667694091797, 91.4613104248047, 91.55095596313477, 91.42731750488281, 91.39078338623047, 91.35545677185058, 91.1931365966797, 91.22664123535156, 91.34032348632813, 91.29905197143555, 91.19990051269531, 91.17785095214843, 91.03748443603516, 91.14641426086426, 91.02631469726562, 91.02016418457032, 91.07926818847656, 91.17668487548828, 91.03695343017579, 91.06348999023437, 91.13747955322266, 91.09613456726075, 91.0124951171875, 91.03497436523438, 91.11513000488281, 91.10722503662109, 91.0077734375, 91.05446746826172, 91.14527282714843, 91.18249481201173, 91.0549038696289, 91.06507049560547, 91.11053436279298, 91.18023193359375, 91.13560668945313, 91.09894989013672, 91.05186798095703, 91.10524429321289, 91.14735061645507, 91.10389923095703, 91.06577102661133, 91.17342758178711, 91.07400100708008, 91.14831298828125, 91.11019409179687, 91.14971145629883, 91.16691162109375, 91.2016665649414, 91.13302062988281, 91.21342864990234, 91.1178370666504, 91.18890899658203, 91.20787048339844, 91.03737884521485, 90.99844604492188, 91.14810852050782, 91.09397552490235, 91.2729995727539, 91.20170150756836, 91.2099072265625, 91.08122863769532, 91.30097381591797, 91.19452819824218, 91.16577453613282, 91.22069183349609, 91.11922149658203, 91.03404479980469, 91.40811386108399, 91.00988250732422, 91.09299499511718, 91.05481018066406, 91.04532531738282]
模型参数为：w: Parameter containing:
tensor([[56.4265]], requires_grad=True), b: Parameter containing:
tensor([14.2516], requires_grad=True)
1.神经网络概念
神经网络参考了生物神经网络，数据从输入到输出经过一层一层的神经元产生预测值的过程就是前向传播。
前向传播涉及到人工神经元是如何工作的（也就是神经元的初始化、激活函数），神经网络如何搭建，权重参数计算、数据形状如何变化。
什么是神经网络
人工神经网络（Artificial Neural Network， 简写为ANN）也简称为神经网络（NN），是一种模仿生物神经网络结构和功能的计算模型。它由多个互相连接的人工神经元（也称为节点）构成，可以用于处理和学习复杂的数据模式，尤其适合解决非线性问题。人工神经网络是机器学习中的一个重要模型，在深度学习领域中得到了广泛应用。
人脑可以看做是一个生物神经网络，由众多的神经元连接而成。各个神经元传递复杂的电信号，树突接收到输入信号，然后对信号进行处理，通过轴突输出信号。下图是生物神经元示意图：
当电信号通过树突进入到细胞核时，会逐渐聚集电荷。达到一定的电位后，细胞就会被激活，通过轴突发出电信号。
如何构建神经网络
神经网络是由多个神经元组成，构建神经网络就是在构建神经元。以下是神经网络中神经元的构建说明：
这个流程就像，来源不同树突(树突都会有不同的权重)的信息, 进行加权计算, 输入到细胞核中，通过激活函数输出信号。
同一层的多个神经元可以看作是通过并行计算来处理相同的输入数据，学习输入数据的不同特征。每个神经元可能会关注输入数据中的不同部分，从而捕捉到数据的不同属性。
接下来，我们使用多个神经元来构建神经网络，相邻层之间的神经元相互连接，并给每一个连接分配一个强度，如下图所示：
神经网络中信息只向一个方向移动，即从输入节点向前移动，通过隐藏节点，再向输出节点移动。其中的基本部分是:
-
输入层/输入数据（Input Layer）: 即输入x的那一层（如图像、文本、声音等），实际就是输入数据。输入层不是网络层，没有神经元。输入层将数据传递给下一层的神经元。
-
输出层（Output Layer）: 即输出y的那一层。输出层的神经元根据网络的任务（回归、分类等）生成最终的预测结果。
- 隐藏层（Hidden Layers）: 输入层和输出层之间都是隐藏层，神经网络的“深度”通常由隐藏层的数量决定。隐藏层的神经元通过 加权求和 和 激活函数 处理输入，并将结果传递到下一层。
-
特点是：
-
同一网络层的神经元之间没有连接
- 第N-1层的每个神经元和第N层的所有神经元相连（Fully Connected），全连接的网络层称为全连接层，由一个或多个全连接层组成的网络就是全连接神经网络（FCNN）
- 全连接神经网络接收的输入是2D的，数据在每一层之间以2D的形式传递
- 第N层网络的每个神经元的输入是 第N-1层网络的所有神经元的输出
- 第N层网络的输入维度 = 第N-1层网络的输出维度 = 第N-1层网络的神经元数量
- 每个连接都有一个权重值w
- 网络层的每个神经元的模型参数有：n个权重[w1,...,wn] + 1个偏置b, n为上一网络层的输出维度
神经网络内部状态值和激活值
每一个神经元工作时，前向传播会产生两个值，内部状态值（加权求和值）和激活值；反向传播时会产生激活值梯度和内部状态值梯度。
- 内部状态值
- 神经元或隐藏单元的内部存储值，它反映了当前神经元接收到的输入（上一层的输出）的加权求和结果。
- 每个输入$\(x_i\)\(都有一个与之相乘的权重\)\(w_i\)$，表示每个输入信号的重要性。
-
z=w⋅x+b
- w：权重矩阵
- x：输入值
- b：偏置
-
激活值
- 通过激活函数（如 ReLU、Sigmoid、Tanh）对内部状态值进行非线性变换后得到的结果。激活值决定了当前神经元的输出。
- a=f(z)
- f：激活函数
- z：内部状态值
通过控制每个神经元的内部状态值和激活值的大小、每一层的内部状态值和激活值的方差可让整个神经网络工作的更好。
2. 激活函数
为什么需要激活函数？
- 引入非线性，提高模型的表达能力
网络非线性因素理解
- 没有引入非线性因素的网络等价于一个线性模型
- 通过给网络增加激活函数, 引入非线性因素, 使得网络模型可以逼近任意函数, 提升网络对复杂问题的拟合能力
激活函数用于对每层的输出数据进行变换, 进而为整个网络注入了非线性因素。此时, 神经网络就可以拟合各种曲线。如果不使用激活函数，整个网络虽然看起来复杂，其本质还相当于一种线性模型，如下公式所示:
另外通过图像可视化的形式理解：
实验表明：增加激活函数之后, 对于线性不可分的场景，神经网络的拟合能力更强。
常用激活函数
- 激活函数总结
| 激活函数 | 特点与描述 | 输出范围 | 常见用途 | 优点 | 缺点 |
|---|---|---|---|---|---|
| Sigmoid | 适用于浅层网络；容易梯度消失；输出为概率 | (0, 1) | 二分类输出层 | 输出可解释为概率 | 梯度消失；收敛慢 |
| ReLU | 隐藏层最常用；max(0, x)；训练快 | [0, +∞) | 各类深度网络的隐藏层 | 计算简单；不易梯度消失；稀疏性 | 负区间梯度为 0；可能 dead ReLU(实际很少见) |
| Softmax | 输出概率分布；总和为 1 | (0, 1)，且总和为1 | 多分类输出层 | 概率解释清晰 | 大数值不稳定（CE损失内部已处理） |
| Tanh | 类似 sigmoid，但零均值；仍会梯度消失 | (-1, 1) | 浅层隐藏层；归一化场景 | 比 sigmoid 更稳定 | 深层仍梯度消失 |
- 激活函数选择指南
| 场景 | 推荐激活函数 |
|---|---|
| 隐藏层 | ReLU > Leaky ReLU > PReLU > Tanh > Sigmoid |
| 二分类（输出层） | Sigmoid + BCELoss 或 Softmax + CrossEntropyLoss（内部已 softmax） |
| 多分类（输出层） | Softmax + CrossEntropyLoss（内部自动 softmax） |
| 回归任务（输出层） | Identity（默认，无激活） |
| - 回归值必须 ≥0 | ReLU |
| - 回归值在 (0,1) | Sigmoid（需对标签归一化） |
| - 回归值在 (-1,1) | Tanh（需标签归一化） |
sigmoid
$\(\operatorname{sigmoid}(x) = \frac{1}{1 + e^{-x}}\)$
输出范围(0,1)，适用于二分类任务的输出层, 输出为正类别的概率。最大问题是容易出现梯度消失，目前使用较少
sigmoid导数：
\[\frac{d}{dx} \operatorname{sigmoid}(x) = \frac{e^{-x}}{(1 + e^{-x})^2} = \operatorname{sigmoid}(x)\left(1-\operatorname{sigmoid}(x)\right)\]
x=torch.tensor([-6,6], dtype=torch.float)
y=torch.sigmoid(x)
print(f"y: {y}")
y: tensor([0.0025, 0.9975])
# 1. 创建画布和坐标轴, 1行2列.
fig, axes = plt.subplots(1, 2, figsize=(12, 5))
# 2. 生成 -10 ~ 10之间的 1000个数据点.
x = torch.linspace(-10, 10, 1000)
# 3. 上述1000个点, 输入sigmoid(x)
y = torch.sigmoid(x)
# 4. 在第1个子图中绘制sigmoid激活函数的图像.
axes[0].plot(x, y)
axes[0].set_title('sigmoid激活函数图像')
axes[0].grid()
# 标注点
x_points = torch.tensor([-6.0, 6.0])
y_points = torch.sigmoid(x_points)
axes[0].scatter(x_points, y_points, color='red')
for xp, yp in zip(x_points, y_points):
axes[0].annotate(
f"({xp.item():.0f}, {yp.item():.3f})",
xy=(xp, yp),
xytext=(xp, yp + 0.2),
arrowprops=dict(arrowstyle='->')
)
# 5. 在第2个图上, 绘制sigmoid激活函数的导数图像.
# 重新生成 -10 ~ 10之间的 1000个数据点.
# 参1: 起始值, 参2: 结束值, 参3: 元素的个数, 参4: 是否需要求导.
x = torch.linspace(-10, 10, 1000, requires_grad=True)
# 上述1000个点, 输入sigmoid进行求导，sum().backward()
torch.sigmoid(x).sum().backward()
# 绘制图像
axes[1].plot(x.detach(), x.grad)
axes[1].set_title('sigmoid激活函数导数图像')
axes[1].grid()
# 标注导数点
x_points = torch.tensor([-6.0, 6.0])
y_deriv = torch.sigmoid(x_points)*(1 - torch.sigmoid(x_points))
axes[1].scatter(x_points, y_deriv, color='red')
for xp, yp in zip(x_points, y_deriv):
axes[1].annotate(
f"({xp.item():.0f}, {yp.item():.3f})",
xy=(xp, yp),
xytext=(xp, yp + 0.05),
arrowprops=dict(arrowstyle='->')
)
plt.tight_layout()
plt.show()
ReLU
\[ReLU(x) = max(0, x)\]
特点: 计算公式为max(0,x), 计算简单，收敛快，常用于隐藏层, 目前使用最多. 只考虑 正样本, 可以使用 LeakyReLU, PReLU 来考虑 负样本.
# 1. 创建画布和坐标轴, 1行2列.
fig, axes = plt.subplots(1, 2, figsize=(12, 5))
# 2. 生成 -10 ~ 10之间的 1000个数据点.
x = torch.linspace(-10, 10, 1000)
# 3. 上述1000个点, 输入ReLU(x)
y = torch.relu(x)
# 4. 在第1个子图中绘制ReLU激活函数的图像.
axes[0].plot(x, y)
axes[0].set_title('ReLU激活函数图像')
axes[0].grid()
# 5. 在第2个图上, 绘制ReLU激活函数的导数图像.
# 重新生成 -10 ~ 10之间的 1000个数据点.
# 参1: 起始值, 参2: 结束值, 参3: 元素的个数, 参4: 是否需要求导.
x = torch.linspace(-10, 10, 1000, requires_grad=True)
# 上述1000个点, 输入ReLU进行求导，sum().backward()
torch.relu(x).sum().backward()
# 绘制图像
axes[1].plot(x.detach(), x.grad)
axes[1].set_title('ReLU激活函数导数图像')
axes[1].grid()
plt.tight_layout()
plt.show()
softmax
$\(softmax(x_i) = \frac{e^{x_i}}{\sum_j e^{x_j}}\)$
特点: 输出概率分布, 概率总和为1，用于多分类
# 1. 定义张量, 记录: 分类数据.
scores = torch.tensor([0.2, 0.02, 0.15, 0.15, 1.3, 0.5, 0.06, 1.1, 0.05, 3.75])
# scores = torch.tensor([[0.2, 0.35, 0.1, 0.46], [0.1, 0.13, 0.05, 2.79]])
# 2. dim = 0, 按行计算
y = torch.softmax(scores, dim=0)
print(f"softmax(x): {y}")
print(f"sum: {y.sum()}")
# 3. 再次softmax
z = torch.softmax(y, dim=0)
print(f"softmax(softmax(x)): {z}")
print(f"sum: {z.sum()}")
# 绘制柱状图
plt.subplot(2,1,1)
plt.title("softmax(x)")
plt.bar(range(len(y)), y)
plt.subplot(2,1,2)
plt.title("softmax(softmax(x))")
plt.bar(range(len(z)), z)
plt.tight_layout()
plt.show()
softmax(x): tensor([0.0212, 0.0177, 0.0202, 0.0202, 0.0638, 0.0287, 0.0185, 0.0522, 0.0183,
0.7392])
sum: 1.0000001192092896
softmax(softmax(x)): tensor([0.0899, 0.0896, 0.0898, 0.0898, 0.0938, 0.0906, 0.0897, 0.0927, 0.0897,
0.1844])
sum: 1.0
二分类任务中：Sigmoid 与 Softmax 的结果是等价的
| 二分类任务 | $\(sigmoid(x)=\frac{1}{1+e^{-x}}\)$ | $\(\text{softmax}(x_i)=\frac{e^{x_i}}{\sum_j e^{x_j}}\)$ |
|---|---|---|
| 输出层网络结构 | nn.Linear(n, 1) |
nn.Linear(n, 2) |
| 原始输出 logits | [y] |
[y0, y1] |
| 人为设定正类 | y表示正类的logit | y1表示正类的logit,注意是人为设定的，可以修改 |
| 正类概率输出 | $\(\frac{1}{1+e^{-y}}\)$ | $\(\frac{e^{y_1}}{e^{y_0}+e^{y_1}}=\frac{1}{1+e^{\,y_0-y_1}}\)$ |
| 负类概率输出 | $\(\frac{e^{-y}}{1+e^{-y}}=\frac{1}{1+e^{y}}\)$ | $\(\frac{e^{y_0}}{e^{y_0}+e^{y_1}}=\frac{1}{1+e^{\,y_1-y_0}}\)$ |
| 转化关系 | y=y1-y0 | 注意是sigmoid对应的logits= softmax对应的正类logits - 负类logits |
| 举例 | y=2, 正类概率:0.8808, 负类概率:1-0.8808=0.1192 | [y0, y1]=[1,3], 正类(y1)概率:0.8808, 负类(y0)概率:0.1192 |
import torch
logits = torch.tensor([[1, 3]], dtype=torch.float)
print(logits)
softmax = torch.softmax(logits, dim=1)
print(softmax)
logits = logits[:,1]-logits[:,0]
sigmoid = torch.sigmoid(logits)
print(sigmoid)
tensor([[1., 3.]])
tensor([[0.1192, 0.8808]])
tensor([0.8808])
Tanh
\[tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}} = \frac{1 - e^{-2x}}{1 + e^{-2x}}\]
特点: 输出范围[-1,1]，比Sigmoid收敛快
tanh函数的导数是：
\[\frac{d}{dx} {tanh}(x) = 1 - {tanh}^2(x)\]
x=torch.tensor([-3,3], dtype=torch.float32)
y=torch.tanh(x)
print(f"y: {y}")
y: tensor([-0.9951, 0.9951])
# 1. 创建画布和坐标轴, 1行2列.
fig, axes = plt.subplots(1, 2, figsize=(12, 5))
# 2. 生成 -10 ~ 10之间的 1000个数据点.
x = torch.linspace(-10, 10, 1000)
# 3. 上述1000个点, 输入tanh(x)
y = torch.tanh(x)
# 4. 在第1个子图中绘制tanh激活函数的图像.
axes[0].plot(x, y)
axes[0].set_title('tanh激活函数图像')
axes[0].grid()
# 标注点：x = -3, 3
x_points = torch.tensor([-3.0, 3.0])
y_points = torch.tanh(x_points)
axes[0].scatter(x_points, y_points, color='red')
for xp, yp in zip(x_points, y_points):
axes[0].annotate(
f"({xp.item():.0f}, {yp.item():.3f})",
xy=(xp, yp),
xytext=(xp, yp + 0.2),
arrowprops=dict(arrowstyle='->')
)
# 5. 在第2个图上, 绘制tanh激活函数的导数图像.
# 重新生成 -10 ~ 10之间的 1000个数据点.
# 参1: 起始值, 参2: 结束值, 参3: 元素的个数, 参4: 是否需要求导.
x = torch.linspace(-10, 10, 1000, requires_grad=True)
# 上述1000个点, 输入tanh进行求导，sum().backward()
torch.tanh(x).sum().backward()
# 绘制图像
axes[1].plot(x.detach(), x.grad)
axes[1].set_title('tanh激活函数导数图像')
axes[1].grid()
# 标注导数点：x = -3, 3
x_points = torch.tensor([-3.0, 3.0])
y_deriv = (1 - torch.tanh(x_points)**2) # tanh' = 1 - tanh^2
axes[1].scatter(x_points, y_deriv, color='red')
for xp, yp in zip(x_points, y_deriv):
axes[1].annotate(
f"({xp.item():.0f}, {yp.item():.3f})",
xy=(xp, yp),
xytext=(xp, yp + 0.05),
arrowprops=dict(arrowstyle='->')
)
plt.tight_layout()
plt.show()
# 激活函数演示
x = torch.linspace(-5, 5, 100)
# sigmoid
sigmoid = torch.sigmoid(x)
# tanh
tanh = torch.tanh(x)
# ReLU
relu = torch.relu(x)
# 绘制
fig, axes = plt.subplots(2, 2, figsize=(10, 8))
axes[0, 0].plot(x.numpy(), sigmoid.numpy())
axes[0, 0].set_title('sigmoid')
axes[0, 0].grid(True)
axes[0, 1].plot(x.numpy(), tanh.numpy())
axes[0, 1].set_title('tanh')
axes[0, 1].grid(True)
axes[1, 0].plot(x.numpy(), relu.numpy())
axes[1, 0].set_title('ReLU')
axes[1, 0].grid(True)
# softmax
softmax_input = torch.randn(5)
softmax_output = torch.softmax(softmax_input, dim=0)
axes[1, 1].bar(range(5), softmax_output.numpy())
axes[1, 1].set_title('softmax')
axes[1, 1].set_ylim([0, 1])
plt.tight_layout()
plt.show()
import torch.nn.functional as F
# 激活函数演示
x = torch.linspace(-5, 5, 100)
# sigmoid
sigmoid = torch.sigmoid(x)
# tanh
tanh = torch.tanh(x)
# ReLU
relu = torch.relu(x)
# gelu
gelu = F.gelu(x)
# 绘制
fig, axes = plt.subplots(2, 2, figsize=(10, 8))
axes[0, 0].plot(x.numpy(), sigmoid.numpy())
axes[0, 0].set_title('sigmoid')
axes[0, 0].grid(True)
axes[0, 1].plot(x.numpy(), tanh.numpy())
axes[0, 1].set_title('tanh')
axes[0, 1].grid(True)
axes[1, 0].plot(x.numpy(), relu.numpy())
axes[1, 0].set_title('ReLU')
axes[1, 0].grid(True)
axes[1, 1].plot(x.numpy(), gelu.numpy())
axes[1, 1].set_title('GELU')
axes[1, 1].grid(True)
# # softmax
# softmax_input = torch.randn(5)
# softmax_output = torch.softmax(softmax_input, dim=0)
# axes[1, 1].bar(range(5), softmax_output.numpy())
# axes[1, 1].set_title('softmax')
# axes[1, 1].set_ylim([0, 1])
plt.tight_layout()
plt.show()
3. 参数初始化
我们在构建网络之后，网络中的参数是需要初始化的。需要初始化的参数主要有权重和偏置，偏置一般初始化为0即可，而对权重的初始化则会更加重要。
参数初始化的作用：
- 防止梯度消失或爆炸：初始权重值过大或过小会导致梯度在反向传播中指数级增大或缩小。
- 提高收敛速度：合理的初始化使得网络的激活值分布适中，有助于梯度高效更新。
- 打破对称性：权重的初始化需要打破对称性，否则网络的学习能力会受到限制。
为什么要打破模型参数的对称性，防止模型参数完全相同
对称性是指：
某一层所有神经元有
- 完全相同的初始权重
- 完全相同的输入
- 完全相同的激活函数
会导致模型训练无效：
- 前向传播结果完全一样
- 反向传播时梯度完全一样
- 梯度下降更新无效 → 所有参数永远一样
- z1 = x1 * w11 + x2 * w12 + x3 * w13 + b1
- z2 = x1 * w21 + x2 * w22 + x3 * w23 + b2
- z3 = x1 * w31 + x2 * w32 + x3 * w33 + b3
- z4 = x1 * w41 + x2 * w42 + x3 * w43 + b4
- z5 = x1 * w51 + x2 * w52 + x3 * w53 + b5
- w1 = w0 - lr*grad
# 演示 模型参数的对称性 导致了 模型训练无效
layer = nn.Linear(3, 5, bias=True)
nn.init.constant_(layer.weight, 0)
nn.init.constant_(layer.bias, 0)
# 打印模型参数
print(f"模型参数:")
print(layer.weight)
print(layer.bias)
print("-"*60)
x = torch.tensor([[1.0, 2.0, 3.0]])
# 模型前向传播
print(f"前向传播:")
print(layer(x))
print("-"*60)
y = layer(x)
loss = y.sum() # 简单损失
loss.backward()
# 获取梯度
print(f"梯度:")
print(layer.weight.grad)
print(layer.bias.grad)
print("-"*60)
# 模型参数更新
layer.weight.data -= 0.01 * layer.weight.grad
layer.bias.data -= 0.01 * layer.bias.grad
print(f"更新后的模型参数:")
print(layer.weight)
print(layer.bias)
print("-"*60)
# 结论
print("结论:")
print("模型参数的对称性 导致 梯度完全相同，更新后的模型参数完全相同，模型训练无效")
模型参数:
Parameter containing:
tensor([[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]], requires_grad=True)
Parameter containing:
tensor([0., 0., 0., 0., 0.], requires_grad=True)
------------------------------------------------------------
前向传播:
tensor([[0., 0., 0., 0., 0.]], grad_fn=<AddmmBackward0>)
------------------------------------------------------------
梯度:
tensor([[1., 2., 3.],
[1., 2., 3.],
[1., 2., 3.],
[1., 2., 3.],
[1., 2., 3.]])
tensor([1., 1., 1., 1., 1.])
------------------------------------------------------------
更新后的模型参数:
Parameter containing:
tensor([[-0.0100, -0.0200, -0.0300],
[-0.0100, -0.0200, -0.0300],
[-0.0100, -0.0200, -0.0300],
[-0.0100, -0.0200, -0.0300],
[-0.0100, -0.0200, -0.0300]], requires_grad=True)
Parameter containing:
tensor([-0.0100, -0.0100, -0.0100, -0.0100, -0.0100], requires_grad=True)
------------------------------------------------------------
结论:
模型参数的对称性 导致 梯度完全相同，更新后的模型参数完全相同，模型训练无效
常见参数初始化方法
- 均匀分布初始化：权重参数初始化从区间均匀随机取值，默认区间为（0，1）。可以设置为在 $\(U(\frac{-1}{\sqrt{d}}, \frac{1}{\sqrt{d}})\)$ 均匀分布中生成当前神经元的权重，其中d为神经元的输入维度。
-
优点：能有效打破对称性
-
缺点：随机选择范围不当可能导致梯度问题
-
适用场景：浅层网络或低复杂度模型。隐藏层1-3层，总层数不超过5层。
print("1. 均匀分布初始化")
# 创建一个线性层，输入维度3，输出维度5
linear = nn.Linear(3, 5)
# 对权重w 进行初始化
nn.init.uniform_(linear.weight)
# 对偏置b 进行初始化
nn.init.uniform_(linear.bias)
# 打印
print(linear.weight.data)
print(linear.bias.data)
print("-"*60)
1. 均匀分布初始化
tensor([[0.5843, 0.7241, 0.6673],
[0.3415, 0.0420, 0.7143],
[0.6089, 0.1226, 0.5414],
[0.0546, 0.1942, 0.7059],
[0.7172, 0.8982, 0.1690]])
tensor([0.4065, 0.5680, 0.8627, 0.7599, 0.5236])
------------------------------------------------------------
-
正态分布初始化：随机初始化从均值为0，标准差是1的高斯分布 $\(N(0,1)\)$中取样，使用一些很小的值对参数W进行初始化
-
优点：能有效打破对称性
-
缺点：随机选择范围不当可能导致梯度问题
-
适用场景：浅层网络或低复杂度模型。隐藏层1-3层，总层数不超过5层。
print("2. 正态分布初始化")
# 创建一个线性层，输入维度3，输出维度5
linear = nn.Linear(3, 5)
# 对权重w 进行初始化
nn.init.normal_(linear.weight)
# 对偏置b 进行初始化
nn.init.normal_(linear.bias)
# 打印
print(linear.weight.data)
print(linear.bias.data)
print("-"*60)
2. 正态分布初始化
tensor([[ 0.1715, -0.7074, -1.4796],
[-0.1207, 0.8655, -0.4058],
[-0.4245, -1.0690, -0.1430],
[ 0.1090, 0.9648, 0.5519],
[-0.9374, -0.7841, 1.3551]])
tensor([ 0.4894, -1.2228, 0.0186, 2.4750, -1.2078])
------------------------------------------------------------
- 全0初始化：将神经网络中的所有权重参数初始化为0
- 优点：实现简单
- 缺点：无法打破对称性，所有神经元更新方向相同，无法有效训练
- 适用场景：几乎不使用，仅用于偏置项的初始化
print("3. 全0初始化")
# 创建一个线性层，输入维度3，输出维度5
linear = nn.Linear(3, 5)
# 对权重w 进行初始化
nn.init.zeros_(linear.weight)
# 对偏置b 进行初始化
nn.init.zeros_(linear.bias)
# 打印
print(linear.weight.data)
print(linear.bias.data)
print("-"*60)
3. 全0初始化
tensor([[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]])
tensor([0., 0., 0., 0., 0.])
------------------------------------------------------------
- 全1初始化：将神经网络中的所有权重参数初始化为1
- 优点：实现简单
- 缺点
- 无法打破对称性，所有神经元更新方向相同，无法有效训练
- 会导致激活值在网络中呈指数增长，容易出现梯度爆炸
- 适用场景
- 测试或调试：比如验证神经网络是否能正常前向传播和反向传播
- 特殊模型结构：某些稀疏网络或特定的自定义网络中可能需要手动设置部分参数为1
print("4. 全1初始化")
# 创建一个线性层，输入维度3，输出维度5
linear = nn.Linear(3, 5)
# 对权重w 进行初始化
nn.init.ones_(linear.weight)
# 对偏置b 进行初始化
nn.init.ones_(linear.bias)
# 打印
print(linear.weight.data)
print(linear.bias.data)
print("-"*60)
4. 全1初始化
tensor([[1., 1., 1.],
[1., 1., 1.],
[1., 1., 1.],
[1., 1., 1.],
[1., 1., 1.]])
tensor([1., 1., 1., 1., 1.])
------------------------------------------------------------
- 固定值初始化：将神经网络中的所有权重参数初始化为某个固定值
- 优点：实现简单
- 缺点
- 无法打破对称性，所有神经元更新方向相同，无法有效训练
- 初始权重过大或过小可能导致梯度爆炸或梯度消失
- 适用场景
- 测试或调试
- 偏置初始化：偶尔可以将偏置初始化为小的正值（如 0.1）
print("5. 固定值初始化")
# 创建一个线性层，输入维度3，输出维度5
linear = nn.Linear(3, 5)
# 对权重w 进行初始化
nn.init.constant_(linear.weight, 6)
# 对偏置b 进行初始化
nn.init.constant_(linear.bias, 6)
# 打印
print(linear.weight.data)
print(linear.bias.data)
print("-"*60)
5. 固定值初始化
tensor([[6., 6., 6.],
[6., 6., 6.],
[6., 6., 6.],
[6., 6., 6.],
[6., 6., 6.]])
tensor([6., 6., 6., 6., 6.])
------------------------------------------------------------
- kaiming初始化(针对ReLU)，也叫做HE初始化：专为ReLU和其变体设计，考虑到ReLU激活函数的特性，对输入维度进行缩放
- HE初始化分为正态分布的HE初始化、均匀分布的HE初始化
- 正态分布的he初始化
- w权重值
$\(W \sim N(0, \sqrt{\frac{2} {fan\_in}})\)$ - std值较大，w权重值离均值0分布相对较广，计算得到的内部状态值有较大的正值或负值
- 均匀分布的he初始化
- w权重值
$\(W \sim U(-\sqrt{\frac{6} {fan\_in}}, \sqrt{\frac{6} {fan\_in}})\)$ fan_in当前层的输入维度，即当前层接受的来自上一层的神经元的数量。简单来说，就是当前层接收多少个输入
- 优点：适合 ReLU，能保持梯度稳定
- 缺点：对非 ReLU 激活函数效果一般
- 适用场景：深度网络(10层及以上)，使用 ReLU、Leaky ReLU 激活函数
print("6. kaiming 初始化")
# 创建一个线性层，输入维度3，输出维度5
linear = nn.Linear(3, 5)
# 正态分布的he初始化
print("正态分布的he初始化")
# 对权重w 进行初始化
nn.init.kaiming_normal_(linear.weight)
# 打印
print(linear.weight.data)
# 均匀分布的he初始化
print("均匀分布的he初始化")
# 对权重w 进行初始化
nn.init.kaiming_uniform_(linear.weight)
# 打印
print(linear.weight.data)
print("-"*60)
6. kaiming 初始化
正态分布的he初始化
tensor([[ 0.5621, 0.2847, 0.5249],
[-0.5627, 0.7431, -0.3000],
[-0.5720, 0.2098, -0.8993],
[ 0.2722, -1.2356, 1.8173],
[ 1.0670, 1.0488, 0.3180]])
均匀分布的he初始化
tensor([[-0.7910, 0.5988, -1.3454],
[ 1.2842, -1.3248, -0.2973],
[-0.6440, 0.0660, 0.2353],
[-0.6605, 1.1916, -0.4887],
[ 0.1652, -0.9336, 0.0619]])
------------------------------------------------------------
- xavier初始化(针对Sigmoid、Tanh)，也叫做Glorot初始化：根据网络输入和输出的维度自动选择权重范围，使输入和输出的方差相同
-
xavier初始化分为正态分布的xavier初始化、均匀分布的xavier初始化
- 正态分布的Xavier初始化
- w权重值
$\(W \sim N(0, \sqrt{\frac{2} {fan\_in + fan\_out}})\)$ - std值较小，w权重值离均值0分布相对集中，计算得到的内部状态值有较小的正值或负值
- 均匀分布的Xavier初始化
- w权重值
$\(W \sim U(-\sqrt{\frac{6} {fan\_in + fan\_out}}, \sqrt{\frac{6} {fan\_in + fan\_out}})\)$ fan_in当前层的输入维度，当前层接受的来自上一层的神经元的数量。简单来说，就是当前层接收多少个输入fan_out当前层的输出维度，当前层输出的神经元的数量，也就是当前层会传递给下一层的神经元的数量。简单来说，就是当前层会产生多少个输出。
-
优点：适用于Sigmoid、Tanh 等激活函数，解决梯度消失问题
- 缺点：对 ReLU 等激活函数表现欠佳
- 适用场景：深度网络(10层及以上)，使用 Sigmoid 或 Tanh 激活函数
print("7.xavier 初始化")
# 创建一个线性层，输入维度3，输出维度5
linear = nn.Linear(3, 5)
# 正态分布的xavier初始化
print("正态分布的xavier初始化")
# 对权重w 进行初始化
nn.init.xavier_normal_(linear.weight)
# 打印
print(linear.weight.data)
# 均匀分布的xavier初始化
print("均匀分布的xavier初始化")
# 对权重w 进行初始化
nn.init.xavier_uniform_(linear.weight)
# 打印
print(linear.weight.data)
print("-"*60)
7.xavier 初始化
正态分布的xavier初始化
tensor([[ 1.8420e-01, -5.0197e-01, 2.1592e-01],
[ 3.8240e-02, 4.3374e-01, -5.6280e-01],
[ 1.8755e-01, 3.0699e-01, -6.3362e-01],
[-6.4917e-01, -4.0213e-01, 2.7394e-01],
[ 2.1878e-04, -4.5354e-01, -4.0259e-01]])
均匀分布的xavier初始化
tensor([[ 0.0448, 0.2639, -0.5287],
[-0.3559, -0.0131, 0.2428],
[ 0.6667, 0.1415, -0.3036],
[ 0.0828, -0.5579, 0.8583],
[-0.5814, 0.8398, 0.0760]])
------------------------------------------------------------
常见面试题
1. 简述人工神经网络的计算模型
人工神经网络 具有多层网络结构，每层网络包含多个神经元，每个神经元都有权重和偏置，能够执行 加权求和+激活函数的运算：
- 加权求和：\(z = w \cdot x + b\)（\(w\)权重，\(x\)输入，\(b\)偏置）
- 激活值：\(a = f(z)\)（\(f\)是激活函数）
- 前向传播：数据从输入层→隐藏层→输出层逐层计算
- 特点：层与层间全连接，同层神经元无连接
2. 激活函数的作用是什么，常用的激活函数有哪些？
- 引入非线性，使网络具有非线性表达能力
- 常用的激活函数有ReLU, LeakyReLU, softmax, sigmoid等
3. 为什么sigmoid激活函数有梯度消失问题，如何解决？
- 原因：Sigmoid导数为 \(sigmoid'(x) = sigmoid(x)(1-sigmoid(x))\)，最大值仅为0.25。深层网络反向传播时，梯度连乘会指数衰减 为十分接近0（如\(0.25^{10} \approx 0\)），导致深层参数无法更新
- 解决方案：
- 使用ReLU激活函数（导数为0或1，梯度不衰减）
- Batch Normalization归一化（标准化输入分布）
- 残差连接（Skip Connection，绕过深层）
- 采用更好的初始化方法（Xavier、Kaiming）
4. ReLU激活函数的公式是什么，有哪些特点？
- 公式 \(max(0,x)\)
- 特点：
- 简单快速
- 梯度不衰减
- 隐藏层首选
- 存在Dead ReLU问题，但实际情况十分少见
5. 如何选择激活函数？
隐藏层激活函数：
1. ReLU
- 最快最简单，大多数场景够用
- 可能出现Dead ReLU，实际很少发生
- 用法：torch.nn.ReLU()
-
LeakyReLU（当ReLU不理想时）
- 完全解决Dead ReLU问题
- 速度与ReLU接近
- 用法：torch.nn.LeakyReLU(negative_slope=0.01) -
GELU（Transformer时代推荐）
- 平滑可微，性能优秀
- 计算量是ReLU的3-4倍
- 用法：torch.nn.GELU()或torch.nn.GELU(approximate='tanh')
输出层激活函数：
- 多分类：Linear + CrossEntropyLoss(内部使用Softmax)
- 二分类：Linear + CrossEntropyLoss(内部使用Softmax) 或 Sigmoid + BCELoss
- 回归：无激活函数（线性输出）
6. 说一下kaiming初始化的原理和应用场景
kaiming初始化有两种方式：正太分布 和 均匀分布
- 正态分布的kaiming初始化为
$\(模型权重W \sim N(0, \sqrt{\frac{2} {fan\_in}})\)$
- fan_in 当前层的输入维度
- 均匀分布的kaiming初始化为
$\(模型权重W \sim U(-\sqrt{\frac{6} {fan\_in}}, \sqrt{\frac{6} {fan\_in}})\)$
- fan_in 当前层的输入维度
应用场景：
适用于 使用ReLU的深度网络的模型权重的参数初始化，能够保持梯度稳定
代码作业
- 1.使用torch.randn创建形状为(3,4)的随机张量，开启梯度计算，使用detach()方法将张量转换为NumPy数组, 并打印。
- 2.修改pytorch模拟线性回归中的参数或设置，比如n_samples改为200，batch_size改为32，epochs改为200，学习率改为0.001，loss_fn改为MAE，并绘制训练过程中损失值的变化曲线。
- 3.使用代码演示 二分类任务中，softmax和sigmoid函数的等价关系，比如原始输出的logits为：两个类别[1,3]，单个类别为[2],分别使用softmax和sigmoid计算正类别和负类别的概率，并打印。
- 4.创建一个线性层，输入维度3，输出维度5，权重初始化采用kaiming正态分布，偏置初始化采用全0初始化，并打印初始化的权重和偏置。
扩展-推荐学习资料
-
动手学机器学习
https://hml.boyuai.com/ -
动手学深度学习
https://zh-v2.d2l.ai/ -
动手学强化学习
https://hrl.boyuai.com/
扩展-GELU激活函数（Gaussian Error Linear Unit）
定义公式
GELU 的常用近似形式（Transformer 默认版本）为：
\[
\text{GELU}(x)
= 0.5x\left(1 + \tanh\left(\sqrt{\frac{2}{\pi}}(x + 0.044715x^3)\right)\right)
\]
其精确形式为：
\[
\text{GELU}(x) = x \cdot \Phi(x)
\]
其中 \(\Phi(x)\) 是标准正态分布的 CDF。
2. 特点
- 平滑非线性：相比 ReLU 的硬截断，GELU 更平滑，有助于梯度稳定。
- 概率式门控：输出大小取决于输入落在正态分布右侧的概率，具有一定“自适应性”。
- 轻微保留负值：不像 ReLU 将负值全部置零，有助于表示能力提升。
- 性能优势明显：在 NLP 模型中普遍优于 ReLU、Swish 等传统激活函数。
3. 应用场景
- Transformer / BERT / GPT 系列默认激活函数
- Vision Transformer (ViT)
- 大多数现代 NLP、CV 大模型中广泛应用
- 已逐渐成为“默认的高性能激活函数”
import torch
import torch.nn.functional as F
import matplotlib.pyplot as plt
# 1. 创建画布和坐标轴（1行2列）
fig, axes = plt.subplots(1, 2, figsize=(12, 5))
# =====================
# GELU 激活函数曲线
# =====================
# 2. 生成 -10 到 10 的 1000 个数据点
x = torch.linspace(-10, 10, 1000)
# 3. GELU(x)
y = F.gelu(x)
# 4. 绘制 GELU 激活函数
axes[0].plot(x, y)
axes[0].set_title('GELU 激活函数')
axes[0].grid()
# =====================
# GELU 激活函数导数
# =====================
# 5. 重新生成可求导的数据
x = torch.linspace(-10, 10, 1000, requires_grad=True)
# 6. GELU(x) 求导
F.gelu(x).sum().backward()
# 7. 绘制导函数曲线
axes[1].plot(x.detach(), x.grad)
axes[1].set_title('GELU 激活函数导数')
axes[1].grid()
plt.tight_layout()
plt.show()
💻 代码
Day03 代码：自动微分 / 梯度下降 / 线性回归
4 个文件📄 01_自动微分模块入门案例.py 1.5KB
""" 演示 自动微分模块 如何 计算梯度/导数 介绍： 梯度：损失函数对模型参数的导数/梯度，针对某个确定的模型参数，损失函数的导函数的数值，模型参数就是权重w,偏置b 损失函数对模型参数的梯度不需要手动计算，是loss.backward反向传播过程中pytorch调用自动微分模块来自动计算的 梯度下降法的参数更新公式： w新 = w旧 - 学习率 * 梯度 过程： 1.定义模型权重参数 2.定义一个损失函数 3.backward()反向传播计算导数/梯度 4.w.grad获取导数值/梯度 5.更新参数：w新 = w旧 - 学习率 * 梯度 """ # 导包 import torch # 1.定义变量,模拟模型权重参数w w = torch.tensor([10,20],requires_grad=True,dtype=torch.float32) print(f"w:{w}, requires_grad: {w.requires_grad}") # 2.定义函数，模拟损失函数 loss = w**2 + 10 # [110,410] print(f"loss:{loss}") # 1D: (2,) # 打印导函数 print(f"loss.grad_fn:{loss.grad_fn}") loss_mean = loss.mean() print(f"loss_mean:{loss_mean}") # loss' = 2*w = [2*w1,2*w2] # loss_mean' = 2*w/2 = [w1,w2] # 3.需要执行反向传播来调用自动微分模块 # 反向传播的张量必须是标量 loss.mean().backward() # 4.打印loss_mean对w的梯度 print(f"w.grad:{w.grad}") # [w1,w2]=[10.0,20.0] # 5.更新参数：w新 = w旧 - 学习率 * 梯度 w.data = w.data - 0.01 * w.grad.data # 10 - 0.01*20=9.8 # [10,20] - 0.01*[10.,20.]=[9.9,19.8] print(f"w:{w}")
📄 02_梯度下降法求最小值.py 1.9KB
""" 演示 梯度下降法 求损失函数的最小值 循环实现，计算梯度，更新参数w 题目： # 求 loss = w**2 + 20 的极小值点 并打印loss是最小值时 w的值(梯度) 计算过程： # 1 定义点 w=10, requires_grad=True, dtype=torch.float32 # 2 定义函数 loss = w**2 + 20 # 3 利用梯度下降法 循环迭代 求最优解/最小值 # 3.1 前向传播,计算损失 # 3.2 梯度清零 w.grad.zero_() # 3.3 反向传播 loss.backward() # 3.4 更新参数 w.data = w.data - 0.01 * w.grad """ # 导包 import torch import matplotlib.pyplot as plt # 设置中文字体 plt.rcParams['font.sans-serif'] = ['SimHei'] # 微软雅黑 plt.rcParams['axes.unicode_minus'] = False # 解决负号显示问题 # 1 定义点 w=10, requires_grad=True, dtype=torch.float32 w = torch.tensor([10.0],requires_grad=True,dtype=torch.float32) # 2 定义函数 loss = w**2 + 20 loss = w**2 + 20 # 120.0 # 3 利用梯度下降法 循环迭代 求最优解/最小值 print(f"初始值：w:{w.data}, w.grad: {w.grad}, loss.mean():{loss.mean()}") # 记录损失均值 loss_list = [] for i in range(500): # 3.1 前向传播, 计算损失 loss = w**2 + 20 # 3.2 梯度清零 w.grad.zero_() if w.grad is not None: w.grad.zero_() # 3.3 反向传播 loss.backward() loss.mean().backward() print(f"w.grad:{w.grad.data}") # loss_mean' = 2*w # 3.4 更新参数 w.data = w.data - 0.01 * w.grad w.data = w.data - 0.01 * w.grad.data # loss_mean' = 2*w = w = [2*w1] print(f"第{i+1}次迭代：w:{w.data}, w.grad:{w.grad.data}, loss.mean():{loss.mean()}") # 添加到loss_list loss_list.append(loss.mean().item()) # 绘制损失曲线 iterations = range(500) plt.figure(figsize=(12,8)) plt.plot(iterations,loss_list) plt.xlabel("迭代次数") plt.ylabel("损失均值loss.mean") plt.grid() plt.show()
📄 03_演示detach()函数.py 897B
""" 演示 detach() 函数的功能，解决 开启梯度计算 requires_grad=True 的张量，无法转为numpy对象的问题 介绍： 一个张量Tensor，开启了梯度计算requires_grad=True, 就无法Tensor.numpy(),需要Tensor.detach().numpy()来转为numpy数组 需要掌握 n1 = t1.detach().numpy() """ # 导包 import torch # 1.定义张量，开启梯度计算requires_grad = True w = torch.tensor([1.0,2.0,3.0], requires_grad=True, dtype=torch.float32) print(f"w:{w}, shape: {w.shape}, requires_grad: {w.requires_grad}") # # 2.直接转为numpy数组 # n1 = w.numpy() # print(f"n1:{n1}, shape: {n1.shape}") # 3.使用.detach().numpy()来转为numpy数组 n2 = w.detach().numpy() print(f"n2:{n2}, shape: {n2.shape}") # 4.查看是否共享内存/浅拷贝 n2[0]=5.0 print(f"w:{w}, shape: {w.shape}") print(f"n2:{n2}, shape: {n2.shape}")
📄 04_pytorch框架模拟线性回归.py 5.9KB
""" 演示 pytorch 框架 模拟线性回归 案例工作流： 1.准备数据集 numpy数组 -> 张量 -> 数据集对象Dataset -> 数据加载器DataLoader（分批次加载） 2.构建神经网络模型 nn.Linear()线性回归模型 3.设置损失函数和优化器 nn.MSELoss, optim.SGD(w新 = w旧 - 学习率 * 梯度) 4.模型训练 1.前向传播 2.计算损失 3.梯度清零 4.反向传播 5.更新参数:w新 = w旧-学习率*梯度 5.模型测试 """ # 导包 import torch # numpy对象 -> 张量 -> 数据集对象TensorDataset -> 数据加载器DataLoader from torch.utils.data import TensorDataset # 构造数据集对象 from torch.utils.data import DataLoader # 数据加载器，按批次加载数据 from torch import nn # 提供 MSE损失函数 和 线性层，线性层用来模拟线性回归模型 from torch import optim # 提供各种优化器，用于更新模型参数，公式为：w ← w - lr * grad from sklearn.datasets import make_regression # 创建线性回归的示例数据集 import matplotlib.pyplot as plt # 设置matplotlib中文字体 plt.rcParams['font.sans-serif'] = ['SimHei'] # 用来正常显示中文标签 plt.rcParams['axes.unicode_minus'] = False # 用来正常显示负号 # 1.准备数据集 # 定义函数，创建数据集 def create_dataset(): # 创建数据集对象 x, y, coef = make_regression( n_samples=100, # 100条样本 n_features=1, # 1个特征 n_targets=1, # 1个标签/目标 noise=10, # 噪声 bias=13.9, # 偏移 coef=True, # 是否返回系数 random_state=5 # 随机状态 ) # 封装成 张量 x = torch.tensor(x, dtype=torch.float32) y = torch.tensor(y, dtype=torch.float32) # 返回结果 return x, y, coef # 2.模型训练 def train(x,y,coef): # 1.准备数据集：numpy数组 -> 张量 -> 数据集对象 -> 数据加载器（分批次加载） # 1.1 创建数据集对象 dataset = TensorDataset(x, y) # 1.2 创建数据加载器 # 参数1：数据集对象；参数2：批次大小，一般64,32;参数3：是否打乱数据，训练时打乱，测试时不打乱； # 参数4：是否删除最后一个不完整的批次，一般是False # 100/16 = 6...4, 6+1 dataloader = DataLoader(dataset, batch_size=16, shuffle=True, drop_last=False) # 2.构建模型：nn.Linear() model = nn.Linear(1,1) # 3.设置损失函数和优化器 # 损失函数 loss_fn = nn.MSELoss() # 优化器,执行梯度下降法: w新 = w旧 - 学习率 * 梯度 # 参数1: 待训练的模型参数；参数2：学习率，常用0.01,0.001 optimizer = optim.SGD(model.parameters(), lr=0.01) # 4.模型训练 # 定义训练轮次 epochs = 100 # 记录 每个轮次的平均损失 total_losses = [] # 遍历轮数epochs for epoch in range(epochs): # 1.初始化 当前轮次的训练总损失 total_loss = 0.0 # 2.初始化 当前轮次的总样本数 total_samples = 0 # 3.遍历批次，遍历数据加载器，分批训练 for train_x, train_y in dataloader: # print(f"train_x:{train_x.shape}") # print(f"train_y:{train_y.shape}") # 1.前向传播，计算预测值 y_pred = model(train_x) # (16,1) # print(f"y_pred: {y_pred.shape}") # 2.计算损失 loss = loss_fn(y_pred, train_y.reshape(-1, 1)) # 3.梯度清零 optimizer.zero_grad() # 4.反向传播 loss.backward() # 5.更新参数:w新 = w旧-学习率*梯度，更新模型的参数 optimizer.step() # 6.统计 训练损失 和 样本数 # 使用train_x.shape[0]来获取当前批次的样本数量，因为最后一个批次可能不完整 total_loss += loss.item() * train_x.shape[0] total_samples += train_x.shape[0] ... # 4.计算当前轮次的平均损失 train_loss = total_loss / total_samples # 5.添加当前轮次平均损失到列表中 total_losses.append(train_loss) # 6.打印当前轮次的训练平均损失 print(f"{epoch+1}/{epochs}: | " f"{train_loss:.4f}") ... # 5.可视化训练过程 print(f"损失列表：{total_losses}") plt.figure(figsize=(15,8)) # 绘制第一个子图：训练损失曲线图 plt.subplot(1,2,1) plt.title("训练损失曲线") x_epoch = range(epochs) plt.plot(x_epoch, total_losses, label="训练损失") plt.xlabel("epoch") plt.ylabel("loss") plt.grid() plt.legend() # plt.show() # 绘制第二个子图：真实值和预测值的对比 plt.subplot(1, 2, 2) plt.title("真实值和预测值的对比") plt.scatter(x, y, label="真实散点") # 计算理论的真实线性回归直线 y_true = [v*coef.item()+13.9 for v in x] plt.plot(x, y_true, label="真实线性回归") # 计算模型预测值 with torch.no_grad(): # 关闭梯度计算 y_pred = model(x).detach().numpy() # 绘制模型预测的线性回归直线 plt.plot(x, y_pred, label="预测值") plt.xlabel("x") plt.ylabel("y") plt.grid() plt.legend() plt.show() # 测试 if __name__ == '__main__': # 1.准备数据集 x, y, coef = create_dataset() print(f"x:{x.shape}") print(f"y:{y.shape}") print(f"coef:{coef}, shape: {coef.shape}") # 绘制样本散点图 plt.scatter(x, y, label="真实样本") plt.grid() plt.xlabel("x") plt.ylabel("y") plt.legend() plt.show() # 2.模型训练 train(x, y, coef)
📎 资料
每日一图
- 📎 day03.png105.7KB
ℹ️ 离线版不含附件原文件，需要请到在线网站对应课时下载。
第4节 · 神经网络基础（二）：搭建网络
🎯 掌握激活函数、参数初始化（含 Kaiming 初始化）与 nn.Module 建模。
📖 笔记
Day04 笔记：神经网络基础（二）
Day 04 - 神经网络基础2
学习目标
- 动手实现搭建神经网络
- 掌握常用损失函数
- 掌握梯度下降算法
- 了解反向传播算法
- 掌握梯度下降的优化方法
# 导包
import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
import matplotlib.pyplot as plt
# 设置中文字体
plt.rcParams['font.sans-serif'] = ['SimHei'] # 微软雅黑
plt.rcParams['axes.unicode_minus'] = False # 解决负号显示问题
# 设置设备
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
day04_01.搭建神经网络
"""
案例：
演示 搭建神经网络 的流程
要求：
第1个隐藏层：权重初始化采用标准化的xavier初始化 激活函数使用sigmoid
第2个隐藏层：权重初始化采用标准化的He初始化 激活函数采用relu
out输出层 采用softmax做数据归一化，权重初始化采用标准化的xavier初始化
深度学习的一般工作流：
1.准备数据集
2.搭建神经网络:
前向传播方法forward
3.模型训练：
构造损失函数 优化器
前向传播，获取预测值
计算损失
梯度清零
反向传播，计算梯度
更新参数，梯度下降法
4.模型测试
搭建神经网络的流程：
1.定义一个模型类，继承 nn.Module
2.定义__init__方法，搭建网络层
3.定义forward方法，实现前向传播
"""
# 导包
import torch
import torch.nn as nn
from torchsummary import summary # 计算模型参数,查看模型结构, pip install torchsummary
# 设置设备
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(device)
# 1.定义模型类，搭建神经网络
class MyModel(nn.Module):
# 1.定义__init__方法，搭建网络层
def __init__(self):
# 1.初始化父类成员
super().__init__()
# 2.搭建网络层
self.linear1 = nn.Linear(3, 3)
self.linear2 = nn.Linear(3, 2)
self.linear3 = nn.Linear(2, 2)
# 3.参数初始化
# # 初始化linear1
# nn.init.xavier_normal_(self.linear1.weight)
# nn.init.zeros_(self.linear1.bias)
# # 初始化linear2
# nn.init.kaiming_normal_(self.linear2.weight)
# nn.init.zeros_(self.linear2.bias)
# # 初始化linear3
# nn.init.xavier_normal_(self.linear3.weight)
# nn.init.zeros_(self.linear3.bias)
# 2.定义forward方法，实现前向传播
def forward(self, x):
# 1.输出隐藏层1，linear1
# x = self.linear1(x)
# x = torch.sigmoid(x)
# 合在一起
x = torch.sigmoid(self.linear1(x))
# 2.输入隐藏层2，linear2
x = torch.relu(self.linear2(x))
# 3.输入输出层，linear3
x = torch.softmax(self.linear3(x), dim=-1)
# 4.返回输出结果
return x
# 2.模型训练，演示前向传播，并统计模型参数
def train():
# 0.设置随机种子，保证结果可复现
torch.manual_seed(42)
# 1.创建模型对象
model = MyModel()
print(model)
# # 2.准备数据集，需要是二维张量
# data = torch.randn(10,3) # 10个样本，每个样本3个特征
# print(f"data: {data}")
# print(f"shape: {data.shape}")
# print(f"data.requires_grad: {data.requires_grad}")
# # 3.前向传播，获取预测结果
# y_pred = model(data) # 底层自动调用forward方法，实现前向传播，返回预测结果，(10,2)
# print(f"y_pred: {y_pred}")
# print(f"shape: {y_pred.shape}")
# print(f"y_pred.requires_grad: {y_pred.requires_grad}")
# print("-" * 60)
# 4.查看模型的参数数量
print("模型参数情况：")
model.to(device)
summary(model, (3,))
for name, param in model.named_parameters():
print(f"{name}: {param}")
# 3.测试
if __name__ == '__main__':
train()
cuda
MyModel(
(linear1): Linear(in_features=3, out_features=3, bias=True)
(linear2): Linear(in_features=3, out_features=2, bias=True)
(linear3): Linear(in_features=2, out_features=2, bias=True)
)
模型参数情况：
----------------------------------------------------------------
Layer (type) Output Shape Param #
================================================================
Linear-1 [-1, 3] 12
Linear-2 [-1, 2] 8
Linear-3 [-1, 2] 6
================================================================
Total params: 26
Trainable params: 26
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.00
Params size (MB): 0.00
Estimated Total Size (MB): 0.00
----------------------------------------------------------------
linear1.weight: Parameter containing:
tensor([[ 0.4414, 0.4792, -0.1353],
[ 0.5304, -0.1265, 0.1165],
[-0.2811, 0.3391, 0.5090]], device='cuda:0', requires_grad=True)
linear1.bias: Parameter containing:
tensor([-0.4236, 0.5018, 0.1081], device='cuda:0', requires_grad=True)
linear2.weight: Parameter containing:
tensor([[ 0.4266, 0.0782, 0.2784],
[-0.0815, 0.4451, 0.0853]], device='cuda:0', requires_grad=True)
linear2.bias: Parameter containing:
tensor([-0.2695, 0.1472], device='cuda:0', requires_grad=True)
linear3.weight: Parameter containing:
tensor([[-0.3258, -0.0829],
[-0.2872, 0.4691]], device='cuda:0', requires_grad=True)
linear3.bias: Parameter containing:
tensor([-0.5582, -0.3260], device='cuda:0', requires_grad=True)
4. 损失函数
损失函数概念
在深度学习中, 损失函数是用来衡量模型参数质量的函数, 衡量的方式是比较网络输出（预测值）和真实输出（真实值）的差异。
我们通过最小化损失函数的值来调整模型参数，使其输出更接近真实值。
损失函数在不同的文献中名称是不一样的，主要有以下几种命名方式：
损失函数作用：
- 评估性能：反映模型预测结果与目标值的匹配程度。
- 指导优化：通过梯度下降等算法最小化损失函数，优化模型参数。
多分类交叉熵损失 - nn.CrossEntropyLoss()
分类任务中使用最多的是交叉熵损失函数, 多分类交叉熵损失函数中包含了softmax运算，模型输出层无需softmax
在多分类任务通常使用softmax将logits转换为概率的形式，所以多分类交叉熵损失也叫做softmax损失，计算方法是：
$$ loss = - \sum_{i=1}^q y_i \log(S(\hat{y}_i)) $$
其中:
yi是样本x属于某一个类别的真实概率- 而f(x)是样本属于某一类别的预测分数
- S是softmax激活函数，将属于某一类别的预测分数转换成概率
loss用来衡量真实值y和预测值f(x)之间差异性的损失结果- \(\hat{y}_i=S(f(x_i))\)是预测概率
例子：
上图中的交叉熵损失为：
$\(-(0log(0.10) + 1log(0.70) + 0log(0.2)) = -log(0.7)\)$
从概率角度理解，我们的目的是最小化正确类别所对应的预测概率的对数的负值(损失值最小)，如下图所示：
# 绘制 y=-log(x) 曲线
x = np.linspace(1e-4, 1, 100) # 避
y = -np.log(x)
# 设置x刻度为0到1
plt.xticks(np.arange(0, 1.1, 0.1))
plt.yticks(np.arange(0, 10, 1))
# 设置坐标轴标签
plt.xlabel("正确类别的预测概率")
plt.ylabel("损失值")
plt.plot(x, y, label="y=-log(x)")
plt.grid(True)
plt.title("多分类交叉熵CrossEntropyLoss")
plt.tight_layout()
plt.legend()
plt.show()
"""
案例：
演示 多分类交叉熵损失函数
介绍：
损失函数 也叫 成本函数，目标函数，代价函数，误差函数，用来 衡量模型好坏，也就是模型预测表现
分类问题：
多分类交叉熵损失CrossEntropyLoss
多分类交叉熵损失CrossEntropyLoss
公式：CrossEntropyLoss = -Σylog(S(f(x)))
其中，x: 样本，f(x): 模型预测的原始输出logits，S(f(x)): 模型预测的概率分布，y:样本x属于某个类别的 真实概率，比如[0,0,1]
注意：
CrossEntropyLoss内部进行了Softmax运算，模型输出层无需进行softmax运算，除非在新的数据上预测类别，需要获取预测概率，则需要在模型的原始输出logits上进行softmax
"""
# 导包
import torch
# 1.定义函数，演示 多分类交叉熵损失函数
def demo01():
# 1.创建 样本的真实类别,针对一个三分类任务，3个样本，真实类别为0，0，1
# 1.1 使用 one-hot 编码
# y_true = torch.tensor([[1,0,0],[1,0,0],[0,1,0]], dtype=torch.float)
# 1.2 使用label编码
y_true = torch.tensor([0,0,1])
# 2.创建 模型预测的原始输出logits
y_logits = torch.tensor([[1.5,0.2,0.9],[0.1,2,0.9],[0.1,0.2,9.4]], requires_grad=True)
# 3.创建多分类交叉熵损失函数对象
loss_fn = torch.nn.CrossEntropyLoss()
# 4.计算损失值
loss = loss_fn(y_logits,y_true)
print(f"损失值：{loss}")
# 测试
if __name__ == '__main__':
demo01()
损失值：4.03115177154541
二分类交叉熵损失 - nn.BCELoss()
二分类任务中可以使用二分类交叉熵损失函数 和 sigmoid
公式为：
$\(loss = -y \log(\hat{y}) - (1-y)\log(1-\hat{y})\)$
其中:
-
y 是样本x属于某一个类别的真实概率
-
而 \(\hat{y}\) 是样本属于某一类别的预测概率
-
loss用来衡量真实值 y 与预测值 \(\hat{y}\) 之间差异的损失函数
# 绘制 y = -log(x), y=-log(1-x)
x = np.linspace(1e-3, 1, 1000) # 避免对数函数的零值
y1 = -np.log(x)
y2 = -np.log(1 - x)
plt.figure(figsize=(8, 6))
plt.xticks(np.arange(0, 1.1, 0.1))
plt.yticks(np.arange(0, 10, 1))
plt.xlabel("预测概率")
plt.ylabel("损失值")
plt.plot(x, y1, color='blue', label='y=-log(x), 正类样本')
plt.plot(x, y2, color='red', label='y=-log(1-x), 负类样本')
plt.grid(True)
plt.title("二分类交叉熵BCELoss")
plt.legend()
plt.tight_layout()
plt.show()
C:\Users\angmi\AppData\Local\Temp\ipykernel_7796\193180856.py:4: RuntimeWarning: divide by zero encountered in log
y2 = -np.log(1 - x)
针对二分类问题，CrossEntropyLoss(softmax) 和 BCELoss(sigmoid) 可以互相转化
"""
案例：
演示 二分类交叉熵损失函数
介绍：
损失函数 也叫 成本函数，目标函数，代价函数，误差函数，用来 衡量模型好坏，模型预测表现
分类问题：
多分类交叉熵损失CrossEntropyLoss
二分类交叉熵损失BCELoss
多分类交叉熵损失CrossEntropyLoss
公式：CrossEntropyLoss = -Σylog(S(f(x)))
其中，x: 样本，f(x): 模型预测的原始输出logits，S(f(x)): 模型预测的概率分布，y:样本x属于某个类别的 真实概率，比如[0,0,1]
注意：
CrossEntropyLoss内部进行了Softmax运算，模型输出层无需进行softmax运算，除非在新的数据上预测类别，需要获取预测概率，则需要在模型的原始输出logits上进行softmax
二分类交叉熵损失BCELoss
公式：BCELoss = -ylog(y_hat)-(1-y)log(1-y_hat)
注意：
BCELoss内部没有进行sigmoid运算，使用BCELoss时，模型输出层需要手动添加sigmoid
针对二分类问题，CrossEntropyLoss 和 BCELoss 可以互相转化
注意正类的人为设定，会影响BCELoss 和 CrossEntropyLoss 中 logits的 转换关系
"""
# 导包
import torch
import torch.nn as nn
# 定义函数，演示 二分类交叉熵损失
def demo01():
print("二分类交叉熵损失")
# 1. 创建 样本的真实类别, [正，正，负]
y_true = torch.tensor([1, 1, 0], dtype=torch.float)
# 2.创建 模型预测 的原始输出logits值
y_logits = torch.tensor([1,-4,3], dtype=torch.float)
# sigmoid=1/(1+exp(-x))
# 经过sigmoid,转化为正类别的 概率
y_pred = torch.sigmoid(y_logits)
print(f"y_pred: {y_pred}")
# 3.创建 交叉熵损失函数
loss_fn = nn.BCELoss()
# 4.计算损失
loss = loss_fn(y_pred, y_true)
print(f"loss: {loss}")
# 定义函数，演示 多分类交叉熵损失
def demo02():
print("多分类交叉熵损失")
# 1. 创建 样本的真实类别, [正，正，负]
y_true = torch.tensor([1, 1, 0])
# 2.创建 模型预测 的原始输出logits值
y_pred = torch.tensor([[2,3], [5,1],[1, 4]], dtype=torch.float)
# 3.创建 交叉熵损失函数
loss_fn = nn.CrossEntropyLoss()
# 4.计算损失
loss = loss_fn(y_pred, y_true)
print(f"loss: {loss}")
# 测试
if __name__ == '__main__':
demo01()
demo02()
二分类交叉熵损失
y_pred: tensor([0.7311, 0.0180, 0.9526])
loss: 2.4599997997283936
多分类交叉熵损失
loss: 2.4599995613098145
回归任务 - MAE损失函数
mean absolute loss(MAE) 也被称为L1 Loss，计算绝对误差的平均值
损失函数公式:
$\(L = \frac{1}{n}\sum_{i=1}^{n}|y_i - \hat{y}_i|\)$
误差-损失曲线如下图所示：
# 绘制 y=|x|的函数图像
x = np.linspace(-2, 2, 1000)
y = np.abs(x)
plt.figure(figsize=(8, 4))
plt.plot(x, y, label="y=|x|")
# 设置坐标轴标签
plt.xlabel("误差 $y_i - \hat{y}_i$")
plt.ylabel("绝对值")
# 设置刻度
plt.xticks(np.arange(-2, 2.5, 0.5))
plt.yticks(np.arange(0, 2.5, 0.5))
plt.title("L1Loss")
plt.grid()
plt.legend()
plt.tight_layout()
plt.show()
<>:7: SyntaxWarning: invalid escape sequence '\h'
<>:7: SyntaxWarning: invalid escape sequence '\h'
C:\Users\angmi\AppData\Local\Temp\ipykernel_7796\3224521475.py:7: SyntaxWarning: invalid escape sequence '\h'
plt.xlabel("误差 $y_i - \hat{y}_i$")
特点是：
- L1 loss具有稀疏性，可以惩罚较大的值，常常作为正则项添加到其他loss中作为约束。(导数恒定为 1或-1，0点不可导, 产生稀疏矩阵)
- L1 loss的最大问题是梯度在零点不平滑，导致会跳过极小值
- 适用于回归问题中存在异常值或噪声数据时，可以减少对离群点的敏感性
# 定义变量，记录 真实值
y_true = torch.tensor([2.0,2.5,3.0], dtype=torch.float)
# 定义变量，记录 预测值
y_pred = torch.tensor([1.5,2.0,2.5], dtype=torch.float, requires_grad=True)
# 创建损失函数，MAE
loss_fn = nn.L1Loss()
# loss_fn = nn.MSELoss()
# loss_fn = nn.SmoothL1Loss()
# 计算损失值
loss = loss_fn(y_pred, y_true)
# 打印损失值
print(f"MAE: {loss}")
MAE: 0.5
回归任务 - MSE损失函数
Mean Squared Loss/ Quadratic Loss(MSE loss) 也被称为L2 loss，或欧氏距离，计算误差平方的平均值
公式:
$\(L = \frac{1}{n}\sum_{i=1}^{n}(y_i - \hat{y}_i)^2\)$
误差-损失曲线如下图所示：
# 绘制 y=x^2 的函数图像
x = np.linspace(-2, 2, 1000)
y = x ** 2
plt.figure(figsize=(8, 7))
plt.plot(x, y, label="y=x^2")
# 设置坐标轴标签
plt.xlabel("误差 $y_i - \hat{y}_i$")
plt.ylabel("平方")
# 设置刻度
plt.xticks(np.arange(-2, 2.5, 0.5))
plt.yticks(np.arange(0, 4.5, 0.5))
plt.title("MSELoss")
plt.grid()
plt.legend()
plt.tight_layout()
plt.show()
<>:7: SyntaxWarning: invalid escape sequence '\h'
<>:7: SyntaxWarning: invalid escape sequence '\h'
C:\Users\angmi\AppData\Local\Temp\ipykernel_7796\1418239404.py:7: SyntaxWarning: invalid escape sequence '\h'
plt.xlabel("误差 $y_i - \hat{y}_i$")
特点是：
- L2 loss也常常作为正则项，对于离群点（outliers）敏感，因为平方项会放大大误差
- 当预测值与目标值相差很大时, 梯度容易爆炸
-
梯度爆炸:网络层之间的梯度（值大于1.0）重复相乘导致的指数级增长
-
适用于大多数标准回归问题，如房价预测、温度预测等
# 定义变量，记录 真实值
y_true = torch.tensor([2.0,2.5,3.0], dtype=torch.float)
# 定义变量，记录 预测值
y_pred = torch.tensor([1.5,2.0,2.5], dtype=torch.float, requires_grad=True)
# 创建损失函数，MSE
# loss_fn = nn.L1Loss()
loss_fn = nn.MSELoss()
# loss_fn = nn.SmoothL1Loss()
# 计算损失值
loss = loss_fn(y_pred, y_true)
# 打印损失值
print(f"MSE: {loss}")
MSE: 0.25
回归任务 - Smooth L1 Loss (也称为 Huber Loss)
Smooth L1 损失函数结合了 MAE 和 MSE，适用于回归任务，尤其是在面对异常值时，它能平滑过渡，避免大误差的影响。
公式如下：
\[
L_{\text{SmoothL1}} = \frac{1}{n} \sum_{i=1}^{n}
\begin{cases}
0.5 \cdot (x_i)^2 & \text{if } |x_i| < 1 \\
|x_i| - 0.5 & \text{if } |x_i| \geq 1
\end{cases}
\]
其中：
- \(x_i = y_{\text{pred}, i} - y_{\text{true}, i}\)
- \(n\) 是样本数量
- \(x_i\) 是每个样本的预测误差
解释：
- 当误差 \(|x_i|\) 小于 1 时，使用 L2 损失（平方误差），这意味着损失的增长是 二次方，避免了在训练过程中使用绝对误差导致的梯度不连续问题。
- 当误差 \(|x_i|\) 大于或等于 1 时，使用 L1 损失（绝对误差），避免了大误差对梯度的影响，避免了MSE对离群点过度敏感，可以减少梯度爆炸。
这种方式使得 Smooth L1 Loss 在小误差时更精确，在大误差时能较为稳健地优化。
Smooth L1损失函数图像为：
从上图中可以看出，该函数实际上就是一个分段函数
- 在[-1,1]之间实际上就是L2损失，这样解决了L1的不光滑问题
- 在[-1,1]区间外，实际上就是L1损失，这样就解决了离群点梯度爆炸的问题
# 定义变量，记录 真实值
y_true = torch.tensor([2.0,2.5,3.0], dtype=torch.float)
# 定义变量，记录 预测值
y_pred = torch.tensor([1.5,2.0,2.5], dtype=torch.float, requires_grad=True)
# 创建损失函数，SmoothL1Loss
# loss_fn = nn.L1Loss()
# loss_fn = nn.MSELoss()
loss_fn = nn.SmoothL1Loss()
# 计算损失值
loss = loss_fn(y_pred, y_true)
# 打印损失值
print(f"SmoothL1Loss: {loss}")
SmoothL1Loss: 0.125
5.神经网络优化方法
多层神经网络的学习能力比单层网络强得多。想要训练多层网络，需要更强大的学习算法。误差反向传播算法（Back Propagation）是其中最杰出的代表，它是目前最成功的神经网络学习算法。现实任务使用神经网络时，大多是在使用 BP 算法进行训练，值得指出的是 BP 算法不仅可用于多层前馈神经网络，还可以用于其他类型的神经网络。通常说 BP 网络时，一般是指用 BP 算法训练的多层前馈神经网络。
这就需要了解两个概念：
- 正向传播：指的是数据通过网络从输入层到输出层的传递过程。这个过程的目的是计算网络的输出值（预测值），从而与目标值（真实值）比较以计算误差。
- 反向传播：指的是计算损失函数相对于网络中各参数（权重和偏置）的梯度，指导优化器更新参数，从而使神经网络的预测更接近目标值。
6. 梯度下降法
梯度下降法就是一种寻找损失函数最小值的方法。从数学角度来看，梯度的方向是函数增长速度最快的方向，梯度的反方向就是函数减少最快的方向。
基本步骤
- 初始化参数
- 前向传播计算损失
- 反向传播计算梯度
- 更新参数:
$\(w = w - \alpha * \frac{\partial L}{\partial w}\)$
其中\(\alpha\)是学习率。如果学习率太小，那么每次训练之后得到的参数更新效果都太小，增大训练的时间成本。如果，学习率太大，那就有可能直接跳过最优解，进入无限的训练中。解决的方法是，学习率也需要随着训练的进行而变化。
\(\frac{\partial L}{\partial w}\) 是损失函数对模型参数的梯度。
在上图中我们展示了一维和多维的损失函数，损失函数呈碗状。在训练过程中损失函数对权重的偏导数就是损失函数在该位置的梯度。可以看到，沿着负梯度方向移动，就可以到达损失函数底部，从而使损失函数最小化。这种利用损失函数的梯度迭代地寻找最小值的过程就是梯度下降的过程。
\[
\frac{\partial \text{loss}}{\partial (w_1, w_2)} = (2w_1, 2w_2)
\]
在进行模型训练时，有三个基础的概念：
- Epoch: 训练轮数，使用全部数据对模型进行此轮完整训练，比如
Epochs = 100 - Batch: 批次大小，使用训练集中的小部分样本的梯度对模型权重进行更新,比如
batch_size=16 - Iteration: 每次训练的批次数，使用一个 Batch 数据对模型进行一次参数更新的过程，比如
iterations=7
假设数据集有 50000 个训练样本，现在选择 Batch Size = 256 对模型进行训练。
- 每个 Epoch 要训练的图片数量：50000
- 训练集具有的 Batch 个数：(50000-1)//256+1=196
- 每个 Epoch 具有的 Iteration 个数：196
- 10个 Epoch 具有的 Iteration 个数：1960
在深度学习中，梯度下降的几种方式的根本区别就在于Batch Size不同,如下表所示：
| 梯度下降方式 | Training Set Size | Batch Size | Number of Batches |
|--------------|-------------------|------------|-------------------|
| BGD | \(N\) | \(N\) | \(1\) |
| SGD | \(N\) | \(1\) | \(N\) |
| Mini-Batch | \(N\) | \(B\) | $ (N-1)//B + 1 $ |
演示梯度下降法
# 目标函数 & 导数
def f(x):
return x**2 + 20
def grad(x):
return 2 * x
# 准备绘图
x_vals = np.linspace(-10, 10, 200)
y_vals = f(x_vals)
plt.figure(figsize=(10,6))
plt.title("梯度下降法演示-二维曲线")
plt.plot(x_vals,y_vals, label="y = $w^{2}$ + 20")
plt.grid()
plt.legend()
plt.tight_layout()
plt.show()
# 梯度下降参数
lr = 0.2 # learning rate
x = -8 # 初始点
steps = 20 # 迭代次数
xs = [x] # 保存每一步的 x
ys = [f(x)] # 保存每一步的 y
# 梯度下降过程
for i in range(steps):
g = grad(x)
x = x - lr * g
xs.append(x)
ys.append(f(x))
# 绘制函数曲线
plt.figure(figsize=(10,6))
plt.plot(x_vals, y_vals, label=r"y = $w^{2}$ + 20")
# 绘制梯度下降路径
plt.scatter(xs, ys, color='red', s=30, label="梯度下降路径")
plt.plot(xs, ys, color='red', linestyle='--')
# 最终点标注
plt.scatter(xs[-1], ys[-1], color='green', s=60, label="最低点")
plt.grid()
plt.legend()
plt.title("梯度下降法演示-二维曲线")
plt.xlabel("w")
plt.ylabel("y")
plt.tight_layout()
plt.show()
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
# 定义函数及梯度
def f(x, y):
return x**2 + y**2
def grad_f(x, y):
return np.array([2*x, 2*y])
# 创建网格
x = np.linspace(-5, 5, 100)
y = np.linspace(-5, 5, 100)
X, Y = np.meshgrid(x, y)
Z = f(X, Y)
# 梯度下降初始化
lr = 0.1 # 学习率
steps = 25
point = np.array([4.0, 4.0]) # 起点
trajectory = [point.copy()]
# 梯度下降迭代
for _ in range(steps):
grad = grad_f(point[0], point[1])
point -= lr * grad
trajectory.append(point.copy())
trajectory = np.array(trajectory)
Z_traj = f(trajectory[:,0], trajectory[:,1])
# 绘图
# Plot surface
fig = plt.figure(figsize=(12,8))
ax = fig.add_subplot(111, projection='3d')
ax.set_title("梯度下降法演示——三维曲面")
ax.plot_surface(X, Y, Z, alpha=0.5, cmap='viridis', label="z = $w_1^{2}$ + $w_2^{2}$") # No colors specified
ax.set_xlabel(r"$w_1$")
ax.set_ylabel(r"$w_2$")
ax.legend()
plt.tight_layout()
fig = plt.figure(figsize=(12,8))
ax = fig.add_subplot(111, projection='3d')
ax.set_title("梯度下降法演示——三维曲面")
ax.plot_surface(X, Y, Z, alpha=0.5, cmap='viridis', label="z = $w_1^{2}$ + $w_2^{2}$") # 半透明曲面
ax.plot(trajectory[:,0], trajectory[:,1], Z_traj, color='r', marker='o', label='梯度下降路径')
ax.set_xlabel(r"$w_1$")
ax.set_ylabel(r"$w_2$")
# ax.set_zlabel(r"z = $w_1^{2}$ + $w_2^{2}$")
ax.legend()
plt.tight_layout()
plt.show()
7.反向传播（BP算法）
利用反向传播算法对神经网络进行训练。该方法与梯度下降算法相结合，对网络中所有权重计算损失函数的梯度，并利用梯度值来更新权值以最小化损失函数。
概念
前向传播：数据输入到神经网络中，逐层向前传播，一直运算到输出层为止, 输出预测值。
反向传播（Back Propagation）：利用损失函数loss值，从后往前，依次求各个参数的偏导，结合梯度下降算法，更新模型参数。
在网络的训练过程中经过前向传播得到的最终结果跟训练样本的真实值总是存在一定误差，这个误差便是损失函数 ERROR。想要减小这个误差，就用损失函数 ERROR，从后往前，依次求各个参数的偏导，这就是反向传播（Back Propagation）, 然后用损失梯度来更新模型参数来减小损失值。
详解
反向传播算法利用链式求导法则对神经网络中的各个节点的权重进行求导运算。
【举个栗子🌰：】
下图是一个简单的神经网络示例：激活函数为sigmoid
前向传播运算：
反向传播运算（求损失函数对各个模型参数的梯度）：
先来求损失函数E对\(w_{5}\)的导数。根据链式求导法则，损失函数E对\(w_{5}\)的导数为：
\[\frac{\partial E}{\partial w_{5}} = \frac{\partial E}{\partial out_{o1}} * \frac{\partial out_{o1}}{\partial net_{o1}} * \frac{\partial net_{o1}}{\partial w_{5}}\]
如下图所示：
导数（梯度）已经计算出来了，下面就是更新参数过程：
损失函数E对\(w_{1}\)的导数，损失函数E对\(w_{1}\)的求导路径不止一条，这会稍微复杂一点，但换汤不换药，计算过程如下所示：
8.梯度下降优化方法
梯度下降优化算法中，可能会碰到以下情况：
- 碰到平缓区域，梯度值较小，参数优化变慢
- 碰到 “鞍点” ，梯度为0，参数无法优化
- 碰到局部最小值，参数不是最优
对于这些问题, 出现了一些对梯度下降算法的优化方法，例如：Momentum、AdaGrad、RMSprop、Adam 等
指数加权平均
常见的算数平均指的是将所有数加起来除以数的个数，每个数的权重是相同的。指数加权平均指的是给每个数赋予不同的权重求平均数。移动平均数，指的是计算最近邻的 N 个数来获得平均数。
指数移动加权平均则是参考各数值，并且各数值的权重都不同，距离越远的数字对平均数计算的贡献越小（权重较小），距离越近则对平均数计算的贡献越大（权重越大）。
比如：明天气温怎么样，和昨天气温有很大关系，而和一个月前的气温关系就小一些。
计算公式可以用下面的式子来表示：
\[
S_t = \begin{cases}
Y_0, & t = 0 \\
\beta * S_{t-1} + (1 - \beta) * Y_t, & t > 0
\end{cases}
\]
- \(S_t\) 表示指数加权平均值;
- \(Y_t\) 表示t时刻的值;
- β 调节权重系数，该值越大平均数越平缓。
举个例子，假设 权重 \(\beta\) 为 0.9，则：
- $S_0 = Y_0 $
- \(S_1 = 0.9 \times S_0 + 0.1 \times Y_1\)
- \(S_2 = 0.9 \times S_1 + 0.1 \times Y_2 = 0.9^2 \times Y_0 + 0.9 \times 0.1 \times Y_1 + 0.1 \times Y_2\)
- \(S_3 = 0.9 \times S_2 + 0.1 \times Y_3 = 0.9^3 \times Y_0 + 0.9^2 \times 0.1 \times Y_1 + 0.9 \times 0.1 \times Y_2 + 0.1 \times Y_3\)
- ...
-
\(S_n = 0.9 \times S_{n-1} + 0.1 \times Y_n = 0.9^{n} \times Y_0 + 0.9^{n-1} \times 0.1 \times Y_1 + \dots + 0.9 \times 0.1 \times Y_{n-1} + 0.1 \times Y_n\)
-
$S_{100} = 0.9 \times S_{99} + 0.1 \times Y_{100} = 0.9^{100} \times Y_0 + 0.9^{99} \times 0.1 \times Y_1 + \dots + 0.9 \times 0.1 \times Y_{99} + 0.1 \times Y_{100} $
\[ = 0.1 \times Y_{100} + 0.9 \times 0.1 \times Y_{99} + 0.9^2 \times 0.1 \times Y_{98} + ... \]
结论
1. \(S_n\)是 \([Y_0, Y_1, Y_2, ..., Y_n]\) 的加权求和结果 — 指数加权平均值
2. 离当前值\(S_n\)越近的数据，系数越大
"""
案例:
演示 移动加权平均，使用近30天的天气温度数据.
结论:
对于移动加权平均, β值(调节权重系数)越大，移动加权平均值就越依赖历史数据；
反之，β值(调节权重系数)越小，移动加权平均值就越依赖当天的数据。
极端情况：β=1，移动加权平均值 = 第一天的数据；β=0，移动加权平均值 = 当天的数据。
"""
import torch
import matplotlib.pyplot as plt
ELEMENT_NUMBER = 30
# 1. 实际平均温度
def demo01():
# 固定随机数种子
torch.manual_seed(0)
# 产生30天的随机温度
temperature = torch.randn(size=[ELEMENT_NUMBER, ]) * 10
print(temperature)
# 绘制平均温度
days = torch.arange(1, ELEMENT_NUMBER + 1, 1)
plt.plot(days, temperature, color='r')
plt.scatter(days, temperature)
plt.tight_layout()
plt.show()
# 2. 指数加权平均温度
def demo02(beta=0.9):
# 固定随机数种子，确保每次运行得到相同的随机温度数据
torch.manual_seed(0)
# 产生30天的随机温度数据，通过乘以10放大温度范围
temperature = torch.randn(size=[ELEMENT_NUMBER, ]) * 10
print(temperature)
# 存储指数加权平均后的温度值
exp_weight_avg = []
# 遍历每天的温度数据，idx从0开始计数
for idx, temp in enumerate(temperature):
# 第一天的指数加权平均值就是当天温度本身
if idx == 0:
exp_weight_avg.append(temp)
continue
# 从第二天开始，按照指数加权平均公式计算：
# V_t = β * V_{t-1} + (1-β) * θ_t
# 其中 V_t 是当前时刻的加权平均值，V_{t-1} 是上一时刻的加权平均值
# θ_t 是当前时刻的实际温度值，β 是权重参数(0<=β<1)
# idx-1 是因为要访问前一天的加权平均值
new_temp = exp_weight_avg[idx - 1] * beta + (1 - beta) * temp
exp_weight_avg.append(new_temp)
# 创建天数序列用于绘图
days = torch.arange(1, ELEMENT_NUMBER + 1, 1)
# 设置图表标题，显示当前使用的beta值
plt.title(f"beta={beta}")
# 绘制指数加权平均温度曲线（红色）
plt.plot(days, exp_weight_avg, color='r')
# 绘制原始温度数据点
plt.scatter(days, temperature)
plt.tight_layout()
# 显示图表
plt.show()
if __name__ == '__main__':
demo01()
demo02(0)
demo02(0.5)
demo02(0.9)
demo02(1)
tensor([-11.2584, -11.5236, -2.5058, -4.3388, 8.4871, 6.9201, -3.1601,
-21.1522, 3.2227, -12.6333, 3.4998, 3.0813, 1.1984, 12.3766,
-1.4347, -1.1161, -6.1358, 0.3159, -4.9268, 2.4841, 4.3970,
1.1241, -8.4106, -23.1604, -1.0231, 7.9244, -2.8967, 0.5251,
5.2286, 23.0221])
tensor([-11.2584, -11.5236, -2.5058, -4.3388, 8.4871, 6.9201, -3.1601,
-21.1522, 3.2227, -12.6333, 3.4998, 3.0813, 1.1984, 12.3766,
-1.4347, -1.1161, -6.1358, 0.3159, -4.9268, 2.4841, 4.3970,
1.1241, -8.4106, -23.1604, -1.0231, 7.9244, -2.8967, 0.5251,
5.2286, 23.0221])
tensor([-11.2584, -11.5236, -2.5058, -4.3388, 8.4871, 6.9201, -3.1601,
-21.1522, 3.2227, -12.6333, 3.4998, 3.0813, 1.1984, 12.3766,
-1.4347, -1.1161, -6.1358, 0.3159, -4.9268, 2.4841, 4.3970,
1.1241, -8.4106, -23.1604, -1.0231, 7.9244, -2.8967, 0.5251,
5.2286, 23.0221])
tensor([-11.2584, -11.5236, -2.5058, -4.3388, 8.4871, 6.9201, -3.1601,
-21.1522, 3.2227, -12.6333, 3.4998, 3.0813, 1.1984, 12.3766,
-1.4347, -1.1161, -6.1358, 0.3159, -4.9268, 2.4841, 4.3970,
1.1241, -8.4106, -23.1604, -1.0231, 7.9244, -2.8967, 0.5251,
5.2286, 23.0221])
tensor([-11.2584, -11.5236, -2.5058, -4.3388, 8.4871, 6.9201, -3.1601,
-21.1522, 3.2227, -12.6333, 3.4998, 3.0813, 1.1984, 12.3766,
-1.4347, -1.1161, -6.1358, 0.3159, -4.9268, 2.4841, 4.3970,
1.1241, -8.4106, -23.1604, -1.0231, 7.9244, -2.8967, 0.5251,
5.2286, 23.0221])
动量算法Momentum
当梯度下降碰到 “峡谷” 、”平缓”、”鞍点” 区域时, 参数更新变慢。 Momentum 通过指数加权平均法，累计历史梯度值，进行参数更新，越近的梯度值对当前参数更新的影响越大。
梯度计算公式：
$\(s_t=βs_{t−1}+(1−β)g_t\)$
参数更新公式：
$\(w_t=w_{t−1}−ηs_t\)$
\(s_t\)是当前时刻的梯度的指数移动加权平均值
\(s_{t-1}\)是上一时刻的梯度的指数移动加权平均值
\(g_t\)是当前时刻的梯度值
β 是调节权重系数，通常取 0.9 或 0.99
η是学习率
\(w_t\)是当前时刻模型权重参数
print("梯度下降优化方法 —— 动量法Momentum")
# 1.初始化模型参数w
w = torch.tensor([1.0], requires_grad=True, dtype=torch.float32)
# 2.求损失值
loss = 0.5*(w**2) # w
# 3.创建优化器，基于SGD, 加入动量 momentum(动量参数)，就是动量法
# 参数1：待优化的模型参数列表，参数2：学习率，参数3：动量参数
optimizer = optim.SGD([w], lr=0.01, momentum=0.9) # momentum默认0，也就是无动量的随机梯度下降法
# 4.计算梯度，梯度清零 + 反向传播 + 更新参数
optimizer.zero_grad() # 梯度清零
loss.mean().backward() # 反向传播
optimizer.step() # 更新参数
print(f"第一次梯度更新结果：\nw:{w}, w.grad: {w.grad}")
# 第二次计算损失值
loss = 0.5 * (w ** 2) # w
optimizer.zero_grad() # 梯度清零
loss.mean().backward() # 反向传播
optimizer.step() # 更新参数
print(f"第二次梯度更新结果：\nw:{w}, w.grad: {w.grad}")
梯度下降优化方法 —— 动量法Momentum
第一次梯度更新结果：
w:tensor([0.9900], requires_grad=True), w.grad: tensor([1.])
第二次梯度更新结果：
w:tensor([0.9711], requires_grad=True), w.grad: tensor([0.9900])
- Momentum 理论公式 vs SGD-Momentum 真实公式
| 方面 | 理论公式 (Classical Momentum) | 框架公式 (PyTorch SGD-Momentum) |
|---|---|---|
| 动量计算 | \(s_t = \beta s_{t-1} + (1-\beta) g_t\) | \(s'_t = \beta s'_{t-1} + \eta g_t\) , 等价于\(s_t = \beta s_{t-1} + g_t\) |
| 参数更新 | \(w_t = w_{t-1} - \eta s_t\) | \(w_t = w_{t-1} - s'_t\) |
| 梯度定义 | \(g_t = \nabla L(w_{t-1})\) | \(g_t = \nabla L(w_{t-1})\) |
| 动量定义 | \(s_t\) 为累积梯度（速度） | \(s'_t = \eta s_t\) 已包含学习率 |
| 说明 | 学习率在参数更新时乘 | 学习率提前合并到动量计算中，更新直接减去 \(s'_t\) |
区别：框架公式的动量计算中，取消了\((1-\beta)\)缩放。
详细解释计算逻辑
w_init = 1.0
momentum = 0.9
lr = 0.01
# 计算梯度
g0 = w_init # 1.0
# 计算s0
s0 = 0 + lr*g0 # 0.01*1.0
# 计算w0
w0 = w_init - s0 # 1.0 - 0.01*1.0 = 0.99
print(f"第一次更新参数之后：w0: {w0}, g0: {g0}")
# 计算g1
g1 = w0 # 0.99
# 计算s1
s1 = momentum * s0 + lr * g1 # 0.9*0.01 + 0.01*0.99 = 0.009+0.0099 = 0.0189
# 计算w1
w1 = w0 - s1 # 0.99 - 0.0189 = 0.9711
print(f"w1: {w1}, g1: {g1}")
第一次更新参数之后：w0: 0.99, g0: 1.0
w1: 0.9711, g1: 0.99
AdaGrad
通过对不同的参数分量使用不同的学习率，AdaGrad 的学习率总体会逐渐减小，AdaGrad 认为：在起初时，我们距离最优目标仍较远，可以使用较大的学习率，加快训练速度，随着迭代次数的增加，学习率逐渐下降。
其计算步骤如下：
-
初始化学习率 η、初始化参数w、小常数 \(σ = 1e-10\)
-
初始化梯度累计变量 s = 0
- 从训练集中采样 m 个样本的小批量，计算梯度\(g_t\)
-
累积平方梯度，⊙ 表示各个分量相乘, 公式为：
$\(s_t = s_{t-1} + g_t⊙g_t\)$
-
学习率 η 的计算公式如下：
\[η = \frac{η}{\sqrt{s_t}+σ}\]
-
权重参数更新公式如下：
$\(w_t = w_{t-1} - \frac{η}{\sqrt{s_t}+σ} * g_t\)$
-
重复 3-7 步骤
AdaGrad 缺点是可能会使得学习率过早、过量的降低，导致模型训练后期学习率太小，较难找到最优解。
print("梯度下降优化方法 —— adagrad")
# 1.初始化模型参数w
w = torch.tensor([1.0], requires_grad=True, dtype=torch.float32)
# 2.求损失值
loss = 0.5*(w**2) # w
# 3.创建优化器，基于SGD, 加入动量 momentum(动量参数)，就是动量法
# 参数1：待优化的模型参数列表，参数2：学习率，参数3：动量参数
# optimizer = optim.SGD([w], lr=0.01, momentum=0.9) # momentum默认0，也就是无动量的随机梯度下降法
# 参数1：待优化的模型参数列表，参数2：学习率
optimizer = optim.Adagrad([w], lr=0.01)
# 4.计算梯度，梯度清零 + 反向传播 + 更新参数
optimizer.zero_grad() # 梯度清零
loss.mean().backward() # 反向传播
optimizer.step() # 更新参数
print(f"第一次梯度更新结果：\nw:{w}, w.grad: {w.grad}")
# 第二次计算损失值
loss = 0.5 * (w ** 2) # w
optimizer.zero_grad() # 梯度清零
loss.mean().backward() # 反向传播
optimizer.step() # 更新参数
print(f"第二次梯度更新结果：\nw:{w}, w.grad: {w.grad}")
梯度下降优化方法 —— adagrad
第一次梯度更新结果：
w:tensor([0.9900], requires_grad=True), w.grad: tensor([1.])
第二次梯度更新结果：
w:tensor([0.9830], requires_grad=True), w.grad: tensor([0.9900])
RMSProp
RMSProp 优化算法是对 AdaGrad 的优化。最主要的不同是，其使用指数加权平均梯度替换累计平方梯度。
其计算过程如下：
-
初始化学习率 η、初始化权重参数w、小常数 σ = 1e-10
-
初始化梯度累计变量 s = 0
-
从训练集中采样 m 个样本的小批量，计算梯度 \(g_t\)
-
使用指数加权平均累计历史梯度，⊙ 表示各个分量相乘，公式如下：
$\(s_t = βs_{t-1} + (1-β)g_t⊙g_t\)$
-
学习率 η 的计算公式如下：
\[η = \frac{η}{\sqrt{s_t}+σ}\]
- 权重参数更新公式如下：
\[w_t = w_{t-1} - \frac{η}{\sqrt{s_t}+σ} * g_t\]
- 重复 3-7 步骤
RMSProp 与 AdaGrad 最大的区别是对梯度的累积方式不同，对于每个梯度分量仍然使用不同的学习率。
RMSProp 通过引入衰减系数β，控制对历史梯度信息获取的多少. 被证明在神经网络非凸条件下的优化更好，学习率衰减更加合理。
需要注意的是：AdaGrad 和 RMSProp 都是对于不同的参数分量使用不同的学习率，如果某个参数分量的梯度值较大，则对应的学习率就会较小，反之，则对应的学习率就会较大。
print("梯度下降优化方法 —— rmsprop")
# 1.初始化模型参数w
w = torch.tensor([1.0], requires_grad=True, dtype=torch.float32)
# 2.求损失值
loss = 0.5*(w**2) # w
# 3.创建优化器，基于SGD, 加入动量 momentum(动量参数)，就是动量法
# 参数1：待优化的模型参数列表，参数2：学习率，参数3：动量参数
# optimizer = optim.SGD([w], lr=0.01, momentum=0.9) # momentum默认0，也就是无动量的随机梯度下降法
# 参数1：待优化的模型参数列表，参数2：学习率
# optimizer = optim.Adagrad([w], lr=0.01)
# 参数1：待优化的模型参数列表，参数2：学习率，参数3：调和权重系数
optimizer = optim.RMSprop([w], lr=0.01, alpha=0.99)
# 4.计算梯度，梯度清零 + 反向传播 + 更新参数
optimizer.zero_grad() # 梯度清零
loss.mean().backward() # 反向传播
optimizer.step() # 更新参数
print(f"第一次梯度更新结果：\nw:{w}, w.grad: {w.grad}")
# 第二次计算损失值
loss = 0.5 * (w ** 2) # w
optimizer.zero_grad() # 梯度清零
loss.mean().backward() # 反向传播
optimizer.step() # 更新参数
print(f"第二次梯度更新结果：\nw:{w}, w.grad: {w.grad}")
梯度下降优化方法 —— rmsprop
第一次梯度更新结果：
w:tensor([0.9000], requires_grad=True), w.grad: tensor([1.])
第二次梯度更新结果：
w:tensor([0.8329], requires_grad=True), w.grad: tensor([0.9000])
Adam
-
Momentum 使用指数加权平均计算当前的梯度值
-
AdaGrad、RMSProp 使用自适应的学习率
-
Adam优化算法（Adaptive Moment Estimation，自适应矩估计）将 Momentum 和 RMSProp 算法结合在一起
- 修正梯度: 使⽤梯度的指数加权平均
-
修正学习率: 使⽤梯度平⽅的指数加权平均
-
原理：Adam 是结合了 Momentum 和 RMSProp 的优点的自适应学习率算法。它计算了梯度的一阶矩（平均值）和二阶矩（梯度的平方）的自适应估计，从而动态调整学习率。
-
梯度计算公式：
$\(m_t=β_1m_{t−1}+(1−β_1)g_t\)$
$\(s_t=β_2s_{t−1}+(1−β_2)g_t^2\)$
$\(\hat{m_t} = \frac{m_t}{1−β_1^t}\)$
$\(\hat{s_t}=\frac{s_t}{1−β_2^t}\)$
-
权重参数更新公式:
$\(w_t = w_{t−1} − \frac{η}{\sqrt{\hat{s_t}}+ϵ}\hat{m_t}\)$
其中，\(m_t\) 是梯度的一阶矩估计，\(s_t\) 是梯度的二阶矩估计，\(\hat{m_t}\) 和 \(\hat{s_t}\) 是偏差校正后的估计。
print("梯度下降优化方法 —— adam")
# 1.初始化模型参数w
w = torch.tensor([1.0], requires_grad=True, dtype=torch.float32)
# 2.求损失值
loss = 0.5*(w**2) # w
# 3.创建优化器，基于SGD, 加入动量 momentum(动量参数)，就是动量法
# 参数1：待优化的模型参数列表，参数2：学习率，参数3：动量参数
# optimizer = optim.SGD([w], lr=0.01, momentum=0.9) # momentum默认0，也就是无动量的随机梯度下降法
# 参数1：待优化的模型参数列表，参数2：学习率
# optimizer = optim.Adagrad([w], lr=0.01)
# 参数1：待优化的模型参数列表，参数2：学习率，参数3：调和权重系数
# optimizer = optim.RMSprop([w], lr=0.01, alpha=0.99)
optimizer = optim.Adam([w], lr=0.01, betas=(0.9, 0.999), weight_decay=1e-3)
# 4.计算梯度，梯度清零 + 反向传播 + 更新参数
optimizer.zero_grad() # 梯度清零
loss.mean().backward() # 反向传播
optimizer.step() # 更新参数
print(f"第一次梯度更新结果：\nw:{w}, w.grad: {w.grad}")
# 第二次计算损失值
loss = 0.5 * (w ** 2) # w
optimizer.zero_grad() # 梯度清零
loss.mean().backward() # 反向传播
optimizer.step() # 更新参数
print(f"第二次梯度更新结果：\nw:{w}, w.grad: {w.grad}")
梯度下降优化方法 —— adam
第一次梯度更新结果：
w:tensor([0.9900], requires_grad=True), w.grad: tensor([1.])
第二次梯度更新结果：
w:tensor([0.9800], requires_grad=True), w.grad: tensor([0.9900])
AdamW 优化算法
AdamW（Adam with Weight Decay）是 Adam 优化器的一个变种，它在保持 Adam 优点的同时，修正了权重衰减（Weight Decay）的实现方式，在各种深度学习任务中展现出更好的泛化性能。
与Adam的核心区别：从梯度计算到参数更新
- Adam 的实现方式
梯度计算(权重衰减添加到梯度中)：
g_t^{\text{Adam}} = ∇L_{\text{data}}(w_{t-1}) + λ w_{t-1}
权重衰减项 \(λ w_{t-1}\) 被直接添加到梯度中，参与后续的自适应计算。
- AdamW 的实现方式
梯度计算(纯数据梯度)：
g_t = ∇L_{\text{data}}(w_{t-1}) \quad
梯度计算不包含权重衰减项。权重衰减在参数更新阶段独立处理。
- AdamW/Adam 共同步骤
梯度矩估计：
m_t = β_1 m_{t−1} + (1 − β_1) g_t
s_t = β_2 s_{t−1} + (1 − β_2) g_t^2
偏差校正：
\hat{m_t} = \frac{m_t}{1 − β_1^t}, \quad \hat{s_t} = \frac{s_t}{1 − β_2^t}
- Adam参数更新 耦合了 权重衰减\(λ w_{t-1}\) 和 梯度的一阶矩\(\hat{m_t}\)与二阶矩\(\hat{s_t}\)
w_t = w_{t−1} - \frac{η}{\sqrt{\hat{s_t}} + ϵ} \hat{m_t}
其中
g_t^{\text{Adam}} = ∇L_{\text{data}}(w_{t-1}) + λ w_{t-1}
- AdamW参数更新：
w_t = (1 - ηλ) w_{t−1} - \frac{η}{\sqrt{\hat{s_t}} + ϵ} \hat{m_t}
- 关键优势
AdamW 通过将权重衰减从梯度计算中解耦，实现了真正的 L2 正则化，训练更稳定，成为现代深度学习特别是 Transformer 模型的首选优化器。
print("梯度下降优化方法 —— adamw")
# 1.初始化模型参数w
w = torch.tensor([1.0], requires_grad=True, dtype=torch.float32)
# 2.求损失值
loss = 0.5*(w**2) # w
# 3.创建优化器，基于SGD, 加入动量 momentum(动量参数)，就是动量法
# 参数1：待优化的模型参数列表，参数2：学习率，参数3：动量参数
# optimizer = optim.SGD([w], lr=0.01, momentum=0.9) # momentum默认0，也就是无动量的随机梯度下降法
# 参数1：待优化的模型参数列表，参数2：学习率
# optimizer = optim.Adagrad([w], lr=0.01)
# 参数1：待优化的模型参数列表，参数2：学习率，参数3：调和权重系数
# optimizer = optim.RMSprop([w], lr=0.01, alpha=0.99)
optimizer = optim.AdamW([w], lr=0.01, betas=(0.9, 0.999), weight_decay=1e-3)
# 4.计算梯度，梯度清零 + 反向传播 + 更新参数
optimizer.zero_grad() # 梯度清零
loss.mean().backward() # 反向传播
optimizer.step() # 更新参数
print(f"第一次梯度更新结果：\nw:{w}, w.grad: {w.grad}")
# 第二次计算损失值
loss = 0.5 * (w ** 2) # w
optimizer.zero_grad() # 梯度清零
loss.mean().backward() # 反向传播
optimizer.step() # 更新参数
print(f"第二次梯度更新结果：\nw:{w}, w.grad: {w.grad}")
梯度下降优化方法 —— adamw
第一次梯度更新结果：
w:tensor([0.9900], requires_grad=True), w.grad: tensor([1.])
第二次梯度更新结果：
w:tensor([0.9800], requires_grad=True), w.grad: tensor([0.9900])
| 优化算法 | 核心思想 | 核心变量（递推） | 参数更新 | 典型场景 | PyTorch 示例 |
|---|---|---|---|---|---|
| SGD | 使用当前梯度直接更新参数 | \(g_t = \nabla L(w_{t-1})\) | \(w_t = w_{t-1} - \eta g_t\) | 凸优化、简单模型 | optim.SGD(params, lr=1e-3) |
| Momentum | 累积历史梯度，加速收敛、减小震荡，可能跨过 鞍点 或 局部最小值 | \(g_t = \nabla L(w_{t-1})\) \(s_t = \beta s_{t-1} + (1-\beta) g_t\) |
\(w_t = w_{t-1} - \eta s_t\) | 深层网络 | optim.SGD(params, lr=1e-2, momentum=0.9) |
| AdaGrad | 自适应学习率，可能过早过快的降低学习率 | \(g_t = \nabla L(w_{t-1})\) \(s_t = s_{t-1} + g_t^2\) |
\(w_t = w_{t-1} - \dfrac{\eta}{\sqrt{s_t}+\epsilon} g_t\) | 稀疏特征（NLP） | optim.Adagrad(params, lr=1e-2) |
| RMSProp | 指数移动平均，缓解 AdaGrad 衰减过快，没有调整梯度，可能陷入 鞍点 或 局部最小值 | \(g_t = \nabla L(w_{t-1})\) \(s_t = \beta s_{t-1} + (1-\beta) g_t^2\) |
\(w_t = w_{t-1} - \dfrac{\eta}{\sqrt{s_t}+\epsilon} g_t\) | RNN / 非平稳目标 | optim.RMSprop(params, lr=1e-3) |
| Adam | 同时调整 学习率 和 梯度： 一阶矩(调整梯度) + 二阶矩(调整学习率)自适应优化 | $\(g_t=\nabla L(w_{t-1})+\lambda w_{t-1}\)$ $\(m_t=\beta_1 m_{t-1}+(1-\beta_1)g_t\)$ $\(s_t=\beta_2 s_{t-1}+(1-\beta_2)g_t^2\)$ $\(\hat m_t=\frac{m_t}{1-\beta_1^t}\)$ $\(\hat s_t=\frac{s_t}{1-\beta_2^t}\)$ | \(w_t=w_{t-1}-\dfrac{\eta}{\sqrt{\hat s_t}+\epsilon}\hat m_t\) | 通用深度学习 | optim.Adam(params, lr=1e-3) |
| AdamW | Adam + 解耦权重衰减 | \(g_t=\nabla L(w_{t-1})\) \(m_t,s_t\) 同 Adam |
\(w_t=w_{t-1}-\dfrac{\eta}{\sqrt{\hat s_t}+\epsilon}\hat m_t-\eta\lambda w_{t-1}\) | Transformer / CV / NLP | optim.AdamW(params, lr=1e-3, weight_decay=1e-3) |
-
如何选择：AdamW（首选）/Adam -> SGD/Momentum
-
简单任务和较小的模型：SGD 或 Momentum
-
复杂任务或有大量数据： AdamW（首选）或 Adam，在大部分任务上都表现优秀
常见面试题
1. pytorch神经网络训练中为什么要在反向传播之前进行梯度清零？
因为pytorch默认梯度累加
2. 多分类交叉熵损失函数的公式是什么？
-sum(真实类别*log(softmax(预测分数logits))), 正确类别的预测概率的对数的负数
3. 二分类任务中，使用 多分类交叉熵损失 和 二分类交叉熵损失 时 应该注意哪些，有何区别？
1.多分类交叉熵损失，自带了softmax, 模型输出不需要激活函数
2.二分类交叉熵损失，没有sigmoid，模型输出需要激活函数sigmoid
4. 简述梯度下降法的原理
w新 = w旧 - 学习率 * 梯度(当前批次)
沿着梯度的反方向，更新模型参数
5. 梯度下降法的缺点有哪些？
1.平缓区域，更新慢
2.鞍点，停止更新
3.局部最小值，停止更新
6. 梯度下降法的优化方法有哪些？
动量法，Adagrad, RMSProp, Adam(AdamW)
7. 动量法是如何更新模型参数的？
1.使用梯度的指数移动加权平均来替代梯度
2.w新 = w旧 - 学习率 * st
8. Adam是如何更新模型参数的？
1.使用梯度的一阶矩来修正梯度
2.使用梯度二阶矩来修正学习率
9. 神经网络的反向传播算法是如何求导的？
1.链式法则求导
2.沿着求导路径对模型参数进行链式法则求导之后，然后累加求导结果
1. pytorch神经网络训练中为什么要在反向传播之前进行梯度清零？
- 因为PyTorch默认梯度会累积，如果不清零，多个反向传播梯度会相加，可能导致梯度爆炸或梯度消失，从而影响训练效果。
- 所以在每次反向传播前，需要调用
optimizer.zero_grad()清零梯度。
2. 多分类交叉熵损失函数的公式是什么？
- 公式：\(-\sum 真实类别 \log(预测概率)\)，等价于 – log(正确类别的预测概率)。
- 内部已包含 softmax 运算。
3. 二分类任务中，使用 多分类交叉熵损失 和 二分类交叉熵损失 时 应该注意哪些，有何区别？
- 二分类任务，使用多分类交叉熵损失时，需要注意两个方面：
- 1.网络输出层的输出维度为2，分别对应正类别和负类别；
- 2.无需在网络输出层添加softmax，因为多分类交叉熵损失函数内部有softmax运算。
- 使用二分类交叉熵损失时，需要注意两个方面：
- 1.网络输出层的输出维度为1，输出对应正类别概率，负类别概率为1-正类别概率；
- 2.需要在网络输出层添加sigmoid，因为二分类交叉熵损失函数内部没有sigmoid运算。
4.简述梯度下降法的原理
- 梯度下降法的目标是找到损失函数的最小值，从而获取最优的模型参数。
- 具体做法为，首先计算损失函数对模型参数的梯度，然后沿着梯度的反方向迭代更新模型参数，直到找到损失函数的最小值。
- 更新参数的公式为：Wt = Wt-1 - 学习率 * 梯度
5. 梯度下降法的缺点有哪些？
主要有三个：
- 1.碰到小梯度的平缓区域，参数更新变慢；
- 2.碰到0梯度的 “鞍点”，参数无法更新；
- 3.可能陷入局部最小值，训练提前终止
6. 梯度下降法的优化方法有哪些？
- 1.动量法Momentum
- 2.自适应学习率AdaGrad
- 3.RMSProp
- 4.Adam/AdamW
7. 动量法是如何更新模型参数的？
- 动量法Momentum 是一种优化方法，通过引入动量参数，来加速模型参数的更新。
- 动量法Momentum 的更新公式为：
St = β * St-1(上一次) + (1-β) * 梯度(当前)，
Wt = Wt-1 - 学习率 * St,
St为梯度的指数加权平均值，β为权重系数，通常为0.9。
8. Adam是如何更新模型参数的？
Adam 是结合 动量法Momentum 和 RMSProp 的自适应优化算法。
- 1.对梯度做一阶矩和二阶矩的指数加权移动平均，得到mt 和 st
- 2.再进行偏差校正得到mt_hat 和 st_hat
- 3.然后按公式 Wt=Wt-1−学习率/(√st_hat + 小常数) * mt_hat，来更新模型参数，从而实现自适应学习率, 并加速收敛。
9. 神经网络的反向传播算法是如何求导的？
- 反向传播通过链式法则在计算图上从输出向输入反向计算梯度。
- 对任意参数，梯度是损失函数到该参数的所有路径的梯度累加，其中每条路径的梯度按链式法则相乘得到。
- 本质上，损失函数对每个参数 w 的梯度是损失函数到参数 \( w \) 的所有路径的链式法则乘积之和：
\[
\frac{\partial L}{\partial w}
=
\sum_{\text{paths}}
\prod
\frac{\partial(\text{中间变量})}{\partial(\text{前一层})}
\]
代码作业
- 1.搭建一个神经网络，有3个隐藏层和1个输出层，输入数据的输入特征维度为20，3个隐藏层的神经元数量分别为128，64，32，输出层的神经元数量为4。激活函数要求为：隐藏层使用relu激活函数，输出层使用softmax激活函数。并打印模型结构 和 模型参数。
- 2.使用代码演示 二分类任务中，分别使用 多分类交叉熵CrossEntropyLoss 和 二分类交叉熵BCELoss，比如原始输出的logits为：两个类别[[3,1], [4,3], [2,4]]，单个类别为[2,1,-2],分别计算对应的 多分类交叉熵损失值 和 二分类交叉熵损失值，并打印。
- 3.使用pytorch计算MSE损失值。已知 y_ture 为 [1, 2, 3, 4, 5, 6], y_pred 为 [2, 3, 4, 5, 6, 7]，计算MSE损失值，并打印。
- 4.使用梯度下降法，使用optim.SGD实现，循环迭代800次，求 loss = 0.5*w**2 + 10 的极小值点, 并打印loss最小值，以及loss是最小值时 w的值 和 梯度。
- 5.使用Adam优化器，使用optim.Adam实现，循环迭代800次，求 loss = 0.5*w**2 + 10 的极小值点, 并打印loss最小值，以及loss是最小值时 w的值 和 梯度。
- 6.使用AdamW优化器，使用optim.Adamw实现，循环迭代800次，求 loss = 0.5*w**2 + 10 的极小值点, 并打印loss最小值，以及loss是最小值时 w的值 和 梯度。
💻 代码
Day04 代码：激活函数 / 初始化 / 网络搭建
3 个文件📄 01_激活函数.py 6.7KB
""" 演示 常用的激活函数 人工神经元组成： 加权求和 + 激活函数(引入非线性) sigmoid 用于二分类任务的输出层，通常用于浅层网络，不超过5层，容易出现梯度消失，目前使用很少 输出范围(0,1),导数范围(0,0.25] ReLU 常用于 隐藏层，目前使用最多. 计算公式max(0,x),输出范围[0,+∞),导数{0,1} 优势： 1.计算简单，模型收敛快 2.正输入的导数恒为1，缓解了梯度消失问题 DeadReLU问题： 随着训练的推进，部分输入会落入负区域，恒为负,模型输出恒为0,导致对应权重无法更新。被称为“神经元死亡”。大模型训练时，通常采用GELU。 tanh 用于隐藏层，浅层网络，同样存在梯度消失问题，很少使用，主流网络中只有RNN(LSTM,GRU)使用了tanh 输出范围(-1,1),导数范围(0,1] softmax 用于单标签多分类任务的输出层，二分类任务上可以替代sigmoid. 输出各个类别的概率分布，范围(0,1), 并且概率总和为1. 激活函数的选择方法 隐藏层: ReLU -> Leaky ReLU/PReLU/GELU -> (Tanh) 输出层: 多分类问题选择 softmax 二分类问题选择 sigmoid 或 softmax 回归问题多数选择 identity (就是没有激活函数)，正数ReLU，区间Sigmoid/Tanh 需要掌握的： sigmoid公式、应用(面试) torch.relu torch.softmax bug解决： 绘制激活函数图像时出现以下提示: OMP: Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized. 解决方法: 方法1：更换版本matplotlib, pip install matplotlib==3.7.5 -i https://pypi.tuna.tsinghua.edu.cn/simple 方法2：更换版本numpy, pip install numpy==1.26.3 -i https://pypi.tuna.tsinghua.edu.cn/simple """ # 导包 import torch # pytorch框架 import torch.nn.functional as F # 函数模块 import matplotlib.pyplot as plt import torch.nn as nn # 神经网络模块，提供各种网络层 # 设置中文字体 plt.rcParams['font.sans-serif'] = ['SimHei'] # 黑体 plt.rcParams['axes.unicode_minus'] = False # 解决负号显示问题 # 1.定义函数，演示sigmoid激活函数的函数图像和导函数图像 def demo01(): # 1.创建画布 fig, axes = plt.subplots(1, 2, figsize=(12, 5)) # 2.生成x值，-10到10之间，总数量100 x = torch.linspace(-10, 10, 100) # 3.计算y值，sigmoid激活函数 y = torch.sigmoid(x) # y = F.sigmoid(x) # y = nn.Sigmoid()(x) # 4.绘制函数图像，在第一个子图上 axes[0].plot(x, y) axes[0].set_title('sigmoid激活函数') axes[0].grid() # 5.绘制导函数图像，在第二个子图上 x = torch.linspace(-10, 10, 100,requires_grad=True) # 1D:(100,) y = torch.sigmoid(x) # 1D:(100,) # y = F.sigmoid(x) # y = nn.Sigmoid()(x) y.sum().backward() # 方向传播获取导数，x.grad # 如果.mean()就是先求和，再除以元素数量，会变成导数的1/100 axes[1].plot(x.detach(), x.grad) axes[1].set_title('sigmoid激活函数导数') axes[1].grid() plt.show() ... # 2.定义函数，演示relu激活函数的函数图像和导函数图像 def demo02(): # 1.创建画布 fig, axes = plt.subplots(1, 2, figsize=(12, 5)) # 2.生成x值，-10到10之间，总数量100 x = torch.linspace(-10, 10, 1000) # 3.计算y值，relu激活函数 y = torch.relu(x) # y = F.relu(x) # y = nn.ReLU()(x) # 4.绘制函数图像，在第一个子图上 axes[0].plot(x, y) axes[0].set_title('relu激活函数') axes[0].grid() # 5.绘制导函数图像，在第二个子图上 x = torch.linspace(-10, 10, 1000,requires_grad=True) y = torch.relu(x) y.sum().backward() # 方向传播获取导数，x.grad axes[1].plot(x.detach(), x.grad) axes[1].set_title('relu激活函数导数') axes[1].grid() plt.show() ... # 3.定义函数，演示tanh激活函数的函数图像和导函数图像 def demo03(): # 1.创建画布 fig, axes = plt.subplots(1, 2, figsize=(12, 5)) # 2.生成x值，-10到10之间，总数量100 x = torch.linspace(-10, 10, 1000) # 3.计算y值，tanh激活函数 y = torch.tanh(x) # 4.绘制函数图像，在第一个子图上 axes[0].plot(x, y) axes[0].set_title('tanh激活函数') axes[0].grid() # 5.绘制导函数图像，在第二个子图上 x = torch.linspace(-10, 10, 1000,requires_grad=True) y = torch.tanh(x) y.sum().backward() # 方向传播获取导数，x.grad axes[1].plot(x.detach(), x.grad) axes[1].set_title('tanh激活函数导数') axes[1].grid() plt.show() ... # 4.定义函数，演示softmax激活函数的函数图像和softmax(softmax)图像 def demo04(): # 1.创建画布 fig, axes = plt.subplots(1, 2, figsize=(12, 5)) # 2.定义x,模拟模型的预测分数logits x = torch.tensor([0.2,-0.9,1,-10,1,2,3,4,2.5,3.9]) # (10,) # 举例说明：老板请客，你可以点一个菜，菜单上有10个菜，选了2对应的菜，最大预测分数10对应的类别 # 3.经过softmax运算 y = torch.softmax(x, dim=-1) # 通常特征维度/轴为-1轴 # y = F.softmax(x, dim=-1) # 4.绘制softmax函数图像，在第一个子图上 axes[0].bar(range(len(y)), y) axes[0].set_title('softmax激活函数') axes[0].grid() # 5.绘制softmax(softmax)函数图像，在第二个子图上 z = torch.softmax(y, dim=-1) axes[1].bar(range(len(z)), z) axes[1].set_title('softmax(softmax)激活函数') axes[1].grid() plt.show() ... # 5.定义函数，演示gelu激活函数的函数图像和导函数图像 def demo05(): # 1.创建画布 fig, axes = plt.subplots(1, 2, figsize=(12, 5)) # 2.生成x值，-10到10之间，总数量100 x = torch.linspace(-10, 10, 1000) # 3.计算y值，gelu激活函数 # y = torch.gelu(x) # y = F.gelu(x) y = nn.GELU()(x) # 4.绘制函数图像，在第一个子图上 axes[0].plot(x, y) axes[0].set_title('gelu激活函数') axes[0].grid() # 5.绘制导函数图像，在第二个子图上 x = torch.linspace(-10, 10, 1000,requires_grad=True) # y = torch.gelu(x) # y = F.gelu(x) y = nn.GELU()(x) y.sum().backward() # 方向传播获取导数，x.grad axes[1].plot(x.detach(), x.grad) axes[1].set_title('gelu激活函数导数') axes[1].grid() plt.show() ... # 测试 if __name__ == '__main__': # demo01() # demo02() # demo03() demo04() # demo05()
📄 02_演示参数初始化.py 4.1KB
""" 案例： 演示 参数初始化 方法 是什么： 创建神经网络时，初始化网络层的参数 注意： 实际工程中，pytorch神经网络会自动进行参数初始化，一般不需要手动参数初始化 参数初始化的作用： 1. 防止 梯度消失 或 梯度爆炸 2. 提高收敛速度 3. 打破对称性：如果所有神经元的参数都一样，则输入数据经过前向传播得到结果相同，再反向传播时，得到的梯度相同。梯度下降法(W新 = W旧 - 学习率*梯度)更新后参数还是相同，无法训练网络 参数初始化的方法： 1. 均匀分布初始化 2. 正态分布初始化 3. 全0初始化 4. 全1初始化 5. 固定值初始化 6. kaiming 初始化，也叫做 HE 初始化 正态分布的he初始化 均匀分布的he初始化 7. xavier 初始化，也叫做 Glorot初始化 正态化的Xavier初始化 均匀分布的Xavier初始化 需要掌握： 无 总结： 一般采用pytorch默认初始化方法，不需要手动初始化 极少数场景，手动参数初始化： 权重w: relu网络: kaiming 非relu网络: Xavier 偏置b: 全0：1.简单；2.计算快 """ # 导包 import torch.nn as nn # 1.定义函数，演示 均匀分布初始化 def demo01(): print("均匀分布初始化") # 1.定义一个线性层，输入维度3，输出维度5 linear = nn.Linear(3,5) # 2.对权重w进行初始化 nn.init.uniform_(linear.weight) # 3.对偏置b进行初始化 nn.init.uniform_(linear.bias) # 4.打印权重和偏置 print(f"w:{linear.weight}, shape: {linear.weight.shape}") print(f"b:{linear.bias}, shape: {linear.bias.shape}") ... # 2.定义函数，演示 正态分布初始化 def demo02(): print("正态分布初始化") # 1.定义一个线性层，输入维度3，输出维度5 linear = nn.Linear(3,5) # 2.对权重w进行初始化 nn.init.normal_(linear.weight) # 3.对偏置b进行初始化 nn.init.normal_(linear.bias) # 4.打印权重和偏置 print(f"w:{linear.weight}, shape: {linear.weight.shape}") print(f"b:{linear.bias}, shape: {linear.bias.shape}") ... # 3.定义函数，演示 全0分布初始化 def demo03(): print("全0分布初始化") # 1.定义一个线性层，输入维度3，输出维度5 linear = nn.Linear(3,5) # 2.对权重w进行初始化 # nn.init.normal_(linear.weight) # 3.对偏置b进行初始化 nn.init.zeros_(linear.bias) # inplace=True # 4.打印权重和偏置 # print(f"w:{linear.weight}, shape: {linear.weight.shape}") print(f"b:{linear.bias}, shape: {linear.bias.shape}") ... # 4.定义函数，演示 kaiming初始化 def demo04(): print("kaiming初始化") # 1.定义一个线性层，输入维度3，输出维度5 linear = nn.Linear(3,5) # 2.对权重w进行初始化,kaiming正态初始化 nn.init.kaiming_normal_(linear.weight) # 3.打印权重和偏置 print(f"正态:\nw:{linear.weight}, shape: {linear.weight.shape}") # # print(f"b:{linear.bias}, shape: {linear.bias.shape}") # 4.对权重w进行初始化,kaiming均匀初始化 nn.init.kaiming_uniform_(linear.weight) print(f"均匀:\nw:{linear.weight}, shape: {linear.weight.shape}") ... # 5.定义函数，演示 xavier初始化 def demo05(): print("xavier初始化") # 1.定义一个线性层，输入维度3，输出维度5 linear = nn.Linear(3,5) # 2.对权重w进行初始化,xavier正态初始化 nn.init.xavier_normal_(linear.weight) # 3.打印权重和偏置 print(f"正态:\nw:{linear.weight}, shape: {linear.weight.shape}") # print(f"b:{linear.bias}, shape: {linear.bias.shape}") # 4.对权重w进行初始化,xavier均匀初始化 nn.init.xavier_uniform_(linear.weight) print(f"均匀\nw:{linear.weight}, shape: {linear.weight.shape}") ... # 测试 if __name__ == '__main__': # demo01() # demo02() # demo03() demo04() demo05()
📄 03_搭建神经网络.py 2.4KB
""" 演示 搭建神经网络，这里是一个简单的全连接神经网络 这里的全连接神经网络组成： 隐藏层1：nn.Linear(3,3),sigmoid,采用默认参数初始化 隐藏层2：nn.Linear(3,2),relu,采用默认参数初始化 输出层: nn.Linear(2,2),采用softmax做数据归一化 神经网络训练的一般工作流 1.准备数据集 2.搭建神经网络 继承nn.Module父类 定义__init__方法，定义网络层 定义forward方法，实现前向传播 3.模型训练 前向传播 计算损失 梯度清零 反向传播 更新参数 4.模型测试 """ # 导包 import torch from torch import nn from torchsummary import summary # 计算模型参数,查看模型结构, pip install torchsummary -i https://mirrors.aliyun.com/pypi/simple/ # 设置设备,优先级 CUDA > CPU device = torch.device("cuda" if torch.cuda.is_available() else "cpu") print(f"设备:{device}") # 1.定义模型类，搭建神经网络 class MyModel(nn.Module): # 1.定义__init__方法，定义网络层 def __init__(self): # 1.初始化父类成员 super().__init__() # 2.定义网络层 # 2.1 定义隐藏层1 self.linear1 = nn.Linear(3,3) # 2.2 定义隐藏层2 self.linear2 = nn.Linear(3,2) # 2.3 定义输出层 self.linear3 = nn.Linear(2,2) # 3.参数初始化，一般不需要人为初始化，采用默认初始化即可 # 2.定义forward方法，实现前向传播 def forward(self,x): # x: 2D:(batch_size, 3) # 1.把数据输入隐藏层1 # x = self.linear1(x) # # 经过sigmoid # x = torch.sigmoid(x) # 合并起来 x = torch.sigmoid(self.linear1(x)) # 2.把数据输入隐藏层2 x = torch.relu(self.linear2(x)) # 3.把数据输入输出层 x = torch.softmax(self.linear3(x),dim=-1) # (batch_size,2) # 4.返回结果 return x # 测试 if __name__ == '__main__': # 1.创建模型对象 model = MyModel() print(model) # 迁移模型到设备，如果有CUDA设备，迁移到CUDA设备上，否则迁移到CPU设备上 model = model.to(device) # 2.查看模型的参数数量 summary(model, (3,)) ...
📎 资料
论文：Kaiming 初始化
- 📎 kaiming.pdf2.2MB
📄 kaiming.pdf 文字提取版
以下为 PDF 文字提取版（排版有简化，原文见附件）。
第 1 页
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xiangyu Zhang Shaoqing Ren Jian Sun Microsoft Research {kahe, v-xiangz, v-shren, jiansun}@microsoft.com Abstract Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit. PReLU improves model fitting with nearly zero extra com- putational cost and little overfitting risk. Second, we de- rive a robust initialization method that particularly consid- ers the rectifier nonlinearities. This method enables us to train extremely deep rectified models directly from scratch and to investigate deeper or wider network architectures. Based on our PReLU networks (PReLU-nets), we achieve 4.94% top-5 test error on the ImageNet 2012 classifica- tion dataset. This is a 26% relative improvement over the ILSVRC 2014 winner (GoogLeNet, 6.66% [29]). To our knowledge, our result is the first to surpass human-level per- formance (5.1%, [22]) on this visual recognition challenge. 1. Introduction Convolutional neural networks (CNNs) [17, 16] have demonstrated recognition accuracy better than or compara- ble to humans in several visual recognition tasks, includ- ing recognizing traffic signs [3], faces [30, 28], and hand- written digits [3, 31]. In this work, we present a result that surpasses human-level performance on a more generic and challenging recognition task - the classification task in the 1000-class ImageNet dataset [22]. In the last few years, we have witnessed tremendous im- provements in recognition performance, mainly due to ad- vances in two technical directions: building more powerful models, and designing effective strategies against overfit- ting. On one hand, neural networks are becoming more ca- pable of fitting training data, because of increased complex- ity (e.g., increased depth [25, 29], enlarged width [33, 24], and the use of smaller strides [33, 24, 2, 25]), new non- linear activations [21, 20, 34, 19, 27, 9], and sophisti- cated layer designs [29, 11]. On the other hand, bet- ter generalization is achieved by effective regularization techniques [12, 26, 9, 31], aggressive data augmentation [16, 13, 25, 29], and large-scale data [4, 22]. Among these advances, the rectifier neuron [21, 8, 20, 34], e.g., Rectified Linear Unit (ReLU), is one of several keys to the recent success of deep networks [16]. It expe- dites convergence of the training procedure [16] and leads to better solutions [21, 8, 20, 34] than conventional sigmoid- like units. Despite the prevalence of rectifier networks, recent improvements of models [33, 24, 11, 25, 29] and theoretical guidelines for training them [7, 23] have rarely focused on the properties of the rectifiers. In this paper, we investigate neural networks from two aspects particularly driven by the rectifiers. First, we propose a new generalization of ReLU, which we call Parametric Rectified Linear Unit (PReLU). This activation function adaptively learns the parameters of the rectifiers, and improves accuracy at negligible extra computational cost. Second, we study the difficulty of training rectified models that are very deep. By explicitly modeling the non- linearity of rectifiers (ReLU/PReLU), we derive a theoret- ically sound initialization method, which helps with con- vergence of very deep models (e.g., with 30 weight layers) trained directly from scratch. This gives us more flexibility to explore more powerful network architectures. On the 1000-class ImageNet 2012 dataset, our PReLU network (PReLU-net) leads to a single-model result of 5.71% top-5 error, which surpasses all existing multi-model results. Further, our multi-model result achieves 4.94% top-5 error on the test set, which is a 26% relative improve- ment over the ILSVRC 2014 winner (GoogLeNet, 6.66% [29]). To the best of
第 2 页
f (y) = y y f (y) f (y) = y f (y) = ay y f (y) f (y) = 0 Figure 1. ReLU vs. PReLU. For PReLU, the coefficient of the negative part is not constant and is adaptively learned. 2. Approach In this section, we first present the PReLU activation function (Sec. 2.1). Then we derive our initialization method for deep rectifier networks (Sec. 2.2). Lastly we discuss our architecture designs (Sec. 2.3). 2.1. Parametric Rectifiers We show that replacing the parameter-free ReLU activa- tion by a learned parametric activation unit improves clas- sification accuracy1. Definition Formally, we consider an activation function defined as: f(yi) = { yi, ifyi > 0 aiyi, ifyi≤ 0. (1) Hereyi is the input of the nonlinear activation f on theith channel, and ai is a coefficient controlling the slope of the negative part. The subscript i inai indicates that we allow the nonlinear activation to vary on different channels. When ai = 0, it becomes ReLU; whenai is a learnable parameter, we refer to Eqn.(1) as Parametric ReLU (PReLU). Figure 1 shows the shapes of ReLU and PReLU. Eqn.(1) is equiva- lent tof(yi) = max(0,y i) +ai min(0,y i). If ai is a small and fixed value, PReLU becomes the Leaky ReLU (LReLU) in [20] ( ai = 0.01). The motiva- tion of LReLU is to avoid zero gradients. Experiments in [20] show that LReLU has negligible impact on accuracy compared with ReLU. On the contrary, our method adap- tively learns the PReLU parameters jointly with the whole model. We hope for end-to-end training that will lead to more specialized activations. PReLU introduces a very small number of extra param- eters. The number of extra parameters is equal to the total number of channels, which is negligible when considering the total number of weights. So we expect no extra risk of overfitting. We also consider a channel-shared variant: 1Concurrent with our work, Agostinelli et al . [1] also investigated learning activation functions and showed improvement on other tasks. f(yi) = max(0,y i) +a min(0,y i) where the coefficient is shared by all channels of one layer. This variant only intro- duces a single extra parameter into each layer. Optimization PReLU can be trained using backpropagation [17] and opti- mized simultaneously with other layers. The update formu- lations of{ai} are simply derived from the chain rule. The gradient ofai for one layer is: ∂E ∂ai = ∑ yi ∂E ∂f (yi) ∂f (yi) ∂ai , (2) whereE represents the objective function. The term ∂E∂f (yi) is the gradient propagated from the deeper layer. The gradi- ent of the activation is given by: ∂f (yi) ∂ai = { 0, ifyi > 0 yi, ifyi≤ 0. (3) The summation ∑yi runs over all positions of the feature map. For the channel-shared variant, the gradient of a is ∂E ∂a =∑i∑yi ∂E∂f (yi) ∂f (yi)∂a , where∑i sums over all chan- nels of the layer. The time complexity due to PReLU is negligible for both forward and backward propagation. We adopt the momentum method when updatingai: ∆ai :=µ∆ai +ϵ∂E∂ai. (4) Here µ is the momentum and ϵ is the learning rate. It is worth noticing that we do not use weight decay (l2 regular- ization) when updatingai. A weight decay tends to pushai to zero, and thus biases PReLU toward ReLU. Even without regularization, the learned coefficients rarely have a magni- tude larger than 1 in our experiments. Further, we do not constrain the range ofai so that the activation function may be non-monotonic. We use ai = 0.25 as the initialization throughout this paper. Comparison Experiments We conducted comparisons on a deep but efficient model with 14 weight layers. The model was studied in [10] (model E of [10]) and its architecture is described in Ta- ble 1. We choose this model because it is sufficient for rep- resenting a category of very deep models, as well as to make the experiments feasible. As a baseline, we train this model with ReLU applied in the convolutional (conv) layers and the first two fully- connected (fc) layers. The training implementation follows [10]. The top-1 and top-5 errors are 33.82% and 13.34% o
第 3 页
learned coefficients layer channel-shared channel-wise conv1 7 ×7, 64, /2 0.681 0.596 pool1 3 ×3, /3 conv21 2×2, 128 0.103 0.321 conv22 2×2, 128 0.099 0.204 conv23 2×2, 128 0.228 0.294 conv24 2×2, 128 0.561 0.464 pool2 2 ×2, /2 conv31 2×2, 256 0.126 0.196 conv32 2×2, 256 0.089 0.152 conv33 2×2, 256 0.124 0.145 conv34 2×2, 256 0.062 0.124 conv35 2×2, 256 0.008 0.134 conv36 2×2, 256 0.210 0.198 spp {6, 3, 2, 1} fc1 4096 0.063 0.074 fc2 4096 0.031 0.075 fc3 1000 Table 1. A small but deep 14-layer model [10]. The filter size and filter number of each layer is listed. The number / s indicates the stride s that is used. The learned coefficients of PReLU are also shown. For the channel-wise case, the average of {ai} over the channels is shown for each layer. top-1 top-5 ReLU 33.82 13.34 PReLU, channel-shared 32.71 12.87 PReLU, channel-wise 32.64 12.75 Table 2. Comparisons between ReLU and PReLU on the small model. The error rates are for ImageNet 2012 using 10-view test- ing. The images are resized so that the shorter side is 256, during both training and testing. Each view is 224×224. All models are trained using 75 epochs. Then we train the same architecture from scratch, with all ReLUs replaced by PReLUs (Table 2). The top-1 error is reduced to 32.64%. This is a 1.2% gain over the ReLU baseline. Table 2 also shows that channel-wise/channel- shared PReLUs perform comparably. For the channel- shared version, PReLU only introduces 13 extra free pa- rameters compared with the ReLU counterpart. But this small number of free parameters play critical roles as ev- idenced by the 1.1% gain over the baseline. This implies the importance of adaptively learning the shapes of activa- tion functions. Table 1 also shows the learned coefficients of PReLUs for each layer. There are two interesting phenomena in Ta- ble 1. First, the first conv layer (conv1) has coefficients (0.681 and 0.596) significantly greater than 0. As the fil- ters of conv1 are mostly Gabor-like filters such as edge or texture detectors, the learned results show that both positive and negative responses of the filters are respected. We be- lieve that this is a more economical way of exploiting low- level information, given the limited number of filters ( e.g., 64). Second, for the channel-wise version, the deeper conv layers in general have smaller coefficients. This implies that the activations gradually become “more nonlinear” at in- creasing depths. In other words, the learned model tends to keep more information in earlier stages and becomes more discriminative in deeper stages. 2.2. Initialization of Filter Weights for Rectifiers Rectifier networks are easier to train [8, 16, 34] com- pared with traditional sigmoid-like activation networks. But a bad initialization can still hamper the learning of a highly non-linear system. In this subsection, we propose a robust initialization method that removes an obstacle of training extremely deep rectifier networks. Recent deep CNNs are mostly initialized by random weights drawn from Gaussian distributions [16]. With fixed standard deviations ( e.g., 0.01 in [16]), very deep models (e.g., >8 conv layers) have difficulties to converge, as re- ported by the VGG team [25] and also observed in our ex- periments. To address this issue, in [25] they pre-train a model with 8 conv layers to initialize deeper models. But this strategy requires more training time, and may also lead to a poorer local optimum. In [29, 18], auxiliary classifiers are added to intermediate layers to help with convergence. Glorot and Bengio [7] proposed to adopt a properly scaled uniform distribution for initialization. This is called “Xavier” initialization in [14]. Its derivation is based on the assumption that the activations are linear. This assumption is invalid for ReLU and PReLU. In the following, we derive a theoretically more sound initialization by taking ReLU/PReLU into account. In our experiments, our initialization method allows for extremely deep models (e.g., 30 conv
第 4 页
We let the initialized elements in Wl be mutually inde- pendent and share the same distribution. As in [7], we as- sume that the elements in xl are also mutually independent and share the same distribution, andxl and Wl are indepen- dent of each other. Then we have: Var[yl] =nlVar[wlxl], (6) where nowyl,xl, andwl represent the random variables of each element in yl, Wl, and xl respectively. We letwl have zero mean. Then the variance of the product of independent variables gives us: Var[yl] =nlVar[wl]E[x2l ]. (7) HereE[x2l ] is the expectation of the square ofxl. It is worth noticing thatE[x2l ]⁄= Var[xl] unlessxl has zero mean. For the ReLU activation, xl = max(0,y l−1) and thus it does not have zero mean. This will lead to a conclusion different from [7]. If we letwl−1 have a symmetric distribution around zero andbl−1 = 0, thenyl−1 has zero mean and has a symmetric distribution around zero. This leads to E[x2l ] = 12Var[yl−1] whenf is ReLU. Putting this into Eqn.(7), we obtain: Var[yl] = 12nlVar[wl]Var[yl−1]. (8) WithL layers put together, we have: Var[yL] = Var[y1] ( L∏ l=2 1 2nlVar[wl] ) . (9) This product is the key to the initialization design. A proper initialization method should avoid reducing or magnifying the magnitudes of input signals exponentially. So we ex- pect the above product to take a proper scalar ( e.g., 1). A sufficient condition is: 1 2nlVar[wl] = 1, ∀l. (10) This leads to a zero-mean Gaussian distribution whose stan- dard deviation (std) is √ 2/nl. This is our way of initializa- tion. We also initialize b = 0. For the first layer (l = 1), we should haven1Var[w1] = 1 because there is no ReLU applied on the input signal. But the factor 1/2 does not matter if it just exists on one layer. So we also adopt Eqn.(10) in the first layer for simplicity. Backward Propagation Case For back-propagation, the gradient of a conv layer is com- puted by: ∆xl = ˆWl∆yl. (11) Here we use ∆x and ∆y to denote gradients ( ∂E∂x and ∂E∂y ) for simplicity. ∆y representsk-by-k pixels in d channels, and is reshaped into ak2d-by-1 vector. We denoteˆn =k2d. Note that ˆn⁄= n = k2c. ˆW is a c-by-ˆn matrix where the filters are rearranged in the way of back-propagation. Note that W and ˆW can be reshaped from each other. ∆x is ac- by-1 vector representing the gradient at a pixel of this layer. As above, we assume that wl and ∆yl are independent of each other, then ∆xl has zero mean for all l, when wl is initialized by a symmetric distribution around zero. In back-propagation we also have ∆yl = f′(yl)∆xl+1 wheref′ is the derivative of f. For the ReLU case, f′(yl) is zero or one, and their probabilities are equal. We as- sume thatf′(yl) and ∆xl+1 are independent of each other. Thus we have E[∆yl] = E[∆xl+1]/2 = 0 , and also E[(∆yl)2] = Var[∆yl] = 12Var[∆xl+1]. Then we compute the variance of the gradient in Eqn.(11): Var[∆xl] = ˆ nlVar[wl]Var[∆yl] = 12 ˆnlVar[wl]Var[∆xl+1]. (12) The scalar 1/2 in both Eqn.(12) and Eqn.(8) is the result of ReLU, though the derivations are different. With L layers put together, we have: Var[∆x2] = Var[∆xL+1] ( L∏ l=2 1 2 ˆnlVar[wl] ) . (13) We consider a sufficient condition that the gradient is not exponentially large/small: 1 2 ˆnlVar[wl] = 1, ∀l. (14) The only difference between this equation and Eqn.(10) is that ˆnl =k2ldl whilenl =k2lcl =k2ldl−1. Eqn.(14) results in a zero-mean Gaussian distribution whose std is √ 2/ˆnl. For the first layer ( l = 1 ), we need not compute ∆x1 because it represents the image domain. But we can still adopt Eqn.(14) in the first layer, for the same reason as in the forward propagation case - the factor of a single layer does not make the overall product exponentially large/small. We note that it is sufficient to use either Eqn.(14) or Eqn.(10) alone. For example, if we use Eqn.(14), then in Eqn.(13) the product∏Ll=2 12 ˆnlVar[wl] = 1, and in Eqn.(9) the product ∏Ll=2 12nlVar[wl] = ∏Ll=2nl/ˆnl = c2/dL, which is not a diminishing number in common network de- signs. This means that
第 5 页
0 0.5 1 1.5 2 2.5 3 0.75 0.8 0.85 0.9 0.95 1 Epoch Error ---------- ---------- ours Xavier Figure 2. The convergence of a 22-layer large model (B in Ta- ble 3). The x-axis is the number of training epochs. The y-axis is the top-1 error of 3,000 random val samples, evaluated on the cen- ter crop. We use ReLU as the activation for both cases. Both our initialization (red) and “Xavier” (blue) [7] lead to convergence, but ours starts reducing error earlier. 0 1 2 3 4 5 6 7 8 9 0.75 0.8 0.85 0.9 0.95 Epoch Error ---------- ---------- ours Xavier Figure 3. The convergence of a30-layer small model (see the main text). We use ReLU as the activation for both cases. Our initial- ization (red) is able to make it converge. But “ Xavier” (blue) [7] completely stalls - we also verify that its gradients are all dimin- ishing. It does not converge even given more epochs. will be rescaled by a factor of βL afterL layers, where L can represent some or all layers. WhenL is large, ifβ >1, this leads to extremely amplified signals and an algorithm output of infinity; if β < 1, this leads to diminishing sig- nals2. In either case, the algorithm does not converge - it diverges in the former case, and stalls in the latter. Our derivation also explains why the constant standard deviation of 0.01 makes some deeper networks stall [25]. We take “model B” in the VGG team’s paper [25] as an example. This model has 10 conv layers all with 3×3 filters. The filter numbers (dl) are 64 for the 1st and 2nd layers, 128 for the 3rd and 4th layers, 256 for the 5th and 6th layers, and 512 for the rest. The std computed by Eqn.(14) ( √ 2/ˆnl) is 0.059, 0.042, 0.029, and 0.021 when the filter numbers are 64, 128, 256, and 512 respectively. If the std is initialized 2In the presence of weight decay (l 2 regularization of weights), when the gradient contributed by the logistic loss function is diminishing, the total gradient is not diminishing because of the weight decay. A way of diagnosing diminishing gradients is to check whether the gradient is mod- ulated only by weight decay. as 0.01, the std of the gradient propagated from conv10 to conv2 is 1/(5.9× 4.22× 2.92× 2.14) = 1/(1.7× 104) of what we derive. This number may explain why diminishing gradients were observed in experiments. It is also worth noticing that the variance of the input signal can be roughly preserved from the first layer to the last. In cases when the input signal is not normalized ( e.g., it is in the range of [−128, 128]), its magnitude can be so large that the softmax operator will overflow. A solu- tion is to normalize the input signal, but this may impact other hyper-parameters. Another solution is to include a small factor on the weights among all or some layers, e.g., L √ 1/128 onL layers. In practice, we use a std of 0.01 for the first two fc layers and 0.001 for the last. These numbers are smaller than they should be ( e.g., √ 2/4096) and will address the normalization issue of images whose range is about [−128, 128]. For the initialization in the PReLU case, it is easy to show that Eqn.(10) becomes: 1 2(1 +a2)nlVar[wl] = 1, ∀l, (15) wherea is the initialized value of the coefficients. Ifa = 0, it becomes the ReLU case; if a = 1, it becomes the linear case (the same as [7]). Similarly, Eqn.(14) becomes 12(1 + a2)ˆnlVar[wl] = 1. Comparisons with “Xavier” Initialization [7] The main difference between our derivation and the “Xavier” initialization [7] is that we address the rectifier nonlinearities3. The derivation in [7] only considers the linear case, and its result is given by nlVar[wl] = 1 (the forward case), which can be implemented as a zero-mean Gaussian distribution whose std is √ 1/nl. When there are L layers, the std will be 1/ √ 2 L of our derived std. This number, however, is not small enough to completely stall the convergence of the models actually used in our paper (Table 3, up to 22 layers) as shown by experiments. Fig- ure 2 compares the convergence of a 22-layer model. Both
第 6 页
Table 1. Figure 3 shows the convergence of the 30-layer model. Our initialization is able to make the extremely deep model converge. On the contrary, the “Xavier” method completely stalls the learning, and the gradients are dimin- ishing as monitored in the experiments. These studies demonstrate that we are ready to investi- gate extremely deep, rectified models by using a more prin- cipled initialization method. But in our current experiments on ImageNet, we have not observed the benefit from train- ing extremely deep models. For example, the aforemen- tioned 30-layer model has 38.56/16.59 top-1/top-5 error, which is clearly worse than the error of the 14-layer model in Table 2 (33.82/13.34). Accuracy saturation or degrada- tion was also observed in the study of small models [10], VGG’s large models [25], and in speech recognition [34]. This is perhaps because the method of increasing depth is not appropriate, or the recognition task is not enough com- plex. Though our attempts of extremely deep models have not shown benefits, our initialization method paves a foundation for further study on increasing depth. We hope this will be helpful in other more complex tasks. 2.3. Architectures The above investigations provide guidelines of designing our architectures, introduced as follows. Our baseline is the 19-layer model (A) in Table 3. For a better comparison, we also list the VGG-19 model [25]. Our model A has the following modifications on VGG-19: (i) in the first layer, we use a filter size of 7×7 and a stride of 2; (ii) we move the other three conv layers on the two largest feature maps (224, 112) to the smaller feature maps (56, 28, 14). The time complexity (Table 3, last row) is roughly unchanged because the deeper layers have more filters; (iii) we use spatial pyramid pooling (SPP) [11] before the first fc layer. The pyramid has 4 levels - the numbers of bins are 7×7, 3×3, 2×2, and 1×1, for a total of 63 bins. It is worth noticing that we have no evidence that our model A is a better architecture than VGG-19, though our model A has better results than VGG-19’s result reported by [25]. In our earlier experiments with less scale aug- mentation, we observed that our model A and our repro- duced VGG-19 (with SPP and our initialization) are com- parable. The main purpose of using model A is for faster running speed. The actual running time of the conv lay- ers on larger feature maps is slower than those on smaller feature maps, when their time complexity is the same. In our four-GPU implementation, our model A takes 2.6s per mini-batch (128), and our reproduced VGG-19 takes 3.0s, evaluated on four Nvidia K20 GPUs. In Table 3, our model B is a deeper version of A. It has three extra conv layers. Our model C is a wider (with more filters) version of B. The width substantially increases the complexity, and its time complexity is about 2.3× of B (Ta- ble 3, last row). Training A/B on four K20 GPUs, or train- ing C on eight K40 GPUs, takes about 3-4 weeks. We choose to increase the model width instead of depth, because deeper models have only diminishing improvement or even degradation on accuracy. In recent experiments on small models [10], it has been found that aggressively in- creasing the depth leads to saturated or degraded accuracy. In the VGG paper [25], the 16-layer and 19-layer models perform comparably. In the speech recognition research of [34], the deep models degrade when using more than 8 hid- den layers (all being fc). We conjecture that similar degra- dation may also happen on larger models for ImageNet. We have monitored the training procedures of some extremely deep models (with 3 to 9 layers added on B in Table 3), and found both training and testing error rates degraded in the first 20 epochs (but we did not run to the end due to limited time budget, so there is not yet solid evidence that these large and overly deep models will ultimately degrade). Be- cause of the possible degradation, we choose not to further increase
第 7 页
input size VGG-19 [25] model A model B model C 224 3×3, 64 7×7, 96, /2 7×7, 96, /2 7×7, 96, /2 3×3, 64 2×2 maxpool, /2 112 3×3, 128 3×3, 128 2×2 maxpool, /2 2×2 maxpool, /2 2×2 maxpool, /2 2×2 maxpool, /2 56 3×3, 256 3×3, 256 3×3, 256 3×3, 384 3×3, 256 3×3, 256 3×3, 256 3×3, 384 3×3, 256 3×3, 256 3×3, 256 3×3, 384 3×3, 256 3×3, 256 3×3, 256 3×3, 384 3×3, 256 3×3, 256 3×3, 384 3×3, 256 3×3, 384 2×2 maxpool, /2 2×2 maxpool, /2 2×2 maxpool, /2 2×2 maxpool, /2 28 3×3, 512 3×3, 512 3×3, 512 3×3, 768 3×3, 512 3×3, 512 3×3, 512 3×3, 768 3×3, 512 3×3, 512 3×3, 512 3×3, 768 3×3, 512 3×3, 512 3×3, 512 3×3, 768 3×3, 512 3×3, 512 3×3, 768 3×3, 512 3×3, 768 2×2 maxpool, /2 2×2 maxpool, /2 2×2 maxpool, /2 2×2 maxpool, /2 14 3×3, 512 3×3, 512 3×3, 512 3×3, 896 3×3, 512 3×3, 512 3×3, 512 3×3, 896 3×3, 512 3×3, 512 3×3, 512 3×3, 896 3×3, 512 3×3, 512 3×3, 512 3×3, 896 3×3, 512 3×3, 512 3×3, 896 3×3, 512 3×3, 896 2×2 maxpool, /2 spp,{7, 3, 2, 1} spp,{7, 3, 2, 1} spp,{7, 3, 2, 1} fc1 4096 fc2 4096 fc3 1000 depth (conv+fc) 19 19 22 22 complexity (ops.,×1010) 1.96 1.90 2.32 5.30 Table 3. Architectures of large models. Here “/2” denotes a stride of 2. and 1e-4, and is switched when the error plateaus. The total number of epochs is about 80 for each model. Testing We adopt the strategy of “multi-view testing on feature maps” used in the SPP-net paper [11]. We further im- prove this strategy using the dense sliding window method in [24, 25]. We first apply the convolutional layers on the resized full image and obtain the last convolutional feature map. In the feature map, each 14×14 window is pooled using the SPP layer [11]. The fc layers are then applied on the pooled features to compute the scores. This is also done on the horizontally flipped images. The scores of all dense sliding windows are averaged [24, 25]. We further combine the results at multiple scales as in [11]. Multi-GPU Implementation We adopt a simple variant of Krizhevsky’s method [15] for parallel training on multiple GPUs. We adopt “data paral- lelism” [15] on the conv layers. The GPUs are synchronized before the first fc layer. Then the forward/backward prop- agations of the fc layers are performed on a single GPU - this means that we do not parallelize the computation of the fc layers. The time cost of the fc layers is low, so it is not necessary to parallelize them. This leads to a simpler implementation than the “model parallelism” in [15]. Be- sides, model parallelism introduces some overhead due to the communication of filter responses, and is not faster than computing the fc layers on just a single GPU. We implement the above algorithm on our modification of the Caffe library [14]. We do not increase the mini-batch size (128) because the accuracy may be decreased [15]. For the large models in this paper, we have observed a 3.8x speedup using 4 GPUs, and a 6.0x speedup using 8 GPUs. 7
第 8 页
model A ReLU PReLU scale s top-1 top-5 top-1 top-5 256 26.25 8.25 25.81 8.08 384 24.77 7.26 24.20 7.03 480 25.46 7.63 24.83 7.39 multi-scale 24.02 6.51 22.97 6.28 Table 4. Comparisons between ReLU/PReLU on model A in Ima- geNet 2012 using dense testing. 4. Experiments on ImageNet We perform the experiments on the 1000-class ImageNet 2012 dataset [22] which contains about 1.2 million training images, 50,000 validation images, and 100,000 test images (with no published labels). The results are measured by top- 1/top-5 error rates [22]. We only use the provided data for training. All results are evaluated on the validation set, ex- cept for the final results in Table 7, which are evaluated on the test set. The top-5 error rate is the metric officially used to rank the methods in the classification challenge [22]. Comparisons between ReLU and PReLU In Table 4, we compare ReLU and PReLU on the large model A. We use the channel-wise version of PReLU. For fair comparisons, both ReLU/PReLU models are trained us- ing the same total number of epochs, and the learning rates are also switched after running the same number of epochs. Table 4 shows the results at three scales and the multi- scale combination. The best single scale is 384, possibly because it is in the middle of the jittering range [256, 512]. For the multi-scale combination, PReLU reduces the top- 1 error by 1.05% and the top-5 error by 0.23% compared with ReLU. The results in Table 2 and Table 4 consistently show that PReLU improves both small and large models. This improvement is obtained with almost no computational cost. Comparisons of Single-model Results Next we compare single-model results. We first show 10- view testing results [16] in Table 5. Here, each view is a 224-crop. The 10-view results of VGG-16 are based on our testing using the publicly released model [25] as it is not reported in [25]. Our best 10-view result is 7.38% (Table 5). Our other models also outperform the existing results. Table 6 shows the comparisons of single-model results, which are all obtained using multi-scale and multi-view (or dense) test. Our results are denoted as MSRA. Our base- line model (A+ReLU, 6.51%) is already substantially better than the best existing single-model result of 7.1% reported for VGG-19 in the latest update of [25] (arXiv v5). We be- lieve that this gain is mainly due to our end-to-end training, without the need of pre-training shallow models. Moreover, our best single model (C, PReLU) has5.71% top-5 error. This result is even better than all previous multi-model results (Table 7). Comparing A+PReLU with B+PReLU, we see that the 19-layer model and the 22-layer model perform comparably. On the other hand, increasing the width (C vs. B, Table 6) can still improve accuracy. This indicates that when the models are deep enough, the width becomes an essential factor for accuracy. Comparisons of Multi-model Results We combine six models including those in Table 6. For the time being we have trained only one model with architec- ture C. The other models have accuracy inferior to C by con- siderable margins. We conjecture that we can obtain better results by using fewer stronger models. The multi-model results are in Table 7. Our result is 4.94% top-5 error on the test set. This number is evaluated by the ILSVRC server, because the labels of the test set are not published. Our result is 1.7% better than the ILSVRC 2014 winner (GoogLeNet, 6.66% [29]), which represents a ∼26% relative improvement. This is also a ∼17% relative improvement over the latest result (Baidu, 5.98% [32]). Analysis of Results Figure 4 shows some example validation images success- fully classified by our method. Besides the correctly pre- dicted labels, we also pay attention to the other four predic- tions in the top-5 results. Some of these four labels are other objects in the multi-object images,e.g., the “horse-cart” im- age (Figure 4, row 1, col 1) contains a “mini-bus” and it is also recognized by th
第 9 页
model top-1 top-5 MSRA [11] 29.68 10.95 VGG-16 [25] 28.07† 9.33† GoogLeNet [29] - 9.15 A, ReLU 26.48 8.59 A, PReLU 25.59 8.23 B, PReLU 25.53 8.13 C, PReLU 24.27 7.38 Table 5. The single-model 10-view results for ImageNet 2012 val set. †: Based on our tests. team top-1 top-5 in competition ILSVRC 14 MSRA [11] 27.86 9.08 † VGG [25] - 8.43 † GoogLeNet [29] - 7.89 post-competition VGG [25] (arXiv v2) 24.8 7.5 VGG [25] (arXiv v5) 24.4 7.1 Baidu [32] 24.88 7.42 MSRA (A, ReLU) 24.02 6.51 MSRA (A, PReLU) 22.97 6.28 MSRA (B, PReLU) 22.85 6.27 MSRA (C, PReLU) 21.59 5.71 Table 6. The single-model results for ImageNet 2012 val set. †: Evaluated from the test set. team top-5 (test) in competition ILSVRC 14 MSRA, SPP-nets [11] 8.06 VGG [25] 7.32 GoogLeNet [29] 6.66 post-competition VGG [25] (arXiv v5) 6.8 Baidu [32] 5.98 MSRA, PReLU-nets 4.94 Table 7. The multi-model results for the ImageNet 2012 test set. Comparisons with Human Performance from [22] Russakovsky et al. [22] recently reported that human per- formance yields a 5.1% top-5 error on the ImageNet dataset. This number is achieved by a human annotator who is well trained on the validation images to be better aware of the existence of relevant classes. When annotating the test images, the human annotator is given a special interface, where each class title is accompanied by a row of 13 ex- ample training images. The reported human performance is estimated on a random subset of 1500 test images. Our result (4.94%) exceeds the reported human-level performance. To our knowledge, our result is the first pub- lished instance of surpassing humans on this visual recog- nition challenge. The analysis in [22] reveals that the two major types of human errors come from fine-grained recog- nition and class unawareness. The investigation in [22] sug- gests that algorithms can do a better job on fine-grained recognition (e.g., 120 species of dogs in the dataset). The second row of Figure 4 shows some example fine-grained objects successfully recognized by our method - “coucal”, “komondor”, and “yellow lady’s slipper”. While humans can easily recognize these objects as a bird, a dog, and a flower, it is nontrivial for most humans to tell their species. On the negative side, our algorithm still makes mistakes in cases that are not difficult for humans, especially for those requiring context understanding or high-level knowledge (e.g., the “spotlight” images in Figure 5). While our algorithm produces a superior result on this particular dataset, this does not indicate that machine vision outperforms human vision on object recognition in general. On recognizing elementary object categories (i.e., common objects or concepts in daily lives) such as the Pascal VOC 9
第 10 页
GT: forklift 1: forklift 2: garbage truck 3: tow truck 4: trailer truck 5: go-kart GT: birdhouse 1: birdhouse 2: sliding door 3: window screen 4: mailbox 5: pot GT: microwave 1: microwave 2: washer 3: toaster 4: stove 5: dishwasher GT: sunscreen 1: hair spray 2: ice lolly 3: sunscreen 4: water bottle 5: lotion GT: wooden spoon 1: wok 2: frying pan 3: spatula 4: wooden spoon 5: hot pot GT: flute 1: flute 2: oboe 3: panpipe 4: trombone 5: bassoon GT: geyser 1: geyser 2: volcano 3: sandbar 4: breakwater 5: leatherback turtle GT: mountain tent 1: sleeping bag 2: mountain tent 3: parachute 4: ski 5: flagpole GT: go-kart 1: go-kart 2: crash helmet 3: racer 4: sports car 5: motor scooter GT: banjo 1: acoustic guitar 2: shoji 3: bow tie 4: cowboy hat 5: banjo GT: torch 1: stage 2: spotlight 3: torch 4: microphone 5: feather boa GT: yellow lady's slipper 1: yellow lady's slipper 2: slug 3: hen-of-the-woods 4: stinkhorn 5: coral fungus GT: horse cart 1: horse cart 2: minibus 3: oxcart 4: stretcher 5: half track GT: coucal 1: coucal 2: indigo bunting 3: lorikeet 4: walking stick 5: custard apple GT: komondor 1: komondor 2: patio 3: llama 4: mobile home 5: Old English sheepdog Figure 4. Example validation images successfully classified by our method. For each image, the ground-truth label and the top-5 la- bels predicted by our method are listed. task [6], machines still have obvious errors in cases that are trivial for humans. Nevertheless, we believe that our re- sults show the tremendous potential of machine algorithms to match human-level performance on visual recognition. GT: restaurant 1: plate 2: meat loaf 3: ice cream 4: chocolate sauce 5: potpie GT: restaurant 1: goblet 2: plate 3: candle 4: red wine 5: dining table GT: restaurant 1: wine bottle 2: candle 3: red wine 4: French loaf 5: wooden spoon GT: spotlight 1: grand piano 2: folding chair 3: rocking chair 4: dining table 5: upright piano GT: spotlight 1: altar 2: candle 3: perfume 4: restaurant 5: confectionery GT: spotlight 1: acoustic guitar 2: stage 3: microphone 4: electric guitar 5: banjo GT: letter opener 1: drumstick 2: candle 3: wooden spoon 4: spatula 5: ladle GT: letter opener 1: Band Aid 2: ruler 3: rubber eraser 4: pencil box 5: wallet GT: letter opener 1: fountain pen 2: ballpoint 3: hammer 4: can opener 5: ruler Figure 5. Example validation images incorrectly classified by our method, in the three classes with the highest top-5 test error. Top: “letter opener” (49% top-5 test error). Middle: “spotlight” (38%). Bottom: “restaurant” (36%). For each image, the ground-truth label and the top-5 labels predicted by our method are listed. References [1] F. Agostinelli, M. Hoffman, P. Sadowski, and P. Baldi. Learning activation functions to improve deep neural net- works. arXiv:1412.6830, 2014. [2] K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman. Return of the devil in the details: Delving deep into convo- lutional nets. In BMVC, 2014. [3] D. Ciresan, U. Meier, and J. Schmidhuber. Multi-column deep neural networks for image classification. In CVPR, 2012. [4] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei- Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009. [5] D. Eigen, J. Rolfe, R. Fergus, and Y . LeCun. Understanding deep architectures using a recursive convolutional network. arXiv:1312.1847, 2013. [6] M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The Pascal Visual Object Classes (VOC) Challenge. IJCV, pages 303–338, 2010. 10
第 11 页
/g1004 /g1004/g856/g1005 /g1004/g856/g1006 /g1004/g856/g1007 /g1004/g856/g1008 /g1004/g856/g1009 /g1005/g1004/g1004 /g1006/g1004/g1004 /g1007/g1004/g1004 /g1008/g1004/g1004 /g1009/g1004/g1004 /g1010/g1004/g1004 /g1011/g1004/g1004 /g1012/g1004/g1004 /g1013/g1004/g1004 Figure 6. The per-class top-5 errors of our result (average of 4.94%) on the test set. Errors are displayed in ascending order. /g882/g1004/g856/g1004/g1009 /g1004 /g1004/g856/g1004/g1009 /g1004/g856/g1005 /g1004/g856/g1005/g1009 /g1004/g856/g1006 /g1005/g1004/g1004 /g1006/g1004/g1004 /g1007/g1004/g1004 /g1008/g1004/g1004 /g1009/g1004/g1004 /g1010/g1004/g1004 /g1011/g1004/g1004 /g1012/g1004/g1004 /g1013/g1004/g1004 Figure 7. The difference of top-5 error rates between our result (av- erage of 4.94%) and our team’s in-competition result for ILSVRC 2014 (average of 8.06%) on the test set, displayed in ascending order. A positive number indicates a reduced error rate. [7] X. Glorot and Y . Bengio. Understanding the difficulty of training deep feedforward neural networks. In International Conference on Artificial Intelligence and Statistics , pages 249–256, 2010. [8] X. Glorot, A. Bordes, and Y . Bengio. Deep sparse rectifier networks. In Proceedings of the 14th International Confer- ence on Artificial Intelligence and Statistics, pages 315–323, 2011. [9] I. J. Goodfellow, D. Warde-Farley, M. Mirza, A. Courville, and Y . Bengio. Maxout networks.arXiv:1302.4389, 2013. [10] K. He and J. Sun. Convolutional neural networks at con- strained time cost. arXiv:1412.1710, 2014. [11] K. He, X. Zhang, S. Ren, and J. Sun. Spatial pyramid pool- ing in deep convolutional networks for visual recognition. arXiv:1406.4729v2, 2014. [12] G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov. Improving neural networks by pre- venting co-adaptation of feature detectors. arXiv:1207.0580, 2012. [13] A. G. Howard. Some improvements on deep convolutional neural network based image classification.arXiv:1312.5402, 2013. [14] Y . Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Gir- shick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. arXiv:1408.5093, 2014. [15] A. Krizhevsky. One weird trick for parallelizing convolu- tional neural networks. arXiv:1404.5997, 2014. [16] A. Krizhevsky, I. Sutskever, and G. Hinton. Imagenet clas- sification with deep convolutional neural networks. In NIPS, 2012. [17] Y . LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Backpropagation applied to handwritten zip code recognition. Neural compu- tation, 1989. [18] C.-Y . Lee, S. Xie, P. Gallagher, Z. Zhang, and Z. Tu. Deeply- supervised nets. arXiv:1409.5185, 2014. [19] M. Lin, Q. Chen, and S. Yan. Network in network. arXiv:1312.4400, 2013. [20] A. L. Maas, A. Y . Hannun, and A. Y . Ng. Rectifier nonlin- earities improve neural network acoustic models. In ICML, 2013. [21] V . Nair and G. E. Hinton. Rectified linear units improve restricted boltzmann machines. In ICML, pages 807–814, 2010. [22] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge. arXiv:1409.0575, 2014. [23] A. M. Saxe, J. L. McClelland, and S. Ganguli. Exact so- lutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv:1312.6120, 2013. [24] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y . LeCun. Overfeat: Integrated recognition, localization and detection using convolutional networks. 2014. [25] K. Simonyan and A. Zisserman. Very deep con- volutional networks for large-scale image recognition. arXiv:1409.1556, 2014. [26] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, pages 1929–1958, 2014. [27] R. K. Srivastava, J. Masci, S. Kazerounian, F. Gomez, and
ℹ️ 离线版不含附件原文件，需要请到在线网站对应课时下载。
📎 资料
每日一图
- 📎 day04.png55.0KB
ℹ️ 离线版不含附件原文件，需要请到在线网站对应课时下载。
第5节 · 损失函数
🎯 掌握交叉熵损失、回归损失与 EMA。
📖 笔记
Day05 笔记：ANN 人工神经网络
Day 05 - ANN人工神经网络
学习目标
- 了解学习率衰减策略
- 掌握正则化方法
- 掌握神经网络训练的一般工作流
- 实现手机价格分类案例
# 导包
import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
import matplotlib.pyplot as plt
# 设置中文字体
plt.rcParams['font.sans-serif'] = ['SimHei'] # 微软雅黑
plt.rcParams['axes.unicode_minus'] = False # 解决负号显示问题
# 设置设备
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
2.学习率衰减策略
在训练神经网络时，一般情况下学习率都会随着训练而变化。主要是由于，在神经网络训练的后期，如果学习率过高，会造成loss的振荡，但是如果学习率减小的过慢，又会造成收敛变慢。
import torch
import matplotlib.pyplot as plt
def func(x_t):
return torch.pow(2*x_t, 2) # y = 4 x ^2
# 采用较小的学习率，梯度下降的速度慢
# 采用较大的学习率，梯度下降太快越过了最小值点，导致不收敛，甚至震荡
def demo01():
# 定义不同的学习率进行实验
learning_rates = [0.01, 0.1, 0.125, 0.2, 0.4]
max_iteration = 4
# 创建图像和子图
fig, axes = plt.subplots(2, len(learning_rates), figsize=(5 * len(learning_rates), 8))
for idx, lr in enumerate(learning_rates):
# 重置参数
x = torch.tensor([2.], requires_grad=True)
iter_rec, loss_rec, x_rec = list(), list(), list()
# 梯度下降过程
for i in range(max_iteration):
y = func(x)
y.backward()
x_rec.append(x.item())
x.data.sub_(lr * x.grad)
x.grad.zero_()
iter_rec.append(i)
loss_rec.append(y.item())
# 绘制损失曲线
axes[0, idx].plot(iter_rec, loss_rec, '-ro')
axes[0, idx].set_title(f'LR={lr}\nLoss Curve')
axes[0, idx].set_xlabel("Iteration")
axes[0, idx].set_ylabel("Loss")
axes[0, idx].grid(True)
# 绘制函数曲线和下降轨迹
x_t = torch.linspace(-3, 3, 100)
y_t = func(x_t)
axes[1, idx].plot(x_t.detach().numpy(), y_t.detach().numpy(), 'b-', alpha=0.7, label="y = 4*x²")
y_rec = [func(torch.tensor(i)).item() for i in x_rec]
axes[1, idx].plot(x_rec, y_rec, '-ro', label="Gradient Descent")
axes[1, idx].set_title(f'LR={lr}\nOptimization Path')
axes[1, idx].set_xlabel("x")
axes[1, idx].set_ylabel("y")
axes[1, idx].grid(True)
axes[1, idx].legend()
plt.tight_layout()
# plt.show()
# 测试
if __name__ == '__main__':
demo01()
C:\Users\angmi\AppData\Local\Temp\ipykernel_20328\4057500276.py:54: UserWarning: Glyph 178 (\N{SUPERSCRIPT TWO}) missing from font(s) SimHei.
plt.tight_layout()
C:\Users\angmi\AppData\Roaming\Python\Python313\site-packages\IPython\core\events.py:96: UserWarning: Glyph 178 (\N{SUPERSCRIPT TWO}) missing from font(s) SimHei.
func(*args, **kwargs)
C:\Users\angmi\AppData\Roaming\Python\Python313\site-packages\IPython\core\pylabtools.py:170: UserWarning: Glyph 178 (\N{SUPERSCRIPT TWO}) missing from font(s) SimHei.
fig.canvas.print_figure(bytes_io, **kw)
可以看出：采用较小的学习率，梯度下降的速度慢；采用较大的学习率，梯度下降太快越过了最小值点，导致震荡，甚至不收敛（梯度爆炸）。
等间隔学习率衰减 optim.lr_scheduler.StepLR
固定步长衰减
# 定义函数，演示 等间隔学习率衰减
def demo01():
# 1.定义变量，记录初始的 学习率，总训练轮数，每轮的批次数
lr = 0.1
epochs = 200
iteration = 10
# 2.创建数据集，y_true, x, w
y_true = torch.tensor([0], dtype=torch.float32)
# 输入特征
x = torch.tensor([1.0], dtype=torch.float32)
# 权重w，需要开启自动微分
w = torch.tensor([1.0], requires_grad=True, dtype=torch.float32)
# 3.创建优化器，动量法Momentum
# 参数1：待优化的模型参数，参数2：学习率，参数3：动量系数
optimizer = optim.SGD([w], lr=lr, momentum=0.9)
# 4.创建学习率衰减策略的对象
scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=50, gamma=0.5) # [0.1, 0.05, 0.025, 0.0125, ...]
# 5.记录 训练轮数 和 每轮训练用的学习率
lr_list = []
epoch_list = []
# 6.遍历训练轮数
for epoch in range(epochs):
# 7.获取 当前轮数 和 学习率，保存到记录列表中
epoch_list.append(epoch)
lr_list.append(scheduler.get_last_lr()[0])
# 8.遍历批次，开始训练
for batch in range(iteration):
# 9.前向传播，计算预测值
y_pred = w * x
# 10.计算损失值, 平方差
loss = (y_pred - y_true) ** 2
# 11.梯度清零
optimizer.zero_grad()
# 12.反向传播
loss.mean().backward()
# 13.更新参数
optimizer.step()
# 14.更新学习率
scheduler.step()
# 15.打印结果
print(f"lr_list: {lr_list}")
# 16.绘制 学习率lr-轮数epoch 折线图
plt.plot(epoch_list, lr_list)
plt.xlabel("epoch")
plt.ylabel("lr")
plt.title("等间隔学习率衰减")
plt.show()
demo01()
lr_list: [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125]
指定间隔学习率衰减 optim.lr_scheduler.MultiStepLR
指定步长衰减
# 定义函数，演示 指定间隔学习率衰减
def demo02():
# 1.定义变量，记录初始的 学习率，总训练轮数，每轮的批次数
lr = 0.1
epochs = 200
iteration = 10
# 2.创建数据集，y_true, x, w
y_true = torch.tensor([0], dtype=torch.float32)
# 输入特征
x = torch.tensor([1.0], dtype=torch.float32)
# 权重w，需要开启自动微分
w = torch.tensor([1.0], requires_grad=True, dtype=torch.float32)
# 3.创建优化器，动量法Momentum
# 参数1：待优化的模型参数，参数2：学习率，参数3：动量系数
optimizer = optim.SGD([w], lr=lr, momentum=0.9)
# 4.创建学习率衰减策略的对象
# 思路1：等间隔学习率衰减策略
# scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=50, gamma=0.5) # [0.1, 0.05, 0.025, 0.0125, ...]
# 思路2：指定间隔学习率衰减策略
milestones = [50, 125, 160]
scheduler = optim.lr_scheduler.MultiStepLR(optimizer, milestones=milestones, gamma=0.5)
# 5.记录 训练轮数 和 每轮训练用的学习率
lr_list = []
epoch_list = []
# 6.遍历训练轮数
for epoch in range(epochs):
# 7.获取 当前轮数 和 学习率，保存到记录列表中
epoch_list.append(epoch)
lr_list.append(scheduler.get_last_lr()[0])
# 8.遍历批次，开始训练
for batch in range(iteration):
# 9.前向传播，计算预测值
y_pred = w * x
# 10.计算损失值, 平方差
loss = (y_pred - y_true) ** 2
# 11.梯度清零
optimizer.zero_grad()
# 12.反向传播
loss.mean().backward()
# 13.更新参数
optimizer.step()
# 14.更新学习率
scheduler.step()
# 15.打印结果
print(f"lr_list: {lr_list}")
# 16.绘制 学习率lr-轮数epoch 折线图
plt.plot(epoch_list, lr_list)
plt.xlabel("epoch")
plt.ylabel("lr")
plt.title("指定间隔学习率衰减")
plt.show()
demo02()
lr_list: [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125, 0.0125]
指数学习率衰减 optim.lr_scheduler.ExponentialLR
按照指数衰减，本轮的学习率 = 上一轮的学习率 * 衰减系数
# 定义函数，演示 指数学习率衰减
def demo03():
# 1.定义变量，记录初始的 学习率，总训练轮数，每轮的批次数
lr = 0.1
epochs = 200
iteration = 10
# 2.创建数据集，y_true, x, w
y_true = torch.tensor([0], dtype=torch.float32)
# 输入特征
x = torch.tensor([1.0], dtype=torch.float32)
# 权重w，需要开启自动微分
w = torch.tensor([1.0], requires_grad=True, dtype=torch.float32)
# 3.创建优化器，动量法Momentum
# 参数1：待优化的模型参数，参数2：学习率，参数3：动量系数
optimizer = optim.SGD([w], lr=lr, momentum=0.9)
# 4.创建学习率衰减策略的对象
# 思路1：等间隔学习率衰减策略
# scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=50, gamma=0.5) # [0.1, 0.05, 0.025, 0.0125, ...]
# 思路2：指定间隔学习率衰减策略
# milestones = [50, 125, 160]
# scheduler = optim.lr_scheduler.MultiStepLR(optimizer, milestones=milestones, gamma=0.5)
# 思路3：指数学习率衰减策略
# 参数1：优化器对象，参数2：衰减因子
scheduler = optim.lr_scheduler.ExponentialLR(optimizer, gamma=0.95)
# 5.记录 训练轮数 和 每轮训练用的学习率
lr_list = []
epoch_list = []
# 6.遍历训练轮数
for epoch in range(epochs):
# 7.获取 当前轮数 和 学习率，保存到记录列表中
epoch_list.append(epoch)
lr_list.append(scheduler.get_last_lr()[0])
# 8.遍历批次，开始训练
for batch in range(iteration):
# 9.前向传播，计算预测值
y_pred = w * x
# 10.计算损失值, 平方差
loss = (y_pred - y_true) ** 2
# 11.梯度清零
optimizer.zero_grad()
# 12.反向传播
loss.mean().backward()
# 13.更新参数
optimizer.step()
# 14.更新学习率
scheduler.step()
# 15.打印结果
print(f"lr_list: {lr_list}")
# 16.绘制 学习率lr-轮数epoch 折线图
plt.plot(epoch_list, lr_list)
plt.xlabel("epoch")
plt.ylabel("lr")
plt.title("指数学习率衰减")
plt.show()
demo03()
lr_list: [0.1, 0.095, 0.09025, 0.0857375, 0.08145062499999998, 0.07737809374999999, 0.07350918906249998, 0.06983372960937498, 0.06634204312890622, 0.0630249409724609, 0.05987369392383786, 0.05688000922764597, 0.05403600876626367, 0.05133420832795048, 0.04876749791155295, 0.046329123015975304, 0.04401266686517654, 0.04181203352191771, 0.039721431845821824, 0.037735360253530734, 0.035848592240854196, 0.03405616262881148, 0.03235335449737091, 0.030735686772502362, 0.029198902433877242, 0.027738957312183378, 0.026352009446574207, 0.025034408974245494, 0.023782688525533217, 0.022593554099256556, 0.021463876394293726, 0.020390682574579037, 0.019371148445850084, 0.01840259102355758, 0.0174824614723797, 0.016608338398760712, 0.015777921478822676, 0.014989025404881541, 0.014239574134637464, 0.01352759542790559, 0.012851215656510309, 0.012208654873684792, 0.011598222130000552, 0.011018311023500524, 0.010467395472325497, 0.009944025698709221, 0.00944682441377376, 0.00897448319308507, 0.008525759033430816, 0.008099471081759275, 0.007694497527671311, 0.007309772651287745, 0.006944284018723357, 0.006597069817787189, 0.006267216326897829, 0.005953855510552938, 0.005656162735025291, 0.005373354598274026, 0.005104686868360324, 0.004849452524942308, 0.004606979898695193, 0.004376630903760433, 0.0041577993585724116, 0.0039499093906437905, 0.0037524139211116006, 0.0035647932250560204, 0.003386553563803219, 0.003217225885613058, 0.0030563645913324047, 0.0029035463617657843, 0.002758369043677495, 0.00262045059149362, 0.002489428061918939, 0.002364956658822992, 0.002246708825881842, 0.00213437338458775, 0.0020276547153583622, 0.001926271979590444, 0.0018299583806109217, 0.0017384604615803755, 0.0016515374385013568, 0.0015689605665762888, 0.0014905125382474742, 0.0014159869113351004, 0.0013451875657683454, 0.001277928187479928, 0.0012140317781059316, 0.001153330189200635, 0.0010956636797406032, 0.001040880495753573, 0.0009888364709658944, 0.0009393946474175996, 0.0008924249150467197, 0.0008478036692943836, 0.0008054134858296644, 0.0007651428115381812, 0.000726885670961272, 0.0006905413874132084, 0.0006560143180425479, 0.0006232136021404205, 0.0005920529220333994, 0.0005624502759317294, 0.0005343277621351429, 0.0005076113740283857, 0.00048223080532696635, 0.00045811926506061804, 0.0004352133018075871, 0.00041345263671720774, 0.00039278000488134735, 0.00037314100463728, 0.000354483954405416, 0.00033675975668514516, 0.0003199217688508879, 0.00030392568040834347, 0.0002887293963879263, 0.00027429292656852995, 0.00026057828024010345, 0.00024754936622809826, 0.00023517189791669334, 0.00022341330302085867, 0.00021224263786981574, 0.00020163050597632494, 0.0001915489806775087, 0.00018197153164363326, 0.0001728729550614516, 0.00016422930730837902, 0.00015601784194296006, 0.00014821694984581206, 0.00014080610235352146, 0.0001337657972358454, 0.0001270775073740531, 0.00012072363200535044, 0.00011468745040508291, 0.00010895307788482875, 0.00010350542399058731, 9.833015279105794e-05, 9.341364515150504e-05, 8.874296289392978e-05, 8.430581474923329e-05, 8.009052401177162e-05, 7.608599781118304e-05, 7.228169792062389e-05, 6.866761302459269e-05, 6.523423237336306e-05, 6.19725207546949e-05, 5.8873894716960144e-05, 5.5930199981112136e-05, 5.3133689982056524e-05, 5.0477005482953695e-05, 4.7953155208806006e-05, 4.55554974483657e-05, 4.327772257594741e-05, 4.111383644715004e-05, 3.9058144624792534e-05, 3.7105237393552906e-05, 3.524997552387526e-05, 3.34874767476815e-05, 3.1813102910297426e-05, 3.0222447764782554e-05, 2.8711325376543424e-05, 2.727575910771625e-05, 2.5911971152330435e-05, 2.461637259471391e-05, 2.3385553964978216e-05, 2.2216276266729303e-05, 2.1105462453392836e-05, 2.0050189330723194e-05, 1.9047679864187035e-05, 1.8095295870977683e-05, 1.71905310774288e-05, 1.6331004523557357e-05, 1.5514454297379488e-05, 1.4738731582510512e-05, 1.4001795003384986e-05, 1.3301705253215736e-05, 1.2636619990554949e-05, 1.2004788991027201e-05, 1.140454954147584e-05, 1.0834322064402047e-05, 1.0292605961181944e-05, 9.777975663122847e-06, 9.289076879966705e-06, 8.82462303596837e-06, 8.38339188416995e-06, 7.964222289961452e-06, 7.566011175463379e-06, 7.18771061669021e-06, 6.8283250858556995e-06, 6.4869088315629144e-06, 6.162563389984768e-06, 5.85443522048553e-06, 5.561713459461253e-06, 5.28362778648819e-06, 5.019446397163781e-06, 4.768474077305592e-06, 4.530050373440312e-06, 4.303547854768296e-06, 4.088370462029881e-06, 3.883951938928387e-06, 3.689754341981967e-06]
余弦退火学习率调度策略
- torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_0, T_mult=1, eta_min=0, last_epoch=-1)
- torch.optim.lr_scheduler.CosineAnnealingWarmRestarts(optimizer, T_0, T_mult=1, eta_min=0, last_epoch=-1)
余弦退火（Cosine Annealing）学习率调度公式：
\[
\eta_t = \eta_{\min} + \frac{1}{2} (\eta_{\max} - \eta_{\min})
\Bigg( 1 + \cos\Big(\frac{t \pi}{T_{\max}}\Big) \Bigg)
\]
其中：
- \(\eta_t\) ：第 \(t\) 步的学习率
- \(\eta_{\min}\) ：最小学习率
- \(\eta_{\max}\) ：初始/最大学习率
- \(T_{\max}\) ：一个周期的总步数
- \(t\) ：当前步数（或 epoch 数），\(0 \le t \le T_{\max}\)
# 演示 周期重启的余弦退火策略
# 导包
import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
import matplotlib.pyplot as plt
# 设置中文字体
plt.rcParams['font.sans-serif'] = ['SimHei'] # 微软雅黑
plt.rcParams['axes.unicode_minus'] = False # 解决负号显示问题
# 设置设备
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
# 1.定义变量，记录初始的 学习率，总训练轮数，每轮的批次数
lr = 0.1
epochs = 200
iteration = 10
# 2.创建数据集，y_true, x, w
y_true = torch.tensor([0], dtype=torch.float32)
# 输入特征
x = torch.tensor([1.0], dtype=torch.float32)
# 权重w，需要开启自动微分
w = torch.tensor([1.0], requires_grad=True, dtype=torch.float32)
# 3.创建优化器，动量法Momentum
# 参数1：待优化的模型参数，参数2：学习率，参数3：动量系数
optimizer = optim.SGD([w], lr=lr, momentum=0.9)
# 4.创建学习率衰减策略的对象
# T_0=50：第一个周期长度为50 epoch
# T_mult=1：周期增大的倍数（1 表示始终50 epoch；2 表示第一个周期为100，第二个周期为200）
# eta_min=0：学习率最小值
# last_epoch=-1：初始轮数，-1 表示从0开始
scheduler = optim.lr_scheduler.CosineAnnealingWarmRestarts(
optimizer,
T_0=50, # 核心：第一个周期的长度
T_mult=1, # 周期增大的倍数
eta_min=0, # 学习率最小值
)
# 5.记录 训练轮数 和 每轮训练用的学习率
lr_list = []
epoch_list = []
# 6.遍历训练轮数
for epoch in range(epochs):
# 7.获取 当前轮数 和 学习率，保存到记录列表中
epoch_list.append(epoch)
lr_list.append(scheduler.get_last_lr()[0])
# 8.遍历批次，开始训练
for batch in range(iteration):
# 9.前向传播，计算预测值
y_pred = w * x
# 10.计算损失值, 平方差
loss = (y_pred - y_true) ** 2
# 11.梯度清零
optimizer.zero_grad()
# 12.反向传播
loss.mean().backward()
# 13.更新参数
optimizer.step()
# 14.更新学习率
scheduler.step()
# 15.打印结果
print(f"lr_list: {lr_list}")
# 16.绘制 学习率lr-轮数epoch 折线图
# plt.figure(figsize=(10, 5))
plt.plot(epoch_list, lr_list)
plt.xlabel("epoch")
plt.ylabel("lr")
# plt.grid()
plt.title("周期重启的余弦退火策略")
plt.show()
lr_list: [0.1, 0.09990133642141358, 0.0996057350657239, 0.09911436253643445, 0.09842915805643156, 0.09755282581475769, 0.09648882429441258, 0.09524135262330098, 0.09381533400219318, 0.09221639627510075, 0.09045084971874738, 0.08852566213878947, 0.08644843137107058, 0.08422735529643444, 0.08187119948743449, 0.07938926261462367, 0.07679133974894983, 0.07408768370508577, 0.07128896457825364, 0.06840622763423392, 0.06545084971874737, 0.062434494358242734, 0.05936906572928624, 0.05626666167821523, 0.05313952597646568, 0.05, 0.04686047402353433, 0.0437333383217848, 0.04063093427071377, 0.03756550564175726, 0.03454915028125265, 0.03159377236576612, 0.02871103542174637, 0.02591231629491423, 0.02320866025105016, 0.02061073738537635, 0.018128800512565515, 0.015772644703565566, 0.013551568628929435, 0.011474337861210543, 0.009549150281252633, 0.007783603724899258, 0.006184665997806821, 0.004758647376699033, 0.0035111757055874327, 0.0024471741852423235, 0.0015708419435684518, 0.000885637463565564, 0.0003942649342761118, 9.866357858642206e-05, 0.1, 0.09990133642141358, 0.0996057350657239, 0.09911436253643445, 0.09842915805643156, 0.09755282581475769, 0.09648882429441258, 0.09524135262330098, 0.09381533400219318, 0.09221639627510075, 0.09045084971874738, 0.08852566213878947, 0.08644843137107058, 0.08422735529643444, 0.08187119948743449, 0.07938926261462367, 0.07679133974894983, 0.07408768370508577, 0.07128896457825364, 0.06840622763423392, 0.06545084971874737, 0.062434494358242734, 0.05936906572928624, 0.05626666167821523, 0.05313952597646568, 0.05, 0.04686047402353433, 0.0437333383217848, 0.04063093427071377, 0.03756550564175726, 0.03454915028125265, 0.03159377236576612, 0.02871103542174637, 0.02591231629491423, 0.02320866025105016, 0.02061073738537635, 0.018128800512565515, 0.015772644703565566, 0.013551568628929435, 0.011474337861210543, 0.009549150281252633, 0.007783603724899258, 0.006184665997806821, 0.004758647376699033, 0.0035111757055874327, 0.0024471741852423235, 0.0015708419435684518, 0.000885637463565564, 0.0003942649342761118, 9.866357858642206e-05, 0.1, 0.09990133642141358, 0.0996057350657239, 0.09911436253643445, 0.09842915805643156, 0.09755282581475769, 0.09648882429441258, 0.09524135262330098, 0.09381533400219318, 0.09221639627510075, 0.09045084971874738, 0.08852566213878947, 0.08644843137107058, 0.08422735529643444, 0.08187119948743449, 0.07938926261462367, 0.07679133974894983, 0.07408768370508577, 0.07128896457825364, 0.06840622763423392, 0.06545084971874737, 0.062434494358242734, 0.05936906572928624, 0.05626666167821523, 0.05313952597646568, 0.05, 0.04686047402353433, 0.0437333383217848, 0.04063093427071377, 0.03756550564175726, 0.03454915028125265, 0.03159377236576612, 0.02871103542174637, 0.02591231629491423, 0.02320866025105016, 0.02061073738537635, 0.018128800512565515, 0.015772644703565566, 0.013551568628929435, 0.011474337861210543, 0.009549150281252633, 0.007783603724899258, 0.006184665997806821, 0.004758647376699033, 0.0035111757055874327, 0.0024471741852423235, 0.0015708419435684518, 0.000885637463565564, 0.0003942649342761118, 9.866357858642206e-05, 0.1, 0.09990133642141358, 0.0996057350657239, 0.09911436253643445, 0.09842915805643156, 0.09755282581475769, 0.09648882429441258, 0.09524135262330098, 0.09381533400219318, 0.09221639627510075, 0.09045084971874738, 0.08852566213878947, 0.08644843137107058, 0.08422735529643444, 0.08187119948743449, 0.07938926261462367, 0.07679133974894983, 0.07408768370508577, 0.07128896457825364, 0.06840622763423392, 0.06545084971874737, 0.062434494358242734, 0.05936906572928624, 0.05626666167821523, 0.05313952597646568, 0.05, 0.04686047402353433, 0.0437333383217848, 0.04063093427071377, 0.03756550564175726, 0.03454915028125265, 0.03159377236576612, 0.02871103542174637, 0.02591231629491423, 0.02320866025105016, 0.02061073738537635, 0.018128800512565515, 0.015772644703565566, 0.013551568628929435, 0.011474337861210543, 0.009549150281252633, 0.007783603724899258, 0.006184665997806821, 0.004758647376699033, 0.0035111757055874327, 0.0024471741852423235, 0.0015708419435684518, 0.000885637463565564, 0.0003942649342761118, 9.866357858642206e-05]
| 方法 | 衰减方式 | 适用场景 | PyTorch API |
|---|---|---|---|
| 等间隔衰减 (Step Decay) | 固定步长衰减 | 大型数据集、简单任务 | torch.optim.lr_scheduler.StepLR |
| 指定间隔衰减 (MultiStep Decay) | 指定间隔衰减 | 需分段调整学习率的任务 | torch.optim.lr_scheduler.MultiStepLR |
| 指数衰减 (Exponential Decay) | 指数比例衰减 | 对训练平稳性要求高的任务 | torch.optim.lr_scheduler.ExponentialLR |
| 周期重启的余弦退火 (Cosine Annealing Warm Restarts) | 周期性余弦下降并重启 | 复杂非凸优化、易陷局部极小值的任务 | torch.optim.lr_scheduler.CosineAnnealingWarmRestarts |
3.正则化方法
什么是正则化
- 在设计机器学习算法时希望在新样本上的泛化能力强。许多机器学习算法都采用相关的策略来减小测试误差，这些策略被统称为正则化
- 神经网络强大的表示能力经常遇到过拟合，所以需要使用不同形式的正则化策略
- 目前在深度学习中使用较多的策略有范数惩罚，DropOut，特殊的网络层等，接下来我们对其进行详细的介绍
L1与L2正则化
L1和L2正则化通过在损失函数上添加惩罚项来实现。
| 内容 | L1 正则化（Lasso） | L2 正则化（Ridge / 权重衰减） |
|---|---|---|
| 公式 | \(loss = loss + λ \sum \|w\|\) | \(loss = loss + \frac{λ}{2} \sum w^2\) |
| 核心特点 | 稀疏解、特征选择、鲁棒性强 | 权重衰减、平滑解、稳定性高 |
| 适用场景 | 高维特征选择、模型压缩 | 深度学习防过拟合、数值稳定 |
Dropout正则化
在训练深层神经网络时，由于模型参数较多，在数据量不足的情况下，很容易过拟合。Dropout 随机失活 是一个简单有效的正则化方法。
- 在训练过程中，Dropout的实现是让神经元以超参数p（丢弃概率）的概率停止工作或者激活被置为0,未被置为0的进行缩放，缩放比例为1/(1-p)。训练过程可以认为是对完整的神经网络的一些子集进行训练，每次只更新子网络的参数
- 在实际应用中，Dropout参数丢弃概率p通常取值在0.2到0.5之间
- 对于较小的模型或较复杂的任务，丢弃概率p可以选择0.3或更小
- 对于非常深的网络，较大的丢弃概率p（如0.5或0.6）可能会有效防止过拟合
- 实际应用中，通常会在全连接层（激活函数后）之后添加Dropout层
- 在测试过程中，随机失活不起作用
- 在测试阶段，使用所有的神经元进行预测，以获得更稳定的结果
- 直接使用训练好的模型进行测试，由于所有的神经元都参与计算，输出的期望值会比训练阶段高。测试阶段的期望输出是 E[x_test] = x
- 测试/推理模式：==model.eval()==
- 缩放的必要性
- 在训练阶段，将参与计算的神经元的输出除以(1-p)
- 经过Dropout后的期望输出变为 E[x_dropout] = [(1-p) * x] / (1-p) = x，与测试阶段的期望输出一致
- 训练模型：==model.train()==
# 1.创建输入数据
t1 = torch.randint(0,10,size=(1,4))
# 转换为float类型
t1 = t1.to(dtype=torch.float32)
print(f"输入特征t1: {t1}")
# 2.进行下一层 加权求和 和 激活函数计算
linear1 = nn.Linear(4,10)
# 输入linear1，进行加权求和
t2 = linear1(t1)
print(f"linear1输出t2: {t2}")
# 激活函数
t3 = torch.relu(t2)
print(f"激活函数输出t3: {t3}")
# 3.对激活函数的输出t3 进行dropout随机失活
dropout = nn.Dropout(p=0.5)
# 输入dropout
t4 = dropout(t3)
print(f"dropout输出t4: {t4}")
输入特征t1: tensor([[9., 8., 1., 6.]])
linear1输出t2: tensor([[ 0.4872, -4.8918, 1.5698, 7.4191, -0.4140, -2.0614, 3.9062, -1.5255,
-2.3680, -8.5525]], grad_fn=<AddmmBackward0>)
激活函数输出t3: tensor([[0.4872, 0.0000, 1.5698, 7.4191, 0.0000, 0.0000, 3.9062, 0.0000, 0.0000,
0.0000]], grad_fn=<ReluBackward0>)
dropout输出t4: tensor([[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 7.8123, 0.0000, 0.0000,
0.0000]], grad_fn=<MulBackward0>)
批量归一正则化(Batch Normalization)
在神经网络的训练过程中，流经网络的数据都是一个batch，每个batch之间的数据分布变化非常剧烈，==这就使得网络参数频繁的进行大的调整以适应流经网络的不同分布的数据==，给模型训练带来非常大的不稳定性，使得模型难以收敛。如果我们对每一个batch的数据进行标准化之后，数据分布就变得稳定，参数的梯度变化也变得稳定，有助于加快模型的收敛。
通过标准化每一层的输入，使其均值接近0，方差接近1，从而加速训练并提高泛化能力。
先对数据标准化，再对数据重构（缩放+平移），写成公式如下所示：
$$
f(x) = \lambda \cdot \frac{x - \mathbb{E}(x)}{\sqrt{\mathrm{Var}(x)} + \epsilon} + \beta
$$
λ和β是可学习的参数，它相当于对标准化后的值做了一个线性变换，λ为系数，β为偏置；
eps 通常指为 1e-5，避免分母为 0；
E(x) 表示变量的均值；
Var(x) 表示变量的方差；
批量归一化的步骤如下：
批量归一化的作用：
-
减少内部协方差偏移： 通过对每层的输入进行标准化，减少了输入数据分布的变化，从而加速了训练过程，并使得网络在训练过程中更加稳定。
-
加速训练：
- 在没有批量归一化的情况下，神经网络的训练通常会很慢，尤其是深度网络。因为在每层的训练过程中，输入数据的分布（特别是前几层）会不断变化，这会导致网络学习速度缓慢。
-
批量归一化通过确保每层的输入数据在训练时分布稳定，有效减少了这种变化，从而加速了训练过程。
-
起到正则化作用： 批量归一化可以视作一种正则化方法，因为它引入了对训练样本的噪声（不同批次的统计信息不同，批次较小的均值和方差估计会更加不准确），使得模型不容易依赖特定的输入特征，从而起到一定的正则化效果，减少了对其他正则化技术（如Dropout）的需求。
-
提升泛化能力： 由于其正则化效果，批量归一化能帮助网络在测试集上取得更好的性能。
==批量归一化层在计算机视觉领域使用较多==
Batch Normalization 的使用步骤：
- 在网络层后添加 BN 层：
- 通常，BN 层会添加在卷积层 (Conv2d) 或全连接层 (Linear) 之后，激活函数之前。
- 例如：Conv2d -> BN -> ReLU 或者 Linear -> BN -> ReLU。 - 训练时： ==model.train()==
- BN 层会计算当前批次的均值 \(μ\) 和方差 \(σ²\)。
- 然后，利用这两个统计量对当前批次的数据进行规范化。
- 规范化后的数据会被缩放 \(γ\) 和平移 \(β\)。
- 同时，BN 层还会维护一个全局均值和全局方差的移动平均值，用于推理阶段。 - 推理时： ==model.eval()==
- 推理时，不会再使用当前批次的均值和方差，而是使用训练阶段计算的全局均值和全局方差。
- 同样，规范化后的数据会被缩放 \(γ\) 和平移 \(β\)。
"""
案例：
演示 批量归一化，属于正则化方法，缓解 模型过拟合
批量归一化：
先标准化为均值0，方差1的分布, 再进行 缩放(✖λ) 和 平移(➕β), 数据分布(均值β, 标准差λ)，找回一些数据信息
应用场景：
计算机视觉领域应用较多
BatchNorm1d：处理2D/3D数据, 主要应用于全连接层或处理一维数据的网络，例如文本处理。它接收形状为 (N, num_features) 的张量作为输入。
BatchNorm2d：处理4D数据, 为主要应用于卷积神经网络，处理二维图像数据或特征图。它接收形状为 (N, C, H, W) 的张量作为输入。
图片数据CHW，输入到CNN里面，需要转换为NCHW格式，批次大小N，通道C，高度H，宽度W
BatchNorm3d：处理5D数据, 主要用于三维卷积神经网络 (3D CNN)，处理三维数据，例如视频或医学图像。它接收形状为 (N, C, D, H, W) 的张量作为输入。
"""
# 导包
import torch
import torch.nn as nn
# 定义函数，演示 BatchNorm2d 批量归一化, 处理4D数据
def demo01():
# 1.创建图像样本数据
# 批次大小1，通道数2，高度3，宽度4
input_2d = torch.rand(size=(1,2,3,4))
print(f"input_2d: {input_2d}, shape: {input_2d.shape}")
# 2.创建 BatchNorm2d 层
# 参数1：输入特征数，也就是通道数；参数2：极小值；参数4：是否启用可学习的缩放和平移参数
# 参数3：用于计算运行统计量的动量参数
# running_mean = (1 - momentum) * running_mean + momentum * batch_mean
# running_var = (1 - momentum) * running_var + momentum * batch_var
bn_2d = nn.BatchNorm2d(num_features=2, eps=1e-5, momentum=0.1, affine=True)
# 3.输入数据 到 BatchNorm2d层
output = bn_2d(input_2d)
# 4.打印BatchNorm2d层输出
print(f"output: {output}, shape: {output.shape}")
# 定义函数，演示 BatchNorm1d 批量归一化, 处理3D/2D数据
def demo02():
# 1.创建样本数据
# 批次大小1，通道数2，长度3
input_1d = torch.rand(size=(1,2,3))
print(f"input_1d: {input_1d}, shape: {input_1d.shape}")
# 2.创建 BatchNorm2d 层
# 参数1：输入特征数，也就是通道数；参数2：极小值；参数4：是否启用可学习的缩放和平移参数
# 参数3：用于计算运行统计量的动量参数
# running_mean = (1 - momentum) * running_mean + momentum * batch_mean
# running_var = (1 - momentum) * running_var + momentum * batch_var
# bn_2d = nn.BatchNorm2d(num_features=2, eps=1e-5, momentum=0.1, affine=True)
# 创建 BatchNorm1d 层
bn_1d = nn.BatchNorm1d(num_features=2, eps=1e-5, momentum=0.1, affine=True)
# 3.输入数据 到 BatchNorm2d层
output = bn_1d(input_1d)
# 4.打印BatchNorm2d层输出
print(f"output: {output}, shape: {output.shape}")
# 测试
if __name__ == "__main__":
demo01()
demo02()
input_2d: tensor([[[[0.3020, 0.4624, 0.2669, 0.6466],
[0.4142, 0.2483, 0.6573, 0.9378],
[0.1171, 0.7750, 0.9441, 0.3912]],
[[0.1435, 0.9811, 0.7332, 0.5805],
[0.1117, 0.3462, 0.6789, 0.6437],
[0.4877, 0.6269, 0.6914, 0.9162]]]]), shape: torch.Size([1, 2, 3, 4])
output: tensor([[[[-0.8033, -0.1941, -0.9364, 0.5049],
[-0.3774, -1.0070, 0.5457, 1.6102],
[-1.5049, 0.9925, 1.6343, -0.4644]],
[[-1.6857, 1.5606, 0.6000, 0.0080],
[-1.8087, -0.8999, 0.3894, 0.2530],
[-0.3516, 0.1878, 0.4379, 1.3090]]]],
grad_fn=<NativeBatchNormBackward0>), shape: torch.Size([1, 2, 3, 4])
input_1d: tensor([[[0.8270, 0.7527, 0.6871],
[0.4526, 0.0340, 0.1730]]]), shape: torch.Size([1, 2, 3])
output: tensor([[[ 1.2475, -0.0509, -1.1967],
[ 1.3367, -1.0676, -0.2691]]], grad_fn=<NativeBatchNormBackward0>), shape: torch.Size([1, 2, 3])
4. ANN手机价格分类案例
小明创办了一家手机公司，他不知道如何估算手机产品的价格。为了解决这个问题，他收集了多家公司的手机销售数据。该数据为二手手机的各个性能的数据，最后根据这些性能得到4个价格区间，作为这些二手手机售出的价格区间。主要包括：
我们需要帮助小明找出手机的功能 与 其售价 之间的某种关系。可以使用机器学习的方法，也可以构建一个全连接的神经网络。
需要注意的是: 在这个问题中，不需要预测实际价格，而是预测一个价格范围，使用 0、1、2、3 来表示，所以该问题是一个分类问题。
神经网络训练的一般工作流：
- 1.准备数据集
- 2.搭建神经网络模型
- 3.模型训练
1. 前向传播：\(y\_pred = model(X)\)
2. 计算损失：\(loss = loss\_fn(y\_pred, y)\)
3. 梯度清零：\(optimizer.zero\_grad()\)
4. 反向传播：\(loss.backward()\)
5. 更新参数：\(optimizer.step()\)
- 4.模型测试
import torch # pytorch框架
from torch.utils.data import TensorDataset # 数据集对象
from torch.utils.data import DataLoader # 数据加载器
import torch.nn as nn # 神经网络层的操作
import torch.optim as optim # 优化器
from sklearn.model_selection import train_test_split # 拆分 训练集 和 测试集
import matplotlib.pyplot as plt # 绘图
import numpy as np # 数组操作
import pandas as pd # 数据处理
import time # 时间模块
from torchsummary import summary
device = "cuda" if torch.cuda.is_available() else "cpu"
# 1.准备数据集
def create_dataset():
# 1.加载CSV数据集
data = pd.read_csv("./data/手机价格预测.csv")
# 2.获取特征x 和 标签y
x = data.iloc[:, :-1]
y = data.iloc[:, -1]
# print(f"x: {x.head()}")
# print(f"y: {y.head()}")
# 3.特征列 转换为 float
x = x.astype(np.float32)
# print(f"x: {x}")
# 4.拆分数据集
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=6, stratify=y)
# 5.把数据集转换为张量, numpy数组 -> 张量 -> 数据集对象TensorDataset -> 数据加载器DataLoader
train_dataset = TensorDataset(torch.tensor(x_train.values), torch.tensor(y_train.values))
test_dataset = TensorDataset(torch.tensor(x_test.values), torch.tensor(y_test.values))
# 6.返回结果
return train_dataset, test_dataset, x_train.shape[1], len(y_train.unique())
# 2.搭建神经网络
class PhonePriceClassifier(nn.Module):
# 1.定义__init__方法, 初始化父类成员，搭建神经网络
def __init__(self, input_dim, output_dim):
# 初始化父类成员
super().__init__()
# 搭建神经网络
self.linear1 = nn.Linear(input_dim, 128)
self.linear2 = nn.Linear(128, 256)
self.linear3 = nn.Linear(256, output_dim)
# 2.定义forward方法，实现前向传播
def forward(self, x):
# 隐藏层1
x = torch.relu(self.linear1(x))
# 隐藏层2
x = torch.relu(self.linear2(x))
# 输出层
x = self.linear3(x)
# 返回结果
return x
# 3.模型训练
def train(train_dataset, input_dim, output_dim):
# 1.创建数据加载器DataLoader
# 参数1：数据集对象；参数2：批量大小；参数3：是否打乱数据
train_loader = DataLoader(train_dataset, batch_size=16, shuffle=True)
# 2.创建神经网络模型
model = PhonePriceClassifier(input_dim, output_dim)
model.to(device)
# 3.定义损失函数
loss_fn = nn.CrossEntropyLoss()
# 4.创建优化器对象
optimizer = optim.SGD(model.parameters(), lr=1e-3)
# 5.模型训练
# 定义变量，记录 训练总轮数
epochs = 50
# 遍历轮数
for epoch in range(epochs):
# 记录 每次训练的 损失值，训练样本数
total_loss = 0.
total_sample = 0
# 记录训练开始时间
start_time = time.time()
# 遍历批次
for x, y in train_loader:
# 训练模式
model.train()
# 移动数据到GPU
x = x.to(device)
y = y.to(device)
# 前向传播
y_pred = model(x)
# 计算损失值
loss = loss_fn(y_pred, y)
# 更新总损失
batch_size = x.shape[0]
total_loss += loss.item()*batch_size
# 更新总样本数
total_sample += batch_size
# 梯度清零
optimizer.zero_grad()
# 反向传播
loss.backward()
# 更新参数
optimizer.step()
# 打印训练信息
print(f"epoch: {epoch}, loss: {total_loss / total_sample}, time: {time.time() - start_time:.2f}")
# 6.保存模型参数
# print(f"\n\n模型的参数信息: {model.state_dict()}\n\n")
torch.save(model.state_dict(), "./model/phone_price_classifier.pth") # 模型文件，后缀名 .pth, .pkl, .pickle 均可
# 4.模型测试
@torch.no_grad()
def evaluate(test_dataset, input_dim, output_dim):
# 1. 创建神经网络分类对象.
model = PhonePriceClassifier(input_dim, output_dim)
model.to(device)
# 2.加载模型参数
model.load_state_dict(torch.load('./model/phone_price_classifier.pth'))
# 3.创建数据加载器DataLoader
# 参数1：数据集对象；参数2：批量大小；参数3：是否打乱数据
test_loader = DataLoader(test_dataset, batch_size=16, shuffle=False)
# 4.记录正确样本的数量
correct_sample = 0
total_sample = 0
# 5.开始测试, 遍历数据加载器的批次数据
for x, y in test_loader:
# 切换到测试模式
model.eval()
# 张量 迁移 到 GPU
x = x.to(device)
y = y.to(device)
# 6.前向传播，模型预测
y_pred = model(x) # 形状(batch_size, 4), 每个类别的输出logits
# print(f"预测结果y_pred：{y_pred}")
# 7.计算预测正确的数量
correct = y_pred.argmax(dim=1) == y
correct_sample += correct.sum().item()
total_sample += y.size(0)
print(f"测试集准确率：{correct_sample / total_sample}")
# 5.测试
if __name__ == '__main__':
# 1.准备数据集
train_dataset, test_dataset, input_dim, output_dim = create_dataset()
# 打印
print(f"train_dataset: {train_dataset}")
print(f"test_dataset: {test_dataset}")
print(f"input_dim: {input_dim}")
print(f"output_dim: {output_dim}")
# 2.构建神经网络
model = PhonePriceClassifier(input_dim, output_dim)
# 3.统计模型参数
model.to(device)
# summary(model,input_size=(input_dim,))
# 4.模型训练
train(train_dataset, input_dim, output_dim)
# 5.模型测试
evaluate(test_dataset, input_dim, output_dim)
train_dataset: <torch.utils.data.dataset.TensorDataset object at 0x00000197BA052BA0>
test_dataset: <torch.utils.data.dataset.TensorDataset object at 0x00000197F9C17610>
input_dim: 20
output_dim: 4
epoch: 0, loss: 10.34764018535614, time: 0.22
epoch: 1, loss: 0.8868235325813294, time: 0.10
epoch: 2, loss: 0.8461017781496047, time: 0.10
epoch: 3, loss: 0.8364109423756599, time: 0.10
epoch: 4, loss: 0.8181111803650856, time: 0.13
epoch: 5, loss: 0.8058439743518829, time: 0.10
epoch: 6, loss: 0.8025904560089111, time: 0.10
epoch: 7, loss: 0.8014548885822296, time: 0.10
epoch: 8, loss: 0.779120919406414, time: 0.12
epoch: 9, loss: 0.7771800068020821, time: 0.11
epoch: 10, loss: 0.7424208781123162, time: 0.10
epoch: 11, loss: 0.7729402646422386, time: 0.12
epoch: 12, loss: 0.7419938072562218, time: 0.12
epoch: 13, loss: 0.7461637026071548, time: 0.10
epoch: 14, loss: 0.7451482152938843, time: 0.10
epoch: 15, loss: 0.7622780457139016, time: 0.10
epoch: 16, loss: 0.7363531678915024, time: 0.11
epoch: 17, loss: 0.7171687591075897, time: 0.10
epoch: 18, loss: 0.7375901317596436, time: 0.10
epoch: 19, loss: 0.7328856790065765, time: 0.10
epoch: 20, loss: 0.7176006978750229, time: 0.10
epoch: 21, loss: 0.7251139596104622, time: 0.10
epoch: 22, loss: 0.7137130743265152, time: 0.10
epoch: 23, loss: 0.7312264454364776, time: 0.10
epoch: 24, loss: 0.7118452110886574, time: 0.10
epoch: 25, loss: 0.7094027370214462, time: 0.10
epoch: 26, loss: 0.690052843093872, time: 0.10
epoch: 27, loss: 0.6843650743365288, time: 0.10
epoch: 28, loss: 0.6829788282513618, time: 0.10
epoch: 29, loss: 0.6869563218951226, time: 0.10
epoch: 30, loss: 0.6813377672433854, time: 0.10
epoch: 31, loss: 0.6886233869194984, time: 0.11
epoch: 32, loss: 0.7100735640525818, time: 0.11
epoch: 33, loss: 0.6711059054732322, time: 0.10
epoch: 34, loss: 0.7061515259742737, time: 0.10
epoch: 35, loss: 0.6886743503808975, time: 0.10
epoch: 36, loss: 0.6809764394164085, time: 0.10
epoch: 37, loss: 0.6769312492012978, time: 0.10
epoch: 38, loss: 0.6803314900398254, time: 0.10
epoch: 39, loss: 0.6642144033312798, time: 0.10
epoch: 40, loss: 0.6802034932374954, time: 0.10
epoch: 41, loss: 0.6694290074706077, time: 0.10
epoch: 42, loss: 0.683846652507782, time: 0.10
epoch: 43, loss: 0.6785068655014038, time: 0.10
epoch: 44, loss: 0.6693502110242844, time: 0.10
epoch: 45, loss: 0.6568958036601543, time: 0.10
epoch: 46, loss: 0.6516988828778267, time: 0.10
epoch: 47, loss: 0.6618712189793586, time: 0.11
epoch: 48, loss: 0.6546808457374573, time: 0.10
epoch: 49, loss: 0.684197007715702, time: 0.10
测试集准确率：0.59
- 模型优化
1. 对输入数据进行标准化
2. 调整优化方法
3. 调整学习率
4. 增加批量归一化层
5. 增加网络层数、神经元个数
6. 增加训练轮数
7. 使用GPU加速训练
8. 等等...
import torch # pytorch框架
from torch.utils.data import TensorDataset # 数据集对象
from torch.utils.data import DataLoader # 数据加载器
import torch.nn as nn # 神经网络层的操作
import torch.optim as optim # 优化器
from sklearn.model_selection import train_test_split # 拆分 训练集 和 测试集
import matplotlib.pyplot as plt # 绘图
import numpy as np # 数组操作
import pandas as pd # 数据处理
import time # 时间模块
from torchsummary import summary
from sklearn.preprocessing import StandardScaler
device = "cuda" if torch.cuda.is_available() else "cpu"
print(f"使用设备：{device}")
# 1.准备数据集
def create_dataset():
# 1.加载CSV数据集
data = pd.read_csv("./data/手机价格预测.csv")
# 2.获取特征x 和 标签y
x = data.iloc[:, :-1]
y = data.iloc[:, -1]
# print(f"x: {x.head()}")
# print(f"y: {y.head()}")
# 3.特征列 转换为 float
x = x.astype(np.float32)
# print(f"x: {x}")
# 4.拆分数据集
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=6, stratify=y)
# 优化1：数据标准化
transfer = StandardScaler()
x_train = transfer.fit_transform(x_train)
x_test = transfer.transform(x_test)
# 5.把数据集转换为张量, numpy数组 -> 张量 -> 数据集对象TensorDataset -> 数据加载器DataLoader
train_dataset = TensorDataset(torch.tensor(x_train), torch.tensor(y_train.values))
test_dataset = TensorDataset(torch.tensor(x_test), torch.tensor(y_test.values))
# 6.返回结果
return train_dataset, test_dataset, x_train.shape[1], len(y_train.unique())
# 2.搭建神经网络
class PhonePriceClassifier(nn.Module):
# 1.定义__init__方法, 初始化父类成员，搭建神经网络
def __init__(self, input_dim, output_dim):
# 初始化父类成员
super().__init__()
# 搭建神经网络
# 优化2：增加网络深度
self.linear1 = nn.Linear(input_dim, 512)
# self.linear2 = nn.Linear(512, 512)
self.linear3 = nn.Linear(512, 256)
self.linear4 = nn.Linear(256, 256)
self.linear5 = nn.Linear(256, output_dim)
self.dropout = nn.Dropout(0.1)
# 2.定义forward方法，实现前向传播
def forward(self, x):
# 隐藏层1
x = torch.relu(self.linear1(x))
# 隐藏层2
# x = torch.relu(self.linear2(x))
x = torch.relu(self.linear3(x))
x = torch.relu(self.linear4(x))
x = self.dropout(x)
# 输出层
x = self.linear5(x)
# 返回结果
return x
# 3.模型训练
def train(train_dataset, input_dim, output_dim):
# 1.创建数据加载器DataLoader
# 参数1：数据集对象；参数2：批量大小；参数3：是否打乱数据
train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)
# 2.创建神经网络模型
model = PhonePriceClassifier(input_dim, output_dim)
model.to(device)
# 3.定义损失函数
loss_fn = nn.CrossEntropyLoss()
# 4.创建优化器对象
# 优化3：使用Adam优化方法
optimizer = optim.Adam(model.parameters(), lr=1e-4)
# 优化4：添加学习率调度，optim.lr_scheduler.CosineAnnealingWarmRestarts
scheduler = optim.lr_scheduler.CosineAnnealingWarmRestarts(optimizer, T_0=10, T_mult=1)
# 5.模型训练
# 定义变量，记录 训练总轮数
epochs = 50
# 遍历轮数
for epoch in range(epochs):
# 记录 每次训练的 损失值，训练样本数
total_loss = 0.
total_sample = 0
# 记录训练开始时间
start_time = time.time()
# 遍历批次
for x, y in train_loader:
# 训练模式
model.train()
# 移动数据到GPU
x = x.to(device)
y = y.to(device)
# 前向传播
y_pred = model(x)
# 计算损失值
loss = loss_fn(y_pred, y)
# 更新总损失
batch_size = x.shape[0]
total_loss += loss.item()*batch_size
# 更新总样本数
total_sample += batch_size
# 梯度清零
optimizer.zero_grad()
# 反向传播
loss.backward()
# 更新参数
optimizer.step()
scheduler.step()
# 打印训练信息
print(f"epoch: {epoch}, loss: {total_loss / total_sample}, time: {time.time() - start_time:.2f}")
# 6.保存模型参数
# print(f"\n\n模型的参数信息: {model.state_dict()}\n\n")
torch.save(model.state_dict(), "./model/phone_price_classifier.pth") # 模型文件，后缀名 .pth, .pkl, .pickle 均可
# 4.模型测试
@torch.no_grad()
def evaluate(test_dataset, input_dim, output_dim):
# 1. 创建神经网络分类对象.
model = PhonePriceClassifier(input_dim, output_dim)
model.to(device)
# 2.加载模型参数
model.load_state_dict(torch.load('./model/phone_price_classifier.pth'))
# 3.创建数据加载器DataLoader
# 参数1：数据集对象；参数2：批量大小；参数3：是否打乱数据
test_loader = DataLoader(test_dataset, batch_size=32, shuffle=False)
# 4.记录正确样本的数量
correct_sample = 0
total_sample = 0
# 5.开始测试, 遍历数据加载器的批次数据
for x, y in test_loader:
# 切换到测试模式
model.eval()
# 张量 迁移 到 GPU
x = x.to(device)
y = y.to(device)
# 6.前向传播，模型预测
y_pred = model(x) # 形状(batch_size, 4), 每个类别的输出logits
# print(f"预测结果y_pred：{y_pred}")
# 7.计算预测正确的数量
correct = y_pred.argmax(dim=1) == y
correct_sample += correct.sum().item()
total_sample += y.size(0)
print(f"测试集准确率：{correct_sample / total_sample}")
# 5.测试
if __name__ == '__main__':
# 1.准备数据集
train_dataset, test_dataset, input_dim, output_dim = create_dataset()
# 打印
print(f"train_dataset: {train_dataset}")
print(f"test_dataset: {test_dataset}")
print(f"input_dim: {input_dim}")
print(f"output_dim: {output_dim}")
# 2.构建神经网络
model = PhonePriceClassifier(input_dim, output_dim)
# 3.统计模型参数
model.to(device)
# summary(model,input_size=(input_dim,))
# 4.模型训练
train(train_dataset, input_dim, output_dim)
# 5.模型测试
evaluate(test_dataset, input_dim, output_dim)
使用设备：cuda
train_dataset: <torch.utils.data.dataset.TensorDataset object at 0x00000197F9C17890>
test_dataset: <torch.utils.data.dataset.TensorDataset object at 0x00000197F9CF4C30>
input_dim: 20
output_dim: 4
epoch: 0, loss: 1.3706597256660462, time: 0.11
epoch: 1, loss: 1.3022879910469056, time: 0.07
epoch: 2, loss: 1.1458162045478821, time: 0.08
epoch: 3, loss: 0.9078893768787384, time: 0.07
epoch: 4, loss: 0.6967931640148163, time: 0.08
epoch: 5, loss: 0.5534972769021987, time: 0.09
epoch: 6, loss: 0.4496357005834579, time: 0.08
epoch: 7, loss: 0.37318617820739747, time: 0.07
epoch: 8, loss: 0.32060630530118944, time: 0.08
epoch: 9, loss: 0.2785828632116318, time: 0.08
epoch: 10, loss: 0.2471187022328377, time: 0.07
epoch: 11, loss: 0.22411519527435303, time: 0.08
epoch: 12, loss: 0.20272956281900406, time: 0.07
epoch: 13, loss: 0.18643852710723877, time: 0.07
epoch: 14, loss: 0.17427322000265122, time: 0.07
epoch: 15, loss: 0.157532174885273, time: 0.07
epoch: 16, loss: 0.14617112323641776, time: 0.07
epoch: 17, loss: 0.14078537061810492, time: 0.08
epoch: 18, loss: 0.12825253903865813, time: 0.08
epoch: 19, loss: 0.11951646484434604, time: 0.07
epoch: 20, loss: 0.11523462533950805, time: 0.08
epoch: 21, loss: 0.10727430664002896, time: 0.07
epoch: 22, loss: 0.10047829926013946, time: 0.08
epoch: 23, loss: 0.09744384035468101, time: 0.07
epoch: 24, loss: 0.09036574810743332, time: 0.07
epoch: 25, loss: 0.08763951905071736, time: 0.08
epoch: 26, loss: 0.08140164084732532, time: 0.07
epoch: 27, loss: 0.07587030805647373, time: 0.07
epoch: 28, loss: 0.0731380008906126, time: 0.08
epoch: 29, loss: 0.07344199813902379, time: 0.08
epoch: 30, loss: 0.06597149554640054, time: 0.08
epoch: 31, loss: 0.06322498302906751, time: 0.07
epoch: 32, loss: 0.06085667755454779, time: 0.08
epoch: 33, loss: 0.06113092955201864, time: 0.08
epoch: 34, loss: 0.057474353462457654, time: 0.08
epoch: 35, loss: 0.05293474618345499, time: 0.08
epoch: 36, loss: 0.05207266837358475, time: 0.09
epoch: 37, loss: 0.04969668610021472, time: 0.09
epoch: 38, loss: 0.04556395657360554, time: 0.09
epoch: 39, loss: 0.04432915959507227, time: 0.07
epoch: 40, loss: 0.04232580278068781, time: 0.08
epoch: 41, loss: 0.0396079907566309, time: 0.08
epoch: 42, loss: 0.03937803667970002, time: 0.09
epoch: 43, loss: 0.03654556540772319, time: 0.07
epoch: 44, loss: 0.033677131067961455, time: 0.07
epoch: 45, loss: 0.03449654405936599, time: 0.07
epoch: 46, loss: 0.030163878463208674, time: 0.08
epoch: 47, loss: 0.029301316421478988, time: 0.07
epoch: 48, loss: 0.02753810239955783, time: 0.08
epoch: 49, loss: 0.02798120280727744, time: 0.08
测试集准确率：0.9175
常见面试题
1. 常见的学习率衰减策略 或 学习率调度策略 有哪些？
2. 常用的正则化方法有哪些？
3. 简述dropout的计算原理？
4. dropout训练时为什么要将输出/(1-p)？
5. 简述BN批量归一化的计算原理？
6. 简述神经网络训练的一般工作流？
7. 神经网络训练中，如何避免或缓解梯度爆炸或者梯度消失？
1. 常见的学习率衰减策略 或 学习率调度策略 有哪些？
- 等间隔衰减：每隔固定训练轮次，将学习率乘以一个固定比例
- 余弦退火：学习率按余弦曲线平滑下降，可周期性重启来跳出局部最优解。
- 自适应学习率：根据梯度自动调整学习率，如 Adam、RMSProp
- 其他
- 指定间隔衰减：指定间隔衰减学习率
- 指数衰减：学习率按指数连续衰减
2. 常用的正则化方法有哪些？
- L1/L2正则化：L1惩罚权重绝对值，L2惩罚权重的平方。
- dropout：训练时随机失活神经元，防止过拟合。
- 批量归一化：标准化输入数据，减小数据分布偏差。
3. 简述dropout的计算原理？
- 训练时，dropout让神经元以随机概率p 随机失活，将输出置为0, 其余输出 /(1-p)。
- 测试时，dropout随机失活不起作用。
4. dropout训练时为什么要将输出/(1-p)？
- Dropout以概率p让神经元随机失活，将输出置为0，会使得输出的期望变为原始期望*(1-p)，
- 为了保持 训练 和 预测 时的数值一致性，Dropout之后需要/(1-p)，缩放为原始期望
5. 简述BN批量归一化的计算原理？
批量归一化在训练时，先对数据标准化，再进行 缩放+平移，从而减小数据分布的偏差。
- 训练时，BN层先将数据标准化为均值0，方差1的分布，然后进行缩放和平移，同时还会维护一个全局均值和全局方差，用于测试时进行归一化。
- 测试时，BN层使用训练时的全局均值和方差先对数据进行标准化，然后使用训练时的缩放和平移参数进行缩放和平移。
- 缩放参数和平移参数是在训练过程中学习的。
6. 简述神经网络训练的一般工作流？
- 1.准备数据集
- 2.搭建神经网络模型
- 3.模型训练
- 前向传播：\(\hat{y} = model(X)\)
- 计算损失：\(loss = loss\_fn(\hat{y}, y)\)
- 梯度清零：\(optimizer.zero\_grad()\)
- 反向传播：\(loss.backward()\)
- 更新参数：\(optimizer.step()\)
- 4.模型测试
7. 神经网络训练中，如何避免或缓解梯度爆炸或者梯度消失？
-
梯度爆炸
-
- 降低学习率
-
- 合理参数初始化方法（Kaiming / Xavier）
-
- 使用稳定激活函数（ReLU / Leaky ReLU）
-
- 使用批归一化（BatchNorm）改善输入分布
-
- 梯度裁剪
-
-
梯度消失
-
- 非饱和激活函数（ReLU）
-
- 合理参数初始化方法（Kaiming / Xavier）
-
- 批量归一化（BatchNorm）
-
- 残差或跳跃连接（ResNet / DenseNet）
-
代码作业
- 1.使用两层dropout处理数据。输入数据形状为(2,4),隐藏层形状为(4,20),后接dropout(p=0.5),输出层形状为(20,10),得到输出结果，并打印。注意通常输出层之后不加dropout
- 2.修改手机价格分类案例代码。采用周期启动的余弦退火学习率调度策略torch.optim.lr_scheduler.CosineAnnealingWarmRestarts来调整学习率，训练模型，并打印最终的测试准确率。
- 3.修改手机价格分类案例代码。模型中添加dropout层，优化器改为AdamW, 采用周期启动的余弦退火学习率调度策略torch.optim.lr_scheduler.CosineAnnealingWarmRestarts来调整学习率，训练模型，并打印最终的测试准确率。注意dropout的p应该小于0.1，因为数据集较小。
💻 代码
Day05 代码：损失函数 / EMA
4 个文件📄 01_多分类交叉熵损失函数.py 2.0KB
""" 演示 多分类交叉熵损失函数 介绍 损失函数 也叫 成本函数/目标函数/代价函数/误差函数 用来 衡量预测值和真实值的差距，从而衡量模型好坏，损失越小模型越好 多分类交叉熵损失 公式：CrossEntropLoss = -Σy*log(softmax(y_logits)) 其中：y_logits是模型的原始输出/预测分数 softmax(y_logits)是模型输出的概率分布，也就是每个类别的预测概率，最大概率对应预测类别 注意： 多分类交叉熵CrossEntropLoss里面有softmax运算，所以模型的输出层不需要softmax。 但是如果要获取预测概率，则需要在模型输出的原始预测分数后面经过softmax 应用： 多分类问题，如：10个类别，预测图片属于哪一类 API: nn.CrossEntropyLoss() 需要掌握： loss_fn = nn.CrossEntropLoss() loss = loss_fn(y_logits, y_true) """ # 导包 import torch import torch.nn as nn # 1.定义函数，演示 多分类交叉熵损失函数 # 注意：1.多分类交叉熵损失函数里面有softmax运算 # 2.多分类交叉熵nn.CrossEntropLoss支持真实值的one-hot编码和标签编码(默认) def demo01(): # 1.创建 样本的真实值，假设是一个三分类任务, 真实类别为[1,0,2,0]，4个样本 # 使用标签编码,1D y_true = torch.tensor([1,0,2,0], dtype=torch.long) # 形状(batch_size,) # 2.创建 模型的预测分数y_logits, 模拟的 # (4,3) y_logits = torch.tensor([[0.1,2.0,1.0],[0.2,0.5,0.3],[0.3,0.2,0.5],[0.5,0.2,0.3]]) # (batch_size, 3) # 3.创建 多分类交叉熵损失函数对象 loss_fn = nn.CrossEntropyLoss() # 4.计算损失 # y_logits: 2D(4, 3), y_true: 1D(4,) # print(f"y_logits.shape: {y_logits.shape}, y_true.shape: {y_true.shape}") loss = loss_fn(y_logits, y_true) print(f"多分类交叉熵损失函数：{loss}") # 测试 if __name__ == '__main__': demo01()
📄 02_二分类交叉熵损失函数.py 2.8KB
""" 演示 二分类交叉熵损失函数 介绍 损失函数 也叫 成本函数/目标函数/代价函数/误差函数，用来 衡量模型好坏，也就是模型预测表现 分类问题 多分类:多分类交叉熵损失函数，nn.CrossEntropLoss 二分类:二分类交叉熵，nn.BCELoss; 或 多分类交叉熵损失函数，nn.CrossEntropLoss(建议使用) 多分类交叉熵损失 公式：CrossEntropLoss = -Σylog(softmax(y_logits)) 其中：y_logits是模型的原始输出/预测分数 softmax(y_logits)是模型输出的概率分布，也就是每个类别的预测概率，最大概率对应预测类别 注意： 多分类交叉熵CrossEntropLoss里面有softmax运算，所以模型的输出层不需要softmax。 但是如果要获取预测概率，则需要在模型输出的原始预测分数后面经过softmax 应用： 多分类问题，如：10个类别，预测图片属于哪一类 API: nn.CrossEntropyLoss() 二分类交叉熵损失 公式：BCELoss = -y*log(y_hat)-(1-y)*log(1-y_hat) 其中：y_hat是模型的预测值，也就是经过sigmoid之后的预测概率，对应正类 y 是真实值，0或1， 负类：标签编码0，one-hot编码[1,0],正类：标签编码1，one-hot[0,1] 注意： 二分类交叉熵损失BCELoss里面没有sigmoid，所以模型输出层最后需要添加sigmoid API: nn.BCELoss() 需要掌握： loss_fn = nn.CrossEntropLoss loss = loss_fn(y_logits, y_true) 二分类任务可以用多分类交叉熵损失函数CrossEntropLoss(建议使用) / 二分类交叉熵损失BCELoss """ # 导包 import torch from torch import nn # 1.定义函数，演示 二分类交叉熵损失函数，注意sigmoid要添加到模型输出层 # 二分类任务,通常1为正类，0为负类 def demo01(): # 1.创建 真实值, 只能标签编码 # 1D:(batch_size,) y_true = torch.tensor([0, 1, 0, 1], dtype=torch.float32) # 2.创建 预测分数，1D:(batch_size,) y_logits = torch.tensor([10, -10, -50, 2.5], dtype=torch.float32) # 预测类别[1,0,0,1] # 经过sigmoid，转换为概率，这里才是模型输出的预测值 y_preds = torch.sigmoid(y_logits) print(f"y_preds:{y_preds}, shape: {y_preds.shape}") print(f"y_true:{y_true}, shape: {y_true.shape}") # print(f"y_logits:{y_logits}, shape: {y_logits.shape}") # print(f"y_preds:{y_preds}, shape: {y_preds.shape}") # 3.创建 损失函数，二分类交叉熵损失函数 loss_fn = nn.BCELoss() # 4.计算损失 # y_preds:1D(4,);y_true:1D(4,) loss = loss_fn(y_preds, y_true) print(f"二分类交叉熵损失:{loss}") ... # 测试 if __name__ == '__main__': demo01()
📄 03_回归任务_损失函数.py 2.4KB
""" 演示 回归任务的 损失函数 MAE: Mean Absolute Error, 平均绝对误差 误差绝对值的平均数 类似于L1正则化，理论上权重不可以降到0, 实际工程中会使用小阈值操作来让权重小于某个阈值时强制为0，由此造成权重的稀疏性(存在部分权重为精确0) 缺点： 在0点不平滑，可能错过最小值 MSE: Mean Squared Error, 均方误差 误差平方之和 / 样本总数 缺点: 如果差值过大，可能导致梯度爆炸，无法有效训练模型 Smooth L1: 结合了MAE和MSE, MSE:(-1,1), MAE:(-∞,1]U[1,+∞) 解决L1(MAE)在0点不平滑、以及L2(MSE)的梯度爆炸问题 如何选择回归任务的损失函数 MSE > Smooth L1 > MAE 涉及到的API: nn.L1Loss()-MAE nn.MSELoss()-MSE nn.SmoothL1Loss() 需要掌握的： nn.MSELoss() """ # 导包 import torch import torch.nn as nn # 1.定义函数，演示 MAE def demo01(): # 1.创建 样本的真实值 # 4个样本 y_true = torch.tensor([20.0,21.0,22.0,23.0],dtype=torch.float32) # 2.创建 模型的预测值 y_pred = torch.tensor([21.0,22.0,23.0,24.0],dtype=torch.float32,requires_grad=True) # 3.定义损失函数对象 loss_fn = nn.L1Loss() # 4.计算损失值 loss = loss_fn(y_pred,y_true) # 5.打印损失值 print(f"损失值：{loss}") # 1 # 2.定义函数，演示 MSE def demo02(): # 1.创建 样本的真实值 # 4个样本 y_true = torch.tensor([20.0,21.0,22.0,23.0],dtype=torch.float32) # 2.创建 模型的预测值 y_pred = torch.tensor([21.0,22.0,23.0,24.0],dtype=torch.float32,requires_grad=True) # 3.定义损失函数对象 loss_fn = nn.MSELoss() # 4.计算损失值 loss = loss_fn(y_pred,y_true) # 5.打印损失值 print(f"损失值：{loss}") # 1 # 3.定义函数，演示 Smooth L1 Loss def demo03(): # 1.创建 样本的真实值 # 4个样本 y_true = torch.tensor([20.0,21.0,22.0,23.0],dtype=torch.float32) # 2.创建 模型的预测值 y_pred = torch.tensor([21.0,22.0,23.0,24.0],dtype=torch.float32,requires_grad=True) # 3.定义损失函数对象 loss_fn = nn.SmoothL1Loss() # 4.计算损失值 loss = loss_fn(y_pred,y_true) # 5.打印损失值 print(f"损失值：{loss}") # 1 # 测试 if __name__ == "__main__": # demo01() # demo02() demo03()
📄 04_演示指数移动加权平均.py 2.5KB
""" 演示 指数移动加权平均, 使用近30天的天气温度数据. 结论 1.对于指数移动加权平均值，β值（调节系数）越大，移动加权平均值越平缓，越考虑历史数据。 β值越小，移动加权平均值越接近当前数据。 2.特例：β=0，就是当前数据的原始值；β=1，就是第一天的初始值 """ # 导包 import torch import matplotlib.pyplot as plt ELEMENT_NUMBER = 30 # 1. 实际平均温度 def demo01(): # 固定随机数种子 torch.manual_seed(0) # 产生30天的随机温度 temperature = torch.randn(size=[ELEMENT_NUMBER, ]) * 10 print(temperature) # 绘制平均温度 days = torch.arange(1, ELEMENT_NUMBER + 1, 1) plt.plot(days, temperature, color='r') plt.scatter(days, temperature) plt.tight_layout() plt.show() # 2. 指数加权平均温度 def demo02(beta=0.9): # 固定随机数种子，确保每次运行得到相同的随机温度数据 torch.manual_seed(0) # 产生30天的随机温度数据，通过乘以10放大温度范围 temperature = torch.randn(size=[ELEMENT_NUMBER, ]) * 10 print(temperature) # 存储指数加权平均后的温度值 exp_weight_avg = [] # 遍历每天的温度数据，idx从0开始计数 for idx, temp in enumerate(temperature): # 第一天的指数加权平均值就是当天温度本身 if idx == 0: exp_weight_avg.append(temp) continue # 从第二天开始，按照指数加权平均公式计算： # V_t = β * V_{t-1} + (1-β) * θ_t # 其中 V_t 是当前时刻的加权平均值，V_{t-1} 是上一时刻的加权平均值 # θ_t 是当前时刻的实际温度值，β 是权重参数(0<=β<1) # idx-1 是因为要访问前一天的加权平均值 new_temp = exp_weight_avg[idx - 1] * beta + (1 - beta) * temp exp_weight_avg.append(new_temp) # 创建天数序列用于绘图 days = torch.arange(1, ELEMENT_NUMBER + 1, 1) # 设置图表标题，显示当前使用的beta值 plt.title(f"beta={beta}") # 绘制指数加权平均温度曲线（红色） plt.plot(days, exp_weight_avg, color='r') # 绘制原始温度数据点 plt.scatter(days, temperature) plt.tight_layout() # 显示图表 plt.show() if __name__ == '__main__': demo01() demo02(beta=0) demo02(beta=0.5) demo02(beta=0.9) demo02(beta=1)
📎 资料
每日一图
- 📎 day05.png99.7KB
ℹ️ 离线版不含附件原文件，需要请到在线网站对应课时下载。
第6节 · 优化器与网络优化技巧
🎯 掌握优化器、学习率衰减、Dropout、BatchNorm。
💻 代码
Day06 代码：优化器 / 学习率衰减 / Dropout / BN
4 个文件📄 01_梯度下降法优化方法.py 8.8KB
""" 演示 梯度下降法的优化方法 介绍： 梯度下降法，由损失函数的导数/梯度、学习率 来更新模型参数 公式：w新 = w旧 - 学习率 * 梯度 存在问题： 1.平缓区域，梯度较小，参数更新慢 2.“鞍点” ，梯度为 0，参数无法更新 3.局部最小值，无法跳出，参数不是最优 梯度下降优化方法： 1.调整梯度，比如使用指数移动加权平均来替代原始梯度 动量法Momentum 梯度的指数移动加权平均：st = β*st-1 + (1-β)*gt 更新参数：wt = wt-1 - η*st 2.调整学习率 AdaGrad自适应学习率(Adaptive Gradient Estimation): 累计平方梯度： St = St-1 + gt*gt 其中，St为本次的累计平方梯度，St-1为上次的累计平方梯度，gt为本次的梯度 学习率： η = η/(sqrt(St)+σ) 更新参数公式： Wt = Wt-1 - 调整后的η * gt 缺点： 可能会使得学习率过早、过量的降低，导致模型训练后期学习率太小，较难找到最优解 RMSProp(Root Mean Square Propagation)自适应学习率： 对AdaGrad的优化，在计算累计平方梯度时，加入 调和权重系数 指数加权累计平方梯度： St = β*St-1 + (1-β)*gt*gt 其中St为本次的累计平方梯度，St-1为上次的累计平方梯度，gt为本次的梯度，β为 调和权重系数 学习率： η = η/(sqrt(St)+σ) 更新参数公式： Wt = Wt-1 - 调整后的η * gt 优点：引入 调和权重系数，控制历史梯度 对 累计平方梯度的影响，防止累计平方梯度过大，从而防止学习率过早降低 Adam(Adaptive Moment Estimation)自适应矩估计: 同时优化 学习率 和 梯度 公式： 梯度(一阶矩)： Mt = β1 * Mt-1 +(1-β1)*gt Mt_hat = Mt / (1-β1^t) 学习率(二阶矩): St = β2 * St-1 + (1-β2)*gt*gt St_hat = St / (1-β2^t) 更新参数公式： Wt = Wt-1 - 学习率/(sqrt(St_hat)+σ) * 调整后的梯度Mt_hat 就是 RMSProp + Momentum AdamW: Adam的优化，解偶了权重衰减，在更新参数时直接添加了权重衰减项。 公式: 梯度(使用一阶矩)： Mt = β1 * Mt-1 +(1-β1)*gt Mt_hat = Mt / (1-β1^t) 学习率(使用二阶矩): St = β2 * St-1 + (1-β2)*gt*gt St_hat = St / (1-β2^t) 更新参数公式： Wt = Wt-1 - 学习率/(sqrt(St_hat)+σ) * 调整后的梯度Mt_hat - η * λ * Wt-1 需要掌握： Adam,AdamW: optim.Adam/AdamW(实际使用) Momentum: optim.SGD(面试) """ # 导包 import torch import torch.nn as nn import torch.optim as optim # 优化器模块，实现梯度下降法以及梯度下降的优化方法 # 1.定义函数，演示 梯度下降的优化方法--动量法Momentum def demo01(): print("动量法Momentum") # 1.初始化模型参数w w = torch.tensor([1.0],requires_grad=True, dtype=torch.float32) print(f"初始权重w:{w}") # 2.定义损失函数,并计算损失 loss = w**2 # loss'=2w # 3.创建优化器，动量法Momentum # [w]:待训练的参数；lr：学习率，momentum：动量因子 optimizer = optim.SGD([w],lr=0.01,momentum=0.95) # 4.梯度清零 optimizer.zero_grad() # 5.反向传播，计算梯度 loss.mean().backward() # 6.更新参数，第一次更新参数 # dloss_mean/dw = 2*w = 2 # w = w - 0.01 * 2 = 1 - 0.01*2 = 0.98 optimizer.step() print(f"第一次更新参数：w:{w.data}, w.grad:{w.grad.data}") # 7.第二次更新参数 loss = w ** 2 optimizer.zero_grad() loss.mean().backward() optimizer.step() print(f"第二次更新参数：w:{w.data}, w.grad:{w.grad.data}") ... # 2.定义函数，演示 梯度下降的优化方法--AdaGrad自适应学习率 def demo02(): print("AdaGrad自适应学习率") # 1.初始化模型参数w w = torch.tensor([1.0],requires_grad=True, dtype=torch.float32) print(f"初始权重w:{w}") # 2.定义损失函数,并计算损失值 loss = w**2 # loss'=2w # 3.创建优化器，动量法Momentum # optimizer = optim.SGD([w],lr=0.01,momentum=0.95) # AdaGrad自适应学习率 optimizer = optim.Adagrad([w],lr=0.01) # 4.梯度清零 optimizer.zero_grad() # 5.反向传播，计算梯度 loss.mean().backward() # 6.更新参数，第一次更新参数 optimizer.step() print(f"第一次更新参数：w:{w.data}, w.grad:{w.grad.data}") # 7.第二次更新参数 loss = w ** 2 optimizer.zero_grad() loss.mean().backward() optimizer.step() print(f"第二次更新参数：w:{w.data}, w.grad:{w.grad.data}") ... # 3.定义函数，演示 梯度下降的优化方法--RMSProp自适应学习率 def demo03(): print("RMSProp自适应学习率") # 1.初始化模型参数w w = torch.tensor([1.0],requires_grad=True, dtype=torch.float32) print(f"初始权重w:{w}") # 2.定义损失函数,并计算损失值 loss = w**2 # loss'=2w # 3.创建优化器，动量法Momentum # optimizer = optim.SGD([w],lr=0.01,momentum=0.95) # AdaGrad自适应学习率 # optimizer = optim.Adagrad([w],lr=0.01) # RMSProp自适应学习率 optimizer = optim.RMSprop([w],lr=0.01) # 4.梯度清零 optimizer.zero_grad() # 5.反向传播，计算梯度 loss.mean().backward() # 6.更新参数，第一次更新参数 optimizer.step() print(f"第一次更新参数：w:{w.data}, w.grad:{w.grad.data}") # 7.第二次更新参数 loss = w ** 2 optimizer.zero_grad() loss.mean().backward() optimizer.step() print(f"第二次更新参数：w:{w.data}, w.grad:{w.grad.data}") ... # 4.定义函数，演示 梯度下降的优化方法--Adam自适应矩估计 def demo04(): print("Adam自适应矩估计") # 1.初始化模型参数w w = torch.tensor([1.0],requires_grad=True, dtype=torch.float32) print(f"初始权重w:{w}") # 2.定义损失函数,并计算损失值 loss = w**2 # loss'=2w # 3.创建优化器，动量法Momentum # optimizer = optim.SGD([w],lr=0.01,momentum=0.9) # AdaGrad自适应学习率 # optimizer = optim.Adagrad([w],lr=0.01) # RMSProp自适应学习率 # optimizer = optim.RMSprop([w],lr=0.01) # Adam自适应矩估计 optimizer = optim.Adam([w],lr=0.01) # 4.梯度清零 optimizer.zero_grad() # 5.反向传播，计算梯度 loss.mean().backward() # 6.更新参数，第一次更新参数 optimizer.step() print(f"第一次更新参数：w:{w.data}, w.grad:{w.grad.data}") # 7.第二次更新参数 loss = w ** 2 optimizer.zero_grad() loss.mean().backward() optimizer.step() print(f"第二次更新参数：w:{w.data}, w.grad:{w.grad.data}") ... # 5.定义函数，演示 梯度下降的优化方法--AdamW def demo05(): print("AdamW") # 1.初始化模型参数w w = torch.tensor([1.0],requires_grad=True, dtype=torch.float32) print(f"初始权重w:{w}") # 2.定义损失函数,并计算损失值 loss = w**2 # loss'=2w # 3.创建优化器，动量法Momentum # optimizer = optim.SGD([w],lr=0.01,momentum=0.9) # AdaGrad自适应学习率 # optimizer = optim.Adagrad([w],lr=0.01) # RMSProp自适应学习率 # optimizer = optim.RMSprop([w],lr=0.01) # Adam自适应矩估计 # optimizer = optim.Adam([w],lr=0.01,decoupled_weight_decay=True,weight_decay=1e-2) # AdamW # [w]:待训练的模型参数；lr：学习率，weight_decay：权重衰减因子，λ*w optimizer = optim.AdamW([w],lr=0.01,weight_decay=1e-2) # 4.梯度清零 optimizer.zero_grad() # 5.反向传播，计算梯度 loss.mean().backward() # 6.更新参数，第一次更新参数 optimizer.step() print(f"第一次更新参数：w:{w.data}, w.grad:{w.grad.data}") # 7.第二次更新参数 loss = w ** 2 optimizer.zero_grad() loss.mean().backward() optimizer.step() print(f"第二次更新参数：w:{w.data}, w.grad:{w.grad.data}") ... # 测试 if __name__ == '__main__': demo01() demo02() demo03() demo04() demo05()
📄 02_演示学习率衰减策略.py 8.6KB
""" 演示 学习率衰减策略/学习率调度策略 介绍： 之前学的优化器，可以调节学习率：Adam/AdamW, AdaGrad, RMSProp, Momentum(调节梯度) 也可以手动调节学习率： 等间隔学习率衰减 指定间隔学习率衰减 指数间隔学习率衰减 周期重启的余弦退火策略 等间隔学习率衰减 固定间隔来衰减学习率，当前学习率lr = 上一次的学习率lr*γ, 比如γ=0.5 指定间隔学习率衰减 人为指定间隔来衰减学习率，当前学习率lr = 上一次的学习率lr*γ, 比如γ=0.5 指数间隔学习率衰减 当前轮次的lr = 上个轮次lr*γ, 比如γ=0.95 周期重启的余弦退火策略 带有周期重启的余弦曲线，可以让学习率重新启动，有可能冲过局部最小值 神经网络的训练流程： 1.准备数据集 2.构建神经网络模型 3.设置损失函数和优化器，以及学习率调度器 4.模型训练 1.前向传播 2.计算损失 3.梯度清零 4.反向传播 5.更新参数:w新 = w旧-学习率*梯度 6.更新学习率 5.模型测试 需要掌握： 无 建议周期重启的余弦退火策略 optim.lr_scheduler.CosineAnnealingWarmRestarts """ # 导包 import torch import torch.nn as nn import torch.optim as optim # 优化器模块，提供各种优化器对象，比如SGD,Adam import matplotlib.pyplot as plt # 绘图 # 设置中文字体 plt.rcParams['font.sans-serif'] = ['SimHei'] # 微软雅黑 plt.rcParams['axes.unicode_minus'] = False # 解决负号显示问题 # 1.定义函数，演示 等间隔学习率衰减策略 def demo01(): # 0.初始化参数 lr = 0.1 epochs = 200 iteration = 10 # 1.准备数据集 x = torch.tensor([1.0],dtype=torch.float32) y_true = torch.tensor([0.0],dtype=torch.float32) # 2.构建神经网络模型 # 创建张量，模拟网络参数 w = torch.tensor([1.0],dtype=torch.float32,requires_grad=True) # 3.设置损失函数和优化器，以及学习率调度器 # 损失函数 loss_fn = nn.MSELoss() # 优化器 optimizer = optim.SGD([w],lr=lr) # 学习率调度器 scheduler = optim.lr_scheduler.StepLR(optimizer,step_size=50,gamma=0.5) # 4.模型训练 # 定义列表，记录训练轮数和学习率 lr_list = [] epoch_list = [] for epoch in range(epochs): # 0.获取当前轮数 和 学习率，保存到记录列表中 epoch_list.append(epoch) lr_list.append(scheduler.get_last_lr()[0]) # 遍历批次 for i in range(iteration): # 1.前向传播 y_pred = w * x # 2.计算损失 # 3.梯度清零 # 4.反向传播 # 5.更新参数:w新 = w旧-学习率*梯度 # 6.更新学习率 scheduler.step() # 5.可视化学习率变化 plt.plot(epoch_list,lr_list) plt.title("等间隔学习率衰减") plt.xlabel("epoch") plt.ylabel("lr") plt.show() # 2.定义函数，演示 指定间隔学习率衰减策略 def demo02(): # 0.初始化参数 lr = 0.1 epochs = 200 iteration = 10 # 1.准备数据集 x = torch.tensor([1.0],dtype=torch.float32) y_true = torch.tensor([0.0],dtype=torch.float32) # 2.构建神经网络模型 # 创建张量，模拟网络参数 w = torch.tensor([1.0],dtype=torch.float32,requires_grad=True) # 3.设置损失函数和优化器，以及学习率调度器 # 损失函数 loss_fn = nn.MSELoss() # 优化器 optimizer = optim.SGD([w],lr=lr) # 学习率调度器 # scheduler = optim.lr_scheduler.StepLR(optimizer,step_size=50,gamma=0.5) # 指定间隔学习率衰减策略 scheduler = optim.lr_scheduler.MultiStepLR(optimizer,milestones=[50,125,160],gamma=0.5) # 4.模型训练 # 定义列表，记录训练轮数和学习率 lr_list = [] epoch_list = [] for epoch in range(epochs): # 0.获取当前轮数 和 学习率，保存到记录列表中 epoch_list.append(epoch) lr_list.append(scheduler.get_last_lr()[0]) # 遍历批次 for i in range(iteration): # 1.前向传播 y_pred = w * x # 2.计算损失 # 3.梯度清零 # 4.反向传播 # 5.更新参数:w新 = w旧-学习率*梯度 # 6.更新学习率 scheduler.step() # 5.可视化学习率变化 plt.plot(epoch_list,lr_list) plt.title("指定间隔学习率衰减") plt.xlabel("epoch") plt.ylabel("lr") plt.show() # 3.定义函数，演示 指数间隔学习率衰减策略 def demo03(): # 0.初始化参数 lr = 0.1 epochs = 200 iteration = 10 # 1.准备数据集 x = torch.tensor([1.0],dtype=torch.float32) y_true = torch.tensor([0.0],dtype=torch.float32) # 2.构建神经网络模型 # 创建张量，模拟网络参数 w = torch.tensor([1.0],dtype=torch.float32,requires_grad=True) # 3.设置损失函数和优化器，以及学习率调度器 # 损失函数 loss_fn = nn.MSELoss() # 优化器 optimizer = optim.SGD([w],lr=lr) # 学习率调度器 # scheduler = optim.lr_scheduler.StepLR(optimizer,step_size=50,gamma=0.5) # 指定间隔学习率衰减策略 # scheduler = optim.lr_scheduler.MultiStepLR(optimizer,milestones=[50,125,160],gamma=0.5) # 指数间隔学习率衰减策略 scheduler = optim.lr_scheduler.ExponentialLR(optimizer,gamma=0.95) # 4.模型训练 # 定义列表，记录训练轮数和学习率 lr_list = [] epoch_list = [] for epoch in range(epochs): # 0.获取当前轮数 和 学习率，保存到记录列表中 epoch_list.append(epoch) lr_list.append(scheduler.get_last_lr()[0]) # 遍历批次 for i in range(iteration): # 1.前向传播 y_pred = w * x # 2.计算损失 # 3.梯度清零 # 4.反向传播 # 5.更新参数:w新 = w旧-学习率*梯度 # 6.更新学习率 scheduler.step() # 5.可视化学习率变化 plt.plot(epoch_list,lr_list) plt.title("指数间隔学习率衰减") plt.xlabel("epoch") plt.ylabel("lr") plt.show() # 4.定义函数，演示 周期重启的余弦退火策略 def demo04(): # 0.初始化参数 lr = 0.1 epochs = 200 iteration = 10 # 1.准备数据集 x = torch.tensor([1.0],dtype=torch.float32) y_true = torch.tensor([0.0],dtype=torch.float32) # 2.构建神经网络模型 # 创建张量，模拟网络参数 w = torch.tensor([1.0],dtype=torch.float32,requires_grad=True) # 3.设置损失函数和优化器，以及学习率调度器 # 损失函数 loss_fn = nn.MSELoss() # 优化器 optimizer = optim.SGD([w],lr=lr) # 学习率调度器 # scheduler = optim.lr_scheduler.StepLR(optimizer,step_size=50,gamma=0.5) # 指定间隔学习率衰减策略 # scheduler = optim.lr_scheduler.MultiStepLR(optimizer,milestones=[50,125,160],gamma=0.5) # 指数间隔学习率衰减策略 # scheduler = optim.lr_scheduler.ExponentialLR(optimizer,gamma=0.95) # 周期重启的余弦退火策略 scheduler = optim.lr_scheduler.CosineAnnealingWarmRestarts( optimizer, # 优化器对象 T_0=25, # 第1个周期的轮数,通常总训练轮数至少有5~10个周期 eta_min=0, # 最小学习率 T_mult=1, # 周期倍增因子，默认为1，表示每个周期的轮数相同 ) # 4.模型训练 # 定义列表，记录训练轮数和学习率 lr_list = [] epoch_list = [] for epoch in range(epochs): # 0.获取当前轮数 和 学习率，保存到记录列表中 epoch_list.append(epoch) lr_list.append(scheduler.get_last_lr()[0]) # 遍历批次 for i in range(iteration): # 1.前向传播 y_pred = w * x # 2.计算损失 # 3.梯度清零 # 4.反向传播 # 5.更新参数:w新 = w旧-学习率*梯度 # 6.更新学习率 scheduler.step() # 5.可视化学习率变化 plt.plot(epoch_list,lr_list) plt.title("周期重启的余弦退火策略") plt.xlabel("epoch") plt.ylabel("lr") plt.show() # 测试 if __name__ == '__main__': demo01() demo02() demo03() demo04()
📄 03_dropout随机失活.py 1.7KB
""" 演示 dropout随机失活 注意： dropout是一个特殊的网络层，没有可训练的参数，只有一个超参数p。dropout层通常添加到输出层之前 正则化作用： 缓解模型过拟合，提高模型泛化能力 正则化的方法： L1正则化：在损失函数中添加绝对值项|w|, 权重可以为0，相当于降维 L2正则化：在损失函数中添加平方项w^2, 权重可以接近0，Adam/AdamW权重衰减，对离散值敏感 Dropout随机失活： 1.训练时，model.train(),随机以概率p让一部分神经元失活，输出设置为0。没有失活的神经元的输出*1/(1-p) 2.测试时，model.eval(),关闭dropout随机失活，让所有神经元一起参与 需要掌握： nn.Dropout(p=0.1) 参数p设置：0<p<1，通常p<0.5, 当模型参数较多 数据量较大 时，可以适当增加p,比如0.4~0.5 """ # 导包 import torch import torch.nn as nn # 1.定义函数，演示 dropout随机失活 def demo01(): # 0.设置随机种子 torch.manual_seed(50) # 1.创建输入数据，(batch_size,input_dim) x = torch.randint(0,10,(1,5),dtype=torch.float32) print(f"x: {x},x.shape: {x.shape}") # 2.创建神经网络，一个简单的线性层 linear1 = nn.Linear(5,20) # 3.创建一个dropout层，来实现dropout随机失活 dropout = nn.Dropout(p=0.5) # 4.把数据输入到线性层 x = linear1(x) # (1,5) -> (1,20) print(f"x: {x},x.shape: {x.shape}") # 5.把数据输入到droput层 x = dropout(x) # 6.打印dropout输出 print(f"x: {x},x.shape: {x.shape}") ... # 测试 if __name__ == '__main__': demo01()
📄 04_BN批量归一化.py 1.8KB
""" 演示 批量归一化，属于正则化方法，缓解 模型过拟合，主要用于CV计算机视觉，比如CNN网络 批量归一化： 先标准化为均值0，方差1的分布，然后再进行重构(缩放+平移，缩放系数和平移系数都是可学习的)，进一步提取特征 涉及到的API： CV计算机视觉领域 BatchNorm2d: 处理4D数据，主要用于二维卷积神经网络Conv2d来处理图片数据，NCHW,(N,C,H,W), N批次大小，C通道数，H高度，W宽度 BatchNorm1d: 处理2D/3D数据，主要用于处理文本数据/序列数据，NCS BatchNorm3d: 处理5D数据，主要用于三维卷积神经网络Conv3d,来处理高维的视觉数据，比如医学图像，卫星遥感视频，接收形状为 NCDHW/NCTHW 的张量作为输入。 需要掌握的： 针对CV方向，BatchNorm2d """ # 导包 import torch import torch.nn as nn # 定义函数，演示 BatchNorm2d 批量归一化, 处理4D数据，NCHW def demo01(): # 1.创建数据，模拟NCHW图片数据 torch.randint(0,256,size=(2,3,32,32)) x = torch.randint(0,256,size=(1,3,32,32)).to(dtype=torch.float32) print(f"原始数据x:{x}, shape: {x.shape}") # 2.创建BatchNorm2d层 # running_mean = (1 - momentum) * running_mean + momentum * batch_mean # running_var = (1 - momentum) * running_var + momentum * batch_var bn2d = nn.BatchNorm2d( num_features=3, # 通道数，C eps=1e-05, # 极小值，防止除零 momentum=0.1, # 动量，控制参数的更新 affine=True, # 缩放参数和平移参数是否可学习 ) # 3.输入数据到BatchNorm2d层中 y = bn2d(x) # (1,3,32,32) # 4.打印输出 print(f"处理后的数据y:{y}, shape: {y.shape}") # 测试 if __name__ == '__main__': demo01()
📎 资料
每日一图
- 📎 day06.png123.4KB
ℹ️ 离线版不含附件原文件，需要请到在线网站对应课时下载。
第7节 · 完整案例：手机价格分类
🎯 综合运用前面知识完成手机价格分类案例与调优。
💻 代码
Day07 代码：手机价格分类案例+调优
4 个文件📁 data/
- ⚙️ 手机价格预测.csv 119.5KB
📄 01_手机价格分类案例.py 11.8KB
""" 演示 手机价格分类案例 注意： 这里是预测手机价格区间，0，1，2，3，是一个单标签多分类任务，我们用全连接神经网络实现 神经网络训练的工作流 1.准备数据集 拆分数据集为训练集和测试集。这里简化了数据拆分，实际工程中会拆分为训练集、验证集(模拟考试)、测试集(最终考试) numpy数值 -> 张量 -> 数据集对象 -> 数据加载器 2.搭建神经网络 使用全连接神经网络 1.继承nn.Module 2.定义__init__方法，构造网络层 3.定义forward方法，前向传播 3.模型训练-在训练集上训练 model.train() 前向传播 计算损失 梯度清零 反向传播 更新参数 4.模型测试-在测试集上测试 model.eval() """ # 导入相关模块 import os import torch from torch.utils.data import TensorDataset, DataLoader # 数据集对象 数据加载器 import torch.nn as nn # 神经网络模块，实现各种网络层 import torch.optim as optim # 优化器模块，实现各种优化器，比如SGD,Adam/AdamW from sklearn.model_selection import train_test_split # 划分数据集,pip install scikit-learn import matplotlib.pyplot as plt # pip install matplotlib import numpy as np # pip install matplotlib import pandas as pd # pip install pandas import time from torchsummary import summary # 统计模型参数量 # 设置中文字体 plt.rcParams['font.sans-serif'] = ['SimHei'] # 微软雅黑 plt.rcParams['axes.unicode_minus'] = False # 解决负号显示问题 # 设置设备，GPU > MPS > CPU device = torch.device( "cuda" if torch.cuda.is_available() else "mps" if torch.backends.mps.is_available() else "cpu" ) print(f"当前设备：{device}") # 设置随机种子 torch.manual_seed(28) # 1.准备数据集 def create_dataset(): # 1.加载CSV df = pd.read_csv(r"data/手机价格预测.csv") # 2.构造输入特征x,输出标签y # 前20列是输入特征，最后一列是标签y x = df.iloc[:, :-1] y = df.iloc[:, -1] # 转换x的元素类型为浮点数float32 x = x.astype(np.float32) # print(f"x:{x}, y:{y}") # print(f"x.shape:{x.shape}, y.shape:{y.shape}") # 3.拆分数据集为训练集和测试集 # 参数1：输入特征x；参数2：标签y；参数3：训练集拆分比例；参数4：随机状态，固定随机性 # 参数5：stratify,按哪个标签比例进行层次拆分，比如y中类别比例为1：1：1：1，拆分之后的数据也要符合这个比例 x_train, x_test, y_train, y_test = train_test_split(x,y,train_size=0.8,random_state=5,stratify=y) # print(f"x_train.shape:{x_train.shape}, x_test.shape:{x_test.shape}, y_train.shape:{y_train.shape}, y_test.shape:{y_test.shape}") # 4.numpy数值 -> 张量 torch.tensor() x_train = torch.tensor(x_train.values,dtype=torch.float32) x_test = torch.tensor(x_test.values,dtype=torch.float32) y_train = torch.tensor(y_train.values,dtype=torch.long) y_test = torch.tensor(y_test.values,dtype=torch.long) print(f"x_train.shape:{x_train.shape}, x_test.shape:{x_test.shape}, y_train.shape:{y_train.shape}, y_test.shape:{y_test.shape}") # 5.张量 -> 数据集对象, 封装输入特征x和输出标签y train_dataset = TensorDataset(x_train,y_train) # 数据集对象用于定义如何构造一个样本 test_dataset = TensorDataset(x_test,y_test) # 6.返回 return train_dataset, test_dataset # 2.搭建神经网络 class MyPhonePriceClassifier(nn.Module): # 1.定义__init__方法，构造网络层 def __init__(self, input_dim=20, output_dim=4): # 1.初始化父类成员 super().__init__() # 2.定义第一个线性层,自动进行参数初始化,kaiming初始化 self.linear1 = nn.Linear(input_dim,256) # 3.定义第二个线性层 self.linear2 = nn.Linear(256,128) # 4.定义第三个线性层，输出层 self.linear3 = nn.Linear(128,output_dim) # 2.定义forward方法，前向传播 def forward(self,x): # x:(batch_size,20) # 1.数据输入linear1，relu x = torch.relu(self.linear1(x)) # (batch_size,256) # 2.数据输入linear2,relu x = torch.relu(self.linear2(x)) # (batch_size,128) # 3.数据输入linear3 x = self.linear3(x) # (batch_size,output_dim) # 4.返回结果 return x # 3.模型训练-在训练集上训练 # train_dataset:训练集数据集对象 # input_dim:数据的特征维度，20 # output_dim: 输出维度，4，也就是有4个类别 def train(train_dataset, input_dim=20, output_dim=4, epochs=10): # 1.创建数据加载器对象 # 参数1：数据集对象；参数2：批次大小；参数3：是否打乱数据，训练时打乱，测试时不打乱；参数4: 是否删除最后一个不完整的批次数据 train_dataloader = DataLoader(train_dataset,batch_size=32,shuffle=True,drop_last=False) # 2.创建神经网络模型对象 model = MyPhonePriceClassifier( input_dim=input_dim, output_dim=output_dim ).to(device) # 迁移模型到设备 # 3.定义损失函数和优化器 loss_fn = nn.CrossEntropyLoss() optimizer = optim.AdamW(model.parameters(),lr=1e-3,weight_decay=1e-3) # 定义变量，记录 每个轮次的 训练损失 和 准确率, 用于可视化 train_losses = [] train_accs = [] # 4.模型训练 # 遍历轮数epochs,举个栗子：值日生扫地，要扫10个轮次 for epoch in range(epochs): # 1.设置训练模式 model.train() # 2.初始化 当前轮次的 训练的总损失，总样本数，预测正确的样本数 train_loss = 0.0 total_samples = 0 total_correct = 0 # 获取当前时刻 start_time = time.time() # 3.遍历数据加载器，分批次训练，举个栗子：值日生扫地，在扫第N个轮次的时候，要逐个扫过每一排，就是每个batch for x, y in train_dataloader: # x:(batch_size,input_dim) # y:(batch_size,) # 0.迁移数据到device x = x.to(device) y = y.to(device) # 1.前向传播 y_pred = model(x) # (batch_size,4) # print(f"y_pred.shape:{y_pred.shape}") # 2.计算损失 loss = loss_fn(y_pred, y) # print(f"loss:{loss}, shape:{loss.shape}") # 0D # 3.梯度清零 optimizer.zero_grad() # 4.反向传播 loss.backward() # 5.更新参数 optimizer.step() # 6.更新训练指标，训练损失，训练样本数，预测正确的样本数 train_loss += loss.item() * x.shape[0] total_samples += x.shape[0] # 更新预测正确的数量 total_correct += (y_pred.argmax(dim=-1) == y).sum().item() # 4.计算当前轮次的训练平均损失和训练准确率 avg_loss = train_loss / total_samples acc = total_correct / total_samples # 获取训练损失和准确率 train_losses.append(avg_loss) train_accs.append(acc) # 5.打印训练损失和训练准确率 print(f"Epoch: {epoch+1}/{epochs} | " f"训练损失：{avg_loss:.4f} | " f"训练准确率：{acc:.4f} | " f"训练时间：{(time.time() - start_time):.2f}s") ... # 5.保存模型文件 # 新建文件夹model os.makedirs("./model", exist_ok=True) # 保存模型参数文件，因为在不同设备上迁移更加方便，如果把模型结构也保存了，那么模型在不同设备上迁移困难 print(f"模型参数：{model.state_dict()}") torch.save(model.state_dict(), r"./model/phone_price_classifier.pth") # python模型文件格式有.pth/.pt,两个都一样 # 返回训练损失和训练准确率 return train_losses, train_accs # 3.1 可视化训练过程 def plot_train_process(train_losses, train_accs): plt.figure(figsize=(12, 6)) # 绘制训练损失曲线 plt.subplot(1, 2, 1) plt.plot(train_losses, label="train_loss") plt.xlabel("epoch") plt.ylabel("loss") plt.legend() plt.grid(True) # 绘制训练准确率曲线 plt.subplot(1, 2, 2) plt.plot(train_accs, label="train_acc") plt.xlabel("epoch") plt.ylabel("acc") plt.legend() plt.grid(True) plt.tight_layout() plt.show() # 4.模型测试-在测试集上测试 # 参数1：测试集 @torch.no_grad() # 关闭梯度计算，节省显卡内存20% def evaluate(test_dataset, input_dim=20, output_dim=4): # 1.创建数据加载器对象 # 参数1：数据集对象；参数2：批次大小；参数3：是否打乱数据，训练时打乱，测试时不打乱；参数4: 是否删除最后一个不完整的批次数据 test_dataloader = DataLoader(test_dataset, batch_size=32, shuffle=False, drop_last=False) # 2.创建神经网络模型对象 model = MyPhonePriceClassifier( input_dim=input_dim, output_dim=output_dim ).to(device) # 迁移模型到设备 # 3.加载模型参数，将模型参数文件的模型参数加载到示例化的模型对象中 model.load_state_dict(torch.load(r"model/phone_price_classifier.pth")) # 定义损失函数 loss_fn = nn.CrossEntropyLoss() # 4.模型测试，设置为评估模式model.eval()，告诉模型，现在是训练/评估，不要开启dropout等特殊网络层 model.eval() # 定义变量，记录测试 的总损失，总样本数，预测正确的样本数 test_loss = 0.0 total_samples = 0 total_correct = 0 # 遍历批次，也就是遍历数据加载器 for x, y in test_dataloader: # x:(batch_size,input_dim) # y:(batch_size,) # 0.迁移数据到device x = x.to(device) y = y.to(device) # 1.前向传播 y_pred = model(x) # (batch_size,4) # 2.计算损失 loss = loss_fn(y_pred, y) # 梯度清零 # 反向传播 # 更新参数 # 3.更新指标，损失，样本数，预测正确的样本数 batch_size = x.shape[0] # 更新总损失 test_loss += loss.item() * batch_size # 当前批次的总损失=平均损失*当前批次的批次大小 # 更新样本数 total_samples += batch_size # 更新预测正确的样本数 total_correct += (y_pred.argmax(dim=-1) == y).sum().item() ... # 计算测试损失和测试准确率 avg_test_loss = test_loss / total_samples avg_test_acc = total_correct / total_samples # 返回结果 return avg_test_loss, avg_test_acc # 测试 if __name__ == "__main__": # 1.准备数据集 train_dataset, test_dataset=create_dataset() print(f"训练集数量：{len(train_dataset)}") print(f"测试集数量：{len(test_dataset)}") print(f"train_dataset:{train_dataset}, test_dataset:{test_dataset}") # # 2.创建模型对象 # model = MyPhonePriceClassifier() # 默认模型对象在CPU上 # model = model.to(device) # 将模型对象移动到device上，比如GPU # print(model) # summary(model, input_size=(20,)) # 3.模型训练 train_losses, train_accs=train(train_dataset, input_dim=20, output_dim=4, epochs=10) # 可视化训练过程 plot_train_process(train_losses, train_accs) # 4.模型测试 avg_test_loss, avg_test_acc = evaluate(test_dataset, input_dim=20, output_dim=4) print(f"测试损失：{avg_test_loss:.4f} | 测试准确率：{avg_test_acc:.4f}")
📄 02_手机价格分类案例_模型调优.py 8.2KB
""" 模型调优思路： 1. 对输入数据进行标准化 2. 调整优化器 3. 调整学习率 4. 增加批量归一化层BN1D 5. 增加网络层数、神经元个数 6. 增加dropout层，缓解过拟合 7. 增加训练轮数 8. 使用GPU加速训练 9. ... """ import os import torch # pytorch框架 from torch.utils.data import TensorDataset # 数据集对象 from torch.utils.data import DataLoader # 数据加载器 import torch.nn as nn # 神经网络层的操作 import torch.optim as optim # 优化器 from sklearn.model_selection import train_test_split # 拆分 训练集 和 测试集 import matplotlib.pyplot as plt # 绘图 import numpy as np # 数组操作 import pandas as pd # 数据处理 import time # 时间模块 from torchsummary import summary from sklearn.preprocessing import StandardScaler # 设置中文字体 plt.rcParams['font.sans-serif'] = ['SimHei'] # 微软雅黑 plt.rcParams['axes.unicode_minus'] = False # 解决负号显示问题 device = "cuda" if torch.cuda.is_available() else "cpu" print(f"使用设备：{device}") # 1.准备数据集 def create_dataset(): # 1.加载CSV数据集 data = pd.read_csv("data/手机价格预测.csv") # 2.获取特征x 和 标签y x = data.iloc[:, :-1] y = data.iloc[:, -1] # 3.特征列 转换为 float x = x.astype(np.float32) # 4.拆分数据集 x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=6, stratify=y) # 数据标准化 # scaler = StandardScaler() # x_train = scaler.fit_transform(x_train) # x_test = scaler.transform(x_test) # 5.把数据集转换为张量, numpy数组 -> 张量 -> 数据集对象TensorDataset -> 数据加载器DataLoader train_dataset = TensorDataset(torch.tensor(x_train.values), torch.tensor(y_train.values)) test_dataset = TensorDataset(torch.tensor(x_test.values), torch.tensor(y_test.values)) # 6.返回结果 return train_dataset, test_dataset, x_train.shape[1], len(y_train.unique()) # 2.搭建神经网络 class PhonePriceClassifier(nn.Module): # 1.定义__init__方法, 初始化父类成员，搭建神经网络 def __init__(self, input_dim, output_dim): # 初始化父类成员 super().__init__() # 搭建神经网络 # 优化1：增加网络深度+使用BN1D层 self.linear1 = nn.Linear(input_dim, 512) self.bn1 = nn.BatchNorm1d(512) self.linear2 = nn.Linear(512, 512) self.bn2 = nn.BatchNorm1d(512) self.linear3 = nn.Linear(512, 256) self.bn3 = nn.BatchNorm1d(256) self.linear4 = nn.Linear(256, output_dim) self.dropout = nn.Dropout(0.1) # 2.定义forward方法，实现前向传播 def forward(self, x): # 隐藏层1, linear -> BN层 -> 激活函数 -> dropout x = torch.relu(self.bn1(self.linear1(x))) x = self.dropout(x) # 隐藏层2 x = torch.relu(self.bn2(self.linear2(x))) x = self.dropout(x) x = torch.relu(self.bn3(self.linear3(x))) x = self.dropout(x) # 输出层 x = self.linear4(x) # 返回结果 return x # 3.模型训练 def train(train_dataset, input_dim, output_dim): # 1.创建数据加载器DataLoader # 参数1：数据集对象；参数2：批量大小；参数3：是否打乱数据 train_loader = DataLoader(train_dataset, batch_size=64, shuffle=True) # 2.创建神经网络模型 model = PhonePriceClassifier(input_dim, output_dim) model.to(device) # 3.定义损失函数 loss_fn = nn.CrossEntropyLoss() # 4.创建优化器对象 # 优化3：使用Adam优化方法 optimizer = optim.AdamW(model.parameters(), lr=1e-3, weight_decay=1e-2) # 优化4：添加学习率调度，optim.lr_scheduler.CosineAnnealingWarmRestarts scheduler = optim.lr_scheduler.CosineAnnealingWarmRestarts(optimizer, T_0=5*len(train_loader), T_mult=1) # 5.模型训练 # 定义变量，记录 训练总轮数 epochs = 50 # 定义遍历，记录每轮次的损失和准确率 train_losses = [] train_accs = [] # 遍历轮数 for epoch in range(epochs): # 记录 每次训练的 损失值，训练样本数，预测正确的样本数 total_loss = 0. total_sample = 0 total_correct = 0 # 记录训练开始时间 start_time = time.time() # 遍历批次 for x, y in train_loader: # 训练模式 model.train() # 移动数据到GPU x = x.to(device) y = y.to(device) # 前向传播 y_pred = model(x) # 计算损失值 loss = loss_fn(y_pred, y) # 更新总损失 batch_size = x.shape[0] total_loss += loss.item() * batch_size # 更新总样本数 total_sample += batch_size # 更新预测正确的样本数 total_correct += (torch.argmax(y_pred, dim=-1) == y).sum().item() # 梯度清零 optimizer.zero_grad() # 反向传播 loss.backward() # 更新参数 optimizer.step() scheduler.step() # 打印训练信息 print( f"epoch: {epoch}, loss: {total_loss / total_sample}, acc: {total_correct / total_sample}, time: {time.time() - start_time:.2f}") train_losses.append(total_loss / total_sample) train_accs.append(total_correct / total_sample) # 6.保存模型参数 os.makedirs("./model", exist_ok=True) # print(f"\n\n模型的参数信息: {model.state_dict()}\n\n") torch.save(model.state_dict(), "./model/phone_price_classifier.pth") # 模型文件，后缀名 .pth, .pkl, .pickle 均可 # 7.返回训练结果 return train_losses, train_accs # 可视化训练过程 def plot_train_process(train_losses, train_accs): plt.figure(figsize=(12, 4)) plt.subplot(1, 2, 1) plt.plot(train_losses) plt.title("训练损失") plt.grid(True) plt.subplot(1, 2, 2) plt.plot(train_accs) plt.title("训练准确率") plt.grid(True) plt.tight_layout() plt.show() # 4.模型测试 @torch.no_grad() def evaluate(test_dataset, input_dim, output_dim): # 1. 创建神经网络分类对象. model = PhonePriceClassifier(input_dim, output_dim) model.to(device) # 2.加载模型参数 model.load_state_dict(torch.load('./model/phone_price_classifier.pth')) # 3.创建数据加载器DataLoader # 参数1：数据集对象；参数2：批量大小；参数3：是否打乱数据 test_loader = DataLoader(test_dataset, batch_size=32, shuffle=False) # 4.记录正确样本的数量 correct_sample = 0 total_sample = 0 # 5.开始测试, 遍历数据加载器的批次数据 for x, y in test_loader: # 切换到测试模式 model.eval() # 张量 迁移 到 GPU x = x.to(device) y = y.to(device) # 6.前向传播，模型预测 y_pred = model(x) # 形状(batch_size, 4), 每个类别的输出logits # print(f"预测结果y_pred：{y_pred}") # 7.计算预测正确的数量 correct = y_pred.argmax(dim=1) == y correct_sample += correct.sum().item() total_sample += y.size(0) print(f"测试集准确率：{correct_sample / total_sample}") # 5.测试 if __name__ == '__main__': # 1.准备数据集 train_dataset, test_dataset, input_dim, output_dim = create_dataset() # 打印 print(f"train_dataset: {train_dataset}") print(f"test_dataset: {test_dataset}") print(f"input_dim: {input_dim}") print(f"output_dim: {output_dim}") # 2.构建神经网络 model = PhonePriceClassifier(input_dim, output_dim) # 3.统计模型参数 model.to(device) # summary(model,input_size=(input_dim,)) # 4.模型训练 train_losses, train_accs = train(train_dataset, input_dim, output_dim) plot_train_process(train_losses, train_accs) # 5.模型测试 evaluate(test_dataset, input_dim, output_dim)
📄 03_扩展_解释布尔张量sum.py 593B
""" y_pred.argmax(dim=-1) (y_pred_class == y).sum().item() """ import torch torch.manual_seed(5) # 1.创建logits, (batch_size, 4) logits = torch.randn(size=(8, 4)) print(f"logits:{logits},shape: {logits.shape}") # 2.经过argmax(dim=-1),(batch_size,) y_pred_class = logits.argmax(dim=-1) print(f"y_pred_class:{y_pred_class},shape: {y_pred_class.shape}") # 3.创建y_true, (batch_size,) y_true = torch.ones((8,)) print(f"y_true:{y_true},shape: {y_true.shape}") # 4.计算预测正确的数量(y_pred_class == y).sum().item() print((y_pred_class == y_true).sum().item())
📎 资料
数据集
- 📎 手机价格预测.csv119.5KB
ℹ️ 离线版不含附件原文件，需要请到在线网站对应课时下载。
📎 资料
每日一图
- 📎 day07.png89.9KB
ℹ️ 离线版不含附件原文件，需要请到在线网站对应课时下载。
📎 思维导图
📎 资料
PyTorch 框架总结 / 人工神经网络基础
- 📎 01pytorch框架总结.xmind227.1KB
- 📎 02人工神经网络基础.xmind231.7KB
🧠 01pytorch框架总结.xmind 大纲
pytorch框架总结
深度学习介绍
概念
机器学习的分支
人工神经网络为基础, 实现自动特征提取的算法
特点
自动提取特征
大数据和计算能力
可解释性差
多层非线性变换
...
算法/模型
ANN 人工神经网络 MLP
CNN 卷积神经网络
RNN 循环神经网络
transformer
Diffusion
BERT文本理解
GPT文本生成
...
应用场景
自然语言处理 NLP
聊天机器人
语音翻译
生成式AI 大模型
计算机视觉 CV
图像识别
图形分割
目标检测
推荐系统
电商推荐
视频推荐
电影/音乐/文章/推荐
多模态大模型
预备的数学知识
线性代数
标量
向量
矩阵
张量
导数和微分
导数定义
求导公式
张量的维度和形状
维度/轴数：表示数据有几个方向
形状：数据在每个方向的长度组成的数组
维度/轴的长度：数据在某个方向的长度
pytorch框架
概念
当前最主流的深度学习框架, 实现神经网络的各种组件或功能
数据是以张量类型存储
创建张量
基本方式
torch.tensor(data=,dtype=)
torch.Tensor(data=, size=())
torch.IntTensor()/FloatTensor()...
线性和随机张量
torch.arange(start=,end=,step=)
torch.linspace(start=,end=,steps=)
torch.randn/rand(size=())
torch.randint(low=,high=,size=())
0/1/指定值张量
torch.zeros/ones(size=)
torch.zeros_like/ones_like(input=tensor)
torch.full(size=, fill_value=)
torch.full_like(input=tensor, fill_value=)
张量元素的类型转换
torch.tensor(data, dtype=)
tensor.type(dtype=)
tensor.to(dtype=, device=)
tensor.half()/float()/double()/int()/long()
类型
torch.int64/float32
cpu: torch.IntTensor/FloatTensor
gpu: torch.cuda.IntTensor/FloatTensor
张量类型转换
张量转换成numpy数组
tensor.numpy()
共享内存
tensor.numpy().copy()
不共享内存
numpy数值转换成张量
torch.from_numpy(data=ndarray)
共享内存
torch.tensor(data=ndarray)
不共享内存
提取单个元素张量数值
tensor.item()
张量运算
基本运算
+ - * /
add()/sub()/mul()/div()/neg()
add_()/sub_()/mul_()/div_()/neg_()
修改原数据，inplace=True
单词: add, substract, multiply, divide
按元素相乘运算/哈达玛积
元素级乘法, 对应位置的元素进行相乘
两个张量形状相同
mul()/*
矩阵乘法运算 1
行*列, 行每个数据和列每个数据相乘求和
规则: (n, m) @ (m, p) =(n, p)
@/torch.matmul()
张量运算函数
min/max/mean/sum(dim=)
不设置dim参数, 对所有元素进行计算
设置dim参数, 对应维度元素进行计算
sqrt()/**0.5
平方根, 开根号
log()/log2()/log10()/log1p()
对数
pow()/**
幂次方 x^n
exp()
指数 e^x
张量索引操作1
作用
根据索引获取对应位置的数据
tensor[0轴索引, 1轴索引, ...]
从左到右从0开始, 0->第一个数据
从右到左从-1开始, -1->最后一个数据
简单行列索引
tensor[0]->0轴第一个数据
tensor[:, 0] ->1轴第一个数据
列表取值
tensor[[0,1], [2,4]]->0轴第1个1轴第3个值, 0轴第2个1轴第5个值
范围取值(切片)
tensor[起始索引:结束索引:步长,...]
tensor[::2],tensor[::]
布尔取值
tensor[:, 0]>10
判断1轴第1组数据大于10->返回T/F
tensor[tensor[:, 0]>10]
获取True对应的0轴数据
多维索引
指定每个维度/轴的索引, tensor[1,2,3]
张量形状操作 1.5
tensor.reshape(shape=)
在保证数据不变的前提下, 修改连续和非连续张量形状
修改前后的张量元素个数一致
-1:自动计算
tensor.squeeze(dim=)
删除长度为1的维度/轴, 可以通过dim指定删除维度/轴
tensor.unsqueeze(dim=)
在指定维度上增加轴，长度1
tensor.transpose(dim0=, dim1=)
交换指定两个维度的数据
参数值是维度下标值
tensor.permute(dims=())
交换任意维度/轴的数据
参数值是维度下标值
tensor.view(shape=)
修改连续张量的形状, 操作等同于reshape
tensor.contiguous()
转换成连续张量
tensor.is_contiguous()
判断张量是否连续
张量拼接操作2
torch.cat([t1, t2, ...], dim=)
在指定维度上对张量进行拼接
其他维度值相同, 不改变新张量的维度
指定维度的维度值相加
torch.stack([t1, t2, ...], dim=)
在指定维度上对张量进行堆叠
其他维度值相同, 新张量在指定维度新增维度
指定维度的维度值就是张量个数
自动微分模块(入门案例,梯度下降法案例)2.5
微分就是求导, 计算导数(梯度)
计算损失函数的梯度/导数
损失函数对模型参数的导数/梯度
使用loss.backward()反向传播中自动求导
单词:
Learning Rate 学习率, 简称: lr
w.grad, 梯度
梯度下降法
找到损失函数的最小值
找到最优模型参数, 对应最佳模型
使用 W新= W旧 - 学习率 * 梯度值 更新参数
w和b一定是可自动微分的张量, 类型是float类型
torch.tensor(requries_grad=True),我是自变量，可以对我进行求导，后面定义了loss=loss_fn(w)
自动微分的张量不能转换成numpy数组, 通过.detach()张量剥离成不自动微分, 可以转换
loss.backward()
计算梯度值
loss值是标量张量
梯度值累加, 通过.grad.zero_() 清空上一次的梯度值
自动微分模块的真实应用场景，使用pytorch API来实现损失函数MSELoss
pytorch构建回归模型
准备数据集: 张量 - 数据集对象 - 数据加载器
将数据按batch_size划分,保存到对象中
遍历对象时, 将batch数据加载到内存中, 避免内存溢出
构建回归模型 nn.Linear()
构建MSE损失函数 nn.MSEloss()
构建优化器 optim.SGD(model.parameters(), lr=)
模型训练：遍历轮数
遍历批次，数据加载器
前向传播：数据输入模型得到预测值y
计算损失：调用MSE损失函数计算损失值
梯度清零：调用优化器对象进行梯度清零
反向传播：loss.backward()计算梯度
更新参数：优化器对象.step()
🧠 02人工神经网络基础.xmind 大纲
人工神经网络基础
神经网络介绍
人工神经元模型是什么
加权求和+激活函数，仿照生物神经元构造的计算模型
加权求和 z=wx+b(线性因素)
激活函数(引入非线性)
神经网络模型是什么
由多层神经网络层组成的计算模型, 神经网络层由多个神经元组成
神经网络的组成
输入层/数据
输入数据，没有网络参数，输入样本特征值 x, 一层
隐藏层
多层，提取复杂的特征值
输出层
1层，输出模型预测结果
激活函数
作用
引入非线性因素
sigmoid
1/(1+exp(-x))
激活值范围(0, 1), 没有负信号
导数范围(0, 0.25]
梯度消失 0.25**5
sigmoid(1-sigmoid)
只有输入在[-6,6]时，激活值才有明显变化
适用于二分类输出层
tanh
激活值范围(-1, 1), 正负信号都存在
导数范围(0, 1]
梯度消失 也存在
1-thah^2
只有输入在[-3,3], 激活值才有明显变化
适用于浅层神经网络隐藏层
RNN网络隐藏层默认
relu
max(0,x)，激活值范围[0,+∞)
导数0或1
1: 梯度永远为1, 收敛效果更好
0: 神经元死亡, 无法学习特征
隐藏层首选
DeadReLU问题，可以考虑relu变体
relu变体：leaky relu/ prelu
负区间有输出值
softmax
线性层输出值转换成概率, 概率和1
softmax(dim=-1)
通常神经网络训练中，张量的最后一个轴/维度代表特征/类别
适用于多分类输出层
参数初始化
作用
防止梯度消失或梯度爆炸
打破对称性
如果初始化时采用完全相同的w,b,则前向传播的预测值完全一样，反向传播计算的梯度完全一样，更新模型参数w新=w旧-学习率*梯度，每个神经元的网络参数完全一样，训练无效
随机初始化
均匀分布
默认值是(0, 1), 可以设置为(-1/sqrt(d), 1/sqrt(d))
正态分布
均值为0, 标准差为1
全0/1/指定值初始化
测试环境中可以使用
神经元学习到的特征一样, 对称性
kaiming初始化
均匀分布
(-sqrt(6/输入神经元个数), sqrt(6/输入神经元个数))
正态分布
w在(0, sqrt(2/输入神经元个数))
匹配relu
xavier初始化
均匀分布
(-sqrt(6/输入神经元个数+输出神经元个数), sqrt(6/输入神经元个数+输出神经元个数))
正态分布
w在(0, sqrt(2/输入神经元个数+输出神经元个数))
匹配sigmoid/tanh
重点：搭建神经网络模型
自定义模型类,继承nn.module基类
__init__构造函数实现网络层
初始化父类成员
初始化隐藏层和输出层 self.linear1 = nn.Linear()
可选：初始化网络参数 nn.init.kaiming_normal_(self.linear1.weight)
forward方法实现前向传播
前向传播, 计算预测y值
调用神经网络模型时, 会自动执行 model(data)
每一层加权求和
每一层激活函数计算
损失函数
什么是损失函数
衡量模型参数质量的函数，越小代表模型越好
损失函数, 代价函数, 误差函数, 目标函数...
根据损失函数计算损失值, 结合反向传播以及梯度下降法更新参数
分类损失函数
重点：多分类交叉熵损失函数
nn.CrossEntropyLoss(reduction='mean')
自带softmax概率计算+损失计算
了解：二分类交叉熵损失函数
nn.BCELoss(reduction='mean')
回归损失函数
重点：MSE均方误差
nn.MSEloss()
L2正则化, 将特征权重接近0
任意位置都有导数, loss越大导数越大
对异常样本效果不好, 放大样本误差
MAE平均绝对误差
nn.L1Loss()
L1 正则化, 将特征权重置为0, 特征筛选
导数为-1/1, 0点不可导用0代替
对异常样本效果比较好, 不会放大异常样本误差
Smooth L1/Huber
nn.SmoothL1loss()
MAE+MSE
任意位置都有导数, [-1,1]导数减小, 小于-1或大于1导数为-1或1
对异常样本效果好, 缓解梯度爆炸
网络参数优化方法
梯度下降法
重点：w1 = w0 - lr*grad
w0是上一版模型的参数
lr是学习率
grad是梯度,反向传播得到
epoch
训练轮数
batch_size
每次迭代需要的样本数
iteration
每次训练需要迭代多少次
样本数/batch_size[+1, 不能整除需要加1]
梯度下降法的优化方法
momentum动量法
当前梯度是指数移动加权平均梯度
St=(1-β)*Gt + β*St-1
结合当前梯度*系数+历史加权平均梯度
当前梯度为0, St = 0.1*0 + 0.9*St-1 != 0
W0 = W1 - lr*St
adagrad
自动调整学习率, 初始训练时学习率大, 后期训练学习率小
新学习率=原始学习率 / 历史梯度平方
初始训练累加梯度小, 分母小, 新的学习率大
后期训练累加梯度大, 分母大, 新的学习率小
适用于高维稀疏数据或文本数据
rmsprop
自动调整学习率, 初始训练时学习率大, 后期训练学习率小
新学习率=原始学习率 / 历史指数加权平均梯度平方
初始训练累加梯度小, 分母小, 新的学习率大
后期训练累加梯度大, 分母大, 新的学习率小
避免adagrad优化方法的学习率下降过快, 导致模型震荡
适用于高维稀疏数据或文本数据
重点：adam/adamW
momentum+rmsprop
优先选择
了解loss.backward()：反向传播算法BP
从输出层向输入层，反向逐层计算损失梯度
链式求导：梯度连乘
E/W = E/a * a/b * b/w
了解：学习率衰减/调度策略
等间隔学习率衰减
训练次数(步长)相等, 学习率随着训练次数减小
lr = lr * gamma
scheduer = optim.lr_scheduler.StepLR(optimizer=optimizer, step_size=50, gamma=0.5)
scheduer.step()
指定间隔学习率衰减
自定义训练次数(步长), 学习率随着训练次数减小
lr = lr * gamma
optim.lr_scheduler.MultiStepLR(optimizer=optimizer, milestones=[50, 100, 160], gamma=0.5, last_epoch=-1)
指数学习率衰减
按指数减小, 前期减小快, 后期减小慢
lr = lr*gamma**epoch
optim.lr_scheduler.ExponentialLR(optimizer=optimizer, gamma=0.9)
重点：周期重启的余弦退火策略
optim.lr_scheduler.CosineAnnealingWarmRestarts(optimizer, T_0, T_mult=1, eta_min=0, last_epoch=-1)
正则化
什么是正则化
一种防止过拟合提高模型泛化能力的策略
泛化能力/预测稳定性
模型在测试样本/新样本效果要好
策略
L1正则化
L2正则化
torch.optim.AdamW(weight_decay=)
重点：dropout层
以指定概率p让神经元随机失活
每批次训练时失活的神经元不固定, 得到不同的子网络, 防止预测结果过分依赖某些神经元
注意点
在激活层后使用
未失活的神经元输出 /(1-p)
dropout在测试集上不生效
模型状态切换
训练模式 model.train()
测试模式 model.eval()
API
nn.Dropout(p=0.4)
BN批量归一化层
对线性结果进行标准化处理
根据每批样本的均值和标准差计算标准化的值
作用
正则化
不同批次样本的均值和标准差不一样, 有可能会引入噪声样本, 降低训练模型效果
加快模型收敛速度
数据分布均匀, 落入不同激活函数要求的范围内
sigmoid/tanh激活函数->数据分布在0附近,导数最大
注意点
在激活层前使用(卷积层后/线性层后)
多数在计算机视觉领域使用
可以引入γ和β可学习参数, 不同层的样本分布在不同范围内(不同层使用的激活函数不同); 可以补回标准化丢失的信号
API
nn.BatchNorm1d(num_features=4)
nn.BatchNorm2d(num_features=3)
nn.BatchNorm3d(num_features=4)
重点：手机价格分类案例
需求
分类问题：预测手机价格区间
20个特征列, 4个类别标签
样本量2000：训练1600，测试400
一般工作流/实现步骤
准备数据集
加载csv文件得到df对象
获取df对象中的x和y
对df对象划分成训练集和测试集
创建张量数据集对象
搭建神经网络模型
自定义模型类, 继承nn.module基类
__init__方法
搭建神经网络模型
隐藏层
输出层
参数初始化
正则化
forward方法
前向传播计算预测y值
隐藏层和输出层进行加权求和和激活值计算
模型训练
创建张量数据加载器, min-batch
创建神经网络模型对象 初版模型
创建优化器对象
创建损失函数对象
遍历epoch轮数
遍历批次，数据加载器, 获取batch样本
切换模型训练模式 model.train()
前向传播，模型预测, 得到预测y
计算损失值
梯度清零
反向传播
更新参数
模型保存
将模型的参数字典保存到pth文件中
模型测试
创建测试集数据加载器
创建模型对象加载pth文件中的参数字典
循环遍历数据加载器实现预测
切换模型推理模式 model.eval()
模型预测, 计算得到预测y(线性输出的分值)
torch.argmax(output, dim=-1)
获取线性输出最大分值对应的下标(y标签)
计算准确率
关闭梯度计算：@torch.no_grad()
ℹ️ 离线版不含附件原文件，需要请到在线网站对应课时下载。
📎 阶段作业
✏️ 作业
深度学习基础代码作业
作业答案
day01
- 1.使用torch.tensor()、torch.arange()和torch.randn()分别创建一个形状为(3, 4)的张量，并打印。
- 2.创建一个形状为(5,10)的全0张量 和 全1张量，并打印。
- 3.使用torch.randint()创建一个形状为(3, 4)的随机整数张量,并将元素类型转换为torch.float32，并打印。
- 4.使用torch.rand()创建一个形状为(3, 4)的张量，然后转换为numpy数组，然后更改其中的一个元素为42，然后再转换为张量，并打印。
- 5.创建一个标量张量，并转换为 python数值，并打印。
# 1.使用torch.tensor()、torch.arange()和torch.randn()分别创建一个形状为(3, 4)的张量，并打印
import torch
# 创建一个形状为(3, 4)的张量
t1 = torch.tensor([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
print(f"t1: {t1}")
# 使用torch.arange()创建一个形状为(3, 4)的张量
t2 = torch.arange(1, 13).reshape(3, 4)
print(f"t2: {t2}")
# 使用torch.randn()创建一个形状为(3, 4)的张量
t3 = torch.randn(3, 4)
print(f"t3: {t3}")
t1: tensor([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
t2: tensor([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
t3: tensor([[ 0.5081, -0.2216, -0.2112, -0.8687],
[ 0.1202, 2.4337, -1.0001, -0.5479],
[-0.1211, -0.2320, -0.5798, -0.0413]])
# 2.创建一个形状为(5,10)的全0张量 和 全1张量，并打印。
import torch
# 创建一个形状为(5,10)的全0张量
t1 = torch.zeros(5, 10)
print(f"t1: {t1}")
# 创建一个形状为(5,10)的全1张量
t2 = torch.ones(5, 10)
print(f"t2: {t2}")
t1: tensor([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]])
t2: tensor([[1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1., 1., 1., 1., 1., 1.]])
# 3.使用torch.randint()创建一个形状为(3, 4)的随机整数张量,并将元素类型转换为torch.float32，并打印。
import torch
# 创建一个形状为(3, 4)的随机整数张量
t1 = torch.randint(0, 10, (3, 4))
# 将元素类型转换为torch.float32
# t1 = t1.type(torch.float32)
t1 = t1.to(torch.float32)
# 打印张量
print(t1)
tensor([[2., 4., 9., 9.],
[8., 5., 8., 4.],
[7., 3., 9., 3.]])
# 4.使用torch.rand()创建一个形状为(3, 4)的张量，然后转换为numpy数组，然后更改其中的一个元素为42，然后再转换为张量，并打印。
import torch
# 创建一个形状为(3, 4)的随机张量
t1 = torch.rand(3, 4)
print(f"原始张量: {t1}")
# 将张量转换为numpy数组
t1_np = t1.numpy()
print(f"numpy数组: {t1_np}")
# 更改numpy数组中的某个元素为42
t1_np[0, 0] = 42
print(f"更改后的numpy数组: {t1_np}")
# 将numpy数组转换为张量
t1 = torch.tensor(t1_np)
print(f"转换后的张量: {t1}")
原始张量: tensor([[0.2265, 0.5340, 0.4939, 0.7156],
[0.7442, 0.6439, 0.3041, 0.5015],
[0.3087, 0.5015, 0.0788, 0.6442]])
numpy数组: [[0.22649407 0.5339812 0.49387354 0.71557003]
[0.7442052 0.6439322 0.30407512 0.50146997]
[0.30869353 0.5014638 0.07881838 0.644166 ]]
更改后的numpy数组: [[42. 0.5339812 0.49387354 0.71557003]
[ 0.7442052 0.6439322 0.30407512 0.50146997]
[ 0.30869353 0.5014638 0.07881838 0.644166 ]]
转换后的张量: tensor([[42.0000, 0.5340, 0.4939, 0.7156],
[ 0.7442, 0.6439, 0.3041, 0.5015],
[ 0.3087, 0.5015, 0.0788, 0.6442]])
# 5.创建一个标量张量，并转换为 python数值，并打印。
import torch
t1 = torch.tensor(10)
print(f"t1: {t1}")
print(f"t1.item(): {t1.item()}")
t1: 10
t1.item(): 10
day02
- 1.使用torch.tensor()创建两个形状为(3,4)的张量t1和t2，计算(t1+t2-2)*2, 并打印。
- 2.使用torch.arange创建一个形状为(3,4)的张量t1,创建一个形状为(3,4)的随机张量t2，计算t1*t2,然后将t2转换形状为(4,3)，注意不要改变t2中元素顺序，计算t1@t2,并打印。
- 3.使用torch.randint()创建一个形状为(3,4)的随机整数张量,求所有元素的均值 和 第1个轴的均值，并打印。(注意元素类型转换)
- 4.使用torch.randint()创建一个形状为(3,4)的张量，获取第2个轴的 索引为1的所有数据，然后获取第二个轴的 索引为1的 大于5的元素 对应的 第一个轴的数据，并打印。
- 5.创建一个两个形状为(3,4)的随机张量t1和t2，都转换形状为(4,3), 然后按0轴进行拼接，得到形状为(8,3)的张量，并打印。
- 6.创建一个形状为(3)的随机张量w，开启requires_grad, 计算loss = 0.5*w**2, 使用loss.mean().backward()获取梯度，并打印梯度w.grad。（尝试手动计算梯度）
# 1.使用torch.tensor()创建两个形状为(3, 4)的张量t1和t2，计算(t1+t2-2)*2, 并打印。
import torch
t1 = torch.tensor(
[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]
)
t2 = torch.tensor(
[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]
)
t3 = (t1 + t2 - 2) * 2
print(f"(t1 + t2 - 2) * 2:\n {t3}")
(t1 + t2 - 2) * 2:
tensor([[ 0, 4, 8, 12],
[16, 20, 24, 28],
[32, 36, 40, 44]])
# 2.使用torch.arange创建一个形状为(3,4)的张量t1,创建一个形状为(3,4)的随机张量t2，计算t1*t2,然后将t2转换形状为(4,3)，注意不要改变t2中元素顺序，计算t1@t2,并打印。
import torch
t1 = torch.arange(12).reshape(3, 4)
t2 = torch.randint(1,10, (3, 4))
print(f"t1: {t1}")
print(f"t2: {t2}")
t3 = t1 * t2
print(f"t1*t2: {t3}")
t2 = t2.reshape(4, 3)
t4 = t1 @ t2
print(f"t1@t2: {t4}")
t1: tensor([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
t2: tensor([[7, 9, 1, 5],
[5, 2, 5, 3],
[7, 1, 8, 5]])
t1*t2: tensor([[ 0, 9, 2, 15],
[20, 10, 30, 21],
[56, 9, 80, 55]])
t1@t2: tensor([[ 18, 35, 31],
[ 90, 135, 91],
[162, 235, 151]])
# 3.使用torch.randint()创建一个形状为(3, 4)的随机整数张量,求所有元素的均值 和 第1个轴的均值，并打印。(注意元素类型转换)
import torch
# 创建一个形状为(3, 4)的随机整数张量
t1 = torch.randint(0, 10, (3, 4))
t1 = t1.to(torch.float32)
# 计算所有元素的均值
mean_all = torch.mean(t1)
print(f"所有元素的均值: {mean_all}")
# 计算第1个轴 / 0轴 的均值
mean_axis_0 = torch.mean(t1, dim=0)
print(f"第1个轴的均值: {mean_axis_0}")
所有元素的均值: 4.916666507720947
第1个轴的均值: tensor([3.6667, 5.0000, 6.0000, 5.0000])
# 4.使用torch.randint()创建一个形状为(3, 4)的张量，获取第2个轴的 索引为1的所有数据，然后获取第二个轴的 索引为1的 大于5的元素 对应的 第一个轴的数据，并打印。
import torch
# 创建一个形状为(3, 4)的随机张量
t1 = torch.randint(0, 10, (3, 4))
print(f"t1: {t1}")
# 获取第2个轴的 索引为1的所有数据
t1_2 = t1[:, 1]
print(f"第2个轴的 索引为1的所有数据: {t1_2}")
# 获取第二个轴的 大于5的元素 对应的 所有轴的数据
t2 = t1[t1[:, 1] > 5]
print(f"第二个轴的 索引为1的 大于5的元素 对应的 第一个轴的数据: {t2}")
t1: tensor([[3, 8, 1, 8],
[7, 3, 2, 0],
[7, 5, 7, 2]])
第2个轴的 索引为1的所有数据: tensor([8, 3, 5])
第二个轴的 索引为1的 大于5的元素 对应的 第一个轴的数据: tensor([[3, 8, 1, 8]])
# 5.创建两个形状为(3,4)的随机张量t1和t2，都转换形状为(4,3), 然后按0轴进行拼接，得到形状为(8,3)的张量，并打印。
import torch
# 创建两个形状为(3,4)的随机张量
t1 = torch.rand(3, 4)
t2 = torch.rand(3, 4)
print(f"t1: {t1}")
print(f"t2: {t2}")
# 转换形状为(4,3)
t1 = t1.reshape(4, 3)
t2 = t2.reshape(4, 3)
# 按0轴进行拼接
t3 = torch.cat([t1, t2], dim=0)
print(f"t3: {t3}")
print(f"t3.shape: {t3.shape}")
t1: tensor([[0.8463, 0.3997, 0.3458, 0.2899],
[0.2253, 0.5289, 0.4712, 0.9801],
[0.3612, 0.9313, 0.7176, 0.0849]])
t2: tensor([[0.1855, 0.1685, 0.6392, 0.5308],
[0.1238, 0.4026, 0.0390, 0.1994],
[0.1992, 0.9355, 0.3620, 0.9237]])
t3: tensor([[0.8463, 0.3997, 0.3458],
[0.2899, 0.2253, 0.5289],
[0.4712, 0.9801, 0.3612],
[0.9313, 0.7176, 0.0849],
[0.1855, 0.1685, 0.6392],
[0.5308, 0.1238, 0.4026],
[0.0390, 0.1994, 0.1992],
[0.9355, 0.3620, 0.9237]])
t3.shape: torch.Size([8, 3])
# 6.创建一个形状为(3)的随机张量w，开启requires_grad, 计算loss = 0.5*w**2, 使用loss.mean().backward()获取梯度，并打印梯度。
import torch
# 创建一个形状为(3)的随机张量w，开启requires_grad
w = torch.randn(3, requires_grad=True)
print(f"w: {w}")
print(f"w.shape: {w.shape}")
print(f"w.grad: {w.grad}")
# 计算loss = 0.5*w**2
loss = 0.5 * w ** 2
# 使用loss.mean().backward()获取梯度
loss.mean().backward()
# 打印梯度
print(f"梯度: {w.grad}")
w: tensor([-0.7115, 1.2408, -0.7791], requires_grad=True)
w.shape: torch.Size([3])
w.grad: None
梯度: tensor([-0.2372, 0.4136, -0.2597])
day03
- 1.使用torch.randn创建形状为(3,4)的随机张量，开启梯度计算，使用detach()方法将张量转换为NumPy数组, 并打印。
- 2.修改pytorch模拟线性回归中的参数或设置，比如n_samples改为200，batch_size改为32，epochs改为200，学习率改为0.001，loss_fn改为MAE，并绘制训练过程中损失值的变化曲线。
- 3.绘制 prelu的激活函数图像 和 导数图像。
- 4.使用代码演示 二分类任务中，softmax和sigmoid函数的等价关系，比如原始输出的logits为：两个类别[1,3]，单个类别为[2],分别使用softmax和sigmoid计算正类别和负类别的概率，并打印。
- 5.创建一个线性层，输入维度3，输出维度5，权重初始化采用kaiming正态分布，偏置初始化采用全0初始化，并打印初始化的权重和偏置。
# 1.使用torch.randn创建形状为(3,4)的随机张量，开启梯度计算，使用detach()方法将张量转换为NumPy数组并打印出来。
import torch
# 创建形状为(3,4)的随机张量
t1 = torch.randn(3, 4, requires_grad=True)
# 将张量转换为NumPy数组并打印出来
t1_np = t1.detach().numpy()
print(f"t1_np: \n{t1_np}")
t1_np:
[[-0.85907346 1.0835776 -0.8895768 0.00883052]
[ 1.3591934 1.233448 -0.6031356 0.31076854]
[ 0.17767143 -1.4833153 0.30477268 0.5143065 ]]
# 2.修改pytorch模拟线性回归中的参数或设置，比如n_samples改为500，batch_size改为32，epochs改为200，学习率改为0.001，loss_fn改为MAE，并绘制训练过程中损失值的变化曲线。
# 导入相关模块
import torch
# numpy对象 -> 张量 -> 数据集对象TensorDataset -> 数据加载器DataLoader
from torch.utils.data import TensorDataset # 构造数据集对象
from torch.utils.data import DataLoader # 数据加载器
from torch import nn # nn模块中有平方损失函数和假设函数
from torch import optim # optim模块中有优化器函数
from sklearn.datasets import make_regression # 创建线性回归模型数据集
import matplotlib.pyplot as plt
plt.rcParams['font.sans-serif'] = ['SimHei'] # 用来正常显示中文标签
plt.rcParams['axes.unicode_minus'] = False # 用来正常显示负号
# 1.定义函数，创建线性回归数据
def create_dataset():
# 创建数据集对象
x, y, coef = make_regression(
n_samples=200, # 100条样本
n_features=1, # 1个特征
noise=10, # 噪声
bias=13.9, # 偏移
coef=True, # 是否返回系数
random_state=6 #
)
# 封装成 张量
x = torch.tensor(x, dtype=torch.float32)
y = torch.tensor(y, dtype=torch.float32)
# 返回结果
return x, y, coef
# 2. 定义函数，用于模型训练
def train(x, y, coef):
# 1. 构建DataLoader, tensor -> 数据集对象 -> 数据加载器
dataset = TensorDataset(x, y)
# 参数1：数据集对象，参数2：批次大小，参数3：是否打乱数据，训练集打乱，测试集不打乱，参数4：是否去掉最有一个不完整的批次
dataloader = DataLoader(dataset, batch_size=32, shuffle=True, drop_last=False) # 100个样本，批次数16，6*16，4
# 2. 创建模型，线性回归模型
model = nn.Linear(1, 1)
# 3. 创建 损失函数 和 优化器
loss_fn = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=1e-3)
# 4.训练过程
# 定义训练轮数
epochs = 200
# 记录 训练过程的 平均损失值
total_train_loss_list = []
# 遍历epoch
for epoch in range(epochs):
# 计算 每轮 的总的平均损失 和 总的样本数
total_train_loss = 0
total_train_sample = 0
# 分 批次 训练
for train_x, train_y in dataloader:
# 前向传播，模型预测
y_pred = model(train_x) # [batch_size, 1]
# 计算损失值
loss = loss_fn(y_pred, train_y.reshape(-1, 1))
# 计算 总损失 和 总样本数
batch_size = train_x.shape[0]
total_train_loss += loss.item() * batch_size
total_train_sample += batch_size
# 梯度清零
optimizer.zero_grad()
# 反向传播，计算梯度
loss.backward()
# 更新模型参数
optimizer.step()
# 添加 本epoch 的平均损失 到 total_train_loss_list
train_loss_mean = total_train_loss / total_train_sample
total_train_loss_list.append(train_loss_mean)
# print(f"epoch: {epoch}, 平均损失值: {train_loss_mean}")
# 打印整个 训练损失记录
# print(f"全部{epochs}轮的平均损失为：{total_train_loss_list}")
print(f"模型参数：权重：{model.weight}, 偏置：{model.bias}")
# 5. 可视化
plt.figure(figsize=[15, 6])
plt.subplot(1, 2, 1)
# 绘制损失曲线
plt.title("训练损失曲线")
x_range = range(epochs)
plt.plot(x_range, total_train_loss_list, label="train loss")
# 添加网格
plt.grid(True)
# 添加X轴标签
plt.xlabel("epoch")
plt.ylabel("loss")
# 添加曲线标签
plt.legend()
# 绘制预测值与真实值的对比图
plt.subplot(1, 2, 2)
plt.title('预测值和真实值的对比图')
plt.scatter(x, y, label="真实数据点")
# 计算真实的理论直线的y值
y_true = [v*coef.item() + 13.8 for v in x]
plt.plot(x, y_true, label="真实值")
# 计算预测值
# 关闭自动微分
with torch.no_grad():
y_pred = model(x).detach().numpy()
# y_pred = [v*model.weight.item() + model.bias.item() for v in x]
plt.plot(x, y_pred, label="预测值")
# 添加网格
plt.grid(True)
# 添加X轴标签
plt.xlabel("x")
plt.ylabel("y")
# 添加曲线标签
plt.legend()
plt.show()
# 测试
if __name__ == "__main__":
x, y, coef = create_dataset()
print(f"x.shape: {x.shape}, y.shape: {y.shape}")
# 训练模型
train(x, y, coef)
x.shape: torch.Size([200, 1]), y.shape: torch.Size([200])
模型参数：权重：Parameter containing:
tensor([[76.4290]], requires_grad=True), 偏置：Parameter containing:
tensor([13.7581], requires_grad=True)
# 3.绘制 prelu的激活函数图像 和 导数图像。
import torch
import matplotlib.pyplot as plt
# 1. 创建画布和坐标轴
fig, axes = plt.subplots(1, 2, figsize=(12, 5))
# -------------------------
# 第 1 个图：pReLU 函数本身
# -------------------------
x = torch.linspace(-10, 10, 1000)
a = torch.tensor([0.1]) # pReLU 的参数
y = torch.prelu(x, a)
axes[0].plot(x, y)
axes[0].set_title("pReLU 激活函数")
axes[0].grid()
# -------------------------
# 第 2 个图：pReLU 的导数
# -------------------------
x = torch.linspace(-10, 10, 1000, requires_grad=True)
a = torch.tensor([0.1])
y = torch.prelu(x, a)
# 求导
y.sum().backward()
axes[1].plot(x.detach(), x.grad)
axes[1].set_title("pReLU 激活函数导数")
axes[1].grid()
plt.tight_layout()
plt.show()
# 4.使用代码演示 二分类任务中，softmax和sigmoid函数的等价关系，比如原始输出的logits为：两个类别[1,3]，单个类别为[2],分别使用softmax和sigmoid计算正类别和负类别的概率，并打印。
import torch
logits = torch.tensor([[1, 3]], dtype=torch.float)
print(logits)
softmax = torch.softmax(logits, dim=1)
print(softmax)
logits = logits[:,1]-logits[:,0]
sigmoid = torch.sigmoid(logits)
print(sigmoid)
tensor([[1., 3.]])
tensor([[0.1192, 0.8808]])
tensor([0.8808])
# 5.创建一个线性层，输入维度3，输出维度5，权重初始化采用kaiming正态分布，偏置初始化采用全0初始化，并打印初始化的权重和偏置。
import torch
import torch.nn as nn
# 创建一个线性层，输入维度3，输出维度5
linear = nn.Linear(3, 5, bias=True)
# 对权重w 进行初始化
nn.init.kaiming_normal_(linear.weight)
# 对偏置b 进行初始化
nn.init.zeros_(linear.bias)
# 打印
print(linear.weight.data)
print(linear.bias.data)
tensor([[ 0.5717, -0.0778, -1.3657],
[-0.3323, 0.0229, -1.3459],
[-1.0581, 0.8332, -0.0232],
[ 0.7131, 1.6377, 0.3035],
[-0.8480, 0.7763, 0.2937]])
tensor([0., 0., 0., 0., 0.])
day04
- 1.搭建一个神经网络，有3个隐藏层和1个输出层，输入数据的输入特征维度为20，3个隐藏层的神经元数量分别为128，64，32，输出层的神经元数量为4。激活函数要求为：隐藏层使用relu激活函数，输出层使用softmax激活函数。并打印模型结构 和 模型参数。
- 2.使用代码演示 二分类任务中，分别使用 多分类交叉熵CrossEntropyLoss 和 二分类交叉熵BCELoss，比如原始输出的logits为：两个类别[[3,1], [4,3], [2,4]]，单个类别为[2,1,-2],分别计算对应的 多分类交叉熵损失值 和 二分类交叉熵损失值，并打印。
- 3.使用pytorch计算MSE损失值。已知 y_ture 为 [1, 2, 3, 4, 5, 6], y_pred 为 [2, 3, 4, 5, 6, 7]，计算MSE损失值，并打印。
- 4.使用梯度下降法，使用optim.SGD实现，循环迭代800次，求 loss = 0.5*w**2 + 10 的极小值点, 并打印loss最小值，以及loss是最小值时 w的值 和 梯度。
- 5.使用Adam优化器，使用optim.Adam实现，循环迭代800次，求 loss = 0.5*w**2 + 10 的极小值点, 并打印loss最小值，以及loss是最小值时 w的值 和 梯度。
- 6.使用AdamW优化器，使用optim.Adamw实现，循环迭代800次，求 loss = 0.5*w**2 + 10 的极小值点, 并打印loss最小值，以及loss是最小值时 w的值 和 梯度。
# 1.搭建一个神经网络，有3个隐藏层和1个输出层，输入数据的输入特征维度为20，3个隐藏层的神经元数量分别为128，64，32，输出层的神经元数量为4。
# 激活函数要求为：隐藏层使用relu激活函数，输出层使用softmax激活函数。并打印模型结构 和 模型参数。
# 导包
import torch
import torch.nn as nn
from torchsummary import summary # 计算模型参数,查看模型结构 pip install torchsummary -i https://mirrors.aliyun.com/pypi/simple/
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
# 1.搭建神经网络
class ModelDemo(nn.Module):
# 1.1 定义__init__方法，搭建网络层，并且进行参数初始化
def __init__(self):
# 初始化父类成员
super().__init__()
# 搭建网络层
# 隐藏层1,输入特征数 20，输出特征数 128
self.linear1 = nn.Linear(20, 128)
# 隐藏层2，输入特征数 128，输出特征数 64
self.linear2 = nn.Linear(128, 64)
# 隐藏层3，输入特征数 64，输出特征数 32
self.linear3 = nn.Linear(64, 32)
# 输出层，输入特征数 32，输出特征数 4
self.output = nn.Linear(32, 4)
# 参数初始化
# 初始化隐藏层1
nn.init.xavier_normal_(self.linear1.weight)
nn.init.zeros_(self.linear1.bias)
# 初始化隐藏层2
nn.init.kaiming_normal_(self.linear2.weight)
nn.init.zeros_(self.linear2.bias)
# 初始化输出层
nn.init.xavier_normal_(self.output.weight)
nn.init.zeros_(self.output.bias)
# 1.2 定义forward方法，实现前向传播
def forward(self, x):
# 输入隐藏层1，加权求和 + 激活函数
x = torch.relu(self.linear1(x))
# 输入隐藏层2，加权求和 + 激活函数
x = torch.relu(self.linear2(x))
# 输入隐藏层3，加权求和 + 激活函数
x = torch.relu(self.linear3(x))
# 输入输出层，加权求和 + 激活函数
x = torch.softmax(self.output(x), dim=-1)
# 返回预测值
return x
# 2.模型训练
def train():
# 2.1 创建模型对象ModelDemo
my_model = ModelDemo()
print("查看模型结构：")
print(my_model)
# 2.2 创建数据集
data = torch.randn(size=(10, 20))
print(f"data: {data}")
print(f"data.shape: {data.shape}")
print(f"data.requires_grad: {data.requires_grad}")
# 2.3 前向传播，模型预测
output = my_model(data) # 底层自动调用forward方法, 执行前向传播
print(f"output: {output}")
print(f"output.shape: {output.shape}")
print(f"output.requires_grad: {output.requires_grad}")
print("-" * 60)
# 2.4 计算 查看 模型参数
print("计算模型参数：")
# 参数1：模型对象；参数2：输入数据形状，10*20
my_model.to(device)
summary(my_model, input_size=(10, 20))
print("查看模型参数：")
for name,param in my_model.named_parameters():
print(f"name: {name}")
print(f"param: {param}")
# 3.测试
if __name__ == '__main__':
train()
查看模型结构：
ModelDemo(
(linear1): Linear(in_features=20, out_features=128, bias=True)
(linear2): Linear(in_features=128, out_features=64, bias=True)
(linear3): Linear(in_features=64, out_features=32, bias=True)
(output): Linear(in_features=32, out_features=4, bias=True)
)
data: tensor([[ 0.7222, -0.0863, 0.5356, 0.6470, 0.4322, 0.4818, -0.7776, 0.4272,
1.7723, -1.4479, 1.1387, 1.1514, 1.4133, -1.6729, 0.8887, -0.3169,
0.6311, -0.9161, 0.7694, 0.9562],
[-0.0066, 0.9702, 0.2877, -0.8346, 1.9843, 0.1039, 1.5156, 0.8568,
0.8353, 1.1827, 0.4811, -0.4231, 0.0559, -0.3341, -2.3958, -0.5815,
0.9180, -2.0646, 1.6093, -1.2262],
[-0.7296, 0.9468, -0.3157, 0.6687, 1.7211, -1.7048, 0.1297, 0.8802,
-0.4132, -0.7198, -0.3060, 0.9133, 2.1737, 1.0186, -1.9547, -0.6534,
1.3868, -0.0124, 1.5287, -0.7931],
[-1.4792, 1.4121, 1.4354, -1.7197, -0.6993, -0.1139, -0.9799, 0.3609,
0.8258, 0.0564, -0.6916, -1.7462, -2.6413, 0.3140, 0.4434, -0.8825,
-0.3473, -0.4817, -0.5086, 0.7418],
[-0.9664, -1.3159, -0.5170, 0.0229, -2.3985, 1.1254, 1.3761, -0.6514,
-0.7067, -0.3633, -0.9831, -1.1118, 0.5575, -0.4361, 1.2631, 0.3355,
-0.2688, 0.0861, -1.6839, 1.8270],
[ 1.3895, 1.5704, 0.7095, -0.2546, -0.0447, 1.2636, -0.1387, -1.1365,
-0.1842, 0.0393, 0.1378, 0.3613, 0.7736, -0.9648, -0.0535, -0.3629,
-0.5721, 1.1692, -0.6664, -1.0393],
[ 1.0083, -0.1898, -0.1666, -0.7376, -1.0179, -2.1106, 1.4209, -0.5041,
0.4916, 1.3665, 0.9918, 0.5540, 0.6099, -0.7724, -0.6299, 0.0195,
-1.8812, 1.4333, -0.4333, -0.4589],
[-0.9778, -0.2505, -0.2861, -1.2180, 0.1773, 1.2218, -0.1429, -2.0637,
-0.5776, 1.1070, 1.2113, 1.5190, -0.4912, -0.5720, 0.4268, 0.0361,
-0.0939, -0.3756, 1.2241, 0.5089],
[-1.7184, -1.5688, -0.2747, 0.8946, 0.3975, 0.3351, -1.9165, 0.8653,
1.3837, 0.6506, 1.6941, 0.4650, 2.4609, -1.2372, 0.7207, 0.1670,
1.6150, -0.9680, -0.5443, -0.4207],
[ 0.8018, -0.7720, 1.1704, 0.1219, 0.9244, 0.6195, -3.0844, 1.9890,
-0.8557, 0.8801, 2.4049, 0.2876, -0.1910, 1.0796, -0.7060, 0.3741,
1.9421, 0.9616, 0.2133, 0.9347]])
data.shape: torch.Size([10, 20])
data.requires_grad: False
output: tensor([[0.2306, 0.3313, 0.2732, 0.1649],
[0.2333, 0.3103, 0.2911, 0.1653],
[0.1898, 0.3517, 0.2753, 0.1832],
[0.2292, 0.3664, 0.2637, 0.1408],
[0.1914, 0.3519, 0.3073, 0.1494],
[0.2339, 0.3018, 0.2789, 0.1854],
[0.1594, 0.3846, 0.3095, 0.1465],
[0.2339, 0.3517, 0.2739, 0.1405],
[0.2374, 0.3553, 0.2432, 0.1641],
[0.2689, 0.3402, 0.2550, 0.1359]], grad_fn=<SoftmaxBackward0>)
output.shape: torch.Size([10, 4])
output.requires_grad: True
------------------------------------------------------------
计算模型参数：
----------------------------------------------------------------
Layer (type) Output Shape Param #
================================================================
Linear-1 [-1, 10, 128] 2,688
Linear-2 [-1, 10, 64] 8,256
Linear-3 [-1, 10, 32] 2,080
Linear-4 [-1, 10, 4] 132
================================================================
Total params: 13,156
Trainable params: 13,156
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.02
Params size (MB): 0.05
Estimated Total Size (MB): 0.07
----------------------------------------------------------------
查看模型参数：
name: linear1.weight
param: Parameter containing:
tensor([[ 0.0970, 0.0855, -0.0611, ..., 0.0448, 0.0180, 0.0465],
[ 0.0419, 0.0806, 0.2274, ..., 0.0700, 0.0055, 0.0337],
[ 0.1548, -0.1215, -0.0058, ..., -0.0043, 0.2355, -0.0220],
...,
[-0.0980, 0.0343, -0.1124, ..., 0.1626, 0.0046, 0.0468],
[ 0.0029, -0.1358, -0.1149, ..., -0.0369, -0.0391, 0.0790],
[ 0.0543, 0.2061, -0.1433, ..., 0.1157, 0.0934, 0.1384]],
device='cuda:0', requires_grad=True)
name: linear1.bias
param: Parameter containing:
tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0.], device='cuda:0', requires_grad=True)
name: linear2.weight
param: Parameter containing:
tensor([[-0.2266, 0.2350, -0.1647, ..., 0.0821, 0.0954, 0.2066],
[ 0.0241, 0.1515, 0.1049, ..., -0.1931, -0.0177, -0.0244],
[-0.0572, -0.1117, 0.0932, ..., -0.0845, 0.0368, -0.1538],
...,
[ 0.2678, 0.1597, 0.1124, ..., 0.0255, 0.0058, 0.1341],
[-0.1320, 0.2293, 0.2222, ..., -0.2828, 0.1095, -0.0857],
[ 0.0711, -0.0697, 0.0811, ..., -0.0872, -0.1307, 0.1419]],
device='cuda:0', requires_grad=True)
name: linear2.bias
param: Parameter containing:
tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
device='cuda:0', requires_grad=True)
name: linear3.weight
param: Parameter containing:
tensor([[ 0.1118, 0.0029, -0.1118, ..., 0.0698, -0.1208, -0.0676],
[-0.1060, -0.0712, 0.0464, ..., 0.0650, 0.0687, 0.1015],
[ 0.0430, -0.0873, 0.0759, ..., -0.0639, 0.0143, 0.1047],
...,
[-0.0564, 0.0564, 0.0747, ..., -0.0459, 0.0229, -0.0639],
[-0.0459, 0.0059, -0.0845, ..., 0.1211, 0.0231, -0.1027],
[-0.0260, 0.0935, -0.0812, ..., 0.1056, -0.0500, 0.1228]],
device='cuda:0', requires_grad=True)
name: linear3.bias
param: Parameter containing:
tensor([-0.0117, 0.1063, 0.0107, -0.0916, -0.0816, 0.1044, -0.0399, -0.0129,
-0.0823, 0.1182, -0.1186, 0.0661, 0.0077, -0.0874, 0.0205, -0.0349,
-0.0220, 0.0624, 0.1220, 0.0773, 0.1086, 0.0973, 0.0833, 0.0550,
0.0207, -0.0176, -0.0507, -0.1212, -0.0415, 0.1025, 0.0176, 0.0105],
device='cuda:0', requires_grad=True)
name: output.weight
param: Parameter containing:
tensor([[ 0.2973, 0.1266, 0.0873, -0.5482, 0.2228, -0.2754, -0.2585, -0.1005,
-0.0771, -0.3397, 0.3239, -0.1660, -0.1010, 0.0278, 0.1315, 0.0336,
-0.1121, 0.1414, 0.2368, 0.2339, -0.1915, -0.3699, -0.3244, -0.4605,
0.1247, -0.1190, 0.0061, 0.3321, 0.1629, 0.2699, -0.1127, 0.5832],
[ 0.1846, -0.2821, -0.1558, -0.2147, -0.0241, 0.2201, 0.0849, 0.1045,
-0.0028, 0.2088, 0.0071, -0.1830, -0.0200, -0.0178, -0.0116, 0.1165,
0.0257, -0.2384, 0.3226, 0.1483, 0.1442, 0.7120, 0.4201, 0.1805,
-0.3795, 0.0289, -0.0815, 0.3679, -0.1334, -0.0531, 0.3140, 0.0128],
[ 0.5472, 0.0324, 0.5694, -0.2798, 0.5592, 0.3089, 0.4404, -0.0490,
0.2804, -0.1889, -0.1140, 0.0887, -0.0787, 0.3601, 0.2508, -0.0791,
0.2545, -0.1007, 0.3294, 0.0665, 0.2633, 0.3503, -0.1244, -0.1213,
-0.2067, -0.4668, -0.1748, -0.1859, 0.1820, -0.3398, 0.0034, -0.0975],
[ 0.4904, 0.3744, 0.1387, -0.1629, -0.0548, 0.1310, -0.0254, -0.2700,
0.0400, -0.1855, 0.2361, -0.0216, 0.1238, -0.0948, -0.2460, -0.3875,
-0.0354, -0.0374, -0.4726, -0.4871, 0.0747, 0.1405, 0.3730, 0.5118,
0.0252, 0.0100, 0.1564, 0.2469, -0.2971, -0.0988, -0.0510, -0.1716]],
device='cuda:0', requires_grad=True)
name: output.bias
param: Parameter containing:
tensor([0., 0., 0., 0.], device='cuda:0', requires_grad=True)
# 2.使用代码演示 二分类任务中，分别使用 多分类交叉熵CrossEntropyLoss 和 二分类交叉熵BCELoss，
# 比如原始输出的logits为：两个类别[[3,1], [4,3]]，单个类别为[2,1],分别计算对应的 多分类交叉熵损失值 和 二分类交叉熵损失值，并打印。
import torch
import torch.nn as nn
# 定义函数，计算 多分类交叉熵损失值
def demo01():
print("二分类任务使用多分类交叉熵损失")
# 1. 创建 样本的真实类别
y_true = torch.tensor([0, 0, 1])
# 2.创建 模型预测 的原始输出logits值
y_logits = torch.tensor([[3,6], [4,3], [2,4]], dtype=torch.float)
# 3.创建 交叉熵损失函数
loss_fn = nn.CrossEntropyLoss()
# 4.计算损失
loss = loss_fn(y_logits, y_true)
print(f"loss: {loss}")
# 5.计算预测概率
y_pred = torch.softmax(y_logits, dim=-1)
print(f"y_pred: {y_pred}")
# 定义函数，计算 二分类交叉熵损失值
def demo02():
print("二分类任务使用二分类交叉熵损失")
# 1. 创建 样本的真实类别
y_true = torch.tensor([0, 0, 1], dtype=torch.float)
# 2.创建 模型预测 的原始输出logits值
y_logits = torch.tensor([3,-1,2], dtype=torch.float)
# 经过sigmoid,转化为正类别的 概率
y_pred = torch.sigmoid(y_logits)
# 3.创建 交叉熵损失函数
loss_fn = nn.BCELoss()
# 4.计算损失
loss = loss_fn(y_pred, y_true)
print(f"loss: {loss}")
print(f"y_pred: {y_pred}")
# 测试
if __name__ == '__main__':
demo01()
demo02()
二分类任务使用多分类交叉熵损失
loss: 1.1629257202148438
y_pred: tensor([[0.0474, 0.9526],
[0.7311, 0.2689],
[0.1192, 0.8808]])
二分类任务使用二分类交叉熵损失
loss: 1.1629258394241333
y_pred: tensor([0.9526, 0.2689, 0.8808])
# 2.使用代码演示 二分类任务中，分别使用 多分类交叉熵CrossEntropyLoss 和 二分类交叉熵BCELoss，
# 比如原始输出的logits为：两个类别[[3,1], [4,3]]，单个类别为[2,1],分别计算对应的 多分类交叉熵损失值 和 二分类交叉熵损失值，并打印。
import torch
import torch.nn as nn
# 定义函数，计算 多分类交叉熵损失值
def demo01():
print("多分类交叉熵损失")
# 1. 创建 样本的真实类别
# y_true = torch.tensor([[0,0,1], [0,1,0]], dtype=torch.float)
y_true = torch.tensor([0, 0, 1])
# 2.创建 模型预测 的原始输出logits值
y_logits = torch.tensor([[3,1], [4,3], [2,4]], dtype=torch.float)
# 3.创建 交叉熵损失函数
loss_fn = nn.CrossEntropyLoss()
# 4.计算损失
loss = loss_fn(y_logits, y_true)
print(f"loss: {loss}")
# 5.计算预测概率
y_pred = torch.softmax(y_logits, dim=-1)
print(f"y_pred: {y_pred}")
# 定义函数，计算 二分类交叉熵损失值
def demo02():
print("二分类交叉熵损失")
# 1. 创建 样本的真实类别
y_true = torch.tensor([0,0,1], dtype=torch.float)
# 2.创建 模型预测 的原始输出logits值
y_logits = torch.tensor([-2,-1,2], dtype=torch.float)
# 经过sigmoid,转化为正类别的 概率
y_pred = torch.sigmoid(y_logits)
# 3.创建 交叉熵损失函数
loss_fn = nn.BCELoss()
# 4.计算损失
loss = loss_fn(y_pred, y_true)
print(f"loss: {loss}")
print(f"y_pred: {y_pred}")
# 测试
if __name__ == '__main__':
demo01()
demo02()
多分类交叉熵损失
loss: 0.18903924524784088
y_pred: tensor([[0.8808, 0.1192],
[0.7311, 0.2689],
[0.1192, 0.8808]])
二分类交叉熵损失
loss: 0.18903927505016327
y_pred: tensor([0.1192, 0.2689, 0.8808])
# 3.使用pytorch计算MSE损失值。已知 y_ture 为 [1, 2, 3, 4, 5, 6], y_pred 为 [2, 3, 4, 5, 6, 7]，计算MSE损失值，并打印。
import torch
import torch.nn as nn
# 获取真实值和预测值
y_true = torch.tensor([1, 2, 3, 4, 5, 6], dtype=torch.float)
y_pred = torch.tensor([2, 3, 4, 5, 6, 7], dtype=torch.float)
# 定义损失函数
loss_fn = nn.MSELoss()
# 计算损失值
loss = loss_fn(y_pred, y_true)
print(f"loss: {loss}")
loss: 1.0
# 4.使用梯度下降法，求 loss = 0.5*w**2 + 10 的极小值点 并打印loss最小值，以及loss是最小值时 w的值 和 梯度。
# 导包
import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
import matplotlib.pyplot as plt
# 设置中文字体
plt.rcParams['font.sans-serif'] = ['SimHei'] # 微软雅黑
plt.rcParams['axes.unicode_minus'] = False # 解决负号显示问题
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print("梯度下降法 —— sgd")
# 1.初始化模型参数w
w = torch.tensor([1.0], requires_grad=True, dtype=torch.float32)
# 2.求损失值
loss = 0.5*w**2 + 10 # w
# 3.创建优化器
# 参数1：待优化的模型参数列表，参数2：学习率
optimizer = optim.SGD([w], lr=0.01)
# 定义变量，记录损失值
loss_list = []
for i in range(800):
# 4.计算损失值
loss = 0.5 * (w ** 2) # w
# 5.计算梯度，梯度清零 + 反向传播 + 更新参数
optimizer.zero_grad() # 梯度清零
loss.mean().backward() # 反向传播
optimizer.step() # 更新参数
loss_list.append(loss.mean().item())
# print(f"第{i+1}次更新结果：w:{w}, w.grad: {w.grad}, loss: {loss}")
# 6.打印结果
print(f"最终结果：w:{w}, w.grad: {w.grad}, loss: {loss}")
# 7.绘制loss曲线
import matplotlib.pyplot as plt
plt.figure(figsize=(12,8))
plt.plot(range(len(loss_list)),loss_list)
plt.xlabel("迭代次数")
plt.ylabel("损失均值loss.mean")
plt.grid()
plt.show()
梯度下降法 —— sgd
最终结果：w:tensor([0.0003], requires_grad=True), w.grad: tensor([0.0003]), loss: tensor([5.2968e-08], grad_fn=<MulBackward0>)
# 5.使用Adam优化器，循环迭代800次，求 loss = 0.5*w**2 + 10 的极小值点, 并打印loss最小值，以及loss是最小值时 w的值 和 梯度。
# 导包
import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
import matplotlib.pyplot as plt
# 设置中文字体
plt.rcParams['font.sans-serif'] = ['SimHei'] # 微软雅黑
plt.rcParams['axes.unicode_minus'] = False # 解决负号显示问题
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print("梯度下降优化方法 —— adam")
# 1.初始化模型参数w
w = torch.tensor([1.0], requires_grad=True, dtype=torch.float32)
# 2.求损失值
loss = 0.5*w**2 + 10 # w
# 3.创建优化器
# 参数1：待优化的模型参数列表，参数2：学习率，参数3：衰减系数
optimizer = optim.Adam([w], lr=0.01, betas=(0.9, 0.999), weight_decay=1e-3)
# 定义变量，记录损失值
loss_list = []
for i in range(800):
# 4.计算损失值
loss = 0.5 * (w ** 2) # w
# 5.计算梯度，梯度清零 + 反向传播 + 更新参数
optimizer.zero_grad() # 梯度清零
loss.mean().backward() # 反向传播
optimizer.step() # 更新参数
loss_list.append(loss.mean().item())
# print(f"第{i+1}次更新结果：w:{w}, w.grad: {w.grad}, loss: {loss}")
# 6.打印结果
print(f"最终结果：w:{w}, w.grad: {w.grad}, loss: {loss}")
# 7.绘制loss曲线
import matplotlib.pyplot as plt
plt.figure(figsize=(12,8))
plt.plot(range(len(loss_list)),loss_list)
plt.xlabel("迭代次数")
plt.ylabel("损失均值loss.mean")
plt.grid()
plt.show()
梯度下降优化方法 —— adam
最终结果：w:tensor([-3.2689e-16], requires_grad=True), w.grad: tensor([-3.1520e-16]), loss: tensor([4.9676e-32], grad_fn=<MulBackward0>)
# 6.使用AdamW优化器，使用optim.Adamw实现，循环迭代800次，求 loss = 0.5*w**2 + 10 的极小值点, 并打印loss最小值，以及loss是最小值时 w的值 和 梯度。
# 导包
import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
import matplotlib.pyplot as plt
# 设置中文字体
plt.rcParams['font.sans-serif'] = ['SimHei'] # 微软雅黑
plt.rcParams['axes.unicode_minus'] = False # 解决负号显示问题
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print("梯度下降优化方法 —— adamW")
# 1.初始化模型参数w
w = torch.tensor([1.0], requires_grad=True, dtype=torch.float32)
# 2.求损失值
loss = 0.5*w**2 + 10 # w
# 3.创建优化器，基于SGD, 加入动量 momentum(动量参数)，就是动量法
# 参数1：待优化的模型参数列表，参数2：学习率，参数3：衰减系数
optimizer = optim.AdamW([w], lr=0.01, betas=(0.9, 0.999), weight_decay=1e-3)
# 定义变量，记录损失值
loss_list = []
for i in range(800):
# 4.计算损失值
loss = 0.5 * (w ** 2) # w
# 5.计算梯度，梯度清零 + 反向传播 + 更新参数
optimizer.zero_grad() # 梯度清零
loss.mean().backward() # 反向传播
optimizer.step() # 更新参数
loss_list.append(loss.mean().item())
# print(f"第{i+1}次更新结果：w:{w}, w.grad: {w.grad}, loss: {loss}")
# 6.打印结果
print(f"最终结果：w:{w}, w.grad: {w.grad}, loss: {loss}")
# 7.绘制loss曲线
import matplotlib.pyplot as plt
plt.figure(figsize=(12,8))
plt.plot(range(len(loss_list)),loss_list)
plt.xlabel("迭代次数")
plt.ylabel("损失均值loss.mean")
plt.grid()
plt.show()
梯度下降优化方法 —— adamW
最终结果：w:tensor([-3.3323e-16], requires_grad=True), w.grad: tensor([-3.2264e-16]), loss: tensor([5.2048e-32], grad_fn=<MulBackward0>)
day05
- 1.使用两层dropout处理数据。输入数据形状为(2,4),隐藏层形状为(4,20),后接dropout(p=0.5),输出层形状为(20,10),得到输出结果，并打印。注意通常输出层之后不加dropout
- 2.修改ANN手机价格分类案例代码。采用周期启动的余弦退火学习率调度策略torch.optim.lr_scheduler.CosineAnnealingWarmRestarts来调整学习率，训练模型，并打印最终的测试准确率。
- 3.修改ANN手机价格分类案例代码。模型中添加dropout层，优化器改为AdamW, 采用周期启动的余弦退火学习率调度策略torch.optim.lr_scheduler.CosineAnnealingWarmRestarts来调整学习率，训练模型，并打印最终的测试准确率。注意dropout的p应该小于0.1，因为数据集较小。
# 1.使用两层dropout处理数据。输入数据形状为(2,4),隐藏层形状为(4,20),后接dropout(p=0.5),输出层形状为(20,10),得到输出结果，并打印。注意通常输出层之后不加dropout
# 1.创建输入数据
t1 = torch.randint(0,10,size=(2,4))
# 转换为float类型
t1 = t1.to(dtype=torch.float32)
print(f"输入特征t1: {t1}")
# 2.搭建神经网络
class ModelDemo(nn.Module):
# 1.1 定义__init__方法，搭建网络层，并且进行参数初始化
def __init__(self):
# 初始化父类成员
super().__init__()
# 搭建网络层
# 隐藏层1
self.linear1 = nn.Linear(4, 20)
# 隐藏层2
self.linear2 = nn.Linear(20, 10)
# dropout层
self.dropout = nn.Dropout(p=0.5)
# 1.2 定义forward方法，实现前向传播
def forward(self, x):
# 输入隐藏层1，加权求和 + 激活函数
x = torch.relu(self.linear1(x))
# 输入dropout层，进行随机失活
x = self.dropout(x)
# 输入隐藏层2，加权求和 + 激活函数
x = self.linear2(x)
# 返回预测值
return x
# 3.创建模型
model = ModelDemo()
# 前向传播，模型预测
y_pred = model(t1)
print(f"预测结果y_pred: {y_pred}")
输入特征t1: tensor([[6., 2., 6., 6.],
[8., 5., 9., 0.]])
预测结果y_pred: tensor([[ 3.4134, -0.7866, 1.1466, -1.1044, -3.8304, -3.5764, -0.4554, 0.5703,
2.6228, -1.6571],
[ 3.2008, -0.6340, 1.3045, -2.6729, -6.2804, 0.1498, 1.3902, -3.3826,
2.1994, 1.7634]], grad_fn=<AddmmBackward0>)
# 2.修改ANN手机价格分类案例代码。采用周期启动的余弦退火学习率调度策略torch.optim.lr_scheduler.CosineAnnealingWarmRestarts来调整学习率，训练模型，并打印最终的测试准确率。
import torch # pytorch框架
from torch.utils.data import TensorDataset # 数据集对象
from torch.utils.data import DataLoader # 数据加载器
import torch.nn as nn # 神经网络层的操作
import torch.optim as optim # 优化器
from sklearn.model_selection import train_test_split # 拆分 训练集 和 测试集
import matplotlib.pyplot as plt # 绘图
import numpy as np # 数组操作
import pandas as pd # 数据处理
import time # 时间模块
from torchsummary import summary
from sklearn.preprocessing import StandardScaler
device = "cuda" if torch.cuda.is_available() else "cpu"
# 1.准备数据集
def create_dataset():
# 1.加载CSV数据集
data = pd.read_csv("./data/手机价格预测.csv")
# 2.获取特征x 和 标签y
x = data.iloc[:, :-1]
y = data.iloc[:, -1]
# print(f"x: {x.head()}")
# print(f"y: {y.head()}")
# 3.特征列 转换为 float
x = x.astype(np.float32)
# print(f"x: {x}")
# 4.拆分数据集
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=6, stratify=y)
# 优化1：数据标准化
transfer = StandardScaler()
x_train = transfer.fit_transform(x_train)
x_test = transfer.transform(x_test)
# 5.把数据集转换为张量, numpy数组 -> 张量 -> 数据集对象TensorDataset -> 数据加载器DataLoader
train_dataset = TensorDataset(torch.tensor(x_train), torch.tensor(y_train.values))
test_dataset = TensorDataset(torch.tensor(x_test), torch.tensor(y_test.values))
# 6.返回结果
return train_dataset, test_dataset, x_train.shape[1], len(y_train.unique())
# 2.搭建神经网络
class PhonePriceClassifier(nn.Module):
# 1.定义__init__方法, 初始化父类成员，搭建神经网络
def __init__(self, input_dim, output_dim):
# 初始化父类成员
super().__init__()
# 搭建神经网络
# 优化2：增加网络深度
self.linear1 = nn.Linear(input_dim, 128)
self.linear2 = nn.Linear(128, 256)
self.linear3 = nn.Linear(256, 512)
self.linear4 = nn.Linear(512, 256)
self.linear5 = nn.Linear(256, output_dim)
pass
# 2.定义forward方法，实现前向传播
def forward(self, x):
# 隐藏层1
x = torch.relu(self.linear1(x))
# 隐藏层2
x = torch.relu(self.linear2(x))
x = torch.relu(self.linear3(x))
x = torch.relu(self.linear4(x))
# 输出层
x = self.linear5(x)
# 返回结果
return x
# 3.模型训练
def train(train_dataset, input_dim, output_dim):
# 1.创建数据加载器DataLoader
# 参数1：数据集对象；参数2：批量大小；参数3：是否打乱数据
train_loader = DataLoader(train_dataset, batch_size=16, shuffle=True)
# 2.创建神经网络模型
model = PhonePriceClassifier(input_dim, output_dim)
model.to(device)
# 3.定义损失函数
loss_fn = nn.CrossEntropyLoss()
# 4.创建优化器对象
# 优化3：使用Adam优化方法
optimizer = optim.Adam(model.parameters(), lr=1e-4)
# 创建余弦退火学习率调度器
scheduler = optim.lr_scheduler.CosineAnnealingWarmRestarts(optimizer, T_0=20, T_mult=1, eta_min=0)
# 5.模型训练
# 定义变量，记录 训练总轮数
epochs = 100
# 遍历轮数
for epoch in range(epochs):
# 记录 每次训练的 损失值，训练样本数
total_loss = 0.
total_sample = 0
# 记录训练开始时间
start_time = time.time()
# 遍历批次
for x, y in train_loader:
# 训练模式
model.train()
# 移动数据到GPU
x = x.to(device)
y = y.to(device)
# 前向传播
y_pred = model(x)
# 计算损失值
loss = loss_fn(y_pred, y)
# 更新总损失
batch_size = x.shape[0]
total_loss += loss.item()*batch_size
# 更新总样本数
total_sample += batch_size
# 梯度清零
optimizer.zero_grad()
# 反向传播
loss.backward()
# 更新参数
optimizer.step()
# 调整学习率
scheduler.step()
# 打印训练信息
# print(f"epoch: {epoch}, loss: {total_loss / total_sample}, time: {time.time() - start_time:.2f}")
pass
# 6.保存模型参数
# print(f"\n\n模型的参数信息: {model.state_dict()}\n\n")
torch.save(model.state_dict(), "./model/phone_price_classifier.pth") # 模型文件，后缀名 .pth, .pkl, .pickle 均可
pass
# 4.模型测试
@torch.no_grad()
def evaluate(test_dataset, input_dim, output_dim):
# 1. 创建神经网络分类对象.
model = PhonePriceClassifier(input_dim, output_dim)
model.to(device)
# 2.加载模型参数
model.load_state_dict(torch.load('./model/phone_price_classifier.pth'))
# 3.创建数据加载器DataLoader
# 参数1：数据集对象；参数2：批量大小；参数3：是否打乱数据
test_loader = DataLoader(test_dataset, batch_size=16, shuffle=False)
# 4.记录正确样本的数量
correct_sample = 0
total_sample = 0
# 5.开始测试, 遍历数据加载器的批次数据
for x, y in test_loader:
# 切换到测试模式
model.eval()
# 张量 迁移 到 GPU
x = x.to(device)
y = y.to(device)
# 6.前向传播，模型预测
y_pred = model(x) # 形状(batch_size, 4), 每个类别的输出logits
# print(f"预测结果y_pred：{y_pred}")
# 7.计算预测正确的数量
correct = y_pred.argmax(dim=1) == y
correct_sample += correct.sum().item()
total_sample += y.size(0)
print(f"测试集准确率：{correct_sample / total_sample}")
# 5.测试
if __name__ == '__main__':
# 1.准备数据集
train_dataset, test_dataset, input_dim, output_dim = create_dataset()
# 打印
print(f"train_dataset: {train_dataset}")
print(f"test_dataset: {test_dataset}")
print(f"input_dim: {input_dim}")
print(f"output_dim: {output_dim}")
# 2.构建神经网络
model = PhonePriceClassifier(input_dim, output_dim)
# 3.统计模型参数
model.to(device)
# summary(model,input_size=(input_dim,))
# 4.模型训练
train(train_dataset, input_dim, output_dim)
# 5.模型测试
evaluate(test_dataset, input_dim, output_dim)
pass
train_dataset: <torch.utils.data.dataset.TensorDataset object at 0x000001C733E1E710>
test_dataset: <torch.utils.data.dataset.TensorDataset object at 0x000001C733E1E990>
input_dim: 20
output_dim: 4
测试集准确率：0.9075
# 3.修改ANN手机价格分类案例代码。模型中添加dropout层，优化器改为AdamW, 采用周期启动的余弦退火学习率调度策略torch.optim.lr_scheduler.CosineAnnealingWarmRestarts来调整学习率，训练模型，并打印最终的测试准确率。注意dropout的p应该小于0.1，因为数据集较小。
import torch # pytorch框架
from torch.utils.data import TensorDataset # 数据集对象
from torch.utils.data import DataLoader # 数据加载器
import torch.nn as nn # 神经网络层的操作
import torch.optim as optim # 优化器
from sklearn.model_selection import train_test_split # 拆分 训练集 和 测试集
import matplotlib.pyplot as plt # 绘图
import numpy as np # 数组操作
import pandas as pd # 数据处理
import time # 时间模块
from torchsummary import summary
from sklearn.preprocessing import StandardScaler
device = "cuda" if torch.cuda.is_available() else "cpu"
# 1.准备数据集
def create_dataset():
# 1.加载CSV数据集
data = pd.read_csv("./data/手机价格预测.csv")
# 2.获取特征x 和 标签y
x = data.iloc[:, :-1]
y = data.iloc[:, -1]
# print(f"x: {x.head()}")
# print(f"y: {y.head()}")
# 3.特征列 转换为 float
x = x.astype(np.float32)
# print(f"x: {x}")
# 4.拆分数据集
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=6, stratify=y)
# 优化1：数据标准化
transfer = StandardScaler()
x_train = transfer.fit_transform(x_train)
x_test = transfer.transform(x_test)
# 5.把数据集转换为张量, numpy数组 -> 张量 -> 数据集对象TensorDataset -> 数据加载器DataLoader
train_dataset = TensorDataset(torch.tensor(x_train), torch.tensor(y_train.values))
test_dataset = TensorDataset(torch.tensor(x_test), torch.tensor(y_test.values))
# 6.返回结果
return train_dataset, test_dataset, x_train.shape[1], len(y_train.unique())
# 2.搭建神经网络
class PhonePriceClassifier(nn.Module):
# 1.定义__init__方法, 初始化父类成员，搭建神经网络
def __init__(self, input_dim, output_dim):
# 初始化父类成员
super().__init__()
# 搭建神经网络
# 优化2：增加网络深度
self.linear1 = nn.Linear(input_dim, 128)
self.linear2 = nn.Linear(128, 256)
self.linear3 = nn.Linear(256, 512)
self.linear4 = nn.Linear(512, 256)
self.linear5 = nn.Linear(256, output_dim)
# 添加dropout层
self.dropout = nn.Dropout(0.05)
pass
# 2.定义forward方法，实现前向传播
def forward(self, x):
# 隐藏层1
x = torch.relu(self.linear1(x))
# x = self.dropout(x)
# 隐藏层2
x = torch.relu(self.linear2(x))
# x = self.dropout(x)
x = torch.relu(self.linear3(x))
x = self.dropout(x)
x = torch.relu(self.linear4(x))
x = self.dropout(x)
# 输出层
x = self.linear5(x)
# 返回结果
return x
# 3.模型训练
def train(train_dataset, input_dim, output_dim):
# 1.创建数据加载器DataLoader
# 参数1：数据集对象；参数2：批量大小；参数3：是否打乱数据
train_loader = DataLoader(train_dataset, batch_size=16, shuffle=True)
# 2.创建神经网络模型
model = PhonePriceClassifier(input_dim, output_dim)
model.to(device)
# 3.定义损失函数
loss_fn = nn.CrossEntropyLoss()
# 4.创建优化器对象
# 优化3：使用AdamW优化方法
optimizer = optim.AdamW(model.parameters(), lr=1e-4, weight_decay=1e-4)
# 创建余弦退火学习率调度器
scheduler = optim.lr_scheduler.CosineAnnealingWarmRestarts(optimizer, T_0=20, T_mult=1, eta_min=0)
# 5.模型训练
# 定义变量，记录 训练总轮数
epochs = 100
# 遍历轮数
for epoch in range(epochs):
# 记录 每次训练的 损失值，训练样本数
total_loss = 0.
total_sample = 0
# 记录训练开始时间
start_time = time.time()
# 遍历批次
for x, y in train_loader:
# 训练模式
model.train()
# 移动数据到GPU
x = x.to(device)
y = y.to(device)
# 前向传播
y_pred = model(x)
# 计算损失值
loss = loss_fn(y_pred, y)
# 更新总损失
batch_size = x.shape[0]
total_loss += loss.item()*batch_size
# 更新总样本数
total_sample += batch_size
# 梯度清零
optimizer.zero_grad()
# 反向传播
loss.backward()
# 更新参数
optimizer.step()
# 调整学习率
scheduler.step()
# 打印训练信息
# print(f"epoch: {epoch}, loss: {total_loss / total_sample}, time: {time.time() - start_time:.2f}")
pass
# 6.保存模型参数
# print(f"\n\n模型的参数信息: {model.state_dict()}\n\n")
torch.save(model.state_dict(), "./model/phone_price_classifier.pth") # 模型文件，后缀名 .pth, .pkl, .pickle 均可
pass
# 4.模型测试
@torch.no_grad()
def evaluate(test_dataset, input_dim, output_dim):
# 1. 创建神经网络分类对象.
model = PhonePriceClassifier(input_dim, output_dim)
model.to(device)
# 2.加载模型参数
model.load_state_dict(torch.load('./model/phone_price_classifier.pth'))
# 3.创建数据加载器DataLoader
# 参数1：数据集对象；参数2：批量大小；参数3：是否打乱数据
test_loader = DataLoader(test_dataset, batch_size=16, shuffle=False)
# 4.记录正确样本的数量
correct_sample = 0
total_sample = 0
# 5.开始测试, 遍历数据加载器的批次数据
for x, y in test_loader:
# 切换到测试模式
model.eval()
# 张量 迁移 到 GPU
x = x.to(device)
y = y.to(device)
# 6.前向传播，模型预测
y_pred = model(x) # 形状(batch_size, 4), 每个类别的输出logits
# print(f"预测结果y_pred：{y_pred}")
# 7.计算预测正确的数量
correct = y_pred.argmax(dim=1) == y
correct_sample += correct.sum().item()
total_sample += y.size(0)
print(f"测试集准确率：{correct_sample / total_sample}")
# 5.测试
if __name__ == '__main__':
# 1.准备数据集
train_dataset, test_dataset, input_dim, output_dim = create_dataset()
# 打印
print(f"train_dataset: {train_dataset}")
print(f"test_dataset: {test_dataset}")
print(f"input_dim: {input_dim}")
print(f"output_dim: {output_dim}")
# 2.构建神经网络
model = PhonePriceClassifier(input_dim, output_dim)
# 3.统计模型参数
model.to(device)
# summary(model,input_size=(input_dim,))
# 4.模型训练
train(train_dataset, input_dim, output_dim)
# 5.模型测试
evaluate(test_dataset, input_dim, output_dim)
pass
train_dataset: <torch.utils.data.dataset.TensorDataset object at 0x000001A2506E42F0>
test_dataset: <torch.utils.data.dataset.TensorDataset object at 0x000001A250659310>
input_dim: 20
output_dim: 4
测试集准确率：0.92
day06
- 1.使用 matplotlib 绘制360✖360✖3的 全黑图 和 全白图，并打印。
- 2.使用 matplotlib 加载一张图片，然后通过Conv2d(3, 8, 3, 1, 1)卷积层，打印输出特征图的形状，并展示所有特征图。
- 3.使用 matplotlib 加载一张图片，然后通过Conv2d(3, 8, 5, 2, 0)卷积层，打印输出特征图的形状，并展示所有特征图。
- 4.使用 matplotlib 加载一张图片，然后通过MaxPool2d(2, 2, 0)池化层，打印输出特征图的形状，并展示所有特征图。
- 5.使用 matplotlib 加载一张图片，然后通过MaxPool2d(10, 10, 0)池化层，打印输出特征图的形状，并展示所有特征图。
- 6.动手实现CIFAR10图像分类任务，尝试修改模型或参数，看看模型性能如何。
# 1.使用 matplotlib 绘制360✖360✖3的 全黑图 和 全白图，并打印。
import matplotlib.pyplot as plt
import torch
# 创建全黑图
black_img = torch.zeros((360, 360, 3))
plt.imshow(black_img)
plt.tight_layout()
plt.show()
# 创建全白图
white_img = torch.ones((360, 360, 3))
plt.imshow(white_img)
plt.tight_layout()
plt.show()
# 2.使用 matplotlib 加载一个自己获取的图片，然后通过Conv2d(3, 8, 3, 1, 1)卷积层，打印输出特征图的形状，并展示所有特征图。
import torch
import torch.nn as nn
import matplotlib.pyplot as plt
# 1.加载图片
img = plt.imread("./data/img.jpg")
print(f"img.shape: {img.shape}")
# 2.创建Conv2d(3, 8, 3, 1, 1)卷积层
conv = nn.Conv2d(3, 8, 3, 1, 1)
# 3.把图片形状从 HWC 转换为 CHW，因为卷积层要求输入的图片形状为CHW,思路：np数组 -> torch张量
img_tensor = torch.tensor(img, dtype=torch.float).permute(2, 0, 1)
print(f"img_tensor.shape: {img_tensor.shape}")
# 4.添加一个维度，表示批次N，卷积层要求输入图片形状为NCHW
img_tensor = img_tensor.unsqueeze(0)
print(f"img_tensor.shape: {img_tensor.shape}")
# 5.卷积层前向传播，得到输出特征图
out = conv(img_tensor)
# 6.打印输出特征图的形状
print(f"特征图形状out.shape: {out.shape}")
# 7.展示所有特征图
out = out.squeeze(0).detach()
print(f"out.shape: {out.shape}")
plt.figure(figsize=(10, 10))
for i in range(out.shape[0]):
plt.subplot(4, 4, i+1)
plt.imshow(out[i])
plt.show()
img.shape: (640, 640, 3)
img_tensor.shape: torch.Size([3, 640, 640])
img_tensor.shape: torch.Size([1, 3, 640, 640])
特征图形状out.shape: torch.Size([1, 8, 640, 640])
out.shape: torch.Size([8, 640, 640])
# 3.使用 matplotlib 加载一个自己获取的图片，然后通过Conv2d(3, 8, 5, 2, 0)卷积层，打印输出特征图的形状，并展示所有特征图。
import torch
import torch.nn as nn
import matplotlib.pyplot as plt
# 1.加载图片
img = plt.imread("./data/img.jpg")
print(f"img.shape: {img.shape}")
# 2.创建Conv2d(3, 8, 5, 2, 0)卷积层
conv = nn.Conv2d(3, 8, 5, 2, 0)
# 3.把图片形状从 HWC 转换为 CHW，因为卷积层要求输入的图片形状为CHW,思路：np数组 -> torch张量
img_tensor = torch.tensor(img, dtype=torch.float).permute(2, 0, 1)
print(f"img_tensor.shape: {img_tensor.shape}")
# 4.添加一个维度，表示批次N，卷积层要求输入图片形状为NCHW
img_tensor = img_tensor.unsqueeze(0)
print(f"img_tensor.shape: {img_tensor.shape}")
# 5.卷积层前向传播，得到输出特征图
out = conv(img_tensor)
# 6.打印输出特征图的形状
print(f"特征图形状out.shape: {out.shape}")
# 7.展示所有特征图
out = out.squeeze(0).detach()
print(f"out.shape: {out.shape}")
plt.figure(figsize=(10, 10))
for i in range(out.shape[0]):
plt.subplot(4, 4, i+1)
plt.imshow(out[i])
plt.show()
img.shape: (640, 640, 3)
img_tensor.shape: torch.Size([3, 640, 640])
img_tensor.shape: torch.Size([1, 3, 640, 640])
特征图形状out.shape: torch.Size([1, 8, 318, 318])
out.shape: torch.Size([8, 318, 318])
# 4.使用 matplotlib 加载一个自己获取的图片，然后通过MaxPool2d(2, 2, 0)池化层，打印输出特征图的形状，并展示所有特征图。
import torch
import torch.nn as nn
import matplotlib.pyplot as plt
# 1.加载图片
img = plt.imread("./data/img.jpg")
print(f"img.shape: {img.shape}")
# 2.创建MaxPool2d(2, 2, 0)池化层
pool = nn.MaxPool2d(2, 2, 0)
# 3.把图片形状从 HWC 转换为 CHW，因为卷积层要求输入的图片形状为CHW,思路：np数组 -> torch张量
img_tensor = torch.tensor(img, dtype=torch.float).permute(2, 0, 1)
print(f"img_tensor.shape: {img_tensor.shape}")
# 4.添加一个维度，表示批次N，卷积层要求输入图片形状为NCHW
img_tensor = img_tensor.unsqueeze(0)
print(f"img_tensor.shape: {img_tensor.shape}")
# 5.前向传播，得到输出特征图
out = pool(img_tensor)
# 6.打印输出特征图的形状
print(f"特征图形状out.shape: {out.shape}")
# 7.展示所有特征图
out = out.squeeze(0).detach()
print(f"out.shape: {out.shape}")
plt.figure(figsize=(10, 10))
for i in range(out.shape[0]):
plt.subplot(1, 3, i+1)
plt.imshow(out[i])
plt.show()
# 8.out转换CHW为HWC
out = out.permute(1, 2, 0)
out = out / 255.0
print(f"out.shape: {out.shape}")
plt.imshow(out)
plt.show()
img.shape: (640, 640, 3)
img_tensor.shape: torch.Size([3, 640, 640])
img_tensor.shape: torch.Size([1, 3, 640, 640])
特征图形状out.shape: torch.Size([1, 3, 320, 320])
out.shape: torch.Size([3, 320, 320])
out.shape: torch.Size([320, 320, 3])
# 5.使用 matplotlib 加载一个自己获取的图片，然后通过MaxPool2d(10, 10, 0)池化层，打印输出特征图的形状，并展示所有特征图。
import torch
import torch.nn as nn
import matplotlib.pyplot as plt
# 1.加载图片
img = plt.imread("./data/img.jpg")
print(f"img.shape: {img.shape}")
# 2.创建MaxPool2d(10, 10, 0)池化层
pool = nn.MaxPool2d(10, 10, 0)
# 3.把图片形状从 HWC 转换为 CHW，因为卷积层要求输入的图片形状为CHW,思路：np数组 -> torch张量
img_tensor = torch.tensor(img, dtype=torch.float).permute(2, 0, 1)
print(f"img_tensor.shape: {img_tensor.shape}")
# 4.添加一个维度，表示批次N，卷积层要求输入图片形状为NCHW
img_tensor = img_tensor.unsqueeze(0)
print(f"img_tensor.shape: {img_tensor.shape}")
# 5.前向传播，得到输出特征图
out = pool(img_tensor)
# 6.打印输出特征图的形状
print(f"特征图形状out.shape: {out.shape}")
# 7.展示所有特征图
out = out.squeeze(0).detach()
print(f"out.shape: {out.shape}")
plt.figure(figsize=(10, 10))
for i in range(out.shape[0]):
plt.subplot(1, 3, i+1)
plt.imshow(out[i])
plt.show()
# 8.out转换CHW为HWC
out = out.permute(1, 2, 0)
out = out / 255.0
print(f"out.shape: {out.shape}")
plt.imshow(out)
plt.show()
img.shape: (640, 640, 3)
img_tensor.shape: torch.Size([3, 640, 640])
img_tensor.shape: torch.Size([1, 3, 640, 640])
特征图形状out.shape: torch.Size([1, 3, 64, 64])
out.shape: torch.Size([3, 64, 64])
out.shape: torch.Size([64, 64, 3])
day07
- 1.使用 nn.Embedding 获取128维度的词向量，输入文本为"星星
一颗两颗三颗四颗 连成线背著背默默许下心愿
看远方的星如果听的见
它一定实现它一定实现
载著你 彷彿载著阳光
不管到哪里都是晴天
蝴蝶自在飞
花也布满天
一朵一朵因你而香
试图让夕阳飞翔
带领你我环绕大自然
迎著风 开始共渡每一天
手牵手
一步两步三步四步望著天"，并打印。要求词表中的token按照词频从高到底排序。 - 2.尝试优化RNN文本生成案例，看看生成歌词的效果。比如修改词嵌入维度embedding_dim，修改批次数量batch_size，修改序列长度sequence_length，添加学习率调度等。
# 1.使用 nn.Embedding 获取128维度的词向量，输入文本为"星星
# 一颗两颗三颗四颗 连成线背著背默默许下心愿
# 看远方的星如果听的见
# 它一定实现它一定实现
# 载著你 彷彿载著阳光
# 不管到哪里都是晴天
# 蝴蝶自在飞
# 花也布满天
# 一朵一朵因你而香
# 试图让夕阳飞翔
# 带领你我环绕大自然
# 迎著风 开始共渡每一天
# 手牵手
# 一步两步三步四步望著天"，并打印。要求词表中的token按照词频从高到底排序。
import torch
import torch.nn as nn
import jieba
from collections import Counter
def demo01():
text = '''
星星
一颗两颗三颗四颗 连成线背著背默默许下心愿
看远方的星如果听的见
它一定实现它一定实现
载著你 彷彿载著阳光
不管到哪里都是晴天
蝴蝶自在飞
花也布满天
一朵一朵因你而香
试图让夕阳飞翔
带领你我环绕大自然
迎著风 开始共渡每一天
手牵手
一步两步三步四步望著天
'''
# 2.使用jieba 进行分词
words = jieba.lcut(text)
print(f"词序列words: {words}")
# === 按词频排序 ===
word_freq = Counter(words)
unique_words = [w for w, _ in word_freq.most_common()]
print(f"按照词频排序后的unique_words: {unique_words}")
# === 构建词表 ===
vocab = {word: index for index, word in enumerate(unique_words)}
print(f"vocab: {vocab}")
# 4.创建词嵌入层
embed = nn.Embedding(num_embeddings=len(vocab), embedding_dim=128)
print(f"embed: {embed}")
# 5.将每个词转换为词向量
for word in words:
idx = vocab[word]
idx_tensor = torch.tensor([idx])
word_vector = embed(idx_tensor)
print(f"{idx}: {word}\t词向量: {word_vector.detach()}")
if __name__ == '__main__':
demo01()
词序列words: ['\n', '星星', '\n', ' ', '一颗', '两颗', '三颗', '四颗', ' ', '连成线', '背著', '背', '默默', '许下', '心愿', '\n', ' ', '看', '远方', '的', '星', '如果', '听', '的', '见', '\n', ' ', '它', '一定', '实现', '它', '一定', '实现', '\n', ' ', '载著', '你', ' ', '彷', '彿', '载', '著', '阳光', '\n', ' ', '不管', '到', '哪里', '都', '是', '晴天', '\n', ' ', '蝴蝶', '自在', '飞', '\n', ' ', '花', '也', '布满', '天', '\n', ' ', '一朵', '一朵', '因', '你', '而', '香', '\n', ' ', '试图', '让', '夕阳', '飞翔', '\n', ' ', '带领', '你', '我', '环绕', '大自然', '\n', ' ', '迎著风', ' ', '开始', '共渡', '每', '一天', '\n', ' ', '手牵手', '\n', ' ', '一步', '两步', '三步', '四步', '望', '著天', '\n']
按照词频排序后的unique_words: [' ', '\n', '你', '的', '它', '一定', '实现', '一朵', '星星', '一颗', '两颗', '三颗', '四颗', '连成线', '背著', '背', '默默', '许下', '心愿', '看', '远方', '星', '如果', '听', '见', '载著', '彷', '彿', '载', '著', '阳光', '不管', '到', '哪里', '都', '是', '晴天', '蝴蝶', '自在', '飞', '花', '也', '布满', '天', '因', '而', '香', '试图', '让', '夕阳', '飞翔', '带领', '我', '环绕', '大自然', '迎著风', '开始', '共渡', '每', '一天', '手牵手', '一步', '两步', '三步', '四步', '望', '著天']
vocab: {' ': 0, '\n': 1, '你': 2, '的': 3, '它': 4, '一定': 5, '实现': 6, '一朵': 7, '星星': 8, '一颗': 9, '两颗': 10, '三颗': 11, '四颗': 12, '连成线': 13, '背著': 14, '背': 15, '默默': 16, '许下': 17, '心愿': 18, '看': 19, '远方': 20, '星': 21, '如果': 22, '听': 23, '见': 24, '载著': 25, '彷': 26, '彿': 27, '载': 28, '著': 29, '阳光': 30, '不管': 31, '到': 32, '哪里': 33, '都': 34, '是': 35, '晴天': 36, '蝴蝶': 37, '自在': 38, '飞': 39, '花': 40, '也': 41, '布满': 42, '天': 43, '因': 44, '而': 45, '香': 46, '试图': 47, '让': 48, '夕阳': 49, '飞翔': 50, '带领': 51, '我': 52, '环绕': 53, '大自然': 54, '迎著风': 55, '开始': 56, '共渡': 57, '每': 58, '一天': 59, '手牵手': 60, '一步': 61, '两步': 62, '三步': 63, '四步': 64, '望': 65, '著天': 66}
embed: Embedding(67, 128)
1:
词向量: tensor([[ 1.4771, 0.8061, -0.4299, -0.5632, -1.3281, -0.5809, -0.4281, 0.5693,
0.6510, -0.1457, -0.2895, -1.2239, -0.8608, -1.1156, 1.4625, 0.9993,
2.6551, -0.2841, -0.1222, -0.1759, 0.2747, 0.8773, -1.2195, 0.0723,
-1.1253, -0.3896, -0.5094, -0.3521, 0.9901, 1.1108, 1.1191, -1.1345,
-0.5516, -1.0788, 0.2044, 0.1993, 0.2574, -0.3992, 0.0040, 0.1144,
-0.7567, 1.0407, -0.4901, -0.9804, 0.4055, 0.4972, -0.4097, -0.2506,
-0.0374, -0.2787, 0.5761, -0.0540, -0.8492, -0.7583, 1.0121, -0.6011,
-1.4103, 0.6407, 0.3240, 0.5556, -1.6938, 0.2747, 0.7353, -2.0933,
-1.7379, -0.5252, 1.3509, -0.2661, -0.3593, -0.2447, -1.2453, 0.2860,
-1.1808, 1.6014, -1.1948, 1.3895, 0.4737, -1.4571, 1.1382, -0.8261,
0.5641, -2.5230, 0.2975, 1.0076, -2.1213, 0.0967, 0.6917, 1.7999,
0.4339, 1.3127, 1.9793, 1.2312, -0.2442, 2.5434, 0.1964, 0.1350,
-1.6063, 0.6460, 0.2100, 0.4153, -0.0708, -0.2820, -0.3491, 0.3494,
-1.0825, 1.4995, 1.1523, 0.9549, 0.7601, 0.5949, 1.0576, 0.5871,
-1.3045, -0.5725, 0.5301, 0.5057, 0.8488, 0.5929, 1.0189, 0.0787,
0.9573, -0.7332, 0.9140, -0.4795, 0.2905, 0.9838, 0.3004, -0.2756]])
8: 星星 词向量: tensor([[ 0.1835, 0.4872, -0.9369, -1.3920, 1.5523, 1.4023, -0.8046, 0.3765,
0.5342, 0.0189, -0.0990, 0.6168, 1.4651, 0.3475, 0.6838, -1.5251,
1.0523, 0.1220, 0.0798, 0.0283, 1.6665, 0.8973, -0.3719, -0.2188,
1.7416, -2.9042, 1.1965, -0.5555, -0.1711, 1.5246, 1.3031, 0.9790,
0.4646, 0.8355, -0.6908, -0.3641, 0.3912, -0.7358, 0.3831, 2.1132,
0.2264, 1.4131, 1.2979, 1.3129, 0.4895, 0.7551, 0.1144, -0.3902,
-0.4115, -0.6710, -1.7081, 1.0087, -0.4943, -2.7130, 0.6042, 0.4462,
-0.5541, 0.5180, 0.2259, -0.0657, 0.1277, -0.7363, -0.0821, -0.6322,
1.5498, -1.4887, -0.6614, -0.1978, 0.4339, -0.2104, -0.9799, 1.6317,
1.2892, -2.0543, -0.7570, 0.6127, -1.4891, 1.1725, 1.1340, -1.0580,
1.7516, 0.3899, 0.4097, 0.7502, -0.4266, -1.2378, -1.1773, 1.0381,
0.8003, 0.9402, -0.6176, -0.9990, 1.8417, 0.5616, 0.0824, 0.4689,
-0.1857, -1.4487, -1.0235, 1.3938, 1.0608, 0.1206, -0.4064, -0.7393,
0.1459, 0.2270, -1.8763, -1.0721, -0.1382, -1.0131, -0.3761, -0.7022,
-0.6367, 0.6296, -0.1648, -1.4059, -0.1349, 1.0895, -0.1716, 0.1845,
0.3640, -0.3664, -1.2385, -0.2331, 1.2419, 1.7809, 0.5578, 1.2833]])
1:
词向量: tensor([[ 1.4771, 0.8061, -0.4299, -0.5632, -1.3281, -0.5809, -0.4281, 0.5693,
0.6510, -0.1457, -0.2895, -1.2239, -0.8608, -1.1156, 1.4625, 0.9993,
2.6551, -0.2841, -0.1222, -0.1759, 0.2747, 0.8773, -1.2195, 0.0723,
-1.1253, -0.3896, -0.5094, -0.3521, 0.9901, 1.1108, 1.1191, -1.1345,
-0.5516, -1.0788, 0.2044, 0.1993, 0.2574, -0.3992, 0.0040, 0.1144,
-0.7567, 1.0407, -0.4901, -0.9804, 0.4055, 0.4972, -0.4097, -0.2506,
-0.0374, -0.2787, 0.5761, -0.0540, -0.8492, -0.7583, 1.0121, -0.6011,
-1.4103, 0.6407, 0.3240, 0.5556, -1.6938, 0.2747, 0.7353, -2.0933,
-1.7379, -0.5252, 1.3509, -0.2661, -0.3593, -0.2447, -1.2453, 0.2860,
-1.1808, 1.6014, -1.1948, 1.3895, 0.4737, -1.4571, 1.1382, -0.8261,
0.5641, -2.5230, 0.2975, 1.0076, -2.1213, 0.0967, 0.6917, 1.7999,
0.4339, 1.3127, 1.9793, 1.2312, -0.2442, 2.5434, 0.1964, 0.1350,
-1.6063, 0.6460, 0.2100, 0.4153, -0.0708, -0.2820, -0.3491, 0.3494,
-1.0825, 1.4995, 1.1523, 0.9549, 0.7601, 0.5949, 1.0576, 0.5871,
-1.3045, -0.5725, 0.5301, 0.5057, 0.8488, 0.5929, 1.0189, 0.0787,
0.9573, -0.7332, 0.9140, -0.4795, 0.2905, 0.9838, 0.3004, -0.2756]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
9: 一颗 词向量: tensor([[-0.0952, -0.4625, -2.1070, -0.0037, -0.8910, 0.6835, -0.2864, 0.5136,
-0.7912, -0.5482, -0.4416, -1.3080, 0.7276, -0.0275, 1.1303, 1.2569,
0.7802, 0.0361, -0.5211, -1.5997, 1.3849, -1.5849, -1.0076, -2.0865,
0.4133, 1.2842, -0.7531, 0.0661, -0.3880, -1.6827, 0.7954, -0.3109,
1.0018, 0.0746, 0.3924, -1.8521, 1.0004, -0.5827, -0.7694, 0.4591,
0.0626, 0.2473, 0.0056, 0.9459, 0.7252, -1.3239, -2.2854, -0.2967,
0.9872, -0.2565, 1.5954, -1.6574, 0.9963, -2.0413, -1.1700, 0.3044,
-0.6900, -0.8878, 1.7776, 0.6022, -0.9321, 1.1377, 1.1811, -0.9509,
-0.7105, 0.1396, -1.5480, 0.2982, -0.4047, 2.2029, 1.0299, 0.6874,
0.8834, 1.5306, 0.3635, -1.2961, -0.9731, -0.8142, 0.6423, -1.3030,
0.4076, 0.3296, -1.0752, 1.5335, 2.0811, -0.9509, 0.0708, -2.4400,
0.4917, 0.0704, 1.8784, 1.3073, 0.3118, -0.6245, -0.0804, -1.2101,
0.3584, 0.2284, 0.1327, 0.2522, 0.0545, 0.2045, -1.3562, -1.0401,
0.8476, 1.7826, -0.1080, 0.0579, 0.2105, -0.2579, -0.3626, -1.2293,
-0.3020, -0.2536, -1.6700, -0.1959, 1.1102, 1.1040, 0.6529, -0.3119,
0.7558, -0.6343, -0.4792, -0.2486, -0.3554, 0.2107, 1.0031, -0.6590]])
10: 两颗 词向量: tensor([[-0.4241, 0.3842, -1.0727, -1.4128, 1.4355, 1.1545, -0.2644, -0.7621,
1.2800, -1.3823, 0.7247, 1.3148, 2.4522, 0.9155, 1.1893, -0.1881,
1.0175, -0.6195, -1.1411, 1.4055, -0.8957, 1.6052, -1.0145, 0.0874,
-0.7277, -0.1779, -1.6127, -1.1583, 0.0521, 0.1163, 0.0885, -0.6367,
-1.4189, -0.3521, 0.2732, 0.6589, -0.9027, 1.0495, -0.0275, 0.2075,
-1.0504, -2.8300, 0.7126, 1.5965, -0.6808, 1.3230, 0.2472, -1.0436,
-0.8222, -0.2674, -0.1792, 1.3296, 0.3519, -0.3567, -1.7306, 0.8841,
-0.4993, 0.6366, 0.3238, 1.7503, 0.4911, -1.3844, -0.9417, 1.1072,
0.2915, -0.7031, 0.6059, -0.4484, -1.4499, -0.9167, 0.3285, 0.8220,
0.4488, 0.7519, -0.5930, -1.3105, 0.5181, -0.3260, 0.8733, -1.1038,
-0.1815, 0.2122, -0.8948, 0.6983, 0.4185, -0.5262, 0.4956, -0.7615,
0.1492, -1.0103, 1.3746, -2.0112, 0.7675, -0.2039, 2.6824, -0.0474,
-0.2814, 1.3921, -0.7379, -1.7518, 2.1407, 0.6210, 0.5455, 1.8880,
0.4104, 0.0215, -0.3853, -0.3236, 0.6369, 1.2165, -0.4176, -0.7624,
0.2078, 0.1622, -0.3580, 1.1864, 0.9641, 0.5357, -1.4547, -0.3602,
0.0064, -0.3495, -0.1450, 1.7285, -0.8818, -0.1155, -0.6609, 0.4982]])
11: 三颗 词向量: tensor([[ 1.1972, 0.3026, 0.6861, -1.4936, 1.1214, 1.1350, 0.2260, -0.0322,
0.8376, -2.1491, 0.9146, 0.1889, -1.4430, -0.4548, 0.7127, -0.9101,
-0.8638, 0.3050, -0.8244, 0.2154, 0.1484, 1.4157, -0.6944, -0.0055,
0.7060, -0.3078, 0.2376, 0.6826, 2.1443, -0.2376, 0.0289, 2.0716,
0.2080, -0.2745, -0.5286, -0.0173, 0.7995, -0.2597, 0.1030, -0.9618,
0.3836, -0.2703, 0.8932, -1.3908, 0.3540, 0.7494, 1.9005, 0.7303,
-1.6385, 0.7400, -0.2965, 0.6675, 0.3963, 0.1274, 1.8464, 1.7222,
-1.1801, 1.0254, -0.8271, 0.0415, -0.4320, 0.8026, -0.0814, -0.7572,
-1.5163, 0.9426, -1.1508, -1.6043, 0.9627, -0.4874, 1.8824, 0.3738,
-0.9084, -0.7828, 1.7626, 0.0361, -1.0051, -0.1684, -1.7264, 0.5657,
-0.1280, 1.4076, 0.4904, -0.8532, -0.4779, -1.0025, 0.6173, -0.5496,
0.9116, 0.2816, 0.1610, -0.4029, 1.1215, -1.1204, 0.2299, 0.0634,
0.0428, -0.0110, 0.1010, -1.2060, -1.2150, -0.0319, 1.2655, -0.8516,
0.9262, 0.9088, 0.7849, -1.6405, -1.9032, -1.6078, 0.8318, -0.9396,
0.6116, -2.6312, -0.0104, 1.2789, 0.3844, -0.8725, -0.8324, 1.2025,
2.0355, -0.9117, 1.4998, 0.4595, -0.2576, -0.6752, 0.7153, -1.2831]])
12: 四颗 词向量: tensor([[ 2.2653, -0.0990, -0.4151, 1.7691, 0.0125, 0.0868, -0.2665, 0.1249,
0.6308, -1.0945, -0.7040, 0.4018, 0.6617, -1.7021, -1.4128, 0.5096,
-0.8145, 0.7628, -0.0089, -0.4675, 1.6505, 1.2618, 0.2549, 0.3889,
-0.7146, 0.3026, 0.8814, -0.1159, 0.1816, -0.8549, -1.1286, -0.2564,
3.1843, 0.2464, -0.9187, 0.9241, 0.1091, 0.1357, 0.4793, -1.6307,
-0.3821, -0.7562, 0.6073, -0.0850, 1.6427, 0.4002, -0.8012, 0.2096,
-1.5070, 1.4767, -0.6066, 0.9431, -1.7317, 0.0050, -2.2413, -1.9389,
-1.3615, 0.5443, 0.3790, -1.5023, -0.9213, -0.4121, 1.5587, -0.0514,
0.0400, 0.8381, -0.6121, -1.7556, 0.9796, 0.4298, 0.4889, -0.6758,
-0.6474, 0.1093, -0.7632, -0.2690, -1.3831, 1.3360, -0.9560, 0.4525,
0.2931, 1.3745, -0.2168, 0.3472, 1.0812, 0.1240, -0.0416, 1.7732,
-0.3745, -2.5600, -0.4112, -0.2439, -0.6894, 0.1595, 0.2341, 0.7727,
-0.7041, 1.1107, 0.2782, 0.1178, -0.4656, -1.3687, -0.0236, -0.6206,
0.6577, 0.4913, 0.6707, -0.2326, 0.4894, -1.6483, 0.4459, -0.8915,
0.1402, 1.6414, -0.6836, 0.7385, -2.5696, -1.5428, 0.3977, -0.5403,
0.9011, -0.9073, -0.3712, 0.4510, 0.0162, 0.2468, -0.1403, -0.1803]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
13: 连成线 词向量: tensor([[-1.0089, 1.9068, 0.1027, 2.6734, -0.5348, -0.6113, 0.1489, 0.8982,
1.1933, -0.9432, 1.3036, -0.6435, -0.5043, -0.2422, 2.4747, 0.7518,
0.5725, 1.8265, 0.7251, -0.3818, 0.4764, -0.6340, -1.4265, 0.4799,
-0.4832, 0.0947, 1.5408, 0.5760, 0.7311, -1.2044, -0.2357, -1.4807,
1.8719, -1.8763, 0.5489, 0.6711, -0.1352, -0.2332, -1.1570, -0.4624,
1.1595, 1.5324, 0.9102, 0.7717, 0.0449, 0.4176, -0.5125, 1.6061,
-1.4227, -1.1869, -1.2770, 0.3141, -0.3876, -0.2360, -0.3205, 0.4684,
0.5411, 1.0962, -2.0393, -0.1298, 0.4447, -1.0904, -1.3099, 0.7801,
0.2982, -0.9387, -0.0730, -0.7430, -0.5198, 0.8293, 0.2138, 0.1790,
0.9236, 0.9367, -0.0730, -0.5329, 0.1728, 0.0316, -0.3757, -2.4010,
-2.8439, 0.8637, 1.6214, 1.0395, -0.6404, -1.1814, -0.3843, 1.4762,
0.5888, 0.0333, -1.2155, 0.7936, -0.9066, -1.1480, -1.2870, 0.5459,
-0.2048, -0.5562, 0.6448, -0.8447, 0.9164, -3.3514, -1.1823, 1.2056,
-0.8367, 1.2647, -1.4880, -1.6201, -0.2069, -0.0443, 1.3610, 2.5970,
1.5484, 0.0419, 0.3360, -0.1804, -1.3359, 0.9980, -1.1611, -0.7898,
0.6173, -2.0333, -1.6544, -0.7136, 0.0444, -1.3843, 1.0230, 0.1293]])
14: 背著 词向量: tensor([[ 0.7876, 2.1792, 2.2059, 0.3068, 1.0159, 0.2849, -0.4607, 2.0255,
0.1264, -1.1946, 0.7437, 0.3287, 0.5098, -0.6641, -1.1045, -1.6229,
-0.6425, -0.0654, -2.1731, -0.0184, -1.2673, -0.7136, 0.9171, -1.4792,
-0.0204, -0.2011, 1.3700, 0.8129, 0.8334, -0.6800, -1.3692, 0.0295,
2.7637, 0.8466, -0.2823, -0.0622, -0.5180, -0.2318, 0.9640, 0.7181,
0.5606, -0.2445, -0.6748, -0.0374, -0.4695, 1.4243, -1.4139, 0.1825,
-0.1100, 0.0981, -1.6561, -2.0866, -1.4798, -0.4042, 1.6569, -0.2554,
1.3079, -0.2598, -1.4106, -1.6662, 1.0674, 0.5720, -0.1954, -0.6007,
0.8412, -0.8076, -1.1911, 0.8255, 0.9852, -1.8515, 0.1371, 1.5273,
1.4887, 0.1104, -1.1449, -0.4188, -1.0321, -0.2365, 1.0290, -1.3560,
0.4288, 2.0059, 0.1673, 1.2921, -0.3048, 1.0754, -0.3310, -1.8982,
-1.1110, 1.3395, -0.6198, -1.3844, -0.2225, -0.3677, -0.0481, -0.0558,
-0.2748, -2.0517, 0.7060, 0.7633, -1.7125, 0.1903, 0.2197, -0.2919,
0.8530, 0.9545, 1.1665, -0.2727, -1.2817, -0.1791, 0.7877, -0.4660,
-1.0399, -0.3832, 0.6765, -0.9848, 0.7830, -0.1709, -0.1176, -0.3470,
0.0316, -0.0032, -0.6891, 0.3376, 0.9826, 0.9525, 0.7986, -0.1708]])
15: 背 词向量: tensor([[ 2.3734e-01, 1.2708e+00, 2.5140e-01, 1.2089e-01, -1.5100e-02,
5.2751e-01, 5.9030e-01, 1.4860e+00, 1.3934e+00, 3.5726e-01,
9.1020e-01, -2.2218e+00, 3.7391e-01, 9.5507e-01, -4.4845e-01,
2.5683e-01, 2.3311e+00, -1.0414e+00, 1.3710e+00, 4.8989e-01,
-1.2335e+00, -6.0398e-01, -3.6508e-01, -9.8114e-01, 8.4317e-01,
4.8411e-02, 1.4439e+00, 4.6530e-02, 1.6278e-01, -4.2410e-01,
4.2322e-01, -1.2813e-01, -1.2249e+00, 3.5543e-01, 5.3040e-01,
1.9645e+00, -5.2517e-01, -9.5649e-01, -6.0416e-01, 1.7216e+00,
2.0524e-01, -1.1140e+00, 5.7797e-01, 2.4478e-01, 1.1673e+00,
-1.8646e+00, -4.1053e-01, 1.3479e+00, 1.4761e+00, -5.1711e-01,
7.3766e-02, 2.0067e-01, 3.1754e-01, 9.2665e-01, 2.1854e+00,
3.1614e-01, -2.8709e-02, 2.2981e-01, -4.9327e-01, 8.8275e-01,
-6.8942e-01, 4.4588e-01, -3.0888e-01, -7.8357e-01, -6.0020e-01,
1.0101e-01, 8.9447e-01, -8.2727e-01, 1.0017e+00, 6.5177e-01,
2.0916e+00, -3.2217e-01, 7.6594e-01, -1.0357e+00, 1.4412e-01,
-2.2681e-01, 3.1293e-01, -4.7175e-01, -6.6384e-01, -1.2676e+00,
5.0739e-01, 1.6433e-01, -3.1845e-01, -6.0663e-01, 1.4500e+00,
8.3583e-01, -1.9677e+00, 8.0365e-02, 4.8449e-01, -2.3415e-01,
1.5498e+00, -1.0268e+00, -1.1252e+00, -3.6096e-01, -9.8169e-02,
-1.1103e+00, 2.4518e+00, -2.9703e-04, -4.7274e-01, 4.6164e-01,
1.3386e+00, -1.0836e+00, 6.7105e-01, 1.0665e+00, -1.0827e+00,
1.3817e+00, -4.3349e-01, 5.8398e-01, 6.0266e-01, 4.5929e-01,
1.4923e+00, 1.8565e-01, -5.4926e-01, 7.7824e-02, -5.4307e-01,
8.9839e-01, -5.7604e-01, -1.0120e+00, -5.1401e-02, -4.5502e-01,
1.9257e+00, 3.5499e-01, 3.3945e-01, 1.1403e-01, -5.6473e-01,
2.9557e-01, -2.5702e-01, -1.4799e+00]])
16: 默默 词向量: tensor([[ 0.5855, 0.5649, 1.2516, -1.0654, 0.0194, 0.7558, -0.9596, -1.8844,
-0.8768, -0.9258, 0.7908, -1.4043, -1.2133, -0.8897, 1.1674, 0.4095,
0.2884, -0.2062, -0.3164, 1.5688, -0.8875, -0.3280, -0.7993, 1.1081,
1.7414, 2.3032, 0.4477, 0.3683, 1.3404, 0.5538, -0.3748, -0.9234,
-1.6365, -1.1173, 0.8437, 1.5516, -0.8803, 0.3150, 0.1510, 0.6790,
0.1267, 3.1082, -2.0576, -1.5079, 0.7249, -0.5747, -0.8275, -0.2894,
0.7550, 0.3751, -1.0428, 1.2397, -0.3644, -0.0388, -0.5385, 0.3079,
0.0749, 0.1902, 1.7752, 0.2350, -0.8458, -0.2173, -1.0261, 0.1243,
1.8935, -0.6879, 0.2402, 0.1903, -0.2396, 0.2941, 3.3073, -0.3757,
-1.8381, 2.1608, 0.0395, 0.5703, 0.6058, -0.2424, 0.0935, -1.1924,
-1.1714, 0.1079, 0.0437, -0.7684, 0.2372, 0.8401, 0.0348, 0.0776,
1.7754, -1.3459, 1.0290, -1.6607, -0.8305, -0.9023, -1.0465, -0.9252,
-0.6129, -0.5499, -1.0712, 0.5483, 2.3140, -0.7705, -1.3616, -0.4511,
1.5649, 0.0098, -1.5093, -0.7352, 0.0066, -1.0004, -0.2411, 0.5052,
1.9473, 0.1918, 0.3365, -0.0226, 0.9867, -0.8436, -1.3735, -0.8803,
-0.9717, -0.7108, -1.7845, -1.5892, -1.1767, -1.5818, -1.2533, 0.3954]])
17: 许下 词向量: tensor([[-1.9168e+00, 4.8427e-01, 1.6830e+00, 1.7918e+00, 1.2230e+00,
2.1999e-01, -5.0684e-01, 1.4321e+00, -6.7233e-01, 8.0671e-01,
-4.4662e-02, 1.6716e+00, -1.0406e+00, -2.6862e-01, 9.4382e-01,
-6.5747e-02, 1.6865e+00, -2.2126e-02, 7.2788e-02, 9.4403e-01,
-2.3236e+00, -1.3564e-02, 1.4283e+00, -1.2860e+00, -5.1389e-01,
1.6175e+00, 1.0914e+00, -1.1820e+00, -1.4320e+00, -4.6109e-02,
-9.6742e-01, 2.1611e+00, -5.6264e-01, -1.1565e+00, 1.9390e-01,
9.2234e-01, 1.0021e-02, -6.5935e-01, -8.2297e-01, -5.2027e-01,
-1.2664e+00, -7.5104e-01, 3.7712e-01, -1.0337e+00, 2.1133e-01,
1.1996e-01, 1.7219e-01, -4.2914e-01, -7.7655e-01, -1.2369e+00,
1.4316e+00, 5.9886e-01, 2.7075e-01, -1.0055e+00, -4.1905e-01,
2.0641e-01, 1.0546e+00, -1.0288e+00, -4.2799e-01, 1.7075e+00,
3.4562e-01, -6.3579e-01, 1.8177e+00, -3.1740e-01, 1.3355e+00,
8.3655e-01, -1.2970e-01, -5.6623e-01, 1.3618e+00, -8.1217e-01,
-5.9931e-01, 1.7257e+00, -1.7200e+00, -2.0489e-01, 1.8407e+00,
1.9718e-01, 5.7118e-01, 1.6514e+00, 1.3401e+00, -1.3435e+00,
-4.0144e-01, -1.0155e-01, -1.1944e+00, -4.5984e-02, 6.7646e-01,
-4.0648e-01, 5.7433e-01, -1.1582e+00, 2.7859e-01, -1.0964e+00,
-1.3926e+00, -7.5777e-01, -2.0582e+00, 2.8480e-01, -1.1150e-01,
-1.1630e+00, -1.2977e+00, -1.3570e+00, -9.2553e-01, -2.9389e-01,
-8.6199e-01, 6.0487e-01, -7.3591e-01, 8.8837e-01, 2.7932e+00,
-2.0715e-01, 4.4492e-01, -1.5744e-01, -1.2521e+00, 2.5512e-01,
2.5749e-03, 1.0270e+00, 1.4577e+00, 1.2816e+00, 1.1000e+00,
8.1408e-01, -8.5781e-01, 3.9352e-01, -1.1353e+00, -6.6646e-01,
-2.3900e-01, -8.6223e-01, 1.4470e+00, 1.7640e-01, 2.9097e-01,
4.9116e-01, -1.0895e-02, 7.6424e-01]])
18: 心愿 词向量: tensor([[-0.2722, 1.5364, -1.3689, -1.7894, -1.6638, -1.2606, 1.5057, -0.3066,
0.2299, 0.3172, -0.1636, -0.5218, 0.6939, -2.5031, 0.8794, -0.3524,
0.6943, -0.2636, -0.0768, -0.4456, 0.7122, -0.2961, 0.5403, 0.9556,
0.6172, 0.1403, 1.6154, -0.6350, 1.7847, 0.2646, -0.6640, -0.1668,
2.0648, 0.0089, 0.3429, -2.2141, 1.8766, -1.4421, 1.6883, 1.8436,
-0.7833, -0.5841, 0.2226, 0.1777, -1.4001, -1.2400, -1.3822, 2.6764,
0.8447, -0.4104, -0.4826, 0.9347, 0.0041, 0.4420, 0.2884, 0.1651,
-0.0656, 1.5672, 0.7912, -0.1630, -0.6801, 0.9197, -0.2025, -0.5427,
0.8947, -1.0273, 1.2590, -1.7895, 0.1924, -0.7166, 1.5944, 0.2242,
-0.9353, 0.1765, 0.9089, -0.6721, -0.5409, -1.2609, -0.5078, -0.2503,
0.1845, 1.0438, -0.0463, -1.0236, 0.0978, -0.9157, -0.8683, -0.0148,
-1.1952, 2.4714, -0.2442, -1.3386, -1.5403, -1.3036, -1.4829, -0.6095,
-0.5726, -1.0416, -0.3513, 0.3971, 1.3704, 0.0423, -0.5206, -1.7684,
-0.7518, -0.4393, 0.4668, 0.5841, -1.1916, -1.0110, 0.6833, -1.2590,
-0.7883, 1.3484, 0.1021, -0.1271, 0.3129, -0.0804, -1.5882, -0.8672,
0.3482, 1.3307, 0.1446, 0.0331, -1.0778, -0.1092, -1.1599, -0.4233]])
1:
词向量: tensor([[ 1.4771, 0.8061, -0.4299, -0.5632, -1.3281, -0.5809, -0.4281, 0.5693,
0.6510, -0.1457, -0.2895, -1.2239, -0.8608, -1.1156, 1.4625, 0.9993,
2.6551, -0.2841, -0.1222, -0.1759, 0.2747, 0.8773, -1.2195, 0.0723,
-1.1253, -0.3896, -0.5094, -0.3521, 0.9901, 1.1108, 1.1191, -1.1345,
-0.5516, -1.0788, 0.2044, 0.1993, 0.2574, -0.3992, 0.0040, 0.1144,
-0.7567, 1.0407, -0.4901, -0.9804, 0.4055, 0.4972, -0.4097, -0.2506,
-0.0374, -0.2787, 0.5761, -0.0540, -0.8492, -0.7583, 1.0121, -0.6011,
-1.4103, 0.6407, 0.3240, 0.5556, -1.6938, 0.2747, 0.7353, -2.0933,
-1.7379, -0.5252, 1.3509, -0.2661, -0.3593, -0.2447, -1.2453, 0.2860,
-1.1808, 1.6014, -1.1948, 1.3895, 0.4737, -1.4571, 1.1382, -0.8261,
0.5641, -2.5230, 0.2975, 1.0076, -2.1213, 0.0967, 0.6917, 1.7999,
0.4339, 1.3127, 1.9793, 1.2312, -0.2442, 2.5434, 0.1964, 0.1350,
-1.6063, 0.6460, 0.2100, 0.4153, -0.0708, -0.2820, -0.3491, 0.3494,
-1.0825, 1.4995, 1.1523, 0.9549, 0.7601, 0.5949, 1.0576, 0.5871,
-1.3045, -0.5725, 0.5301, 0.5057, 0.8488, 0.5929, 1.0189, 0.0787,
0.9573, -0.7332, 0.9140, -0.4795, 0.2905, 0.9838, 0.3004, -0.2756]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
19: 看 词向量: tensor([[-1.6378, 1.1989, -0.6366, 0.0372, 2.0751, 1.3119, 0.4727, 1.2676,
-0.3800, 0.3326, -0.7496, -0.7480, -0.0793, -0.6415, -1.0095, 0.9856,
0.5956, -0.5635, -0.7542, 0.3592, 0.9146, -0.2246, -1.1856, -0.9349,
-0.1095, -0.8371, 0.9123, -1.0214, -0.7292, 0.4273, 1.1791, -0.6683,
-1.8901, -0.2477, -0.1068, -0.7313, 0.4921, -0.0978, 0.1945, 0.4086,
-0.3137, 0.2452, -0.8536, -0.2235, -0.6163, 0.8495, 1.2052, -0.4979,
-1.2943, 0.4710, -0.2844, -0.5532, -1.4932, 0.7001, -0.1473, -0.9754,
1.8101, -0.0287, 0.6692, -0.6750, 0.6471, 1.9667, 0.2542, 1.1195,
-0.7262, -0.3463, -0.2518, -2.9235, 0.7129, 0.8904, 0.2089, 1.8507,
-0.0445, -0.3453, 0.6186, -0.1506, 0.5771, 0.2950, -1.2261, -1.7731,
0.2479, 0.3745, 0.0143, -0.4963, 0.4838, -0.1708, -1.1928, -0.1565,
-0.4288, -1.4751, 1.0072, 1.3855, 1.6219, 1.7316, -0.4740, -0.4865,
-0.5089, 0.2501, 0.6099, -0.2189, -0.0841, -0.9087, 0.6845, 0.1901,
0.0069, 2.8011, -1.9380, 0.2300, -1.2650, 0.9575, 0.0194, -0.1019,
-0.6718, -0.7557, 1.0150, 0.5572, 0.0731, 0.0102, 1.2525, -0.9054,
0.4344, 0.5597, 0.6715, -0.4902, 0.1535, -0.7845, -0.7937, -1.4195]])
20: 远方 词向量: tensor([[-2.2151e-01, 2.7015e-01, -2.5084e-01, -4.4097e-01, -1.2715e+00,
-2.9254e-01, 6.1142e-01, -1.9627e+00, 5.2096e-01, 1.2656e-01,
-1.3257e+00, 3.0418e-01, 3.0586e-02, -1.2778e+00, -8.1623e-04,
7.6972e-01, 4.1588e-01, -9.7389e-01, 6.3766e-01, -7.4420e-03,
3.2223e-01, 6.0338e-02, 7.8877e-04, 6.7707e-02, 1.1984e+00,
4.5039e-01, 5.3452e-01, -3.4807e-02, 1.5357e+00, 3.0241e-01,
3.5626e-01, -4.7850e-01, -3.9241e-01, -5.7744e-01, 6.6992e-01,
8.0270e-01, 3.1736e-01, 9.4924e-01, 5.7952e-01, -6.4206e-01,
1.0187e+00, -2.1815e+00, -8.4608e-01, 1.6100e+00, 1.7417e+00,
-7.7409e-01, -1.0568e+00, 1.0163e+00, -1.8650e+00, -7.5423e-01,
1.7711e-01, 1.1056e+00, -5.5982e-01, 2.5708e-01, -1.1924e+00,
5.4293e-01, 1.2407e+00, 1.0870e-02, -1.0442e+00, 2.4535e-01,
-3.9359e-01, -5.6523e-01, 6.7341e-02, 7.0170e-01, -9.0430e-01,
-2.1122e-01, 3.7679e-01, -3.6847e-01, -1.1514e+00, 3.1001e-01,
-7.4094e-01, -1.8485e+00, 1.4766e+00, 2.2901e-01, 1.5023e+00,
-3.2748e-01, 1.2463e+00, 3.8577e-01, -3.4778e-01, 7.4142e-01,
6.4087e-01, -3.2994e-03, -1.3176e+00, 9.4887e-01, 2.8549e-01,
-3.3758e-01, 4.4792e-01, 8.2648e-01, -5.9769e-01, 9.3186e-01,
8.1247e-01, 8.7909e-01, -2.0067e+00, -1.1447e+00, -2.2422e-01,
1.3603e+00, 1.7765e-01, -1.0203e+00, -4.3007e-01, -3.1473e-01,
-9.5439e-01, -1.6629e+00, 1.5106e+00, -3.3230e-01, -2.0346e-01,
1.0844e+00, -1.2731e+00, 1.3654e+00, -3.3133e-01, 1.4257e+00,
2.8431e-01, -7.4210e-01, 2.0834e-01, 1.9805e-01, -5.5987e-01,
7.3535e-01, 6.8641e-01, -1.0080e+00, 8.2997e-01, 8.7261e-01,
-5.4841e-01, -5.6013e-01, -1.1264e+00, -9.1787e-01, -1.5226e-01,
1.5964e+00, -1.1799e+00, -1.3598e+00]])
3: 的 词向量: tensor([[-0.5050, -0.7864, 0.4733, 0.5857, -0.3920, -1.0719, 0.7779, 0.7414,
1.1750, -0.0547, -0.0028, 0.5988, -0.9794, 0.5023, -0.9066, 0.3467,
0.1048, 1.0200, 0.9090, 0.7166, -0.0776, 0.2908, -0.0487, 0.2978,
-0.9683, 1.3314, -0.7385, 0.3636, -1.8954, 1.8396, 0.0174, 0.5640,
1.0021, -0.3806, -1.3091, -0.3162, 0.6868, -1.1819, 0.9884, -0.5532,
0.3543, -0.0881, -0.7711, -0.3639, 0.0058, -0.3205, 0.2947, -0.9907,
-1.1410, -2.0466, 1.1630, -0.0579, 1.5855, -1.0066, -1.8496, 0.9140,
-0.5690, 0.7007, -0.0466, -1.4705, 1.4759, -1.2716, 0.1965, 0.1914,
-1.4481, 1.0102, -0.9801, -0.8014, -0.0049, 0.0173, -2.2484, 1.9075,
-0.9655, 0.9100, -1.4905, -0.6812, -1.1242, -0.8694, -0.7923, -0.7349,
0.3795, -0.1257, 0.4920, -1.4052, 0.5717, 0.9546, -0.9034, -1.5733,
-2.4029, -0.4214, -1.7982, 0.4383, -1.5436, 0.4382, -1.0642, -0.6521,
-0.8291, 0.3413, 0.5453, -0.3017, 1.9028, 1.1402, -1.1944, 0.7615,
-1.3941, 0.7679, 0.4553, -0.0162, -0.6040, 0.6493, 0.3233, 2.2985,
0.6532, -1.0028, 0.1517, 0.1298, 0.7207, -1.3053, -1.0424, 1.2343,
1.0513, -0.2350, 0.3177, -1.6808, 0.9402, -0.4769, -0.0186, 0.3269]])
21: 星 词向量: tensor([[ 0.9343, -0.6351, 2.5973, -0.1343, -0.5938, 0.6907, -1.9802, 1.2802,
-0.7350, -2.3115, 1.1310, -0.5244, 2.0018, -0.0078, -0.4999, 0.1240,
1.3521, -1.0262, -0.3435, 1.1001, -1.4145, -0.5343, -0.3016, 1.3090,
-0.4084, -0.0651, 1.6233, -1.3712, -1.4570, -1.3325, 0.5027, 1.6271,
1.7070, -0.2588, 0.4852, -0.2199, -0.4746, -2.1739, -0.3085, -0.7908,
0.6772, 1.5464, 0.6278, 0.4157, 0.0978, 1.3029, -0.1206, 0.9472,
1.2165, 0.5168, -0.6556, 2.5364, 0.7907, -0.1370, -1.3545, -0.2952,
-1.6427, -2.1023, 0.3443, 0.2859, -1.1653, 0.3909, -0.1909, -1.0621,
0.8516, -0.3537, -0.3861, -0.1562, -0.7356, 0.8104, 0.6886, -0.6016,
0.5568, -0.6973, 0.6600, -0.7863, -0.3930, 0.2882, -1.1542, 1.0403,
-0.8155, 0.8802, -1.9443, 0.8925, 1.1167, -1.6002, -0.2377, 0.4621,
0.8763, -0.8899, 0.2207, 0.3117, 0.4775, 0.8951, -0.3195, -1.9292,
-0.1436, -0.6130, 1.0702, -0.2847, -0.1454, -0.2137, -0.6822, 0.0947,
-0.7980, -0.3340, -0.4143, 0.4056, 0.8545, 0.3359, -0.3429, 1.8712,
0.2058, 0.4364, 0.2536, 0.7652, 0.0403, -0.0690, -0.6465, 0.0350,
-0.3616, 0.3416, 0.4873, -0.6190, 0.2839, 1.4134, -0.8080, -0.1496]])
22: 如果 词向量: tensor([[ 0.1246, 1.2048, 1.4993, 0.2169, -0.6324, 0.3480, 0.5321, -0.4643,
0.1853, -0.2201, -0.4717, 0.7266, -0.3807, -1.1129, 1.1964, 0.6585,
1.2434, -0.0832, -1.5133, 0.7713, -0.8878, 0.5984, 0.7236, 0.3181,
-0.7097, 1.2266, -0.4268, -0.3424, 0.6792, -1.2795, 0.4660, -0.9344,
-1.3268, 0.2262, -1.2773, 1.2925, 0.4822, 0.6024, 0.2507, 0.2106,
0.1559, 0.0466, 0.7026, 0.8076, -2.9059, -1.9419, 2.2187, -1.6862,
-0.3676, -0.8750, 0.3967, -1.0897, 0.7214, -1.0434, -0.9400, 0.9683,
0.4671, 0.7030, -0.8523, 0.0555, -0.6136, 0.5558, -0.4495, 0.4403,
0.8187, -1.5777, -0.7845, 1.0370, 0.4199, 0.7749, 1.2567, -0.3439,
-0.3614, -1.3200, -1.0704, -0.9868, 1.0314, 1.9150, 1.4884, 0.2266,
-2.5519, -0.6866, -1.1764, 0.7929, -2.9061, 0.5399, 0.0444, -0.2442,
-0.1906, 0.4717, 1.3004, 0.4692, 0.6341, -0.8144, 0.0730, -1.9063,
0.6340, 1.0164, 0.7114, 3.6052, 0.6208, 0.4212, -0.8814, 0.8068,
-1.5279, -0.1897, 1.7267, 0.2021, 1.9491, 2.0983, -0.2440, -0.0369,
-0.9810, 0.0416, -1.5853, -0.9668, -0.3975, 0.6687, 1.0153, -1.6619,
0.2119, 0.1046, -0.0508, 0.1504, -0.0602, 1.0055, 0.1538, 0.1054]])
23: 听 词向量: tensor([[ 1.2544, 0.9021, -0.4938, 2.3340, 0.3996, -0.6306, 0.2922, -0.8091,
1.6602, 1.8169, -0.8131, -0.1887, 1.7214, -1.0289, 1.1330, -1.0458,
-0.7295, -0.1894, 0.1186, 1.1407, 0.8980, -1.5787, 0.3620, -0.9011,
-0.5341, 0.4732, -0.0771, -1.0971, 0.6488, 0.5311, -1.2628, -0.3052,
-0.1396, -1.3593, 0.9019, 0.5569, 0.8465, -1.5078, 0.5944, 0.1621,
0.3068, 1.3318, -0.7125, 0.5898, -1.1402, 1.4340, 1.3863, 1.4169,
1.3509, -0.7427, 0.4680, 0.8152, 0.6923, 0.1683, -0.0130, 0.2283,
-0.0542, -0.4965, 0.1802, -0.8256, 0.4025, -0.0358, -0.5260, 0.3125,
1.2364, 1.0025, 0.2867, -2.1741, 2.1622, -1.1709, 1.3145, 0.8909,
0.7970, 0.3928, -0.2470, -0.9782, 1.9170, 0.1687, 0.1538, -0.6906,
-0.5534, -0.3445, 1.1270, -0.1708, 0.9009, 2.3711, -1.2261, 0.2222,
-2.5274, -0.4691, 0.6888, 0.5456, 0.7658, -0.6966, 1.4674, -0.6560,
0.7415, -0.4231, 0.1107, -0.9721, 0.4802, -0.0400, -0.3285, -0.9941,
-2.7940, 1.0055, -0.3630, -0.4238, 0.4130, 0.4586, 0.4073, 1.1068,
-0.4705, -1.2360, 0.7293, 1.0668, -1.2555, -0.1846, 0.9244, -2.0131,
1.5062, 0.2711, -1.8606, -0.3787, 1.3292, 2.0386, 0.7979, -1.6072]])
3: 的 词向量: tensor([[-0.5050, -0.7864, 0.4733, 0.5857, -0.3920, -1.0719, 0.7779, 0.7414,
1.1750, -0.0547, -0.0028, 0.5988, -0.9794, 0.5023, -0.9066, 0.3467,
0.1048, 1.0200, 0.9090, 0.7166, -0.0776, 0.2908, -0.0487, 0.2978,
-0.9683, 1.3314, -0.7385, 0.3636, -1.8954, 1.8396, 0.0174, 0.5640,
1.0021, -0.3806, -1.3091, -0.3162, 0.6868, -1.1819, 0.9884, -0.5532,
0.3543, -0.0881, -0.7711, -0.3639, 0.0058, -0.3205, 0.2947, -0.9907,
-1.1410, -2.0466, 1.1630, -0.0579, 1.5855, -1.0066, -1.8496, 0.9140,
-0.5690, 0.7007, -0.0466, -1.4705, 1.4759, -1.2716, 0.1965, 0.1914,
-1.4481, 1.0102, -0.9801, -0.8014, -0.0049, 0.0173, -2.2484, 1.9075,
-0.9655, 0.9100, -1.4905, -0.6812, -1.1242, -0.8694, -0.7923, -0.7349,
0.3795, -0.1257, 0.4920, -1.4052, 0.5717, 0.9546, -0.9034, -1.5733,
-2.4029, -0.4214, -1.7982, 0.4383, -1.5436, 0.4382, -1.0642, -0.6521,
-0.8291, 0.3413, 0.5453, -0.3017, 1.9028, 1.1402, -1.1944, 0.7615,
-1.3941, 0.7679, 0.4553, -0.0162, -0.6040, 0.6493, 0.3233, 2.2985,
0.6532, -1.0028, 0.1517, 0.1298, 0.7207, -1.3053, -1.0424, 1.2343,
1.0513, -0.2350, 0.3177, -1.6808, 0.9402, -0.4769, -0.0186, 0.3269]])
24: 见 词向量: tensor([[ 0.7362, -1.4712, 1.2106, -0.4481, 0.3059, -0.3769, -0.3850, 1.1246,
-0.4344, 0.9222, -0.5994, -0.4479, 1.4162, -1.3390, -1.6539, 1.8394,
-1.2221, 1.5845, -1.1930, 1.1310, 0.3554, -0.0841, -0.7046, -1.2362,
-0.6700, 2.4575, 1.5531, 0.2896, 0.6560, 1.0525, -0.1828, 0.7062,
-1.3173, -0.0787, -0.5038, -2.3971, 0.4665, 2.1829, -0.1454, -1.2375,
0.4456, -0.1730, 0.9735, -0.0640, -0.9733, 0.9785, 0.9200, 0.0025,
-0.1969, -0.4680, -1.8549, 0.5413, -1.8929, 0.3384, 0.9737, -0.2777,
0.1088, 1.2688, -0.0298, -0.6080, -1.5854, 0.0967, -0.1646, 0.8230,
-1.1896, -0.0592, -0.0293, 0.5168, 0.2211, -0.5586, -0.2930, -1.6045,
0.5938, 0.2630, -1.0017, 0.4197, -0.3070, -1.9273, -1.8329, -0.1540,
0.5780, -0.9059, 0.5529, 1.3260, 0.3143, 0.5037, 0.0775, 0.2414,
-0.5057, 0.3852, -0.4010, 1.0541, 1.1464, 0.8323, -0.5967, 1.2612,
-0.6690, -0.6840, -0.9773, -0.8996, 0.0482, -0.0055, 0.1377, 0.7025,
0.0617, 0.7621, 1.5706, 0.8205, -0.4286, 1.3250, -0.1531, -0.0703,
-1.0932, -0.5865, -0.4240, -1.2477, -0.1975, -0.2549, 0.5158, -0.0439,
-0.5585, -0.0480, -0.7398, 2.1300, -0.9802, -0.0223, 1.9863, -2.0320]])
1:
词向量: tensor([[ 1.4771, 0.8061, -0.4299, -0.5632, -1.3281, -0.5809, -0.4281, 0.5693,
0.6510, -0.1457, -0.2895, -1.2239, -0.8608, -1.1156, 1.4625, 0.9993,
2.6551, -0.2841, -0.1222, -0.1759, 0.2747, 0.8773, -1.2195, 0.0723,
-1.1253, -0.3896, -0.5094, -0.3521, 0.9901, 1.1108, 1.1191, -1.1345,
-0.5516, -1.0788, 0.2044, 0.1993, 0.2574, -0.3992, 0.0040, 0.1144,
-0.7567, 1.0407, -0.4901, -0.9804, 0.4055, 0.4972, -0.4097, -0.2506,
-0.0374, -0.2787, 0.5761, -0.0540, -0.8492, -0.7583, 1.0121, -0.6011,
-1.4103, 0.6407, 0.3240, 0.5556, -1.6938, 0.2747, 0.7353, -2.0933,
-1.7379, -0.5252, 1.3509, -0.2661, -0.3593, -0.2447, -1.2453, 0.2860,
-1.1808, 1.6014, -1.1948, 1.3895, 0.4737, -1.4571, 1.1382, -0.8261,
0.5641, -2.5230, 0.2975, 1.0076, -2.1213, 0.0967, 0.6917, 1.7999,
0.4339, 1.3127, 1.9793, 1.2312, -0.2442, 2.5434, 0.1964, 0.1350,
-1.6063, 0.6460, 0.2100, 0.4153, -0.0708, -0.2820, -0.3491, 0.3494,
-1.0825, 1.4995, 1.1523, 0.9549, 0.7601, 0.5949, 1.0576, 0.5871,
-1.3045, -0.5725, 0.5301, 0.5057, 0.8488, 0.5929, 1.0189, 0.0787,
0.9573, -0.7332, 0.9140, -0.4795, 0.2905, 0.9838, 0.3004, -0.2756]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
4: 它 词向量: tensor([[ 0.5916, -1.0563, 1.2493, 0.7314, 1.2289, 0.7857, 0.1050, 0.5233,
-0.3904, 1.3847, -1.5685, -0.4917, -0.1517, 1.7520, -0.8190, -0.1675,
-1.3133, 0.8705, 0.7957, 0.1744, 0.8354, 0.6104, 0.2760, -0.4395,
-0.2033, 0.0680, 0.9529, -1.0169, -0.7344, 1.0691, 0.3209, 0.4052,
0.7633, 1.9166, 0.0961, -1.2044, -0.2926, -0.1764, -0.7288, 0.1354,
0.5887, 0.2954, 0.1310, 1.3626, 0.0274, 0.1746, -1.3382, 1.2703,
0.9801, -0.6425, 1.1956, 0.6877, 2.4781, -0.7597, -1.1181, -0.3879,
0.1513, -2.1913, -0.1680, -2.7475, 1.8495, -0.6694, 0.1333, 0.7756,
-1.0243, -1.3028, -0.9127, 0.4272, 0.3535, 0.7610, 1.2806, 0.9609,
-0.6524, 0.1703, 1.1705, -0.5499, 0.8303, 1.2069, 2.1746, 0.3545,
-0.1621, -0.1059, -1.1698, -0.4564, 0.1297, 0.0899, -1.5461, 0.0992,
0.0704, 2.3755, -0.4975, -0.0843, -0.1291, 2.9284, 0.0932, -1.7693,
1.0329, -1.0583, 0.7528, 0.2621, -0.6719, 0.9021, -0.5467, -0.9155,
0.2840, -0.1062, -1.0615, 0.2068, -1.1440, 1.1453, -0.2993, 2.2820,
1.1952, 1.1511, 0.6852, 0.4618, -1.6429, 0.2492, 0.5235, -1.0211,
-1.6268, -1.1363, -1.7695, 0.6146, -0.0123, 0.2200, 0.4970, -0.4545]])
5: 一定 词向量: tensor([[ 1.2313e+00, 4.5803e-01, -1.0245e+00, 8.1082e-01, 5.8335e-01,
-8.5283e-01, 1.0435e+00, 8.9049e-01, 1.4090e+00, 4.7953e-01,
2.1237e+00, -1.4484e+00, 1.0245e+00, -4.9148e-01, -1.1194e-01,
-9.3104e-01, -1.7203e+00, -3.3872e-01, 7.6409e-01, 1.1902e+00,
-1.5168e+00, -1.8148e-01, -5.0774e-01, 6.8826e-01, 8.9191e-01,
2.2935e-01, 1.3293e-01, 9.2889e-04, 2.3604e+00, -2.2407e+00,
1.5633e+00, -2.8127e-01, -1.7008e+00, -4.7021e-01, -3.5090e-01,
2.1340e+00, -7.8498e-01, -1.1078e+00, -1.3837e-01, -5.3655e-01,
4.7906e-01, 1.6595e+00, -1.0748e+00, -2.5242e-01, 6.4267e-02,
5.7673e-01, -7.6405e-01, 3.6373e-01, -1.1225e+00, -1.1487e+00,
2.8528e-01, 8.0939e-01, -7.5029e-01, -1.7576e-01, -9.6853e-01,
-5.7401e-01, -9.3717e-01, 5.6623e-01, -3.4602e-01, -2.3836e+00,
2.3034e-01, 1.0035e+00, 1.7825e+00, -1.0826e+00, -2.3943e-03,
1.2059e+00, -3.3547e-01, 8.3382e-01, -3.0155e-01, -7.3170e-01,
-1.1627e+00, 1.1664e+00, -1.0954e+00, -1.4304e+00, 2.4684e-01,
-9.4767e-01, -3.6145e-01, -1.0142e+00, 1.8561e-01, 2.7217e+00,
2.8907e-01, 1.4693e-01, 8.9869e-01, -6.0849e-02, 2.1282e-01,
1.2202e+00, -2.2507e-01, 8.0473e-01, 9.2867e-01, 4.1824e-01,
-9.1914e-01, 5.4643e-01, -6.9789e-01, 1.0888e+00, 7.4573e-01,
-1.1722e+00, -1.5506e+00, 1.3988e+00, -1.2377e+00, 7.9709e-01,
-4.1322e-02, -2.2761e-01, -5.1566e-01, 1.7174e+00, -2.4986e+00,
1.5490e+00, 8.7587e-01, -6.0875e-01, -1.8899e-01, -1.9690e-01,
-2.7263e-01, 1.4313e+00, -1.1703e+00, -1.0822e+00, 1.7274e+00,
-7.1926e-01, 1.4563e+00, 1.7508e-01, 1.1314e+00, -6.4319e-01,
9.1328e-01, -7.9965e-01, 1.4625e+00, -9.2181e-01, 1.3153e+00,
3.3298e-01, -2.0157e+00, -1.5097e+00]])
6: 实现 词向量: tensor([[ 1.0926e-01, -3.9009e-01, -1.9415e-01, 6.4970e-02, -2.4715e-01,
5.0884e-01, -6.3888e-01, -1.0772e+00, -6.8568e-01, -9.5716e-01,
-2.4326e-01, -5.2359e-02, -9.9621e-01, 2.3951e+00, -1.4536e-01,
9.9262e-01, -6.5230e-01, -6.0988e-01, 8.1966e-01, 2.2814e+00,
-6.1732e-01, -9.4800e-01, -2.1926e-01, -5.0263e-01, 2.0673e-01,
-1.1291e+00, -8.8755e-01, 8.2712e-01, -6.9201e-02, -2.8709e-01,
-3.6867e-02, 1.7624e+00, -2.8595e-01, -5.4195e-03, 8.5137e-01,
-5.8569e-01, 8.4966e-01, 3.5678e-01, -6.3139e-02, 1.0690e+00,
-4.6396e-01, -4.6860e-01, -6.7270e-01, -7.8498e-01, 3.1458e-01,
-3.3875e-01, 2.7294e+00, 3.2299e-01, -1.9131e+00, 6.1479e-01,
-3.6987e-01, -1.4290e-02, -1.7148e+00, -1.5682e-02, -1.3152e+00,
-1.0467e+00, 1.5903e+00, 6.2825e-01, 9.9813e-01, -1.3689e+00,
4.5732e-01, -6.2837e-01, 1.0551e+00, 6.7639e-01, 1.2566e-01,
-1.1856e-02, -7.7169e-01, -1.9395e+00, 5.4226e-01, 1.5230e+00,
3.7949e-01, -5.4299e-01, 3.9010e-01, 9.7130e-02, -1.2557e+00,
-4.7758e-01, 2.2550e-01, 1.2194e+00, 7.4347e-01, -5.6995e-01,
6.9029e-01, 4.0881e-02, 7.5411e-01, -9.8277e-01, -8.7063e-01,
3.2578e-01, -7.3621e-01, -1.0762e+00, 9.4652e-01, 8.2750e-01,
-1.0640e+00, 7.8649e-01, 1.5135e+00, -9.6734e-01, 9.6932e-03,
-1.3051e+00, -3.2069e-01, -1.0847e+00, 7.4306e-01, 3.5581e-01,
-2.4428e-01, 4.5607e-01, 7.4739e-01, -6.4819e-01, -1.5006e-01,
-6.1586e-01, 2.9964e+00, 1.3413e+00, 2.8650e-01, -1.8197e-03,
-6.7891e-01, -9.4347e-02, -9.0407e-01, 4.4775e-01, -6.3878e-01,
-1.0743e+00, 1.3872e+00, 3.3750e-01, -8.2506e-01, -2.2975e-01,
-1.5826e+00, -1.9349e+00, -4.6668e-01, 6.8606e-01, -1.8853e-01,
-8.4801e-01, -1.4381e+00, 7.6804e-01]])
4: 它 词向量: tensor([[ 0.5916, -1.0563, 1.2493, 0.7314, 1.2289, 0.7857, 0.1050, 0.5233,
-0.3904, 1.3847, -1.5685, -0.4917, -0.1517, 1.7520, -0.8190, -0.1675,
-1.3133, 0.8705, 0.7957, 0.1744, 0.8354, 0.6104, 0.2760, -0.4395,
-0.2033, 0.0680, 0.9529, -1.0169, -0.7344, 1.0691, 0.3209, 0.4052,
0.7633, 1.9166, 0.0961, -1.2044, -0.2926, -0.1764, -0.7288, 0.1354,
0.5887, 0.2954, 0.1310, 1.3626, 0.0274, 0.1746, -1.3382, 1.2703,
0.9801, -0.6425, 1.1956, 0.6877, 2.4781, -0.7597, -1.1181, -0.3879,
0.1513, -2.1913, -0.1680, -2.7475, 1.8495, -0.6694, 0.1333, 0.7756,
-1.0243, -1.3028, -0.9127, 0.4272, 0.3535, 0.7610, 1.2806, 0.9609,
-0.6524, 0.1703, 1.1705, -0.5499, 0.8303, 1.2069, 2.1746, 0.3545,
-0.1621, -0.1059, -1.1698, -0.4564, 0.1297, 0.0899, -1.5461, 0.0992,
0.0704, 2.3755, -0.4975, -0.0843, -0.1291, 2.9284, 0.0932, -1.7693,
1.0329, -1.0583, 0.7528, 0.2621, -0.6719, 0.9021, -0.5467, -0.9155,
0.2840, -0.1062, -1.0615, 0.2068, -1.1440, 1.1453, -0.2993, 2.2820,
1.1952, 1.1511, 0.6852, 0.4618, -1.6429, 0.2492, 0.5235, -1.0211,
-1.6268, -1.1363, -1.7695, 0.6146, -0.0123, 0.2200, 0.4970, -0.4545]])
5: 一定 词向量: tensor([[ 1.2313e+00, 4.5803e-01, -1.0245e+00, 8.1082e-01, 5.8335e-01,
-8.5283e-01, 1.0435e+00, 8.9049e-01, 1.4090e+00, 4.7953e-01,
2.1237e+00, -1.4484e+00, 1.0245e+00, -4.9148e-01, -1.1194e-01,
-9.3104e-01, -1.7203e+00, -3.3872e-01, 7.6409e-01, 1.1902e+00,
-1.5168e+00, -1.8148e-01, -5.0774e-01, 6.8826e-01, 8.9191e-01,
2.2935e-01, 1.3293e-01, 9.2889e-04, 2.3604e+00, -2.2407e+00,
1.5633e+00, -2.8127e-01, -1.7008e+00, -4.7021e-01, -3.5090e-01,
2.1340e+00, -7.8498e-01, -1.1078e+00, -1.3837e-01, -5.3655e-01,
4.7906e-01, 1.6595e+00, -1.0748e+00, -2.5242e-01, 6.4267e-02,
5.7673e-01, -7.6405e-01, 3.6373e-01, -1.1225e+00, -1.1487e+00,
2.8528e-01, 8.0939e-01, -7.5029e-01, -1.7576e-01, -9.6853e-01,
-5.7401e-01, -9.3717e-01, 5.6623e-01, -3.4602e-01, -2.3836e+00,
2.3034e-01, 1.0035e+00, 1.7825e+00, -1.0826e+00, -2.3943e-03,
1.2059e+00, -3.3547e-01, 8.3382e-01, -3.0155e-01, -7.3170e-01,
-1.1627e+00, 1.1664e+00, -1.0954e+00, -1.4304e+00, 2.4684e-01,
-9.4767e-01, -3.6145e-01, -1.0142e+00, 1.8561e-01, 2.7217e+00,
2.8907e-01, 1.4693e-01, 8.9869e-01, -6.0849e-02, 2.1282e-01,
1.2202e+00, -2.2507e-01, 8.0473e-01, 9.2867e-01, 4.1824e-01,
-9.1914e-01, 5.4643e-01, -6.9789e-01, 1.0888e+00, 7.4573e-01,
-1.1722e+00, -1.5506e+00, 1.3988e+00, -1.2377e+00, 7.9709e-01,
-4.1322e-02, -2.2761e-01, -5.1566e-01, 1.7174e+00, -2.4986e+00,
1.5490e+00, 8.7587e-01, -6.0875e-01, -1.8899e-01, -1.9690e-01,
-2.7263e-01, 1.4313e+00, -1.1703e+00, -1.0822e+00, 1.7274e+00,
-7.1926e-01, 1.4563e+00, 1.7508e-01, 1.1314e+00, -6.4319e-01,
9.1328e-01, -7.9965e-01, 1.4625e+00, -9.2181e-01, 1.3153e+00,
3.3298e-01, -2.0157e+00, -1.5097e+00]])
6: 实现 词向量: tensor([[ 1.0926e-01, -3.9009e-01, -1.9415e-01, 6.4970e-02, -2.4715e-01,
5.0884e-01, -6.3888e-01, -1.0772e+00, -6.8568e-01, -9.5716e-01,
-2.4326e-01, -5.2359e-02, -9.9621e-01, 2.3951e+00, -1.4536e-01,
9.9262e-01, -6.5230e-01, -6.0988e-01, 8.1966e-01, 2.2814e+00,
-6.1732e-01, -9.4800e-01, -2.1926e-01, -5.0263e-01, 2.0673e-01,
-1.1291e+00, -8.8755e-01, 8.2712e-01, -6.9201e-02, -2.8709e-01,
-3.6867e-02, 1.7624e+00, -2.8595e-01, -5.4195e-03, 8.5137e-01,
-5.8569e-01, 8.4966e-01, 3.5678e-01, -6.3139e-02, 1.0690e+00,
-4.6396e-01, -4.6860e-01, -6.7270e-01, -7.8498e-01, 3.1458e-01,
-3.3875e-01, 2.7294e+00, 3.2299e-01, -1.9131e+00, 6.1479e-01,
-3.6987e-01, -1.4290e-02, -1.7148e+00, -1.5682e-02, -1.3152e+00,
-1.0467e+00, 1.5903e+00, 6.2825e-01, 9.9813e-01, -1.3689e+00,
4.5732e-01, -6.2837e-01, 1.0551e+00, 6.7639e-01, 1.2566e-01,
-1.1856e-02, -7.7169e-01, -1.9395e+00, 5.4226e-01, 1.5230e+00,
3.7949e-01, -5.4299e-01, 3.9010e-01, 9.7130e-02, -1.2557e+00,
-4.7758e-01, 2.2550e-01, 1.2194e+00, 7.4347e-01, -5.6995e-01,
6.9029e-01, 4.0881e-02, 7.5411e-01, -9.8277e-01, -8.7063e-01,
3.2578e-01, -7.3621e-01, -1.0762e+00, 9.4652e-01, 8.2750e-01,
-1.0640e+00, 7.8649e-01, 1.5135e+00, -9.6734e-01, 9.6932e-03,
-1.3051e+00, -3.2069e-01, -1.0847e+00, 7.4306e-01, 3.5581e-01,
-2.4428e-01, 4.5607e-01, 7.4739e-01, -6.4819e-01, -1.5006e-01,
-6.1586e-01, 2.9964e+00, 1.3413e+00, 2.8650e-01, -1.8197e-03,
-6.7891e-01, -9.4347e-02, -9.0407e-01, 4.4775e-01, -6.3878e-01,
-1.0743e+00, 1.3872e+00, 3.3750e-01, -8.2506e-01, -2.2975e-01,
-1.5826e+00, -1.9349e+00, -4.6668e-01, 6.8606e-01, -1.8853e-01,
-8.4801e-01, -1.4381e+00, 7.6804e-01]])
1:
词向量: tensor([[ 1.4771, 0.8061, -0.4299, -0.5632, -1.3281, -0.5809, -0.4281, 0.5693,
0.6510, -0.1457, -0.2895, -1.2239, -0.8608, -1.1156, 1.4625, 0.9993,
2.6551, -0.2841, -0.1222, -0.1759, 0.2747, 0.8773, -1.2195, 0.0723,
-1.1253, -0.3896, -0.5094, -0.3521, 0.9901, 1.1108, 1.1191, -1.1345,
-0.5516, -1.0788, 0.2044, 0.1993, 0.2574, -0.3992, 0.0040, 0.1144,
-0.7567, 1.0407, -0.4901, -0.9804, 0.4055, 0.4972, -0.4097, -0.2506,
-0.0374, -0.2787, 0.5761, -0.0540, -0.8492, -0.7583, 1.0121, -0.6011,
-1.4103, 0.6407, 0.3240, 0.5556, -1.6938, 0.2747, 0.7353, -2.0933,
-1.7379, -0.5252, 1.3509, -0.2661, -0.3593, -0.2447, -1.2453, 0.2860,
-1.1808, 1.6014, -1.1948, 1.3895, 0.4737, -1.4571, 1.1382, -0.8261,
0.5641, -2.5230, 0.2975, 1.0076, -2.1213, 0.0967, 0.6917, 1.7999,
0.4339, 1.3127, 1.9793, 1.2312, -0.2442, 2.5434, 0.1964, 0.1350,
-1.6063, 0.6460, 0.2100, 0.4153, -0.0708, -0.2820, -0.3491, 0.3494,
-1.0825, 1.4995, 1.1523, 0.9549, 0.7601, 0.5949, 1.0576, 0.5871,
-1.3045, -0.5725, 0.5301, 0.5057, 0.8488, 0.5929, 1.0189, 0.0787,
0.9573, -0.7332, 0.9140, -0.4795, 0.2905, 0.9838, 0.3004, -0.2756]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
25: 载著 词向量: tensor([[ 0.0130, -1.4442, -0.1159, 1.7674, 1.1045, -0.0127, -2.1541, 0.8694,
-1.5281, -0.3458, -0.3053, -0.3947, 1.3160, -1.9775, 1.9472, 0.0337,
-0.1300, 1.6106, 0.3540, 0.1020, 0.1342, 0.3241, 1.0377, 0.0238,
-1.0631, 1.0980, 1.3228, 0.1006, -0.3086, 0.0924, -1.0263, -0.3253,
-0.6838, 0.6012, -0.9547, 0.1487, -0.7481, -1.2261, -1.8033, -0.7746,
-0.9505, 0.0416, 0.4279, -1.8811, -0.6799, -0.6689, 0.4292, 0.2898,
-1.6674, -0.4637, 0.3331, 2.2861, -1.2305, -0.5207, -0.4132, -1.6243,
0.9616, 0.1419, 1.2081, -1.0937, -0.0821, -0.5604, 0.5080, -1.0553,
2.7479, -0.7037, 0.1400, -1.8362, -1.9336, -0.2423, 0.7988, -1.2030,
-0.6264, -0.1811, 1.9042, -1.3571, 1.4924, 0.1017, 1.9865, 0.3771,
-0.8864, -0.5777, -1.1338, -0.3651, 0.5492, -2.2099, 0.4397, 1.7691,
-0.3087, -0.2277, -1.1842, -1.0426, 0.4771, -0.3086, -0.9765, 0.6117,
-0.1084, -0.1830, -0.2647, 0.1288, -0.9891, 0.4511, 1.7160, -0.2620,
-1.1222, 0.3238, -2.1957, 1.3118, -0.5672, 0.5805, 0.1584, -1.9377,
-0.8131, -2.2030, 0.6335, 0.2707, -1.4031, 0.2158, -1.5485, 1.2137,
1.1471, 0.2171, 0.6865, -0.3507, 0.0420, -0.2295, -0.3680, 2.6944]])
2: 你 词向量: tensor([[-0.2350, 0.2923, -0.5940, 0.4420, -0.5620, 0.6781, 0.6996, 1.1685,
-1.1202, 1.1702, 1.7431, 1.1335, -0.2978, -0.8452, -0.1611, -0.7057,
-1.2757, -0.4136, -0.8584, 0.7088, 1.0133, 0.6883, 0.7639, 0.3339,
-0.0689, -0.8740, 0.1080, -0.2372, -0.0766, -1.1628, 1.2577, -1.4728,
-0.8391, 0.1434, -0.7532, 0.1123, -0.0730, -0.3895, -0.8914, 1.0943,
0.9030, 0.6517, 1.9855, -0.5613, -0.9821, -0.0886, 0.6993, 1.1337,
0.0435, -1.1503, 1.6857, -0.9903, -0.1018, -2.5759, 0.3399, 0.7758,
-1.5326, -0.9689, -0.0901, 0.1434, 2.1109, 1.5606, -1.2908, -0.8750,
-0.1149, 0.3210, 1.2538, 0.2164, -1.1185, -1.3347, -0.1046, -1.3265,
-0.0519, 1.7280, -0.8752, -0.2027, 0.0870, 0.3137, 2.1226, 0.7773,
-0.4965, 1.2673, 0.9419, 0.0753, 1.3247, -0.2009, -0.1344, 1.8694,
-0.8257, -1.2295, -0.1927, 0.9116, -0.9511, 2.7437, -0.3050, 0.6188,
-0.6865, 0.1427, 0.6941, -0.4649, -1.3865, 0.1717, 0.4214, -1.9276,
0.1772, 0.2489, -0.8077, -0.8780, -0.4985, 0.4747, -2.3380, 1.7014,
-2.1964, 0.3989, 0.0545, 0.6022, -0.4869, -1.9906, -1.0134, 0.2028,
-0.0669, 0.2856, -0.4412, 0.8217, 0.0568, 1.1643, -1.0116, 1.9836]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
26: 彷 词向量: tensor([[ 2.0638e-01, 1.7918e-01, -1.6845e+00, -1.6166e+00, -9.5709e-01,
-9.1953e-01, 8.3249e-01, 6.5709e-01, -4.5395e-01, 4.9919e-01,
3.3684e+00, -9.6222e-01, -2.5708e-01, 9.1349e-01, 8.9903e-01,
-9.6211e-01, -3.0636e-01, -1.0191e+00, -7.3408e-01, 1.1168e+00,
8.6342e-01, 1.9096e+00, -7.0161e-01, 8.4195e-01, -2.8671e-02,
3.1279e-01, -1.7202e+00, 9.0489e-01, 2.1504e-01, 4.9763e-01,
-1.6546e-02, -2.4402e-01, 1.1222e+00, -3.2448e-02, -9.7688e-02,
8.5689e-01, -3.4795e-02, 1.2967e-01, 3.7968e-01, 2.2878e+00,
1.7545e-01, 3.3064e-01, -1.2222e-01, -1.2034e-01, 1.3826e-01,
1.2994e+00, 8.4120e-01, -2.3508e+00, -1.4964e-03, 1.4723e+00,
-6.2672e-01, 7.6151e-01, -1.0613e+00, -1.5221e-01, 2.6675e-01,
5.1785e-01, 2.1635e-01, 2.2183e+00, -1.6828e+00, 5.3576e-01,
3.2945e-01, 1.3455e+00, -2.8959e-01, -1.6541e-01, -7.2238e-01,
1.5208e+00, 3.2218e+00, -4.7667e-01, -2.0124e+00, 7.7005e-01,
-3.1791e-01, -2.2316e-01, 5.5996e-02, 1.4482e+00, 5.8317e-01,
6.7232e-02, -1.0836e+00, 1.9237e+00, 2.2125e-01, -2.2501e-02,
1.2048e-03, 3.8885e-01, -1.9154e+00, -1.4272e+00, 1.9411e-02,
-5.8705e-01, 1.3982e+00, 1.3480e-01, -6.3466e-01, -2.4412e-01,
-2.3098e-01, -1.4719e+00, -7.8848e-01, 5.1966e-01, -5.0305e-01,
1.3106e+00, -7.7203e-01, -1.4542e+00, -7.8097e-01, 7.1610e-01,
6.2247e-01, -1.4318e+00, -1.0222e+00, 6.9049e-01, 5.4608e-01,
-4.3482e-01, 3.2407e-01, -1.3702e-03, 1.5834e+00, 3.8256e-01,
-6.8818e-01, 1.2793e+00, -1.2762e-01, -2.0133e+00, -2.1265e+00,
-7.7334e-01, 7.4891e-01, -1.7430e+00, -9.0149e-01, -1.2098e+00,
-1.0013e+00, 5.9874e-02, 1.3382e+00, 1.8211e+00, -1.0879e+00,
2.5116e-01, -3.5087e-01, 1.3276e+00]])
27: 彿 词向量: tensor([[ 0.3545, 0.9200, 0.2450, 0.1973, -0.4150, 0.0871, 0.6044, -0.6435,
-1.8667, -0.7829, 0.0790, 0.9342, 1.7092, -1.4955, 0.4697, 1.3505,
-1.2273, 0.4795, 1.5599, 1.2343, -0.9477, -2.0274, -0.0503, -0.0280,
0.4261, -0.9967, -0.9661, 0.5095, 0.5977, -0.9845, -0.9109, 1.7268,
-0.6637, -0.7102, -0.1051, 0.1794, -0.6248, -0.0737, 2.2374, -1.4089,
1.3406, -0.4111, -1.2105, -1.5885, -1.7177, -0.0593, -0.5647, -0.2260,
-0.9655, 0.4805, 0.0861, -1.0524, -0.7211, -0.2645, 0.9490, 0.3667,
-0.3931, -0.6123, -0.5245, -2.5402, -0.2376, 0.2304, -1.7667, -0.0738,
-0.5167, 0.0438, 0.1273, 0.3209, 0.1512, -0.3844, 1.1880, 0.2504,
0.2282, 0.7113, 0.7361, 0.7894, 0.1982, 0.4231, 1.2974, -0.3185,
-0.0531, -0.5355, 1.0994, -0.6103, -1.3390, 1.1379, -0.3036, -0.2812,
-1.4810, -0.2329, -2.0360, -0.5474, 0.1371, 1.6883, 0.6546, 1.4727,
-0.8790, 0.8791, -0.6492, -0.4888, -0.0276, -0.8042, -1.2861, -3.0403,
-0.3311, -0.5223, -0.8616, -0.9277, -0.1015, -0.7776, 1.2963, 0.3413,
0.3724, -0.0179, 0.2722, 0.8740, 0.1257, 2.4065, 2.2139, -0.9071,
-0.6479, 0.1401, 1.6464, -0.1034, -0.2409, -0.1685, 1.0669, 0.2147]])
28: 载 词向量: tensor([[ 2.2927, -0.1556, 0.1948, 0.1058, -0.9095, 1.5347, 0.1037, -0.9865,
1.5968, 0.7869, 1.7521, -0.7507, -2.4286, -0.0770, -0.4187, 1.0340,
0.4136, 0.0871, -1.1497, 0.1112, -0.9691, 0.0651, 0.4673, 0.5560,
1.7580, 0.7309, 1.0291, -0.7700, 0.3698, 0.0497, 1.3715, -1.2193,
1.2144, 1.2866, -1.2489, -0.0215, 0.6812, -0.4840, -1.2453, 0.1715,
-0.5785, -1.0193, 0.0938, 1.1645, -1.1046, 0.4792, 1.1986, 0.0237,
2.5452, 0.7969, -0.3165, 0.7955, -0.4315, 0.8652, -0.7987, 0.4060,
-0.6243, -0.5340, -1.5854, -0.3623, -1.0397, -0.4544, 0.4102, -0.6597,
0.5227, -1.1922, 1.3843, 0.1093, -0.0911, -0.0289, -0.1817, 1.6703,
0.2879, -1.5454, 1.8103, 1.1089, 0.4454, 1.2236, 0.1834, 2.2438,
0.4571, 0.5952, 0.0425, -0.6352, 1.0600, 0.5906, -0.4696, -1.0117,
-0.7786, -0.0038, 0.3714, -0.2240, 0.0452, -0.9149, -1.5518, 0.4848,
-0.2984, 1.4754, -0.5693, -0.3460, -0.1203, -0.9630, 0.7055, 1.8842,
0.2574, -0.1630, 0.1578, 0.4158, -0.2017, 1.4251, 0.2964, 1.6244,
-1.5865, 1.5117, -1.0052, -1.3300, 1.3928, 1.3397, 0.8070, 0.5173,
0.1104, -0.6624, -1.2985, -0.9819, 0.8713, -0.7251, -0.6626, 1.2367]])
29: 著 词向量: tensor([[-0.4609, -0.2471, 0.4085, -0.5945, 0.4617, 2.6537, -1.6894, -1.4264,
-0.0373, 0.4902, 0.5070, -2.7035, 0.4626, -0.0360, 1.8401, 1.0522,
-1.3496, -0.0990, 1.4639, -0.4736, 0.9515, 0.9412, -0.3063, 0.7427,
0.3580, 0.3058, 0.3931, -0.3376, 0.8213, 0.2612, -1.6250, 1.7013,
1.1387, 1.6242, -0.0913, -0.0833, 0.6456, 0.9076, 0.1719, 0.0250,
1.3384, 1.7722, -1.1699, 0.3260, -1.9999, -0.0369, -0.3020, -0.4866,
-0.0539, 0.4095, 0.6190, -0.7274, -0.5045, 0.6066, 0.8903, -0.1151,
0.6718, 0.1379, -1.1420, 1.2265, -1.9408, 0.0518, -0.0717, -0.1400,
0.5750, 1.8808, 0.1026, 1.2920, -0.2349, 0.4570, 3.0292, -0.6879,
1.0702, 0.1245, 0.8513, -0.0917, 0.0102, -0.4408, -1.2079, -0.0123,
-0.2064, 0.8972, -0.4601, -0.1962, -1.0646, -0.6595, 1.0718, -0.4501,
-0.1222, 2.4056, -0.6845, -0.0965, 2.5040, -0.8885, -0.4753, 2.0764,
-1.7804, -0.7555, -0.3801, 1.0043, 0.5187, -0.8202, -0.2324, 0.4584,
1.2358, -0.2425, 0.4599, 1.2283, 0.6889, 0.4875, -0.9370, -1.0898,
1.5884, 1.4127, 2.1541, 0.3219, 2.0027, 0.4316, -0.4091, -0.2188,
-1.5602, -0.5742, 0.1215, 0.7322, -1.3016, 0.3541, 0.1630, 0.8797]])
30: 阳光 词向量: tensor([[-6.6681e-01, 3.1388e-01, 1.9891e-01, -1.9579e+00, -1.3236e+00,
8.0021e-01, -6.9249e-01, -5.1662e-01, -2.2410e-01, 7.0207e-01,
1.6730e+00, -1.9667e-01, -2.0531e+00, -1.0278e-02, -1.4487e+00,
1.2462e+00, 1.7303e-01, 1.2454e+00, 1.6392e-01, 1.1565e+00,
-9.8996e-01, -1.2486e-01, 1.0502e+00, -1.9623e-01, -1.5182e+00,
2.9193e-01, 8.4160e-01, 2.0578e-01, -9.4992e-01, 2.1812e+00,
-8.3680e-02, -2.4540e-02, 4.8220e-01, -1.3317e+00, -5.8058e-01,
-1.1528e+00, -1.4336e+00, -8.0645e-01, 8.2314e-01, -4.6465e-01,
2.6950e-01, 2.5201e+00, 4.7925e-01, 1.8191e-01, 5.4016e-01,
-1.0156e+00, -9.0166e-01, -8.2198e-01, -2.0085e+00, 2.7966e-01,
1.7198e+00, 1.6327e-01, -5.4052e-01, 7.8048e-01, -5.6496e-02,
-1.7522e+00, -1.4818e+00, -3.5830e-02, 1.1451e+00, 1.1324e+00,
4.3634e-01, 3.9944e-01, -1.0802e-01, -1.4442e+00, -7.1127e-01,
-1.7754e+00, 2.4793e-01, 2.2581e+00, 9.2790e-01, -1.4974e-01,
3.1582e-01, 1.0705e+00, 1.2466e-01, -2.7784e-01, -1.7259e-01,
6.2649e-01, 6.6608e-01, -4.2799e-01, 6.7389e-01, 1.6137e+00,
1.4399e-01, 2.5740e+00, -2.4464e+00, 2.6337e-02, -9.0736e-01,
-3.0761e-01, 9.4740e-01, -8.6411e-01, -5.6676e-01, 4.4355e-01,
-6.0534e-01, -2.0130e+00, -1.1328e+00, -1.8299e-01, -6.1543e-01,
-2.7019e-01, -9.6866e-01, -2.8558e-01, -1.6923e+00, 1.3407e+00,
-4.8356e-02, 1.2291e+00, 1.5349e+00, -9.8115e-01, -9.6204e-01,
1.4603e+00, -3.8333e-01, 1.1623e+00, 1.8353e+00, -1.1845e+00,
-1.5038e-03, -1.3031e+00, 1.8868e+00, -1.2254e+00, 1.7250e+00,
-4.6208e-01, 8.9387e-01, -1.6488e+00, 7.6882e-01, -1.7052e-01,
2.6908e-01, -6.5193e-02, -5.0647e-01, -4.8133e-01, -2.4054e-02,
-3.8189e-01, 1.0427e+00, 1.3326e-01]])
1:
词向量: tensor([[ 1.4771, 0.8061, -0.4299, -0.5632, -1.3281, -0.5809, -0.4281, 0.5693,
0.6510, -0.1457, -0.2895, -1.2239, -0.8608, -1.1156, 1.4625, 0.9993,
2.6551, -0.2841, -0.1222, -0.1759, 0.2747, 0.8773, -1.2195, 0.0723,
-1.1253, -0.3896, -0.5094, -0.3521, 0.9901, 1.1108, 1.1191, -1.1345,
-0.5516, -1.0788, 0.2044, 0.1993, 0.2574, -0.3992, 0.0040, 0.1144,
-0.7567, 1.0407, -0.4901, -0.9804, 0.4055, 0.4972, -0.4097, -0.2506,
-0.0374, -0.2787, 0.5761, -0.0540, -0.8492, -0.7583, 1.0121, -0.6011,
-1.4103, 0.6407, 0.3240, 0.5556, -1.6938, 0.2747, 0.7353, -2.0933,
-1.7379, -0.5252, 1.3509, -0.2661, -0.3593, -0.2447, -1.2453, 0.2860,
-1.1808, 1.6014, -1.1948, 1.3895, 0.4737, -1.4571, 1.1382, -0.8261,
0.5641, -2.5230, 0.2975, 1.0076, -2.1213, 0.0967, 0.6917, 1.7999,
0.4339, 1.3127, 1.9793, 1.2312, -0.2442, 2.5434, 0.1964, 0.1350,
-1.6063, 0.6460, 0.2100, 0.4153, -0.0708, -0.2820, -0.3491, 0.3494,
-1.0825, 1.4995, 1.1523, 0.9549, 0.7601, 0.5949, 1.0576, 0.5871,
-1.3045, -0.5725, 0.5301, 0.5057, 0.8488, 0.5929, 1.0189, 0.0787,
0.9573, -0.7332, 0.9140, -0.4795, 0.2905, 0.9838, 0.3004, -0.2756]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
31: 不管 词向量: tensor([[-0.9464, -0.6052, 0.3126, 1.4290, 2.0005, 0.8584, 1.6002, -0.5967,
-0.3659, -1.6638, -0.1917, -0.1917, -0.6897, -0.5995, 2.3674, 0.6679,
0.5741, -2.0621, 1.3792, 0.3328, -0.9829, 0.2667, -2.0164, -0.3677,
0.6888, -2.6394, 0.0337, -1.4745, 1.8685, -1.0525, -1.1835, -0.0969,
0.5602, 1.3218, -0.2591, 0.5099, 1.5861, 1.2293, -0.3418, 0.9655,
0.1812, 0.3926, 0.6548, 1.1521, 1.0141, 1.0634, -1.6467, 0.8662,
0.2042, 0.8573, 1.9409, -0.6745, -1.7629, -0.5597, 1.3174, -0.0809,
-1.3265, -1.1189, 0.1612, -1.2166, 0.0435, 1.6677, 0.6189, -0.4808,
1.5639, -0.3744, -0.4279, -0.1212, 0.4054, -0.7304, 0.8854, 0.3845,
-1.5092, 0.1070, -0.0445, -0.2511, -1.3624, -0.0352, 1.2683, 1.3496,
0.5806, 0.9147, -0.4457, -0.7512, 1.0327, 0.6721, -0.1766, -1.0790,
0.0667, 0.4618, -0.5581, 0.7611, -2.1172, 0.4174, 0.5315, 0.5692,
-0.8661, 0.5378, 0.4122, 1.0156, -0.1269, 1.0359, 0.1050, 0.1159,
-2.0003, -0.7383, -0.9426, 1.0606, 1.3359, -0.2512, -0.5589, -0.4096,
-0.9388, 1.1260, -2.0243, -2.3810, -2.5837, -1.0050, 0.3578, -0.6826,
1.6660, -1.7785, -1.4657, 1.0506, 0.3798, 0.8354, 1.0325, -0.6190]])
32: 到 词向量: tensor([[-1.7293e+00, 7.6618e-01, 1.2457e+00, -8.3565e-02, -1.6795e+00,
-6.8624e-01, -1.8255e-02, 1.0258e+00, 1.1433e+00, 5.5758e-01,
1.5512e+00, 2.0042e-01, -1.4405e+00, -8.9225e-01, -7.9894e-02,
-8.3573e-01, -8.9205e-01, -1.5358e+00, -2.3136e-01, -1.5133e+00,
-2.0101e-01, 9.2057e-01, -1.5428e+00, 5.5261e-01, -5.8868e-01,
3.9964e-01, 9.4172e-01, 1.3467e+00, 1.6078e+00, 8.8467e-01,
1.2418e-01, -4.2634e-01, -6.0308e-01, -1.8785e-01, 5.9931e-01,
5.3682e-01, -1.6145e-01, 6.0872e-01, 1.2291e+00, 1.7928e+00,
1.2258e+00, -5.0628e-01, 7.3103e-01, -7.6231e-01, -4.9622e-01,
-9.2898e-01, 2.6941e+00, 2.9617e-01, -8.6824e-01, -1.6541e+00,
-6.4260e-01, 1.5887e+00, 1.9627e-02, 1.8426e+00, 6.6561e-01,
-1.2808e+00, 1.0860e+00, 1.1127e+00, 3.3658e-01, 8.2967e-01,
7.8014e-02, -1.4274e+00, 1.1754e+00, 4.4043e-01, -9.9647e-02,
7.8550e-01, 1.5082e+00, 1.7038e+00, 6.4063e-01, 1.9863e+00,
1.9860e+00, 1.3358e+00, 2.1502e-01, -1.0011e-01, 1.7259e+00,
-1.4419e+00, -8.8087e-01, 1.6717e+00, 1.7081e+00, -8.1647e-01,
-2.1245e+00, -6.6736e-01, -1.3917e+00, 8.8490e-01, -4.5910e-01,
1.0833e+00, 9.0440e-02, -1.6128e-01, -2.4166e+00, -1.9114e+00,
-4.9039e-01, -1.0321e+00, -1.9241e+00, -4.1589e-01, -4.9464e-01,
3.9884e-02, 1.4264e+00, -1.0090e+00, 7.2417e-01, -6.8247e-01,
-5.4365e-01, 3.1351e-01, -3.3199e-01, -1.1638e+00, -6.8961e-01,
9.4218e-01, -5.3865e-01, 1.3979e+00, -4.9486e-01, -6.6712e-02,
-5.6468e-02, -4.6435e-01, -3.9127e-01, -2.1858e+00, -2.6374e-03,
-4.6120e-02, -2.3910e-01, -6.2278e-01, -4.8175e-02, -1.1165e-01,
8.4639e-01, 3.3255e-01, 1.6826e-01, 8.6989e-01, -1.3513e+00,
7.0078e-01, 1.1347e+00, -1.7916e+00]])
33: 哪里 词向量: tensor([[-1.4643, -0.6733, 0.2959, -0.1952, -0.1018, 0.0386, -0.1922, -0.5421,
2.5666, -0.2306, 0.4469, -0.1740, -1.6989, 0.8954, 0.4466, -0.5638,
0.5447, -0.6224, 0.9208, 0.5154, -0.5738, -0.7667, -1.8900, 1.0274,
-0.1189, 0.2136, 0.3690, 0.5807, 0.9170, 0.1215, -0.0820, -0.1953,
-0.1378, 0.1823, -0.4439, 0.5595, -0.1301, -1.3967, -0.4519, 0.5513,
0.2657, 0.1050, 0.8477, -0.1732, -0.5251, -0.7948, 0.7711, 0.2346,
0.9837, 1.1456, -1.2671, 0.4402, -1.0754, 0.4043, 0.4300, -1.1614,
-0.8005, 1.4687, -0.6400, -0.0339, 1.2569, -0.6062, 1.4264, 0.7395,
0.1630, -1.1712, 0.5811, -0.3621, 0.6328, -0.4657, 0.4631, 0.1303,
0.2278, -1.0573, 0.4471, -1.5146, -0.9939, -0.5790, -1.9654, 1.8992,
-1.1681, 0.5045, 1.2638, -1.4513, 0.4989, -2.2016, 0.7756, -0.9715,
1.2353, 0.0658, 0.1500, -0.2204, -1.1382, -1.0660, -0.6612, -2.2103,
0.3402, 1.0039, 0.0782, -1.2468, -0.8603, -1.1246, -0.7365, 0.3039,
1.0979, 1.0798, -0.5041, -0.3765, -0.5015, 1.4325, 0.3214, 0.5275,
0.8239, -0.3697, -1.5350, 0.0238, 0.0119, 0.4711, 0.2432, 1.0015,
0.0086, 0.3860, 0.5784, -0.8344, 1.1504, 0.4801, 0.2475, 1.3448]])
34: 都 词向量: tensor([[-1.1040, -0.2448, -1.7101, -0.6372, 0.8819, 0.3487, -1.1393, 0.2709,
-0.7410, -0.1067, 1.0191, -0.6669, 1.1108, -1.0200, 1.1394, 0.2519,
1.3242, -0.8859, -0.1807, -0.0494, -1.2886, 1.2777, 1.0099, 0.2565,
0.5103, -0.8095, 1.5098, 0.1844, -0.8281, 1.0727, 0.5986, -0.2436,
-1.0959, -0.2728, -0.6333, 0.5404, 1.3417, 0.4069, 0.3982, 1.4103,
-1.0120, -0.7256, -0.6730, 0.0247, -0.1195, -0.3666, 0.9982, -1.4035,
0.6242, -1.8441, 0.5665, -0.1932, 1.0155, -0.0217, -0.9399, 1.6921,
0.7970, -1.1289, 0.9723, 0.5233, 0.8992, 0.5504, -0.0088, -1.7802,
0.7315, -1.1603, 0.2731, 1.0375, 0.8651, -0.0506, 0.2135, -0.0511,
-1.2038, -0.4909, -0.4693, 0.4387, 0.7884, 0.1814, 0.4348, 0.7209,
0.6599, -1.2706, -0.3124, -0.5817, 0.5481, -0.5841, -1.7155, -0.8999,
0.6258, -0.3484, -0.7053, 0.4315, -1.6577, -0.0441, 0.0373, 1.2164,
0.7550, -1.0862, -0.2914, 0.2415, 1.0807, 1.7424, -0.8073, -0.0807,
-0.6910, 0.2095, 1.0607, -1.3497, 0.8184, -0.7912, 0.5244, -1.0989,
0.0108, 0.5212, 1.2531, -2.2000, 0.0591, -0.7317, -2.4573, -1.2097,
-0.6044, -0.1179, -0.7387, 0.8334, -0.3104, 1.5265, 0.3675, 2.6542]])
35: 是 词向量: tensor([[-8.2120e-01, -2.8328e-01, 1.6662e+00, -1.6087e+00, 8.6736e-01,
-6.4053e-01, 4.2528e-01, -6.1368e-01, 6.3542e-01, -1.1060e+00,
-2.0113e+00, 2.9655e-01, 1.0306e+00, 4.6830e-01, 8.3836e-01,
-1.6699e-03, 4.6096e-01, 2.2086e+00, -9.2179e-01, 1.9870e+00,
3.4076e-01, -2.8053e-01, -9.0661e-01, 2.4770e-01, -1.1387e+00,
2.7342e-01, 4.2476e-01, -1.0907e+00, -1.4633e+00, 1.0174e-01,
-8.6484e-01, -1.4023e+00, -2.2307e+00, -1.8555e+00, -3.4721e-01,
-1.7547e+00, -8.0795e-01, -1.3987e+00, -3.8316e-01, 2.1896e-01,
-2.9757e+00, 9.9645e-01, -4.1250e-01, -1.1160e-01, -1.4714e+00,
-9.8250e-01, 4.3480e-01, 2.6648e-01, -1.5392e+00, -1.6790e+00,
-1.8937e+00, 3.8134e-01, -1.4923e+00, -1.5945e-02, 1.8075e+00,
1.2530e+00, 1.0465e+00, 4.2364e-01, -2.8934e-02, -1.7606e+00,
-1.7925e+00, -1.6269e+00, 1.8391e-01, 1.5703e+00, -1.7618e-02,
-5.1381e-01, -4.7056e-01, 6.7110e-01, 1.5954e+00, 5.7706e-01,
-2.5244e-01, 1.6121e+00, -1.6114e+00, -3.3068e-01, 6.9184e-02,
-1.7144e-01, -1.0965e-01, 1.2530e+00, 8.9262e-02, 5.7882e-01,
-1.1490e+00, -3.2062e-01, 3.2360e-01, 4.6964e-02, 7.8878e-03,
1.5564e-01, 1.5038e+00, -6.0567e-01, -3.7252e-01, -1.1011e+00,
6.1680e-01, -2.0176e-01, -3.2489e-01, 3.1328e-01, 6.0072e-01,
3.2597e-01, 6.9676e-01, -6.5529e-01, 6.6205e-02, -4.3353e-01,
-3.2025e-03, -5.4171e-01, 1.2523e+00, 3.3848e-01, -7.5469e-01,
-1.8495e+00, 1.3784e+00, -1.1285e+00, 1.1601e+00, 9.1605e-02,
-3.3379e-01, -1.2874e+00, 1.1333e+00, -6.9849e-01, 3.0190e-01,
-6.3544e-01, 4.8548e-01, 1.9634e-01, -2.7538e-01, 3.2318e-01,
1.2272e+00, 1.7607e+00, 6.6594e-01, 3.8842e-01, 6.7394e-01,
4.0367e-01, -1.8401e-01, 3.1367e-01]])
36: 晴天 词向量: tensor([[-0.9515, -2.0035, 0.9765, -0.2883, -0.8134, 0.0556, -0.2098, -0.1265,
1.5195, 0.0802, -1.1466, -0.4319, -0.7498, -0.5207, -0.4725, 0.6365,
-0.0726, -0.7961, 0.2653, 0.0717, 0.9894, 0.3911, 0.7375, 0.8026,
0.7547, 0.2801, 0.4399, -0.3265, -0.9765, -1.2219, 1.6562, -0.3701,
-1.9560, -1.1429, -0.1170, 2.2568, 0.7193, -0.6739, 0.2834, 0.5517,
-0.4053, -0.3769, 0.4368, 2.0275, 0.5619, -0.1478, 0.4074, 1.3673,
-0.2386, 0.7686, -2.0170, 0.1169, 1.2757, -1.6200, -2.4805, -0.5298,
-0.5516, 0.3519, 0.8227, -0.1155, -0.4592, -0.2786, -0.7938, 0.1115,
2.1405, -0.5330, 1.7106, 0.4140, -0.1481, 1.0451, 2.1707, -1.6154,
-0.6824, -0.7025, 1.4571, 0.5462, 0.2533, -1.9664, 0.4303, -0.2169,
-0.9560, -0.3117, 0.3301, -0.1070, -0.9566, 0.3362, 1.8394, 1.0272,
0.0161, 0.7523, 0.0521, 1.5430, 0.6700, 0.4177, 1.4022, -0.3000,
0.5417, 1.8098, -0.0651, 0.7423, 0.3519, 0.3802, -1.0462, 1.3886,
0.4424, 1.4787, -0.7734, 1.4286, 0.4675, 0.1979, 0.9512, 0.0780,
-1.0070, -1.0718, 0.2628, 0.1733, -0.0740, -1.6008, -2.0330, 0.9557,
0.7744, -2.1375, 1.4451, -0.4186, 0.7063, -0.3801, 0.1497, 2.5435]])
1:
词向量: tensor([[ 1.4771, 0.8061, -0.4299, -0.5632, -1.3281, -0.5809, -0.4281, 0.5693,
0.6510, -0.1457, -0.2895, -1.2239, -0.8608, -1.1156, 1.4625, 0.9993,
2.6551, -0.2841, -0.1222, -0.1759, 0.2747, 0.8773, -1.2195, 0.0723,
-1.1253, -0.3896, -0.5094, -0.3521, 0.9901, 1.1108, 1.1191, -1.1345,
-0.5516, -1.0788, 0.2044, 0.1993, 0.2574, -0.3992, 0.0040, 0.1144,
-0.7567, 1.0407, -0.4901, -0.9804, 0.4055, 0.4972, -0.4097, -0.2506,
-0.0374, -0.2787, 0.5761, -0.0540, -0.8492, -0.7583, 1.0121, -0.6011,
-1.4103, 0.6407, 0.3240, 0.5556, -1.6938, 0.2747, 0.7353, -2.0933,
-1.7379, -0.5252, 1.3509, -0.2661, -0.3593, -0.2447, -1.2453, 0.2860,
-1.1808, 1.6014, -1.1948, 1.3895, 0.4737, -1.4571, 1.1382, -0.8261,
0.5641, -2.5230, 0.2975, 1.0076, -2.1213, 0.0967, 0.6917, 1.7999,
0.4339, 1.3127, 1.9793, 1.2312, -0.2442, 2.5434, 0.1964, 0.1350,
-1.6063, 0.6460, 0.2100, 0.4153, -0.0708, -0.2820, -0.3491, 0.3494,
-1.0825, 1.4995, 1.1523, 0.9549, 0.7601, 0.5949, 1.0576, 0.5871,
-1.3045, -0.5725, 0.5301, 0.5057, 0.8488, 0.5929, 1.0189, 0.0787,
0.9573, -0.7332, 0.9140, -0.4795, 0.2905, 0.9838, 0.3004, -0.2756]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
37: 蝴蝶 词向量: tensor([[-0.0400, -3.0574, -0.3635, 0.0669, -0.5588, -1.2091, -1.4738, 0.2054,
-0.0921, -0.4293, -2.2230, -0.0302, 1.1037, 0.8118, -1.5666, 0.2460,
-1.0714, 1.3454, -1.0974, -1.7518, 0.9955, -0.2803, -0.1690, 0.9053,
-1.7695, 0.1259, 1.1383, 0.1016, 0.6136, 0.5402, -0.4232, 0.7554,
-0.2779, -0.3137, 0.5117, 0.2531, -0.6667, 1.6624, -0.7515, 1.1473,
-0.9900, -0.3361, -1.6921, -0.9011, 0.0125, -0.2491, 0.9530, 0.7645,
-1.0743, 1.3598, -0.6045, 0.0985, -0.4107, 0.5621, 0.3941, 1.8453,
-1.3194, 0.5754, -1.5461, -0.3941, -0.0374, -0.5618, 0.5198, -0.4568,
-1.5868, -1.1016, -2.0161, -0.2921, 1.1768, 1.0603, -0.9682, -0.4863,
0.3831, -2.2404, 0.1762, 0.8192, -2.0637, 1.1402, 0.6617, 1.8851,
1.7079, -0.2097, 0.5858, 0.0674, -0.2334, -0.4106, -0.9226, 1.4199,
-1.3866, -0.3694, -1.6176, 1.1050, 0.8684, 1.0019, -0.4293, -0.0600,
-0.8311, -0.3207, -0.5480, -0.3166, 0.6045, 1.1389, -0.2693, -1.2763,
0.7155, -0.7850, 1.4891, 0.3402, -0.1004, -0.9299, 0.9118, 1.5175,
0.9601, 0.5925, -0.3684, 0.1902, 0.1856, 0.4328, -1.1945, -0.8295,
-0.2797, 0.1495, 0.1475, 1.5402, -0.3102, 1.7637, 0.4542, 0.2214]])
38: 自在 词向量: tensor([[-0.0890, 2.0051, -0.4212, 0.7407, -0.1412, -0.2897, -1.2797, -0.1035,
0.8262, 0.1403, 1.6216, -0.4490, 2.0291, -0.7813, 0.5643, 0.9159,
-0.6230, -2.3437, 0.8992, -1.1990, 0.8511, -0.9259, 1.0098, 0.1918,
-0.1001, -0.2540, 0.2757, -0.3057, 1.0016, -0.7917, 0.6704, 1.8217,
-1.3798, -2.3096, 0.2124, 0.0288, 2.3481, 0.1914, -0.9464, 0.2106,
-0.9816, 1.1996, -0.1687, -0.5714, 2.5692, 0.4879, -1.0856, -0.3265,
0.3012, 0.6901, 2.0890, 0.5059, -0.9193, 1.1435, -0.9719, -0.6070,
-0.2419, -0.6951, 0.8119, 0.3433, -0.7767, -0.6102, -0.0567, -0.1336,
1.7577, -1.5125, 0.7527, 0.2311, -0.8408, -0.2683, -0.3418, -1.5201,
-0.9887, -1.1599, -1.3392, -0.6399, 0.0456, 1.6694, 0.8850, 0.4662,
-0.8431, -0.4383, 1.4793, -0.1043, -0.4125, 1.5138, -1.0306, 0.4325,
-1.4426, 1.3891, 2.2106, 0.0119, 1.3662, -1.0865, 0.2993, 1.6427,
0.9929, -0.4652, 0.3751, 0.1675, 0.5255, -1.7891, -0.5643, 2.0388,
-0.4236, 2.2841, 0.4414, -0.8529, -0.2652, 2.0532, 0.9428, -2.3291,
0.7312, -0.2203, -2.6520, 0.7018, -0.6359, -1.1981, -1.3476, 0.5686,
0.6346, -1.3652, 0.0989, 0.3841, 0.7982, 0.3921, 0.2562, 1.1164]])
39: 飞 词向量: tensor([[ 0.1505, -0.5211, -1.3555, -1.6991, 0.0248, 1.0908, -2.4161, -0.4290,
-0.9339, 0.9067, -0.9893, 0.2206, -0.3759, 0.3800, -1.3682, -0.1715,
-1.0890, -0.8062, -0.7725, -1.5024, 1.2533, -1.1394, -2.2211, 0.7388,
0.2162, 0.3735, 1.1901, 1.4049, 1.4705, 0.3772, 0.0564, 0.7979,
0.3808, -1.5826, 0.8833, -0.0071, -1.5239, -0.0058, -0.1841, -0.6579,
-0.2287, -0.9190, 0.2217, 0.6823, 0.5909, 0.6026, -0.7251, 0.3107,
0.3377, -0.7829, 0.6419, -1.3618, -1.1473, -2.2249, 0.1155, -0.0491,
-0.2913, 1.2379, 1.0540, -1.1968, 0.4840, -0.2500, 0.4143, 0.0701,
-1.6958, -1.0107, -1.0176, -0.0821, -0.4596, 0.6628, 0.5114, 0.8228,
-0.0263, -0.3108, -0.0664, 0.7817, -0.0792, -0.6642, 0.4899, -0.7746,
-1.2100, 0.0668, 0.5627, 1.9854, 1.2941, -0.4958, 1.6721, 0.5782,
-0.4971, 0.8496, -1.3295, 0.8021, -0.8112, -0.1706, 0.1133, -1.7094,
1.1023, 0.8259, 1.9681, -0.7125, -1.6230, -0.0514, -0.1862, 0.0737,
0.5846, 0.7409, 1.2824, -1.1207, 0.0472, -1.7442, -0.3456, -0.5543,
0.1329, 0.5167, 0.4778, -0.6580, 0.2221, -1.7735, -0.2137, 0.5213,
0.3882, -0.2364, 0.6884, 0.3610, -0.2606, 0.7823, -1.1167, -0.2934]])
1:
词向量: tensor([[ 1.4771, 0.8061, -0.4299, -0.5632, -1.3281, -0.5809, -0.4281, 0.5693,
0.6510, -0.1457, -0.2895, -1.2239, -0.8608, -1.1156, 1.4625, 0.9993,
2.6551, -0.2841, -0.1222, -0.1759, 0.2747, 0.8773, -1.2195, 0.0723,
-1.1253, -0.3896, -0.5094, -0.3521, 0.9901, 1.1108, 1.1191, -1.1345,
-0.5516, -1.0788, 0.2044, 0.1993, 0.2574, -0.3992, 0.0040, 0.1144,
-0.7567, 1.0407, -0.4901, -0.9804, 0.4055, 0.4972, -0.4097, -0.2506,
-0.0374, -0.2787, 0.5761, -0.0540, -0.8492, -0.7583, 1.0121, -0.6011,
-1.4103, 0.6407, 0.3240, 0.5556, -1.6938, 0.2747, 0.7353, -2.0933,
-1.7379, -0.5252, 1.3509, -0.2661, -0.3593, -0.2447, -1.2453, 0.2860,
-1.1808, 1.6014, -1.1948, 1.3895, 0.4737, -1.4571, 1.1382, -0.8261,
0.5641, -2.5230, 0.2975, 1.0076, -2.1213, 0.0967, 0.6917, 1.7999,
0.4339, 1.3127, 1.9793, 1.2312, -0.2442, 2.5434, 0.1964, 0.1350,
-1.6063, 0.6460, 0.2100, 0.4153, -0.0708, -0.2820, -0.3491, 0.3494,
-1.0825, 1.4995, 1.1523, 0.9549, 0.7601, 0.5949, 1.0576, 0.5871,
-1.3045, -0.5725, 0.5301, 0.5057, 0.8488, 0.5929, 1.0189, 0.0787,
0.9573, -0.7332, 0.9140, -0.4795, 0.2905, 0.9838, 0.3004, -0.2756]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
40: 花 词向量: tensor([[ 1.6223e+00, -9.5817e-01, -1.4592e+00, 1.5443e+00, 2.2209e-01,
-1.9822e+00, -1.8519e+00, 1.1029e-01, -1.6615e-01, 8.0731e-02,
3.7431e-01, 7.9663e-01, 1.2330e+00, -4.7161e-01, -1.6916e+00,
-4.4269e-01, 7.8954e-02, 1.1167e+00, -3.8532e-01, -1.1191e+00,
4.6523e-01, 1.4931e+00, 2.8343e-01, 7.2036e-01, 6.2302e-01,
1.1892e-01, -1.4169e+00, -1.3503e+00, 9.5121e-01, -1.8075e+00,
1.3349e+00, -3.1257e-01, 5.2157e-01, 9.8751e-02, 9.1501e-01,
6.4720e-01, -6.1483e-01, -1.1998e-01, 4.8999e-01, -1.0645e+00,
-2.1452e-01, -1.3770e+00, 1.8499e+00, -1.2342e+00, 9.5056e-01,
2.1517e+00, -3.6696e-01, -6.7400e-01, -2.1884e-01, 6.2584e-01,
4.9954e-01, -4.5436e-01, -9.6446e-01, 1.5823e-01, 7.8843e-01,
8.5788e-01, -1.0194e+00, -5.0947e-03, -1.1680e+00, -7.5109e-01,
-7.4497e-01, 9.8633e-01, 1.6304e+00, -3.2880e-01, 8.0259e-01,
-5.8733e-01, -1.2889e+00, 2.8109e-01, -8.9083e-01, -2.0554e+00,
-1.2636e+00, 6.7099e-02, 1.0834e+00, 1.1612e+00, 9.3985e-01,
-2.0706e+00, 1.8427e-01, -4.6557e-01, 9.6158e-01, -1.0032e+00,
1.7373e+00, 7.1857e-01, -1.3122e+00, -1.6312e+00, 7.2751e-01,
-6.9314e-01, -6.9697e-02, -8.0559e-01, 2.6991e-01, 8.4679e-01,
2.1277e-01, 1.3291e-01, 4.0936e-01, -1.1848e+00, -6.7349e-01,
7.4269e-01, -6.2325e-01, -2.4662e-01, -5.1447e-01, 3.5489e-01,
-9.5475e-01, -1.0932e+00, -1.2717e-01, -1.3606e+00, -7.7625e-01,
1.0224e+00, -2.9855e+00, -4.8703e-01, 5.2271e-01, 1.9643e+00,
1.3821e+00, -3.1085e-01, 1.0357e+00, -1.4043e-01, -2.9865e-02,
-2.0081e+00, -5.4855e-01, 2.8518e-01, 1.2019e-01, 1.4236e-03,
2.7062e-01, 1.1406e+00, -2.8399e-01, 5.6629e-02, -5.7652e-01,
6.6034e-01, -2.9776e-01, 8.1386e-01]])
41: 也 词向量: tensor([[-6.7628e-01, -1.2231e+00, -8.5826e-01, -1.5907e+00, 9.6659e-02,
-1.0435e+00, 7.8038e-01, 6.3931e-01, -2.2397e-01, 1.0919e+00,
-3.6329e-01, 1.7266e+00, 8.5129e-02, 1.7637e+00, 1.0250e+00,
-4.6704e-01, 1.3918e+00, -1.1509e+00, 6.4847e-01, -7.5462e-01,
3.0993e-01, 1.3801e+00, 1.2410e+00, -6.6719e-01, -9.9027e-02,
1.3417e+00, -1.0160e+00, -1.7841e+00, 4.6521e-01, 1.0478e+00,
-9.1776e-01, -2.4109e+00, -1.0396e-01, -1.4302e+00, 1.3115e+00,
1.4434e+00, 4.4277e-01, -4.1619e-02, 8.4129e-01, 3.5324e-01,
1.6084e+00, 1.1552e+00, 8.7772e-01, 3.2182e-01, -5.6103e-01,
6.8060e-01, -4.5383e-01, 1.0746e+00, 1.4843e-01, 1.3553e-01,
2.4592e+00, -2.1877e-01, 5.8189e-01, 1.3227e+00, 1.0808e+00,
-2.1273e-01, 9.7085e-01, -1.1326e+00, 2.3754e-01, -2.1791e+00,
-1.3799e-01, -7.2653e-01, -2.1827e+00, -4.4514e-02, 5.2606e-01,
6.2988e-01, -5.2788e-02, -7.5509e-01, 9.4722e-01, 2.7789e-01,
-7.2505e-01, -8.6593e-01, 7.2531e-01, 1.0444e+00, -4.5420e-01,
1.0472e+00, -7.1261e-01, -4.4027e-01, -7.8621e-01, -2.6592e+00,
-7.1783e-01, -7.4633e-01, -1.2442e+00, 2.0033e+00, 5.3387e-01,
1.5952e-01, -2.3704e-03, 1.6082e+00, 4.8052e-01, -4.1368e-01,
-1.2486e+00, -3.4285e-02, 1.1333e-02, -6.2764e-01, -4.7374e-01,
-4.3883e-01, 7.5377e-01, -1.4190e+00, 5.0723e-01, 1.4401e+00,
-1.1395e+00, 5.3203e-01, 2.5143e-01, 2.4624e-01, 2.0348e-01,
2.8935e-01, -2.9100e+00, 4.4372e-01, 4.0474e-01, 8.5836e-01,
-1.4738e-01, 4.4940e-01, -1.2206e+00, -1.9185e-01, 1.1428e+00,
1.1633e-01, 1.2609e-01, -9.0215e-01, -5.8231e-01, 2.5477e-01,
6.3429e-01, -4.0369e-01, 9.3054e-01, -2.9329e-01, -3.4951e-01,
8.1288e-02, -2.0352e+00, -1.3720e-01]])
42: 布满 词向量: tensor([[-4.5253e-01, 1.2684e+00, 1.7057e-01, 9.4326e-01, -4.1709e-01,
-4.4314e-02, -5.4650e-01, -1.4884e+00, -1.7525e+00, 1.7944e+00,
-5.1579e-01, 6.5650e-01, -7.9460e-01, 5.9025e-01, 2.1915e+00,
-2.1443e-01, -1.0747e+00, -1.0729e+00, 4.1925e-01, -1.3990e-01,
2.2697e-01, 1.5757e+00, 5.9371e-01, 7.1543e-02, -2.1005e-01,
1.9483e+00, 1.0135e+00, 8.0003e-02, 6.8458e-01, 1.3643e+00,
2.3798e+00, 5.0634e-01, -1.3400e+00, 3.5532e-01, -1.9955e-01,
1.7431e+00, -1.0468e+00, 1.3643e-01, 1.6486e-01, -1.4376e+00,
4.1655e-02, -1.2915e+00, -1.2906e-01, -1.1715e+00, -2.1027e+00,
1.7695e-01, -3.6445e-01, 5.1146e-01, -1.3261e+00, -1.2071e+00,
-1.0221e+00, -2.7931e-01, 2.0997e+00, -1.0898e+00, 3.2626e-01,
-7.9737e-01, -5.6796e-01, -9.3576e-01, 2.8312e+00, -6.4564e-02,
-1.6108e+00, 1.0971e+00, 5.6300e-01, -2.1304e+00, 8.5615e-01,
-1.6809e-01, -1.8408e-01, 2.5422e-02, -4.0539e-01, -1.1370e+00,
-4.2277e-01, 2.2772e-01, 8.1919e-01, -4.0291e-01, 7.0270e-04,
-1.4806e+00, 4.6460e-01, 1.5019e+00, -2.8855e+00, 2.5376e-01,
7.6675e-03, -2.1994e-01, 2.0591e+00, -3.9508e-01, -1.1114e+00,
1.1211e+00, 3.9303e-01, 1.0362e+00, -5.5384e-01, 1.2727e+00,
-1.2753e+00, 7.9594e-01, 3.9921e-01, -2.1216e-01, 4.1584e-01,
1.1322e+00, 4.2213e-01, -8.8632e-01, -1.4457e+00, 1.2618e+00,
1.4342e+00, 7.5508e-02, 7.7594e-01, -1.2284e+00, -1.4598e+00,
-8.1313e-01, -7.7488e-01, 3.7278e-01, 1.4813e+00, -1.8646e-01,
-8.9246e-01, -8.6041e-01, -3.5563e-01, 1.5553e+00, 1.2483e+00,
-1.2193e+00, -1.6430e+00, 5.2919e-01, 9.0076e-01, -6.3405e-01,
2.4560e-04, -6.9363e-01, -2.7712e-01, 2.0122e-01, 9.3922e-01,
-1.2071e+00, -9.4478e-01, -2.4938e+00]])
43: 天 词向量: tensor([[ 0.6787, 0.5562, 1.2362, 1.3772, -0.0162, -1.2678, 0.3929, 0.1114,
0.7642, 0.0986, 0.1139, 1.0605, 1.4362, -2.0456, 0.8168, -1.6170,
1.6338, -0.1835, -1.5192, 0.7876, -2.0830, 0.3325, -0.2090, 0.5700,
-1.4786, -1.2677, 0.1142, -0.2664, 0.7965, -1.6880, 0.0061, 0.0397,
2.3898, -0.2688, -0.9833, -1.4358, -0.0378, -0.1717, -0.1442, -0.8028,
1.0593, -0.3457, 0.2422, 0.2237, -0.7803, 1.0759, 0.8889, 0.0160,
-1.7623, -2.9322, -0.9920, -0.6178, 0.0272, 0.0790, -1.1477, 2.4297,
-0.9211, 1.6385, 0.5648, -1.5323, -1.1366, -0.5470, -0.6924, 1.8343,
-1.2098, 1.1172, 1.7328, -0.4984, -1.4761, -0.0402, -0.6028, 0.7889,
-0.4026, 1.0294, -0.5683, 1.0917, -1.0194, -1.5754, -0.2576, -1.3578,
-0.1577, 1.6544, -0.3081, 1.5000, 1.0419, -0.2156, 1.7456, -0.1576,
1.0433, 0.6680, -0.3189, 0.2969, -0.2939, -1.7309, 2.0627, -0.6846,
-0.8100, -2.1868, 0.5834, 2.0086, -0.5590, -0.5323, 0.2037, -1.5073,
2.2249, 0.7154, 0.0607, -1.2099, -0.4010, 1.0810, 0.9910, 0.4656,
0.3364, -0.9702, 0.3221, -0.3232, -1.8185, 0.0825, 1.0955, 0.1303,
-0.1656, 0.3935, -0.1586, 0.9302, 1.2459, 2.4649, -0.2167, -0.1001]])
1:
词向量: tensor([[ 1.4771, 0.8061, -0.4299, -0.5632, -1.3281, -0.5809, -0.4281, 0.5693,
0.6510, -0.1457, -0.2895, -1.2239, -0.8608, -1.1156, 1.4625, 0.9993,
2.6551, -0.2841, -0.1222, -0.1759, 0.2747, 0.8773, -1.2195, 0.0723,
-1.1253, -0.3896, -0.5094, -0.3521, 0.9901, 1.1108, 1.1191, -1.1345,
-0.5516, -1.0788, 0.2044, 0.1993, 0.2574, -0.3992, 0.0040, 0.1144,
-0.7567, 1.0407, -0.4901, -0.9804, 0.4055, 0.4972, -0.4097, -0.2506,
-0.0374, -0.2787, 0.5761, -0.0540, -0.8492, -0.7583, 1.0121, -0.6011,
-1.4103, 0.6407, 0.3240, 0.5556, -1.6938, 0.2747, 0.7353, -2.0933,
-1.7379, -0.5252, 1.3509, -0.2661, -0.3593, -0.2447, -1.2453, 0.2860,
-1.1808, 1.6014, -1.1948, 1.3895, 0.4737, -1.4571, 1.1382, -0.8261,
0.5641, -2.5230, 0.2975, 1.0076, -2.1213, 0.0967, 0.6917, 1.7999,
0.4339, 1.3127, 1.9793, 1.2312, -0.2442, 2.5434, 0.1964, 0.1350,
-1.6063, 0.6460, 0.2100, 0.4153, -0.0708, -0.2820, -0.3491, 0.3494,
-1.0825, 1.4995, 1.1523, 0.9549, 0.7601, 0.5949, 1.0576, 0.5871,
-1.3045, -0.5725, 0.5301, 0.5057, 0.8488, 0.5929, 1.0189, 0.0787,
0.9573, -0.7332, 0.9140, -0.4795, 0.2905, 0.9838, 0.3004, -0.2756]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
7: 一朵 词向量: tensor([[ 0.3096, -1.0921, 0.0507, -1.4102, -1.6641, -0.0777, 0.7555, 0.7956,
0.7948, 0.8727, 1.9170, 3.4420, -0.8712, 0.1161, -1.0912, -2.0824,
0.0259, -1.0647, -0.5117, 1.5172, -0.0385, -0.3116, 1.7770, -0.8633,
-0.5530, 0.4903, 0.1786, -0.0834, -0.1454, -1.0511, -0.6035, 0.3540,
-0.6492, -0.6907, -0.5937, 0.2177, -0.3159, 0.4670, 0.4429, -1.8321,
-0.1668, 0.3966, 1.5405, 0.4017, 0.9508, -1.8645, 0.4707, -0.5419,
0.2048, -0.0637, 1.7019, 0.6493, 1.0908, 0.6885, -1.8013, -0.9422,
-0.1981, -0.4192, -0.7153, -1.2981, 1.0295, -0.5339, 0.1464, 0.2236,
-1.4971, -0.4581, -1.1740, 1.1611, 0.3127, -0.0834, 0.5868, 1.5329,
-0.9288, 1.8301, -1.2567, -0.4465, -1.2352, 0.4739, -0.8452, 2.2686,
0.4098, 1.6155, -0.0646, 1.8567, -1.8655, 0.2523, 1.0969, -0.5596,
-0.1209, 0.4286, 1.5780, 0.0447, 0.7614, -1.2196, -0.5788, 0.4128,
0.2822, -0.7087, 0.7638, -1.2023, -1.4040, -0.3361, 0.0961, -1.2491,
-1.2159, 1.7288, -1.5975, -0.7892, -0.3213, 0.4886, -0.4395, 0.8248,
-0.1356, 1.0946, -0.0798, -1.0341, 0.5240, 0.0456, -0.4374, 0.5594,
-0.8723, -0.1566, -2.3145, -1.2582, 0.4562, 0.2723, -1.0719, -0.1717]])
7: 一朵 词向量: tensor([[ 0.3096, -1.0921, 0.0507, -1.4102, -1.6641, -0.0777, 0.7555, 0.7956,
0.7948, 0.8727, 1.9170, 3.4420, -0.8712, 0.1161, -1.0912, -2.0824,
0.0259, -1.0647, -0.5117, 1.5172, -0.0385, -0.3116, 1.7770, -0.8633,
-0.5530, 0.4903, 0.1786, -0.0834, -0.1454, -1.0511, -0.6035, 0.3540,
-0.6492, -0.6907, -0.5937, 0.2177, -0.3159, 0.4670, 0.4429, -1.8321,
-0.1668, 0.3966, 1.5405, 0.4017, 0.9508, -1.8645, 0.4707, -0.5419,
0.2048, -0.0637, 1.7019, 0.6493, 1.0908, 0.6885, -1.8013, -0.9422,
-0.1981, -0.4192, -0.7153, -1.2981, 1.0295, -0.5339, 0.1464, 0.2236,
-1.4971, -0.4581, -1.1740, 1.1611, 0.3127, -0.0834, 0.5868, 1.5329,
-0.9288, 1.8301, -1.2567, -0.4465, -1.2352, 0.4739, -0.8452, 2.2686,
0.4098, 1.6155, -0.0646, 1.8567, -1.8655, 0.2523, 1.0969, -0.5596,
-0.1209, 0.4286, 1.5780, 0.0447, 0.7614, -1.2196, -0.5788, 0.4128,
0.2822, -0.7087, 0.7638, -1.2023, -1.4040, -0.3361, 0.0961, -1.2491,
-1.2159, 1.7288, -1.5975, -0.7892, -0.3213, 0.4886, -0.4395, 0.8248,
-0.1356, 1.0946, -0.0798, -1.0341, 0.5240, 0.0456, -0.4374, 0.5594,
-0.8723, -0.1566, -2.3145, -1.2582, 0.4562, 0.2723, -1.0719, -0.1717]])
44: 因 词向量: tensor([[-0.6953, 0.0299, 1.1689, 0.5407, 0.2593, 1.0880, -0.6120, -0.4362,
-0.5349, 0.0470, -1.7298, -0.3189, -0.6521, -1.1419, -0.1377, -0.3708,
0.0674, -0.0340, -0.6774, 1.3706, 1.4224, 1.3449, 0.3215, -0.3885,
1.4160, -0.8970, -0.3351, 0.2710, 0.1199, -0.4192, -0.8231, 0.0205,
-0.8621, -0.0056, 0.4288, -0.8688, 0.6278, -0.6215, 1.2195, -1.8778,
-1.1621, -0.6641, -2.2388, 0.1523, 0.9739, -0.2387, -1.1108, 0.0185,
-1.3954, 0.4345, -1.9215, 0.0101, 0.8398, -0.7043, 1.2207, -0.6346,
-0.3100, -1.4357, 0.5465, -0.1911, -0.6763, -0.2377, 1.2261, 0.3085,
1.0818, 1.4969, 0.5146, 0.3250, -0.8032, 0.7045, -0.0147, 0.1613,
-0.6856, -0.0940, 0.7458, 1.2374, 1.1497, 0.0072, -0.6461, -0.9718,
-0.7268, 1.5200, -0.2985, -1.9819, -0.7442, 1.6588, 0.2809, -0.0664,
-0.4558, -1.3838, -0.1158, -0.9900, 0.3545, 0.1350, -0.3233, -1.0548,
-0.6536, -0.0381, 1.4860, -1.0694, 1.1341, -1.5340, -0.7540, 0.6109,
-0.3733, -1.9303, -1.0210, -0.3443, 2.2516, 0.0581, -0.0224, -1.3393,
-0.5132, -0.3895, 1.2804, -0.6350, 0.5694, 0.7077, -0.8996, -0.2569,
3.1786, -2.2239, 0.2082, -0.4805, 0.6633, 0.4823, 1.0565, -1.2659]])
2: 你 词向量: tensor([[-0.2350, 0.2923, -0.5940, 0.4420, -0.5620, 0.6781, 0.6996, 1.1685,
-1.1202, 1.1702, 1.7431, 1.1335, -0.2978, -0.8452, -0.1611, -0.7057,
-1.2757, -0.4136, -0.8584, 0.7088, 1.0133, 0.6883, 0.7639, 0.3339,
-0.0689, -0.8740, 0.1080, -0.2372, -0.0766, -1.1628, 1.2577, -1.4728,
-0.8391, 0.1434, -0.7532, 0.1123, -0.0730, -0.3895, -0.8914, 1.0943,
0.9030, 0.6517, 1.9855, -0.5613, -0.9821, -0.0886, 0.6993, 1.1337,
0.0435, -1.1503, 1.6857, -0.9903, -0.1018, -2.5759, 0.3399, 0.7758,
-1.5326, -0.9689, -0.0901, 0.1434, 2.1109, 1.5606, -1.2908, -0.8750,
-0.1149, 0.3210, 1.2538, 0.2164, -1.1185, -1.3347, -0.1046, -1.3265,
-0.0519, 1.7280, -0.8752, -0.2027, 0.0870, 0.3137, 2.1226, 0.7773,
-0.4965, 1.2673, 0.9419, 0.0753, 1.3247, -0.2009, -0.1344, 1.8694,
-0.8257, -1.2295, -0.1927, 0.9116, -0.9511, 2.7437, -0.3050, 0.6188,
-0.6865, 0.1427, 0.6941, -0.4649, -1.3865, 0.1717, 0.4214, -1.9276,
0.1772, 0.2489, -0.8077, -0.8780, -0.4985, 0.4747, -2.3380, 1.7014,
-2.1964, 0.3989, 0.0545, 0.6022, -0.4869, -1.9906, -1.0134, 0.2028,
-0.0669, 0.2856, -0.4412, 0.8217, 0.0568, 1.1643, -1.0116, 1.9836]])
45: 而 词向量: tensor([[-6.9493e-01, 1.4062e+00, -3.1675e-01, 5.8409e-01, 2.9410e-01,
-5.5122e-01, -6.2750e-01, -1.2793e+00, 6.2230e-01, 1.8255e+00,
-1.5399e+00, 7.1456e-01, 3.0716e-03, -1.0304e+00, 7.4607e-01,
-1.3288e+00, -5.4598e-01, 2.0768e+00, -5.0377e-01, 1.5654e+00,
-1.1130e+00, -6.5534e-01, -5.2108e-01, 1.0062e+00, -4.0439e-01,
-4.0440e-01, 1.0674e+00, -9.5448e-01, -2.7082e-01, 1.4654e+00,
-2.8040e-01, -1.4891e+00, 6.3615e-01, 6.6646e-01, -4.0095e-01,
-1.0941e+00, 1.9780e-02, 1.0787e-01, -1.5074e-01, -6.5446e-01,
-5.3278e-01, -4.2440e-01, 4.4326e-01, -1.9796e+00, -2.3452e-03,
1.1360e+00, -6.2523e-01, 2.4874e-01, -1.2869e+00, -1.2564e+00,
1.8399e+00, 1.1310e+00, 3.7163e-01, 1.0387e-01, 1.9941e+00,
-1.9027e+00, 5.3621e-02, -3.1132e-01, -4.8634e-01, 1.2258e+00,
4.1567e-01, 8.8325e-02, -4.1522e-01, -7.3539e-01, -1.2774e+00,
-6.7218e-01, -9.5278e-01, 1.8209e+00, 1.5850e+00, -1.4090e+00,
-3.7985e-01, 4.8194e-01, 1.5726e+00, 1.3843e+00, 1.2500e+00,
6.6419e-01, -5.8809e-01, -2.1640e-01, 1.2158e+00, 7.3180e-01,
1.0677e+00, 2.6748e-01, -6.7863e-01, -3.5348e-01, 2.0592e-01,
-1.6295e+00, 1.2518e+00, 6.7684e-02, 8.8990e-01, -1.4817e-01,
-3.5291e-02, 1.1091e+00, -2.4658e+00, 8.8627e-01, 5.1475e-01,
1.1680e+00, -9.7546e-01, -5.5057e-01, -3.8526e-01, -1.2595e+00,
8.0320e-01, -1.1163e+00, -4.2267e-01, -1.5712e+00, -8.3920e-02,
-7.5662e-01, 3.3516e-01, -1.0455e+00, -8.5412e-01, -1.0817e-01,
-1.9216e-01, 8.4686e-01, -4.6614e-01, 6.8509e-01, 6.1812e-01,
-1.9532e+00, 2.3169e+00, -1.3680e-01, -7.9031e-01, -2.6187e-01,
-1.0526e+00, -1.5326e-01, 1.7926e-01, 3.3464e-01, -4.1420e-01,
6.3608e-01, 1.5391e+00, -1.4365e+00]])
46: 香 词向量: tensor([[ 1.1219, 1.0796, 0.2386, -0.2531, 1.3683, 1.0109, 1.4057, 0.4779,
-1.7352, 0.8844, -1.0220, -0.6492, -0.6238, -0.9067, -1.2073, -1.2844,
0.1932, 2.4944, -2.6507, 1.5242, 2.0159, 0.8329, -0.1949, -0.0109,
0.4642, 0.4089, -1.5732, -0.6178, 0.0168, -0.8314, -2.0872, 2.4771,
-0.7312, -2.2930, -0.2236, 0.5033, 0.9182, 0.8936, -0.8301, 1.6399,
-1.0809, -1.2183, -0.3412, -0.5223, 1.7513, -0.5539, 0.6528, 0.0027,
1.8203, -0.9402, 0.1190, 0.3335, -0.5194, 0.3879, -0.6065, 0.2250,
0.5431, -0.7531, 0.2786, -0.0649, 0.8879, 1.9610, -0.3204, -1.3830,
-0.0650, 0.9430, -2.0940, -1.3006, 0.3104, -1.8551, 0.8262, -1.4165,
-1.1421, 1.4812, -1.1354, -0.1728, 0.4486, -0.0232, -1.9880, 0.2007,
-0.9809, -1.0533, -1.0997, 0.8863, -0.5470, 1.1782, 0.9364, 0.9233,
-0.6922, -1.0484, -0.4710, -0.9735, -0.8595, -1.3355, -2.3149, 0.6993,
-0.5502, -1.1625, -1.2600, 0.9010, -0.1639, 0.2649, 2.1328, -0.2760,
1.5169, 0.0307, -1.7372, 0.7917, -0.1726, 0.6966, -0.6777, 0.2696,
-0.5333, -0.1086, -0.2842, -0.7344, -0.2377, -0.2307, -0.7686, -1.1642,
-0.7456, -0.8460, -0.9036, 0.0933, -0.5102, -0.2190, 0.0434, 0.5659]])
1:
词向量: tensor([[ 1.4771, 0.8061, -0.4299, -0.5632, -1.3281, -0.5809, -0.4281, 0.5693,
0.6510, -0.1457, -0.2895, -1.2239, -0.8608, -1.1156, 1.4625, 0.9993,
2.6551, -0.2841, -0.1222, -0.1759, 0.2747, 0.8773, -1.2195, 0.0723,
-1.1253, -0.3896, -0.5094, -0.3521, 0.9901, 1.1108, 1.1191, -1.1345,
-0.5516, -1.0788, 0.2044, 0.1993, 0.2574, -0.3992, 0.0040, 0.1144,
-0.7567, 1.0407, -0.4901, -0.9804, 0.4055, 0.4972, -0.4097, -0.2506,
-0.0374, -0.2787, 0.5761, -0.0540, -0.8492, -0.7583, 1.0121, -0.6011,
-1.4103, 0.6407, 0.3240, 0.5556, -1.6938, 0.2747, 0.7353, -2.0933,
-1.7379, -0.5252, 1.3509, -0.2661, -0.3593, -0.2447, -1.2453, 0.2860,
-1.1808, 1.6014, -1.1948, 1.3895, 0.4737, -1.4571, 1.1382, -0.8261,
0.5641, -2.5230, 0.2975, 1.0076, -2.1213, 0.0967, 0.6917, 1.7999,
0.4339, 1.3127, 1.9793, 1.2312, -0.2442, 2.5434, 0.1964, 0.1350,
-1.6063, 0.6460, 0.2100, 0.4153, -0.0708, -0.2820, -0.3491, 0.3494,
-1.0825, 1.4995, 1.1523, 0.9549, 0.7601, 0.5949, 1.0576, 0.5871,
-1.3045, -0.5725, 0.5301, 0.5057, 0.8488, 0.5929, 1.0189, 0.0787,
0.9573, -0.7332, 0.9140, -0.4795, 0.2905, 0.9838, 0.3004, -0.2756]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
47: 试图 词向量: tensor([[ 3.8229e-02, 1.0295e+00, -9.8116e-01, 3.0016e+00, -8.3988e-04,
-5.4871e-01, 1.6143e-01, -7.7370e-01, -4.2112e-01, 2.1031e-02,
-1.9869e+00, -5.6874e-02, -3.0638e-01, 1.8294e+00, 1.5476e+00,
-1.5998e-02, -1.3612e+00, -1.0680e+00, 9.1975e-01, 9.6268e-01,
-4.1225e-01, -2.4728e-01, 8.7029e-01, -3.1911e-01, -7.3283e-01,
6.1494e-01, 2.7506e+00, 2.5409e+00, 6.3492e-01, 5.7630e-01,
-7.7426e-02, 8.4821e-01, -1.0945e-01, 1.0364e+00, -2.0155e+00,
-2.4681e-01, -1.9374e-01, -2.0869e-01, 8.7123e-01, -6.9003e-01,
-2.2480e+00, 1.2094e+00, 4.3536e-01, -8.7690e-01, -8.0195e-01,
1.3994e-01, -1.4737e+00, 1.1885e+00, -8.2735e-01, 1.1662e-01,
-2.6052e-01, 2.5670e-01, 9.9016e-01, -3.6616e-01, 1.4097e+00,
-4.0346e-01, 5.5548e-01, 5.6929e-01, 1.6270e-01, 1.8525e+00,
1.0757e+00, 3.4564e-01, -2.4647e-01, 4.0178e-01, 9.3586e-02,
-2.0745e+00, -9.6670e-02, 1.2683e+00, -1.4990e+00, 2.9566e+00,
9.4927e-01, -1.9397e-01, 1.7072e+00, 1.3692e+00, -1.1363e+00,
-1.8812e+00, -4.4998e-01, 2.9471e-01, 8.6469e-01, 8.5077e-02,
-7.3075e-01, -8.7080e-01, 1.0867e+00, -1.2369e+00, -4.6418e-01,
-7.6873e-02, -1.4832e+00, -1.5348e+00, -2.6368e-01, 2.0422e+00,
-1.4194e+00, -1.0408e+00, -6.6020e-02, 3.2093e-01, -1.3257e-01,
5.0488e-01, -1.1369e+00, -7.6361e-01, -2.5426e-01, -1.0196e-01,
6.7761e-01, 5.2637e-01, -8.7972e-01, -1.2195e-01, 1.5258e+00,
-6.6660e-02, 3.2165e-02, 3.9270e-01, 1.2309e+00, 1.9312e+00,
-4.9214e-01, -8.3518e-01, -2.4525e-01, 9.7059e-01, 1.3650e+00,
-6.3579e-01, 5.3070e-01, 1.8185e-01, -1.5820e+00, 3.4072e-01,
-1.2839e+00, 8.3954e-01, -1.7604e+00, -6.7730e-01, -3.8595e-01,
-4.6726e-01, 5.9705e-01, -9.1093e-01]])
48: 让 词向量: tensor([[ 0.1047, -0.1963, -0.5827, 0.2373, 0.5354, 1.1712, 0.4129, 0.3905,
1.7455, -0.1996, 1.4604, -0.1104, 1.5858, -0.2527, -0.6230, 0.4800,
-0.7811, 0.2958, -0.3782, 1.9233, -0.5916, -0.3274, -1.5021, -0.0400,
-1.7086, -1.2303, 0.3489, -0.2678, 0.0236, 0.4103, 0.6926, -0.3335,
-0.0477, -0.4302, 2.1077, -0.1851, 0.8022, -0.1936, -1.5385, -0.9973,
0.3996, -1.0661, -0.5265, 0.9906, 1.3254, 0.4078, 1.6247, -0.3208,
-0.5164, -1.8549, 1.9178, 0.6265, -0.5813, -0.4273, -1.3287, -0.7632,
0.0737, -0.5155, -0.2682, 1.9832, -0.7008, -1.9192, -0.1252, -1.5195,
-0.1937, -0.5278, 0.8634, -0.0307, 0.0053, 1.1085, -0.4354, 1.0311,
-0.3634, 1.1479, 0.5621, 1.8091, 0.8000, -1.5153, -0.0878, 0.5621,
0.3949, 1.5060, -0.3300, 0.4974, -1.6473, 1.9393, 0.7104, -0.7937,
-2.3827, -0.3346, 0.2019, -0.5378, 1.3844, 0.7436, -2.1666, 2.0400,
0.6024, -0.2300, 0.8634, 0.6597, 1.5599, -1.2148, -0.7414, 0.2471,
-1.4901, 0.9992, 0.9838, -0.0489, -0.4885, -0.4100, 0.1226, 0.5899,
-0.3981, 0.2456, -0.2408, -1.7554, 0.8440, 0.5517, 0.6177, 1.9538,
-0.1266, 0.7740, -0.3234, 1.0270, 0.2702, -0.6555, -0.0128, 1.1648]])
49: 夕阳 词向量: tensor([[-0.6004, 0.9376, -0.6297, -1.8173, 0.2103, 0.3722, 0.0192, 0.9139,
0.5123, -1.6350, -1.4789, 0.2643, 1.5317, 0.8034, -0.0666, 0.4330,
0.2521, 0.6076, 0.4397, -1.1878, 0.6810, -0.4961, 0.6582, -1.2506,
-0.9845, -0.7020, -0.5939, 0.8869, 0.8704, -1.1969, -0.7378, 0.4396,
2.2369, 0.7673, -2.3626, -0.2410, -0.1081, 1.3425, -0.2445, -0.9309,
-0.5149, -0.3644, 0.8627, 0.3312, -1.0202, -0.1720, -0.7266, -1.8455,
-1.8430, -0.8775, -1.7141, -0.5157, 0.0070, -1.5449, 1.2689, -1.0665,
-0.9434, -0.0797, -1.0266, 0.4979, -0.9363, 1.4508, 1.1772, -0.3139,
-0.5204, -0.2885, 0.1240, -0.0211, -2.3010, 0.8195, -0.2835, 1.2481,
-0.5358, -0.8594, -0.4159, 1.3001, -0.6774, 2.3530, -1.6556, 1.9768,
0.3301, 1.6243, -1.2789, -0.8119, 0.9700, -0.2757, 1.4842, 0.0566,
-0.4246, 1.4038, 0.9694, 0.1432, -0.0601, -0.7541, 0.9335, -0.7924,
1.2861, 1.3255, 0.5303, -0.3681, -0.5266, -1.3112, -0.1385, -0.5437,
-0.1600, 0.2397, -1.5492, -0.1492, 1.2562, -0.0595, 0.2199, 0.5822,
-0.3988, -0.0322, 1.8986, -0.9918, 0.9039, -1.7710, -1.4276, 1.7922,
1.1502, -2.5430, -0.1372, 1.4500, -0.3768, -0.4091, -1.5363, 0.4002]])
50: 飞翔 词向量: tensor([[ 1.0860, -0.0171, -0.6277, -1.1933, 0.9315, -0.7330, 0.8131, 0.1651,
0.4671, 1.8656, -0.4305, -1.0249, 2.9707, 1.3919, -0.6975, -1.8236,
-1.0667, 1.5667, -0.9691, 0.2355, 0.1622, -0.5192, 0.0356, 0.3255,
0.3883, -0.6321, -0.5643, 0.2612, 0.5231, -2.6157, -1.4175, 1.1954,
1.7983, -0.3155, 0.6007, -0.2746, -0.7162, -0.1186, -0.2207, 0.6713,
-0.5339, 0.4534, 0.5201, 0.9344, 1.6918, -1.7041, -0.4986, 0.1152,
0.4424, -1.0363, 0.0898, 1.0369, -0.6468, -0.9246, -0.2190, 0.0517,
-0.9905, -0.5032, 0.8691, -0.3064, -0.9427, -0.1224, -0.5199, -0.3677,
-1.7578, -0.3239, -0.5745, 2.1304, 2.5637, 0.0856, -0.5059, 0.2896,
1.3139, -1.7180, 0.0164, -0.7947, 0.1157, 0.4066, 1.0226, 0.9724,
1.0858, -0.5026, 1.0933, -0.7102, -0.9489, -0.5639, -0.6002, -0.2920,
1.4527, -0.6295, 1.3500, 0.6058, 0.9799, -0.4328, 0.1221, -0.9199,
0.6476, -0.5325, -1.3911, 1.7790, -1.3404, -0.3926, -0.0288, 0.1779,
0.6804, 1.3036, 0.8603, 0.4150, 0.6042, -0.6114, 1.3965, -2.1026,
0.4287, 1.1885, 1.7315, -1.3131, 0.4715, 0.2487, 0.5676, -0.6143,
-1.0070, 0.9936, -0.0798, -1.1731, -1.6360, -1.3797, -1.1460, -0.6632]])
1:
词向量: tensor([[ 1.4771, 0.8061, -0.4299, -0.5632, -1.3281, -0.5809, -0.4281, 0.5693,
0.6510, -0.1457, -0.2895, -1.2239, -0.8608, -1.1156, 1.4625, 0.9993,
2.6551, -0.2841, -0.1222, -0.1759, 0.2747, 0.8773, -1.2195, 0.0723,
-1.1253, -0.3896, -0.5094, -0.3521, 0.9901, 1.1108, 1.1191, -1.1345,
-0.5516, -1.0788, 0.2044, 0.1993, 0.2574, -0.3992, 0.0040, 0.1144,
-0.7567, 1.0407, -0.4901, -0.9804, 0.4055, 0.4972, -0.4097, -0.2506,
-0.0374, -0.2787, 0.5761, -0.0540, -0.8492, -0.7583, 1.0121, -0.6011,
-1.4103, 0.6407, 0.3240, 0.5556, -1.6938, 0.2747, 0.7353, -2.0933,
-1.7379, -0.5252, 1.3509, -0.2661, -0.3593, -0.2447, -1.2453, 0.2860,
-1.1808, 1.6014, -1.1948, 1.3895, 0.4737, -1.4571, 1.1382, -0.8261,
0.5641, -2.5230, 0.2975, 1.0076, -2.1213, 0.0967, 0.6917, 1.7999,
0.4339, 1.3127, 1.9793, 1.2312, -0.2442, 2.5434, 0.1964, 0.1350,
-1.6063, 0.6460, 0.2100, 0.4153, -0.0708, -0.2820, -0.3491, 0.3494,
-1.0825, 1.4995, 1.1523, 0.9549, 0.7601, 0.5949, 1.0576, 0.5871,
-1.3045, -0.5725, 0.5301, 0.5057, 0.8488, 0.5929, 1.0189, 0.0787,
0.9573, -0.7332, 0.9140, -0.4795, 0.2905, 0.9838, 0.3004, -0.2756]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
51: 带领 词向量: tensor([[-0.3883, 1.4575, 2.0290, -0.8347, -0.5549, -0.4177, 0.9295, 1.0170,
-0.6016, 0.4783, -0.8574, 0.3708, -1.2610, -0.8413, 0.9271, 0.3631,
-0.8457, 0.1038, -1.3275, 0.2564, -0.0925, 1.5057, -0.4423, -1.3217,
-0.9225, -1.1700, 0.6291, 1.5080, -0.3643, -0.8193, 1.8530, 0.8711,
-1.9185, -0.8187, 0.7976, 1.1875, -0.7883, 0.7925, -0.7433, 0.2557,
0.9899, 0.5820, 0.9995, -0.0171, -0.2620, 0.2623, 0.5431, -0.7220,
-0.0887, -0.2825, -0.7457, -0.5655, -0.5712, -0.2949, 1.7468, -0.8723,
0.1218, -0.4338, -1.5607, -0.1263, 1.4716, 0.1932, -1.9922, 0.2615,
0.7598, -0.5381, 1.3246, 2.7312, 0.2112, 1.1132, -0.2877, -0.8743,
0.0557, 0.6472, 0.0501, -2.3401, 0.3528, 0.2412, 0.3257, 0.2621,
0.7255, 2.1305, 0.6389, -0.4104, -0.4304, 1.1592, 0.1603, 0.3825,
0.2950, 0.0406, 0.7059, 0.6141, -1.3077, 0.4607, -0.3397, -0.6514,
-0.4276, -1.3638, 0.6839, 0.2592, -0.2928, 0.2293, -0.4305, 0.9677,
2.0352, -0.2551, 0.4089, 1.4816, -1.1304, -0.4020, -0.4308, 1.0968,
0.6394, -1.0796, -1.0582, 1.2715, -0.1240, 0.4402, -0.8747, 0.0545,
0.4919, -1.9369, 0.8893, -1.1005, 0.9882, 0.3336, 0.8308, 1.3954]])
2: 你 词向量: tensor([[-0.2350, 0.2923, -0.5940, 0.4420, -0.5620, 0.6781, 0.6996, 1.1685,
-1.1202, 1.1702, 1.7431, 1.1335, -0.2978, -0.8452, -0.1611, -0.7057,
-1.2757, -0.4136, -0.8584, 0.7088, 1.0133, 0.6883, 0.7639, 0.3339,
-0.0689, -0.8740, 0.1080, -0.2372, -0.0766, -1.1628, 1.2577, -1.4728,
-0.8391, 0.1434, -0.7532, 0.1123, -0.0730, -0.3895, -0.8914, 1.0943,
0.9030, 0.6517, 1.9855, -0.5613, -0.9821, -0.0886, 0.6993, 1.1337,
0.0435, -1.1503, 1.6857, -0.9903, -0.1018, -2.5759, 0.3399, 0.7758,
-1.5326, -0.9689, -0.0901, 0.1434, 2.1109, 1.5606, -1.2908, -0.8750,
-0.1149, 0.3210, 1.2538, 0.2164, -1.1185, -1.3347, -0.1046, -1.3265,
-0.0519, 1.7280, -0.8752, -0.2027, 0.0870, 0.3137, 2.1226, 0.7773,
-0.4965, 1.2673, 0.9419, 0.0753, 1.3247, -0.2009, -0.1344, 1.8694,
-0.8257, -1.2295, -0.1927, 0.9116, -0.9511, 2.7437, -0.3050, 0.6188,
-0.6865, 0.1427, 0.6941, -0.4649, -1.3865, 0.1717, 0.4214, -1.9276,
0.1772, 0.2489, -0.8077, -0.8780, -0.4985, 0.4747, -2.3380, 1.7014,
-2.1964, 0.3989, 0.0545, 0.6022, -0.4869, -1.9906, -1.0134, 0.2028,
-0.0669, 0.2856, -0.4412, 0.8217, 0.0568, 1.1643, -1.0116, 1.9836]])
52: 我 词向量: tensor([[ 1.8025, -0.8686, 0.6503, 1.3381, 0.0555, 0.6407, 0.0161, 0.5962,
-2.1178, 0.9846, -0.1072, 1.3838, 0.8264, -1.4054, 0.2844, -0.2970,
-2.3557, -0.5851, 0.1094, -0.5116, -1.8826, -1.1737, 1.0207, 1.2605,
1.6016, -0.4241, 0.9683, -1.0203, 0.2468, 0.4037, 1.1892, 0.5581,
1.0928, 0.2257, 0.7151, -1.2975, -0.4777, 0.8343, 0.7666, 0.9375,
-0.2058, 1.6071, -1.5055, 1.1093, 0.5306, 0.5420, -0.6054, -1.3755,
-0.8689, 0.7478, -0.9576, 0.0944, 0.0334, -1.0172, 0.7365, 0.3529,
1.0343, -1.5456, 0.5741, -0.0613, 0.8689, 0.7043, -2.2342, 1.3287,
0.4374, -0.2889, -1.6983, 1.7067, -0.9939, -0.2796, -0.4290, -0.4194,
1.0979, -0.5970, -0.9135, 0.4163, 1.6215, 0.1321, -0.4211, 0.1524,
1.1856, 0.3440, 0.5450, -0.0427, -1.0537, -0.5387, -0.1554, 0.5156,
1.3124, -0.7575, 0.0543, 0.4914, -0.7407, 0.5013, -0.1477, -0.7364,
2.0451, -1.9390, -0.9945, -0.4166, -2.1268, 0.7785, -0.7775, 0.7935,
-0.7087, -0.9547, 0.8181, 0.2678, 0.0905, 0.9333, -0.6942, 1.2687,
1.4517, 0.6902, -0.5689, 1.0928, 0.6622, 1.2018, -0.5590, -0.6284,
-0.4608, 0.9066, -1.3882, -1.0355, -0.6040, -0.8719, -0.2974, 2.1041]])
53: 环绕 词向量: tensor([[ 2.1902e-01, -1.1552e+00, -1.8424e-01, 3.3614e-01, -3.8256e-01,
-1.1324e+00, -6.7924e-02, -5.9995e-01, 6.1576e-02, -1.0614e+00,
4.6223e-01, -1.1970e+00, 1.1080e+00, -1.8975e+00, -1.6733e+00,
-7.9665e-02, -8.5991e-01, -9.1305e-01, -3.2379e-01, -2.4269e-01,
1.7524e+00, -1.2605e-01, -1.1945e+00, 5.8631e-01, -5.3558e-01,
5.2792e-01, -1.1576e+00, 6.4780e-02, -6.7487e-01, 7.4069e-01,
2.1513e+00, -7.9131e-01, 4.3649e-01, -4.6189e-01, -8.5121e-01,
-2.4487e-01, -1.5749e+00, -1.1254e+00, -7.6424e-01, -1.6875e-02,
8.4802e-01, 6.9762e-01, -8.4007e-02, 1.5265e+00, 1.7715e+00,
-1.3288e+00, -9.2405e-01, 3.3903e-03, -3.5675e-02, -1.0754e+00,
3.8780e-01, 1.2380e+00, -1.5078e+00, 1.4308e-02, -1.8832e-01,
2.4368e-01, 1.6547e+00, 3.3784e-01, 6.2146e-01, -2.0486e-01,
2.9040e-01, 2.3162e+00, 1.3580e+00, -1.5030e+00, -3.1261e-01,
2.5586e-01, 3.7365e-01, 6.7240e-01, -1.4438e-02, 6.4064e-01,
-8.4326e-01, 4.9617e-01, 3.2818e-01, -2.4743e+00, -1.1754e+00,
-2.2826e-01, 2.0747e+00, 3.5032e-01, 9.2976e-02, 5.0810e-01,
3.9344e-01, 1.0260e+00, -2.1320e-01, 6.9975e-02, 2.4769e+00,
-3.3605e-01, 1.5252e+00, -8.1502e-02, 7.5484e-01, 2.8961e-01,
9.4699e-01, 5.1192e-01, 2.2573e+00, -1.2976e+00, -6.8307e-01,
-5.0647e-01, -9.5794e-02, 9.7939e-01, 9.8515e-01, 1.1481e+00,
1.0263e+00, -1.4036e+00, 1.1095e+00, -1.0258e+00, 1.7887e-01,
-1.8991e-01, 1.7981e+00, -1.5916e+00, 9.0513e-01, 8.2632e-01,
-1.2119e+00, 7.6358e-01, 1.8939e-01, 5.4378e-01, 4.9077e-01,
-9.7766e-01, 5.6718e-01, -4.8325e-01, 9.4810e-01, -6.4625e-01,
2.1275e+00, -2.6082e+00, 4.6068e-01, -9.4026e-01, -1.8495e-03,
7.4016e-01, -1.3644e+00, -9.7325e-01]])
54: 大自然 词向量: tensor([[ 1.0683e+00, -9.3921e-01, 1.0142e+00, 5.2375e-01, -1.6918e+00,
-1.3720e+00, -2.8962e-01, -4.4843e-01, -1.4024e+00, -7.2348e-01,
-1.6988e+00, 2.6401e-01, -1.3191e-03, -5.3966e-01, 1.2056e+00,
-1.2444e+00, -8.3017e-02, -4.6160e-03, 1.0160e+00, 5.7973e-01,
8.4230e-01, -1.1997e+00, 3.1302e-01, -7.8104e-04, 1.4088e-01,
-9.4516e-01, 1.1195e-01, 9.8150e-01, 4.8767e-01, 4.8473e-01,
-6.9974e-02, 1.1046e+00, -9.6830e-01, -1.2903e+00, 1.0750e+00,
-1.3531e+00, -3.2607e-01, 1.4716e+00, 1.3152e+00, -1.0358e+00,
-4.1777e-02, 8.5979e-01, 7.0251e-01, 4.8289e-01, 2.6085e-01,
7.8900e-01, -5.6025e-02, -1.3694e+00, -1.4287e-01, -7.6493e-01,
9.9712e-01, -1.2491e+00, -1.2640e+00, 8.3255e-01, 2.3806e-01,
-2.4682e-01, 1.4890e+00, -2.1768e+00, -1.1718e+00, -1.2710e-01,
-7.2177e-01, 1.6948e+00, -3.6733e-01, 2.4117e+00, -1.2189e+00,
-1.0218e+00, 8.6482e-01, 1.1552e+00, -9.7631e-01, 1.7594e+00,
-3.4284e-01, 3.7660e-01, -1.5496e+00, 4.1770e-01, 1.4202e+00,
-3.3451e-02, -1.0973e+00, -7.3886e-01, -1.1802e+00, 2.7324e-01,
1.1071e+00, -1.8545e-01, -1.0871e+00, -3.2923e-01, 8.6681e-01,
2.2645e+00, 6.7991e-01, 7.6886e-01, 8.8457e-02, -7.2791e-01,
1.0182e+00, -1.7361e+00, -3.4060e-01, 3.0723e-01, -5.5604e-01,
-6.3626e-02, -2.0292e-01, 1.8210e+00, 5.6976e-01, -3.8184e-01,
3.1810e-01, -1.1195e+00, -2.4723e+00, 9.2755e-01, -4.2846e-01,
5.0195e-01, -3.5629e-01, 1.1201e+00, -1.7522e+00, -1.1405e-01,
-1.1771e+00, 4.1570e-01, -2.2039e-01, 7.2794e-01, 6.5139e-01,
-4.2230e-01, -4.1998e-01, 1.1662e+00, 2.7355e-02, -1.2218e+00,
-7.0014e-01, 2.1131e+00, 6.1906e-01, -2.2585e-01, -3.8449e-01,
-2.2723e-01, -4.5542e-01, -1.1120e+00]])
1:
词向量: tensor([[ 1.4771, 0.8061, -0.4299, -0.5632, -1.3281, -0.5809, -0.4281, 0.5693,
0.6510, -0.1457, -0.2895, -1.2239, -0.8608, -1.1156, 1.4625, 0.9993,
2.6551, -0.2841, -0.1222, -0.1759, 0.2747, 0.8773, -1.2195, 0.0723,
-1.1253, -0.3896, -0.5094, -0.3521, 0.9901, 1.1108, 1.1191, -1.1345,
-0.5516, -1.0788, 0.2044, 0.1993, 0.2574, -0.3992, 0.0040, 0.1144,
-0.7567, 1.0407, -0.4901, -0.9804, 0.4055, 0.4972, -0.4097, -0.2506,
-0.0374, -0.2787, 0.5761, -0.0540, -0.8492, -0.7583, 1.0121, -0.6011,
-1.4103, 0.6407, 0.3240, 0.5556, -1.6938, 0.2747, 0.7353, -2.0933,
-1.7379, -0.5252, 1.3509, -0.2661, -0.3593, -0.2447, -1.2453, 0.2860,
-1.1808, 1.6014, -1.1948, 1.3895, 0.4737, -1.4571, 1.1382, -0.8261,
0.5641, -2.5230, 0.2975, 1.0076, -2.1213, 0.0967, 0.6917, 1.7999,
0.4339, 1.3127, 1.9793, 1.2312, -0.2442, 2.5434, 0.1964, 0.1350,
-1.6063, 0.6460, 0.2100, 0.4153, -0.0708, -0.2820, -0.3491, 0.3494,
-1.0825, 1.4995, 1.1523, 0.9549, 0.7601, 0.5949, 1.0576, 0.5871,
-1.3045, -0.5725, 0.5301, 0.5057, 0.8488, 0.5929, 1.0189, 0.0787,
0.9573, -0.7332, 0.9140, -0.4795, 0.2905, 0.9838, 0.3004, -0.2756]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
55: 迎著风 词向量: tensor([[ 0.2499, -0.9195, -2.0798, 0.8443, -0.4590, -0.2433, 0.4116, 1.4193,
0.1332, -0.1165, -0.6010, -0.3951, 0.7606, -1.3181, 0.2098, -1.1948,
0.2119, -0.1856, 0.5986, 1.0093, -1.6948, -0.9851, 0.9495, 0.3580,
-0.0412, -0.6933, 0.3507, 0.5312, -2.0586, 0.0343, -0.0656, 0.1630,
1.2536, -0.0250, 0.8471, 0.3254, 0.4763, 0.6172, 0.0852, 0.0637,
-0.4052, 0.5236, -0.2595, -1.4127, 0.3243, 1.0533, -0.6825, -1.5045,
-0.9738, -1.9627, 0.7090, 0.0586, -0.8406, -0.9009, -0.6829, -0.9942,
0.0103, 0.1358, 0.6260, -0.0610, 1.4292, -1.4707, -1.2501, 0.1482,
-1.9348, -0.7603, 0.2080, -1.4380, -0.2337, 0.5236, 0.3099, -0.0222,
0.1745, 2.7300, -1.4230, -0.4661, -0.7483, 2.5051, 0.8559, -0.1612,
0.8224, -0.2808, 0.3730, -0.9974, 0.6265, 1.1285, 0.1511, 0.0687,
0.7169, 0.3940, -0.0870, 0.4539, -1.6034, -0.0756, 0.1008, -1.5667,
0.2044, 0.2164, 0.4385, -1.1896, -0.7260, 0.0813, -1.5283, -0.8356,
-0.5925, -0.2488, -0.2587, -1.6830, -1.5480, -0.3825, -0.5161, -0.5260,
-0.7144, -0.5617, -1.1504, -0.0983, 0.9098, 0.5436, -0.6765, -1.9885,
-2.2751, -0.4951, -0.6814, 0.6681, -1.1405, 0.7472, 0.0320, 0.8300]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
56: 开始 词向量: tensor([[-1.9813, 1.6995, 1.1226, -1.1644, 0.5912, 0.1666, -1.0282, 1.3264,
-0.4648, -0.0262, 1.7921, 0.3861, 1.2850, -0.5048, -0.1850, -1.2177,
-0.9231, -0.4529, 0.7043, -1.9624, -0.9191, -0.4489, 0.7742, 0.0458,
-0.4288, 0.5045, -0.4517, -0.0529, -0.6371, 0.3220, -0.2698, -1.5022,
-1.3722, 0.7447, 0.1231, 0.9082, -0.3852, -0.6957, 0.4795, -0.3938,
0.2775, -1.2321, -0.3383, 0.7813, 0.0436, -1.2526, -0.6505, 0.3326,
1.0836, -0.4308, -1.2012, -1.0896, -1.2444, 2.0645, -0.2010, 1.4258,
0.1280, 0.5159, 0.0596, -0.2161, 0.2984, 0.4174, -0.0073, 1.3804,
0.1315, 0.7714, 0.9009, -0.4024, 1.0126, 1.4795, 0.9897, -0.0254,
-0.9798, -0.4357, 2.2943, -1.0560, 1.2949, -0.8752, 1.4327, 1.0656,
0.1439, -1.4196, -1.4354, -0.7374, 0.0495, 1.0598, -0.3429, 0.2520,
-1.1802, -0.9050, -0.7626, -0.5806, -0.9929, -2.2615, -0.0110, -0.7846,
0.2553, -0.2644, -0.5114, -0.5074, -0.3472, 0.1222, 0.3443, 0.3440,
-1.3284, -0.6892, 0.5428, 0.0414, -0.1834, -0.3867, -2.0045, 0.6607,
-0.6262, 1.8186, -1.2601, -0.1855, -1.2118, 1.4943, 0.5222, -2.3380,
0.6423, 0.2073, -1.1112, 0.8694, 1.3488, -1.7096, -2.0201, -0.9811]])
57: 共渡 词向量: tensor([[ 1.1552, -1.1775, -0.7923, -0.8759, 0.3838, -0.3569, 0.2196, -1.7299,
-0.2527, -0.4146, -0.2309, -0.0932, -1.3646, 0.0742, 0.0528, 1.2435,
0.2058, 1.0656, 0.1760, -0.0908, 0.2878, -0.5905, -0.8570, -1.0456,
0.0119, -0.1301, -0.5770, -0.7324, -1.2146, -0.4203, 0.3345, -1.6111,
-0.0893, -0.5048, -0.4912, 0.5011, -0.6233, -0.0686, 0.5077, 1.3853,
-1.1493, -0.6771, -0.0249, 0.0469, -1.8951, 0.7383, 1.6796, 0.1959,
-1.0561, -1.3896, -1.5098, 2.3460, 1.8851, 0.6651, 0.7328, 0.4741,
-0.4156, -2.0029, -0.0087, 1.3089, -1.1811, -0.3713, 1.4896, -1.9756,
-0.2270, 0.7772, -1.0457, 0.8554, -0.3493, -1.0754, -0.3900, 0.0935,
-1.2546, -1.4160, -0.4934, -1.6591, -0.8026, -0.4168, 1.4469, 1.1436,
0.5042, -0.8564, 1.9937, -0.2030, -0.7135, -1.4389, 0.9389, 0.1246,
-1.0682, 2.0665, -2.5335, 2.5368, -0.9056, -0.5825, 0.5129, 0.6345,
0.1575, -0.3034, -0.7676, -1.3009, 0.6037, -0.5790, -1.8223, -0.7235,
0.9397, -0.3880, -0.4059, 0.9193, -0.3312, 0.2401, 0.9551, -1.3676,
-0.2662, -0.9645, 1.3275, -0.3785, 0.0332, -0.4419, 0.9355, 1.4195,
-1.2115, 1.2054, 2.5421, -0.3062, -1.9236, -0.5395, -0.0240, 0.7995]])
58: 每 词向量: tensor([[ 0.3498, -0.7574, 0.1146, 0.7873, 1.4814, -1.2129, -0.3470, 1.5027,
-0.2193, -1.9263, 0.0346, -0.5626, -0.3347, 0.5228, -0.6182, -0.7035,
-0.3006, -0.8527, 0.3731, 0.0857, -0.0111, -0.8429, -1.6461, 0.3679,
-0.3584, 0.9093, 2.0222, -1.1867, 2.0389, 0.0766, -1.0236, -0.9392,
-0.0163, -0.1390, -1.1153, 1.8077, 1.4790, 0.6615, 0.1883, -1.0539,
2.0340, 0.7649, 0.0168, -0.9907, -1.6050, -0.3988, -0.8727, 0.5305,
-0.6364, 0.2739, 0.7177, -2.1755, -1.6382, -1.4612, 1.9361, 0.0580,
-1.2769, 1.1145, 0.3545, -0.9117, -0.2648, 1.2646, 1.9668, 0.4406,
0.8857, -0.3670, 0.8295, 1.4161, -0.2115, 1.2174, 0.5693, -1.2825,
1.0954, -1.2001, 0.6513, -0.3302, -1.2100, -2.3221, 0.4044, 0.0423,
-0.9381, -0.4300, 2.2154, 1.0064, 0.5278, 0.2070, -1.0909, -2.0609,
-0.5837, 2.0414, -1.2969, -0.0257, -0.8994, -0.8459, -0.7661, 0.9830,
-0.9493, 0.3878, 1.7131, -1.0839, 0.3617, -1.7164, -0.5890, -1.4442,
-0.5854, 1.5730, -0.6862, 1.0519, -1.9228, 0.3219, 1.0994, -1.8571,
0.8783, -0.3197, 0.0278, 1.1004, 0.0468, -0.8312, 0.2939, -0.1949,
-0.2218, 1.3453, 0.1781, 0.2764, 1.5048, 0.7907, -1.4415, -0.5297]])
59: 一天 词向量: tensor([[-0.8323, 0.0881, -1.4882, 0.2288, -1.0547, 1.1075, -0.1441, 0.2287,
0.8294, -1.3625, 0.9649, 0.5535, -0.7352, 0.1774, -0.7410, -1.4113,
0.9171, 1.3320, 0.2866, 0.1871, 0.7052, 1.2598, 0.3351, 2.0226,
-1.3257, -1.3111, 2.5196, 0.1810, 0.6007, -0.9601, 1.2443, -1.5912,
0.2706, -1.9888, 0.0807, -1.1416, 1.5431, 0.5763, -0.4852, -0.5902,
-0.8565, -1.2463, 1.2020, 0.3245, 0.8880, 1.7908, -0.5500, 0.2925,
0.6405, 1.2001, 0.2597, 0.1210, 1.8376, 0.5919, -0.9829, 0.0814,
0.7686, -0.8528, 0.8874, -1.2020, 1.0796, 0.1069, 1.1454, 1.5961,
0.9683, -0.6176, 1.5075, -0.0719, 0.1401, 1.6648, 1.6374, 0.0056,
0.6043, 0.8292, -0.7032, -0.2443, 0.8741, -0.2997, 1.5568, 1.2950,
-1.4252, 0.6444, -0.9421, -0.6760, -0.5903, 0.0040, 0.0740, -0.1053,
0.0359, 1.1091, 0.8509, -1.5936, -1.1683, -1.0063, -0.1447, -0.3768,
-0.0526, -0.2548, -0.2461, -0.6924, -0.5115, 0.3421, 0.9276, -0.4238,
0.5287, 0.2878, 0.4261, -2.0163, -0.9782, 0.6345, -0.3501, 0.2344,
1.3634, 1.1329, -2.0690, 0.7777, -1.3949, -0.7385, 1.0592, 0.7805,
-1.1326, 0.1079, -0.3565, -0.0959, -1.0551, 1.1710, 0.3118, -1.5953]])
1:
词向量: tensor([[ 1.4771, 0.8061, -0.4299, -0.5632, -1.3281, -0.5809, -0.4281, 0.5693,
0.6510, -0.1457, -0.2895, -1.2239, -0.8608, -1.1156, 1.4625, 0.9993,
2.6551, -0.2841, -0.1222, -0.1759, 0.2747, 0.8773, -1.2195, 0.0723,
-1.1253, -0.3896, -0.5094, -0.3521, 0.9901, 1.1108, 1.1191, -1.1345,
-0.5516, -1.0788, 0.2044, 0.1993, 0.2574, -0.3992, 0.0040, 0.1144,
-0.7567, 1.0407, -0.4901, -0.9804, 0.4055, 0.4972, -0.4097, -0.2506,
-0.0374, -0.2787, 0.5761, -0.0540, -0.8492, -0.7583, 1.0121, -0.6011,
-1.4103, 0.6407, 0.3240, 0.5556, -1.6938, 0.2747, 0.7353, -2.0933,
-1.7379, -0.5252, 1.3509, -0.2661, -0.3593, -0.2447, -1.2453, 0.2860,
-1.1808, 1.6014, -1.1948, 1.3895, 0.4737, -1.4571, 1.1382, -0.8261,
0.5641, -2.5230, 0.2975, 1.0076, -2.1213, 0.0967, 0.6917, 1.7999,
0.4339, 1.3127, 1.9793, 1.2312, -0.2442, 2.5434, 0.1964, 0.1350,
-1.6063, 0.6460, 0.2100, 0.4153, -0.0708, -0.2820, -0.3491, 0.3494,
-1.0825, 1.4995, 1.1523, 0.9549, 0.7601, 0.5949, 1.0576, 0.5871,
-1.3045, -0.5725, 0.5301, 0.5057, 0.8488, 0.5929, 1.0189, 0.0787,
0.9573, -0.7332, 0.9140, -0.4795, 0.2905, 0.9838, 0.3004, -0.2756]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
60: 手牵手 词向量: tensor([[-0.4457, -0.0111, -0.1142, -0.3385, -2.3170, -1.1889, -0.4656, 1.8851,
1.0001, -0.9519, 0.9699, 0.2183, 0.6482, -1.7934, 0.1949, -0.2373,
-0.4917, 0.0193, -0.2512, -0.1193, -1.0027, -0.0518, -0.1406, -0.1449,
0.2086, -0.2187, -0.9120, 0.1278, -0.2654, -0.9425, -0.7463, 0.6436,
-2.1115, 0.1178, 1.7578, 0.0539, 0.1830, 0.8112, -0.0274, 1.0397,
-1.3360, 1.3458, -1.3802, -1.0181, 0.4262, -0.9126, -0.9357, 0.2789,
-0.9752, 0.9468, 0.1352, -0.0782, 1.3297, 2.1012, 0.5217, -1.5189,
1.4051, 1.1851, -0.7222, -0.4523, -0.9668, -1.7462, -0.1036, -0.0657,
0.7856, 1.4936, 0.9012, -0.4188, 0.0092, -1.1457, 0.6428, -0.2174,
2.0091, -1.0033, 2.2029, 0.1472, -0.4140, 1.4386, -0.8736, 0.6306,
0.9984, -1.6063, -0.3681, -0.8818, 1.8783, -0.5745, -0.2927, 2.2076,
-0.9706, -1.5232, -0.3766, 0.3183, -0.0388, 2.2997, -1.0957, 1.2374,
1.7551, 1.2685, -0.5895, -0.6288, 0.3892, -1.2646, 0.4644, -0.3824,
-0.0753, -1.0889, 0.1276, -0.9911, -0.3899, 1.1059, -0.1128, 0.4696,
-0.0387, -0.3349, 0.7895, -0.4671, -0.5814, -1.0417, 0.7999, 0.2012,
-2.0221, 0.1848, 0.2324, 0.8634, 1.2771, 0.5084, 0.3314, -0.4662]])
1:
词向量: tensor([[ 1.4771, 0.8061, -0.4299, -0.5632, -1.3281, -0.5809, -0.4281, 0.5693,
0.6510, -0.1457, -0.2895, -1.2239, -0.8608, -1.1156, 1.4625, 0.9993,
2.6551, -0.2841, -0.1222, -0.1759, 0.2747, 0.8773, -1.2195, 0.0723,
-1.1253, -0.3896, -0.5094, -0.3521, 0.9901, 1.1108, 1.1191, -1.1345,
-0.5516, -1.0788, 0.2044, 0.1993, 0.2574, -0.3992, 0.0040, 0.1144,
-0.7567, 1.0407, -0.4901, -0.9804, 0.4055, 0.4972, -0.4097, -0.2506,
-0.0374, -0.2787, 0.5761, -0.0540, -0.8492, -0.7583, 1.0121, -0.6011,
-1.4103, 0.6407, 0.3240, 0.5556, -1.6938, 0.2747, 0.7353, -2.0933,
-1.7379, -0.5252, 1.3509, -0.2661, -0.3593, -0.2447, -1.2453, 0.2860,
-1.1808, 1.6014, -1.1948, 1.3895, 0.4737, -1.4571, 1.1382, -0.8261,
0.5641, -2.5230, 0.2975, 1.0076, -2.1213, 0.0967, 0.6917, 1.7999,
0.4339, 1.3127, 1.9793, 1.2312, -0.2442, 2.5434, 0.1964, 0.1350,
-1.6063, 0.6460, 0.2100, 0.4153, -0.0708, -0.2820, -0.3491, 0.3494,
-1.0825, 1.4995, 1.1523, 0.9549, 0.7601, 0.5949, 1.0576, 0.5871,
-1.3045, -0.5725, 0.5301, 0.5057, 0.8488, 0.5929, 1.0189, 0.0787,
0.9573, -0.7332, 0.9140, -0.4795, 0.2905, 0.9838, 0.3004, -0.2756]])
0: 词向量: tensor([[-1.0558, 0.8881, -0.9850, 0.8188, -0.7008, -0.0922, -0.2241, 0.4969,
-1.0668, 0.4412, 0.7892, 1.5448, 0.9501, 1.0494, 0.2578, 0.2669,
1.4082, 0.5511, 1.3676, -0.4447, -0.9770, -1.2668, 0.4750, -0.3019,
0.5630, -0.2293, 0.2125, 0.2311, 0.2486, -0.1571, -1.7642, -0.1463,
-3.0483, -2.3720, -1.3424, -0.1577, -0.0171, 0.8787, 1.5240, 0.4730,
-0.8506, -0.0183, 1.1694, 1.2187, 0.5417, 1.3164, 1.8620, 0.4319,
-0.2337, 0.3830, 1.0625, 0.9880, 0.5470, -0.3907, -1.1378, 0.1881,
0.8776, 0.2353, 0.3081, 1.1402, 1.0400, -0.4366, -0.9093, 0.3939,
-0.6285, 1.3242, 0.6147, 1.7434, 0.2010, 1.4402, 0.5377, -0.1547,
-0.5641, 0.8100, 1.5759, 0.3668, 2.9535, 1.2819, 0.5410, -0.3859,
0.0836, 0.3964, -0.1896, 0.6245, 1.9583, 0.2600, -2.2688, 0.7577,
1.0525, -2.5902, 1.4733, 0.5291, -1.9547, -0.6918, 1.7990, -0.7484,
1.2813, -0.3554, 1.5658, -0.5579, -1.4720, 0.1699, 0.3927, 0.9757,
0.5506, -1.0921, 0.6403, 0.3171, 1.4954, -0.6224, 0.2379, -1.1242,
0.2876, 0.7108, -0.7549, 0.5265, -0.1823, 1.0296, -0.0220, 0.9025,
-1.8766, -0.7134, -1.2855, 1.8341, 0.6500, -1.3963, -0.8693, 0.0550]])
61: 一步 词向量: tensor([[ 0.9854, -1.2690, 0.6755, 0.9791, -0.7692, -0.3396, 0.7444, -0.1841,
0.5338, -0.7630, 0.7404, -1.3930, -0.7798, -0.5989, -1.2273, 0.5384,
-2.8233, 1.9800, 1.8217, 0.6808, 0.9290, -0.9703, 0.0657, -1.1967,
-0.4422, 0.2280, 1.2710, -1.2026, 0.6665, 0.5191, -0.9456, 1.9234,
-2.4274, -0.2036, 0.0981, -0.0781, 1.3622, 1.3070, -0.5892, 1.3786,
-2.0082, -0.2287, -0.4984, 0.5516, -0.9018, -0.9672, 0.6945, -1.7003,
-0.0059, -0.7846, 1.2262, 0.8687, 1.8092, -0.9282, -0.4748, -1.0091,
2.1235, -0.4453, 1.4294, 0.0225, -0.5111, 2.0459, -1.4678, -1.2538,
0.3284, -1.7567, -1.9918, 1.4049, 0.4875, -0.7923, -0.4132, 0.2315,
0.7019, -0.6254, -0.0726, 1.5229, 0.1685, -0.8175, -0.3895, 1.4473,
-0.0698, 1.7641, 0.2572, -2.3368, -1.4207, 0.0671, 0.0591, -0.8647,
2.0336, -0.8095, -0.5938, 0.9260, 0.0963, -1.6361, -1.2757, -0.8129,
-0.5823, 0.1888, 1.2362, -1.2889, -0.1477, 0.2733, 0.3116, 0.4198,
-0.9686, 1.3732, 0.7068, -0.0575, -1.3484, 0.4544, -1.2748, -1.1468,
0.4440, 0.5673, 0.9180, 1.2850, 0.4027, 0.2936, 1.3631, -1.0128,
0.6255, 0.7762, 0.7725, 0.7121, 0.8912, 1.0394, 0.4000, 0.2730]])
62: 两步 词向量: tensor([[-1.4841, -0.4666, 0.7924, 0.1195, 0.7606, 1.0563, -0.8920, -1.2274,
-0.5929, 0.4674, -0.1980, 1.0637, -1.2775, -1.4201, 0.5583, -1.0308,
0.1844, 0.0704, -0.1867, 1.3427, 0.3063, 0.0943, 0.0689, 0.4293,
0.8627, -1.7344, -0.5952, 1.9933, -1.1489, 0.1377, -0.3827, 0.6335,
0.3720, -0.0605, 1.7543, 0.2534, -1.0263, -0.8452, 0.8373, -0.8766,
-1.9627, -1.3624, 0.2630, -0.2924, -0.2172, -0.4633, 0.8443, -0.8749,
-0.7271, -0.4718, 1.0398, 0.1794, -2.0009, 1.1092, -1.0449, -0.6830,
-0.6373, 0.0326, -1.9863, 1.4530, -1.1634, -1.9569, 0.5194, 1.4223,
-0.5422, 0.5160, -0.2853, -0.7955, -2.3985, -0.0743, -1.1329, -0.2955,
0.3917, 2.6188, 0.0035, 0.5957, 0.3004, 0.1322, 0.1627, 0.7105,
-1.2862, -1.7003, -1.4462, -0.0824, 0.4808, 1.0575, -0.5548, -1.1678,
0.5736, -0.9303, 0.4457, -0.1778, -0.6941, 1.2100, 0.6334, -1.8079,
0.2552, -0.3786, -1.1633, 1.5506, 0.8175, 0.6522, 0.4500, 0.3415,
0.0154, -0.7068, -0.1257, 0.0550, 0.9416, -0.7952, -0.8868, -1.1407,
-0.9140, -1.6486, 0.7874, 0.5624, -0.4908, -0.3209, -0.0323, -0.5650,
-0.9341, 0.5328, 0.0654, 1.6518, -1.4184, -0.7103, 0.8351, -0.1445]])
63: 三步 词向量: tensor([[ 1.8932e+00, 9.1970e-01, 2.7027e-01, -2.3908e+00, -6.1062e-01,
-6.3633e-01, 1.4612e+00, 2.3411e+00, 1.5788e+00, -3.1986e-01,
-1.2401e+00, 8.2538e-01, 2.3567e+00, 4.0510e-02, -1.2214e+00,
1.2295e-01, -5.2664e-01, 4.5525e-01, 5.6684e-01, 5.8943e-01,
-1.5617e+00, 5.9794e-01, 1.3509e+00, -9.8100e-01, 4.8033e-01,
4.5032e-01, -2.1695e-03, 3.1046e-01, 1.6488e+00, -7.9855e-01,
-4.8367e-01, 5.5191e-01, 3.0998e-01, 6.7324e-01, 1.9096e+00,
-1.0135e-01, -1.2919e+00, -1.4905e-01, -5.3759e-01, -2.9332e-02,
-1.8953e+00, -1.8282e-01, 1.4832e-01, -4.2072e-01, 3.1158e-01,
-1.0317e+00, -3.2119e-01, 5.5970e-01, -3.9222e-02, 3.1496e-01,
7.7765e-01, 3.2088e-03, -9.5037e-01, 1.6842e+00, -2.7472e-01,
8.9495e-01, -6.6780e-01, -2.1917e+00, -8.9231e-01, -1.2503e-01,
-7.6916e-01, 9.2977e-02, -1.0068e+00, 5.6095e-01, -4.3650e-01,
-4.2963e-01, 1.1767e+00, 1.5873e-01, 8.1270e-01, 2.4282e-01,
6.1882e-01, 1.9297e+00, 2.7768e-03, 1.3663e-01, -2.0949e+00,
1.8813e+00, 1.7197e+00, -7.9407e-02, 2.5143e+00, -2.0963e+00,
1.5809e+00, 2.2331e-01, -7.8078e-01, 1.3170e+00, 1.2812e-01,
-1.8651e+00, 1.4497e+00, -1.2340e+00, 1.7757e-01, 9.9390e-01,
-3.9311e-01, -1.6382e-03, 2.3383e-01, 4.5272e-01, 3.5482e-01,
-3.3604e-01, -9.3516e-01, -3.1347e-01, 1.3623e+00, -7.4361e-02,
7.4045e-01, 8.0236e-01, -3.2368e-01, 6.3069e-01, -1.3896e+00,
-5.8012e-02, 7.2827e-01, 4.7116e-01, 5.6173e-01, -6.7654e-01,
5.3626e-01, 9.4554e-01, -3.9603e-02, -1.4229e+00, -5.6620e-01,
4.4772e-01, 2.7705e-01, -5.4747e-01, 9.2418e-01, 6.8657e-01,
-6.2135e-02, 1.4296e-01, 3.5313e-01, 1.3227e+00, -8.6707e-01,
9.8802e-01, -1.8308e+00, -9.7145e-01]])
64: 四步 词向量: tensor([[-1.1519e-01, -9.8370e-01, 8.9683e-01, 7.8583e-02, 6.8171e-01,
1.5929e+00, 1.2414e+00, 4.4744e-02, -8.8165e-01, -3.1066e-01,
-3.2233e-02, 3.3470e-01, -1.4127e-01, -3.6761e-03, 1.9378e+00,
-2.0881e+00, -2.3337e+00, -1.6996e+00, 4.7872e-01, 4.3735e-01,
-1.9272e+00, 4.9519e-02, -3.8391e-01, 1.1140e+00, -1.0786e+00,
7.2027e-01, -1.1500e+00, 7.2772e-02, 1.8115e+00, -3.8328e-01,
-8.6005e-01, -1.2892e+00, 7.5508e-01, 1.4564e-02, 2.1858e+00,
-8.2394e-01, -2.5774e-01, -2.0778e+00, -6.1939e-01, 2.4040e+00,
-7.9077e-01, 1.9725e+00, -3.6691e-01, 1.9984e-01, 5.5778e-01,
1.1229e+00, -8.2752e-01, 1.0133e+00, -1.0626e+00, 1.3553e+00,
9.8108e-01, -2.1485e-01, -6.4929e-01, 1.1640e+00, 7.6984e-01,
-3.5187e-01, -1.6375e+00, 5.7573e-01, 5.0084e-02, -4.0885e-01,
-2.6644e-02, 7.2604e-01, -1.3315e-01, -3.9336e-01, 2.3292e+00,
-1.9080e+00, 1.5909e-01, 9.0571e-02, 1.3194e-04, -1.0185e+00,
1.9428e-01, -1.7433e+00, 9.8780e-01, 8.2421e-02, 1.1351e+00,
-1.3256e+00, 1.2815e+00, -1.3856e+00, 4.4290e-01, -6.5222e-01,
-8.4154e-01, 1.9129e+00, 7.2862e-01, 6.5466e-01, -3.5003e-01,
4.3143e-01, -9.8964e-01, -5.9240e-02, 6.0528e-01, -1.0780e+00,
6.0879e-01, 7.0929e-01, -8.5609e-01, 2.2993e+00, -3.1809e-01,
-1.4632e+00, 1.0808e+00, 9.9197e-01, 1.3919e+00, 4.9169e-01,
-7.9783e-01, -4.6758e-01, -1.3620e-01, 1.0581e+00, 7.7175e-01,
-7.6914e-01, 1.5600e+00, 1.8618e-01, 2.4299e+00, 1.4844e+00,
2.1944e-01, 2.0768e+00, 6.5039e-01, -1.4695e-01, -1.1825e+00,
2.3904e-01, 1.3650e-01, -1.4382e-01, 6.6381e-01, -1.4008e-01,
-1.8280e+00, 1.6798e+00, -2.8428e-01, 1.3919e-01, -1.0410e+00,
-3.9361e-01, 4.9195e-01, 8.6388e-01]])
65: 望 词向量: tensor([[ 0.1241, 0.3408, 0.3753, 0.2032, -0.2694, 0.3849, 0.1945, 2.5496,
-0.2753, 1.3256, -0.8730, 0.3806, 1.0433, -0.1123, -0.9665, -0.9737,
-1.5741, -0.9025, 0.6558, 0.7908, 0.2945, 1.1923, -0.7982, 1.5092,
-0.5291, -0.6104, 0.2368, -0.9181, 0.7606, -0.5059, 0.0167, 2.3759,
0.5298, -0.4443, -0.0848, 0.6960, 0.6826, 1.1171, 1.3161, -0.9244,
0.1064, 1.1438, -1.3754, -0.3363, -0.6080, 1.0188, -0.6544, 0.9740,
-0.4169, 0.5819, 1.0222, -1.3296, 0.6507, 0.7518, -0.9745, -1.5318,
1.8925, 1.1480, -0.6524, -1.0280, 0.6995, 2.5830, -1.0279, 0.0774,
0.2262, -0.0683, 2.0969, 0.5485, 0.9846, 0.3675, 0.6454, 0.1687,
0.9284, 0.2790, 1.5492, -0.5730, 0.5825, -2.0112, -0.6177, -0.4560,
-0.8300, -0.6018, -0.3029, 0.4510, 0.0798, 0.1019, 0.7190, 1.0135,
-1.3040, -1.4357, -0.3265, 0.9620, -0.2163, 1.9211, 0.6877, -0.4949,
-0.4345, -1.4830, 0.2423, 0.7857, -1.5926, 0.6691, -1.2324, 0.4993,
-1.0496, 0.3796, 0.7865, -1.4122, 0.5166, 0.1025, -0.0085, -1.4805,
1.0596, -0.8605, -1.0924, -0.4712, -0.2731, -2.2384, -0.0569, -0.6090,
0.8119, 0.4555, -1.2355, 0.4860, 1.3225, 0.0208, -0.9553, -0.4149]])
66: 著天 词向量: tensor([[-0.9880, 0.0335, 1.3868, -0.8592, -0.6532, -1.1912, -0.1666, -1.6974,
0.4223, 0.7032, -0.8819, 1.3296, -0.2070, -0.3874, 0.4051, -1.1049,
-0.4800, 2.1483, -0.3965, 1.6693, 0.4499, 0.4128, -0.8221, -0.9146,
0.5112, -0.2271, 0.8036, -0.8404, 0.4012, -0.2307, 0.7848, -1.3319,
0.3837, 0.4600, -1.6393, 2.0505, 0.1815, 0.5192, 0.6952, -0.8248,
0.4817, -1.3933, -0.9988, -0.2749, 0.2796, -1.4801, 1.0917, -1.7538,
-0.4039, -1.0303, 0.9663, 1.2007, 1.0668, -0.8318, -2.5792, 1.0778,
0.0976, 1.1202, -2.2210, 2.2378, -0.3914, -0.2341, -0.1081, -0.7162,
-2.4974, -0.4599, -1.9009, -0.4083, -1.3762, 0.7436, -1.1261, -0.9869,
0.2709, 0.7337, 1.0590, 0.3218, 0.7844, 0.3267, 1.7037, 1.0939,
1.0560, -1.4140, -0.9326, -0.2127, -0.8218, 0.5245, -2.0479, 0.9608,
0.0367, -1.5361, -0.4979, -1.0448, -0.5511, -0.0768, 0.6718, -1.8612,
-0.1531, 1.7146, -0.0993, -1.5642, 0.8094, -1.4131, -0.2840, -0.5167,
0.8219, -0.3373, 0.3086, 0.1353, 0.3373, -0.3680, 0.6317, -0.1202,
-0.4661, -0.4880, -0.5316, -0.5198, -0.5909, -0.6734, 2.1747, 1.5025,
0.0948, 0.7073, -0.1861, 1.2045, -1.4991, 0.0775, 2.3362, -1.1228]])
1:
词向量: tensor([[ 1.4771, 0.8061, -0.4299, -0.5632, -1.3281, -0.5809, -0.4281, 0.5693,
0.6510, -0.1457, -0.2895, -1.2239, -0.8608, -1.1156, 1.4625, 0.9993,
2.6551, -0.2841, -0.1222, -0.1759, 0.2747, 0.8773, -1.2195, 0.0723,
-1.1253, -0.3896, -0.5094, -0.3521, 0.9901, 1.1108, 1.1191, -1.1345,
-0.5516, -1.0788, 0.2044, 0.1993, 0.2574, -0.3992, 0.0040, 0.1144,
-0.7567, 1.0407, -0.4901, -0.9804, 0.4055, 0.4972, -0.4097, -0.2506,
-0.0374, -0.2787, 0.5761, -0.0540, -0.8492, -0.7583, 1.0121, -0.6011,
-1.4103, 0.6407, 0.3240, 0.5556, -1.6938, 0.2747, 0.7353, -2.0933,
-1.7379, -0.5252, 1.3509, -0.2661, -0.3593, -0.2447, -1.2453, 0.2860,
-1.1808, 1.6014, -1.1948, 1.3895, 0.4737, -1.4571, 1.1382, -0.8261,
0.5641, -2.5230, 0.2975, 1.0076, -2.1213, 0.0967, 0.6917, 1.7999,
0.4339, 1.3127, 1.9793, 1.2312, -0.2442, 2.5434, 0.1964, 0.1350,
-1.6063, 0.6460, 0.2100, 0.4153, -0.0708, -0.2820, -0.3491, 0.3494,
-1.0825, 1.4995, 1.1523, 0.9549, 0.7601, 0.5949, 1.0576, 0.5871,
-1.3045, -0.5725, 0.5301, 0.5057, 0.8488, 0.5929, 1.0189, 0.0787,
0.9573, -0.7332, 0.9140, -0.4795, 0.2905, 0.9838, 0.3004, -0.2756]])