从0到1：神经网络实现图像识别（上）

“神经网络”是“机器学习”的利器之一，常用算法在TensorFlow、MXNet计算框架上，有很好的支持。

By the study of systems such as the perceptron, it is hoped that those fundamental laws of organization which are common to all information handling systems, machines and men included,may eventually understood.

- Frank Rosenblatt@Connell Aeronautical Laboratory

$$f(x) = sign(\vec w\cdot \vec x + b)$$

sign(自变量)是符号函数，将自变量，进一步映射到yi的输出类别{+1,-1}上。

$$\vec w\cdot \vec x + b = 0$$

$$y_i=+1$$ 有 $$\vec w\cdot \vec x_i + b > 0$$

$$y_i=-1$$ 有  $$\vec w\cdot \vec x_i + b < 0$$

1、由于平面上任意两个不同实例点，$$\vec x_i$$和$$\vec {x_{i+n}}$$都满足

$$\vec w\cdot (\vec x_i-\vec x_{i+n}) = \vec w\cdot \overrightarrow {( x_{i+n}- x_i )}=0$$

2、而内积的几何意义，是一个向量在另一个向量方向上的投影长度，与另一个向量长度的乘积。

$$\displaystyle \min \limits_{f \in F} \frac{1}{N} \sum \limits_{i=1}^N L(y_i , f(x_i))$$

$$\displaystyle \frac {\vert {\vec w \cdot \vec x_i + b}\vert}{ \sqrt { \sum \limits_{j=1}^{D} w_j^2}}$$

$$- y_i ( w \cdot x_{err} + b ) >0$$

$$- \displaystyle \frac {\sum y_{err} (\vec w \cdot \vec x_{err} + b)}{ \sqrt { \sum \limits_{j=1}^{D} w_j^2}} = - \displaystyle \frac {\sum y_{err} (\vec w \cdot \vec x_{err} + b)}{ \Vert w \Vert}$$

$$L( \vec w , b ) =- \sum y_{err} (\vec w \cdot \vec x_{err} + b)$$

$$\min \limits_{f \in F} \frac{1}{N} \sum \limits_{i=1}^N L(y_i , f(x_i))$$

$$\min \limits_{\vec w ,b} L( \vec w , b ) = - \sum y_{err} (\vec w \cdot \vec x_{err} + b)$$

$$\displaystyle \nabla _w Loss = \frac { \partial [- \sum y_{err} ( \vec w \cdot \vec x_{err} + b)]}{ \partial \vec w} = - \sum y_{err}x_{err}$$

$$\displaystyle \nabla _b Loss = \frac { \partial [- \sum y_{err} ( \vec w \cdot \vec x_{err} + b)]}{ \partial b } = - \sum y_{err}$$

$$w = w + \eta \space x_{err} \space y_{err}$$

$$b = b + \eta \space y_{err}$$

(上篇完)

[1] 李航 .统计学习方法. 北京：清华大学出版社，2012

[2] Rosenblatt, F. The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review.1958,65(6), 386-408