활성화 함수 (Activation Function)

step function (계단 함수)

$$ h(x) = \begin{cases} 1 & (x > 0) \\ 0 & (x \leqq 0) \end{cases} $$

import numpy as np

def step_function(x):
    return np.array(x > 0, dtype=np.int)

계단 함수의 그래프

import matplotlib.pylab as plt

x = np.arange(-5.0, 5.0, 0.1)
y = step_function(x)
plt.plot(x, y)
plt.ylim(-0.1, 1.1)
plt.show()

sigmoid (시그모이드)

d

$$ h(x) = \frac{1}{1 + e^{-x}} $$

def sigmoid(x):
    return 1 / (1 + np.exp(-x))
x = np.arange(-5.0, 5.0, 0.1)
y = sigmoid(x)
plt.plot(x, y)
plt.ylim(-0.1, 1.1)
plt.show()

dd

ReLU (Rectified Linear Unit, 렐루)

$$ h(x) = \begin{cases} x & (x > 0) \\ 0 & (x \leqq 0) \end{cases} $$

def relu(x):
    return np.maximum(0, x)
x = np.arange(-5.0, 5.0, 0.1)
y = relu(x)
plt.plot(x, y)
plt.ylim(-1.0, 5.0)
plt.show()