Deep Learning (02) - 퍼셉트론

---

신경망

퍼셉트론(Perceptron)

• 퍼셉트론 알고리즘: 여러 개의 신호를 입력으로 받아 하나의 신호를 출력

• 활성화 함수(Ativation function): 신경망에서 만들어진 값을 적절한 출력값으로 변환해주는 함수

• 계단함수(step function): 단순히 0과 1을 출력하는 함수

  • 최족 결괏값은 사용자가 정해놓은 답과 비교하는데, 이때 알고리즘을 사용하여 최적값을 찾을 때까지 가중치를 업데이트하면서 학습을 반복

In [1]:

import tensorflow as tf

In [2]:

tf.random.set_seed(777)

In [3]:

import numpy as np
from keras.models import Sequential
from keras.layers import Dense # 완전 연결층
from keras.optimizers import SGD # 경사 하강법
from keras.losses import mse # mean squared error

OR 게이트

In [4]:

x = np.array([[0,0],[0,1],[1,0],[1,1]])
y = np.array([[0],[1],[1],[1]])

In [5]:

model = Sequential()

# Dense: 완전 연결층, 입력값을 전부 연결
model.add(Dense(1, input_shape=(2,), activation='linear')) # 입력값을 받는 layer의 수, 들어오는 형태(값이 (2,) 즉, [o, o] 형태), 활성화 함수

# 모델 준비
model.compile(optimizer=SGD(),
              loss=mse,
              metrics=['acc'])
#모델 학습
model.fit(x, y, epochs=500) # 1에폭:전체 데이터를 한번 학습

Out [5]:

Epoch 1/500
1/1 [==============================] - 1s 1s/step - loss: 1.4290 - acc: 0.5000
Epoch 2/500
1/1 [==============================] - 0s 15ms/step - loss: 1.3602 - acc: 0.5000
Epoch 3/500
1/1 [==============================] - 0s 27ms/step - loss: 1.2956 - acc: 0.5000
Epoch 4/500
1/1 [==============================] - 0s 17ms/step - loss: 1.2349 - acc: 0.5000
Epoch 5/500
1/1 [==============================] - 0s 21ms/step - loss: 1.1779 - acc: 0.5000

...

Epoch 496/500
1/1 [==============================] - 0s 9ms/step - loss: 0.0667 - acc: 1.0000
Epoch 497/500
1/1 [==============================] - 0s 28ms/step - loss: 0.0667 - acc: 1.0000
Epoch 498/500
1/1 [==============================] - 0s 11ms/step - loss: 0.0666 - acc: 1.0000
Epoch 499/500
1/1 [==============================] - 0s 16ms/step - loss: 0.0666 - acc: 1.0000
Epoch 500/500
1/1 [==============================] - 0s 42ms/step - loss: 0.0666 - acc: 1.0000

<keras.callbacks.History at 0x7f04796f62b0>

In [6]:

model.get_weights() # array(가중치, 바이어스)

Out [6]:

[array([[0.49431458],
        [0.37552658]], dtype=float32), array([0.32719323], dtype=float32)]

In [7]:

x1 = 1
x2 = 1

In [8]:

(x1*0.49431458) + (x2*0.37552658) + 0.32719323

Out [8]:

1.19703439

임곗값 0.5를 넘으면 1로 반환

AND 게이트

In [9]:

x = np.array([[0,0],[0,1],[1,0],[1,1]])
y = np.array([[0],[0],[0],[1]])

In [10]:

model = Sequential()

model.add(Dense(1, input_shape=(2,), activation='linear'))

model.compile(optimizer=SGD(),
              loss=mse,
              metrics=['acc'])

model.fit(x, y, epochs=500)

Out [10]:

Epoch 1/500
1/1 [==============================] - 1s 781ms/step - loss: 0.2004 - acc: 0.5000
Epoch 2/500
1/1 [==============================] - 0s 10ms/step - loss: 0.1989 - acc: 0.5000
Epoch 3/500
1/1 [==============================] - 0s 9ms/step - loss: 0.1973 - acc: 0.5000
Epoch 4/500
1/1 [==============================] - 0s 9ms/step - loss: 0.1958 - acc: 0.5000
Epoch 5/500
1/1 [==============================] - 0s 8ms/step - loss: 0.1943 - acc: 0.5000

...

Epoch 496/500
1/1 [==============================] - 0s 6ms/step - loss: 0.0647 - acc: 1.0000
Epoch 497/500
1/1 [==============================] - 0s 6ms/step - loss: 0.0647 - acc: 1.0000
Epoch 498/500
1/1 [==============================] - 0s 6ms/step - loss: 0.0647 - acc: 1.0000
Epoch 499/500
1/1 [==============================] - 0s 5ms/step - loss: 0.0646 - acc: 1.0000
Epoch 500/500
1/1 [==============================] - 0s 5ms/step - loss: 0.0646 - acc: 1.0000

<keras.callbacks.History at 0x7f0476510bb0>

In [11]:

model.get_weights() # array(가중치, 바이어스)

Out [11]:

[array([[0.41167912],
        [0.48379776]], dtype=float32), array([-0.18801045], dtype=float32)]

In [12]:

x1 = 0
x2 = 1

In [13]:

(x1*0.41167912) + (x2*0.48379776) - 0.18801045

Out [13]:

0.29578731

NAND 게이트

In [14]:

x = np.array([[0,0],[0,1],[1,0],[1,1]])
y = np.array([[1],[1],[1],[0]])

In [15]:

model = Sequential()

model.add(Dense(1, input_shape=(2,), activation='linear'))

model.compile(optimizer=SGD(),
              loss=mse,
              metrics=['acc'])

model.fit(x, y, epochs=500)

Out [15]:

Epoch 1/500
1/1 [==============================] - 1s 741ms/step - loss: 0.8985 - acc: 0.2500
Epoch 2/500
1/1 [==============================] - 0s 16ms/step - loss: 0.8895 - acc: 0.2500
Epoch 3/500
1/1 [==============================] - 0s 7ms/step - loss: 0.8807 - acc: 0.2500
Epoch 4/500
1/1 [==============================] - 0s 12ms/step - loss: 0.8722 - acc: 0.2500
Epoch 5/500
1/1 [==============================] - 0s 7ms/step - loss: 0.8639 - acc: 0.2500

...

Epoch 496/500
1/1 [==============================] - 0s 7ms/step - loss: 0.0859 - acc: 1.0000
Epoch 497/500
1/1 [==============================] - 0s 7ms/step - loss: 0.0857 - acc: 1.0000
Epoch 498/500
1/1 [==============================] - 0s 7ms/step - loss: 0.0856 - acc: 1.0000
Epoch 499/500
1/1 [==============================] - 0s 10ms/step - loss: 0.0854 - acc: 1.0000
Epoch 500/500
1/1 [==============================] - 0s 6ms/step - loss: 0.0853 - acc: 1.0000

<keras.callbacks.History at 0x7f0479617cd0>

In [16]:

model.get_weights() # array(가중치, 바이어스)

Out [16]:

[array([[-0.23936202],
        [-0.36854893]], dtype=float32), array([1.0174638], dtype=float32)]

In [17]:

x1 = 0
x2 = 1

In [18]:

(x1*-0.23936202) + (x2*-0.36854893) + 1.0174638

Out [18]:

0.64891487

XOR 게이트

In [19]:

x = np.array([[0,0],[0,1],[1,0],[1,1]])
y = np.array([[0],[1],[1],[0]])

In [20]:

model = Sequential()

model.add(Dense(1, input_shape=(2,), activation='linear'))

model.compile(optimizer=SGD(),
              loss=mse,
              metrics=['acc'])

model.fit(x, y, epochs=1000) 

Out [20]:

Epoch 1/1000
1/1 [==============================] - 1s 660ms/step - loss: 2.8644 - acc: 0.5000
Epoch 2/1000
1/1 [==============================] - 0s 9ms/step - loss: 2.7051 - acc: 0.5000
Epoch 3/1000
1/1 [==============================] - 0s 9ms/step - loss: 2.5558 - acc: 0.5000
Epoch 4/1000
1/1 [==============================] - 0s 8ms/step - loss: 2.4159 - acc: 0.5000
Epoch 5/1000
1/1 [==============================] - 0s 9ms/step - loss: 2.2847 - acc: 0.5000

...

Epoch 908/1000
1/1 [==============================] - 0s 15ms/step - loss: 0.2503 - acc: 0.7500
Epoch 909/1000
1/1 [==============================] - 0s 10ms/step - loss: 0.2503 - acc: 0.7500
Epoch 910/1000
1/1 [==============================] - 0s 10ms/step - loss: 0.2503 - acc: 0.7500
Epoch 911/1000
1/1 [==============================] - 0s 10ms/step - loss: 0.2503 - acc: 0.7500
Epoch 912/1000
1/1 [==============================] - ETA: 0s - loss: 0.2503 - acc: 0.7500

<keras.callbacks.History at 0x7f04795804f0>

• acc를 아무리 돌려도 올라가지 않음! -> XOR문제는 단층 퍼셉트론으로는 해결할 수 없다

다층 퍼셉트론(MLP; Multi-Layer Perceptrion)

: 단층 퍼셉트론의 XOR문제를 해결 할 수 있다

In [21]:

from keras.optimizers import RMSprop

In [22]:

x = np.array([[0,0],[0,1],[1,0],[1,1]])
y = np.array([[0],[1],[1],[0]])

In [23]:

model = Sequential()

model.add(Dense(32, input_shape=(2,), activation='relu'))
model.add(Dense(1, activation='linear')) # 다층 추가
model.compile(optimizer=SGD(),
              loss=mse,
              metrics=['acc'])

model.fit(x, y, epochs=500)

Out [23]:

Epoch 1/500
1/1 [==============================] - 0s 339ms/step - loss: 0.7334 - acc: 0.5000
Epoch 2/500
1/1 [==============================] - 0s 7ms/step - loss: 0.6828 - acc: 0.5000
Epoch 3/500
1/1 [==============================] - 0s 9ms/step - loss: 0.6382 - acc: 0.5000
Epoch 4/500
1/1 [==============================] - 0s 8ms/step - loss: 0.5992 - acc: 0.5000
Epoch 5/500
1/1 [==============================] - 0s 6ms/step - loss: 0.5654 - acc: 0.5000

...

Epoch 496/500
1/1 [==============================] - 0s 8ms/step - loss: 0.0454 - acc: 1.0000
Epoch 497/500
1/1 [==============================] - 0s 9ms/step - loss: 0.0452 - acc: 1.0000
Epoch 498/500
1/1 [==============================] - 0s 10ms/step - loss: 0.0450 - acc: 1.0000
Epoch 499/500
1/1 [==============================] - 0s 6ms/step - loss: 0.0448 - acc: 1.0000
Epoch 500/500
1/1 [==============================] - 0s 9ms/step - loss: 0.0446 - acc: 1.0000

<keras.callbacks.History at 0x7f04794c3490>

In [24]:

model.get_weights() # array(가중치, 바이어스)

Out [24]:

[array([[-0.3647865 ,  0.19419493, -0.27322584, -0.12304214,  0.00732531,
          0.4152011 , -0.36740124,  0.02312151, -0.27288848, -0.34067878,
          0.3978779 ,  0.33925995, -0.04183829,  0.12893987, -0.02815302,
          0.31668478, -0.33050963, -0.22045128,  0.34314525, -0.40096074,
          0.2279879 , -0.29325008,  0.08459529,  0.38154343,  0.31770322,
          0.26830775, -0.00500929, -0.32947505,  0.18048538, -0.09777051,
          0.3698536 ,  0.34828937],
        [-0.36095715,  0.35408652,  0.11284499, -0.16528037, -0.40171486,
         -0.07047239,  0.19572088, -0.39503545, -0.12045923, -0.37620205,
          0.4380986 , -0.33948484, -0.41078678, -0.20670712,  0.32728168,
         -0.31648418,  0.31005126,  0.3785067 ,  0.14227697,  0.39699733,
          0.40770897,  0.03798516, -0.37863094, -0.3952569 ,  0.26154408,
         -0.21042714, -0.15236348, -0.06894568, -0.4169179 ,  0.03011136,
          0.26063493,  0.11131967]], dtype=float32),
 array([ 0.0000000e+00, -1.9431205e-01, -1.1298203e-01,  0.0000000e+00,
        -7.4083009e-03,  7.1544960e-02, -1.6878155e-01, -2.3255464e-02,
         0.0000000e+00,  0.0000000e+00,  6.7960896e-02, -2.7892008e-04,
         0.0000000e+00,  6.1826955e-05, -2.6806872e-02, -1.6289880e-05,
         1.9801866e-02, -4.8135631e-03,  6.8675637e-02,  3.7444141e-03,
         5.0821796e-02, -2.3652159e-02, -8.5395813e-02,  1.1525599e-02,
        -2.6145738e-01, -3.7589641e-03,  0.0000000e+00,  0.0000000e+00,
         5.1375737e-05, -3.2379072e-02, -2.6025215e-01, -1.1119527e-01],
       dtype=float32),
 array([[-0.3106395 ],
        [-0.4172637 ],
        [-0.12232165],
        [-0.02042663],
        [-0.08811529],
        [-0.21569595],
        [-0.10583159],
        [-0.417019  ],
        [ 0.26359326],
        [ 0.09979928],
        [ 0.3503362 ],
        [ 0.2736206 ],
        [-0.2794919 ],
        [ 0.09167034],
        [-0.1355925 ],
        [ 0.51988244],
        [ 0.37897652],
        [ 0.12214111],
        [ 0.33911797],
        [ 0.37631428],
        [ 0.27579123],
        [-0.01018383],
        [-0.2801623 ],
        [ 0.491249  ],
        [-0.53207463],
        [ 0.05855707],
        [ 0.04499847],
        [-0.14873469],
        [ 0.14005944],
        [-0.3803438 ],
        [-0.5565973 ],
        [-0.18226637]], dtype=float32),
 array([0.18689457], dtype=float32)]

In [25]:

model.predict(x)

Out [25]:

1/1 [==============================] - 0s 264ms/step

array([[0.24716544],
       [0.7727164 ],
       [0.83058953],
       [0.18972254]], dtype=float32)

• XOR 문제가 해결되었다!

3blue 1brown 유툽

• 활성함수의 종류
  • 시그모이드 함수: 0 ~ 1사이값 리턴
  • 하이퍼볼릭 단젠트 함수: -1 ~ 1사이값 리턴
  • ReLU 함수: 0을 기준으로 -값은 0, +값은 그대로 리턴(양수값만 취하고 음수는 0으로)