Difference between revisions of "DPS921/PyTorch: Convolutional Neural Networks"

From CDOT Wiki
Jump to: navigation, search
(Implementation of a Neural Network)
Line 1: Line 1:
 
= Neural Networks Using Pytorch =
 
= Neural Networks Using Pytorch =
The basic idea was to create a convolutional neural network using the python machine learning Framework PyTorch. The actual code will  
+
The basic idea was to create a simplistic neural network using the python machine learning Framework PyTorch. The actual code will  
  
 
be written in Jupyter Lab both for demonstration and implementation purposes. Furthermore, using the the torchvision dataset, the goal
 
be written in Jupyter Lab both for demonstration and implementation purposes. Furthermore, using the the torchvision dataset, the goal
Line 132: Line 132:
  
 
  print(torch.argmax(net(X[0].view(-1, 784))[0]))
 
  print(torch.argmax(net(X[0].view(-1, 784))[0]))
 +
 +
= Implementation of a Convolutional Neural Network =
 +
 +
''' So essentially we have taken the linear neural network defined above and transformed it into a CNN by transforming
 +
''' our first two layers into convolutional layers. This was achieved by making use of the 'nn' module function called
 +
''' 'conv2d' and making use of 2-d max pooling activation function.
 +
 +
import torch.nn as nn
 +
 +
class Network(nn.Module):
 +
    def __init__(self):
 +
        super(Network, self).__init__()
 +
        self.conv1 = nn.Conv2d(in_channels=1, out_channels=6,kernel_size=5)
 +
        self.conv2 = nn.Conv2d(in_channels=6, out_channels=12,kernel_size=5)
 +
       
 +
        self.fc1 = nn.Linear(in_features=12*4*4, out_features=120)
 +
        self.fc2 = nn.Linear(in_features=120, out_features=60)
 +
        self.out = nn.Linear(in_features=60, out_features=10)
 +
       
 +
    def forward(self, t):
 +
        #implement the forward pass
 +
       
 +
        #(1) Hidden conv Layer
 +
        t = self.conv1(t)
 +
        t = relu(t)
 +
        t = F.max_pool2d(t, kernel_size=2,stride=2)
 +
       
 +
        #(2) Hidden conv Layer
 +
        t = self.conv2(t)
 +
        t = F.relu(t)
 +
        t = F.max_pool2d(t, kernel_size=2,stride=2)
 +
       
 +
        #(3) Hidden linear Layer
 +
        t = t.reshape(-1, 12 * 4 * 4)
 +
        t = self.fc1(t)
 +
        t = F.relu(t)
 +
       
 +
        #(4) Hidden linear Layer
 +
        t = self.fc2(t)
 +
        t = F.relu(t)
 +
        t = F.softmax(t, dim=1)
 +
       
 +
        return t
 +
         
 +
  
 
== Getting Started With Jupyter ==
 
== Getting Started With Jupyter ==

Revision as of 18:30, 29 November 2020

Neural Networks Using Pytorch

The basic idea was to create a simplistic neural network using the python machine learning Framework PyTorch. The actual code will

be written in Jupyter Lab both for demonstration and implementation purposes. Furthermore, using the the torchvision dataset, the goal

was to show the training of the neural network, and show the classification of several images which have a single digit from 0 - 9.

A successful execution will show the correct determination of what number resides in that specific image. As part our of research,

We will explain in detail how an actual convolution neural network works at a fundamental level. Will we will both take a graphical

and mathematical approach to explaining the different parts of the neural network and how it comes together as whole. Furthermore,

we will briefly explain how it relates to parallel computing and how parallel computing plays a significant role in driving the

implementation of the neural network.

Group Members

1. Shervin Tafreshipour

2. Parsa Jalilifar

3. Novell Rasam

Progress

Intro to Neural Networks

Artificial intelligence is an umbrella term with many levels. Machine learning is a subset of AI that focuses on these self-teaching algorithms. Deep learning is a further subset of machine learning where multi-layered artificial neural networks are employed to allow for more versatile, independent learning [1]. Machine learning algorithms that do not utilize deep learning would be less versatile in what they could learn and would need more hand-holding by programmers. A handy infographic is shown below:

Example.jpg


Implementation of a Neural Network

In order to implement a Neural Network using the PyTorch Framework and Jupyter Lab, there are some key steps that need to be followed:


1. Import the required modules to download the datasets required to train the neural network.

import torch 
import torchvision
from torchvision import transforms, datasets


2. Download the needed datasets from the MNIST database, partition them into feasible data batch sizes.

train = datasets.MNIST(, train = True, download = True, transform=transforms.Compose([transforms.ToTensor()]))
test = datasets.MNIST(, train = False, download = True, transform=transforms.Compose([transforms.ToTensor()]))
trainset = torch.utils.data.DataLoader(train, batch_size = 10, shuffle = True)
testset = torch.utils.data.DataLoader(test, batch_size = 10, shuffle = False)


3. Import the necessary modules to define the structure of the neural network

import torch.nn as nn 
import torch.nn.functional as F

4. Define the class structure of the NN, in the following case, we have defined 4 linear layers, that output to three functions that execute rectified linear processing as an activation method. Finally instantiate the NN class.

class Net(nn.Module):
   
   def __init__(self):
       super().__init__()
       self.fc1 = nn.Linear(784, 64)
       self.fc2 = nn.Linear(64, 64)
       self.fc3 = nn.Linear(64, 64)
       self.fc4 = nn.Linear(64, 10)
   
   def forward(self, x):
       x = F.relu(self.fc1(x))
       x = F.relu(self.fc2(x))
       x = F.relu(self.fc3(x))
       x = self.fc4(x)
       return F.log_softmax(x, dim = 1)
 
net = Net()

5. Test our neural network with some randomized values representing a 1 dimensional tensor of pixel values of 28 by 28 pixel image. print the output of the test and ensure that the resultant is a tensor with numerical 10 values.

X = torch.rand((28,28))
X = X.view(-1, 28*28)
output = net(X)
print(output)

6. Use an optimizer to train the neural network. By passing the 1-dimensional tensor of pixel maps of each image in the training data set to the optimizer, it allows the optimizer to update the weight value of each of the layers in our neural network.

import torch.optim as optim
optimizer = optim.Adam(net.parameters(), lr=0.001)
EPOCHS = 3
 for epoch in range(EPOCHS):
    for data in trainset:
        # data is a batch of featuresets and labels
        X, y = data
        net.zero_grad()
        output = net(X.view(-1,28*28))
        loss = F.nll_loss(output, y)
        loss.backward()
        optimizer.step()
    print(loss)

7. Iterate through the trainset data again to verify current accuracy rating of our neural network using the testset data set.

correct = 0 
total = 0 
with torch.no_grad():
    for data in testset:
        X, y = data
        output = net(X.view(-1,784))
        for idx,i in enumerate(output):
            if torch.argmax(i) == y[idx]:
                correct += 1
            total += 1
            
print("Accuracy: ", round(correct/total, 3))

8. View the data at index of 0 in the training data set as 28 x 28 Image, Making sure to import 'matplotlib'.

import matplotlib.pyplot as plt
plt.imshow(X[0].view(28,28))
plt.show()

9. Test the trained neural network to check whether the digit value shown in the image is in fact the number the neural network determined it to be.

print(torch.argmax(net(X[0].view(-1, 784))[0]))

Implementation of a Convolutional Neural Network

So essentially we have taken the linear neural network defined above and transformed it into a CNN by transforming our first two layers into convolutional layers. This was achieved by making use of the 'nn' module function called 'conv2d' and making use of 2-d max pooling activation function.

import torch.nn as nn
class Network(nn.Module):
   def __init__(self):
       super(Network, self).__init__()
       self.conv1 = nn.Conv2d(in_channels=1, out_channels=6,kernel_size=5)
       self.conv2 = nn.Conv2d(in_channels=6, out_channels=12,kernel_size=5)
       
       self.fc1 = nn.Linear(in_features=12*4*4, out_features=120)
       self.fc2 = nn.Linear(in_features=120, out_features=60)
       self.out = nn.Linear(in_features=60, out_features=10)
       
   def forward(self, t):
       #implement the forward pass
       
       #(1) Hidden conv Layer
       t = self.conv1(t)
       t = relu(t)
       t = F.max_pool2d(t, kernel_size=2,stride=2)
       
       #(2) Hidden conv Layer
       t = self.conv2(t)
       t = F.relu(t)
       t = F.max_pool2d(t, kernel_size=2,stride=2)
       
       #(3) Hidden linear Layer
       t = t.reshape(-1, 12 * 4 * 4)
       t = self.fc1(t)
       t = F.relu(t)
       
        #(4) Hidden linear Layer
       t = self.fc2(t)
       t = F.relu(t)
       t = F.softmax(t, dim=1)
       
       return t
          


Getting Started With Jupyter

What is Jupyter Notebook?

This is basically a way for us to run the code interactively within a web browser alongside some visualizations and some markdown text to explain the process of what is going on.


User Interaction Model

User Interaction Model.png


Installation methods


In order to do Jupyter code, the easiest way is using plugin in vs code to install anaconda but if you don’t like environment of vs code IDE to do coding, you can install anaconda directly on your pc. This gives you chance to code on browser or use suggested IDES by anaconda. Also, pip can be used which is a package-management system written in Python used to install and manage software packages.


Points:

  • It does not matter how many operations we have inside a cell it only returns the last one.

1+1
2+3
output:5
  • There are different modes such as command and edit. Command is used to add or removed cells and when we are in this mode the cell cover with the line of blue colour while we are in edit mode to do cell editing cell will cover with the line of green colour.
  • Numbers in '[ ]' shows the order of the execution
  • To go to the new line we simply can use Enter
  • To run the cell and to insert a new cell we just use Shift + Enter
  • running the cell is by using Ctrl + Enter

Some Sample Codes

1. Printing a value

msg = "Hello World!"
print(msg)

2. Finding type of a value

type(1.5)

3. Making an array

my_list = [1,2,3,4,5]
my_list

4. Size of an array

len(my_list)

5. Object

dictionary = {"name":"Matt","age":"21"}
print(dictionary['name'])

6. for loop

for counter in[1,2,3,4]:
    print(counter)

7. Drawing a graph

import matplotlib.pylab as plt
import numpy as np
x = [0,1,2]
y = [0,1,4]
# to increase the size of the figure we can use 
fig = plt.figure(figsize=[12,8])
axes = fig.add_subplot(111)  # we are working only with only one chart
# axes.plot(x, y, color="red", linestyle='dashed') # to give style
# axes.plot(x,y)
axes.plot(x, y, marker='o', markerfacecolor="blue", markersize=6)
plt.show()

Installing PyTorch

The following command line instructions are directed at installing the PyTorch framework for Ubuntu Linux, Mac OS, and Windows using the Python Package Manager.


1. Ubuntu Linux:

  pip install torchvision

2. Mac OS:

  pip install torch torchvision torchaudio


3. Windows:

  pip install torch===1.7.0 torchvision===0.8.1 torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html