top of page
  • Adwith Malpe

Learning about Neural Networks

Updated: Jul 20, 2020

In this post, I will talk about a general introduction of what I have learned about neural networks. This is a high level post which means that I won't be going into any of the fine details just yet. With me so far? Let's begin.

From what I have learned, a neural network is a very powerful learning algorithm. Its concept is inspired by how the brain works. These algorithms are designed to recognize patterns and apply these patterns in a decision making process. The neural network first takes in input, sensory data from any type of machine perception. This type of data include raw, labeled or clustered input data. The data is interpreted by the network until a pattern is identified and these patterns are identified as numerical information organized in vectors.

Simple neural networks tend to help us cluster data from datasets and classify based on pattern recognition, much like a human being. They can also extract data and designated features to use as input for other algorithms pertaining to reinforcement learning, regression, and classification.

Here is an example I learned from one of my classes about a single neural network:

This illustration displays a rectified linear unit (ReLU) relationship since prices can't be negative and the function begins at 0. In this example, the input (x) is the size of the house, the output (y) is the price of the house, and the hidden unit ("neuron") is responsible for implementing the function ReLU. This example conveys a linear regression problem as the price is denoted as a function of sized is a continuous output.

Here is an example of a multiple neural network that I learned in my class:

In this example, the price of a house depends on a series of factors (size, number of bedrooms, zip code, and wealth). The multiple neural network is responsible for taking in all these factors into account and then predict the price once a pattern has been recognized. The hidden units in this case will be automatically generated so the only thing that needs to be given is the inputs, which are denoted by the x-notation. You'll notice that every input layer feature is interconnected with every hidden layer.

There are many different types of neural networks. The ones that I have showed so far represent standard neural networks. Another type of neural network is a Convolutional Neural Network (also known as CNN) which are particularly effective with image recognition and classification. CNNs can be used for facial recognition and objects such as traffic signs (which plays a part in computer vision within autonomous vehicles). Another type of neural network is a Recurrent Neural Network (also known as RNN) which are neural networks that specialize in speech recognition and machine translation. RNNs are frequently used when the input/output is a sequence such as a sequence of words. For autonomous driving, a custom hybrid version of neural nets is used to achieve autonomy.

All these different types of neural nets fall under the Supervised Learning category. Supervised learning is an ML (machine learning) method of learning a function that maps an input to an output and usually infers functions from labeled training data sets. In supervised learning, we are given a data set and know what the correct output should be, meaning that we know there is a relationship between the input and output. Supervised learning can be categorized as "regression" and "classification" problems. With regression problems, we are predicting results within a continuous output, which means that we are mapping input variables to a continuous function. With classification problems, we are predicting results in a discrete output, which means we are mapping input variables into different categories.

Data can be described as structured data or unstructured data. Structured data is refined data organized into datasets that can be easily interpreted by the machine. Structured data has defined meaning and some examples are age and price. Unstructured datasets are datasets that are not filtered or organized in any sort of way and typically consists of erratic data such as audio, pixels, and text.

Deep Learning is becoming a hot topic today because it has been discovered that ML algorithms tend to have a better performance and work optimally with large data sets. Here is a graph that compares ML algorithms efficiency with standard, coded algorithms:

In this graph, it can be seen as the number of training examples (amount of data represented by the variable m) increases, the performance of neural networks increases, making standard algorithms more obsolete as they do not adjust to the size of the datasets. Increasing the size of the training set and the size of the neural network often improves the performance and leads to better results as the machine as more opportunities to learn. It is a lengthy process when training a neural network, however, faster computation helps to iterate and strengthen new algorithms.

This concludes my brief description about neural networks.

10 views1 comment

Recent Posts

See All


Post: Blog2_Post
bottom of page