Neural networks are computing systems with interconnected nodes that work much like neurons in the human brain. Using algorithms, they can recognize hidden patterns and correlations in raw data, cluster and classify it, and – over time – continuously learn and improve.
History of Neural Networks
The first neural network was conceived of by Warren McCulloch and Walter Pitts in 1943. They wrote a seminal paper on how neurons may work and modeled their ideas by creating a simple neural network using electrical circuits.
This breakthrough model paved the way for neural network research in two areas:
Biological processes in the brain.
The application of neural networks to artificial intelligence (AI).
AI research quickly accelerated, with Kunihiko Fukushima developing the first true, multilayered neural network in 1975.
The original goal of the neural network approach was to create a computational system that could solve problems like a human brain. However, over time, researchers shifted their focus to using neural networks to match specific tasks, leading to deviations from a strictly biological approach. Since then, neural networks have supported diverse tasks, including computer vision, speech recognition, machine translation, social network filtering, playing board and video games, and medical diagnosis.
As structured and unstructured data sizes increased to big data levels, people developed deep learning systems, which are essentially neural networks with many layers. Deep learning enables the capture and mining of more and bigger data, including unstructured data.
Why are neural networks important?
Neural networks are also ideally suited to help people solve complex problems in real-life situations. They can learn and model the relationships between inputs and outputs that are nonlinear and complex; make generalizations and inferences; reveal hidden relationships, patterns and predictions; and model highly volatile data (such as financial time series data) and variances needed to predict rare events (such as fraud detection). As a result, neural networks can improve decision processes in areas such as:
- Credit card and Medicare fraud detection.
- Optimization of logistics for transportation networks.
- Character and voice recognition, also known as natural language processing.
- Medical and disease diagnosis.
- Targeted marketing.
- Financial predictions for stock prices, currency, options, futures, bankruptcy and bond ratings.
- Robotic control systems.
- Electrical load and energy demand forecasting.
- Process and quality control.
- Chemical compound identification.
- Ecosystem evaluation.
- Computer vision to interpret raw photos and videos (for example, in medical imaging and robotics and facial recognition).
Our first goal for these neural networks, or models, is to achieve human-level accuracy. Until you get to that level, you always know you can do better. Ivan Gomez Data Scientist and Consultant Zencos
Types of Neural Networks
There are different kinds of deep neural networks – and each has advantages and disadvantages, depending upon the use. Examples include:
- Convolutional neural networks (CNNs) contain five types of layers: input, convolution, pooling, fully connected and output. Each layer has a specific purpose, like summarizing, connecting or activating. Convolutional neural networks have popularized image classification and object detection. However, CNNs have also been applied to other areas, such as natural language processing and forecasting.
- Recurrent neural networks (RNNs) use sequential information such as time-stamped data from a sensor device or a spoken sentence, composed of a sequence of terms. Unlike traditional neural networks, all inputs to a recurrent neural network are not independent of each other, and the output for each element depends on the computations of its preceding elements. RNNs are used in forecasting and time series applications, sentiment analysis and other text applications.
- Feedforward neural networks, in which each perceptron in one layer is connected to every perceptron from the next layer. Information is fed forward from one layer to the next in the forward direction only. There are no feedback loops.
- Autoencoder neural networks are used to create abstractions called encoders, created from a given set of inputs. Although similar to more traditional neural networks, autoencoders seek to model the inputs themselves, and therefore the method is considered unsupervised. The premise of autoencoders is to desensitize the irrelevant and sensitize the relevant. As layers are added, further abstractions are formulated at higher layers (layers closest to the point at which a decoder layer is introduced). These abstractions can then be used by linear or nonlinear classifiers.
Neural Networks in Today’s World
Neural networks are changing how people and organizations interact with systems, solve problems, and make better decisions and predictions. Learn more about the impact of neural networks.
Neural networks for herd health
A passionate data scientist uses neural networks to detect tuberculosis in elephants. Find out how her research can help prevent the spread of tuberculosis.
Learn neural network modeling
Learn how to choose an appropriate neural network architecture, how to determine the relevant training method, how to implement neural network models in a distributed computing environment and how to construct custom neural networks using the NEURAL procedure.
How to do deep learning with SAS®
Need a more technical overview of deep learning techniques and applications? Read this paper and find out how SAS supports the creation of deep neural network models.
Who uses neural networks?
Deep learning systems – and thus the neural networks that enable them – are used strategically in many industries and lines of business.
Life Sciences
Health and life sciences organizations use neural networks to enable predictive diagnostics, biomedical imaging and health monitoring.
Manufacturing
Energy and manufacturing companies use neural networks to optimize supply chains, automate defect detection and forecast energy needs.
Banking
Banks use neural networks to detect fraud, conduct credit analysis and automate financial adviser services.
Public Sector
Public sector organizations use neural networks to support smart cities, security intelligence and facial recognition.
Retail & Consumer Goods
The retail and consumer goods industries use neural networks to power conversational chatbots, enhance and deepen customer intelligence, and perform network analysis.
Learn more about industries using this technology
Neural networks have the ability to identify anomalies. In the future, we can use them to give doctors a second opinion – for example, if something is cancer, or what some unknown problem is. And we’ll be able to provide these second opinions faster and with more accuracy. Leigh Ann Herhold Data Scientist and Consultant Zencos
Building a Neural Network Model
In this video, you learn how to use SAS for Machine Learning and Deep Learning and Machine Learning in the context of neural networks. This example examines the drivers of website visitors and what causes them to download a paper from an IT company’s site.
How Neural Networks Work
A simple neural network includes an input layer, an output (or target) layer and, in between, a hidden layer. The layers are connected via nodes, and these connections form a “network” – the neural network – of interconnected nodes.
A node is patterned after a neuron in a human brain. Similar in behavior to neurons, nodes are activated when there is sufficient stimuli or input. This activation spreads throughout the network, creating a response to the stimuli (output). The connections between these artificial neurons act as simple synapses, enabling signals to be transmitted from one to another. Signals across layers as they travel from the first input to the last output layer – and get processed along the way.
When posed with a request or problem to solve, the neurons run mathematical calculations to figure out if there’s enough information to pass on the information to the next neuron. Put more simply, they read all the data and figure out where the strongest relationships exist. In the simplest type of network, data inputs received are added up, and if the sum is more than a certain threshold value, the neuron “fires” and activates the neurons it’s connected to.
As the number of hidden layers within a neural network increases, deep neural networks are formed. Deep learning architectures take simple neural networks to the next level. Using these layers, data scientists can build their own deep learning networks that enable machine learning, which can train a computer to accurately emulate human tasks, such as recognizing speech, identifying images or making predictions. Equally important, the computer can learn on its own by recognizing patterns in many layers of processing.
So let’s put this definition into action. Data is fed into a neural network through the input layer, which communicates to hidden layers. Processing takes place in the hidden layers through a system of weighted connections. Nodes in the hidden layer then combine data from the input layer with a set of coefficients and assigns appropriate weights to inputs. These input-weight products are then summed up. The sum is passed through a node’s activation function, which determines the extent that a signal must progress further through the network to affect the final output. Finally, the hidden layers link to the output layer – where the outputs are retrieved.
Featured capability for neural networks
SAS® for Machine Learning and Deep Learning
Support the end-to-end data mining and machine-learning process with a comprehensive, visual (and programming) interface that handles all tasks in the analytical life cycle.
Recommended Reading
-
Article
What is synthetic data? And how can you use it to fuel AI breakthroughs?There's no shortage of data in today's world, but it can be difficult, slow and costly to access sufficient high-quality data that’s suitable for training AI models. Learn why synthetic data is so vital for data-hungry AI initiatives, how businesses can use it to unlock growth, and how it can help address ethical challenges.
-
Article
Analytics: A must-have tool for leading the fight on prescription and illicit drug addictionStates and MFCUs now have the analytics tools they need to change the trajectory of the opioid crisis by analyzing data and predicting trouble spots – whether in patients, prescribers, distributors or manufacturers. The OIG Toolkit with free SAS® programming code makes that possible.
-
Article
AI in government: The path to adoption and deploymentThe government sector is lagging in AI adoption, but awareness of the importance of AI in the public sector is increasing. Our survey indicates that operational issues are requiring governments to turn their attention to AI projects as a way to address important public issues.