close

Uncategorized

Dev & DataUncategorized

What is an Artificial Neural Network?

Written by Savaram Ravindra, Content Lead at Mindmajix

Introduction

A neural network is an assembly of simple processing units, nodes, or elements that are interconnected and whose functionality is based on the biological neuron. The network’s processing capability stored in the strength of inter-unit connections (weights) obtained by learning (the process of adoption) from a set of training patterns. Neural network systems perform computational tasks that are much faster than conventional systems and this is their objective. The examples of computational tasks are text to voice translation, zip code recognition, function approximation, and so on. This article provides an in-depth explanation of artificial neural networks. 

Artificial Neural Networks and their Importance

An Artificial Neural Network (ANN) is defined as a model for processing information that is inspired by the structure and functions of a biological neural network. The information processing system’s novel structure is the key element of this model. It consists of a large number of neurons (interconnected processing elements) working simultaneously to solve specific problems. The ANNs learn by example just like humans. 

Artificial neural networks are largely used for data modeling and statistical analysis. The role of ANNs in these techniques is perceived as an alternative to standard cluster analysis or nonlinear regression techniques. Hence, they are generally used in problems that may be formulated in terms of forecasting or classification. A few examples include textual character recognition, speech and image recognition, and domains of human expertise like financial market indicator prediction, medical diagnosis, geological survey for oil, and so on.

How does an Artificial Neural Network Work?

A neuron is the neural network’s fundamental processing element and encompasses a few general capabilities. A biological neuron basically acquires inputs from various other sources, integrates them and carries out a non-linear operation on the result. Then, it outputs the final result.

nervous-system
A Simple Neuron
Image: https://www.nichd.nih.gov/health/topics/neuro/conditioninfo/parts

There are many variations within humans on this basic neuron type further complicating their attempts at replicating the thinking process electrically. Yet, the natural neurons basically have four components. They are synapses, axon, soma (cell body), and dendrites. To the cell body, the hair-like extensions are dendrites and they act as input channels. The dendrites obtain their input via other neurons’ synapses. Over time, the cell body then handles these incoming signals and processes them thereby converting the value that is processed into an output. The output is then sent to other neurons via the axon and then the synapses.

From the present-day experimental data, it is evident that biological neurons are much more complex than the simple explanation given above. They are more complex than today’s artificial neurons in artificial neural networks. As technology advances and as the biology offers a better understanding of neurons, the network designers can enhance their systems by building upon the man’s understanding of the biological brain. The goal of today’s ANNs is not the extravagant recreation of the biological brain. The neural network researchers, on the contrary, are seeking an understanding of the capabilities of nature for which people can develop solutions to problems that haven’t been solved by conventional computing. To accomplish this, the artificial neurons which are the basic unit of ANNs simulate the four basic functions of biological neurons. The below image shows a fundamental representation of an artificial neuron.

Working-Of-Artificial-Neuron
A Basic Artificial Neuron
Image: https://www.xenonstack.com/blog/artificial-neural-network-applications/

In the above figure, various inputs to the network are represented by x1, x2,..xn. These inputs are multiplied by weights of connection which are represented by w1, w2,…wn. These products are summed simply and fed via a transfer function (Threshold Unit) for result generation, and then output. This process contributes to a large scale physical implementation in a small package. This electronic implementation is still achievable with various other network structures as well which use different transfer and summing functions.

Few applications need binary or black and white answers. These applications include image deciphering of scenes, speech identification, and text recognition. For this kind of applications, the real-world inputs are turned into discrete values. The discrete values are limited to some known set such as the common 50,000 English words or the ASCII characters. These applications do not always use networks consisting of neurons that simply add up and smooth the inputs due to the limitation of output options. The binary properties of ANDing and ORing of inputs are used by these networks. These functions, as well as many others, can be integrated into the transfer and summation functions of a network.

The other networks work on problems in which the resolutions are not one of some known values. These types of networks must be capable of an unlimited number of responses. This type of application incorporates the intelligence behind robotic movements. The inputs are processed and the outputs which cause some device to move are created by this intelligence.

The movement of a device can span an unlimited number of precise motions. Indeed, these networks want to smooth their inputs which occur in interrupted bursts due to limitations of sensors (say 30 times a second for example). To accomplish that, they receive these inputs and sum the data thereby producing output by using a hyperbolic tangent as the transfer function. The network’s output values are uninterrupted and they satisfy the additional amount of real-world interfaces in this manner. Other applications may just add and compare to a threshold yielding one of two outputs that can be possible (a one or a zero).

Architectures of ANNs

The artificial neurons are arranged in a series of layers in an artificial neural network. Basically, an artificial neural network consists of an output layer, a hidden layer, and an input layer. The below figure shows the architecture of an ANN.

artificial_neural_network_1-768x768
The Architecture of an Artificial Neural Network Image: https://www.digitaltrends.com/cool-tech/what-is-an-artificial-neural-network/

The input layer contains those artificial neurons that obtain input from the outside world upon which the network will process, recognize, or learn. The output layer consists of units that respond to the information regarding how it has learned any task. These units lie in between the input and output layers. The hidden layer alters the input into something that the output unit can utilize in some way.

Basically, there are four types of neural network architectures. They are single-layer feedforward architecture, multi-layer feedforward architecture, recurrent or feedback architecture, and mesh architecture. Single-Layer Feedforward networks consist of one input layer and one output layer (neural layer). The number of outputs will always coincide with the neuron number in networks that belong to this architecture. The applications include pattern classification and linear filtering. The networks with multi-layer feedforward architecture consist of one or more hidden neural layers. The applications include pattern classification, system identification, and so on.

In the networks with feedback architecture, the outputs of neurons are utilized as feedback inputs for other neurons in these networks. They are used for process control and other time-variant systems. The main features of a network with mesh structures reside in considering the neurons’ spatial arrangement for pattern extraction. This means the neurons’ spatial localization is directly related to the procedure of adjusting their synaptic thresholds and weights. These are used in problems such as data clustering, system optimization, and so on.

Few popular neural network architectures include multilayer perceptron, radial basis function network, perceptron, LTSM, and recurrent neural networks. A perceptron is also known as the single-layer perceptron. It is a neural network that contains one output unit and two input units with zero hidden layers. The radial basis function network is similar to the feedforward neural network. The only difference is that the radial basis function is used as an activation function of these neurons. Different from a single layer perceptron, the multilayer perceptron uses beyond one hidden layer of neurons. The other name for this network is the deep feedforward neural network. The hidden layer neurons are equipped with self-connections in a recurrent neural network. These networks own a memory. In LTSM (Long/Short Term Memory Network), the memory cell is integrated into hidden layer neurons.

The artificial neural network’s architecture defines how its neurons are placed or arranged in relation to each other. By directing the neurons’ synaptic connections, these arrangements are structured essentially. Within a particular architecture, the topology of a given neural network is defined as distinct structural compositions it can assume. Most of the neural networks are fully-connected structures. This implies that each of the hidden neurons is connected entirely to each neuron in the preceding layer(input) and the subsequent (output) layer.

Training Processes

In order to produce the desired output, the neural network learns by adjusting its bias and weights iteratively. The weights and thresholds are also known as free parameters. The neural network is trained first for learning to take place. The defined set of rules with which training is performed is known as the learning algorithm. The network, during its execution, will thus be able to extricate discriminant features about the system that is being mapped from samples obtained from the system. There are five types of learning in a neural network. They are online learning, offline learning, reinforcement learning, unsupervised learning, and supervised learning.

In supervised learning, the training data is the input to the network, and the expected output is known weights are adjusted until the output produces the desired value. In unsupervised learning, the network with a known output is trained with the help of input data. The network categorizes the data at the input and it regulates the weight through the extraction of features in input data. In reinforcement learning, the network gives feedback whether it is a right or wrong output though the output value is not known. This is semi-supervised learning.

In online learning, the adjustment of the threshold and weight vector is carried out only after each training sample is presented to the network. In offline learning, the adjustment of the threshold and weight vector is carried out only after the entire training set is presented to the network. This is also known as batch learning.

Learning Datasets

The learning datasets in an ANN include a test set, validation set, and a training set. A test set is a set of examples utilized to assess the fully specified network’s performance or to successfully apply in predicting the output with a known input. A training set is a set of examples utilized for learning (to fit the network parameters). During the ANNs’ training process, the entire presentation of all the samples that belong to the training set, to adjust the synaptic thresholds and weights, is known as training epoch. A validation set is a set of examples utilized to tune the network parameters.

Major learning Algorithms

The learning algorithms used in a neural network include backpropagation and gradient descent algorithms. They are a set of steps applied for adjusting the thresholds and weights of its neurons. A learning algorithm tunes the network so that its outputs are very close to the desired output values. Backpropagation is a prolongation of the delta learning rule based on gradient. After finding the variance between the target and desired (error), the error is transmitted back towards the input layer from the output layer through the hidden layer. Backpropagation is used for a multilayer neural network.

Gradient Descent is the simplest training algorithm employed in the case of the supervised training model. The error or difference is found out in case the actual output differs from the target output. The gradient descent algorithm transforms the weights of the network such that the error gets minimized. The other learning algorithms include competitive learning, Least Mean Square (LMS) algorithm, Hopfield law, Self-Organizing Kohonen rule, and Hebb rule.

Applications of ANNs

Artificial neural networks are commonly used for clustering, prediction, association, and classification. ANN can be used to determine a particular feature of data and designate them into various categories without any previous knowledge of data. ANNs are trained to produce outputs that can be expected from a given input (stock market prediction for example). ANNs can be trained to classify a given data set or pattern into a predefined class. ANNs can be trained to remember a specific pattern so that when the network is presented with a noise pattern, it associates the noise pattern with the closest pattern in the memory or discard it.

ANNs are being applied in many industries including medicine, business, mineral potential mapping, cooperative distributed environments, image processing, geotechnical problems, nanotechnology, aquatic ecology, analysis of thermal transient processes, and so on. In business, ANNs are used in the areas of credit evaluation and marketing. In medicine, they are used for diagnosing and modeling the cardiovascular system, the implementation of electronic noses, and so on.

Conclusion

This article has given you an in-depth overview of the artificial neural networks. It has started with explaining the neural networks and then moved on to ANNs and the reasons for using them. Later, we dealt with the architecture, training process, and then the applications. Thus, this article gives you a comprehensive overview of ANNs. To gain mastery in this subject and obtain a job in this area, it is recommended that you opt for machine learning training. Please let me know your thoughts by commenting in the comments section.

About the Author

Savaram-Ravindra-cropped

Savaram Ravindra is working as a Content Lead at Mindmajix. His passion lies in writing articles on different niches which include some of the most innovative and emerging software technologies, digital marketing, businesses, and so on. Follow him on LinkedIn and Twitter.

read more
innovation

Innovative Technology Needed to Improve Crisis Response

article by and images from Dr. Victor Soji Ladele

Humanitarian crises in general are affecting more people, for longer. One in every 70 people around the world is caught up in crisis and urgently needs humanitarian assistance. Food insecurity is rising, with the number of people experiencing crisis-level food insecurity or worse increasing from 80 million to 124 million people between 2015 to 2017. The number of forcibly displaced people rose from 59.5 million in 2014 to 68.5 million in 2017. Natural disasters and climate change continue to affect 350 million people on average each year and cause billions of dollars of damage. Pandemics — large disease outbreaks that affect several countries — are rising as a global threat and pose major health, social, and economic risks. (Source: United Nations Coordinated Support to People Affected by Disaster And Conflict)

terrain_sm
Approaching the border between Liberia and Guinea.

My recent experience on the frontlines of a large-scale Ebola outbreak and many years of health emergency response, confirm that health system weaknesses in many countries are a source of global health insecurity. In an article in the influential New England Journal of Medicine in 2015, Bill Gates said, “There is a significant chance that an epidemic of a substantially more infectious disease (than Ebola) will occur sometime in the next 20 years. In fact, of all the things that could kill more than 10 million people around the world, the most likely is an epidemic stemming from either natural causes or bioterrorism.” In this interconnected world, a threat in one place is a threat everywhere.

Humanitarian action is intended to save lives, alleviate suffering, and maintain human dignity during and after crises and disasters, but often fall short of its lofty ideals in practice. The way it works is that most of the estimated $27.3 billion allocated to international humanitarian response comes from governments, but action is via proxies, the “humanitarian actors” (2017 Global Humanitarian Assistance Report by Development Initiatives). Humanitarian actors come from a plethora of UN agencies, the International Federation of Red Cross and Red Crescent Societies, military branches, non-governmental organizations (NGOs), local institutions and donor agencies.

coordination_sm
Operations conference room in Liberia during the Ebola response.

In order for actions to be quick, agile, and impactful, coordination is paramount. Information management is the bedrock of coordination, and it is fair to say that this is one of the most important activities during a crisis response. However, in recent years, networks connecting humanitarians have expanded so quickly, that the volume of data flowing through these pathways — and the number of information sources — have become in and of itself a problem. Data comes fast and hard from many sources, and adoption of ICT applications to improve outcomes has been relatively slow. The innumerable NGOs that are working on international humanitarian issues cannot alone address needs of such magnitude and diversity. To tackle these complex problems requires deeper levels of collaboration between formal humanitarian organizations and tech communities like Data+Creativity.

inspection_sm
An Ebola treatment center closes after the last patient is discharged.

Taking the relatively recent Ebola pandemic of 2014-2016 as a case study, nothing prepared the health emergency responders, myself included, for the difficulty of containing the outbreak.  With its many intense “waves of transmission”, dealing with the deadly pandemic challenged assumptions like never before, but at the end established a new nexus of cooperation among traditionally dissimilar groups like – community chiefs, epidemiologists, pastors, imams, hobbyists, doctors, financiers, anthropologists, logisticians, politicians, computer scientists, and technologists.

map_smBetween its start in 2013 in Guinea’s dense forest, and end in 2016, the Ebola pandemic, which originated in a remote village, spread south to Conakry, Freetown and Monrovia, east to Lagos, north to Bamako, northwest to Dakar, and by jet to the United States, Spain, the U.K. and Italy. The virus killed at least 11,315 people in seven countries and caused more than 28,600 known infections. Beyond the immediate horror and loss of life, the usual routines of daily life in the most affected countries came to a halt: population movement was restricted, harvests interrupted, markets closed, and volume of trade contracted. Reduced commercial activity in the surrounding region reversed recent economic gains. An estimated $2.2 billion was lost just in 2015 from the gross domestic product (GDP) of the three most affected countries. This regional economic decline in turn caused a widespread crisis of food security, affecting hundreds of thousands of people and turned into a separate humanitarian situation of itself.

mtg_sm
A meeting with several partners, including local health authorities in Lofa County.

Prior to Ebola, I had been active in medical humanitarian assistance and health emergency relief for many years. Having gained experience both in tiny, poor village health centers and at higher strategic levels with responsibility for broader policy decisions. I was already convinced of the effectiveness of tackling complex problems with complementary, cross-cultural and cross-disciplinary teams. The idea is gaining broad acceptance in the humanitarian and global development community, but this model of cooperation is yet lacking a purpose-built Information and Communication Technologies (ICT) tool. During the Ebola outbreak, I served as information management officer in addition to my role as team lead with the World Health Organization. Then and now, a framework for blending local knowledge and expert knowledge is absent. The lack of an effective platform to harness local expertise within the humanitarian affairs coordination framework has brought about a shocking amount of missed opportunities in humanitarian crisis response.

Despite the sorrow and devastation, as Bill Gates has noted, “perhaps the only good news from the tragic epidemic (Ebola) is that it may serve as a wake-up call. We must prepare for future epidemics of diseases that may spread more effectively than Ebola.” The world is at greater risk than ever from global health threats. We may not know what the next epidemic will be, but we know that one is coming. Disease threats can spread faster and more unpredictably than ever before. People are traveling more, food and medical product supply chains stretch across the globe, and biological threats as well as drug-resistant illnesses pose a growing danger to people everywhere.

During crises, professionals tend to avoid novel approaches that have not yet been tried and tested. They instead reach for familiar and trusted ways. As a result, humanitarian relief operations often deploy older technologies. Due to poorly adapted tools, training, and strategies, responders are increasingly ill-prepared to produce useful knowledge from the flow of information and data. There is thus an urgent need for innovative groups to engage early with humanitarian organizations, explore joint projects, and strengthen relationships before crises occur. These sorts of engagement will help both sides better understand each other’s modus operandi. As collaborations start to yield fruit, solutions will be developed and deployed to make all of us safer.

celebration_sm
The first celebration of the end of the outbreak. Four days later there was a new case.

For startups and midsize companies, this presents an incredible opportunity and the timing is right. The timing is right because the biggest funding agencies are currently enamored with the idea of private sector engagement (PSE) as a strategic approach to international development and humanitarian crises response (USAID). Furthermore private foundations, traditionally a good source of funding for innovation, are getting more assertive and acting with more ambition. This is a good time for data practitioners, software developers, researchers, creative technologists and thinkers  to build relevant partnerships with humanitarian and international development organizations, who on their part have become more supportive of broader engagement to help forcibly displaced people, respond to natural disasters, and prevent pandemics.

The key to having impact quickly during complex emergencies is through cross-disciplinary collaboration. Funders realize this and are actively seeking for great ideas to get behind. To close with a quote from a popular author, Marianne Williamson: “Success means we go to sleep at night knowing that our talents and abilities were used in a way that served others.”

 
Dr. Victor Soji Ladele presented at the Data+Creativity Meetup in Oklahoma City, OK, on February 21, 2019. Watch the video archive here.

read more