# What is linear separability in neural networks

## to directory mode

### Linear separability

The concept of linear separability can best be demonstrated with a simple example, the well-known XOR problem. Consider a single-stage perceptron network with one output neuron in level 1 and two neurons in level 0. The output of the neuron should be 0, if its binary inputs are the same () otherwise it should be 1. That means that it must apply:

For this is equivalent to the following inequality:

For a constant threshold value, there is a straight line in the plane formed by and (see (Fig. 1), left). With positive combinations of and, all points above this straight line represent for which the neuron fires. If it is negative, all points below the straight line are points for which the neuron fires. Note that this derivation generally applies to real activations, with binary activations only the corner points of the unit square marked with,, and are possible.

A neural network that wants to solve the XOR problem must assign the points and one class, the points and the class. It is obvious that this is not possible by shifting and rotating a single straight line that linearly separates the input space. So the following applies:

The sets and the XOR problem cannot be linearly separated, i.e. there is no combination of values of, and, for which is in for all points and at the same time for all points

For neurons that provide inputs to a neuron, the space of the inputs can be represented as a -dimensional cube (if the input is restricted to, otherwise it is the -dimensional space). The neuron separates this input space by a -dimensional hyperplane. For this is shown in (Fig. 1) on the right. In general:

A single-level perceptron (i.e. a perceptron with only one level of modifiable weights) can only classify linearly separable quantities, i.e. quantities that can be separated by a hyperplane.

For practical applications, the question arises how often real problems can be linearly separated. Since this depends on the problem and the coding chosen, there is no general answer to this question. However, we know or assume that many problems cannot be linearly separated. There is also a theoretical study by Widner, who examined the number of linearly separable functions among all possible binary functions of input neurons. He has found that their percentage decreases very quickly as it increases.

- Tab. 1
- Number of binary functions of inputs and number of linearly separable functions

n | Number of binary functions of n inputs | Number of functions that can be linearly separated from this |
---|---|---|

### Conclusion

The bottom line is that single-stage perceptrons are only suitable for very simple tasks with a small number of inputs per cell.

- What happens to unclaimed illegal drug money
- Did pirates really have plug legs
- What is so important about the Iroquois tribe
- What are the characteristics of the original honey
- Can I exchange my old currency now?
- How does the credit card system work
- Chester Bennington was an Illuminati
- Are optimists healthier than pessimists

- What is education and community development
- The attitude determines your success
- How is used in jQuery
- Why are there people at all
- What are oxidation numbers
- Should Pakistan declare India a terrorist state
- How is stoneware different from ceramics?
- Who is smarter Batman or Lex Luthor
- What are the effects of genomic imprinting
- Did pirates really have plug legs
- What are sales taxes in Fallout
- What is the worst parenting strategy
- Which curve should a tax incentive shift?
- Can you schedule a slot machine
- Is PUBG Mobile nice
- How was Muammar Gaddafi in real life
- How do I get good gimbal strokes
- What is Kim Taehyung's personality
- Why did Hitler have Jewish soldiers?
- Is Lalit Modi really a culprit
- Can dogs learn to like other dogs?
- Which curve should a tax incentive shift?
- Why did Twitter acquire Crashlytics
- What is longshore drift