A dense layer is one in which all of the neurones receives input from all the neurones of previous layer. The dense layer is the most common layer in a deep neural network.

Definitions

A dense neurone has a weight associated with each connection from the past layer and a bias associated with the current neurone.

Dense neurones can be defined using different operations

• Definition of a Single Dense Neurone
• Definition of a Layer of Dense Neurones

Initializing a Neurone

When initializing the neurones weights should be chosen randomly (these should usually be much smaller than the training data values) and the biases should be set to 0, but can be set to any number depending on the use case.