Abstract: (Deep) Neural networks seem to be achieving the targets drawn for Artificial Intelligence in the early 60's. We will present the practical accomplishments. The underlying mathematical models will be discussed: -Feed forward neural networks -Recurrent neural networks -Generative neural networks (Boltzmann Machines, Variational AutoEncoders, Generative Adversary Networks) . All these models have sometimes millions of parameters that are learned (estimated) from examples (training data). The learning procedure will be discussed. Training data is always a small fraction of real data where these models are applied. To avoid overfitting the model parameters to training data, and poor generalization to large real data, regularization techniques are used.