Project : Processing on GPU-FPGA (Image)(SICOM S9) - 5PMSPAM1

Goal(s)

This project presents some of the fundamentals of deep learning, with a particular focus on the parallelisation of large models on clustered servers using multiple GPUs.

There will be an introduction to parallelisation, in particular:
1. Data parallelisation
2. Model parallelization

Then we will pay special attention to data parallelisation and we will study two different modes:
1. Centralised data parallelisation
2. Decentralised data parallelization

We will have a look at the estimation of the bandwidth and the allocation of the right parameter server.

The final project will focus on building a parallelized model for image processing applications such as : Recognition, Classification, Tracking, etc.

Content(s)

This project consists of 8 sessions:

1. In the first four sessions, we will spend about 1 to 2 hours talking about some of the basics needed to understand the project and mainly about data parallelisation.

2. Then we will look at some tutorials and toy problems to get familiar with Pytroch, single GPU, multiple GPUs.

3. The last 3 sessions will be dedicated to building your model and paying special attention to performance analysis.

4. Finally, the last session is dedicated to the Soutenance.

Machine Learning Fundamentals