Project : Processing on GPU-FPGA (Image) - 5PMSPAM1

Goal(s)

This project presents some of the fundamentals of deep learning, with a particular focus on the parallelisation of large models on clustered servers using multiple GPUs.

There will be an introduction to parallelisation, in particular:
1. Data parallelisation
2. Model parallelization

Then we will pay special attention to data parallelisation and we will study two different modes:
1. Centralised data parallelisation
2. Decentralised data parallelization

We will have a look at the estimation of the bandwidth and the allocation of the right parameter server.

The final project will focus on building a parallelized model for image processing applications such as : Recognition, Classification, Tracking, Segmentation, etc.

In this project, students are going to use the GPU cluster Gricad:
https://gricad-doc.univ-grenoble-alpes.fr/hpc/connexion/#se-connecter-aux-bastions-et-clusters-sans-mot-de-passe

By that, students are well prepared to face how to deal with building a real project where resources are usually not available on a single computer but on HPC. In this aspect, students will learn how to connect and configure the server parameters, how to allocate the right resources (1, 2 or more GPUs) and they will have to estimate the proper time of their computation. Typically Gricad consist of OAR system that distribute the tasks of running jobs based on priorities that depend on allocated resources, running time, etc.

Content(s)

This project consists of 8 sessions:

1. In the first four sessions, we will spend about 1 to 2 hours talking about some of the basics needed to understand the project and mainly about data parallelisation.

2. Then we will look at some tutorials and toy problems to get familiar with Pytroch, single GPU, multiple GPUs.

3. The last 3 sessions will be dedicated to building your model and paying special attention to performance analysis.

4. Finally, the last session is dedicated to the Soutenance.

Machine Learning Fundamentals: Supervised, Semi-Supervised and Non-Supervised Based Model.
Image Processing and Computer Vision Fundamentals: Segmentation, Tracking, ...