Introduction

cover

A Complete Introduction to ResNet

This is the final project of PHP 2650 Statistical Learning and Big Data. This project aims to provide a complete introduction to ResNet, a deep learning model that has been widely used in computer vision tasks. It was conducted by William Qian (opens in a new tab) and Dingxuan Zhang (opens in a new tab).

What is ResNet?

ResNet, short for Residual Network, is a deep learning model that was proposed by Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun in 2015. ResNet is a type of Convolutional Neural Network (CNN) that is widely used in computer vision tasks, such as image classification, object detection, and image segmentation.

Why we choose ResNet?

ResNet used to be a breakthrough in the field of deep learning. It solved the problem of accuracy degradation and verify the idea that the deeper network should have better performance. It was once widely used in various computer vision tasks and has been the basis of many other deep learning models. Although Transformer has become the most popular deep learning model in recent years, it is still necessary to understand the basic principles of ResNet.

Project Structure

As you can see in the navigation bar, this project is divided into several sections.

The first section is the Basic Theory section, in this section, we will provide basic theories that will be used in the ResNet, and we will also introduce the basic structure of ResNet in this section.

The second section is the Experiments section. In this section, we analyze the attributes of ResNet through experiments. Also, an implementation of ResNet using PyTorch will be provided in this section.

The third section is the Advanced Topics section. A practical application of ResNet and a variant of ResNet will be introduced in this section.

References

  1. K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 770-778, doi: 10.1109/CVPR.2016.90. (opens in a new tab)
  2. He, K., Zhang, X., Ren, S., Sun, J. (2016). Identity Mappings in Deep Residual Networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds) Computer Vision – ECCV 2016. ECCV 2016. Lecture Notes in Computer Science(), vol 9908. Springer, Cham. (opens in a new tab)
  3. https://www.geeksforgeeks.org/cnn-introduction-to-pooling-layer/ (opens in a new tab)
  4. https://d2l.ai/index.html (opens in a new tab)
  5. https://pytorch.org/docs/stable/index.html (opens in a new tab)
  6. https://nextra.site/ (opens in a new tab)
  7. https://www.cnblogs.com/shine-lee/p/12363488.html (opens in a new tab)
  8. Srivastava, R.K., Greff, K., Schmidhuber, J.: Highway networks. In: ICML Workshop (2015) (opens in a new tab)
  9. Srivastava,R.K.,Greff,K.,Schmidhuber,J.:Training very deep networks.In:NIPS (2015) (opens in a new tab)
Convolutional Layer