Andrej Karpathy, Director of AI at Tesla, wrote in this blog post about how he goes about debugging neural networks. Debugging, experimenting, tweaking your model is probably the biggest and most challenging aspect of training deep learning models.

Here are some tips from the master which everyone should try to follow!

I’m quoting straight from the blog.

PRINT THIS OUT AND USE IT!

Tips & tricks for this stage:

  • fix random seed. Always use a fixed random seed to guarantee that when you run the code twice you will get the same outcome. This removes a factor of variation and will help keep you sane.

I do almost all of these while debugging my models. Let me know if anyone wants me to expand on these points. Thanks, Andrej Karpathy for the wonderful points!