Post for Stanford NLP Course

Reference Course Repo with the slides, and course info Deep Learning overview we can look at deep learning as an algorithm that writes algorithms, like a compiler in this case the source code would be the data: (examples/experiences) excutable code would be the deployable model Deep: Functions compositions $ fl f{l-1} …. f_1$ Learning: Loss, Back-propagation, and Gradient Descent $ L(\theta) \approx J(\theta)$ –> noisy estimate of the objective function due to mini-batching.

Course 1: Classification and vector Spaces Weak 4 Hashing We can use hashing to search for the K-nearest vectors, to heavily reduce the searching space Locality senstive hashing the idea is to put items that are close in the vector space, in the same hashing buckets we can create a set of planes and calculate the relative position of points compated to this plane and then we can calculate the hash value for this point accordingly

Lecture 12 Gausain process infinite dimentional gaussian distribution Lecture 16 Convolution NN a rule of thumb: to have many layers with smaller filters is better than having one big filter, as going deep captures better features and also uses fewer parameters Residual Networks even after using Relu, NN can still suffer from gradient vanishing the idea in to add skip connections so that we can create shorter paths Lecture 18 LSTM vs GRU vs Attention LSTM: 3 gates, one for the cell state, one for the input, one for the output GRU: only two states, one for output, and one for taking weighted probablitiy for the contribution of the input and the hidden state takes less parameters Attention: at every step of producing the output, create a new context vector that gives more attention to the importat input tokens for this output token Lecture 20 Autoencoder takes different input and generates the same output

AutoVC paper Demo Repo Speach Split Paper Demo repo AutoPST Paper Demo repo Sources IMPROVING ZERO-SHOT VOICE STYLE TRANSFER VIA DISENTANGLED REPRESENTATION LEARNING Voice Conversion Challenge 2020 results

Inheritance If you have inherited from parent class then you should call the parent class constructor if you overload it, or simply doesn’t overload it Ex: 1 2 3 4 5 6 7 8 9 class parent: def __init__: class child: def __init__: super().__init__() class child2: ## or simply don't override the constructor and use the parent one Multiple Inheritance when we inherit from two or more classes, whatever class we inherited first(typed first in the list), would be the one to have pariority