You can easily boost the performance of a language model based on RNNs by adding a pointer cache on top of it. The idea was introduce by Grave et al. and their results showed how this simple technique can make your perplexity decrease by 10 points without additional training. This sounds exciting, so let's see what this is all about and implement that in pytorch with the fastai library.

Continue reading

In Natural Language Processing, traditional neural networks struggle to properly execute the task we give them. To predict the next work in a sentence for instance, or grasp its meaning to somehow classify it, you need to have a structure that can keeps some memory of the words it saw before. That's why Recurrent Neural Network have been designed to do, and we'll look into them in this article.

Continue reading