NLP guide towards neural networks

** Statistical Language models

Over the years language related task was used using different techniques. Statistical model was one of the famous model which was used by Google before transforming into different deep learning models. The LM (Language modelling) is based upon probability for next word based upon the given corpus.

differences source:scalr.ai

* * Neural Language models (NLM)

Recently with the better outcome the use of neural network in the development of ml model build has become a preferred choice. NLM is when neural network is being used in natural language processing.

Heuristics for NLM

Size matters. The best models were the largest models, specifically number of memory units.Regularization matters. Use of regularization like dropout on input connections improves results.CNN vs Embedding. Character-level Convolutional Neural Network (CNN) models can be used on the front-end instead of word embedding, achieving similar and sometimes better results.Ensembles matter. Combining the prediction from multiple models can offer large improvements in model performance. 

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store