Question

Natural language processing language (NLP), a subfield of artificial intelligence and computational linguistics, is an essential component of text mining. What is the definition of NLP?Find the challenges associated with implementing NLP?

Answer 1

Natural language processing (NLP) can be defined as the ability of a machine to recognize a statement made by someone, understand its meaning and take decisions accordingly. It should also be able to reply to the speaker in the language which he/she understands. Eg: Siri in apple devices uses NLP to synthesize the user commands.

Challenges associated with implementing NLP:

• Huge number of combinations possible while constructing a sentence, also known as combinatorial explosion. The english vocabulary has over 100,000 words. Suppose we need to form a sentence of 10 words. So this comes out to a total of 1 million (100,000^10) combination of sentences. Computing such huge number of combinations is not effectively possible.
• Each user can ask a particular question in a different way. Eg. Alice can ask a question like "What is the difference between android and ios", while Bob can ask like "Android vs ios". Both these statements have the same meaning. So, training our model for each and every question is difficult. For this reason Semantic analysis is used, which is used for extracting the meaning from the statement.
• Distinguish grammatically correct statements. For eg. "Peter is a smart boy" is correct but "Smart boy is Peter" is grammatically wrong. We as humans can differentiate between the two, but for the machine they are one and the same, as same words are used. An efficient machine should be able to distinguish between a grammatically correct and incorrect statements.
• Current NLP models use recurrent neural networks (RNN) for functioning. RNN is very efficient for processing small sentences, but it begins losing it's efficiency for larger sentences. This can be overcome by training the model for large and huge texts, but such texts are expensive and very difficult to obtain.
• Understanding emotions from a sentence and the tone is still a long way to go. Machine can not understand sarcasm.