Natural Language Processing Components

Hello There,

In previous post I have talked about some basic about Natural language Processing and why it is difficult to understand for machines.

In this post I’m moving forward and will talk about some basic component of NLP. So here it is…

Components of NLP:

Mainly there are two component of NLP namely

     1.   Natural Language Understanding (NLU) and

     2.  Natural Language Generation (NLG)

So what are these actually, let’s take a look on that…

The first one is Natural language Understanding or we can say NLU, as name says “understanding” the main thing to do is to understand the input given by user as a part of natural language. It deals with machine reading comprehension à ability to read text, to do process on it, and understand its meaning. It is involved with mapping the given inputs (let’s take plain text as input) in useful representations and analyzing the different aspects of the language.

Now about second one Natural Language Generation (NLG). Main task of this component is to generate meaningful output or parser in form of natural language from some internal representing according to given input.

It includes :-

     1.  Text planning : - retrieving the relevant content from database. Here database can be includes vocabulary, sentences, knowledge, sample data and many more.

     2.  Sentence planning : - we get our content using text planning now next step to do is choosing required words and forming meaningful sentence setting the words in right grammatical way.

     3.  Text realization :- we have all the thing need to create actual text in humans language.

To understand all of these take example of Eliza. You may heard about it. It is the most popular AI bot of its time, developed at MIT in the mid-1960.

It’s not perfectly understand or not understand at all meaning of input sentence and it is neither pass the Turing test but it is still acknowledgeable for its behavior of giving replies. It is impressive.  

ELIZA

So how it works? It is very simple to understand. There is a database of words that are called keywords. For each keyword there is an integer value which shows the rank of keyword, a pattern to match against the input and a specification of the output. So when user type sentence it separate the words and look for keyword which is similar to database’s keyword (we can say that here program try to understand meaning of sentence. The first component NLU). If there is more than one keyword found it pick the one with highest rank value and according to that give output which is already In database (here it is actually generates the answer. The second component NLG). And if keyword not found in database it simply generate the statements like “tell me more” or “Go on” etc…

So that’s all for this post. See you guys in my next post…