LOADING

Home / Artificial Intelligence / Natural Language Processing

version: 2.0.0

Overview

Language enables you to analyze text to extract meta-data from content such as concepts, entities, keywords, categories, sentiment, emotion, relations, semantic roles, using natural language understanding and identify industry/domain specific entities and relations in unstructured text.


Beginner's Guide

Computers are great at working with structured data like spreadsheets and database tables but we humans usually communicate in words, not in tables. That’s unfortunate for computers. A lot of information in the world is unstructured — raw text in English or another human language. How can we get a computer to understand unstructured text and extract data from it?

NLP is short for natural language processing while NLU is the shorthand for natural language understanding. Similarly named, the concepts both deal with the relationship between natural language (as in, what we as humans speak, not what computers understand) and artificial intelligence.

They share a common goal of making sense of concepts represented in unstructured data, like language, as opposed to structured data like statistics, actions, etc. To that end, NLP and NLU are opposites of a lot of other data mining techniques but that’s where the comparisons stop: NLU and NLP aren’t the same thing.


Quickstart

Following : 5 services/APIs are provided by Language as part of Release 2.0.1.

  • POS Tagger : Part Of Speech tagging for a piece of text.

  • Sentiment Analysis (English) : Analyses the sentiment of the text and returns if it falls under positive, neutral or negative sentiment.

  • NER Tagger : Named Entity Recognition Tagging for a piece of text.

  • Dependency Parser : Dependency Parser represents the Grammatical Structure of the Sentence.

  • Tags : Tags Extractor for extracting keywords from a paragraph.


APIs

All URIs below are relative to https://kong.intellihub.ai

POS Tagger POST   /core/nlp/pos
Sentiment Analysis POST  /core/nlp/sentiment/
Named Entity Recognition POST  /core/nlp/ner
Dependency Parser POST  /core/nlp/dependency_parser/
Tags POST  /core/nlp/tags/

POS Tagger

Description

This API would enable you to do a part of speech tagging of the text.

URI

POST  /core/nlp/pos

Headers

api-key Your App's API Key

Attributes

text Text Sentence (String)
sources Algorithm to use - ibm_watson/spacy (String)

Request Example

{
  "text": "He did a terrific job at the construction site",
  "sources" : "spacy"
}

Response

{
    "spacy": {
        "result": {
            "The": "DT",
            "old": "JJ",
            "tired": "JJ",
            "man": "NN",
            "was": "VBD",
            "sitting": "VBG",
            "under": "IN",
            "a": "DT",
            "tree": "NN",
            "and": "CC",
            "patiently": "RB",
            "waiting": "VBG",
            "for": "IN",
            "his": "PRP$",
            "son": "NN",
            "to": "TO",
            "arrive": "VB"
        }
    }
}

Sentiment Analysis

Description

This API would enable you to do a sentiment analysis and classify if the text is positive, neutral or negative.

URI

POST  /core/nlp/sentiment/

Headers

api-key Your App's API Key

Attributes

text Text Sentence (String)
sources Algorithm to use - azure/ibm_watson/nltk_vader (String)

Request Example

{
    "text" : "I really like the new design of your website",
    "sources" : "nltk_vader"
}

Response

{
    "nltk_vader": {
        "emotion": "POSITIVE",
        "scores": {
            "compound": 0.4201,
            "negative": 0,
            "positive": 0.285,
            "neutral": 0.715
        }
    }
}

NER Tagger

Description

This API would enable you to do a named entity recognition tagging of the text. This API also finds nearest match for the person detected and gives information such as occupation, nationality of the person.

URI

POST  /core/nlp/ner

Headers

api-key Your App's API Key

Attributes

text Text Sentence (String)
sources Algorithm to use - azure/ibm_watson/spacy (String)

Request Example

{
    "text" : "Delhi has a population of 1.3 crore. Arvind Kejriwal is the Chief Minister of Delhi",
    "sources" : "spacy"
}

Response

{
    "spacy": {
        "result": {
            "Delhi": "GPE",
            "1.3": "CARDINAL",
            "Arvind Kejriwal": "PERSON"
        }
    }
}

Dependency Parser

Description

This API would enable you to do dependency-parser

URI

POST  /core/nlp/dependency_parser/

Headers

api-key Your App's API Key

Attributes

text Text Sentence (String)

Request Example

{
    "text" : "And now for something completely different."
}

Response

{
    "And": {
        "dep": "cc",
        "headText": "for",
        "headPOS": "ADP",
        "children": []
    },
    "now": {
        "dep": "advmod",
        "headText": "for",
        "headPOS": "ADP",
        "children": []
    },
    "for": {
        "dep": "ROOT",
        "headText": "for",
        "headPOS": "ADP",
        "children": [
            "And",
            "now",
            "something",
            "."
        ]
    },
    "something": {
        "dep": "pobj",
        "headText": "for",
        "headPOS": "ADP",
        "children": [
            "different"
        ]
    },
    "completely": {
        "dep": "advmod",
        "headText": "different",
        "headPOS": "ADJ",
        "children": []
    },
    "different": {
        "dep": "amod",
        "headText": "something",
        "headPOS": "PRON",
        "children": [
            "completely"
        ]
    },
    ".": {
        "dep": "punct",
        "headText": "for",
        "headPOS": "ADP",
        "children": []
    }
}

Tags Extractor

Description

This API would enable you get tags from a paragraph.

URI

POST  /core/nlp/tags/

Headers

api-key Your App's API Key

Attributes

text Text Sentence (String)
sources Algorithm to use - rake/azure (String)

Request Example

{
    "text": "Artificial intelligence is intelligence demonstrated by machines, unlike the natural intelligence displayed by humans and animals, which involves consciousness and emotionality.",
    "sources": "rake"
}

Response

{
    "rake": {
        "tags": [
            "artificial intelligence",
            "intelligence demonstrated",
            "machines",
            "unlike",
            "natural intelligence displayed",
            "humans",
            "animals",
            "involves consciousness",
            "emotionality"
        ]
    }
}

SDK

Installation

INTELLIHUB requires Python version greater than 3.5. One can install INTELLIHUB SDK using the following command:

pip install intellihub

Creating Client

Create INTELLIHUB client to perform a different task.

client = intellihub.IntellihubClient('Your API Key')

To use these services, one needs to register to intellihub.ai website and create a project. Copy your API key to use different APIs.

Sentiment Analysis

Detects positive or negative sentiment in text. To get the sentiment of the text, one needs to pass specific parameters.
Parameters for training a model are:

  • Text: 'Text Sentence'.

  • Sources: Algorithm to use - azure/ibm_watson/nltk_vader. Default is nltk_vader. 

sentiment_analysis_response = client.sentiment_analysis("I really like the new design of your website.","nltk_vader")
sentiment_analysis_response

POS Tagger

It identifies and marks a word in a text as corresponding to a particular part of speech, based on its definition and context. To get the POS of the text, one needs to pass specific parameters.
Parameters for training a model are:

  • Text: 'Text Sentence'.

  • Sources: Algorithm to use - ibm_watson/spacy. 

pos_tagging_response = client.pos_tagger("The old tired man was sitting under a tree and patiently waiting for his son to arrive.","spacy")
pos_tagging_response

NER Tagger

It identifies key information (entities) in text. Each token is given an appropriate tag such as Person, Location, Organisation, etc.To get the POS of the text, one needs to pass specific parameters.
Parameters for training a model are:

  • Text: 'Text Sentence'.

  • Sources: Algorithm to use - azure/ibm_watson/spacy. 

ner_tagger_response = client.ner_tagger('Delhi has a population of 1.3 crore. Arvind Kejriwal is the Chief Minister of Delhi.',"spacy")
ner_tagger_response

Dependency Parser

It analyses the grammatical structure of a sentence, establishing relationships between “head” words and words which modify those heads.

dependency_parser_response = client.dependency_parser('And now for something completely different.')
dependency_parser_response

Tag Extraction

It identifies the important words in a sentence. To get the POS of the text, one needs to pass specific parameters.
Parameters for training a model are:

  • Text: 'Text Sentence'.

  • Sources: Algorithm to use - azure/rake. 

tagger_response = client.tags('Artificial intelligence is intelligence demonstrated by machines, unlike the natural intelligence displayed by humans and animals, which involves consciousness and emotionality.',"rake")
tagger_response

Annotations

NER Supports below entities

Type Description
PERSON People, including fictional.
NORP Nationalities or religious or political groups.
LANGUAGE Any named language.
QUANTITY Measurements, as of weight or distance.
ORDINAL “first”, “second”, etc.
CARDINAL Numerals that do not fall under another type.
MONEY Monetary values, including unit.
PERCENT Percentage, including ”%“.
TIME Times smaller than a day.
DATE Absolute or relative dates or periods.
EVENT Named hurricanes, battles, wars, sports events, etc.
WORK_OF_ART Titles of books, songs, etc.
LAW Named documents made into laws.
LOC Non-GPE locations, mountain ranges, bodies of water.
PRODUCT Objects, vehicles, foods, etc. (Not services.).
FAC Buildings, airports, highways, bridges, etc.
ORG Companies, agencies, institutions, etc.
GPE Countries, cities, states.

Annotations

POS TAGGER Supports below entities

POS Description
PUNCT Punctuations
CD Cardinal Number
SYM Symbol
DT determiner
NN, NNS Noun
NNP, NNPS proper noun
PRP pronoun
UH interjection
JJ, JJR, JJS, WDT Adjective, comparitive and superlative
ADP AdPosition like in, during
VB, VBG, VBN, VBP, VBZ Verb
RB, RBR, RBS Adverb
AUX auxiliary
CC conjunction
CCONJ coordinating conjunction
IN subordinating conjunction

Release Notes

Folowing are the release notes as part of Release 2.0.1

  • POS Tagger tags supported are DT, NN, IN, JJ, PRP, VBD etc.,

  • NER Tagger tags supported are PER, ORG, LOC, DATE, GPE, PRODUCT, LANGUAGE, TIME, MONEY, QUANTITY etc.,

  • Sentiment Analysis is supported only in English.

  • Tags Extractor works on paragraphs and minimum frequency of any word to detect as keyword must be atleast two.

  • The Info of the person detected in NER is the nearest match.