Text Analytics & MeaningCloud 101

One of the questions we get most often at our helpdesk is how to apply the text analytics functionalities that MeaningCloud provides to specific scenarios.

Users know they want to incorporate text analytics into their processes but are not sure how to translate their business requirements into something they can integrate into their pipeline.

If you add the fact that each provider has a different name for the products they offer to carry out specific text analytics tasks, it becomes difficult not just to get started, but even to know exactly what you need for your scenario.

homer-simpson-confused

In this post, we are going to explain what our different products are used for, the NLP (Natural Language Processing) tasks they are tied to, the added value they provide, and the requirements they fulfill.

Topics Extraction

Topics Extraction is MeaningCloud’s product for “automatically extracting structured information from unstructured and/or semi-structured machine-readable documents” [1]. In other words, Topics Extraction extracts specific pieces of information from collections of text, anything from names of people to locations or amounts of money.

There are a number of ways to refer to this task some, such as Named Entity Recognition are derived from its most popular subtasks. However, the objective is the same: extracting structured information from text.

Let’s take a look at the following text, taken from an article in the New York Times:

It’s Official: Simone Biles Is the World’s Best Gymnast

RIO DE JANEIRO — Simone Biles, already considered the world’s greatest female gymnast before even competing in the Olympics, emphatically confirmed her standing on Thursday by winning the women’s individual all-around gold medal at the Rio Games.

Wearing a stars-and-stripes leotard, Biles, 19, joined Mary Lou Retton, Carly Patterson, Nastia Liukin and Gabby Douglas as American all-around winners.

The American Aly Raisman, 22, won the silver, and Aliya Mustafina, 21, of Russia won bronze.

Victory in this event brings lucrative endorsements and widespread adoration, a popularity bonanza fueled by a prime-time showcase of athletic artistry. At 4 feet 9 inches, with size 5 feet, Biles is someone that young viewers can relate to. Then she performs, and her abilities are unimaginable.

Her ascent has been sudden to those who follow gymnastics only ever four years. At the last Summer Games, in London in 2012, Douglas was the show-stopper. Biles arrived here from Texas and gave the Rio Games a performance for the ages. Whether you know an Amanar from an aardvark, you watch her not because the result is in doubt but rather to witness something without equal.

So, how does this text look when we extract its information using our Topics Extraction API?
demo_meaningcloud_infoextraction

At first, it may seem that it’s just a matter of finding the names that appear in the text, but there’s a little more to it. There are many ways to refer to the same person, nicknames and variants of their name that you need to take into account. For instance, in this text the concept of “Simone Biles” appears five times, two with her full name, and three only using her last name.

But names or named entities, as they are often called are not the only thing we may want to extract. In the text we can also see quantities, dates, and keywords. Depending on the scenario you are working on, you will need to extract different types of structured information.

Sometimes, identifying all the named entities in a text is more than enough. For those instances, entities have a type associated, so you can choose to extract only locations, people, organizations, etc. You can check all the different types we detect in our ontology.

demo_meaningcloud_entitytypes

You can also define your own entries with their corresponding types through our customization engine. By using user dictionaries you will be able to extract the entities/concepts specific to your domain using your own ontology.

These are some scenarios in which Topics Extraction can be applied:

  • Automatic tag suggestions for news articles or blog posts and semantic publishing
  • Popularity analysis according to mentions
  • Key data entity extraction

Text Classification

Text Classification is MeaningCloud’s product for document categorization or document classification, which is the task of “assigning a document to one or more classes or categories” [2]. In this case, instead of extracting something from a text, we analyze it and decide into which available category/categories it should be classified into.

This task assumes that we have a number of categories defined beforehand, and that we know the criteria that determines whether a text should be categorized into any of them. In MeaningCloud, we refer to this definition of both the categories and their criteria as classification models.

Our Text Classification API provides several generic predefined models such as IAB (a standard from the advertisement industry) or IPTC (an international standard to classify news).

Going back to the example we used before, in the image on the right, we can see how our text is classified by the two predefined models we have mentioned.

demo_meaningcloud_classification

Text classification gives us an idea of what a text is about according to specific criteria. This may apply to an article, a tweet or to the feedback you obtain from a customer.

In some instances, this generic criteria may not fit your needs, so for those cases you can define your own classification model through the use of our customization engine.

These are some scenarios in which Text Classification can be applied:

  • Automatic tag suggestions for news articles or blog posts.
  • Complete characterization (visual sorting) of user feedback according to different criteria.

Sentiment Analysis

Sentiment Analysis is MeaningCloud’s product for sentiment analysis or opinion mining, which is the task of “identifying and extracting subjective information in source materials” [3]. One of the most basic tasks in sentiment analysis is classifying the polarity of a given text at the document, sentence, or feature/aspect level.

Our Sentiment Analysis API combines the complete morphosyntactic analysis carried out by our core engine with sentiment information, which allows us to extract a sentiment analysis at every level.

We can obtain a global polarity of the text, or we can go in deeper and see the polarity expressed in each one of the sentences that make up the text.

demo_meaningcloud_globalsentiment

On the right, we can see the global analysis we obtain for the text we’ve used as an example before. We have a polarity value with a level of confidence, an agreement/disagreement value to indicate if within the text all the polarities detected within the text per sentence/segment agree, a subjectivity value, and an irony value.

MeaningCloud also offers the possibility of combining this analysis with the Topics Extraction feature, allowing you to obtain the polarity associated to the entities and the concepts in the text. This is usually referred to as aspect-level sentiment analysis.

On the right, we see some of the entities detected. In the image, the entities detected with a positive polarity are shown in green—the athletes that have won medals—while the ones with no polarity are shown in blank rows.

demo_meaningcloud_aspectlevelsentiment

Just like the other products we’ve mentioned, Sentiment Analysis can be customized through our customization engine, both the sentiment associated to terms, as well as the entities and concepts to analyze at an aspect level.

These are some scenarios in which Sentiment Analysis can be applied:

  • Customer satisfaction analysis
  • Popularity analysis
  • Voice of the Costumer

Other products

Text Clustering

Text Clustering provides cluster analysis, the task of “grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters)” [4].

In this case, the objects in question are texts, and the different types of analysis provided can help us discover patterns in them, either to visually sort the data or to learn new information about it and use it as feedback for other types of analysis. A possible use of Text Clustering is to apply it over the texts we are classifying using Text Classification in order to identify new categories to add to our model.

On the right, we can see the result we would obtain if we analyze the text we have used as an example with the next two paragraphs in the same article. The three texts are grouped into two different clusters: “Rio Games” and “Performs”, which fit their overall themes quite well.

demo_meaningcloud_clusters

Language Identification

Language Identification is MeaningCloud’s product for “determining which natural language given content is in” [5]. Though it is usually considered an auxiliary task, it’s no less important.

Any of the previously mentioned analysis need to know the language of the content to analyze. If you are working in a single language, this is not an issue, but nowadays, multilingual scenarios such as Twitter are more and more common, so having an API to carry out this task is extremely useful.

Lemmatization, PoS and Parsing

Lemmatization, PoS (part of speech) and Parsing provide a complete morphosyntactic analysis of a text:

  • Lemmatizationthe task of “grouping together the different inflected forms of a word so they can be analyzed as a single item” [6]
  • PoS tagging or grammatical tagging—the task of “marking up a word in a text (corpus) as corresponding to a particular part of speech” [7]
  • Parsing or syntactic analysis—the task of “analyzing a string of symbols, either in natural language or in computer languages, conforming to the rules of a formal grammar” [8]

On the right, there’s the morphosyntactic tree obtained from one of the sentences from the example text.

As you can probably guess from the image, this morphosyntactic tree is also combined with the Topics Extraction and Sentiment Analysis.

sentiment-analysis-morphosyntactic

This provides an extremely powerful API, where you can combine sentiment analysis with morphological, syntactical and semantic information. The output is quite complex but it provides a myriad of possibilities for post-processing, including pattern extraction.

Corporate Reputation

Corporate Reputation is not a classic NLP task but a combination of several tasks focused on a specific application. “Reputation of a social entity (a person, a social group, an organization) is an opinion about that entity, typically a result of social evaluation on a set of criteria” [9].

By combining Topics Extraction, Sentiment Analysis and Text Classification, we are able to analyze the sentiment associated to organizations mentioned in a text according to the different categories defined in a reputation model.

This is a small summary of our main capabilities, which can be used through our APIs or using any of our integrations—including our Excel Add-in—if you don’t feel like coding. If you don’t see how they can apply to your scenario or have any questions about it, just drop us a line and we will be happy to help.


Leave a Reply

Your email address will not be published. Required fields are marked *

*
*