Analyzing Your Company’s Social Media Presence With IBM Watson And Node.js

Original Source: https://www.smashingmagazine.com/2018/04/analyzing-social-media-presence-ibm-watson-nodejs/

2018-04-04T13:00:50+02:00
2018-04-04T12:16:56+00:00
Analyzing Your Company’s Social Media Presence With IBM Watson And Node.js

If you are unfamiliar with Machine Learning (ML) technology, it has existed in science fiction for many years and is finally reaching its maturity in our society. One of the first ML examples I saw as a kid was in Star Trek’s The Next Generation when Lieutenant Tasha Yar trains with her holographic opponent that learns how to fight and better defeat in future battles.

In today’s society, China has developed a “lane robot” that is a guard rail controlled by a computer system that can direct the flow of traffic into different lanes, increasing safety and improving traveling time. This is done automatically based on time of day and how much traffic is flowing in each direction.

Another example is Pittsburg unveiling AI traffic signals that automatically detect traffic patterns and alter the traffic lights on-the-fly. Each light is controlled independently to help reduce both the commuting time and the idling time of cars. According to the article, pilot tests have demonstrated a reduced travel time of 25% and idling time by over 40%. There are, of course, hundreds of other examples of ML technology that make intelligent decisions based on the content it consumes.

To accomplish today’s goal, I am going to demonstrate (using Node.js) how to perform a search with Twitter’s API to retrieve content that will be inputted into the ML algorithm to be analyzed. This way, you’ll be provided with characteristics about the users who wrote that specific content so that you can get a better understanding of your audience. The example application will be written using Node.js as the server.

It is beyond the scope of this article to demonstrate how to write an ML algorithm. Instead, to aid in the analysis, I will demonstrate how to use IBM’s Watson to help you understand the general personality of your social media audience.

Getting the process just right ain’t an easy task. That’s why we’ve set up ‘this-is-how-I-work’-sessions — with smart cookies sharing what works really well for them. A part of the Smashing Membership, of course.

Explore features →

What Is IBM Watson?

In 2011, Watson began as a computer system that attempted to index the (entire) Internet. It was originally programmed to answer questions posed in ordinary English. Watson competed and won on the TV show Jeopardy! claiming a $1,000,000 cash prize.

Watson was now a proven success.

With the fame of winning on Jeopardy!, IBM has continued to push Watson’s capabilities. Watson has evolved into an enterprise-level application that is focused on Artificial Intelligence (AI) which you can train to identify what you care about most allowing you to make smarter decisions automatically.

The suite of Watson’s services is divided into six high-level categories:

Conversation
The services in this category allow you to build intelligent chatbot’s or a virtual customer service agent.
Knowledge
This category is focused on teaching Watson how to interpret data to unlock hidden value and monitor trends.
Vision
This service provides the ability to tag content inside an image that is used to train Watson to be able to automatically recognize the same pattern inside of other images.
Speech
These services provide the ability to convert speech to text and the inverse, text to speech.
Language
This category is split between translating one language to another as well as interpreting the text to predict what predefined category the text belongs to.
Empathy
This category is devoted to understanding the content’s tone, personality, and emotional state. Inside this category is a service called “Personality Insights” that will be used in this article to predict the personality characteristics with the social media content we will provide it.

This article will be focusing on understanding the personality of the content that we will fetch from Twitter. However, as you can see, Watson provides many other AI features that you can explore to automate many other processes simply through training and content aggregation.

Personality Insights

Personality Insights will analyze content and help you understand the habits and preferences at an individual level and at scale. This is called the ‘personality profile.’ The profile is split into two high-level groups: Personality characteristics and Consumption preferences. These groups are further broken down into more finite components.

Note: To help understand the high-level concepts (before we deep dive into the results), the Personality Insights documentation provides this helpful summary describing how the profile is inferred from the content you provide it.

Big Five Personality Traits. Image courtesy: IBM.com. (Large preview)

Personality Characteristics

The Personality Insights service infers personality characteristics based on three primary models:

The ‘Big Five’ personality characteristics represent the most widely used model for generally describing how a person engages with the world. The model includes five primary dimensions:

Agreeableness
Conscientiousness
Extraversion
Emotional range
Openness
Note: Each dimension has six facets that further characterize an individual according to the dimension.

Needs describe which aspects of a product will resonate with a person. The model includes twelve characteristic needs:

Excitement
Harmony
Curiosity
Ideal
Closeness
Self-expression
Liberty
Love
Practicality
Stability
Challenge
Structure

Values describe motivating factors that influence a person’s decision making. The model includes five values:

Self-transcendence / Helping others
Conservation / Tradition
Hedonism / Taking pleasure in life
Self-enhancement / Achieving success
Open to change / Excitement

For more information, see Personality models.

Consumption preferences

Based on the personality characteristics inferred from the input text, the service can also return an indication of the author’s consumption preferences. ‘Consumption preferences’ indicate the author’s likelihood to pursue different products, services, and activities. The service groups the individual preferences into eight categories:

Shopping
Music
Movies
Reading and learning
Health and activity
Volunteering
Environmental concern
Entrepreneurship

Each category contains from one to as many as a dozen individual preferences.

Note: For more information, see Consumption preferences. For a more in-depth overview of a particular point of interest, I suggest you refer to the Personality Insights documentation.

To be effective, Watson requires a minimum of a hundred words to provide an insight into the consumer’s personality. The more words provided, the better Watson can analyze and determine the consumer’s preference.

This means, if you wish to target individuals, you will need to collect more data than one or two tweets from a specific person. However, if a user writes a product review, blog post, email, or anything else related to your company, this could be analyzed on both an individual level and at scale.

To begin, let’s start by setting up the Personality Insights service to begin analyzing a real-world example.

Configuring The Personality Insights Service

Watson is an enterprise application but they offer a free, limited service. Once you’ve created an account and are logged in, you will need to add the Personality Insight service. IBM offers a Lite plan that is free. The Lite plan is limited to 1,000 API calls per month and is automatically deleted after 30 days — perfect for our demonstration.

Create the Personality Insights Service. (Large preview)

Once the service has been added, we will need to retrieve the service’s credentials to perform API calls against it. From Watson’s Dashboard, your service should be displayed. After you’ve selected the service, you’ll find a link to view the Service credentials in the left-hand menu. You will need to create a new ‘Credential.’ A unique name is required and optional configuration parameters can be defaulted for this login. For now, we will leave the configuration options empty.

After you have created a credential, select the ‘View’ credentials link. This will display the API’s URL, your username, and password required to securely execute API calls. Save these somewhere safe as we will need them in the next step.

Testing The Personality Insights Service

To perform API calls, I am going to use Node.js. If you already have Node.js installed, you can move on to the next step; otherwise, follow the instructions to setup Node.js from the official download page.

To demonstrate how to use the Personality Insights, I am going to create a new Node.js project on my computer. With a command prompt open, navigate to the directory where your Node.js projects will be stored and create your new project:

mkdir watson-sentiments
cd watson-sentiments
npm init

To assist with making the API calls to Watson, I am going to leverage the NPM Package: Watson Developer Cloud Node.js SDK. This package can be installed via the command prompt:

npm install watson-developer-cloud –save

Before making the first call, the PersonalityInsightsV3 object needs to be instantiated with the credentials from the previous section. Begin by creating a new file called index.js that will contain the Node.js code.

Here is an example of configuring the class so it is ready to make API calls:

var PersonalityInsightsV3 = require(’watson-developer-cloud/personality-insights/v3’);
var personality_insights = new PersonalityInsightsV3({
“url”: “https://gateway.watsonplatform.net/personality-insights/api”,
“username”: “**************************”,
“password”: “*************”,
“version_date”: “2017-12-01”
});

The personality_insights variable is what we will use to interact with the API for the Personality Insights service. Let’s review how to execute a call and return a personality profile:

var fs = require(’fs’);

personality_insights.profile({
“contentItems”: [
{
“content”: “Some content that contains more than 100 words…”,
“contenttype”: “text/plain”,
“created”: 1447639154000,
“id”: “666073008692314113”,
“language”: “en”
}
],
“consumption_preferences”: true
}, (err, response) => {
if (err) throw err;

fs.writeFile(“results.txt”, JSON.stringify(response, null, 2), function(err) {
if (err) throw err;

console.log(“Results were saved!”);
});
});

The profile function accepts an array of contentItems. The ‘content’ item contains the actual content with a few additional properties identifying additional information to help Watson interpret it.

When this is executed, the results are written to a text file (the results are too large to write in the console). The result is an object that contains the following high-level properties:

word_count
The count of words interpreted
processed_language

The language that the content provided, e.g. (en).

Personality
This is an array of the ‘Big Five’ personality characteristics (Openness, Conscientiousness, Extraversion, Agreeableness, and Emotional range). Each characteristic contains an overall percentile for that characteristic (e.g. 0.8100175318417588). To ascertain more detail, there is an array called children that provides more in-depth insight. For example, a child category under ‘Openness’ is ‘Adventurousness’ that contains its own percentile.
Needs
This is an array of the twelve characteristics that define the aspects a person will resonate with a product (Excitement, Harmony, Curiosity, Ideal, Closeness, Self-expression, Liberty, Love, Practicality, Stability, Challenge, and Structure). Each characteristic contains a percentile of how the content was interpreted.
Values
This is an array of the five characteristics that describe motivating factors that influence a person’s decision making (Self-transcendence / Helping others, Conservation / Tradition, Hedonism / Taking pleasure in life, Self-enhancement / Achieving success, and Open to change / Excitement). Each characteristic contains a percentile of how the content was interpreted.
Behavior
This is an array that contains thirty-one elements. Each element provides a percentile of when the content was created. Seven of the elements define the days of the week (Sunday through Saturday). The remaining twenty-four elements define the hours of the day. This helps you understand when customer’s interact with your product.
consumption_preferences
This is an array that contains eight different categories with as much as a twelve child categories providing a percentile of likelihood to pursue different products, services, and activities (Shopping, Music, Movies, Reading and learning, Health and activity, Volunteering, Environmental concern, and Entrepreneurship).
Warnings
This is an array that provides messages if a problem was encountered interpreting the content provided.

Here is a CodePen of the formatted results:

See the Pen Example Watson Results by Jamie Munro (@endyourif) on CodePen.

Configuring Twitter

To search Twitter for relevant tweets, I am going to use the Twitter NPM package. From a console window where the application is hosted, run the following command to install:

npm install twitter –save

Before we can implement the Twitter package, you need to create a Twitter application.

Retrieving Twitter’s Access Tokens. (Large preview)

Once you’ve created your application, you need to retrieve the authorization keys required to perform API calls. With your application created, navigate to the ‘Keys’ and ‘Access Tokens’ page. Since we are not performing API calls against users of Twitter, OAuth integration is not required. Instead, we need only the four following keys:

Consumer Key
Consumer Secret
Access Token
Access Token Secret

The last two keys need to be generated near the bottom of the ‘Keys’ and ‘Access Tokens’ page. With the keys, here is an example of searching for Tweets about #SmashingMagazine:

var Twitter = require(’twitter’);

var client = new Twitter({
consumer_key: ’*********************’,
consumer_secret: ’******************’,
access_token_key: ’******************’,
access_token_secret: ’****************’
});

client.get(’search/tweets’, { q: ’#SmashingMagazine’ }, function(error, tweets, response) {
if(error) throw error;

console.log(tweets);
});

The result of this code will log a list tweets about Smashing Magazine. For the purposes of this demonstration, the following fields are of interest to us:

id
created_at
text
metadata.iso_language_code

These are the fields we will feed Watson.

Integrating Personality Insights With Twitter

With Twitter setup and Watson setup, it’s time to integrate the two together and see the results. To make it interesting, let’s search for #DonaldTrump to see what the world thinks about the President of the United States. Here is the code example to search Twitter, feed the results into Watson, and write the results to a text file:

var fs = require(’fs’);
var Twitter = require(’twitter’);

var client = new Twitter({
consumer_key: ’*********************’,
consumer_secret: ’******************’,
access_token_key: ’******************’,
access_token_secret: ’****************’
});

var PersonalityInsightsV3 = require(’watson-developer-cloud/personality-insights/v3’);
var personality_insights = new PersonalityInsightsV3({
“url”: “https://gateway.watsonplatform.net/personality-insights/api”,
“username”: “**************************”,
“password”: “*************”,
“version_date”: “2017-12-01”
});

client.get(’search/tweets’, { q: ’#DonaldTrump’ }, function(error, tweets, response) {
if(error) throw error;

var contentItems = [];

// Loop through the tweets
for (var i = 0; i {
if (err) throw err;

// Write the results to a file
fs.writeFile(“results.txt”, JSON.stringify(response, null, 2), function(err) {
if (err) throw err;

console.log(“Results were saved!”);
});
});
});

Here is another CodePen of the formatted results that I received:

See the Pen Donald Trump Watson Results by Jamie Munro (@endyourif) on CodePen.

What Do The Results Say?

Once we’ve analyzed the ‘Openness’ trait of the ‘Big Five,’ we can infer the following:

Emotion is quite low at 13%
Imagination is average at 54%
Intellect is very high at 96%
Authority challenging is also quite high at 87%

The ‘Conscientiousness’ trait at a high-level is average at 46% compared with the ‘Openness’ high-level average of 88%. Whereas ‘Agreeableness’ is very low at only 25%. I guess people on Twitter don’t like to agree with Donald Trump.

Moving on to the ‘Needs.’ The sub-categories of ‘Curiosity’ and ‘Structure’ are in the 60 percentile compared to other categories being below the 10th percentile (Excitement, Harmony, etc.).

And finally, under ‘Values,’ the sub-category that stands out to me as interesting is the ‘Openness’ to ‘Change’ at an abysmal 6%.

Based on when you perform your search, your results may vary as the results are limited to the past seven days from executing the example.

From these results, I would determine that the average person who tweets about Donald Trump is quite intellectual, challenges authority, and is not open to change.

With these results, it would allow you to automatically alter how you would target your content towards your audience to match the results received. You will need to determine what categories are of interest and what percentiles do you wish to target. With this ammunition, you can begin automating.

What Else Can I Do With Watson?

As I mentioned at the beginning of this article, Watson offers many other different services. With these services, you could automate many different parts of common business processes. For example:

Building a chat bot that can intelligently answer questions based on a knowledge base of information;
Build an application where you dictate what you want written to Watson by using the speech to text functionality;
Automatically translate your content into different languages to create a multi-lingual site or knowledge base;
Teach Watson how to look for specific patterns in images. This could be used to determine if a logo is embedded into a photo.

This, of course, is a very small subset that my limited imagination can postulate. I’m sure you can think of many other ways to leverage Watson’s immense capabilities.

If you are looking for more examples, IBM has an entire GitHub repository dedicated to their Node.js SDK. The example folder contains over ten sample applications that convert speech to text, text to speech, tone analysis, and visual recognition to name just a few.

Conclusion

Before Watson can runaway with technological growth, resulting in the singularity where Artificial Intelligence destroys mankind, this article demonstrated how you can turn social media content into a powerful understanding of how the people creating the content think. Using the results from Watson, your application can use the categories of interest where the percentile exceeds or is less than a predetermined amount to change how you target your audience.

If you have other interesting uses of Watson or how you are using the Personality Insights, be sure to leave a comment below.

(rb, ra, yk, il)