Sentiment analysis in Tableau with R

With the increasing amount of user content on the web, text analytics is gaining more mainstream adoption. Sentiment analysis, keyword and named entity extraction are the most common tasks since they allow quickly classifying, filtering and turning text into easily consumable metrics. What do the customers like most about your product, what do they not like? Do people perceive your brand more positively or negatively compared to last year?

With the new R integration feature in Tableau 8.1 it is very easy to add these functionality to your dashboards. There are currently two packages in R that can be used for this purpose: sentiment and qdap. In this post we will use sentiment. This package requires tm and Rstem packages, so first you’ll need to install those. You can do this by typing in the commands below, into your R console (or RStudio if that’s the IDE of your choice).

It may be difficult to find the right versions of Rstem and sentiment. If you already have these packages you can skip to the next step. Before you run the workbook please make sure you load the packages either in the calculation by adding library(sentiment); before the classify_ functions or in Rserve config as covered in my previous blog post about Logistic Regression.

download.file("", "Rstem_0.4-1.tar.gz")
install.packages("Rstem_0.4-1.tar.gz", repos=NULL, type="source")

Once you have tm and Rstem installed, here is how you can download and install the sentiment package.

download.file("", "sentiment.tar.gz")
install.packages("sentiment.tar.gz", repos=NULL, type="source")

NOTE: Rstem and sentiment packages are becoming more and more difficult to get to work in newer R versions. If you’re having trouble, please read the comments section where you will find information about alternative packages.

Let’s take the first stab by using the classify_polarity function. Comment Text column contains reviews for a hypothetical product. We are using our calculated field Sentiment for both text and color coding as it returns one of three classifications: negative, neutral and positive.

Results from polarity classification

You will notice that the results are not perfect. Second row from the bottom, is in fact a negative comment about delayed delivery but classified as a positive comment. More on that later. Now let’s have a look at what the calculated field looks like.

Classify polarity calculated field

As you can see the R script is very simple. We are calling the function and retrieving the column corresponding to best_fit. Another method in this package is classify_emotion which classifies text into emotion such as anger, joy, fear… The function call is very similar but we get a different dimension from the results this time. Especially the two lines that are associated with emotion “fear” look far off. But how does this work and how can it be made better?

Classify emotion example

Sentiment analysis techniques can be classified into two high level categories:

  1. Lexicon based :  This technique relies on dictionaries of words annotated with their orientation described as polarity and strength e.g. negative and strong, based on which a polarity score for the text is calculated. This method gives high precision results as long as lexicon used has a good coverage of words encountered in the text being analyzed.
  2. Learning based : These techniques require training a classifier with examples of known polarity presented as text classified into positive, negative and neutral classes.

R’s sentiment package follows a lexicon based approach hence we were able to get right into the action, given it comes with a lexicon for English. In your R package library under \sentiment\data folder you can find the lexicon as a file named subjectivity.csv.gz.

The text that was incorrectly classified as having positive polarity is the following “Took 4 weeks to receive it even though I paid for 2 day delivery. What a scam.” If you open the file, as you probably suspected, you will find out that scam is not a word in the lexicon. Let’s add the following line to the file,


then save, zip the file, restart RServe and refresh our workbook.

Results with the updated lexicon

Now,  you can see that the text is classified correctly as expressing negative sentiment. When using lexicon-based systems, adding new words to the lexicon or using a completely new lexicon  are potential paths to follow if you are not getting good results. Incorrect classifications are more likely if  slang, jargon and colloquial words are being used in the text you’re analyzing since these are not covered extensively in common lexica.

You can download the workbook containing the example HERE.

Happy Holidays!

38 comments on “Sentiment analysis in Tableau with R

  1. Sean Otto says:

    Trying to replicate what you have done and not being competely experienced with R and R scripts, I get the following error message in Tableau: Error in eval(expr,envir, enclos): could not find function “classify_polarity”

    I’m assuming it refers to your comment about adding the sentiment library, but I is that added in the Tableau formula? Not exactly sure on your message of where to add it.

    • Bora Beran says:

      Hi Sean,
      Were you able to install all the libraries successfully? If the answer is yes the most likely culprit is that the library isn’t loaded. You could verify this by adding the library reference to your function call.

      SCRIPT_STR(‘library(sentiment);polarity_data = classify_polarity(.arg1,algorithm=”bayes”,verbose=TRUE)[,4]’,ATTR([Call Script]))

      Let me know if this solves the issue.

      I wrote about how to take advantage of Rserve configuration file to preload packages and other objects here which you may find useful

      If you do this Rserve will load the package on start only once instead of evaluating library(sentiment) command every time you refresh your view in Tableau. It shortens your code in R, also would give you better performance. i will add a pointer from this article to that one and a note that the example assumes the libraries are pre-loaded in Rserve configuration to avoid future confusion.

      ~ Bora

      • Yijie Wang says:

        I think I fixed this issue. You still need to first open sentiment libraries in R and import R library in Tableau.But the key thing is that Tableau distinguishes the single & double quotes (‘ ‘ and ” “) ! Here’s my function call below and it finally works for me:)


  2. Praveen Koppolu says:

    I’m trying to replicate the above, got the error

    Error in base::parse(text = .cmd) : :1:71: unexpected input
    1: library(sentiment);polarity_data = classify_polarity(.arg1,algorithm=��

    Please help me.

    • Bora Beran says:

      I can’t tell much from the snippet. This sort of error commonly happens when the line is a continuation and there is something missing like a trailing comma on the previous line. The other likely cause is ASCII vs UTF. It could be that you’re using a different kind of quote or there is some other non-ASCII character in there. If you are editing in a tool like Word or copy-pasting from a browser etc., it is likely that you get the wrong character while it may appear like the right character on the surface.

  3. kurrabac says:

    there is simpler way doing for Twitter text sentiment analysis in R. Try this pacakge.

    • Hein says:

      I installed R v3.0.2 and tried to install this package. I got an error message that it is not available for this version of R.

  4. Shubho Ray says:

    This is a very interesting article which prompted me to recreate it with my own response data. The only problem is, I’m unable to add any extra lines to the subjectivity.csv file that you had mentioned.
    Can you point out as to what could be the reason?? If my query is not clear, do let me know what extra information you require.

    • Bora Beran says:

      Hi Shunho,
      What were the steps you used? Unzip, edit, save, zip? If the file you saved didn’t work, potential issues could be related to encoding, using different new line etc. If your changes seem to be ignored, it could be because the package is already loaded in which case loading again (if this is done in your Rserve config, restarting Rserve) could be the solution.

      • Shubho Ray says:

        Sorry for the delayed response.
        Steps used were exactly the same that you mentioned, but I didn’t understand what exactly you meant by the “different new line” issue.
        As for reloading Rserve, I did that too, every time I made any changes to the file.
        So, how to proceed from this point?

      • Yijie Wang says:

        I encountered this problem and I found it’s due to the unzipping and zipping process. Basically after you have edited your subjectivity.csv file (Lexicon) you have to zip it back to .gz file (gzip). If you zip it back to .rar .tar it won’t work.

        So I downloaded (7Zip) and chose to compress my subjectivity.csv file to be compressed directly to subjectivity.csv.gz

        Now it works for me in Tableau and you can find R sentiment analysis has changed based on your new words in the lexicon.

  5. Looks like R Sentiment has been discontinued, in the interim do you have any other suggestions?

    • Bora Beran says:

      You can still get the sentiment package from I picked sentiment since it is the most straightforward to use.

      Qdap is another package that can be used for sentiment analysis and it is still in CRAN. You can also write your own function to do it or give a shot to sentiment140 as suggested in the above link.

      • Waikent says:

        Being a novice, and given sentiment does appear to have stopped being updated, has anyone managed to produce the required code in Tableau for qdap?

        Any chance you might do a part 2 of this tutorial to cover the qdap component?

  6. Robert Rouse says:

    I saw another example using this package that recommends removing punctuation, special characters, hyperlinks, etc. in the context of analyzing tweets. In the example above, you didn’t remove punctuation or convert to lowercase. How important do you think it is to clean up the text before using this package?

    • Bora Beran says:

      Good question.

      For this library capitalization makes no difference.

      As for punctuation it makes no difference as long as there is proper spacing which was the case in my demo dataset.

      E.g. if you have something like this is “My experience was terrible,bad,sad.” It won’t catch any of those three words but if it were “terrible, bad, sad” it would work.

      Similarly “My experience was bad.Terrible in fact.” would miss the words bad and terrible. If it were “My experience was bad. Terrible in fact.” though it would work fine.

      For tweets you can assume that the text won’t have proper spacing so it is better to be safe than sorry and replace all the punctuation and special characters with spaces before passing onto this function.

  7. steven says:

    What do i need to do in R outside of firing up RServe? Should i have a bunch of R script already written, or can i just plug tableau into a CSV and drop the calculated field to the right of my comments?

  8. hello bora, i’m just curious with classify_sentiment function in R, that using bayes algorithm to classify sentiment, and if u set verbose = TRUE, there’s some value for each sentiment, value for positive, negative, positive/negative, and best_fit. For some neutral text (according to me), the classify_sentiment function classified into positive sentiment. how’s that work ? and if you know how to calculate it with bayes? sorry if my english is bad😦 , thank you so much bora, you can e-mail me the answer if you don’t mind. , once again. thank you so much🙂

    • Bora Beran says:

      My goal in the blog is to show ways how different R packages can be used with Tableau. For general R package support please rely on R forums or the package documentation. Sentiment package allows you to set how you want the classifier to calculate. If you set algorithm option to bayes it will use a Multinomial Naive Bayes Classifier trained using the lexicon in the package. The other option (default option) is a simple voter algorithm (count negative, positive etc. words and majority wins). If the text is incorrectly classified either the words in your text are not in the lexicon or there is negation that is being not properly acknowledged by the classifier.

  9. Shrutayu Kale says:

    Error in base::parse(text = .cmd) : :2:0: unexpected end of input
    1: classify_polarity(.agr1,algorithm=”bayes”,verbose=TRUE[,4]
    I am getting the above error when i try to execute the script. please help

    • Bora Beran says:

      Is this the actual error? .agr1 should be .arg1 and classify_polarity is missing the closing parentheses. Also unless you’re loading the sentiment library as part of your Rserve configuration, you should add library(sentiment); to the beginning. So it becomes library(sentiment);classify_polarity(.arg1 ….

  10. john says:

    Do these functions run with french text please ?

    • Bora Beran says:

      You will need a lexicon in French. The package doesn’t come with it and I am not sure off the top of my head if there is one readily available. Googling for subjectivity lexicon French didn’t return anything useful.

  11. Kirk says:

    Hi Bora, I am also fighting with the classify_polarity function. When trying to connect to Rserv through Tableau’s help menue, I get a “successful” message when testing the connection, but

    Error in eval(expr, envir, enclos) :
    could not find function “classify_polarity”

    Upon clicking the OK button. I understand that you recomend editing the Rserv.cfg file to add an implicity function call such as, SCRIPT_STR(‘library(sentiment);polarity_data = classify_polarity(.arg1,algorithm=”bayes”,verbose=TRUE)[,4]’,ATTR([Call Script])), But I can not find the file Rserv.cfg. A search turns up Rserv.xml. Can you give me a hint as to where I could look for this file? Is this something I need to create? and if so, where should it be located. Checking in help implies that the config file should be in /etc/. Any cluses would be much appreciated!

    • Bora Beran says:

      You can put Rserv.cfg in the same folder with Rserve.exe. If you don’t want that for some reason, when you initiate Rserve pass config file location using –RS-conf argument.

      • Kirk says:

        Thank you for the reply Bora. I have loaded the packages Rserve, Rtem, and sentiment. I have created an Rserv.cfg file in the same directory as Rserve.exe (C:\Users\kwyther\Documents\R\win-library\3.0\Rserve\libs\x64\Rserv.cfg) with notepad.

        The only text in the file is: SCRIPT_STR(‘library(sentiment);polarity_data = classify_polarity(.arg1,algorithm=”bayes”,verbose=TRUE)[,4]’,ATTR([Call Script]))

        However, I am still getting the: Error in eval(expr, envir, enclos) :
        could not find function “classify_polarity” error when I try and estabolish a Tableau connection to R through Help -> Settings and Performance -> Manage R connection. Again, “Test Connection” reports success, but hitting the OK gives the error.

        I have also tried using SCRIPT_STR(“library(sentiment);classify_polarity(.arg1,algorithm=’bayes’,verbose=TRUE)[,4]”,
        ATTR([CommentText])) as suggested by Yijie Wang. It makes no difference.

        I am using Rstudio, but I can’t imagine that makes a difference. Please excuse what may be overattension to detail, but I am coming from a Unix/Liux envrionment and not terribly familar with Windows and I want to make sure I am not overlooking something silly. I did check the permission of the Rserv.cfg file and they appear to be the standard Full control, Modify, Read and execute, Read, and Write.

        Is there anything else that might prevent the “classify_polarity function from loading when Rserve is started? Thanks in advance.

      • Bora Beran says:

        You just need


        in the config file.

  12. […] string (one song per row). That is the data input format required by R. Bora Beran writes more on his blog about running an R sentiment package from within Tableau, but for a Tableau Public viz I had to do […]

  13. HT says:

    I’m with Waikent.. just spent a few hours trying every permutation of every comment to try to get this to work.

    If there’s an updated list of instructions, we’d appreciate them.

  14. Vijay Rajan says:

    Hello Bora-

    Thank you for the great article. I am trying to replicate what you are doing but I am stumbling at the very first step – how to prepare a data set for the analysis and visualization. Where did you data set come from? How did you convert it to a csv file? Did you scrape it using some R package (such as, rvest or Rcurl)? Could you please direct me to a place which shows how to prepare a data set such as the one you have used to load in Tableau?

    Thank you very much!


  15. mamta says:

    i received the same error: could not find classify_emotion . what i did is i ran this together

    and manually typed in classify_emotion(documents,algorithm = “bayes”,verbose=TRUE) instead of copy pasting.

    so , the issue mainly was the libraries werent loaded correctly and also that, by manully typing in the method, you allow the IDE to pick up the library from its proposed recommendation.

    Hope this helps!

  16. Bora Beran says:

    If you’re having trouble with the sentiment package, you may want to try the syuzhet package. It has several useful methods with a number of knobs. Below is an example how you can use the get_sentiment function in this package in the sample workbook provided in this blog post.

    Rescale, scales the scores to a range of -1 and 1. I classified -0.3-0.3 to neutral, 0.3+ to positive and below 0.3 to negative.

    SCRIPT_STR(‘library(syuzhet);r<-rescale(get_sentiment(.arg1,method = "syuzhet"));ifelse (abs(r) 0.3,”positive”,”negative”))’,

  17. Sachin says:

    Hi Bora,

    I am continously getting the message that classify_polarity function not available. I checked and also noticed that Sentiment package is not available for the version which I am using (3.2.3). Is there any other alternative?

    Please suggest.

    Thanks in advance.


  18. Sarah F says:

    Hi there, I’m wondering what sort of time is expected to calculate the polarity and sentiment, as I am not getting an error, having successfully loaded the Sentiment page and having a valid calculated field, but when I apply the calculation, Tableau sits for many, minutes calculating the dependencies and doesn’t seem to finalise the calculation, Ive left it running for 15 minutes, Im assuming it doesn’t take that long for 30 observations. Any thoughts. It seems like an error but it isn’t throwing an error message.

    • Bora Beran says:

      Hi Sarah,
      I can think of two reasons. 1) Addressing setting on the calculation is causing Tableau to make a separate call to R for each sentence/paragraph. There is connection overhead, so making hundreds of separate calls can cause noticeable delays. 2) Data is large.

      If the latter I would recommend doing it as a preprocessing step, writing results to a file e.g. CSV then opening that from Tableau.

      If the former, the solution is easy. If you click on the pill doing the sentiment analysis and select Edit Table Calculation from the context menu, and put everything in the Addressing box, it will send all the text to R in a single request. This could be called “Compute Using” and may require you to check all the boxes in Tableau 10 (we redesigned table calculation dialog). In earlier versions Compute Using dropdown will have an “Advanced…” option which will pop up another dialog and you will need to put everything into Addressing box.

      I hope this helps.

      ~ Bora

      • Sarah F says:

        Great, thanks so much for your advice. Ill let you know how it goes. I appreciate your approach suggestions.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s