R, Visualization

Sentiment analysis in Tableau with R

With the increasing amount of user content on the web, text analytics is gaining more mainstream adoption. Sentiment analysis, keyword and named entity extraction are the most common tasks since they allow quickly classifying, filtering and turning text into easily consumable metrics. What do the customers like most about your product, what do they not like? Do people perceive your brand more positively or negatively compared to last year?

With the new R integration feature in Tableau 8.1 it is very easy to add these functionality to your dashboards. There are currently two packages in R that can be used for this purpose: sentiment and qdap. In this post we will use sentiment. This package requires tm and Rstem packages, so first you’ll need to install those. You can do this by typing in the commands below, into your R console (or RStudio if that’s the IDE of your choice).

It may be difficult to find the right versions of Rstem and sentiment. If you already have these packages you can skip to the next step. Before you run the workbook please make sure you load the packages either in the calculation by adding library(sentiment); before the classify_ functions or in Rserve config as covered in my previous blog post about Logistic Regression.

download.file("http://cran.cnr.berkeley.edu/src/contrib/Archive/Rstem/Rstem_0.4-1.tar.gz", "Rstem_0.4-1.tar.gz")
install.packages("Rstem_0.4-1.tar.gz", repos=NULL, type="source")

Once you have tm and Rstem installed, here is how you can download and install the sentiment package.

download.file("http://cran.r-project.org/src/contrib/Archive/sentiment/sentiment_0.2.tar.gz", "sentiment.tar.gz")
install.packages("sentiment.tar.gz", repos=NULL, type="source")

NOTE: Rstem and sentiment packages are becoming more and more difficult to get to work in newer R versions. If you’re having trouble, please read the comments section where you will find information about alternative packages.

Let’s take the first stab by using the classify_polarity function. Comment Text column contains reviews for a hypothetical product. We are using our calculated field Sentiment for both text and color coding as it returns one of three classifications: negative, neutral and positive.

Results from polarity classification

You will notice that the results are not perfect. Second row from the bottom, is in fact a negative comment about delayed delivery but classified as a positive comment. More on that later. Now let’s have a look at what the calculated field looks like.

Classify polarity calculated field

As you can see the R script is very simple. We are calling the function and retrieving the column corresponding to best_fit. Another method in this package is classify_emotion which classifies text into emotion such as anger, joy, fear… The function call is very similar but we get a different dimension from the results this time. Especially the two lines that are associated with emotion “fear” look far off. But how does this work and how can it be made better?

Classify emotion example

Sentiment analysis techniques can be classified into two high level categories:

  1. Lexicon based :  This technique relies on dictionaries of words annotated with their orientation described as polarity and strength e.g. negative and strong, based on which a polarity score for the text is calculated. This method gives high precision results as long as lexicon used has a good coverage of words encountered in the text being analyzed.
  2. Learning based : These techniques require training a classifier with examples of known polarity presented as text classified into positive, negative and neutral classes.

R’s sentiment package follows a lexicon based approach hence we were able to get right into the action, given it comes with a lexicon for English. In your R package library under \sentiment\data folder you can find the lexicon as a file named subjectivity.csv.gz.

The text that was incorrectly classified as having positive polarity is the following “Took 4 weeks to receive it even though I paid for 2 day delivery. What a scam.” If you open the file, as you probably suspected, you will find out that scam is not a word in the lexicon. Let’s add the following line to the file,


then save, zip the file, restart RServe and refresh our workbook.

Results with the updated lexicon

Now,  you can see that the text is classified correctly as expressing negative sentiment. When using lexicon-based systems, adding new words to the lexicon or using a completely new lexicon  are potential paths to follow if you are not getting good results. Incorrect classifications are more likely if  slang, jargon and colloquial words are being used in the text you’re analyzing since these are not covered extensively in common lexica.

You can download the workbook containing the example HERE.

Happy Holidays!


61 thoughts on “Sentiment analysis in Tableau with R

  1. Sean Otto says:

    Trying to replicate what you have done and not being competely experienced with R and R scripts, I get the following error message in Tableau: Error in eval(expr,envir, enclos): could not find function “classify_polarity”

    I’m assuming it refers to your comment about adding the sentiment library, but I is that added in the Tableau formula? Not exactly sure on your message of where to add it.

    • Hi Sean,
      Were you able to install all the libraries successfully? If the answer is yes the most likely culprit is that the library isn’t loaded. You could verify this by adding the library reference to your function call.

      SCRIPT_STR(‘library(sentiment);polarity_data = classify_polarity(.arg1,algorithm=”bayes”,verbose=TRUE)[,4]’,ATTR([Call Script]))

      Let me know if this solves the issue.

      I wrote about how to take advantage of Rserve configuration file to preload packages and other objects here which you may find useful


      If you do this Rserve will load the package on start only once instead of evaluating library(sentiment) command every time you refresh your view in Tableau. It shortens your code in R, also would give you better performance. i will add a pointer from this article to that one and a note that the example assumes the libraries are pre-loaded in Rserve configuration to avoid future confusion.

      ~ Bora

      • I think I fixed this issue. You still need to first open sentiment libraries in R and import R library in Tableau.But the key thing is that Tableau distinguishes the single & double quotes (‘ ‘ and ” “) ! Here’s my function call below and it finally works for me:)


  2. Praveen Koppolu says:

    I’m trying to replicate the above, got the error

    Error in base::parse(text = .cmd) : :1:71: unexpected input
    1: library(sentiment);polarity_data = classify_polarity(.arg1,algorithm=��

    Please help me.

    • I can’t tell much from the snippet. This sort of error commonly happens when the line is a continuation and there is something missing like a trailing comma on the previous line. The other likely cause is ASCII vs UTF. It could be that you’re using a different kind of quote or there is some other non-ASCII character in there. If you are editing in a tool like Word or copy-pasting from a browser etc., it is likely that you get the wrong character while it may appear like the right character on the surface.

    • Hein says:

      I installed R v3.0.2 and tried to install this package. I got an error message that it is not available for this version of R.

  3. Shubho Ray says:

    This is a very interesting article which prompted me to recreate it with my own response data. The only problem is, I’m unable to add any extra lines to the subjectivity.csv file that you had mentioned.
    Can you point out as to what could be the reason?? If my query is not clear, do let me know what extra information you require.

    • Hi Shunho,
      What were the steps you used? Unzip, edit, save, zip? If the file you saved didn’t work, potential issues could be related to encoding, using different new line etc. If your changes seem to be ignored, it could be because the package is already loaded in which case loading again (if this is done in your Rserve config, restarting Rserve) could be the solution.

      • Shubho Ray says:

        Sorry for the delayed response.
        Steps used were exactly the same that you mentioned, but I didn’t understand what exactly you meant by the “different new line” issue.
        As for reloading Rserve, I did that too, every time I made any changes to the file.
        So, how to proceed from this point?

      • I encountered this problem and I found it’s due to the unzipping and zipping process. Basically after you have edited your subjectivity.csv file (Lexicon) you have to zip it back to .gz file (gzip). If you zip it back to .rar .tar it won’t work.

        So I downloaded http://www.7-zip.org/download.html (7Zip) and chose to compress my subjectivity.csv file to be compressed directly to subjectivity.csv.gz

        Now it works for me in Tableau and you can find R sentiment analysis has changed based on your new words in the lexicon.

    • You can still get the sentiment package from Omegahat.org. I picked sentiment since it is the most straightforward to use.

      Qdap is another package that can be used for sentiment analysis and it is still in CRAN. You can also write your own function to do it or give a shot to sentiment140 as suggested in the above link.

      • Waikent says:

        Being a novice, and given sentiment does appear to have stopped being updated, has anyone managed to produce the required code in Tableau for qdap?

        Any chance you might do a part 2 of this tutorial to cover the qdap component?

  4. I saw another example using this package that recommends removing punctuation, special characters, hyperlinks, etc. in the context of analyzing tweets. In the example above, you didn’t remove punctuation or convert to lowercase. How important do you think it is to clean up the text before using this package?

    • Good question.

      For this library capitalization makes no difference.

      As for punctuation it makes no difference as long as there is proper spacing which was the case in my demo dataset.

      E.g. if you have something like this is “My experience was terrible,bad,sad.” It won’t catch any of those three words but if it were “terrible, bad, sad” it would work.

      Similarly “My experience was bad.Terrible in fact.” would miss the words bad and terrible. If it were “My experience was bad. Terrible in fact.” though it would work fine.

      For tweets you can assume that the text won’t have proper spacing so it is better to be safe than sorry and replace all the punctuation and special characters with spaces before passing onto this function.

  5. steven says:

    What do i need to do in R outside of firing up RServe? Should i have a bunch of R script already written, or can i just plug tableau into a CSV and drop the calculated field to the right of my comments?

  6. hello bora, i’m just curious with classify_sentiment function in R, that using bayes algorithm to classify sentiment, and if u set verbose = TRUE, there’s some value for each sentiment, value for positive, negative, positive/negative, and best_fit. For some neutral text (according to me), the classify_sentiment function classified into positive sentiment. how’s that work ? and if you know how to calculate it with bayes? sorry if my english is bad 😦 , thank you so much bora, you can e-mail me the answer if you don’t mind. sutamichaerul@gmail.com , once again. thank you so much 🙂

    • My goal in the blog is to show ways how different R packages can be used with Tableau. For general R package support please rely on R forums or the package documentation. Sentiment package allows you to set how you want the classifier to calculate. If you set algorithm option to bayes it will use a Multinomial Naive Bayes Classifier trained using the lexicon in the package. The other option (default option) is a simple voter algorithm (count negative, positive etc. words and majority wins). If the text is incorrectly classified either the words in your text are not in the lexicon or there is negation that is being not properly acknowledged by the classifier.

  7. Shrutayu Kale says:

    Error in base::parse(text = .cmd) : :2:0: unexpected end of input
    1: classify_polarity(.agr1,algorithm=”bayes”,verbose=TRUE[,4]
    I am getting the above error when i try to execute the script. please help

    • Is this the actual error? .agr1 should be .arg1 and classify_polarity is missing the closing parentheses. Also unless you’re loading the sentiment library as part of your Rserve configuration, you should add library(sentiment); to the beginning. So it becomes library(sentiment);classify_polarity(.arg1 ….

    • You will need a lexicon in French. The package doesn’t come with it and I am not sure off the top of my head if there is one readily available. Googling for subjectivity lexicon French didn’t return anything useful.

  8. Kirk says:

    Hi Bora, I am also fighting with the classify_polarity function. When trying to connect to Rserv through Tableau’s help menue, I get a “successful” message when testing the connection, but

    Error in eval(expr, envir, enclos) :
    could not find function “classify_polarity”

    Upon clicking the OK button. I understand that you recomend editing the Rserv.cfg file to add an implicity function call such as, SCRIPT_STR(‘library(sentiment);polarity_data = classify_polarity(.arg1,algorithm=”bayes”,verbose=TRUE)[,4]’,ATTR([Call Script])), But I can not find the file Rserv.cfg. A search turns up Rserv.xml. Can you give me a hint as to where I could look for this file? Is this something I need to create? and if so, where should it be located. Checking in help implies that the config file should be in /etc/. Any cluses would be much appreciated!

    • You can put Rserv.cfg in the same folder with Rserve.exe. If you don’t want that for some reason, when you initiate Rserve pass config file location using –RS-conf argument.

      • Kirk says:

        Thank you for the reply Bora. I have loaded the packages Rserve, Rtem, and sentiment. I have created an Rserv.cfg file in the same directory as Rserve.exe (C:\Users\kwyther\Documents\R\win-library\3.0\Rserve\libs\x64\Rserv.cfg) with notepad.

        The only text in the file is: SCRIPT_STR(‘library(sentiment);polarity_data = classify_polarity(.arg1,algorithm=”bayes”,verbose=TRUE)[,4]’,ATTR([Call Script]))

        However, I am still getting the: Error in eval(expr, envir, enclos) :
        could not find function “classify_polarity” error when I try and estabolish a Tableau connection to R through Help -> Settings and Performance -> Manage R connection. Again, “Test Connection” reports success, but hitting the OK gives the error.

        I have also tried using SCRIPT_STR(“library(sentiment);classify_polarity(.arg1,algorithm=’bayes’,verbose=TRUE)[,4]”,
        ATTR([CommentText])) as suggested by Yijie Wang. It makes no difference.

        I am using Rstudio, but I can’t imagine that makes a difference. Please excuse what may be overattension to detail, but I am coming from a Unix/Liux envrionment and not terribly familar with Windows and I want to make sure I am not overlooking something silly. I did check the permission of the Rserv.cfg file and they appear to be the standard Full control, Modify, Read and execute, Read, and Write.

        Is there anything else that might prevent the “classify_polarity function from loading when Rserve is started? Thanks in advance.

  9. Pingback: Bob Dylan – analyzing 55 years of lyrics » Tableau Picasso

  10. HT says:

    I’m with Waikent.. just spent a few hours trying every permutation of every comment to try to get this to work.

    If there’s an updated list of instructions, we’d appreciate them.

  11. Vijay Rajan says:

    Hello Bora-

    Thank you for the great article. I am trying to replicate what you are doing but I am stumbling at the very first step – how to prepare a data set for the analysis and visualization. Where did you data set come from? How did you convert it to a csv file? Did you scrape it using some R package (such as, rvest or Rcurl)? Could you please direct me to a place which shows how to prepare a data set such as the one you have used to load in Tableau?

    Thank you very much!


  12. mamta says:

    i received the same error: could not find classify_emotion . what i did is i ran this together

    and manually typed in classify_emotion(documents,algorithm = “bayes”,verbose=TRUE) instead of copy pasting.

    so , the issue mainly was the libraries werent loaded correctly and also that, by manully typing in the method, you allow the IDE to pick up the library from its proposed recommendation.

    Hope this helps!

  13. If you’re having trouble with the sentiment package, you may want to try the syuzhet package. It has several useful methods with a number of knobs. Below is an example how you can use the get_sentiment function in this package in the sample workbook provided in this blog post.

    I classified 0.3 or less as negative, over 0.3 to 0.5 (inclusive) as neutral, and above 0.5 as positive in this case.

    len <- length(.arg1);
    result <- numeric(len);
    for (i in 1:len){
    token <- get_tokens(.arg1[i]);
    sentiment <- get_sentiment(token,method = "syuzhet")
    sentiment <- sentiment[sentiment!=0]
    result[i] <- mean(sentiment)
    as.character(cut(result,breaks=c(-Inf, 0.3, 0.5, Inf),labels=c("negative","neutral","positive")))',
    ATTR([Comment Text]))

  14. Sachin says:

    Hi Bora,

    I am continously getting the message that classify_polarity function not available. I checked and also noticed that Sentiment package is not available for the version which I am using (3.2.3). Is there any other alternative?

    Please suggest.

    Thanks in advance.


  15. Sarah F says:

    Hi there, I’m wondering what sort of time is expected to calculate the polarity and sentiment, as I am not getting an error, having successfully loaded the Sentiment page and having a valid calculated field, but when I apply the calculation, Tableau sits for many, minutes calculating the dependencies and doesn’t seem to finalise the calculation, Ive left it running for 15 minutes, Im assuming it doesn’t take that long for 30 observations. Any thoughts. It seems like an error but it isn’t throwing an error message.

    • Hi Sarah,
      I can think of two reasons. 1) Addressing setting on the calculation is causing Tableau to make a separate call to R for each sentence/paragraph. There is connection overhead, so making hundreds of separate calls can cause noticeable delays. 2) Data is large.

      If the latter I would recommend doing it as a preprocessing step, writing results to a file e.g. CSV then opening that from Tableau.

      If the former, the solution is easy. If you click on the pill doing the sentiment analysis and select Edit Table Calculation from the context menu, and put everything in the Addressing box, it will send all the text to R in a single request. This could be called “Compute Using” and may require you to check all the boxes in Tableau 10 (we redesigned table calculation dialog). In earlier versions Compute Using dropdown will have an “Advanced…” option which will pop up another dialog and you will need to put everything into Addressing box.

      I hope this helps.

      ~ Bora

  16. Hi Bora – first off great article and thank you for your tutorial. Having followed all of the guidance above, from yourself and other, I get this error:

    Error in base::parse(text = .cmd) : :2:1: unexpected ‘[‘ 1: library(sentiment);classify_polarity(.arg1,algorithm=’bayes’,verbose=TRUE) 2: [ ^

    When I look at the code in Tableau, it says The calculation is valid.

    Is this to do with the text file by any chance, and there are characters in the tweets that the code is unable to pick up?

    Thanks in advance for your help.

    • Hi Boroon,
      Are you using the example workbook or did you create your own calculated field? If you wrote your own, does it follow the same structure as in the example? If you have extra new lines in your code, errors like this can happen. E.g. if instead of


      you do


      you can get an error like this.

      Similarly if instead of this


      You do something like this


      you can get a similar error.



  17. Nathan G says:

    Bora, Thanks for this guide. I was able to use this on my own comment data, but am having an issue getting sentiment counts. I’ve using your example also, but have been unsuccessful. Any idea if this is possible with this method?

    Something like this.

    Sentiemnt Count of ID
    Positive 25
    Neutral 73
    Negative 12

  18. Gaurav Mathur says:

    Hi Bora,

    Thank you for the great article, I am also attending the advanced analytics webinar series from tableau and learning a lot. I tried to follow the instructions , but while downloading the Rstem package I get the following error:

    Warning: running command ‘make -f “Makevars” -f “C:/PROGRA~1/R/R-33~1.1/etc/i386/Makeconf” -f “C:/PROGRA~1/R/R-33~1.1/share/make/winshlib.mk” SHLIB=”Rstem.dll” OBJECTS=”api.o danish_stem.o dutch_stem.o english_stem.o finnish_stem.o french_stem.o german_stem.o mytest.o norwegian_stem.o portuguese_stem.o russian_stem.o spanish_stem.o swedish_stem.o utilities.o”‘ had status 127
    ERROR: compilation failed for package ‘Rstem’
    installation of package ‘Rstem_0.4-1.tar.gz’ had non-zero exit status

    I tried to do some research but could not get to any solution. Would you be kindly able to provide some suggestions? Thank you again for the great article.


  19. Hi,

    I am currently using R 3.2.3 and when i try to use the code in tableau (SCRIPT_STR(“library(sentiment);classify_polarity(.arg1,algorithm=’bayes’,verbose=TRUE)[,4]”,
    ATTR([LTR Comment – Relationship]))

    Below is the error i get.

    An error Occurred while communicating with RServe service.


      • Hi Bora,

        An error occurred while communicating with the RServe service.
        Error in base::parse(text = .cmd) : :1:55: unexpected input 1: library(sentiment);classify_polarity(.arg1,algorithm=� ^


  20. Hi Bora,

    When i use the syuzhet library. I get this error.

    An error occurred while communicating with the RServe service.
    Error : could not find function “get_tokens”


  21. Hi Bora,

    Sorry. I am unable to reply to your comment.

    This is how i am loading the script.

    len <- length(.arg1);
    result <- numeric(len);
    for (i in 1:len){
    token <- get_tokens(.arg1[i]);
    sentiment <- get_sentiment(token,method = "syuzhet")
    sentiment <- sentiment[sentiment!=0]
    result[i] <- mean(sentiment)
    as.character(cut(result,breaks=c(-Inf, 0.3, 0.5, Inf),labels=c("negative","neutral","positive")))',
    ATTR([LTR Comment – Relationship]))


    • Renato Falcon Lyke says:

      Hi Bora,

      I checked the syuzhet package and i found in the file named syuzhet-vignette get_tokens function missing. I tired to add the same and zip it to install the package. I get the error MD5 SUMS incorrect. Any suggestions how to go about this.


    • Hi Bora,

      I was researching the syuzhet package and found that the function get_token is missing in 3.2.3. I tried to add the same and zip it got and MD5 Sums error. Any suggestions how to go about this. I tried the mentioned suggestions of zipping the file it did not work.


    • That is odd. This (1.0.1) is the latest version on CRAN. https://cran.r-project.org/web/packages/syuzhet/index.html and in the online documentation they still list get_tokens() as a supported function.

      You’re saying that the version you have is 3.2.3 so I am confused. Maybe you mean R 3.2.3?

      I also reinstalled the package from scratch and I see get_tokens also in the syuzhet-vignette.html under the doc folder.

      Something doesn’t add up.

      I have R 3.3.2. If your R version is 3.2.3, you may want to upgrade R first, then try installing syuzhet again.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s