量身定制的情感分析：基于单词及其各自得分对文档进行评分-R NLP

如何解决量身定制的情感分析：基于单词及其各自得分对文档进行评分-R NLP

我正在尝试根据文档中出现的单词对文档评分。对于语料库中出现的每个单词，我都有两种类型的分数。从本质上讲，它类似于情感分析，但具有定制的词典和相应的分数。谢谢

#documents to be scored on 2 dimensions: score1 and score2
documents <- data.frame(textID = 1:3,text = c("Hello everybody,pleased to see everyone together"," DHL postmen have faced difficulties this year","divorcees have trouble finding jobs in this country"),scored1 = rep(NA,3),scored2=rep(NA,3) )

#first scoring dimension
scores1 <- as.matrix(data.frame(words = c("hello","everybody","pleased","to","see","everyone","together","DHL","postmen","have","faced","difficulties","this","year","divorcees","trouble","finding","jobs","in","country" ),scores = 1:20))

#second scoring dimension
scores2 <- as.matrix(data.frame(words = c("hello",scores = 10:29))

#the result should look like this,where each text receives a score that represents the sum of #individual word scores: 

#textID                                                  text      scored1 scored2
#1      1   Hello everybody,pleased to see everyone together       28        91
#2      2       DHL postmen have faced difficulties this year       77        140
#3      3 divorcees have trouble finding jobs in this country       128       200

解决方法

这可以通过

实现

1. tidytext::unnest_token将文档分成一个单词
2. dplyr::left_join单词得分
3. dplyr::summarise计算每个文档的分数

library(dplyr)
library(tidytext)

#documents to be scored on 2 dimensions: score1 and score2
documents <- data.frame(textID = 1:3,text = c("Hello everybody,pleased to see everyone together"," DHL postmen have faced difficulties this year","divorcees have trouble finding jobs in this country"),scored1 = rep(NA,3),scored2=rep(NA,3) )

# 1. Get rid of as.matrix

#first scoring dimension
scores1 <- data.frame(words = c("hello","everybody","pleased","to","see","everyone","together","DHL","postmen","have","faced","difficulties","this","year","divorcees","trouble","finding","jobs","in","country" ),scores = 1:20)

#second scoring dimension
scores2 <- data.frame(words = c("hello",scores = 10:29)       

# 2. Make words lowercase
scores1 <- mutate(scores1,words = tolower(words))
scores2 <- mutate(scores2,words = tolower(words))

# 3. Compute scores
documents %>% 
  select(-scored1,-scored2) %>% 
  tidytext::unnest_tokens(text,output = words,drop = FALSE) %>% 
  left_join(scores1,by = c("words" = "words")) %>% 
  left_join(scores2,by = c("words" = "words"),suffix = c("1","2")) %>% 
  group_by(textID,text) %>% 
  summarise(across(starts_with("scores"),sum,na.rm = TRUE)) %>% 
  rename(scored1 = scores1,scored2 = scores2) %>% 
  ungroup()
#> `summarise()` regrouping output by 'textID' (override with `.groups` argument)
#> # A tibble: 3 x 4
#>   textID text                                                  scored1 scored2
#>    <int> <chr>                                                   <int>   <int>
#> 1      1 "Hello everybody,pleased to see everyone together"        28      91
#> 2      2 " DHL postmen have faced difficulties this year"           77     140
#> 3      3 "divorcees have trouble finding jobs in this country"     128     200