如何解决如何对将标点符号分类为空格的单词进行分词
|| 基于这个问题，很快就结束了： 尝试创建一个程序来读取用户输入，然后将数组拆分为单独的单词，我的指针是否全部有效？ 我认为与其结束讨论，不如说可以做一些额外的工作来帮助OP澄清问题。 问题： 我想标记用户输入并将标记存储到单词数组中。 我想使用标点符号（。，-）作为分隔符，因此将其从令牌流中删除。 在C语言中，我将使用“ 0”将数组拆分为令牌，然后手动构建数组。 像这样： 主要功能：char **findwords(char *str);
int main()
{
int test;
char words[100]; //an array of chars to hold the string given by the user
char **word; //pointer to a list of words
int index = 0; //index of the current word we are printing
char c;
cout << \"die monster !\";
//a loop to place the charecters that the user put in into the array
do
{
c = getchar();
words[index] = c;
}
while (words[index] != \'\\n\');
word = findwords(words);
while (word[index] != 0) //loop through the list of words until the end of the list
{
printf(\"%s\\n\",word[index]); // while the words are going through the list print them out
index ++; //move on to the next word
}
//free it from the list since it was dynamically allocated
free(word);
cin >> test;
return 0;
}
char **findwords(char *str)
{
int size = 20; //original size of the list
char *newword; //pointer to the new word from strok
int index = 0; //our current location in words
char **words = (char **)malloc(sizeof(char *) * (size +1)); //this is the actual list of words
/* Get the initial word,and pass in the original string we want strtok() *
* to work on. Here,we are seperating words based on spaces,commas,*
* periods,and dashes. IE,if they are found,a new word is created. */
newword = strtok(str,\",.-\");
while (newword != 0) //create a loop that goes through the string until it gets to the end
{
if (index == size)
{
//if the string is larger than the array increase the maximum size of the array
size += 10;
//resize the array
char **words = (char **)malloc(sizeof(char *) * (size +1));
}
//asign words to its proper value
words[index] = newword;
//get the next word in the string
newword = strtok(0,.-\");
//increment the index to get to the next word
++index;
}
words[index] = 0;
return words;
}
解决方法
看一下Boost令牌生成器，它在C ++上下文中要比
strtok()
white space
white space
#include <locale>
#include <string>
#include <sstream>
#include <iostream>
// This is my facet that will treat the,.- as space characters and thus ignore them.
class WordSplitterFacet: public std::ctype<char>
{
public:
typedef std::ctype<char> base;
typedef base::char_type char_type;
WordSplitterFacet(std::locale const& l)
: base(table)
{
std::ctype<char> const& defaultCType = std::use_facet<std::ctype<char> >(l);
// Copy the default value from the provided locale
static char data[256];
for(int loop = 0;loop < 256;++loop) { data[loop] = loop;}
defaultCType.is(data,data+256,table);
// Modifications to default to include extra space types.
table[\',\'] |= base::space;
table[\'.\'] |= base::space;
table[\'-\'] |= base::space;
}
private:
base::mask table[256];
};
std::ctype<char>* wordSplitter(new WordSplitterFacet(std::locale()));
<stream>.imbue(std::locale(std::locale(),wordSplitter));
std::vector<std::string> data;
for(std::istream_iterator<std::string> loop(<stream>); loop != std::istream_iterator<std::string>(); ++loop)
{
// In here loop is an iterator that has tokenized the stream using the
// operator >> (which for std::string reads one space separated word.
data.push_back(*loop);
}
std::copy(std::istream_iterator<std::string>(<stream>),std::istream_iterator<std::string>(),std::back_inserter(data));
int main()
{
// Create the facet.
std::ctype<char>* wordSplitter(new WordSplitterFacet(std::locale()));
// Here I am using a string stream.
// But any stream can be used. Note you must imbue a stream before it is used.
// Otherwise the imbue() will silently fail.
std::stringstream teststr;
teststr.imbue(std::locale(std::locale(),wordSplitter));
// Now that it is imbued we can use it.
// If this was a file stream then you could open it here.
teststr << \"This,stri,plop\";
cout << \"die monster !\";
std::vector<std::string> data;
std::copy(std::istream_iterator<std::string>(teststr),std::back_inserter(data));
// Copy the array to cout one word per line
std::copy(data.begin(),data.end(),std::ostream_iterator<std::string>(std::cout,\"\\n\"));
}
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。