Tokenizer for debug use

Tokenizes a character vector and returns all possible results out of the tokenization process. The returned data.frame contains additional attributes for debug usage.

Arguments

text

String.

sys_dic

String.

user_dic

String.

partial

Logical.

Value

data.frame.