Skip to contents

Ngrams tokenizer

Source: R/pack.R
ngram_tokenizer.Rd

Make an ngram tokenizer function.

Usage

ngram_tokenizer(n = 1L)

Arguments

n

Integer.

Value

ngram tokenizer function