Ferret::Analysis::LetterAnalyzer

Summary

A LetterAnalyzer creates a TokenStream that splits the input up into maximal strings of characters as recognized by the current locale. If implemented in Ruby it would look like;

class LetterAnalyzer
  def initialize(lower = true)
    @lower = lower
  end

  def token_stream(field, str)
    return LetterTokenizer.new(str, @lower)
  end
end

As you can see it makes use of the LetterTokenizer.

Public Class Methods

new(lower = true) → analyzer click to toggle source

Create a new LetterAnalyzer which downcases tokens by default but can optionally leave case as is. Lowercasing will be done based on the current locale.

lower

set to false if you don’t want the field’s tokens to be downcased

static VALUE
frb_letter_analyzer_init(int argc, VALUE *argv, VALUE self)
{
    Analyzer *a;
    GET_LOWER(true);
#ifndef POSH_OS_WIN32
    if (!frb_locale) frb_locale = setlocale(LC_CTYPE, "");
#endif
    a = mb_letter_analyzer_new(lower);
    Frt_Wrap_Struct(self, NULL, &frb_analyzer_free, a);
    object_add(a, self);
    return self;
}

Home Classes Methods

In Files

Parent

Methods

Files

Class/Module Index

Ferret::Analysis::LetterAnalyzer

Summary

Public Class Methods