RuleBasedTransliterator is a transliterator
michael@0: * that reads a set of rules in order to determine how to perform
michael@0: * translations. Rule sets are stored in resource bundles indexed by
michael@0: * name. Rules within a rule set are separated by semicolons (';').
michael@0: * To include a literal semicolon, prefix it with a backslash ('\').
michael@0: * Whitespace, as defined by Character.isWhitespace(),
michael@0: * is ignored. If the first non-blank character on a line is '#',
michael@0: * the entire line is ignored as a comment.
michael@0: *
michael@0: * Each set of rules consists of two groups, one forward, and one michael@0: * reverse. This is a convention that is not enforced; rules for one michael@0: * direction may be omitted, with the result that translations in michael@0: * that direction will not modify the source text. In addition, michael@0: * bidirectional forward-reverse rules may be specified for michael@0: * symmetrical transformations.
michael@0: * michael@0: *Rule syntax
michael@0: * michael@0: *Rule statements take one of the following forms:
michael@0: * michael@0: *$alefmadda=\u0622;$alefmadda", will be replaced by
michael@0: * the Unicode character U+0622. Variable names must begin
michael@0: * with a letter and consist only of letters, digits, and
michael@0: * underscores. Case is significant. Duplicate names cause
michael@0: * an exception to be thrown, that is, variables cannot be
michael@0: * redefined. The right hand side may contain well-formed
michael@0: * text of any length, including no text at all ("$empty=;").
michael@0: * The right hand side may contain embedded UnicodeSet
michael@0: * patterns, for example, "$softvowel=[eiyEIY]".ai>$alefmadda;ai<$alefmadda;ai<>$alefmadda;Translation rules consist of a match pattern and an output
michael@0: * string. The match pattern consists of literal characters,
michael@0: * optionally preceded by context, and optionally followed by
michael@0: * context. Context characters, like literal pattern characters,
michael@0: * must be matched in the text being transliterated. However, unlike
michael@0: * literal pattern characters, they are not replaced by the output
michael@0: * text. For example, the pattern "abc{def}"
michael@0: * indicates the characters "def" must be
michael@0: * preceded by "abc" for a successful match.
michael@0: * If there is a successful match, "def" will
michael@0: * be replaced, but not "abc". The final '}'
michael@0: * is optional, so "abc{def" is equivalent to
michael@0: * "abc{def}". Another example is "{123}456"
michael@0: * (or "123}456") in which the literal
michael@0: * pattern "123" must be followed by "456".
michael@0: *
The output string of a forward or reverse rule consists of
michael@0: * characters to replace the literal pattern characters. If the
michael@0: * output string contains the character '|', this is
michael@0: * taken to indicate the location of the cursor after
michael@0: * replacement. The cursor is the point in the text at which the
michael@0: * next replacement, if any, will be applied. The cursor is usually
michael@0: * placed within the replacement text; however, it can actually be
michael@0: * placed into the precending or following context by using the
michael@0: * special character '@'. Examples:
michael@0: *michael@0: * michael@0: *michael@0: *
a {foo} z > | @ bar; # foo -> bar, move cursor michael@0: * before a
michael@0: * {foo} xyz > bar @@|; # foo -> bar, cursor between michael@0: * y and z
UnicodeSet
michael@0: * michael@0: *UnicodeSet patterns may appear anywhere that
michael@0: * makes sense. They may appear in variable definitions.
michael@0: * Contrariwise, UnicodeSet patterns may themselves
michael@0: * contain variable references, such as "$a=[a-z];$not_a=[^$a]",
michael@0: * or "$range=a-z;$ll=[$range]".
UnicodeSet patterns may also be embedded directly
michael@0: * into rule strings. Thus, the following two rules are equivalent:
michael@0: *michael@0: * michael@0: *michael@0: *
$vowel=[aeiou]; $vowel>'*'; # One way to do this
michael@0: * [aeiou]>'*'; michael@0: * # michael@0: * Another way
See {@link UnicodeSet} for more documentation and examples.
michael@0: * michael@0: *Segments
michael@0: * michael@0: *Segments of the input string can be matched and copied to the michael@0: * output string. This makes certain sets of rules simpler and more michael@0: * general, and makes reordering possible. For example:
michael@0: * michael@0: *michael@0: *michael@0: * michael@0: *michael@0: *
([a-z]) > $1 $1; michael@0: * # michael@0: * double lowercase letters
michael@0: * ([:Lu:]) ([:Ll:]) > $2 $1; # reverse order of Lu-Ll pairs
The segment of the input string to be copied is delimited by
michael@0: * "(" and ")". Up to
michael@0: * nine segments may be defined. Segments may not overlap. In the
michael@0: * output string, "$1" through "$9"
michael@0: * represent the input string segments, in left-to-right order of
michael@0: * definition.
Anchors
michael@0: * michael@0: *Patterns can be anchored to the beginning or the end of the text. This is done with the
michael@0: * special characters '^' and '$'. For example:
michael@0: *michael@0: * michael@0: *michael@0: *
^ a > 'BEG_A'; # match 'a' at start of text
michael@0: * a > 'A'; # match other instances michael@0: * of 'a'
michael@0: * z $ > 'END_Z'; # match 'z' at end of text
michael@0: * z > 'Z'; # match other instances michael@0: * of 'z'
It is also possible to match the beginning or the end of the text using a UnicodeSet.
michael@0: * This is done by including a virtual anchor character '$' at the end of the
michael@0: * set pattern. Although this is usually the match chafacter for the end anchor, the set will
michael@0: * match either the beginning or the end of the text, depending on its placement. For
michael@0: * example:
michael@0: *michael@0: * michael@0: *michael@0: *
$x = [a-z$]; # match 'a' through 'z' OR anchor
michael@0: * $x 1 > 2; # match '1' after a-z or at the start
michael@0: * 3 $x > 4; # match '3' before a-z or at the end
Example
michael@0: * michael@0: *The following example rules illustrate many of the features of michael@0: * the rule language.
michael@0: * michael@0: *| Rule 1. | michael@0: *abc{def}>x|y |
michael@0: *
| Rule 2. | michael@0: *xyz>r |
michael@0: *
| Rule 3. | michael@0: *yz>q |
michael@0: *
Applying these rules to the string "adefabcdefz"
michael@0: * yields the following results:
|adefabcdefz |
michael@0: * Initial state, no rules match. Advance michael@0: * cursor. | michael@0: *
a|defabcdefz |
michael@0: * Still no match. Rule 1 does not match michael@0: * because the preceding context is not present. | michael@0: *
ad|efabcdefz |
michael@0: * Still no match. Keep advancing until michael@0: * there is a match... | michael@0: *
ade|fabcdefz |
michael@0: * ... | michael@0: *
adef|abcdefz |
michael@0: * ... | michael@0: *
adefa|bcdefz |
michael@0: * ... | michael@0: *
adefab|cdefz |
michael@0: * ... | michael@0: *
adefabc|defz |
michael@0: * Rule 1 matches; replace "def"
michael@0: * with "xy" and back up the cursor
michael@0: * to before the 'y'. |
michael@0: *
adefabcx|yz |
michael@0: * Although "xyz" is
michael@0: * present, rule 2 does not match because the cursor is
michael@0: * before the 'y', not before the 'x'.
michael@0: * Rule 3 does match. Replace "yz"
michael@0: * with "q". |
michael@0: *
adefabcxq| |
michael@0: * The cursor is at the end; michael@0: * transliteration is complete. | michael@0: *
The order of rules is significant. If multiple rules may match michael@0: * at some point, the first matching rule is applied.
michael@0: * michael@0: *Forward and reverse rules may have an empty output string. michael@0: * Otherwise, an empty left or right hand side of any statement is a michael@0: * syntax error.
michael@0: * michael@0: *Single quotes are used to quote any character other than a
michael@0: * digit or letter. To specify a single quote itself, inside or
michael@0: * outside of quotes, use two single quotes in a row. For example,
michael@0: * the rule "'>'>o''clock" changes the
michael@0: * string ">" to the string "o'clock".
michael@0: *
Notes
michael@0: * michael@0: *While a RuleBasedTransliterator is being built, it checks that michael@0: * the rules are added in proper order. For example, if the rule michael@0: * "a>x" is followed by the rule "ab>y", michael@0: * then the second rule will throw an exception. The reason is that michael@0: * the second rule can never be triggered, since the first rule michael@0: * always matches anything it matches. In other words, the first michael@0: * rule masks the second rule.
michael@0: * michael@0: * @author Alan Liu michael@0: * @internal Use transliterator factory methods instead since this class will be removed in that release. michael@0: */ michael@0: class RuleBasedTransliterator : public Transliterator { michael@0: private: michael@0: /** michael@0: * The data object is immutable, so we can freely share it with michael@0: * other instances of RBT, as long as we do NOT own this object. michael@0: * TODO: data is no longer immutable. See bugs #1866, 2155 michael@0: */ michael@0: TransliterationRuleData* fData; michael@0: michael@0: /** michael@0: * If true, we own the data object and must delete it. michael@0: */ michael@0: UBool isDataOwned; michael@0: michael@0: public: michael@0: michael@0: /** michael@0: * Constructs a new transliterator from the given rules. michael@0: * @param rules rules, separated by ';' michael@0: * @param direction either FORWARD or REVERSE. michael@0: * @exception IllegalArgumentException if rules are malformed. michael@0: * @internal Use transliterator factory methods instead since this class will be removed in that release. michael@0: */ michael@0: RuleBasedTransliterator(const UnicodeString& id, michael@0: const UnicodeString& rules, michael@0: UTransDirection direction, michael@0: UnicodeFilter* adoptedFilter, michael@0: UParseError& parseError, michael@0: UErrorCode& status); michael@0: michael@0: /** michael@0: * Constructs a new transliterator from the given rules. michael@0: * @param rules rules, separated by ';' michael@0: * @param direction either FORWARD or REVERSE. michael@0: * @exception IllegalArgumentException if rules are malformed. michael@0: * @internal Use transliterator factory methods instead since this class will be removed in that release. michael@0: */ michael@0: /*RuleBasedTransliterator(const UnicodeString& id, michael@0: const UnicodeString& rules, michael@0: UTransDirection direction, michael@0: UnicodeFilter* adoptedFilter, michael@0: UErrorCode& status);*/ michael@0: michael@0: /** michael@0: * Covenience constructor with no filter. michael@0: * @internal Use transliterator factory methods instead since this class will be removed in that release. michael@0: */ michael@0: /*RuleBasedTransliterator(const UnicodeString& id, michael@0: const UnicodeString& rules, michael@0: UTransDirection direction, michael@0: UErrorCode& status);*/ michael@0: michael@0: /** michael@0: * Covenience constructor with no filter and FORWARD direction. michael@0: * @internal Use transliterator factory methods instead since this class will be removed in that release. michael@0: */ michael@0: /*RuleBasedTransliterator(const UnicodeString& id, michael@0: const UnicodeString& rules, michael@0: UErrorCode& status);*/ michael@0: michael@0: /** michael@0: * Covenience constructor with FORWARD direction. michael@0: * @internal Use transliterator factory methods instead since this class will be removed in that release. michael@0: */ michael@0: /*RuleBasedTransliterator(const UnicodeString& id, michael@0: const UnicodeString& rules, michael@0: UnicodeFilter* adoptedFilter, michael@0: UErrorCode& status);*/ michael@0: private: michael@0: michael@0: friend class TransliteratorRegistry; // to access TransliterationRuleData convenience ctor michael@0: /** michael@0: * Covenience constructor. michael@0: * @param id the id for the transliterator. michael@0: * @param theData the rule data for the transliterator. michael@0: * @param adoptedFilter the filter for the transliterator michael@0: */ michael@0: RuleBasedTransliterator(const UnicodeString& id, michael@0: const TransliterationRuleData* theData, michael@0: UnicodeFilter* adoptedFilter = 0); michael@0: michael@0: michael@0: friend class Transliterator; // to access following ct michael@0: michael@0: /** michael@0: * Internal constructor. michael@0: * @param id the id for the transliterator. michael@0: * @param theData the rule data for the transliterator. michael@0: * @param isDataAdopted determine who will own the 'data' object. True, the caller should not delete 'data'. michael@0: */ michael@0: RuleBasedTransliterator(const UnicodeString& id, michael@0: TransliterationRuleData* data, michael@0: UBool isDataAdopted); michael@0: michael@0: public: michael@0: michael@0: /** michael@0: * Copy constructor. michael@0: * @internal Use transliterator factory methods instead since this class will be removed in that release. michael@0: */ michael@0: RuleBasedTransliterator(const RuleBasedTransliterator&); michael@0: michael@0: virtual ~RuleBasedTransliterator(); michael@0: michael@0: /** michael@0: * Implement Transliterator API. michael@0: * @internal Use transliterator factory methods instead since this class will be removed in that release. michael@0: */ michael@0: virtual Transliterator* clone(void) const; michael@0: michael@0: protected: michael@0: /** michael@0: * Implements {@link Transliterator#handleTransliterate}. michael@0: * @internal Use transliterator factory methods instead since this class will be removed in that release. michael@0: */ michael@0: virtual void handleTransliterate(Replaceable& text, UTransPosition& offsets, michael@0: UBool isIncremental) const; michael@0: michael@0: public: michael@0: /** michael@0: * Return a representation of this transliterator as source rules. michael@0: * These rules will produce an equivalent transliterator if used michael@0: * to construct a new transliterator. michael@0: * @param result the string to receive the rules. Previous michael@0: * contents will be deleted. michael@0: * @param escapeUnprintable if TRUE then convert unprintable michael@0: * character to their hex escape representations, \uxxxx or michael@0: * \Uxxxxxxxx. Unprintable characters are those other than michael@0: * U+000A, U+0020..U+007E. michael@0: * @internal Use transliterator factory methods instead since this class will be removed in that release. michael@0: */ michael@0: virtual UnicodeString& toRules(UnicodeString& result, michael@0: UBool escapeUnprintable) const; michael@0: michael@0: protected: michael@0: /** michael@0: * Implement Transliterator framework michael@0: */ michael@0: virtual void handleGetSourceSet(UnicodeSet& result) const; michael@0: michael@0: public: michael@0: /** michael@0: * Override Transliterator framework michael@0: */ michael@0: virtual UnicodeSet& getTargetSet(UnicodeSet& result) const; michael@0: michael@0: /** michael@0: * Return the class ID for this class. This is useful only for michael@0: * comparing to a return value from getDynamicClassID(). For example: michael@0: *michael@0: * . Base* polymorphic_pointer = createPolymorphicObject(); michael@0: * . if (polymorphic_pointer->getDynamicClassID() == michael@0: * . Derived::getStaticClassID()) ... michael@0: *michael@0: * @return The class ID for all objects of this class. michael@0: * @internal Use transliterator factory methods instead since this class will be removed in that release. michael@0: */ michael@0: U_I18N_API static UClassID U_EXPORT2 getStaticClassID(void); michael@0: michael@0: /** michael@0: * Returns a unique class ID polymorphically. This method michael@0: * is to implement a simple version of RTTI, since not all C++ michael@0: * compilers support genuine RTTI. Polymorphic operator==() and michael@0: * clone() methods call this method. michael@0: * michael@0: * @return The class ID for this object. All objects of a given michael@0: * class have the same class ID. Objects of other classes have michael@0: * different class IDs. michael@0: */ michael@0: virtual UClassID getDynamicClassID(void) const; michael@0: michael@0: private: michael@0: michael@0: void _construct(const UnicodeString& rules, michael@0: UTransDirection direction, michael@0: UParseError& parseError, michael@0: UErrorCode& status); michael@0: }; michael@0: michael@0: michael@0: U_NAMESPACE_END michael@0: michael@0: #endif /* #if !UCONFIG_NO_TRANSLITERATION */ michael@0: michael@0: #endif