RWCTokenizer(3C++)

Name
     RWCTokenizer - Rogue Wave library class

Synopsis
              #include <rw/ctoken.h>



              RWCString str("a string of tokens");
          RWCTokenizer(str);  // Lex the above string




Description
     Class RWCTokenizer is designed to break a string up into separate tokens,
     delimited by an arbitrary "white space."  It can be thought of as an
     iterator for strings and as an alternative to the ANSI C function
     strtok() which has the unfortunate side effect of changing the string
     being tokenized.

Persistence
     None

Example
              #include <rw/ctoken.h>
          #include <rw/rstream.h>
          main(){
            RWCString a("Something is rotten in the state of Denmark");
            RWCTokenizer next(a);           // Tokenize the string a
            RWCString token;                // Will receive each token
            // Advance until the null string is returned:
            while (!(token=next()).isNull())
              cout << token << "0;
          }


     Program output:

                 Something
             is
             rotten
             in
             the
             state
             of
             Denmark

Public Constructor
              RWCTokenizer(const RWCString& s);
     Construct a tokenizer to lex the string s.

Public Member Operators
              RWCSubString
          operator();


     Advance to the next token and return it as a substring.  The tokens are
     delimited by any of the four characters in " 0. (space, tab, newline and
     null).

              RWCSubString
          operator()(const char* s);


     Advance to the next token and return it as a substring.  The tokens are
     delimited by any character in s, or any embedded null.

              RWCSubString
          operator()(const char* s,size_t num);


     Advance to the next token and return it as a substring.  The tokens are
     delimited by any of the first num characters in s. Buffer s may contain
     nulls, and must contain at least num characters.  Tokens will not be
     delimited by nulls unless s contains nulls.