Iterator

The base class for a set of stream iterators. These operate upon a buffered input stream, and are designed to deal with partial content. That is, stream iterators go to work the moment any data becomes available in the buffer. Contrast this behaviour with the ocean.text.Util iterators, which operate upon the extent of an array.

There are two types of iterators supported; exclusive and inclusive. The former are the more common kind, where a token is delimited by elements that are considered foreign. Examples include space, comma, and end-of-line delineation. Inclusive tokens are just the opposite: they look for patterns in the text that should be part of the token itself - everything else is considered foreign. Currently ocean.io.stream includes the exclusive variety only.

Each pattern is exposed to the client as a slice of the original content, where the slice is transient. If you need to retain the exposed content, then you should .dup it appropriately.

The content provided to these iterators is intended to be fully read-only. All current tokenizers abide by this rule, but it is possible a user could mutate the content through a token slice. To enforce the desired read-only aspect, the code would have to introduce redundant copying or the compiler would have to support read-only arrays (now in D2).

See Delimiters, Lines, Patterns, Quotes.

class Iterator : InputFilter {

size_t scan(const(void)[] data);

this(InputStream stream);

Iterator set(InputStream stream);

cstring get();

int opApply(int delegate(ref cstring) dg);

int opApply(int delegate(ref int, ref cstring) dg);

int opApply(int delegate(ref int, ref cstring, ref cstring) dg);

cstring next();

size_t set(const(char)* content, size_t start, size_t end);

size_t set(const(char)* content, size_t start, size_t end, size_t next);

size_t notFound();

size_t found(size_t i);

bool has(cstring set, char match);

}

Constructors

this this(InputStream stream): Instantiate with a buffer.

Members

Functions

found size_t found(size_t i): Invoked when a scanner matches a pattern. The provided value should be the index of the last element of the matching pattern, which is converted back to a void[] index.
get cstring get(): Return the current token as a slice of the content.
has bool has(cstring set, char match): See if set of characters holds a particular instance.
next cstring next(): Locate the next token. Returns the token if found, null otherwise. Null indicates an end of stream condition. To sweep a conduit for lines using method next():
notFound size_t notFound(): Called when a scanner fails to find a matching pattern. This may cause more content to be loaded, and a rescan initiated.
opApply int opApply(int delegate(ref cstring) dg): Iterate over the set of tokens. This should really provide read-only access to the tokens, but D does not support that at this time.
opApply int opApply(int delegate(ref int, ref cstring) dg): Iterate over a set of tokens, exposing a token count starting at zero.
opApply int opApply(int delegate(ref int, ref cstring, ref cstring) dg): Iterate over a set of tokens and delimiters, exposing a token count starting at zero.
scan size_t scan(const(void)[] data): The pattern scanner, implemented via subclasses.
set Iterator set(InputStream stream): Set the provided stream as the scanning source.
set size_t set(const(char)* content, size_t start, size_t end): Set the content of the current slice to the provided start and end points.
set size_t set(const(char)* content, size_t start, size_t end, size_t next): Set the content of the current slice to the provided start and end points, and delimiter to the segment between end & next (inclusive.)

Variables

delim cstring delim;: Undocumented in source.
slice cstring slice;: Undocumented in source.

Inherited Members

From InputFilter

source InputStream source;: Undocumented in source.
conduit IConduit conduit(): Return the hosting conduit.
read size_t read(void[] dst): Read from conduit into a target array. The provided dst will be populated with content from the conduit.
load void[] load(size_t max): Load the bits from a stream, and return them all in an array. The dst array can be provided as an option, which will be expanded as necessary to consume the input.
flush IOStream flush(): Clear any buffered content.
seek long seek(long offset, Anchor anchor): Seek on this stream. Target conduits that don't support seeking will throw an IOException.
input InputStream input(): Return the upstream host of this filter.
close void close(): Close the input.