| Top |  |
 |
 |
 |
Functions
| AnjutaTokenStream * | anjuta_token_stream_push () |
| AnjutaTokenStream * | anjuta_token_stream_pop () |
| AnjutaTokenStream * | anjuta_token_stream_get_parent () |
| AnjutaToken * | anjuta_token_stream_get_root () |
| GFile * | anjuta_token_stream_get_current_directory () |
| GFile * | anjuta_token_stream_get_current_file () |
| AnjutaToken * | anjuta_token_stream_tokenize () |
| gint | anjuta_token_stream_read () |
| void | anjuta_token_stream_append_token () |
Description
A AnjutaTokenStream object reads and writes a list of tokens. It uses two list. The first list is assigned when the object is created. Each token is read as characters discarding the separation between tokens. The second list is written using the data of the first list, so no new memory is allocated, in order to recreate a new list of tokens.
This is used when the lexer needs several passes. At the beginning the file is read as a single token containing the whole file content. The first pass split this content into tokens. Additional passes are done on some parts of the token list to get a more precise splitting.
It is important to not allocate new memory and keep the same character pointers in the additional passes because the token list does not own the memory. The address of each character is used to find the position of the changed data in the file.
Several objects can be linked together to create a stack. It is used for included file or variable expansion.
Functions
anjuta_token_stream_push ()
AnjutaTokenStream * anjuta_token_stream_push (AnjutaTokenStream *parent,AnjutaToken *root,AnjutaToken *content,GFile *file);
Create a new stream from a list of tokens. If a parent stream is passed, the new stream keep a link on it, so we can return it when the new stream will be destroyed.
Parameters
parent |
a parent AnjutaTokenStream object or |
[allow-none] |
root |
a token or |
[allow-none] |
content |
a token list. |
|
file |
a GFile of the file. |
[allow-none] |
anjuta_token_stream_pop ()
AnjutaTokenStream *
anjuta_token_stream_pop (AnjutaTokenStream *stream);
Destroy the stream object and return the parent stream if it exists.
anjuta_token_stream_get_parent ()
AnjutaTokenStream *
anjuta_token_stream_get_parent (AnjutaTokenStream *stream);
Return the parent stream
anjuta_token_stream_get_root ()
AnjutaToken *
anjuta_token_stream_get_root (AnjutaTokenStream *stream);
Return the root token for the output stream.
anjuta_token_stream_get_current_directory ()
GFile *
anjuta_token_stream_get_current_directory
(AnjutaTokenStream *stream);
Return the current directory.
anjuta_token_stream_get_current_file ()
GFile *
anjuta_token_stream_get_current_file (AnjutaTokenStream *stream);
Return the current file.
anjuta_token_stream_tokenize ()
AnjutaToken * anjuta_token_stream_tokenize (AnjutaTokenStream *stream,gint type,gsize length);
Create a token of type from the last length characters previously read and append it in the output stream. The characters are not copied in the output stream, the new token uses the same characters.
Parameters
stream |
a AnjutaTokenStream object. |
|
type |
a token type. |
|
length |
the token length in character. |
anjuta_token_stream_read ()
gint anjuta_token_stream_read (AnjutaTokenStream *stream,gchar *buffer,gsize max_size);
Read token from the input stream and write the content as a C string in the buffer passed as argument.
Parameters
stream |
a AnjutaTokenStream object. |
|
buffer |
a character buffer to fill with token data. |
|
max_size |
the size of the buffer. |
anjuta_token_stream_append_token ()
void anjuta_token_stream_append_token (AnjutaTokenStream *stream,AnjutaToken *token);
Append an already existing token in the output stream.