[ragel-users] EOF and token termination

[Adrian Thurston]

Hi Brian,

Just to be certain, you'd like a token at the end of the input to match 
even if it is not in a final state?

If that's the case, I'm sorry I don't have an elegant solution for you. 
A state is either final or not, and what you need is to make all states 
final when at the end of the input.

-Adrian



On 10-06-27 06:46 AM, Brian Maher wrote:
> Hello Ragel Users,
>
> First, I just want to say that Ragel is an awesome tool, thank you
> Adrian for sharing this tool with the open source community!
>
> Recently I've been trying to write a grammar for which individual
> tokens may be pre-maturely terminated by EOF.  I came up with the
> following rather hacky solution below (emit the same action as though
> the token was "recognized", then fbreak so that only one token is
> guaranteed to be "seen").  The things that I don't like with this
> solution are that:
>
>    * I'm not keen on having to redundantly call the token's action in
> two places since a maintainer later on may forget to update one of the
> places where $eof(token_action) is done.
>
>    * te is not updated (although this isn't a big deal since I can just
> use p instead of relying on te, and if I really wanted to make a fuss
> about this I could always add $eof{te=p;}).
>
> One idea I had was to $eof{fgoto final;}, but that doesn't work since
> the "final" label is only defined within the context of state charts
> and can't be used as a "normal label".
>
> --------------example.rl
>
> #include<stdio.h>
>
> %% machine t;
> %% write data;
>
> #define INPUT "abcd"
>
> int main() {
>     char *ts, *te;
>     int   cs, act, i;
>     char *input = INPUT;
>     char *p     = input;
>     char *pe    = input + (sizeof(INPUT) - 1);
>     char *eof   = pe;
>
>     fprintf(stderr, "Input[");
>     fwrite(p, 1, pe-p, stderr);
>     fprintf(stderr, "]\n");
>
>     %%{
>         write init;
>
>         action text {
>             fprintf(stderr, "Text [%c]\n", *p);
>         }
>
>         action token {
>             fprintf(stderr, "Token[");
>             fwrite(ts, 1, p-ts, stderr);
>             fprintf(stderr, "]\n");
>         }
>
>         Token = "abc" "def" $eof(token) $eof{fbreak;};
>
>          main := |*
>             Token =>  token;
>             any   =>  text;
>         *|;
>         write exec;
>
>     }%%
>     fprintf(stderr, "end\n");
>     return 0;
> }
>
> --------------/example.rl
>
> Thanks!
> -Brian
>
> _______________________________________________
> ragel-users mailing list
> ragel-users at complang.org
> http://www.complang.org/mailman/listinfo/ragel-users
>

_______________________________________________
ragel-users mailing list
ragel-users at complang.org
http://www.complang.org/mailman/listinfo/ragel-users