[ragel-users] A generic eof action from a ragel scanner

[Adrian Thurston]

There is no native mechanism to express EOF within a scanner. Since your 
grammar doesn't require a scanner (no backtracking will ever take 
place), you can kleene star a union of tokens and then wrap that in an 
eof action. Like this:

main := ( ' ' %{} | '\t' %{} | [0-9]+ %{})** $eof {};

Just note you will be forced to use one character of lookahead. The 
scanner eliminates lookahead whenever possible.

Another option is to send the null character to the state machine, then 
write scanner pattern for that.

Regards,
  Adrian

On 13-02-19 10:14 AM, gonzalo diethelm wrote:
> I am playing with a typical toy calculator written with ragel and lemon,
> in C. The scanner is defined like this:
>
>    main := |*
>
>      number => num_tok;
>
>      '('    => lparen_tok;
>
>      ')'    => rparen_tok;
>
>      '+'    => plus_tok;
>
>      '-'    => minus_tok;
>
>      '*'    => mul_tok;
>
>      '/'    => div_tok;
>
>      '^'    => pow_tok;
>
>      ws;
>
>    *|;
>
> Each of the *_tok actions calls the lemon-generated parser. For example:
>
>    action pow_tok    { CalcParser(parser, TOKEN_POW, 0); }
>
>    action num_tok    { CalcParser(parser, TOKEN_NUM, get_num(ts, te)); }
>
> The idea is to pass each argv[] character as a separate expression to
> compute. So this execution:
>
> ./calc 2+3*4 7-5 “2   +    4”
>
> Should output three lines:
>
> 14
>
> 2
>
> 6
>
> Since the lemon parser has to be informed about the EOF condition, what
> I am doing is manually informing the parser of this condition right when
> I have finished processing each one of the argv[] arguments:
>
> calc_exec(argv[i], strlen(argv[i]));
>
> …
>
> void calc_exec (const char* data, int len)
>
> {
>
>      const char* p = data;
>
>      const char* pe = data + len;  // Should this be len+1?
>
>      const char* eof = pe;
>
>      %% write exec;
>
>      /* Make sure we inform the parser we saw the whole input */
>
>      CalcParser(parser, 0, 0);
>
> }
>
> But what I would really like to do is to add a “generic EOF action” to
> the scanner, that issued this final call to the parser:
>
>    ??? %eof(CalcParser(parser, 0, 0);)
>
> I have not been able to found how to specify this “generic EOF action”
> to my ragel scanner. Is this even possible? Is it the right approach?
>
> Thanks for any help. And thanks for the great tool that ragel is.
>
> --
>
> Gonzalo Diethelm
>
> DCV Chile
>
> ------------------------------------------------------------------------
>
> Declaración de confidencialidad: Este Mensaje esta destinado para el uso
> de la o las personas o entidades a quien ha sido dirigido y puede
> contener información reservada y confidencial que no puede ser
> divulgada, difundida, ni aprovechada en forma alguna. El uso no
> autorizado de la información contenida en este correo podrá ser
> sancionado de conformidad con la ley chilena. Si usted ha recibido este
> correo electrónico por error, le pedimos eliminarlo junto con los
> archivos adjuntos y avisar inmediatamente al remitente, respondiendo
> este mensaje.
>
> Disclosure: This Message is to be used by the individual, individuals or
> entities that it is addressed to and may include private and
> confidential information that may not be disclosed, made public nor used
> in any way at all. Unauthorized use of the information in this
> electronic mail message may be subject to the penalties set forth by
> Chilean law. If you have received this electronic mail message in error,
> we ask you to destroy the message and its attached file(s) and to
> immediately notify the sender by answering this message.
>
>
> _______________________________________________
> ragel-users mailing list
> ragel-users at complang.org
> http://www.complang.org/mailman/listinfo/ragel-users
>

_______________________________________________
ragel-users mailing list
ragel-users at complang.org
http://www.complang.org/mailman/listinfo/ragel-users