[ragel-users] Re: Inline scanner

[Adrian Thurston]

Hi Carlos,

After much thought I've decided that inline scanners are not a suitable
feature for ragel. It is a good technique but there are a few issues.

I think that calls and returns should not be encoded in a language
construct. They need to be treated with care and I'm not comfortable
with encapsulating them.

There is the issue of when to return. Since the feature would be using
the scanning machinery, one might expect to be able to match multiple
tokens as an alternative to matching just a single token. Letting the
user implement the technique allows them to chose. Also, it allows for
the option of erring when no pattern matches.

So far all the inline machine construction operators are compositional.
Inline scanners are not. For example you cannot safely union a machine
that contains an inline scanner because the call necessary to invoke it
could interfere with the other machine. This is related to the issue of
encapsulating calls and returns.

I would however be interested to add a prefix operator which would allow
you to automatically set up the machine that the call is embedded in.
The prefix operator would simply set every state final (this might have
another name in the literature). After you run the prefix operator you
could intersect it with something else to get the correct length (one in
this case).

pat = [a-z][a-z0-9]*;

inline_scanner = |*
    pat => { fret; };
    any => { fhold; fret; };
  *|;

call_pat = ( prefix(pat) & any ) @{ fhold; fcall inline_scanner; };

Regards,
 Adrian

Carlos Antunes wrote:
> On 7/5/07, Adrian Thurston <thurs... at cs.queensu.ca> wrote:
>>
>>> Assuming I understand the idea you are trying to convey here (there's
>>> a good chance that I don't), isn't it possible to transition directly
>>> to the scanner by looking at all the potential transitions of the
>>> scanner as if it was a "normal" state machine?
>> Yes, that's possible. The code I sent does exactly that, only it's
>> explicitly implemented. When it sees a whitespace character it jumps to
>> the whitespace scanner. I'll rework what I sent so hopefully it's clear
>> that it is an implementation of an inline scanner.
>>
> 
> Adrian, now that you've been able to explicitly design an inline
> scanner, would you be able to code ragel to do it implicitly? Although
> one can always do it by hand, as you did, it is error prone and
> sometimes somewhat difficult, depending on what you have inside the
> scanner. This feature would be very much appreciated.
> 
> Thanks!
> 
> Carlos Antunes
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
URL: <http://www.colm.net/pipermail/ragel-users/attachments/20070711/49711107/attachment-0001.sig>