Feature Request: Inline Scanner

[Carlos Antunes]

Hello!

I've been playing with Ragel now for roughly three weeks and I find it
a wonderful piece of software. There is one particular thing, however,
that Ragel could do better, in my opinion, that of capturing matched
input.

Let's look at an example:

# Whitespace including folding
sp = ( ( '\r'? '\n' )? [ \t] )+;

# From mail header (simplified for illustrative purposes)
from_header = sp? ( display_name sp? )? '<' email_address '>';

The state machine Ragel implements is wonderful for ensuring correct
syntax. The scanner Ragel implements is wonderful for repeatedly match
tokens. But how about just capuring input matched by the state
machine? Well, in this case, things get a little bit more complicated.
In my opinion, it would be wonderful to have what I'm calling an
inline scanner as a complement to the two currently implemented
choices.

An example of an inline scanner in action would be something like:

display_name = |> display_name_pattern; { capture_display_name(ts, te); }; <|;

email_address = |> email_address_pattern; { capture_email_address(ts,
te); }; <|;

Both capture_display_name and capture_email_address are used defined
functions accepting locally declared (automatically by Ragel)
variables 'ts' and 'te' (for tokstart and tokend, respectively).

With this kind of inline scanner, not only would the syntax be
enforced but the input easily captured.

Transitions from the state machine to the inline scanner would happen
only if and only if the state machine would transition to the state
machine defined by the inline scanner pattern. And, once the inline
scanner finishes matching, it would transition the the following state
machine as usual.

Although the functionality described above can be achieve with clever
use of the current state machine and scanner paradigms, in my opinion,
things would be a lot easier with this inline scanner concept.

So, Adrian and everybody else, what say you?

Thanks!

Carlos

-- 
"We hold [...] that all men are created equal; that they are
endowed [...] with certain inalienable rights; that among
these are life, liberty, and the pursuit of happiness"
        -- Thomas Jefferson