Feature Request: Inline Scanner

[Adrian Thurston]

Hi Carlos,

> Yes, but is it really useful to have this kind of overlapping or
> containment, in practical terms?

Overlapping ... you're right, that's a little silly. But I still think 
containment could be useful. Maybe you'd want to have one markup for the 
whole email address and other markups which give you the user and host 
names. Maybe not. Hmmmm.

> Hmm, keep a global variable (ex: alltokstarts)? Thss 'alltokstarts'
> var could be defined as min(tokstart, ts1, ts2, ts3, ...).

I wonder if maintaining this could be made fast even when the number of 
variables grows.

>>From the point of view of the FSM, the inline scanner would be a
> virtual state. Transitions to this virtual state would happen if and
> only if at least one of the inline scanner patterns matches. If there
> is no possible match then the FSM would error.

You'll have to bear with me here, I can be thick sometimes! From what you're 
saying it seems like it's not really a scanner, but more like a union 
because if it finishes when it matches a pattern then it won't ever match 
more than one. Is that right? If that's the case then it seems like the 
criteria for it starting is the same as for it finishing.

Cheers,
  Adrian