[ragel-users] Re: fexec bug in v5.15?

[Adrian Thurston]

Hi Carlos,

As it turns out, it's not safe to manipulate the current pointer in 
pattern actions. I had realized this recently and in 5.15 an error is 
emitted if you use fhold in a pattern action. It didn't occur to me that 
I have to do the same for fexec. I should also put a warning in the docs 
about directly manipulating p in pattern actions.

The reason is that as a part of restarting the scanner, it's sometimes 
necessary to manipulate p. Sometimes you need to shift it back one. 
Other times you need to set it to tokend. Other times you don't need to 
do anything at all. This all depends on the contents of the patterns. So 
if you change p first, you mess the scanner up in ways that are hard to 
predict.

Cheers,
  Adrian

Carlos Antunes wrote:
> Adrain,
> 
> Just wanted to clarify that v5.14 with that scanner patch you provided
> also exhibits the problem so I guess that patch is the culprit.
> 
> Thanks!
> 
> Carlos
> 
> Carlos Antunes wrote:
>> Adrian,
>>
>> I have the following defined:
>>
>> consume_sp_optional := |*
>>
>>             ( ( '\r'? '\n' )? [ \t] )* { printf("===== sp_optional
>> (%s)\n", p); };
>>
>>             '\r'? '\n' { printf("===== crlf (%s)\n", p); fexec
>> tokstart; fret; };
>>
>>             any { printf("<---- sp_optional (%s)\n", p); fexec
>> tokstart; fret; };
>>
>>         *|;
>>
>> This machine basically eats up optional spaces and tabs including
>> folding and returns when CRLF is encountered. This worked fine with
>> v5.14. With version 5.15, however, I think there might be a bug in the
>> way fexec is working. Looking at the generated C code, I see
>> (corresponding to the \r'? '\n' line ):
>>
>> tr6811:
>> #line 26 "test.rl"
>>         {tokend = p;{ printf("===== crlf (%s)\n", p); {p =
>> ((tokstart))-1;} {p--;{cs = stack[--top]; goto _again;}} }p--;}
>>         goto st6170;
>>
>>
>> Now, there is this p = ((tokstart))-1 followed by a p-- (and another
>> p-- that is never reached). Shouldn't the first expression be just p =
>> tokstart? The purpose of the fexec in my code is to backtrack and make
>> the CRLF available for further matching.
>>
>> Thanks!
>>
>> Carlos
>>
>>
>>
>> --
>> "We hold [...] that all men are created equal; that they are
>> endowed [...] with certain inalienable rights; that among
>> these are life, liberty, and the pursuit of happiness"
>>         -- Thomas Jefferson
> 
> 
>