[ragel-users] Re: tuning/optimizing scanners

[Chuck Remes]

Adrian,

I am using ragel 5.24 so I can have ruby support.


On Oct 5, 2007, at 11:13 AM, Adrian Thurston wrote:

> Hi Chuck,
>
> The parsing methodology looks fine to me. There is no undue  
> backtracking.
>
> What version of Ragel are you using?
>
> -Adrian
>
> Chuck Remes wrote:
>> I've written a log parsing tool using ragel and ruby. I'm using the
>> scanner construct to perform the parsing, but things appear to be
>> running very slowly. I fear I may have chosen the wrong methodology
>> to parse the log. (And yes, I know ruby isn't the quickest language
>> out there...) :-)
>>
>> The log in question is a set of key/value pairs that look like this
>> (this is one line):
>>
>> Oct  1 09:50:33.37204 [29193]: {market = ICE | type = order |
>> order_id = 4 | buy = 1 | price = 80.83 | volume = 1 | date =
>> 2007-10-01 | time = 09:50:33.37201 | metadata = {l={f=Quote|g=4|j=1|
>> sid=8290182729}|ac=289182|cf=2881|ca= 289182}}
>>
>> I'm uninterested in the date and other data at the line start, so I
>> throw it away. I primarily search for the key (e.g. 'market = ') and
>> then fgoto another machine to parse the value. Upon hitting a pipe
>> character, I fgoto main again and look for another key. I pasted in a
>> section of the machine below to illustrate.
>>
>> Is this the correct approach? Is there a superior method for rapidly
>> parsing long text strings? Be gentle with me... I'm new to this  
>> stuff.
[snip]