[ragel-users] JavaScript code generation for Ragel

Tue Oct 12 07:24:26 UTC 2010

  On 11/10/2010 20:31, Adrian Thurston wrote:
> Ya as far as Ragel code generation goes you shouldn't need to p, pe, 
> or data available in the write init context. If I remember right that 
> was due to a combination of Ruby variable scoping/declaration rules 
> and the typical usage pattern of Ragel+Ruby. The Ruby code generator 
> really is a special case for a number of reasons. What we need to a 
> Ragel and Ruby expert to sort it out. I'm not both of those things.

I'd take it on if I had the time.

> There is also some strange stuff in the java code generator too. Java 
> doesn't have static array data. It makes functions that set the 
> values. But large arrays overflow the max function size so you can't 
> have large static arrays. To get around this the arrays are made using 
> functions (like the compiler does), but they are broken down into 
> smaller parts and concatenated. Ugh. I'm not sure if this would apply 
> to javascript as well.

I wrote a small test program and creating arrays with 1 and 10 million 
elements worked, although at 10 million the VM was stuttering 
(run-pause-run-pause), probably due to the GC not coping too well. How 
large is large? In the case of Node.js there is another data type called 
Buffer which is a fixed size byte array (sort of). This doesn't have a 
literal notation but it does support the array [] operator so it might 
be able to substitute for static arrays in code. It has the benefit of 
being allocated outside of the runtime and so it doesn't pressurise the 
GC as much. The downside is that it would require special IO handling 
code to setup the arrays and it isn't portable.

It occured to me while writing this that I actually need to make at 
least two different code generators for JavaScript. JavaScript can be 
used for client and server programming (which is why I got into doing 
this work in the first place) and the kind of code you want to generate 
in each case is totally different. In the client case you want to 
optimise for page load time first, so the state machine needs to be 
expressed in the minimum possible code, even if it's slower to execute. 
When generating the same machine for the server case you (probably) want 
the normal approach of trading code size for performance.

Should be an interesting challenge.

Dominic

> -Adrian
>
> On 10-10-11 12:14 PM, Dominic Marks wrote:
>> Hey Adrian,
>>
>> I haven't but I will do. Thank you.
>>
>> I'm not sure if the RubyCodeGen::writeInit function is incorrect but
>> it's quite divergent from the CDCodeGen version. The only problem I had
>> doing the work, besides a few missed semicolons, was that the write init
>> directive was trying to initialise pe to the data length. In my testing
>> script the initialisation and execution stages are in separate functions
>> and this produced invalid JavaScript (data being undefined at that 
>> point).
>>
>> I assumed I had misunderstood something about the options that can be
>> given to write init and write exec but after that turned up a blank I
>> looked at CDCodeGen. I was surprised to find that it didn't include
>> similar code relating to the end directive. I made my writeInit function
>> more like CDCodeGen and my problem was resolved.
>>
>> The code in question:
>>
>> http://github.com/dominicmarks/ragel-js/commit/7a8e554f2829fbebe04a26837ebb5512314c376b#diff-0 
>>
>>
>>
>> The difference might be related to the concessions you mention, I didn't
>> look into it too much.
>>
>> Thanks for Ragel, it's a great piece of software!
>>
>> Dominic
>>
>> On 11/10/2010 19:54, Adrian Thurston wrote:
>>> Great, thanks for working on this!
>>>
>>> Did you have a look at the java code generator? There were a few
>>> concessions that had to be made in the Ruby code generator that might
>>> not need to be made in JS.
>>>
>>> -Adrian
>>>
>>> On 10-10-11 04:07 AM, Dominic Marks wrote:
>>>> Hi all,
>>>>
>>>> I've got the table code generation strategy working. I'm going to work
>>>> on building a set of RL files for testing purposes and then have a 
>>>> crack
>>>> at writing a code generation backend that targets modern JavaScript 
>>>> VMs
>>>> (e.g. produces JavaScript that best suits a JIT/tracer). If anyone 
>>>> would
>>>> like to help with any part of this process, I'd be most grateful.
>>>>
>>>> I've forked Ragel trunk into a github repo while I work on this. There
>>>> is a sample script (I translated the maillbox.rl file from the Ragel
>>>> distribution) which you can play with if you install Node.js (a
>>>> JavaScript application framework powered by Google's V8 VM). Note that
>>>> while the sample script uses Node.js for IO, but the code generated is
>>>> plain JavaScript. I look forward to being able to use Ragel in the
>>>> browser soon too.
>>>>
>>>> Github:
>>>>
>>>> http://github.com/dominicmarks/ragel-js
>>>>
>>>> A diff against Ragel trunk is here:
>>>>
>>>> http://goodforbusiness.co.uk/~dom/ragel-js-20101011.diff
>>>> <http://goodforbusiness.co.uk/%7Edom/ragel-js-20101011.diff>
>>>>
>>>> The code is mostly a cut-paste-and-replace from Ruby at the moment. 
>>>> I'll
>>>> be tidying it up over the next few weeks, as my time allows. 
>>>> However, it
>>>> does at least run one Ragel state machine correctly already.
>>>>
>>>> Mailbox.rl/Mailbox.js:
>>>>
>>>> http://goodforbusiness.co.uk/~dom/mailbox.js
>>>> <http://goodforbusiness.co.uk/%7Edom/mailbox.js> (Requires Node.js).
>>>>
>>>> Cheers,
>>>> Dom
>>>>
>>>> _______________________________________________
>>>> ragel-users mailing list
>>>> ragel-users at complang.org
>>>> http://www.complang.org/mailman/listinfo/ragel-users
>>>
>>> _______________________________________________
>>> ragel-users mailing list
>>> ragel-users at complang.org
>>> http://www.complang.org/mailman/listinfo/ragel-users
>>
>>
>> _______________________________________________
>> ragel-users mailing list
>> ragel-users at complang.org
>> http://www.complang.org/mailman/listinfo/ragel-users
>
> _______________________________________________
> ragel-users mailing list
> ragel-users at complang.org
> http://www.complang.org/mailman/listinfo/ragel-users

_______________________________________________
ragel-users mailing list
ragel-users at complang.org
http://www.complang.org/mailman/listinfo/ragel-users