Preprocess Javascript on-the-fly

I am currently looking into using oj, a Objective-C like syntax for JS. Why I didn’t use Objective-J? Its not nodejs-friendly.

But oj lacks a preprocessor…but I need one. Actually, I only need one thing: #include.

I went and browsed npm, but I only found one preprocessor, but it required slashes - which does not look very clean to me. The one I found wanted me to do:

// #ifdef A
…
// #endif

I just want do something like

#include <file>

or

#ifdef A
…
#endif

If I can’t find one, I’ll just write my own…but it’s a bit sad.

Does anyone know a proper, javascript-based preprocessor? I want to use it in a middleware and preprocess the file, before giving it to oj.



cpp should not be used for non-C code.

http://web.mit.edu/ghudson/info/cpp.whynot



When I did that, I used GNU m4 for this. Maybe it'll help in your case as well.

Using GCC preprocessor isn't a bad thing either, but it'll have to be fine-tuned to do that.

Why not use the javascript VM as preprocessor? Node is about putting more javascript into javascript anyway.

Every line starting with '#' is coded in the preprocessor context, every line that isn't, is for the result.

I wrote it in a way, so line numbers are ought to be preserved unless an include( ) happens, in that case line numbers go a little amiss.

example input.js:

For example:

preprocessor.js:

-----------------------------

var fs = require( 'fs' );

var vm = require( 'vm' );

var file = fs.readFileSync( 'input.js' ).toString( );

var lines = file.split('\n');

var skipped = '';

var code = [ ];

var result;

for( var ln in lines )

{

var line = lines[ ln ];

if( line[ 0 ] === '#' )

{

lines[ ln ] = line.substr( 1 );

skipped += '\\n';

}

else

{

lines[ ln ] =

'code.push("'

+ skipped

+ line.replace( /\"/g, '\\"' ).replace( /"/g, '\\"' )

+ '");';

skipped = '';

}

}

lines.push( 'code; ');

result =

vm.runInNewContext(

lines.join( '\n' ),

{

code : code,

require : require,

include :

function( filename )

{

code.push( fs.readFileSync( filename ) );

}

}

)

.join( '\n' );

console.log( result );

-------------------

Do with result what you want, like using it in the http server module, cache it, etc. you might wrap some async code to replace readFileSync() with async alternatives in the flow control of your choice. 

------------------------

#var test = require( './test' );

#if( test ) {

console.log( '"hello"' );

#}

console.log( 'world' );

#include( 'include-me.js' );

-----------------------

example test.js :

------------------------

// just contains a yes/no switch

module.exports = true || false;

----------------------------

Handing the preprocessors own require function to the to be preprocessed file might be a little naive and have quirky effects if they use the same files, I just suppose its trusted code anyway, but for a starter it works. Might be replaced with a custom require if so desired.

One can also directly alter the 'code' array in preprocessor context to add custom stuff.

I assume you are talking about this module : https://www.npmjs.org/package/preprocess

I've updated and published 2.1.0 to accommodate unwrapped directives (when used as a library). The slashes (and other formats) are there to keep the files parsable by their destination environments regardless of whether or not they have been preprocessed.

That is a very impressive example - i wonder why thsi method is not its own module anyway =p I will try and test this! Thank you a lot for this solution. I never saw any ude for the vm module…well, now I do.

Yes, that’s the one that I found originally. So are you saying that I could use the directives without using comments at all? Because, to me, it doesn’t look like a real preprocessor if there are comment dashes infront of it - that might only be a personal taste, and due to my long work in C++ over all the last months...

Crockford offered up JSDev [0].  And as mentioned, using GNU cpp "can" have drawbacks.  Alternatively, you may want to investigate using ucpp [1] which will do C99 style preprocessing without doing parsing or compilation.

[0] https://github.com/douglascrockford/JSDev

[1] https://code.google.com/p/ucpp/

Here you have it: https://github.com/IngwiePhoenix/connect-oj/blob/master/pp.js

You can copy it, put it in your project and use it. Its simple and pretty cool. :) It does not just work with JS, but really everything. I am using it to preprocess .oj files, mainly to use #include() to import base classes.