Pre-Loading Script Libraries

If you plan to embed a Lua script library like LOOP in a C/C++ application you may prefer to generate a C library with your library's script files already pre-compiled. To do so, you may use the scripts precompiler.lua and preloader.lua provided in the LOOP package. Such scripts work as a sort of replacement for the old bin2c application that used to be included in the Lua distribution, however they are primarily intended to pre-compile and pre-load scripts that implement modules following the Lua 5.1 package model. For pre-compilation of ordinary scripts check out bin2c or other implementations of the old bin2c application. Anyway, the scripts provided in the LOOP package are very general and may be used to generate pre-compiled libraries for any set of Lua scripts.

WARNING:

The current version of these scripts require class Argument Processor to handle command-line options. So it might be necessary to make the LOOP library available in the LUA_PATH. Alternatively, you may simply redefine your commands as:

lua -epackage.path=[[<LOOP_HOME>/?.lua]] <LOOP_HOME>/precompiler.lua

lua -epackage.path=[[<LOOP_HOME>/?.lua]] <LOOP_HOME>/preloader.lua

Anyway, these scripts are very easy to read and modify, so feel free to change them in any way to better fit your demands.

On the other hand, these concerns are not valid if you installed LOOP's rock through LuaRocks. In such case, you can use these scripts simply through commands precompiler.lua and preloader.lua.

Pre-Compilation

To generate pre-compiled libraries, first select the script files that should be included in the library. Next, use the precompiler.lua script to generate the C source file that includes the pre-compiled scripts. This script receives a list of optional parameters and the path of all the script files that should be pre-compiled. The path to these script files must follow the same hierarchy of the packages that they implement so the package name shall be correctly inferred from the provided Lua path. The options precompiler.lua script accepts are the following:

-b, -bytecodes: Flag that indicates the provided arguments are file paths containing bytecodes (e.g. instead of source code), like the output of the luac compiler. When this flag is used no compilation is performed by this script.
-d, -directory: Directory where the output files should be generated. Its default is the current directory.
-l, -luapath: Sequence os path templates used to infer package names from file paths and vice versa. These templates follows the same format of the package.path field of Lua. Its default is the value of package.path.
-n, -names: Flag that indicates provided input names are actually package names and the real file path should be inferred from the path defined by -luapath option. This flag can be used in conjunction with the -bytecodes flag to indicate that inferred file paths contains bytecodes instead of source code.
-o, -output: Name used to form the name of the files generated. Two files are generated: a source code file with the sufix .c with the pre-compiled scripts and a header file with the suffix .h with function signatures. Its default is precompiled.
-p, -prefix: Prefix added to the signature of the functions generated. Its default is LUAOPEN_API.

Example

> lua precompiler.lua -o loop -l lua/?.lua \
 lua/loop/base.lua \
 lua/loop/cached.lua \
 lua/loop/multiple.lua \
 lua/loop/scoped.lua \
 lua/loop/simple.lua \
 lua/loop/table.lua \
 lua/loop/collection/MapWithArrayOfKeys.lua \
 lua/loop/collection/ObjectCache.lua \
 lua/loop/collection/OrderedSet.lua \
 lua/loop/collection/PriorityQueue.lua \
 lua/loop/collection/UnorderedArray.lua \
 lua/loop/collection/UnorderedArraySet.lua \
 lua/loop/compiler/Arguments.lua \
 lua/loop/compiler/Conditional.lua \
 lua/loop/compiler/Expression.lua \
 lua/loop/debug/Inspector.lua \
 lua/loop/debug/Matcher.lua \
 lua/loop/debug/Verbose.lua \
 lua/loop/debug/Viewer.lua \
 lua/loop/object/Exception.lua \
 lua/loop/object/Wrapper.lua \
 lua/loop/serial/FileStream.lua \
 lua/loop/serial/Serializer.lua \
 lua/loop/serial/SocketStream.lua \
 lua/loop/serial/StringStream.lua \
 lua/loop/thread/CoSocket.lua \
 lua/loop/thread/IOScheduler.lua \
 lua/loop/thread/Scheduler.lua \
 lua/loop/thread/SocketScheduler.lua \
 lua/loop/thread/Timer.lua

The package name is inferred by matching the provided path with the Lua path patterns defined with option -luapath and no path expansion is performed. Therefore precompiler.lua won't be able to figure out the package name of file mypacks/loop/base.lua if the provided Lua path is /home/user/mypacks/?.lua even though the current directory is /home/user. However if the Lua path is set to mypacks/?.lua the inferred package name would be loop.base. When the Lua path defines multiple patterns, the last (i.e. rightmost) that matches the file path is used to define the package name.

Alternatively, you shall provide the flag -names to precompiler.lua and provide package names instead of file paths. In such case, the provided Lua path is used to figure out the file that contains the actual package implementation. This is useful when package name cannot be correctly inferred from the file path like in case-insensitive file systems. Therefore the above example could be rewritten like below:

Example

> lua precompiler.lua -o loop -l lua/?.lua -n \
 loop.base \
 loop.cached \
 loop.multiple \
 loop.scoped \
 loop.simple \
 loop.table \
 loop.collection.MapWithArrayOfKeys \
 loop.collection.ObjectCache \
 loop.collection.OrderedSet \
 loop.collection.PriorityQueue \
 loop.collection.UnorderedArray \
 loop.collection.UnorderedArraySet \
 loop.compiler.Arguments \
 loop.compiler.Conditional \
 loop.compiler.Expression \
 loop.debug.Inspector \
 loop.debug.Matcher \
 loop.debug.Verbose \
 loop.debug.Viewer \
 loop.object.Exception \
 loop.object.Wrapper \
 loop.serial.FileStream \
 loop.serial.Serializer \
 loop.serial.SocketStream \
 loop.serial.StringStream \
 loop.thread.CoSocket \
 loop.thread.IOScheduler \
 loop.thread.Scheduler \
 loop.thread.SocketScheduler \
 loop.thread.Timer

Additionally, if the flag -bytecodes is used with the precompiler.lua script then it assumes that the provided files contain bytecodes instead of source code. This is useful to create source files with scripts pre-compiled with luac in platforms other than the one where the application is being written.

Finally, if no additional parameter is provided other than the options listed above, then the file paths or package names are read from the standard input. Therefore, if the list of package to be compiled is stored in file loop.lpk and they are already compiled in directory compiled in files with extension .lo that follow the same hierarchy of the packages then the compilation command could be:

Example

> lua precompiler.lua -o loop -l compiled/?.lo -n -b < loop.lpk

As a result of the examples above you will get the files loop.c and loop.h that can be compiled to produce a library that offers functions that load each one of the scripts pre-compiled. Such functions follow the pattern defined by the Lua package model. Examples are:

int luaopen_loop_base(lua_State*);
int luaopen_loop_simple(lua_State*);
int luaopen_loop_collection_ObjectCache(lua_State*);
int luaopen_loop_collection_OrderedSet(lua_State*);

Note: Since these function signatures comply with the standard defined by Lua package model then loop.c can be used to generate a dynamic C library that exports all functions defined in loop.h and is able to load all LOOP packages if properly installed in the directory for C packages (see field package.cpath).

Pre-Loading

However, to effectively load such scripts in an application, you have to call these operations provided by the library in the right loading sequence that respects each package dependencies. Alternatively, you may pre-load such scripts by registering these functions in the package.preload table so they are executed when each package is first required. This can be automatically done by the code generated with script preloader.lua. This script receives a list of optional parameters and the path of the headers of libraries that should be pre-loaded. The options preloader.lua script accepts are the following:

-d, -directory: Directory where the output files should be generated. Its default is the current directory.
-I, -i, -includes: Adds a directory to the list of paths where the header files of pre-compiled libraries are searched.
-f, -funcname: Name of the generated function that pre-loads all library modules. Its default is luapreload_ plus the name defined by option -output.
-n, -names: Flag that indicates provided input names are actually package names and not header files.
-o, -output: Name used to form the name of the files generated. Two files are generated: a source code file with the sufix .c with the pre-loading code and a header file with the suffix .h with the function that pre-loads the scripts. Its default is preload.
-p, -prefix: Prefix added to the signature of the functions generated. Its default is LUAPRELOAD_API.

Example

> lua preloader.lua -o looplib -f luapreload_loop loop.h

As a result you will get the files looplib.c and looplib.h that can be compiled with the files loop.c and loop.h to produce a library that offers a function with the following signature:

int luapreload_loop(lua_State*);

This function registers all the functions defined by loop.h header with the names of the corresponding packages. This way, function luapreload_loop can be used to pre-load all scripts previously compiled in loop.c so they are automatically loaded when required.

The preloader.lua script also works with header files of Lua libraries written in C/C++. For example, to generate a library that pre-loads the full LuaSocket library, including its script files you can use the following commands.

> lua precompiler.lua -o luasocketscripts -l "?.lua" \
 socket.lua \
 socket/ftp.lua \
 socket/http.lua \
 socket/smtp.lua \
 socket/tp.lua \
 socket/url.lua

> lua preloader.lua -o fullluasocket -I <LUASOCKET_HOME>/include \
 luasocket.h \
 luasocketscripts.h

These commands produces files luasocketscripts.c, luasocketscripts.h, fullluasocket.c, and fullluasocket.h that shall be compiled with the C source files of the LuaSocket library to produce a library that provides operation int luapreload_fullluasocket(lua_State*) that may be used to pre-load into the provided Lua state the full LuaSocket library, including its script files.

Alternatively, you can provide the names of the packages that should be pre-loaded instead of header files that contains signature of the package opening functions. To do so, just provide the flag -names to the preloader.lua script. This is useful when package name cannot be inferred correctly from the luaopen_* function. For example a package named my_lib.my_package is loaded by function luaopen_my_lib_my_package that preloader.lua script would register under the name my.lib.my.package. To avoid this, use the package name like in the example below.

> lua preloader.lua -o mylib -n my_lib.my_package

Similar to the precompiler.lua script, if no additional argument is provided then the list of header files or package names is read from the standard input.