This document describes the build system of OX App Suite. It is intended for app developers who use the build system to create apps as well as for OX App Suite developers who not only use the build system but also may wish to extend it.
The build system is used to create source archives from source files. These source archives can be used to compile installable packages for various Linux distributions. The build system can generate archives for the core UI as well as for independently installed apps.
The OX App Suite build system uses Jake, a port of Rake from Ruby to Node.js. Both Rake and Jake are dependency-based build systems like Make, which allows quick incremental builds. But unlike Make, Jake doesn't have its own syntax. Instead, it provides an API in JavaScript. The API is used not only to specify dependencies between files using a full programming language, but also to implement the generation of files in the same language. This allows easy implementation of complex build systems, of which the OX App Suite build system is an example. Using the same language for the developed project and for its build system also allows any core developer to quickly extend the build system without having to switch to another language.
While easily extensible, most of the time the build system will be used as-is. This chapter describes how to set up and use the build system to develop apps and the OX App Suite core.
All command examples are given for the Debian operating system. The instructions should work similarly on other POSIX systems. The first character of each command indicates whether it should be executed as root (#) or as a normal user ($).
The build system comes in two variants: as part of the OX App Suite source, and as a Software Development Kit (SDK). The SDK contains only the build system and can be installed as a package. Its only external dependency is Node.js, which should be installed automatically by your package manager. While the core of OX App Suite is supposed to be built using the included version of the build system, both the source and the SDK can be used to build external (i.e. independently installable) apps.
The actual installation depends on the chosen variant:
First, if not already done, add the Open-Xchange repository to the list of Apt sources.
#
echo deb http://software.open-xchange.com/products/appsuite/\ stable/appsuiteui/DebianSqueeze/ / >> /etc/apt/sources.list.d/ox.list
#
aptitude update
Then, the actual installation is a single command.
#
aptitude install open-xchange-appsuite-dev
Finally, the main executable of the build system should be put in the executable
path for easier calling. This is typically done by either extending
the $PATH
environment variable
$
export PATH=$PATH:/opt/open-xchange-appsuite-dev/bin
or copying or symlinking the binary to a directory already in the path.
#
ln -s /opt/open-xchange-appsuite-dev/bin/build-appsuite /usr/local/bin
Using the source variant can avoid the requirement for root permissions. But,
when building external apps and the build system executable is not invoked
using its full path, the environment variable $OX_APPSUITE_DEV
needs to be set to specify the path to the build system.
$
git clone https://git.open-xchange.com/git/wd/frontend/web
$
export OX_APPSUITE_DEV=$(pwd)/web/ui
Just like with the SDK variant, the executable should be put in the executable
path either by extending the $PATH
variable
$
export PATH=$PATH:$OX_APPSUITE_DEV/bin
or by copying or symlinking the executable.
#
ln -s $OX_APPSUITE_DEV/bin/build-appsuite /usr/local/bin
The build system is executed by invoking the command
build-appsuite
. Similar to most build systems, the build system can
perform multiple tasks, which are specified as parameters on the command line.
Each task can require any number of parameters. These parameters can be
specified either on the command line, using the syntax name=value
,
or as environment variables.
If present, the file local.conf
is sourced by a shell script before
the build process starts. This file can export environment variables which are
specific to the local system, without checking them into a version control
system. Typically, it defines values for builddir
and
debug
.
Since the build system is based on Jake, it also accepts all other
Jake options.
In addition, the environment variable $nodeopts
can be used to pass
command line parameters to Node.js. One of the most useful parameters is
--debug-brk
, which can be used to debug the build system.
When developing external apps, the build system must be run from the top
directory of the app's source. As a safety precaution, execution is aborted if
the subdirectory apps
, which usually contains JavaScript source
code, is not found.
The build system is used not only to create source archives for packaging. It can also directly install and update the built UI in a directory during development, help with the setup of the source of a new external app and more. While all of these tasks are described in the reference section, daily work involves just a few of them.
The first task which should be executed when creating a new external app is
init-packaging
. It creates packaging metadata in the current
directory. It requires the parameter package
to specify the package
name in the generated files. All subsequent tasks will automatically extract
the package name from the files generated by init-packaging
.
Before execution, the apps
subdirectory must be created.
It indicates that the current directory is actually a valid source directory.
$
mkdir apps
$
build-appsuite init-packaging package=example-app Build path: build Build version: 7.0.0-1.20130111.105125 Version [7.0.0]: 0.0.1 Maintainer (Name <e-mail>): Maintainer <maintainer@example.com> Copyright line [2016 OX Software GmbH]: License name [CC-BY-NC-SA-3.0]: Short description: Hello World app to demonstrate usage of the build system
The task presents a number of interactive prompts to get the necessary
information about the generated packages. The entered values should follow
the
Debian Maintainer's Guide. Some or even all prompts can be skipped by
explicitly specifying the information as a build variable. The list of variable
names is available in the reference of init-packaging
.
After the task is finished, the generated files can be customized manually to account for any additional packaging requirements.
The main workhorse task is app
. It is used to process all source
files to an optimized deployable app. During development, it is usually called
automatically by an IDE or a directory monitoring tool. Since the builds are
incremental, the task can be called after every source modification.
For quickest roundtrip times, the directory with the generated files of the app should be made available through the OX App Suite back end by overlaying an existing OX App Suite installation, for example, by symlinking it.
#
ln -s $(readlink -m build/apps/com.example) \ /opt/open-xchange/appsuite/apps/com.example
To make the manifest accessible to the backend, it should be symlinked into the
directory specified by the com.openexchange.apps.manifestPath
setting (from manifests.properties
) of a locally installed
OX App Suite backend.
#
ln -s $(readlink -m build/manifests/example-app.json) \ /opt/open-xchange/appsuite/manifests/example-app.json
Now, any changes can be tested by reloading OX App Suite in the browser, and any changes in the app manifest can be applied by restarting the backend.
WARNING: Do not use build variables builddir
or
manifestDir
when developing an external app to generate the files
directly in their final destination! The clean
task will delete
these directories and all their contents! In general, don't point
builddir
or any other *Dir
variables at existing
directories.
As an example, let's create a small app and build it. It requires only two
files: apps/com.example/main.js
for the source code of the app
and apps/com.example/manifest.json
for the manifest.
Building the app is as easy as calling
$
build-appsuite app
The default
task is used instead of the app
task when
building the core OX App Suite. Since it is the default Jake task, it is not
necessary to specify it on the command line when it's the only task.
The top directory of OX App Suite source code includes the script
build.sh
, which should be used instead of calling a potentially
unrelated version of build-appsuite
. The script changes the current
directory to its own, so that it can be called from any directory. Furthermore,
it sets BASEDIR
, so that $OX_APPSUITE_DIR
is neither
used nor necessary.
$
./build.sh
The build system uses dependencies and file timestamps to decide which files to
rebuild. This assumes that any change to a file increases its timestamp to a
value which is greater than the timestamp of any existing file. When this
assumption is violated (e.g. after switching to a different source control
branch with older files) it may become necessary to rebuild everything to
restore the assumption about timestamps. The simplest way to achieve this is
the clean
task, which simply deletes all generated files.
WARNING: This can be potentially dangerous, since the clean
task
simply deletes the directories specified by the variables builddir
,
destDir
, l10nDir
, manifestDir
, and
helpDir
.
When the app is ready to be shipped, or rather all the time on a continuous build system, the app needs to be packaged in a format suitable for installation on a production system. Since there already exist tools to create packages from suitably arranged source code archives, the OX App Suite build system merely prepares such source archives.
The dist
task creates an archive with the source (the one ending in
.orig.tar.bz2
) and a few additional files necessary for Debian
packaging. RPM packages can be generated using the same source archive and
the .spec
file created by init-packaging
. The version
of the package is extracted from the newest entry in the file
debian/changelog
.
Debian packages can also be generated manually either from the temporary
directory left behind by dist
, or even directly from the source
tree. The second option pollutes the source tree with generated files, so it is
not recommended, although the .gitignore
file created by
init-packaging
can handle these generated files.
$
build-appsuite dist
$
ls tmp/packaging/ example-app-0.0.1 example-app_0.0.1-1.dsc example-app_0.0.1-1.debian.tar.bz2 example-app_0.0.1.orig.tar.bz2
$
cd tmp/packaging/example-app-0.0.1/
$
dpkg-buildpackage -b
The top directory of the build system.
Used by: all tasks.
Default: installation directory of the build system
Required to build external apps, since in this case, the build
system is not installed in the current directory. This variable is
automatically set as an environment variable by the build system
executable based on $OX_APPSUITE_DEV
.
The Subversion branch of the CLDR to checkout.
Used by: update-i18n
.
The target directory for generated files.
Used by: app
, clean
,
default
, dist
, docs
,
jakedeps
.
Default: build
The copyright line to be included in packaging metadata.
Used by: init-packaging
.
Example: 2016 OX Software GmbH
Enables a debug build.
Used by: app
, default
.
Default: false
To simplify debugging of OX App Suite, compression of source code
can be disabled by specifying on
, yes
,
true
or 1
.
Used by: init-packaging
.
The description of the app to be included in packaging metadata.
Output directory for source archives created by
the dist
task.
Used by: clean
, dist
.
Default: tmp/packaging
Removes all "use strict"
directives from processed
JavaScript code.
Used by: app
, default
.
Default: false
Some debugging tools which use code instrumentation have problems
when the debugged code uses strict mode. This setting enables code
processing even whe using debug
mode, so line numbers
will not match the original source code.
The root of the printed dependency tree between Jake tasks.
Used by: jakedeps
.
The location of online help files.
Used by: clean
, default
,
dist
.
Default: same as builddir
If the value contains the string @lang@
, it will be
replaced by the lowercase language code (e.g. en-us
) to
allow per-language directories.
The location of compiled l10n files.
Used by: app
, clean
,
default
, dist
.
Default: same as builddir
If the value contains the string @lang@
, it will be
replaced by the lowercase language code (e.g. en-us
) to
allow per-language directories.
File name of the full text of the distribution license.
Used by: init-packaging
.
Default: based on licenseName
.
Name of the distribution license to be included in packaging metadata.
Used by: init-packaging
.
Default: CC-BY-NC-SA-3.0
The location of the combined manifest file.
Used by: app
, clean
,
default
, dist
.
Default: same as builddir
Name and email address of the package maintainer.
Used by: init-packaging
.
Format: Name <email>
The name of the package for the built app.
Used by: all tasks.
Default: the package name in the first line of
debian/changelog
Since the name of the manifest file contains the package name and it
is required to determine build dependencies, the package name must
be always known. This means either debian/changelog
must exist and contain at least one entry, or the parameter must be
explicitly specified.
Reverses the direction of printed dependencies.
Used by: deps
.
When specified, the deps
task prints modules which
depend on the specified modules, instead of modules on which
the specified module depends.
Revision number of the package for the app.
Used by: app
, default
,
dist
.
Default: 1
The revision number must increase with each rebuild of the same version to enable the creation of unique version strings. These are required in package names and to control content caching in clients.
Specifies for which module to print the dependencies.
Used by: deps
.
Default: print all roots
If specified, only the dependencies of the specified module are printed. Otherwise, the dependencies of all modules are printed.
Whether to skip the generation of Debian source packages.
Used by: dist
.
Default: false
This is useful when Debian packages are not required and/or
dpkg-source
is not available, e.g. on RPM based
systems.
Even when using this flag, at least the file
debian/changelog
is still required, because it is used
to store the package name and version.
Whether to skip the preprocessing of LessCSS files.
Used by: app
, default
.
Default: false
This flag skips the generation of CSS files in
the apps/themes/*/less
directories of all themes.
It is used by the packaging system, where the LessCSS files are
precompiled after installation on the target system instead of
while building the package.
The Subversion tag of the CLDR to checkout.
Used by: update-i18n
.
The leaf task in the printed dependency path between Jake tasks.
Used by: jakedeps
.
When specified, only a single path from the root to the leaf task is printed (in reverse order).
Version number of the app.
Used by: app
, default
,
dist
, init-packaging
.
Default: 0.0.1
The version should consist of a major, minor and patch version separated by dots.
An up-to-date list of tasks can be printed using the -T command line option.
Builds an external app.
BASEDIR
, builddir
,
debug
, disableStrictMode
,
l10nDir
, manifestDir
,
package
, revision
,
version
.
This is the main task used to build external apps.
It works without explicitly specifying any variables, but during
development, builddir
is usually pointed to
the directory of a locally installed OX App Suite UI to avoid
additional copying steps. For debugging, debug
is also
often used. Note that clean
must be called when
changing any of the variables.
Removes all generated files.
BASEDIR
, builddir
,
destDir
, l10nDir
,
manifestDir
, package
.
This task should be executed before a normal build using
app
or default
after changing any build
variables and after a switch between Git branches. Normal
incremental builds can miss changed files if a branch switch
replaces files by older versions.
Builds OX App Suite.
BASEDIR
, builddir
,
debug
, disableStrictMode
,
l10nDir
, manifestDir
,
package
, revision
,
version
.
This is the main task to build OX App Suite. Since it is the default Jake task, it does not need to be specified explicitly on the command line when it is the only task.
It works without explicitly specifying any variables, but during
development, builddir
is usually pointed to
a directory which is accessible to a local web server. For
debugging, debug
is also often used. Note that
clean
must be called when changing any of
the variables.
Prints module dependencies.
BASEDIR
, package
,
reverse
, root
.
This task visualizes dependencies of RequireJS modules. It prints
a tree of dependencies to stdout
.
If root
is specified, then only the dependencies of
that module are printed. Otherwise, the dependencies of all modules,
on which no other module depends are printed in sequence.
If reverse
is specified, then this task prints
dependants instead of dependencies, i.e. modules which depend on
the specified module instead of modules on which the specified
module depends.
Creates source packages.
BASEDIR
, builddir
,
destDir
, l10nDir
,
manifestDir
, package
,
revision
, skipDeb
,
version
.
This task cleans the source tree by calling clean
and
packs the source into an archive which can be used to create Debian
and RPM packages. The necessary Debian metadata is created alongside
the source archive. All files necessary for Debian packaging are
placed in the directory specified by destDir
.
The generated .orig.tar.bz2
archive can also be used
together with the .spec
file to generate RPM packages.
Unless the variable skipDeb
is set to
true
, the program dpkg-source
is required
by this task. It is used to generate Debian-specific packaging
metadata.
The variables version
and revision
specify
the version of the package which will be built from the created
files, and should therefore be specified explicitly.
Generates developer documentation.
BASEDIR
, builddir
,
package
.
This task generates developer documentation for OX App Suite.
The generated HTML files are put into
builddir
/doc
.
Initializes packaging information for a new app.
BASEDIR
, copyright
,
description
, license
,
licenseName
, maintainer
,
package
, version
.
This task is the first task called when starting with
the development of a new external app. The directory from which it
is called must already contain at least the apps
subdirectory. This is also the only task where
the package
variable must be specified explicitly.
Afterwards, the package name is looked up automatically in the file
debian/changelog
, which is created by this task.
The variables version
, maintainer
,
copyright
, licenseName
,
license
, and description
are required to
fill out all necessary packaging metadata. Any of these variables
which are not specified explicitly will cause an interactive prompt.
This avoids the need to remember the list of variables before one
can start developing an app.
Shows the dependency chain between two Jake tasks.
BASEDIR
, from
,
package
, to
.
This task visualized dependencies between Jake tasks. The variable
from
specifies the root of a dependency tree, with all
dependencies of from
as inner and leaf nodes. If
to
is not specified, then that entire tree is printed.
If to
is also specified, then only the first found
dependency path from from
to to
is
.printed with to
at the top and from
at
the bottom.
Updates all .po
files with the generated
ox.pot
.
BASEDIR
, builddir
,
debug
, package
, revision
,
version
.
This task updates the list of extracted i18n strings in
ox.pot
and calls the GNU Gettext tool
msgmerge
for every language in i18n/*.po
.
Updates CLDR data in the source tree.
BASEDIR
, branch
,
package
, tag
.
This task downloads data from the Unicode CLDR and updates date
translations for all languages in i18n/*.po
.
The exact version of CLDR data is specified as Suubversion tag or
branch by the variables tag
and branch
,
respectively. If neither is specified, then the Subversion trunk is
checked out. If both are specified, tag
is used.
Generates a documentation skeleton for extension points.
BASEDIR
, package
.
This task is still under development. Currently, it creates a list of all extension points, which have a constant name in the source code. In the future it is supposed to update an existing list instead of overwriting it, and to handle non-constant extension point names.
The list of extension points is stored as an HTML snippet in
doc/extensionpoints.html
.