287 lines
11 KiB
Markdown
287 lines
11 KiB
Markdown
---
|
|
title: "QMake hackery: Dependencies & external preprocessing"
|
|
date: November 13, 2011
|
|
author: Chris Hodapp
|
|
tags:
|
|
- Project
|
|
- Technobabble
|
|
---
|
|
* TODO: Put the code here into a Gist?
|
|
|
|
[Qt Creator](http://qt-project.org/wiki/Category:Tools::QtCreator) is
|
|
a favorite IDE of mine for when I have to deal with miserably large
|
|
C++ projects. At my job I ported a build in Visual Studio of one such
|
|
large project over to Qt Creator so that builds and development could
|
|
be done on OS X and Linux, and in the process, learned a good deal
|
|
about [QMake](http://doc.qt.nokia.com/latest/qmake-manual.html) and
|
|
how to make it do some unexpected things.
|
|
|
|
While I find Qt Creator to be a vastly cleaner, lighter IDE than
|
|
Visual Studio, and find QMake to be a far more straightforward build
|
|
system for the majority of things than Visual Studio's build system,
|
|
some things the build needed were very tricky to set up in QMake. The
|
|
two main shortcomings I ran into were:
|
|
|
|
* Managing dependencies between projects, as building the application
|
|
in question involved building 40-50 separate subprojects as
|
|
libraries, many of which depended on each other.
|
|
* Having external build events, as the application also had to call an
|
|
external tool (no, not `moc`, this is different) to generate some
|
|
source files and headers from a series of templates.
|
|
|
|
QMake, as it happens, has some commands that actually make the project
|
|
files Turing-complete, albeit in a rather ugly way. The `eval`
|
|
command is the main source of this, and I made heavy use of it.
|
|
|
|
First is the dependency management system. It's a little large, but I'm including it inline here.
|
|
|
|
```bash
|
|
# This file is meant to be included in from other project files, but it needs
|
|
# a particular context:
|
|
# (1) Make sure that the variable TEMPLATE is set to: subdirs, lib, or app.
|
|
# Your project file really should be doing this anyway.
|
|
# (2) Set DEPENDS to a list of dependencies that must be linked in.
|
|
# (3) Set DEPENDS_NOLINK to a list of dependencies from which headers are
|
|
# needed, but which are not linked in. (Doesn't matter for 'subdirs'
|
|
# template)
|
|
# (4) Make sure BASEDIR is set.
|
|
#
|
|
# This script may modify SUBDIRS, INCLUDEPATH, and LIBS. It should always add,
|
|
# not replace.
|
|
# It will halt execution if BASEDIR or TEMPLATE are not set, or if DEPENDS or
|
|
# DEPENDS_NOLINK reference something not defined in the table.
|
|
#
|
|
# Order does matter in DEPENDS for the "subdirs" template. Items which come
|
|
# first should satisfy dependencies for items that come later.
|
|
# You'll often see:
|
|
# include ($$(BASEDIR)/qmakeDefault.pri)
|
|
# which includes this file automatically.
|
|
#
|
|
# -CMH 2011-06
|
|
|
|
# ----------------------------------------------------------------------------
|
|
# Messages and sanity checks
|
|
# ----------------------------------------------------------------------------
|
|
message("Included Dependencies.pro!")
|
|
message("Dependencies: " $$DEPENDS)
|
|
message("Dependencies (INCLUDEPATH only): " $$DEPENDS_NOLINK)
|
|
#message("TEMPLATE is: " $$TEMPLATE)
|
|
|
|
isEmpty(BASEDIR) {
|
|
error("BASEDIR variable is empty here. Make sure it is set!")
|
|
}
|
|
isEmpty(TEMPLATE) {
|
|
error("TEMPLATE variable is empty here. Make sure it is set!")
|
|
}
|
|
|
|
# ----------------------------------------------------------------------------
|
|
# Table of project locations
|
|
# ----------------------------------------------------------------------------
|
|
|
|
# Some common locations, here only to shorten descriptions in the _PROJ table.
|
|
_PROJECT1 = $$BASEDIR/SomeProject
|
|
_PROJECT2 = $$BASEDIR/SomeOtherProject
|
|
_DEPENDENCY = $$BASEDIR/SomeDependency
|
|
|
|
# Table of project file locations
|
|
# (Include paths are also generated based off of these)
|
|
_PROJ.FooLib = $$_PROJECT1/Libs/FooLib
|
|
_PROJ.BarLib = $$_PROJECT1/Libs/BarLib
|
|
_PROJ.OtherStuff = $$_PROJECT2/Libs/BarLib
|
|
_PROJ.MoreStuff = $$_PROJECT2/Libs/BarLib
|
|
_PROJ.ExternalLib = $$BASEDIR/SomeLibrary
|
|
|
|
# ----------------------------------------------------------------------------
|
|
# Iterate over dependencies and update variables, as appropriate for the given
|
|
# template type
|
|
# ----------------------------------------------------------------------------
|
|
|
|
# _valid is a flag telling whether TEMPLATE has matched anything yet
|
|
_valid = false
|
|
|
|
contains(TEMPLATE, "subdirs") {
|
|
for(dependency, DEPENDS) {
|
|
# Look for an item like: _PROJ.(dependency)
|
|
|
|
# Disclaimer: I wrote this and it works. I have no idea why precisely
|
|
# why it works. However, I repeat the pattern several times.
|
|
eval(_dep = $$"_PROJ.$${dependency}")
|
|
isEmpty(_dep) {
|
|
error("Unknown dependency " $${dependency} "!")
|
|
}
|
|
|
|
# If that looks okay, then update SUBDIRS.
|
|
eval(SUBDIRS += $$"_PROJ.$${dependency}")
|
|
}
|
|
message("Setting SUBDIRS=" $$SUBDIRS)
|
|
_valid = true
|
|
}
|
|
|
|
contains(TEMPLATE, "app") | contains(TEMPLATE, "lib") {
|
|
# Loop over every dependency listed in DEPENDS.
|
|
for(dependency, DEPENDS) {
|
|
# Look for an item like: _PROJ.(dependency)
|
|
eval(_dep = $$"_PROJ.$${dependency}")
|
|
isEmpty(_dep) {
|
|
error("Unknown dependency " $${dependency} "!")
|
|
}
|
|
|
|
# If that looks okay, then update both INCLUDEPATH and LIBS.
|
|
eval(INCLUDEPATH += $$"_PROJ.$${dependency}"/include)
|
|
eval(LIBS += -l$${dependency}$${LIBSUFFIX})
|
|
}
|
|
for(dependency, DEPENDS_NOLINK) {
|
|
# Look for an item like: _PROJ.(dependency)
|
|
eval(_dep = $$"_PROJ.$${dependency}")
|
|
isEmpty(_dep) {
|
|
error("Unknown dependency " $${dependency} "!")
|
|
}
|
|
|
|
# If that looks okay, then update INCLUDEPATH.
|
|
eval(INCLUDEPATH += $$"_PROJ.$${dependency}"/include)
|
|
}
|
|
#message("Setting INCLUDEPATH=" $$INCLUDEPATH)
|
|
#message("Setting LIBS=" $$LIBS)
|
|
_valid = true
|
|
}
|
|
|
|
# If no template type has matched, throw an error.
|
|
contains(_valid, "false") {
|
|
error("Don't recognize template type: " $${TEMPLATE})
|
|
}
|
|
```
|
|
|
|
It's been sanitized heavily to remove all sorts of details from the
|
|
huge project it was taken from. Mostly, you need to add your dependent
|
|
projects into the "Table of Project Locations" section, and perhaps
|
|
make another file that set up the necessary variables mentioned at the
|
|
top. Then set the `DEPENDS` variable to a list of project names, and
|
|
then include this QMake file from all of your individual projects (it
|
|
may be necessary to include it pretty close to the top of the file).
|
|
|
|
In general, in this large application, each sub-project had two
|
|
project files:
|
|
|
|
* One with `TEMPLATE = lib` (a few were `app` instead as well). This
|
|
is the project file that is included in as a dependency from any
|
|
project that has `TEMPLATE = subdirs`, and this project file makes
|
|
use of the QMake monstrosity above to set up the include and library
|
|
paths for any dependencies.
|
|
* One with `TEMPLATE = subdirs`. The same QMake monstrosity is used
|
|
here to include in the project files (of the sort in #1) of
|
|
dependencies so that they are built in the first place, and permit
|
|
you to build the sub-project standalone if needed.
|
|
|
|
...and both are needed if you want to be able to build sub-project
|
|
independently and without making to take care of dependencies
|
|
individually.
|
|
|
|
The next project down below sort of shows the use of that QMake
|
|
monstrosity above, though in a semi-useless sanitized form. Its
|
|
purpose is to show another system, but I'll explain that below it.
|
|
|
|
```bash
|
|
QT -= gui
|
|
QT -= core
|
|
TEMPLATE = lib
|
|
|
|
## Include our qmake defaults
|
|
DEPENDS = FooLib BarLib
|
|
include ($$(BASEDIR)/qmakeDefault.pri)
|
|
|
|
TARGET = Project$${LIBSUFFIX}
|
|
LIBS += -llua5.1 -lrt -lLua$${LIBSUFFIX}
|
|
DEFINES += PROJECT_EXPORTS
|
|
|
|
INCLUDEPATH += /usr/include/lua5.1
|
|
./include
|
|
|
|
HEADERS += include/SomeHeader.h
|
|
include/SomeOtherHeader.h
|
|
|
|
SOURCES += source/SomeClass.cpp
|
|
source/SomeOtherClass.cpp
|
|
|
|
# The rest of this is done with custom build steps:
|
|
GENERATOR_INPUTS = templates/TemplateFile.ext
|
|
templates/OtherTemplate.ext
|
|
|
|
gen.input = GENERATOR_INPUTS
|
|
gen.commands = $${DESTDIR}/generator -i $${QMAKE_FILE_IN}
|
|
# -s source$(InputName).cpp -h include$(InputName).h
|
|
|
|
# Set the destination of the source and header files.
|
|
SOURCE_DIR = "source/"
|
|
HEADER_DIR = "include/"
|
|
# What prefix and suffix to replace with paths and .h.cpp, respectively.
|
|
TEMPLATE_PREFIX = "external/"
|
|
TEMPLATE_EXTN = ".ext"
|
|
|
|
#
|
|
# Warning: Here be black magic.
|
|
#
|
|
# We need to use QMAKE_EXTRA_COMPILERS but its functionality does not give us
|
|
# an easy way to explicitly specify the names of multiple output files with a
|
|
# single QMAKE_EXTRA_COMPILERS entry. So, we get around this by making one
|
|
# entry for each input template (the .ext files).
|
|
# The part where this gets tricky is that each entry requires a unique
|
|
# variable name, so we must create these variables dynamically, which would
|
|
# be impossible in QMake ordinarily since it does only a single eval pass.
|
|
# Luckily, QMake has an eval(...) command which explicitly performs an eval
|
|
# pass on a string. We repeatedly use constructs like this:
|
|
# $$CONTENTS = "Some string data"
|
|
# $$VARNAME = "STRING"
|
|
# eval($$VARNAME = $$CONTENTS)
|
|
# These let us dynamically define variables. For sanity, I've tried to use a
|
|
# suffix of _VARNAME on any variable which contains the name of another
|
|
# variable.
|
|
#
|
|
|
|
# Iterate over every filename in GENERATOR_INPUTS
|
|
for(templatefile, GENERATOR_INPUTS) {
|
|
# Generate the name of the header file.
|
|
H1 = $$replace(templatefile, $$TEMPLATE_PREFIX, $$HEADER_DIR)
|
|
HEADER = $$replace(H1, $$TEMPLATE_EXTN, ".h")
|
|
# Generate the name of the source file.
|
|
S1 = $$replace(templatefile, $TEMPLATE_PREFIX, $$SOURCE_DIR)
|
|
SOURCE = $$replace(S1, $$TEMPLATE_EXTN, ".cpp")
|
|
# Generate unique variable name to populate & pass to QMAKE_EXTRA_COMPILERS
|
|
QEC_VARNAME = $$replace(templatefile, ".", "")
|
|
QEC_VARNAME = $$replace(QEC_VARNAME, "/", "")
|
|
VARNAME = $$replace(QEC_VARNAME, "\", "")
|
|
# Append _INPUT to generate another variable name for the input filename
|
|
INPUT_VARNAME = $${QEC_VARNAME}_INPUT
|
|
eval($${INPUT_VARNAME} = $$templatefile)
|
|
|
|
# Now generate an entry to pass to QMAKE_EXTRA_COMPILERS.
|
|
eval($${VARNAME}.commands = $${DESTDIR}/generator -i ${QMAKE_FILE_IN} -s ${QMAKE_FILE_OUT} -h $${HEADER})
|
|
eval($${VARNAME}.name = $$VARNAME)
|
|
# ACHTUNG! The 'input' field is the _variable name_ which contains the
|
|
# input filename, not the filename itself. If you put in a filename or
|
|
# either of those variables don't exist, this will fail, silently, and
|
|
# all attempts at diagnosis will lead you nowhere.
|
|
eval($${VARNAME}.input = $${INPUT_VARNAME})
|
|
eval($${VARNAME}.output = $${SOURCE})
|
|
eval($${VARNAME}.variable_out = SOURCES)
|
|
|
|
# Now tell QMake to actually do this step we meticulously built.
|
|
eval(QMAKE_EXTRA_COMPILERS += $$VARNAME)
|
|
# Also add our header files. I doubt it's really necessary, but here it is.
|
|
HEADERS += $${HEADER}
|
|
}
|
|
```
|
|
|
|
This one uses a bit more black magic. The entire `GENERATOR_INPUTS`
|
|
list is a set of files that are inputs to an external program that is
|
|
called to generate some code, which then must be built with the rest
|
|
of the project. This uses undocumented QMake features, and a couple
|
|
kludges to generate some things dynamically (i.e. the filenames of the
|
|
generated code) from a variable-length list. I highly recommend
|
|
avoiding it. However, it does work.
|
|
|
|
These two links proved indispensable in the creation of this:
|
|
|
|
[QMake Variable Reference](http://qt-project.org/doc/qt-4.8/qmake-variable-reference.html)
|
|
|
|
[Undocumented qmake](http://www.qtcentre.org/wiki/index.php?title=Undocumented_qmake)
|