The best documentation of MAKAO is provided by my PhD dissertation, but this tutorial helps you getting started in no time. First, download MAKAO and extract it into a directory $MAKAO.
- Extraction of Build Dependency Graph
- Laying out the Build Dependency Graph
- Querying of Build Dependency Graph
- Filtering of Build Dependency Graph
- Refactoring of Build Dependency Graph
1. Extraction of Build Dependency Graph
To extract the build dependency graph of a GNU Make build system, you need to invoke
make using its debugging flags:
-w --debug=v --debug=m -p. To make this easier, we supply a simple wrapper script called makewrapper.sh:
$MAKAO/parsing/bin/makewrapper.sh all 2&> somefile.txt. For details on how to obtain the trace file with ClearMake, contact Matthias Rieger.
Make sure that every invocation of
make inside the Makefiles uses the debugging flags (or the wrapper script), for example by overriding the variable used to refer to the
make command or by replacing all invocations of
make. The makewrapper.sh wrapper by default overrides the
Once a trace of a build run has been obtained, MAKAO extracts the build dependency graph from the GNU Make or ClearMake build trace using
$MAKAO/parsing/bin/generate_makao_graph.pl -in somefile.txt -out ourgraph.gdf -format gdf (GNU Make) or
$MAKAO/parsing/bin/generate_makao_graph.clearmake.pl -in somefile.txt -out ourgraph.gdf -format gdf (ClearMake). In addition to extracting the build dependency graph, the extraction process also generates a .xml file with information about the commands that were executed to build each build target.
2. Laying out the Build Dependency Graph
The graph we obtained in the previous step contains the right information (i.e., build targets and dependencies), but looks like a mess because the layout is random. To fix this, issue
$MAKAO/ui/bin/show.sh ourgraph.gdf -p. This starts up GUESS in Prefuse mode, which offers an interactive force-based layout mechanism.
Click on the "Start/Stop" button to let the powerful force-based layout mechanism start clustering the build dependency graph based on the density of build dependencies between build targets. After a while, slowly start to increase the "DefaultSpringLength" value until it is fully cranked up. Nodes can be dragged to another position to influence the layout, and the mouse scroll wheel can be used to zoom in and out. Once you are happy with the layout, click the "Start/Stop" button again.
export("ourgraph") in the console window at the bottom of the screen, and press enter. This will store the layouted graph as "ourgraph-piccolo.gdf". You should quit GUESS now, because we need to restart it in Piccolo mode.
3. Querying of Build Dependency Graph
The extracted, layouted build dependency graph is now ready for manipulation and querying. Open the graph using
$MAKAO/ui/bin/show.sh ourgraph.gdf. Select ourgraph.gdf.xml in the file dialog, i.e., the XML file that was generated as a by-product of the extraction phase.
Once the graph is fully loaded, you can notice the following things:
- On the left of the screen, there is a color legend indicating what color is associated with each concern. For example, red represents object files (.o).
- The build dependency graph is shown in the largest part of the GUI, with each node colored as indicated in the legend panel. White nodes are instances of as yet unidentified concerns (can be added in $MAKAO/ui/scripts/makao.py).
- Zoom in or out by right-clicking and dragging the mouse to the right or to the left.
- The large colored polygons (disabled by default) are convex hulls, i.e., the smallest polygons around all build targets of a particular Makefile. Hulls are disabled since they tend to clutter up the display.
- The bottom panel is an interactive Jython-based interpreter console that allows the user to manipulate the GUESS environment as well as the graph.
Some example commands that can be typed in the Jython console:
center centers the graph.
- The name of a node corresponds to the name of a Jython object (in the OO sense) that represents that node.
all.name gives you the name of target "all", provided this target exists.
- The following expression sets the "visible" field to zero of all nodes with "error" code zero:
(error==0).visible=0. In other words, this command only shows those targets that were built without any error.
- Display -> Information Window opens another tab in the left panel in which all the fields and values of the currently selected node or edge are shown.
- You can load in your own Jython scripts using
4. Filtering of Build Dependency Graph
If the build dependency graph contains too much information, setting the "visible" field of the offending nodes to zero hides these nodes. However, to make the graphs less bloated, it makes sense to systematically remove all nodes and edges that match a particular pattern before the graph is loaded into GUESS. For this, MAKAO provides a filtering mechanism which is based on logic programming (SWI Prolog is needed for this):
- Generate a build trace as usual, say somefile.txt
cp $MAKAO/filtering/prolog-backward/rules/gdf-sample.pl $SOMEDIR/gdf-trace.pl
cp $MAKAO/filtering/prolog-backward/rules/gdf-sample-logic.pl $SOMEDIR/gdf-trace-logic.pl
cp $MAKAO/filtering/prolog-backward/bin/prepare_fact_base.sh $SOMEDIR/prepare_fact_base.sh
- Modify the first three variable definitions of
$SOMEDIR/prepare_fact_base.sh and make sure that
- With the sample rules as inspiration, you can write logic filtering rules in
$SOMEDIR/gdf-trace-logic.pl and configure which rules are active in
$SOMEDIR/gdf-trace.pl (details are given in my PhD dissertation)
- Apply the filtering rules and show the resulting (un-layouted) graph:
$MAKAO/filtering/prolog-backward/bin/show_reduced.sh trace-facts.pl target_name.
- Layout the graph and store the result as usual
- Visualize and query the graph using
$MAKAO/filtering/prolog-backward/ui/show.sh some_file-piccolo.gdf target_name.
5. Refactoring of Build Dependency Graph
MAKAO can refactor Makefiles (only GNU Make) using Aspect-Orientation Software Development (AOSD) technology. The full details are given here, but this tutorial shows how to re-enact our experiments.
We want to preprocess each .c file with a custom tool before compilation. For this, we will add a call to that tool before every compiler invocation, making sure that instead of the original file the compiler gets as input the preprocessed file.
Open the build dependency graph you want to re-engineer in the Piccolo mode of GUESS. Then:
$MAKAO/refactoring/example as directory when prompted (create this directory if it is not there).
- If you check the command list of any red object file node (
commands[t]), you will see that a call to the custom tool has been added before every compilation command. This has only happened on MAKAO's in-memory model of the build dependency graph, NOT on the actual Makefiles ("virtual weaving").
- If the results are not satisfactorily, the virtual weaving can be undone using
- If you are happy with the proposed changes, it is time to modify the real Makefiles ("physical weaving") by:
- Copying your Makefiles to
- Creating an empty directory
./bin/weave.sh example aspicere-cc reset /path/to/build/dir.
- Doing the physical weaving using
./bin/weave.sh example aspicere-cc weave /path/to/directory/during/build.
- To show that the Makefiles in
$MAKAO/refactoring/example/makao have been modified, a diff is shown to compare with the original Makefiles in
- Undoing the physical weaving only requires
./bin/weave.sh example aspicere-cc unweave /path/to/directory/during/build.
This tutorial helped you getting started with MAKAO. The full documentation can be found here. If you have problems or suggestions, do not hesitate to contact me.