Recent News

  • [March 7, 2014] MAKAO 1.7 released, fixing bugs in implicit dependency detection, circular dependency detection (currently flagged when generating the gdf, but ignored in the graph) and with weird target names (containing spaces or plus signs).
  • [August 5, 2013] MAKAO 1.6 released, fixing some small bugs.
  • [July 2, 2011] MAKAO 1.5 released, which fixes a bug with the line numbers of makefile rules.
  • [November 3, 2010] MAKAO 1.4 released, which adds support for GNU Make 3.82 and auto-detection of the version of GNU Make used. Kudos to Reinier Post for his 3.82 patch!
  • [October 26, 2010] MAKAO 1.3 released, which adds support for Windows (using Cygwin 1.7.x).
  • [June 22, 2010] MAKAO 1.2 released, which adds some small fixes and enhancements.
  • [August 7, 2009] Website published!

Screenshots

Functionality

Making sense of hundreds of Makefiles is very hard. Finding out where to modify them is even harder. Enter MAKAO, i.e., the "Makefile Architecture Kernel featuring Aspect Orientation", which uses visualization and some ideas of Aspect-Orientation Software Development (AOSD) to understand and manage build systems.

MAKAO provides the following features:

  • visualization of the build dependency graph
  • querying of build targets and dependencies for their build commands and variables
  • filtering of the build dependency graph using Prolog rules to reduce clutter or to identify build idioms
  • verification of the presence of bad smells in the build dependency graph
  • re-engineering of the build dependency graph and build scripts using aspects

MAKAO builds on GUESS (Graph Exploration System), which is a flexible and scriptable graph manipulation framework. It takes the build trace of a GNU Make or ClearMake build run as input, generates and lays out the build dependency graph, and then provides the above five features to the user. More information can be found in the tutorial and publications sections.

We would like to thank Kris De Schutter, Herman Tromp, Matthias Rieger, Cyrille Artho, Dieter Paeps, Istvan Nagy and Reinier Post for their contributions and suggestions!

PS: Apparently, MAKAO (or Mau-Mau) is the name of a popular Polish card game, but we intended the name to be an allusion on both the "Made in Macao"-statement found on products fabricated in Macao and on GNU Make of course.

Requirements

The latest version of MAKAO can be downloaded here. MAKAO is covered by the Mozilla Public License 1.1 (MPL 1.1), and runs on Linux, Max OSX and Windows (using Cygwin 1.7.x).

System requirements:

MAKAO has been tested extensively on GNU Make 3.80+, and provides preliminary support for ClearMake. As long as a build dependency graph can be extracted from your build tool, MAKAO can be used to re(verse)-engineer your build layer. Example build dependency graphs of ANTLR 2.7.6, Aspicere, CppTest 1.0.2, Gaim 2.0.0beta3, GCC 4.1.1, Inkscape 0.44, Ioquake3, Linux Kernel Graphing Project and other systems are included in the download!

Tutorial

The best documentation of MAKAO is provided by my PhD dissertation, but this tutorial helps you getting started in no time. First, download MAKAO and extract it into a directory $MAKAO.

  1. Extraction of Build Dependency Graph
  2. Laying out the Build Dependency Graph
  3. Querying of Build Dependency Graph
  4. Filtering of Build Dependency Graph
  5. Refactoring of Build Dependency Graph

1. Extraction of Build Dependency Graph

To extract the build dependency graph of a GNU Make build system, you need to invoke make using its debugging flags: -w --debug=v --debug=m -p. To make this easier, we supply a simple wrapper script called makewrapper.sh: $MAKAO/parsing/bin/makewrapper.sh all 2&> somefile.txt. For details on how to obtain the trace file with ClearMake, contact Matthias Rieger.

Make sure that every invocation of make inside the Makefiles uses the debugging flags (or the wrapper script), for example by overriding the variable used to refer to the make command or by replacing all invocations of make. The makewrapper.sh wrapper by default overrides the MAKE variable.

Once a trace of a build run has been obtained, MAKAO extracts the build dependency graph from the GNU Make or ClearMake build trace using $MAKAO/parsing/bin/generate_makao_graph.pl -in somefile.txt -out ourgraph.gdf -format gdf (GNU Make) or $MAKAO/parsing/bin/generate_makao_graph.clearmake.pl -in somefile.txt -out ourgraph.gdf -format gdf (ClearMake). In addition to extracting the build dependency graph, the extraction process also generates a .xml file with information about the commands that were executed to build each build target.

2. Laying out the Build Dependency Graph

The graph we obtained in the previous step contains the right information (i.e., build targets and dependencies), but looks like a mess because the layout is random. To fix this, issue $MAKAO/ui/bin/show.sh ourgraph.gdf -p. This starts up GUESS in Prefuse mode, which offers an interactive force-based layout mechanism.

Click on the "Start/Stop" button to let the powerful force-based layout mechanism start clustering the build dependency graph based on the density of build dependencies between build targets. After a while, slowly start to increase the "DefaultSpringLength" value until it is fully cranked up. Nodes can be dragged to another position to influence the layout, and the mouse scroll wheel can be used to zoom in and out. Once you are happy with the layout, click the "Start/Stop" button again.

Now, type export("ourgraph") in the console window at the bottom of the screen, and press enter. This will store the layouted graph as "ourgraph-piccolo.gdf". You should quit GUESS now, because we need to restart it in Piccolo mode.

3. Querying of Build Dependency Graph

The extracted, layouted build dependency graph is now ready for manipulation and querying. Open the graph using $MAKAO/ui/bin/show.sh ourgraph.gdf. Select ourgraph.gdf.xml in the file dialog, i.e., the XML file that was generated as a by-product of the extraction phase.

Once the graph is fully loaded, you can notice the following things:

  • On the left of the screen, there is a color legend indicating what color is associated with each concern. For example, red represents object files (.o).
  • The build dependency graph is shown in the largest part of the GUI, with each node colored as indicated in the legend panel. White nodes are instances of as yet unidentified concerns (can be added in $MAKAO/ui/scripts/makao.py).
  • Zoom in or out by right-clicking and dragging the mouse to the right or to the left.
  • The large colored polygons (disabled by default) are convex hulls, i.e., the smallest polygons around all build targets of a particular Makefile. Hulls are disabled since they tend to clutter up the display.
  • The bottom panel is an interactive Jython-based interpreter console that allows the user to manipulate the GUESS environment as well as the graph.

Some example commands that can be typed in the Jython console:

  • center centers the graph.
  • The name of a node corresponds to the name of a Jython object (in the OO sense) that represents that node. all.name gives you the name of target "all", provided this target exists.
  • The following expression sets the "visible" field to zero of all nodes with "error" code zero: (error==0).visible=0. In other words, this command only shows those targets that were built without any error.
  • Display -> Information Window opens another tab in the left panel in which all the fields and values of the currently selected node or edge are shown.
  • You can load in your own Jython scripts using execfile(makao_path+"path/relative/to/makao/directory/script.py").

4. Filtering of Build Dependency Graph

If the build dependency graph contains too much information, setting the "visible" field of the offending nodes to zero hides these nodes. However, to make the graphs less bloated, it makes sense to systematically remove all nodes and edges that match a particular pattern before the graph is loaded into GUESS. For this, MAKAO provides a filtering mechanism which is based on logic programming (SWI Prolog is needed for this):

  1. Generate a build trace as usual, say somefile.txt
  2. cp $MAKAO/filtering/prolog-backward/rules/gdf-sample.pl $SOMEDIR/gdf-trace.pl
  3. cp $MAKAO/filtering/prolog-backward/rules/gdf-sample-logic.pl $SOMEDIR/gdf-trace-logic.pl
  4. cp $MAKAO/filtering/prolog-backward/bin/prepare_fact_base.sh $SOMEDIR/prepare_fact_base.sh
  5. Modify the first three variable definitions of $SOMEDIR/prepare_fact_base.sh and make sure that PL_IN=$WORK_DIR/trace-facts.pl
  6. $SOMEDIR/prepare_fact_base.sh
  7. With the sample rules as inspiration, you can write logic filtering rules in $SOMEDIR/gdf-trace-logic.pl and configure which rules are active in $SOMEDIR/gdf-trace.pl (details are given in my PhD dissertation)
  8. Apply the filtering rules and show the resulting (un-layouted) graph: $MAKAO/filtering/prolog-backward/bin/show_reduced.sh trace-facts.pl target_name.
  9. Layout the graph and store the result as usual
  10. Visualize and query the graph using $MAKAO/filtering/prolog-backward/ui/show.sh some_file-piccolo.gdf target_name.

5. Refactoring of Build Dependency Graph

MAKAO can refactor Makefiles (only GNU Make) using Aspect-Orientation Software Development (AOSD) technology. The full details are given here, but this tutorial shows how to re-enact our experiments.

We want to preprocess each .c file with a custom tool before compilation. For this, we will add a call to that tool before every compiler invocation, making sure that instead of the original file the compiler gets as input the preprocessed file.

Open the build dependency graph you want to re-engineer in the Piccolo mode of GUESS. Then:

  1. execfile(makao_path+"scripts/case-cc.py")
  2. Select $MAKAO/refactoring/example as directory when prompted (create this directory if it is not there).
  3. If you check the command list of any red object file node (commands[t]), you will see that a call to the custom tool has been added before every compilation command. This has only happened on MAKAO's in-memory model of the build dependency graph, NOT on the actual Makefiles ("virtual weaving").
  4. If the results are not satisfactorily, the virtual weaving can be undone using cc_weaver.unweave_before().
  5. If you are happy with the proposed changes, it is time to modify the real Makefiles ("physical weaving") by:
    • Copying your Makefiles to $MAKAO/refactoring/examle/orig.
    • Creating an empty directory $MAKAO/refactoring/example/makao.
    • cd $MAKAO/refactoring
    • Synchronizing $MAKAO/refactoring/examle/orig and $MAKAO/refactoring/example/makao using ./bin/weave.sh example aspicere-cc reset /path/to/build/dir.
    • Doing the physical weaving using ./bin/weave.sh example aspicere-cc weave /path/to/directory/during/build.
    • To show that the Makefiles in $MAKAO/refactoring/example/makao have been modified, a diff is shown to compare with the original Makefiles in $MAKAO/refactoring/example/orig.
    • Undoing the physical weaving only requires ./bin/weave.sh example aspicere-cc unweave /path/to/directory/during/build.

This tutorial helped you getting started with MAKAO. The full documentation can be found here. If you have problems or suggestions, do not hesitate to contact me.

Publications

    • MCINTOSH, S., ADAMS, B., KAMEI, Y., NGUYEN, T. and HASSAN, A. E. (2011). An Empirical Study of Build Maintenance Effort, in Proceedings of the 33rd International Conference on Software Engineering, ICSE (Waikiki, Honolulu, Hawaii), to appear. (Acceptance ratio: 62/441=14%) BibTeX
    • MCINTOSH, S., ADAMS, B. and HASSAN, A. E. (2010). The Evolution of ANT Build Systems, in Proceedings of the 7th IEEE Working Conference on Mining Software Repositories, MSR (Cape Town, South Africa), pages 42-51. (Acceptance ratio: 16/51=31.4%) BibTeX
    • ADAMS, B. (2009). Co-Evolution of Source Code and the Build System, PhD Symposium at the 25th IEEE International Conference on Software Maintenance, ICSM (Edmonton, Canada), September 2009. BibTeX
    • ADAMS, B. (2008). Co-evolution of Source Code and the Build System: Impact on the Introduction of AOSD in Legacy Systems, PhD dissertation, ISBN 978-90-8578-203-2, Ghent University, Belgium, May 2009. BibTeX
    • ADAMS, B., DE SCHUTTER, K., TROMP, H. and DE MEUTER, W. (2008). The Evolution of the Linux Build System, in Proceedings of the ERCIM Working Group on Software Evolution 2007 (Paris, France), Electronic Communications of the ECEASST, Vol. 8., 16 p. BibTeX
    • ADAMS, B., DE SCHUTTER, K., TROMP, H. and DE MEUTER, W. (2007). Design Recovery and Maintenance of Build Systems, in Proceedings of the 23rd IEEE International Conference on Software Maintenance, ICSM, pages 114-123, IEEE Computer Society, Paris, France, October 2007. (Acceptance ratio: 46/214=21%) BibTeX
    • ADAMS, B., DE SCHUTTER, K., TROMP, H. and DE MEUTER, W. (2007). MAKAO (demo), in Proceedings of the 23rd IEEE International Conference on Software Maintenance, ICSM (Paris, France), pages 517-518, IEEE Computer Society, October 2007. BibTeX

Download

Jump to ...

My Other Tools