What it does:
- Adds a “–couch-json” option to bsmedberg‘s pymake that takes a URL that crams the state of (global, non-rule associated) variables into a couch document for each makefile context executed. Variables are grouped by general origin as well as specific Makefile origin where relevant. The code as it stands only tells you about make variables that were accessed because this cuts down on things you (I) don’t care about. (For example, if FOO and BAR are defined but only FOO is ever expanded, then only FOO goes in the couch document.)
I think there’s a fair bit of potential in this (pushing “make -p” style data/make output into couch), at least in situations where there’s no escaping make, you don’t already have usable/appropriate tooling, and python/CouchDB/JSON is one of your religions.
I’m hoping to avoid touching the mozilla build system in any meaningful fashion from here on out, so I am unlikely to work on this much more, but it could be useful to others, hence this post. The repo is here, check out the “understand” (p)branch. (The changes are not suitable for upstreaming as things stand.)
Just curious, what were you thinking of doing with the data?
The immediate practical reason I did this was debugging some Makefile trickery I was doing to replace the (comm-central/)mozilla/storage/ implementation with one located in (comm-central/)storage-backport/. There was an asymmetry between what made the “make -f client.mk build” execution and the incremental “cd objdir/storage-backport; make” happy.
The main complexity there is a result of comm-central having a basically forked version of the make configuration from mozilla-central. It seemed like I could either spend a lot of time trying to puzzle out the snafu, spend a lot of time trying to manually parse out a from-clean build with the make “-p” flag, or write a tool to make it easier.
The long-term justification for the tool was that the comm-central build system, by virtue of the forking, seems to require a lot of upkeep and having improved tooling might make it easier for us to take a step back and completely eliminate the whole forking issue.
The pie-in-the-sky justification is that it’s a lot easier to parse build output if you 1) can get direct feeds of the compiler stdout without having to grok what is make output and is compiler output, especially under -j8 and 2) can store it in and process it in a structured format. This makes it easier to have tools figure out what failed, what generated warnings, etc. I think bsmedberg already has the warning problem in hand though and I think there may be various other ongoing projects to parse build output that would address similar use cases, even if they are less cool about it :).
Although making my forays into the C++ platform easier helps my productivity and efficiency (and happier), it’s much more productive and efficient for me to not touch the C++ stuff at all which is why I am planning to not do much more with this.
Oh, also, pymake was doing some weird things on windows for comm-central and I think we may have some ‘bugs’ in our mailnews makefiles in terms of expressing dependencies so that a clean build with -j8 won’t trip up.