As I mentioned in my teaser on rich thunderbird mozmill logs, in order to get the build logs and provide failure clustering you already have to do most of the stuff tinderboxpushlog does. One of the key things is summarizing the builds in a way that is useful, where the definition of “useful” probably varies a lot between users.
While Thunderbird has an extremely boring set of build types (build, xpcshell, mozmill), my first take on summarizing them was no good. While fixing it, I decided to feature creep (out of my hobby time allocation) and see if I could create a presentation that could handle the prolific Firefox and TraceMonkey trees.
While I am not going to claim it’s perfect, I like it. It’s probably also important to caveat that not all Tracemonkey builds are categorized. The mobile talos runs identify themselves by a completely different set of names from the desktop ones, and there’s not really room for columns for that. Additionally, some builds cite a revision for “mobile-browser”, but we ignore that extra meta-data. Although the design was intended to handle Thunderbird’s repository where each build is a tuple of “comm-central” and “mozilla-central” revision used, we really need to have that tuple for every build in the tinderbox tree, and TraceMonkey is not providing it. (We could kick builds without the info to the “outer” push as an enhancement.)
As a gesture of friendship to non-Thunderbird trees, we now also process mochitest and reftest logs, although I’m failing to surface some of the details retrieved in the UI.
Anywho, you can see arbpl in action for yourself at arbpl.visophyte.org. It cron scrapes every 5 minutes. The error recovery logic is not production-grade yet; the scraper can fall victim to partially written tinderbox JSON files on the tinderbox server, which means that some trees might not see updates for longer than that. And various other things may go wrong too. The client does not auto-refresh or use Socket.IO or anything. If you want to run your own, hit github. If you want to read the source, you should probably hit github too, because the production serving mode is reasonably optimized and crams all the JS into a single (gzipped) file.
Cool stuff!
> although I’m failing to surface some of the details retrieved in the UI
What exactly are you referring to here?
>> although I’m failing to surface some of the details retrieved in the UI
> What exactly are you referring to here?
We discover what the test failures are by scanning the tinderbox logs. As a result of this scanning, we have more information than just the test that failed. Especially for mochitests, we have at least one line of the good stuff from “TEST-UNEXPECTED-FAIL | file path | good stuff”. This would presumably be useful to surface.