June ended up being quite a productive month. Many of the ideas from the 2014 QA Hackathon have come to fruition, and I've been updating the sites appropriately. Earlier this month I sorted out several blog posts for the updates, even though the updaes themselves had been live for a few weeks.
 
The CPAN Testers Statistics had plenty of updates, mostly due to ideas from or inspired by Neil Bowers, but also inclding some additional files in the form of backpan100.csv and cpan100.csv. These two files contain the source of data for the Acme-CPANAuthors-CPAN-OneHundred and Acme-CPANAuthors-BACKPAN-OneHundred distributions. A celebration of those authors who have contributed to CPAN with over 100 different distributions. The other updates to the statistics site included the inclusion of Submission Rates, and the lists of New Distributions being uploaded to CPAN. Both sets of pages and graphs, help to show that Perl and CPAN are far and above from their alledged hey days of the early 2000s, and are growing significantly every single year. We are attracting new authors, and they in turn are contributing some great new distributions to CPAN.

The CPAN Testers Admin has now been launched officially, even if the soft launch happened a few months ago. There has been a slight downside to the release, in that the author login doesn't appear to be working. Andreas and I have been looking into it, but have yet to understand why it fails. We'll keep you posted once we have it working. In the meantime all the tester side of the site is working. So if you are a tester and want to claim your addresses, and verify your credentials, including your prefered contact email address, you can now do so.

Alberto Simões recently posted two issues regarding missing dependency tests. The first relates to how a smoker can continually retry to test a distribution, even though the same prerequisite always fails. With help from Alexandr Ciornii, it appears that Alberto's initial problem was solved by setting 'halt_on_failure' to '0', rather than the default of '1'. However, it is worth noting that CPANPLUS::YACSmoke does this by default, as the original design was to prevent unnecessarily testing distributions that could never reach a satisfactory result to produce a report, until the dependency changed versions.

Alberto's second issue centred around the same scenario, but how authors could be notified. However, both David Golden and myself felt this doesn't warrant an NA report as suggested, as the majority of authors are not interested, and many would feel they are powerless to change the report, particularly if it did not accurately represent the testing on a particular platform. The offending prerequisite is tested and a report sent as appropriate, so the core problem is still reported, but perhaps what is needed is a better way to inform authors of this downstream. At least, for those who wanted to know, and would find it useful. My suggestion is a mashup of the CPAN Testers reports, CPAN Dependencies and PerlModules.net, such that authors can sign up to receive reports, similar to the CPAN Testers Preferences site, and get alerts when there are failures for these distributions. If I get some time to write it, I will, but if anyone else fancies creating a new addition to the CPAN Testers family, feel free to get in touch.

On the mailing list, Yves Orton asked why the build output was not included alongside the test output for FAIL reports. In the example provided, it would have helped Yves spot the problem. Without the build output it becomes a little harder to debug, particularly when building XS modules, which this example was. David Golden explained that a decision was made some time ago to exclude the build output to help cut down on the size of reports, as we once had a cut-off when sending via email. As that cut-off doesn't exist any more, the output could potentially be included in the report, perhaps just for XS module distributions. However, currently the logic in CPAN-Reporter is to only report the last failing phase. As the build phase here only reported a warning, it didn't report an exit value of 0, and as such wasn't considered a failure. David has asked that a ticket be created with details, and he will hopefully find some time to change the logic to remember the build phase output, should a smoke test fail in a later phase.

Alberto Simões also asked about a problem he encountered with the compiler being set as ' cc', with a leading space character. Following several posts by others, it was established that a space was quite legal to precede the compiler. From this Alberto was able to resolve the problem, and has been able to move on to the next problem :) Daniel Staal asked about whether tester comments in reports could be configurable. He has a rather unusual setup, and what to have a way to post this into the tester comments automatically. Sadly the use of CPAN::Reporter::Smoke doesn't allow for this currently, but David Golden did follow-up saying he was open to patches to enable the comments to be added from a file.

Recently I had cause to contact Ben Bullock and happened to spot a page on his website, Perl XS modules and CPAN testers. This is a great resource, and thanks to Ben for compiling it. It is very useful to have a knowledge base somewhere to record errors and most importantly what they mean. In 2005 I put together a talk, Preparing For CPAN, that looked at my experiences of testing modules and particularly in regards to portability. It would be great to see a similar updated talk at a future YAPC or Workshop, taking current experiences, and explaining as Ben does, what particular errors mean.

A tweet by Salve J Nilsen highlighted that CPAN has now passed the 30,000 mark, of the number of unique distributions on CPAN. Just in case anyone was in any doubt that CPAN was growing! A great achievement for one and all.