June ended up being quite a productive month. Many of the ideas from the 2014 QA Hackathon have come to fruition, and I've been updating the sites appropriately. Earlier this month I sorted out several blog posts for the updates, even though the updaes themselves had been live for a few weeks.
 
The CPAN Testers Statistics had plenty of updates, mostly due to ideas from or inspired by Neil Bowers, but also inclding some additional files in the form of backpan100.csv and cpan100.csv. These two files contain the source of data for the Acme-CPANAuthors-CPAN-OneHundred and Acme-CPANAuthors-BACKPAN-OneHundred distributions. A celebration of those authors who have contributed to CPAN with over 100 different distributions. The other updates to the statistics site included the inclusion of Submission Rates, and the lists of New Distributions being uploaded to CPAN. Both sets of pages and graphs, help to show that Perl and CPAN are far and above from their alledged hey days of the early 2000s, and are growing significantly every single year. We are attracting new authors, and they in turn are contributing some great new distributions to CPAN.

The CPAN Testers Admin has now been launched officially, even if the soft launch happened a few months ago. There has been a slight downside to the release, in that the author login doesn't appear to be working. Andreas and I have been looking into it, but have yet to understand why it fails. We'll keep you posted once we have it working. In the meantime all the tester side of the site is working. So if you are a tester and want to claim your addresses, and verify your credentials, including your prefered contact email address, you can now do so.

Alberto Simões recently posted two issues regarding missing dependency tests. The first relates to how a smoker can continually retry to test a distribution, even though the same prerequisite always fails. With help from Alexandr Ciornii, it appears that Alberto's initial problem was solved by setting 'halt_on_failure' to '0', rather than the default of '1'. However, it is worth noting that CPANPLUS::YACSmoke does this by default, as the original design was to prevent unnecessarily testing distributions that could never reach a satisfactory result to produce a report, until the dependency changed versions.

Alberto's second issue centred around the same scenario, but how authors could be notified. However, both David Golden and myself felt this doesn't warrant an NA report as suggested, as the majority of authors are not interested, and many would feel they are powerless to change the report, particularly if it did not accurately represent the testing on a particular platform. The offending prerequisite is tested and a report sent as appropriate, so the core problem is still reported, but perhaps what is needed is a better way to inform authors of this downstream. At least, for those who wanted to know, and would find it useful. My suggestion is a mashup of the CPAN Testers reports, CPAN Dependencies and PerlModules.net, such that authors can sign up to receive reports, similar to the CPAN Testers Preferences site, and get alerts when there are failures for these distributions. If I get some time to write it, I will, but if anyone else fancies creating a new addition to the CPAN Testers family, feel free to get in touch.

On the mailing list, Yves Orton asked why the build output was not included alongside the test output for FAIL reports. In the example provided, it would have helped Yves spot the problem. Without the build output it becomes a little harder to debug, particularly when building XS modules, which this example was. David Golden explained that a decision was made some time ago to exclude the build output to help cut down on the size of reports, as we once had a cut-off when sending via email. As that cut-off doesn't exist any more, the output could potentially be included in the report, perhaps just for XS module distributions. However, currently the logic in CPAN-Reporter is to only report the last failing phase. As the build phase here only reported a warning, it didn't report an exit value of 0, and as such wasn't considered a failure. David has asked that a ticket be created with details, and he will hopefully find some time to change the logic to remember the build phase output, should a smoke test fail in a later phase.

Alberto Simões also asked about a problem he encountered with the compiler being set as ' cc', with a leading space character. Following several posts by others, it was established that a space was quite legal to precede the compiler. From this Alberto was able to resolve the problem, and has been able to move on to the next problem :) Daniel Staal asked about whether tester comments in reports could be configurable. He has a rather unusual setup, and what to have a way to post this into the tester comments automatically. Sadly the use of CPAN::Reporter::Smoke doesn't allow for this currently, but David Golden did follow-up saying he was open to patches to enable the comments to be added from a file.

Recently I had cause to contact Ben Bullock and happened to spot a page on his website, Perl XS modules and CPAN testers. This is a great resource, and thanks to Ben for compiling it. It is very useful to have a knowledge base somewhere to record errors and most importantly what they mean. In 2005 I put together a talk, Preparing For CPAN, that looked at my experiences of testing modules and particularly in regards to portability. It would be great to see a similar updated talk at a future YAPC or Workshop, taking current experiences, and explaining as Ben does, what particular errors mean.

A tweet by Salve J Nilsen highlighted that CPAN has now passed the 30,000 mark, of the number of unique distributions on CPAN. Just in case anyone was in any doubt that CPAN was growing! A great achievement for one and all.

After many years of promising to get the CPAN Testers Admin site online, I'm pleased to say it's finally here. You can visit the site at admin.cpantesters.org, where you can login as a tester or an author, select reports and mark them to be deleted and submit them to the respective tester to action.

If you are a tester please review the requests and if the report truly is dubious please delete. If you have any questions as to why an author might have requested the report be deleted, please contact them for more clarification. If you are aware of your smoker going off the rails and submitting bogus reports, this site will enable you to delete them before the author highlights them.

If you are an author, please don't request deletions for reports that you just dont like. This service is meant to help clear out reports that are submitted where the smoker or environment is faulty in some way, and the report it produces does not truly represent the report grade. Requesting reports to be delete simply because you don't want to see any red failures in your pages, will be rejected.

In truth, reports are not actually deleted, they are merely set to be ignored, and the build system will extract them from the reports site so they do not appear in pages or JSON files.

However, the Admin site is more than just a report deletion service. It also allows testers to claim emails (particularly many of their old emails), and set their Name and PAUSE identities in the Statistics Leaderboards. In this regard, please note that the Leaderboard is currently only built once a day, so it may take 24 hours before you see any changes.

In addition to this, the tester can also include a preferred contact address. This may not be one used for testing, but is the one they can receive emails for. As older emails become defunct, it is important that testers keep this email address relevant, to ensure anyone wishing to contact them can do so. Note that this email address will be publicly visable via the Find A Tester service, and will be shown on "raw" reports via the Reports website. If you are a CPAN Author, you may wish this to be your pause email address.

A few testers/authors have already been invited to take a look and hopefully we've ironed out the kinks, but if you do spot any perculiarities, please let me know.

In time there may be other features added to the site, so if you have suggestions for improvements, feel free to file issues on GitHub or in RT.

For a few months now, Slaven Rezić has been asking me to look at the way the Find A Tester tool keeps bring back an old email address of his, rather than the current one. The problem is the way that Metabase now stores the tester reference for a report. Instead of a specific email address it stores a resource fact, that just happens to be a tester. The tester resource fact then has a number of email addresses associated with it, except it has no way of knowing a tester's preference for which email address is most suitable. As such the list I retrieve, is essentially random. There isn't a good way to distinguish between new and old address.

In my workings on the CPAN Admin site (more news on that very soon), I spotted a gap in the knowledge a tester can tell us about themselves. Their current contact address. In many cases, particularly if they are also a CAPN author, this could be their PAUSE email address (AUTHOR@cpan.org), but equally could be a central email address they prefer to be contacted on. This then enables the tester to keep their contact information current, even if they change jobs. The mechanism behind the scenes now uses this, if available, as the primary addresss to display in the Find A Tester tool, but also now in the "raw" reports on the Reports website, and in the Reports Mailer tool that sends out the summary and report mails to authors.

There is still more to be changed to ensure that the contact address is considered the primary address, but hopefully it will get easier and more reliable to contact testers.

To find out how to update your contact address, please see tomorrow's post :)

I was trying to keep track of what distributions I released for my monthly challenge quest, and it made me think about when I released the first version of each distribution I have on CPAN. This then got me thinking about the first versions everyone had on CPAN. Seeing as I was already adding to the statistics, I thought I might as well add some more.

And so now you can see all the first version releases to CPAN. It also helps to keep track of all those first versions that happening each month, in case you wanted to check out what new cool stuff your favourite author was releasing to CPAN. So far this month we've had  21 new distributions (at the time of writing), and we're only 3 days into the month (the stats run in the early hours of the morning for the preceeding days).

However, that then got me thinking about the popularity of version numbers, and in particular what version numbers authors used for their first release of a distribution to CPAN. Unsurprisngly 0.01 is the most popular, but it turns out there is quite a vareity before we get down to single figures. Going that little bit further, looking at all the version numbers used, it was interesting to see there is a reasonably uniform trend, certainly for the top 14 positions.

If you have any ideas for more CPAN statistics, please post them on Github or RT.

Back at the 2014 QA Hackathon in March, Neil Bowers asked about the frequency of testers submitting reports. We looked at a few different stats, and he blogged about it in How soon do CPAN Testers start testing your releases?. However, he also gave me the idea to look some other statistics to do with when authors release their distributions.

This idea is now available as the CPAN Submission Rates pages and graphs on the CPAN Testers Statistics site. When I have a bit more free time, I'm going to look at providing the preceeding few ranges, so that for day of the week, I'll include the previous 4 weeks individually. This will hopefully show the fluctuations in each period a little more clearly. However, as a first step it does show that as authors we are pretty much consistent with our releases over the lifetime of CPAN.

The one graph that does show a noticeable difference is the hour of release. I had expected this to be reasonable even, but the fact that there is a dip around 05:00 UTC and two peaks around 15:00 UTC and 21:00 UTC did suprise me. I'm guessing the first peak equates with European submissions and the second with American submissions. However, we have plenty of authors in Australia, Japan, China and Korea, so I am suprised that there isn't a similar peak somewhere between 03:00 UTC and 05:00 UTC. It might be interesting to filter the results using the wealth of Acme::CPANAuthors::* modules to figure out what periods each group favours. If anyone fancies patching CPAN::Testers::WWW::Statistics, or even setting up their own statistics page for that, please let me know.

If you have any more ideas for these sorts of statistical analysis, please post to GutHub or RT.