Recently Gabor Szabo has been tweeting about his climb up the CPAN Testers Leaderboard. It was something that Damian Learns Perl also picked up on his recent CPAN tester post. It was also exactly the kind of healthy competition I had in mind to encourage people to become CPAN Testers. However, is there anything more we can do to not only attract testers in the first place, but also keep them submitting reports? Do we need to raise the profile of CPAN Testers?

On the CPAN Testers Discussion mailing list, Gabor has been suggesting some ways to promote CPAN Testers, and get more regular testers involved. While I don't have any problems with encouraging anyone to get more involved with CPAN Testers, I still want to keep those occasional and even one-time testers still willing to submit as many reports as they feel comfortable with, which some have seen as a problem. Here is why I believe it's not a problem.

Firstly, occasional and one-time testers often only submit reports when they install modules for their currently working or development environment, which may well be significantly infrequent. This does not mean that their contributions are any less significant though. They are testing on real machines in real life environments, and can often run into to problems that automated testers may not experience. This is particularly true for distributions which rely on libraries not installed on automated smoker environments.

Secondly, the statistics only give a picture based on the user profile. The leaderboard doesn't make any distinction between individual testers and automated smoker networks. As a consequence, someone like Chris Williams, who has several dedicated automated smoker environments, distorts the weighting. The trend would likely be a little more even if we were able to calculate the number of reports from each unique smoker environment. Many of the testers who appear to have submitted only one report, in many cases have submitted others. Although I have put in a lot of effort to consolidate multiple email addresses into a single profile, there are many email addresses which are no longer used, and for which I am unable to pair up to an existing profile.

Thirdly the leaderboard as it stands is for the whole of CPAN Testers history. It took nearly 10 years to reach 1 million reports, while now we have over 500,000 reports submitted each month. The distribution of tester contributions is very different over the last few years than it was in the early years.

However, while it's not a problem, it would be much nicer to be able to encourage more of these low volume testers to submit more reports more frequently, as their contributions are valuable. This primarly means trying to understand why some testers have only posted a few reports and then have choosen not to continue. Is there some reason for this that we can resolve? Has the change to the HTTP submissions confused people and they've not continued?

I'd also like to make it as easy as possible to install the required software, both for infrequent testing as well as for more automated environments. To a large degree we already have this covered, with various instructions on the Wiki, although I'd be delighted if we could encourage contributions to explain more scenarios.

Gabor also asked how we can encourage companies to run smoke testing on their machines, particular during any out-of-hours downtime. I'm not sure of the best way to do this, as many companies have usage policies that prevent the use of machines for this kind of purpose. For those that do want to contribute and want to support CPAN Testers, what is the best way we can help them?

There has also been the suggestion to create logos and banners to promote CPAN Testers, which can be used both by companies and individuals on their websites and blogs. As a starting point we have the smoking onion, which I would like to keep as a brand for CPAN Testers, but if you have graphic skills, I would be very interested for anyone to create some logos and banners that build on this. I'd like to be able to create a page with a variety of images which people can then use to link back to us and show their support for CPAN Testers.

So if you have any ideas for promoting CPAN Testers, or have 133t graphics skills to create some images, please get in touch via the CPAN Testers Dicussion mailing list and share your ideas with the CPAN Testers community.