Understanding the Catalyst Poll: Who Answered the Poll?

Analyzing a poll can be a tricky matter. As my old debating coach taught me to say, "There's Lies, Damn Lies and Statistics." Given that, polling can still be a decent way to get cheap community involvement, as long as you are careful; and caveat the results. So lets start off by trying to answer the question: Who Answered the Poll?

The poll asked a number of self identification questions. The goal of these questions is to help us understand if the people who answered the poll can be meaningfully organized into groups for the purposes of seeing likenesses and differences, and for tracking changes over time. Here's the first basic self identification question: How long have you been using Catalyst?

I gave "I don't use it" as an option both to help us measure perception of those who don't actually use it (but presumable might use it) and also to help filter responses that could very well be outliers.

People that answered the poll trend towards being long time users of Catalyst. Although having long time users is a good thing in general, I think this is a possible indicator that we're not attracting very many newcomers, or that we are not exciting them enough to care to answer the poll. I'm going to bust this down into two groups, and here's my rationale. "Old Timers" (4 or more years, people mostly around since before Moose was ported to Catalyst), and "Medium Term Users", (people 3 years or less, and came to Catalyst after the rise of Plack and are choosing Catalyst over other options like Dancer and Mojolicious). These two groups represent each about half the respondees, so its a pretty natural split.

I am not going to consider a 'Newbie' group (one year or less) because the number of respondees is too small and I don't want to draw unwarrented conclusions. I will try to tease out some possible thoughts when looking deeper into the the poll, but those thoughts have to be heavily caveated.

So we have a lot of long term users. Are these people just the type that likes to hold onto the past? Well, in order to try and understand that I asked people about their computer platform.

This was an open format question, in that you could choose as many answers as you wish. Given that we do appear to have answers weighted toward newer technologies, such as newer versions of Perl and Catalyst. In fact nearly everyone is on a Post Moose port version of Catalyst, and a clear majority on a post Plack engine port as well. Interestingly enough, if we try to break down these answers by our two groups (old timers versus medium term) I didn't find much of a different.

I'm glad to see this poll suggesting that the PSGI Engine port didn't scare people off. We did go to a lot of trouble to maintain backward compatibilty, which probably assisted. It's also good to know that the old timer group doesn't seem to just be maintaining legacy systems, and that they are either making new applications or are being successful in migrating older code to newer systems.

One last look at the data before we close this first post and move on to understand more about the feelings of the different groups we identified. We asked people about the kind of work they do:

I'm a Developer (Catalyst programmer)	259	89%
I am a Project Manager	52	18%
I am a Product Owner/Manager	58	20%
I am a Director of Engineering	23	8%
I am a C level executive (CTO, CEO, etc).	26	9%
I am a Recruiter	7	2%
I work in QA / Testing	20	7%
I lead a team of developers but I don't consider myself a project manager or director level	38	13%
I work on a team using Catalyst, but I am not a programmer	2	1%
I am a HTML/ Frontend/UI developer	48	17%
I prefer not to identify myself	3	1%
Other	23	8%

This was another open ended question, and people could answer as many as they liked. I guess its not a big surprise to see the vast, vast majority of poll respondents are developers. I'm glad we are reaching developers, but I am concerned overall that we need to do a better job reaching people in companies that use Catalyst, but in management level positions, or other general leadership positions. One thing that I'd personally like to change is to make sure that Perl is not just empowering developers but also exciting company leadership. Right now I really don't have a great way of measuring that :(

So what do we know before heading into the next post ( warning you upfront, look for that next week)? From the data I have, people that answered the poll have generally been using Catalyst for at least a moderate amount of time, and that you are choosing Catalyst over other options (at least some of the time) even though you tend to use the latest and greatest. Thats a good base to build from and to try and understand the next part of the poll (which will analyze how you feel about Catalyst). Yet I'd love to see some of you drag some newbies over to the club, if you can :)