A week after The 2012 RPG Podcast Listener Survey went live (and if you haven’t taken it, click that link to the left before you go any further!), and we’re nearly at the number of respondents we had last year. Total.
Last year we had 973 respondents (with some spam entries removed from the real survey) and this year we have 840 respondents, which means we’re about 86.3% of the way to matching our goal from last year, in less that 13% of the total time that we’re running the survey.
We made some changes to the survey this year, and having watched the responses trickle in over the last few days, I had some thoughts about some of the things I’m seeing in the survey already. I’ll avoid posting some specific thoughts I have (I can see the results already, you know) that I think might prompt people to skew results, but some general overview thoughts shouldn’t have too much impact, I wouldn’t think.
Number and Identity of Respondents
Last year, I barely snaked the completion of the software we use to track data under the wire. This year, I had a little more foresight and a little more motivation – my masters’ thesis is about RPG Podcasts and lo, this is a perfect avenue for firsthand research – so I was able to put some more time into building both an interface that’s pleasant to look at and has the functionality to deliver real-time analysis options.
While that may not sound exciting, I was thrilled to be able to really watch as the responses poured in and let me break down the responses without having to export the full list to Excel and do a bunch of manipulation every time. Having it inline has been a fascinating way to keep me obsessively checking the site every hour.
But, numbers without any sort of goal is somewhat useless. So, I made a prediction on the number of responses we would get per day. My suspicion was that we would get a large burst as the initial social media push started on a Tuesday (and I didn’t expect the RPG Podcasters list to have pre-dropped a promo in their episodes), that it would drop off, and then we would see a slight rise in the second week as non-social media listeners discovered the link through show notes or host recommendations.
In day 8 of the survey, I’m a little off, but I had to do a little digging to really find out where and how I was off.
You can see from the graph that the survey technically started on 4/30 when I released our episode and didn’t do the math right on the date. The first bunch of responses there was from the initial Podge release and subsequent social media burst. If I hadn’t messed that up, those would have appeared on 5/1, like they should have, as part of the larger hump.
After the two day blitz, there’s a pretty steep drop, which then jumps back up on 5/7. I assumed that I was right, and that was from shows having promoted it on their feeds and we were picking up listeners who didn’t interact with shows on social media. This wasn’t the case.
Yesterday, Jason who runs the Being a Geek group on Facebook, posted the link to that group. While Facebook doesn’t tell me how many people subscribe to that group, it does tell me that 34,000 some odd people are “talking about this”. That’s a lot of people.
I have mixed feelings about this. On one hand, my original intent with the survey (aside from proving Joe wrong) was to get something to measure just how far the average reach was in the RPG Podcasting community. At the time, I considered myself someone who had at least an average reach, and I was curious what that number was. While this puts that number a lot higher, it also brings in several people who are self-identifying as not being regular listeners to RPG Podcasts.
Is this a problem? I can come up with reasonable justifications for both, I suppose.
Next year I’m building a FAQ.
I took the pool of available, selectable podcasts from RPGPodcasts.com because there isn’t another way to reliably do it. As far as I’m concerned, RPGPodcasts.com is the authority when it comes to RPG Podcasts. If a show isn’t listed there, then I don’t think it’s unreasonable not to include the show on the survey.
I tried to make this clear to the RPG Podcasters group, but there’s a bit of a process problem there. I don’t control the access to that podcasting board, so new shows that sign up for RPG Podcasts have to manually request access from the Yahoo Group (which I can’t grant), and I’m not sure how often that gets done.
We also chopped out shows that haven’t released an episode in the last year. Hard dates from 4/30/2011 on. Also, we had a single show request to be removed from RPGPodcasts.com, so I have removed them from the survey.
I built sourcing information into the URLs that people use to jump to the website this year so that we can track the names and locations of where people are referred to the software. I’m looking forward to seeing just how much of the survey comes from social media links vs. show links.
Any thoughts on the survey thus far from anyone else? The comments on this site are little more suited for long form, public feedback or you can use this thread.