How’s that for a clickbait title? It’s a mystery why internet marketing firms aren’t flooding me with high-paying job offers and consulting gigs.
I’ve realized that in my hurry to look at the stories in my 49 ranked magazines broken down by genre, by nation, by gender, and by pay, I didn’t present the entire dataset. So, here it is:
Under my interpretation of the five star system, about half these stories made me angry or gave me a headache, about a quarter were a waste of my time, and only a quarter were truly worth reading. The overall average is 2.66 stars. Clearly, I’m hoping for better results in rounds two and three.
There were seventeen magazines that had at least one five-star story. Here are the magazines that had more than one:
| Magazine | Five Star Stories |
| Andromeda Spaceways In-Flight Magazine | 4 (out of 15) |
| Red Sun Magazine | 2 (out of 4) |
| Cirsova | 2 (out of 7) |
| Perihelion SF | 2 (out of 11) |
| Storyhack | 2 (out of 9) |
I don’t feel like typing the table of magazines hosting one-star stories. Thirty-five of the forty-nine magazines contained a one-star story, and twenty-one of the magazines contained more than one. The worst offender was Galaxy’s Edge, with seven (out of ten). I’m still scratching my head over that. The magazine’s editor, Mike Resnick, wrote one of my all-time favorite short stories.
Next, I have a couple of wild hypotheses:
Imagine the editorial team at a magazine has sifted through hundreds of stories in their slush pile, and selected the twelve best for their next quarterly issue. They could put all twelve in the magazine, but wouldn’t average quality be higher if they chose the best six of those twelve? Wouldn’t it improve again if they chose the best three of the six?
To see if magazines with fewer stories had higher quality, I plotted the two factors:
While there is a (very slight) downward slope to the trend line, the data points are all over the place, with most of the far-right data points being above the line. For you stats guys, this dataset has the lowest R-squared I’ve ever seen. I’m going to say I was way off-base on my prediction.
My other prediction was that magazine editors would place their best stories at the start and end of the magazine. A good first story could hook pre-sale browsers, or purchasers who are quick to DNF. A good last story would make good reviews or word-of-mouth advertising more likely, and make the reader more likely to purchase the next issue.
Here’s my comparison of first, middle, and last stories:
| Story Position | Average Stars (out of 5) |
| First Story | 2.74 |
| Middle Stories | 2.68 |
| Last Story | 2.99 |
It looks like I was off-base again with first stories. I’ve only rated them two percent higher than stories in the middle. Last stories are rated over eleven percent higher than middle stories, so it is possible editors are saving their best for last.
Finally, I’ve always considered myself a “shorter is better” kind of guy. Once again, the data doesn’t seem to support my opinion, but this time I had the special effects team from Star Wars design the chart:
There is an upward trend here, but as in my other chart above, the data is all over the place, and the R-squared is very low.
For additional insight, I asked a child with a crayon to analyze the data:
To further clarify, I sought the assistance of an intoxicated spider:
This should be my last analysis post until I’ve collected more data. I’m planning to review a classic science fiction novel next week, a recent epic fantasy novel the week after that, and post my Planetary Awards nominations after that.
Planetary Defense Command 
I like analysis of stats, but the stats themselves baffle me. I’m more the child with a crayon kind of guy when it comes to interpreting this kind of thing. We’re lucky we have you.
Thanks, but I didn’t do any real stats work other than a simple regression using MS Excel. I didn’t even boot up my statistics software.
I once did a medical paper where two different people measured distances on CT scans of vertebrae, and ran some statistics relating to differences between measurer. I’ll have to pull those stats back out if I ever get someone else to do a side-by-side read of the same stories.
median might be a nice way to differentiate magazines—averages can get wonky if there is a mass at the high or low end.
I didn’t think a median would be very helpful because the result range is only 1 to 5. I could easily run the numbers, though, and see how much the median differs from the mean.
Also, as a reader, I’m more interested in the 5s and 1s, because that’s what I’m seeking out / trying to avoid. So, I prefer the mean, which gives them more weight.
Median is a tougher scale true. That’s what can make it handy! In theory a median 3 magazine should mean half the stories are better then three stars which would be very good. However a magazine with lots of two star fiction and a handful of five star pieces might end up averaging higher then 3 but have a median of two, i suppose the delts between median and mean is the slog factor, that might be interesting.
wow, even my average is just above 3.
Maybe that drunk spider will bite you and turn you into Alcohol Man! (not sure what his superpower is though)
But your average is for novels, which I’m starting to think are of higher average quality than short stories. Also, you get to read the blurb of a novel and filter out ones that are obviously not for you, which can’t be done as effectively with an anthology or magazine.
My goodreads average is just above 3.7, but that includes about 50% non-fiction, which I tend to rate higher than fiction. Sounds like I’ll eventually have to download that data and calculate my SFF average there.
I’m not sure what intoxicant that spider was using, I think it was something other than alcohol.
[…] a fan of shorter works, but I’ve been having bad luck with short stories recently (see the star ratings I gave to the stories in 49 magazines, my first side quest, and my second side quest). I’m being extremely unscientific by […]
[…] rating system, I’d probably give this collection three stars. That’s higher than the 2.66 average I gave out during the first round of my magazine quest, so I might consider reading the follow-up […]