Differences between Crusader Strike Talents

A short look at how the talents that modify Crusader Strike impact DPS in logs.

I'm going to use abbreviations in this post for each talent build, partly because it's easier to write and partly because the distinction between "Crusader Strike" and "Crusading Strikes" is not really immediately clear to the reader.

TS - Templar Strikes

CSAA - Crusading Strikes

Base CS - Baseline Crusader Strike, ie. not using either TS or CSAA

TL;DR

If you've just clicked on this because you want to know what talent to play: the difference between TS and CSAA is small and you should play the one you enjoy more. If you really care about minmaxing DPS, CSAA is better. Base CS has a decent chance of being the better "alternative build to CSAA" than TS going forward.

Background

I decided to write a post examining the gap between the CS modifying talents since it's easily the most common question I see asked in the Paladin discord. It's not really obvious what the actual difference between the talents is on DPS to the average player, who will often just see top logs and assume that CSAA is the only viable talent choice. I decided to get data from logs to try and do some analysis on the data available to get a better idea on what the difference actually is.

Quick disclaimer: I am not super experienced when it comes to statistical analysis, so there is a decent chance that there could be problems with what I'm doing to analyze this data. If you spot something I'm doing obviously wrong please let me know, and I'll make a followup post correcting any issues if necessary.

I decided to only use raid logs when doing this analysis. This is mostly because it's presented in a way that is very easy to use for analysis - you could theoretically do the same thing for Mythic+, but you'd have to decide whether you wanted to try to figure out the effect of talents on overall key DPS, or score, or some combination metric of the two, and it's also very difficult to actually get data on talents used in Mythic+. Raider.io has some talent data, but it's unreliable since it usually just displays the talents that a player logged out in, and while there are Mythic+ logs on Warcraftlogs too, most players will not upload Mythic+ logs at anything close to the same frequency they do for raid logs. If there is significant interest I could potentially try to do some similar analysis using Mythic+ logs, but I am not planning to currently.

Data

The sample I used was the top 500 ranks for each boss (except Assault, for obvious reasons) on the 10th of July. This data is from prior to 10.1.5, but I don't expect the results to be significantly different in the new patch - the only real major change is that you'll replace the Annulet with another ring, which should not have a significant effect on the difference between CS talents. Unfortunately I don’t have the technological skills to use an API to get data, so I am pretty much limited to what is available from the basic rankings page. The data that I could easily get from the regular rankings page was: DPS, item level, date, fight length, number of external buffs, and CS talent used. There are a ton of factors that can influence DPS a lot that I could not easily check, including:

- Type of external buff: Power Infusion and Blessing of Autumn are not equally valuable, but I did not have an easy way to check how many of each external was given to a player rather than just the total number.

- Windfury Totem: It would not be that difficult to check if an Enhancement player is in the raid, but it's a lot more work to check whether the Ret was actually in the Windfury group. Note that this will not impact CSAA damage at all, as it does not benefit from Windfury, but it will have an effect on all TS/base CS logs.

- Other talents: TS and base CS logs are infrequent enough for it to be reasonable to check those by hand, but I did not check any talents besides these. There were certainly some bizarre talent builds people used if you go far enough down the rankings, but they've just been categorized by the CS talent.

- Padding: This isn't a problem for most bosses, but if you want to you can definitely pad on Forgotten Experiments, Neltharion, or Sarkareth. Just looking at boss damage in these situations isn't really a great solution either, since they all also have adds that actually do need to be focused over the boss at times. I don't expect padding to significantly affect any of the results.

- Wings procs: Raid members dying at opportune times is an ancient Ret parsing strategy, and that’s no different now. There is no simple way to track the uptime of Wings for each log.

- Sampling bias: Only using the top 500 logs for each boss might not necessarily be the best way to get representative data. There could possibly be a talent "x" with a high average and low variance, and another talent "y" with a lower average but much higher variance, and you could end up with a situation where "y" ended up overrepresented in top logs despite not being better on average across the entire population. On the other hand, most players who care about the difference between the talents to this extent are likely to be players aiming for high logs specifically, and might care more about the differences between the top end of the damage distributions for each talent rather than the average.

- Population differences: I don't think that the playerbase that uses each CS talent is likely to be an actual random sample. You could easily imagine CSAA players to be more likely to be savvy, keeping up with the latest talent changes, and TS players to be more likely to not bother doing significant research on talents. Alternatively, you could imagine CSAA players to be more likely to be simply copying the top logs rather than doing a lot of independent research, and TS players to be the ones who would need to feel confident in their decision to go against the grain. I don't know which way the population differences are likely to bias talent choice, but I think it's worth mentioning that this will certainly have an impact on the analysis.

Results

While it would be easy to just look at the average DPS for each talent build, this doesn't take into account the important factors that we do have data on, like item level and fight length. Instead, I'm using multiple linear regression to try to specifically look at just the differences between talents. I'm looking at the statistics for each boss individually. To start with, here's how TS compares to CSAA:

To quickly explain what these numbers mean if you are not a stats person, the main things to care about are the estimate and p-value columns. The estimate is the model's expected change in damage if you were to use TS rather than CSAA; for example, if you were to use TS instead of CSAA on Kazzara, your expected DPS would decrease by 710. The p-values indicate how statistically significant the result is. In essence, a low p-value means that the result is more likely to be significant, a real effect. (More specifically, the p-value is the chance that you’d see this result if the real effect was 0.) Some bosses do not have p-values low enough that I would consider them significant, with the standard significance level being 0.05, but about half do - Amalgamation Chamber, Rashok, Zskarn, and Magmorax. 

Base CS is a lot more difficult to look at - the estimates are all over the place, and none of the p-values are anywhere close to 0.05. There also aren’t actually any base CS logs on Sarkareth at all yet, and only a single one on Neltharion. There's a good chance that base CS is actually a decent alternative to CSAA in my opinion, but there's just not enough data to actually draw any conclusions yet. I would be interested to look at the data again in 10.1.5 if more people run it on logs.

So, what conclusions should you draw about what talent to run? Well, if you purely want to run the best expected performing talent, CSAA is very clearly the safest choice. It outperforms TS on every boss even when taking other relevant factors into consideration, and even if it isn't technically statistically significantly better on all bosses, I wouldn't be expecting TS to suddenly start appearing more competitive any time soon. However, despite TS generally being lower than CSAA, it isn't actually lower by all that much, an average of 1003 DPS less than CSAA. 1k DPS is not an insignificant amount, but choosing a build that does 1k less expected DPS because of personal preference isn't a really unusual thing to do and is unlikely to ever negatively affect your raid. It’s also worth reiterating that Windfury Totem will not have any positive impact on CSAA, but will for the other talents, and if you are able to be in the Windfury group then the damage gap between them will likely be negligible.

Going forward, if you're just opposed to running CSAA rather than wanting to run TS specifically, I would recommend trying out base CS. It doesn't have enough logs to draw any solid conclusions about its viability yet, but in theory it should end up continuing to perform better than TS as we gain more secondary stats, since TS gains a lot less from Haste and Crit than the other builds do.

Bonus Data

Log representation for each talent per boss:

Unsurprisingly, CSAA is the most commonly used talent by a large margin on every boss. As mentioned above, analyzing data doesn’t really give you meaningful results if you don’t have a large enough sample, and pretty much none of the bosses have enough base CS logs to give a good idea of its strength.

Other than CS talents, there's also some other interesting things that I noticed when looking at the data. It is fairly obvious that higher item level, shorter fight length, and number of external buffs received will all increase DPS, but I think it's interesting to look at specifically how much they impact it.

Item level, unsurprisingly, is a good indicator of increased DPS - for most encounters 1 extra item level would increase your expected DPS by about 200, with a couple of encounters having much larger expected increases, especially Sarkareth. Kazzara and Amalgamation Chamber don't have low enough p-values to be significant, but they're not that far off.

Fight length is a really good indicator of DPS. The estimates here are per second, which means that for each second longer you spend fighting Sarkareth, your expected final DPS will decrease by 151. Neltharion is the only encounter where it's not a great predictor, potentially because spending longer in the last phase often means you'll have more wings uptime from other people's deaths and more adds to pad on.

Again, not a shock to anyone, receiving external buffs is a pretty good way to increase your expected damage. These estimates are per external.

I also tried looking at whether the date of the log had any significant effect on the DPS, with the theory that later logs could be more likely to have better optimized gear/trinkets, but it was not a significant predictor of DPS for any boss.