THANK YOU FOR VISITING SWEATSCIENCE.COM!
As of September 2017, new Sweat Science columns are being published at www.outsideonline.com/sweatscience. Check out my bestselling new book on the science of endurance, ENDURE: Mind, Body, and the Curiously Elastic Limits of Human Performance, published in February 2018 with a foreword by Malcolm Gladwell.
- Alex Hutchinson (@sweatscience)
***
Just noticed a new study on a very old debate: the “hot hand” in basketball. There was a very famous study back in 1985 that concluded that our belief that players have hot streaks and cold streaks is simply an example of the “clustering illusion.” Think of it this way: if you flip a coin over and over, you’ll occasionally have streaks of six or seven heads in a row — but the probability of the next toss is still 50-50. What Gilovich, Vallone and Tversky found when they analyzed NBA game data was the same pattern: the probability of hitting a given shot was independent of whether the player had hit or missed his previous attempts.
Of course, most people simply refuse to acknowledge this sort of result that conflicts with what seems “obvious.” As Tversky noted:
I’ve been in a thousand arguments over this topic, won them all, but convinced no one.
After all, we can all remember, say, Michael Jordan taking over a game in the fourth quarter and pouring in basket after basket. But part of this may be because, with the game on the line, he starts taking more shots. And we tend to forget all the nights when he didn’t manage to take over the game, despite presumably making the same effort. This is one of those issues when we can really only trust the cold, hard data.
Anyway, the new study (full text freely available here; press release here) takes advantage of the Moneyball era of statistical abundance to revisit this question with a larger data set. The researchers, from Yale, looked at every free throw taken in the five seasons between 2005 and 2010 — a staggering total of 308,862 free throws — and tried to determine whether the patterns in the data could truly be explained by considering each shot as an independent event. And indeed, they found some evidence that — according to the press release, at least — supports the existence of the hot hand.
The key result they found is that, when players were taking two foul shots, they had a slightly greater chance of hitting the second shot when they hit the first (~76%) compared to when they missed the first (~73%). There are two possible ways to explain this:
- Players have periods when they’re “hot” and “cold.” The success of the first free throw is an indicator of which of those zones (if any) they’re in for the second throw.
- The outcome of the first throw causally influences the outcome of the second throw. For example, if you hit the first, you relax, feel confident, and drain the second; if you miss the first, you tense up, feel the pressure, and (become infinitesimally more likely to) miss.
The researchers argue against that second explanation, for the following reason. When they analyzed the individual data, some players shot better after hitting the first throw, while others shot better after missing the first throw. This is to be expected: since the supposed effect is psychological, different players will react differently to hitting/missing the first throw. But when they drilled deeper and broke the data down into individual seasons, they found that players who shot better after hitting the first shot in one season had a 50-50 chance of showing the opposite pattern the next season. That suggests that the connection between the first and second shots isn’t actually causal.
So what does this all mean? Well, in a sense it makes a fairly obvious point. It would be ludicrous to imagine that pairs of free throws are totally uncorrelated — consider, say, a pair of throws taken late in an insignificant game where the outcome is already decided, just after returning from a prolonged injury and having sustained a hard foul that hurts your shooting hand, the night after a coast-to-coast flight that was delayed by weather for seven hours, during which you got hammered because one of your teammates was celebrating his 21st birthday. The probability of both those throws will be slightly lower than a pair of throws under optimal conditions — and that leaves the kind of statistical footprint detected in this paper.
In a sense, of course, that’s precisely the point: under those conditions, you might say the player has a “cold hand.” But that’s not usually what we think of when we talk about hot and cold hands — we’re usually referring to time frames that are longer than two back-to-back free throws (it usually takes more than two shots before announcers start pulling out the “hot hand” trope), but far shorter than game-to-game variations. So in the end, I’m going to keep believing that the hot hand doesn’t exist until better evidence emerges.
Does the study take into account the current events of the game. for example, a player may purposely miss the second shot if they miss the first in hopes of getting the rebound when down by two at the end of the game. Free throws are a flawed metric to use as there is strategy, jump shots are rarely taken to purposely miss a shot. Just my 2 cents.
Agreed — jump shots would definitely be a better measure to assess what we usually think of as “hot hand.” Of course, that sort of analysis would come with its own flaws, because you’re then affected by a whole bunch of external factors. If you’re covered by a very strong defensive player for a part of the game, but by a weak defensive player when the other guy is resting, it would look like you’re having “hot” and “cold” streaks — whereas in reality, your own abilities are staying perfectly constant and it’s the environment that’s changing. It’s definitely not a simple riddle to untangle!
Jump shots ( or FG attempts in general) would be very difficult to study. Not only are they influenced by the defenders input but also by many other factors that might relate to shot difficulty. Shots early in the shot clock are generally of a higher quality (less difficulty). Near the end of the clock shots are forced to beat the clock. Distance of attempt of course would also matter.
Maybe most important is the confounding between the individual feeling of the ‘hot hand’ and the likelihood of taking a difficult shot. The more confidence a player has the more likely they are to attempt a low percentage shot (speaking from experience).
I think when most media types, or players or coaches refer to the ‘hot hand’ they talking about free throws. I have a pretty good stats background and Tversky is a giant in his field though his work with Kahneman. I find the field of hueristics fascinating.
Having said that I have always had a difficult time accecpting the no ‘hot hand’ work. I have played a whole lot of basketball and am not the type to forget the previous shot or play. For me confidence levels always swang pretty dramatically. When confident the skills that have been developed through practice are allowed to be expressed. When hesitant the mind seems to find a way to interfere. The feeling is very noticable and it seems like it would have to influence results.
I never found the feeling of being hot affecting FTs personally and don’t think it is a good test of what most mean by ‘hot hand’.
Sorry, I meant to write ‘when most media types, or players or coaches refer to the ‘hot hand’ they are not talking about free throws.’
@Seth: Ah, that makes more sense. Yes, I agree that (a) free throws aren’t the appropriate test of hot hand, and (b) testing the theory on jump shots is tremendously problematic. However, it’s worth noting that the original 1985 paper DID analyze in-game sequences of field goals, and found no evidence of hot (or cold) hand.
As for this:
“Maybe most important is the confounding between the individual feeling of the ‘hot hand’ and the likelihood of taking a difficult shot. The more confidence a player has the more likely they are to attempt a low percentage shot.”
I’m not sure I agree that that needs to be taken into account. It’s certainly true that players have the PERCEPTION that they’re hot or cold. But if this doesn’t manifest itself in a greater prevalence of consecutive runs of hits or misses than would be expected from a chance process with a constant hit rate, then how can we say the phenomenon exists? No one’s interested in the internal mental state of the players, they’re only interested in whether the baskets go in.
Now, if feeling “hot” means you try lower percentage shots, then in theory your overall shooting percentage could stay more or less constant — which means that statisticians will fail to find evidence for hot hand despite its notional existence in the mind of the shooter. If that’s the case, then we can split hairs and say (1) the hot hand as perceived by fans (91% believed in it, according to a survey in the 1985 paper) doesn’t exist as an objective reality as recorded in the scoresheets of the game; but (2) it does exist in the minds of the players, and WOULD show up in the stats if they could resist the temptation to alter their shot selection when they feel good.
Proving the second point (that hot hand doesn’t exist even in the minds/bodies of the players) would indeed be very hard. But we should start with point (1): does the hot hand show up in the scoresheet? If it really existed, we should be able to see it.
I think when people refer to the ‘hot hand’ in practice they do take shot difficulty into account.
This is what I was trying to get at. No announcer is likely to state that Ray Allen is red hot if he happens get a couple fast breaks and an easy layup off a great play by Rondo. On the other hand if he makes three straight deep threes while closely guarded by Kobe Bryant thats another story. In this instance you can control shot distance. The point however is that in my opinion what fans and players and coaches call a ‘hot hand’ weighs difficult makes more heavily than easy makes.
I also think that as humans we are pretty good at social recognition and perceiving body and facial language. I think we are judging what we think a a player is feeling not only on results but also on these factors. If a player banks in a 30 footer when obviously attenping a straight make and acknowldeges his surpise at the make he is much less likely to be described as hot than one who confidently rises up and swishes from the same distance with a game face. Think Kobe with clenched jaw.
So I think what is being studied and what is being referred to in practice are quite different. So I tend to agree with your points 1 & 2 except that players would be less likely to be called ‘hot’ if they only took hi percentage shots.
The feeling of being hot exists for sure. It seems likely to me it influence results all other factors being equal.
“No announcer is likely to state that Ray Allen is red hot if he happens get a couple fast breaks and an easy layup off a great play by Rondo. On the other hand if he makes three straight deep threes while closely guarded by Kobe Bryant thats another story.”
But if this is true, then analysis of statistics should OVERESTIMATE the occurrence of “hot hand” streaks, rather than not show them at all. We should be able to analyze games and see lots of hot streaks, and then face the challenge of distinguishing between the “true” hot streaks and the lesser ones that simply consist of a series of layups. Instead, what the data seems to show is that there aren’t ANY hot streaks of either type, beyond what we’d expect for a random process with a constant rate of success.
Again I would disagree based on the primary point I keep returning to.
In basketball there is a balance between being to passive and only taking easy opportunities and being to aggressive and taking too many bad shots. Because there is a 24 second shot clock if you are too passive you put the onus on your teammates to create with less time left.
If a player modulates his aggressiveness based on how ‘hot’ he feels there will be no statistical evidence for the presence of streaks. It would be rational for a player who is at an elevated level of ability for creating scores to try to take more burden .
This is why the top scorers are not necesarily the fg% shooters. They take it upon themselves to take more difficult shots. Some like Iverson go to far in this direction and the greatest of the great scorers (MJ) could maintain the hi fg% while scoring often.
I just think reducing being ‘hot’ to made vs missed streaks is a weak proxy for what it is trying to measure.
@Seth: Fair enough — it’s true that under your definition of “hot hand,” it’s possible that it could exist without leaving a statistical footprint. But that’s a bit of a semantic shift. In the original paper, they were very clear in their definition of what they were looking for: streak shooting. In their survey, they found that 91% of basketball fans believed that “a player has a better chance of making a shot after having just made his last two or three shots than he does after having just missed his last two or three shots.” That’s the proposition they were testing.
So we can split this into two separate questions:
(1) Does the “Seth Leon Hot Hand” exist? I think we probably both agree this is a question that can’t be answered by analyzing statistics (or at least would require an incredibly sophisticated analysis).
(2) Do players shoot in streaks more than would be predicted by a random process with constant probability? On this question — which is entirely independent of the players’ internal mental state and shot selection — most fans would answer yes, while statistical analysis says no. That’s the cognitive dissonance at the heart of Gilo, Vallone and Tversky’s paper.
Of course, basketball is vastly more complex than the simple question of frequency of consecutive hit and miss streaks. This kind of analysis offers no insight into the ebb and flow of player and team confidence and efficacy. But it does tell you that fans believe they’re seeing something that isn’t there.
Haha, Thanks for hanging in there with the intelligent replys.
For the record I am not among those 91% who think making your last 1, or 2 increase the probably your next will also be a make (providing no caveats are mentioned).
As for ‘Does the “Seth Leon Hot Hand” exist?’ Not for me anymore 🙂
But I do think my definition is really what most actually notice when veiwing real players or when playing themselves. I think it has to exist for those whose skills are in a good current state from practice. The state of confidence exists and knowing just a little a bit about how mind influences body to carry out practiced skills it we be hard to accept ‘no influence’.
By the way – You not only are fair in your replys but communicate very clearly in writing. I would be happy with a fraction of your skill in that area.
Funny thing that, if you look at panel A from figure 1, it appears that players who have three free throws, already have a hot hand on the first throw (hit rate between 74 and 80%). How would you explain that?
Confidence? (“hey, I have three throws. Surely one must go in”)Or is it a tactical thing? Something like three throws are somehow more likely to be awarded to the best players.