The Model Has No Clothes
February 27, 2013 7 Comments
A while back an article was making the rounds claiming to show that the government’s implicit ‘too big to fail’ backstop amounts to subsidizing banks to the tune of $83 billion a year. I did not, let’s be clear, read that article. Moreover, I am totally against ‘too big to fail’ (N.B.: in all its forms, not just banks…) so at least superficially, I should be sympathetic to publicizing such a finding as furthering my goals. But then I read Matt Levine’s debunking of the model by which the authors claimed to derive that $83 billion number, and as far as I’m concerned it rendered the matter completely and thoroughly closed. If you don’t believe me, just read it.
He pointed out at least two huge issues with the calculation: 1) the authors looked at the rating impact of TBTF protection, turned rating moves into implied spread moves (ok fine), and then apparently averaged (!) the hi and lo spread impacts. What? Simple averaging spreads? Bzzt, sorry. 2) Worse, the spread impact they come up with relates only to 5-year spreads. Again, sorry; that’s not a good proxy for how banks actually fund themselves. Or it might be, conceivably, just with some appropriate scaling/beta/multiplier? So their “$83 billion” is, at best, correct, up to some unknown scaling? You know what else is correct up to some unknown scaling? Any other number.
From these two facts alone, unless they have been misrepresented by Levine, it should be clear to any knowledgeable reader that the study and therefore the $83 billion number was garbage.
Not to everyone though, especially since “$83 billion subsidy!” made its way into a lot of peoples’ politically-motivated talking-points. Hence mathbabe has written a
rebuttal response to Levine’s piece saying – in effect – that you just can’t critique a model without proposing one of your own.
- But since I’m a modeler, I know it’s a lot easier to push over a model by complaining about an assumption than it is to come up with a better model that doesn’t make such stupid assumptions.
- So anyone who complains should also offer an alternative.
I’m just going to point out that that is completely and totally wrong. In my book people, whether ‘modelers’ or not, can and should feel free to critique others’ models all day long and to point out legitimate problems with them. If those objections are silly or crazy, they can be easily dismissed or ignored; but if they are serious, they should be answered. Either way, the absence of a better model doesn’t mean we are somehow compelled to pay attention to or go with the results of models that are materially flawed and whose flaws have not been satisfactorily defended. (As far as I know, Levine’s objections have not been answered – certainly not in that post.)
A perfectly scientifically valid ‘alternative’ to a bad model doesn’t have to be another model – it could just be no model, and saying “we don’t know”. It is enough, or should be enough, to just point out “That is wrong”, that the Emperor Has No Clothes. This is all a healthy way for science to be done – you know, open critique, questions asked, problems raised, critics answered, and so forth. Part of what bothers me about this ‘well then offer an alternative!’ attitude is that it represents a sort of science-guild protectionism: if no one is allowed to raise problems with existing models without ‘offering an alternative’, that obviously would help to increase the power of…folks who make models. After all, really, who has time to not only dig through such things and find/identify their problems (which is valuable enough already, and kudos to Levin for rolling up his sleeves), but to cook up complete and valid ‘alternative models’ as well? A very small percentage of people. And that’s the way the modelers like it, I suppose.
The post is titled “How much are the taxpayers subsidizing too-big-to-fail banks, if not $83 billion per year?” But if the entire ‘subsidy’ premise came from a flawed model, this is sort of like asking “Okay, if bleeding the patient doesn’t cure their disease, what DOES?”. Or, “On how many giant turtles’ backs does the Earth rest, if not 63?”
You say it’s not 63? Well, how many then, smartypants! Where’s YOUR model? You don’t have one? Ha! I win. 63 it is.
I don’t know what that is, but it’s not science.
Maybe it’s ‘modeling’.
P.S. Apologies in advance to all those who read this post’s title a different way and got their hopes up.