Archive

Ineffective Theory

A note on macro performance in Julia

At least in Julia 1.10 (also 1.9), macros are not compiled. That means that

macro m()
    for i in 1:1000000
        # do something
    end
end

@m

can be quite slow. This is analogous to how running code directly from a script, instead of creating a function main() and calling that function at top-level, is not advised. Only code in functions is compiled/optimized, and macros do not count as functions.

All is not lost; in fact very little is lost. Functions are of course compiled, and functions can be called from macros. Therefore this is fast:

function mfunc()
    for i in 1:1000000
        # do something
    end
end

macro m()
    mfunc()
end

@m

Functions can return Expr objects as well, so even if a macro is slow entirely from AST manipulations, it can be optimized by offloading that work into a function.

One-paragraph book reviews

Bryan Caplan started it (and seems to have dropped it), and I think it’s a good idea, so here goes.

C. S. Lewis, That Hideous Strength. Probably my favorite Lewis book, at least for adults. Like Narnia, clearly has an evangelic thrust towards Lewis’s particular flavor of Christianity. (This will be much more obvious if you’ve read e.g. Screwtape Letters or Mere Christianity.) Reading it today it has an entertaining reactionary feel: a distrust of scientists and scientism, of large institutions, of anything much beyond family, friends, and individual duty.

Neil Gorsuch, A Republic, If You Can Keep It. I’ve often found that I like people less after reading their memoirs/autobiographies. Happily that was not the case here. This is a collection of speeches, essays, and excerpts from opinions. About two-thirds of the book is what might be covered in a decent civics course; the remaining third of the book is legal ethics. On the latter topic, Gorsuch’s speeches (understandably) emphasize the conduct of judges and trial lawyers. I can’t find good statistics on what law school graduates end up doing, but my sense is that this is something around 20% (at most) of lawyers. A great deal of societal power is surely wielded by the remaining 80%, but less examined.

John J. Mearsheimer, The Tragedy of Great Power Politics. An argument for offensive realism. This is the first book I’ve read on international relations, so I’m entirely unable to pass judgement on the claims made. The final part of the book (written 2014) is dedicated to predictions for relations between the U.S. (and east Asian allies) and China, conditioned on China’s economic rise continuing more or less unabated. Offensive realism, at least as defined by Mearsheimer, claims that `hot’ conflict between China and the U.S. is considerably more likely than it was between the U.S. and U.S.S.R. during the cold war. Mearsheimer clears two minimal bars with this book: first, to an uninformed reader, his arguments are convincing; second, his predictions made in 2014 have not yet been obviously falsified.

(Weakly) Against scientific taste

A common sentiment is that trained scientists have an important skill, often called “taste”, which is crucial in identifying worthwhile work. For a recent example see Tim Hwang and Caleb Watney’s most recent piece at Macroscience, which summarizes this view pretty well:

Scientists will often scoff at traditional metrics like citations or patents because, in their view, good science is a “know-it-when-you-see-it” phenomenon and fundamentally about good taste. Large-scale surveys of scientists could help bridge this gap by aggregating the opinions of scientists about promising (or unpromising) developments in their field and in adjacent fields.

If we’re going to assert that thus-and-such mechanism is an effective one, it’s good to have a story for what causes the mechanism to work. In this case the most plausible story looks something like this:

  1. Scientists are repeatedly exposed to examples of historical work in their field and adjacent fields.
  2. Thanks to obvious selection biases, that work is disproportionately the high-impact work.
  3. Scientists begin to develop a feel for what sort of work is “high-impact”.
  4. Work that feels familiar (and thus similar to historical high-impact work) is held to be in good taste.

Note that the selection bias in step 2 is doing heavy lifting here—and it’s good heavy lifting. It’s nice to see selection bias being the good guy for once!

Unfortunately, this procedure breaks down for the sort of work that spawns a new (sub)field. By definition and perhaps design, that work doesn’t look much like historically high-impact work. “Taste”, when driven by the mechanism above, is only sensible within the field that spawns it. This isn’t so surprising—who would expect a particle theorist to identify high-quality immunology research?

Another way to see the problem with taste is to remember that, by assumption, most scientists have pretty good taste. Therefore, within any given field, good-taste work is disproportionately likely to have been explored. That doesn’t mean that everything in good taste has been done, just that low-hanging fruit is likely to have been picked. It follows that if you’re looking for something new, it’s worthwhile to spend a lot of time exploring bad-taste ideas. Most other things being equal, they’re more likely to be the important ones.

For straightforward work, all the above considerations lean in favor of judging via taste. It’s only in the search for unusual (can I type “paradigm changing” without vomiting?), but high-quality, work that using taste becomes a handicap.