The Rules of the Game

John Shim was there at the beginning, just as uber-fast algorithmic trading took over Wall Street in the late 2000s and secretive high-frequency trading (HFT) firms could rake in millions of dollars simply because their technology was one-thousandth of a second faster than their rivals.

Fresh out of college with a background in computer science and statistics, Shim, now an assistant professor of finance at the University of Notre Dame’s Mendoza College of Business, landed a job as a trader at Jump Trading at a time when few outside the industry understood how HFT worked or how firms like his were able to profit so reliably. Shim spent his days building and running the very strategies that would later attract the attention of regulators and the financial press: split-second trades, executed by machines, competing for advantages measured in microseconds. 

He was good at it. But being good at something and understanding why it worked, he found, were two very different things. 

One particular question stuck in his mind: How could people like him with no real knowledge of finance or a company's fundamentals consistently profit at such scale and why would the investors losing money on the other side of those trades tolerate it?

The question ate at him enough that he walked away from a lucrative trading career to look for the answer in a Ph.D. program. What he found helped make him one of the more closely watched experts on the hidden costs built into modern financial markets. Shim has been cited by the Securities and Exchange Commission and helped shape a federal rule on how stock exchanges sell market data in the years since. But at its core, his research is still driven by the same instinct that sent him back to school in the first place: The conviction that markets don't just run on human judgment or bad luck, but on rules — rules that determine, often invisibly, who profits and who pays.

“The rules of the game matter. The rules dictate how market participants behave, whether that's trading to gain an advantage and earn profits, or allocating trillions of dollars in an index fund,” said Shim. “We need to understand the rules to understand economic behavior and to understand whether a market is efficient or inefficient. If it's inefficient, we need to think about how to tweak the rules to get better outcomes.”

The consequences are real. Estimates put the cost of high-frequency arbitrage in U.S. equities alone at roughly $500 million a year, a figure that climbs well past $1 billion once futures and options are included. Shim's later research on index funds suggests the rules governing how funds rebalance their holdings cost ordinary investors billions of dollars a year.

None of that money simply vanishes. It flows to whoever is positioned to capture it, whether it would be a HFT trading firm exploiting a millisecond advantage or a company issuing shares that index funds are required to buy, even at an inflated price. For every winner produced by a market's inefficient rules, there is very often a quiet loser on the other side, and Shim's research is largely an attempt to make those hidden costs visible.

Following the Money

Shim first announced himself with a 2015 paper he co-authored that reframed the HFT arms race as a response to a market-design flaw, and showed that money spent chasing speed doesn't make markets more efficient; it merely wastes resources as rivals compete for the same fixed prize. Shim and co-author Eric Budish of the University of Chicago proposed that exchanges replace continuous, sequential trades with tiny batch auctions that would collect orders within a fraction of a second and execute them together at a single price, eliminating the mechanical advantage that speed buys. The paper, which won the 2014 AQR Insight Award, was cited by the SEC and referenced by the New York Attorney General — proof it had reached an unusually wide audience.

A decade later, Shim turned to a very different corner of the market as a deliberate choice to broaden his scope of work beyond his previous career. Working with Harvard Business School's Marco Sammon, he began studying the plumbing of index funds, which today own roughly a fifth of the U.S. stock market. 

Their research found that value-weighted index funds — marketed as simple buy-and-hold vehicles — are often required to buy high and sell low. Whenever a company issues new shares, often a signal it thinks its stock is overpriced, the index funds tracking it are mechanically required to buy more. When shares are bought back, usually because the company believes they are undervalued, funds are forced to sell. That built-in mistiming costs index investors an estimated 46 to 69 basis points a year, far more than the fund's advertised fee. 

While Shim resists the suggestion that his research is really a story about Wall Street exploiting the little guy, he accepts that sophisticated investors have an edge. But he also argues they should earn their profit doing what markets need them to do: providing liquidity, or trading on real information that makes prices more accurate. What troubles him — the thread that runs through his research from HFT to index fund rebalancing — is when profit does not come from either of those functions. 

“If sophisticated traders make money when they are not doing these things, economists start to ask if maybe the market is not working very well,” he said. 

That same test — is profit earned or extracted — occasionally clears a practice he initially viewed with skepticism. Shim and a co-author recently studied the wave of "front-running" that occurs when traders buy a stock ahead of its addition to a major index, a form of anticipatory trading that looks, on its face, like predatory behavior. Their finding surprised even them: Competition among those early buyers actually smooths out the price spike that hits when index funds are forced to buy in bulk, lowering costs for everyone else. A separate working paper looked at what happened to bond exchange-traded funds during the 2020 pandemic market crash and found something equally surprising: The middlemen who connected ETFs to the bonds they held didn't panic and dump everything at once. Instead, they absorbed the selling pressure themselves for a while, which kept the bond market from spiraling into a full-blown fire sale.

"What we find, more often than not, is that the market is very efficient," Shim said.  

With six working papers in hand and five more projects in progress, Shim's research pipeline continues to grow, though less by design than by organic accumulation, such as a serendipitous conversation with a colleague or an unanswered question from an earlier paper. At times, the thread is more direct, as was the case with one study he wrote with Sammon that led straight to a second paper, which in turn prompted discussions about a future project with another collaborator. 

A Decade Later, Still Waiting

For all its acclaim, Shim's signature fix — frequent batch auctions to neutralize HFT traders — has never been adopted by a major U.S. exchange. A follow-up paper, published in the Journal of Political Economy in 2024, explains why: Exchanges make much of their money selling speed itself, through premium data feeds and co-location services, which means the very firms best positioned to fix the arms race have no incentive to do so. That diagnosis was reflected in a narrower reform, the SEC's 2020 Market Data Infrastructure Rule, which opened the door to competition in market data pricing. Courts upheld the rule, but implementation remains stalled because exchanges still control the fees any competitor would have to pay, proof of the very disincentive his research identified. 

Shim isn’t concerned about the pace of change. What bothers him more is the idea that a single paper, including his own, could be lifted off the shelf and accepted as ready-made policy. Research like his, he said, should inform how regulators think about markets, not dictate what they do; blind adoption, in his words, would be a "complete overreaction." That's why his preferred fixes tend to be narrow and indirect, the kind that clear a path for market forces rather than those designed to force an outcome. It’s an approach that requires patience  — real-world impact, on his terms, arrives slowly, unevenly, and largely outside his control.

"Success, to me, isn't a rule that gets passed," he said. "It's a colleague telling me they learned something from my work. And it's changing the way my students think about problems. I want them to understand how the rules of the game dictate people’s behavior and lead to the economic outcomes we observe. That lesson is much more valuable than any one finding in any of my papers." 

It's the same instinct that pulled him away from a trading career, forgoing the race to profit for a different kind of pursuit — understanding why.