Is more efficiency a good thing anyway?

This really begs at the question are Capital Markets good? I would say a resounding yes. That's not to say that our capital markets are perfect or couldn't be improved upon, but its better than not having an equity market.

Now what does the stock market do for the company and the investor and how is the stock market and in particularly the accuracy of the price at any one moment have to relate to that. Equity is one way for a company to fund itself. You sell a portion of company ownership for cash in an IPO or another offering. Then going forward you can also use the equity you retain and the current price in the market as one of your assets. For an investor equity is part ownership in the future cash flow of the company. If the company has more money they might issue a higher dividend or do a stock buyback thereby increasing the value of what you own.

The Efficiency of Price Discovery

I can try to answer the question in the context of something material happening to JCP. Let's say their holiday sales are released and it's bad. If you are an owner of JCP stock you probably want to compare your expectation of future cash flow with the current price of the stock. Would you rather sell now for cash or hold it for future appreciation. If you think you are much better off selling now then you go to the market and sell. This is going to affect the price in the market, because the level at which someone was willing to buy at may no longer be there. Only so many people are going to be willing to buy at price X until they move the price down to X-1. So the stock price moves down eventually it stops moving down and market participants are willing to push it back up again. This process is called price discovery and it's something that market makers help to achieve. Now here's the kicker, if you as an investor want the option to be able to buy or sell in a reasonable time after material information is released then price discovery has to happen! The price can't sit at it's last price and everyone is allowed to execute at that price. Clearly that doesn't work because who is actually going to receive the sold securities? There has to be real time buyers and sellers to facilitate this movement.

The Efficiency of Alternate Markets

Now above I described the efficiency of price discovery and how market makers help with the process. Now there is also the efficiency of price being equal across different exchanges. This is easier to understand, if there is a resting offer to sell BAC for 12.99 at NYSE and a resting bid to buy BAC at 13.00 at NASDAQ then it only makes sense that someone is going to come along and try to buy BAC at 12.99 at NYSE and sell it for 13.00 at NASDAQ. If you want more than one market center then this has to happen! And you do want more than one market center, because competing market centers help to keep exchange fees low via competition among the exchanges. Not to mention that competition among the exchanges helps them to improve their market technology being able to handle more bandwidth and coming up with new trading technologies that help market participants. The CME futures market is an interesting counterexample to the US equity markets, because although there are about 7 lit equity markets there are only 2 big futures markets. So CME has a lot more power. I cannot say definitively whether it's cheaper to trade in equities or futures because its like comparing apples to oranges. With CME they are also a central clearing house so they help facilitate the risk if a counter-party defaults and cannot fulfill an order (imagine Smithfield goes broke and cannot deliver on its pork futures). The problems with trading equities are not equivalent.

The Efficiency of Related Securities

Many of you are likely familiar with ETFs. Exchange Traded Funds are a security backed by a corporation that is designed to back this security with specific other securities. The simplest one is SPY, which is made up of the securities of the S&P 500. So the person who issues SPY buys the basket of stocks that relate to the S&P 500 then they issues an equivalent amount of SPY shares. When they get dividends from the basket that they own they issue dividends to the owners of SPY. Now why is this a good thing, because owning SPY instead of owning 500 stocks is a cheaper way to diversify your portfolio. Why do you want to diversify your portfolio, because mathematically you have higher expected returns with lower risk (check out modern portfolio theory). Now just with the alternate exchange example above, if I could calculate the fair price of SPY and I see that the price SPY is being traded lower than the cost of its constituents then I'm going to buy SPY and sell the constituents. This pushes the price SPY to the true price of SPY.

Aside, now one other thing that could happen that is different than it works today is there could be mini auctions every few milliseconds instead of continual trading. I wouldn't necessarily object to this idea. The idea is similar to what happens in the morning or closing auction. Everyone who wants to buy or sell a stock issues the price they would buy or sell at. If the market crosses then the exchange finds the fair value and matches as many buyer and sellers as possible at the price. You would have to match everyone pro rata though, because if the match still occurred based on time priority you wouldn't have solved any of the negative sides of the industry such as high technology cost.

I don't consider the high cost of technology to be a huge negative of the trading industry. What automated trading has accomplished is take the job of 1000's of human market makers and automate them. The old human market makers used to make a lot of money for what they did. They also couldn't handle more than a few stocks at a time so there had to be a lot of them. A modern moderately sized team can handle the entire universe of stocks though. This should make it clear why it's cheaper to trade now than it ever has been. In the process they have brought the spread between buyers and sellers down to penny and subpenny levels as well as enable you as an investor to click on your Scotttrade account and actually own the shares you want to own before the page even finishes loading. That's amazing if you think about it.

In response to the this

HFT often works in a way that is agnostic to the market itself.

In my experience strategies do not work in a way that is agnostic to the market. Usually strategies are formulated off of some premise related to the market. One example would be if there are more orders resting on the buy side then the market might move up. Or if the price has been moving up keep moving it up until reach some reversion point that you have calibrated by backtesting. These two strategies here may seem like they are agnostic to the fundamental price information of the stock, but they are not agnostic to the market itself. As a market maker you want to facilitate trading between buyers and sellers and using information that is coming into the market (like resting orders or recent trades) allows you to get a better idea of what the market participants are going to do next.

Now I imagine that there are other things that traders could do that are way out there. Attempting to correlate twitter traffic to price fluctuations seems particularly hard to me. Similarly doing anything that is deep machine learning like Neural networks or Genetic Algorithms run a bigger risk in my opinion because if the premise that you trained your learning algorithm on changes then your strategy might work in the complete wrong way. As an analogy imagine that you trained a neural network facial recognition algorithm in a room with only yellow incandescent light and then you bring in a test corpus to see if your training has worked. It will probably work pretty well but what if the yellow light gets changed to fluorescent light. If it's a neural network it will be hard to know if it will continue working. Similarly in the market there might be a certain market sentiment all of 2013 but if something big happens in 2014 like a sovereign default or another externality you have no idea how the market is going to behave moving forward. If you at least tie your strategy to something simple like order flow you are likely to run less risk.

EDIT: Another thought came to me: Is more efficiency a good thing anyway? I can imagine that some inefficiency between market changes and stock prices could actually be beneficial for the health of the economy. Inertia is a stabilizing factor of many systems.

The slight inefficiency of the market creates a small time buffer that means a lot of the meaningless fluctuations of the market can be ignored (have to be), which would seem to lend stability. HFT lets computers exploit these fluctuations instead, perhaps blowing them out of proportion.

And the other thing you said is true as well -- if you have the computing power it seems much more foolproof to work the market via HFT than it would be through doing actual investment research and accumulating knowledge.

Thanks for the offer. I would like to know what kind of factors your algorithms consider. Are they mostly indicators that traditional investors look for, like product announcements and earnings statements, but HFT aims to react faster than others, or do you put in variables day traders might ignore? Not just Twitter trends, but obscure things like weather events, stock price proximity to multiples of ten, ease of pronunciation of company names.

I can't comment on anything I've specifically been involved in and even that is a small subset of the motivation of all the market participants. The price discovery that I described in mk's answer above is a combination of everyone coming together with their own strategies. These strategies come from a wide variety of ideas and time horizons. Trading on P/E on a month to month basis might be one. While trading on twitter traffic in a minute to minute horizon may be another. Trading on order flow on a microsecond by microsecond basis is another. Predicting how the weather is going to affect Oil and Utility companies is probably something else going on.

Ha it's interesting that you mentioned stock price proximity to multiples of ten because that is something I wouldn't expect someone outside the industry to even know about. Are you a shark trying to extract info? It's true stock prices do weird things when they get close to whole numbers. I've pictured it a matter of psychology, I think it's the same reason that queues at restaurants get busy at 12:00 and 1:00pm but not at 12:23. It's because everyone told their friends to meet at the restaurant at 12:00 not 12:23. I think people do the same thing for stocks. They say "OK, I will finally sell TSLA if it hits 180." Not "I will sell TSLA if it hits 179.97." except me that's exactly what I say because I know that's going to get my order executed before everyone at 180. Some stocks can even trade along 1 penny from a whole number all day long. It's weird and I don't get it completely.

IMO as I described above, in the smaller time horizon space the simpler and more fundamental of a signal you can act on the lower risk you are going to have and the quicker you are going to be able to act on that signal.

The article also mentions that transaction fees approach zero. This seems a critical factor too for a high-frequency trader. How close to zero, and how significant are transaction costs overall?

Here are the costs of all the regular lit exchanges NYSE, NASDAQ, BATS, and Direct Edge. It's less than a penny to to take liquidity of a liquid stock and you also get paid to provide liquidity of that same stock. How much you get paid for taking less how much you get paid for providing is clearly positive resulting in profit for the exchange. Additionally you might be interested to know that every big lit market has an inverted exchange where you get paid for taking and have to pay for providing. There tends to be less trading on these exchanges.

Executing a trade, whether that trade came from a signal or a client is all about finding the cheapest liquidity in the market. If you can provide liquidity and get paid by NASDAQ then do that first. If that doesn't exist then go to the dark pool where you might be able to match between the Best bid and ask. If that doesn't work then go to a lit exchange and take liquidity. You can see from the cost lists above that it's not going to be more than a fraction of a penny to execute in any of these areas.

As far as their significance even though its small how much you are going to get paid for a trade from your client is also pretty small. So you want to be as frugal as possible. Trading is a numbers game. You want to make each fraction of a penny of profit you can make in a transaction as big as possible while doing as many of those transactions as possible while also taking on as little risk as possible.

I just want to point out that this philosophy is a really poor way to approach a conversation on hubski.

Hubski allows me to get insight into so many areas and opinions that are not my own or that I would have never considered to contemplate. I always approach my conversations wanting to get as much as I can out of the person I am talking to not reject as much as I can with prejudice.