Why is Microsoft so ademant about keeping all of the processing on the device? It seems like everything would be a lot better if it ran off of a computer via WiFi. There would be more processing power to take control of, because the hardware wouldn't be limited to something that can be worn on the user's head; which could solve the FoV problem, depending on what's limiting it. The whole thing would weigh less and be a lot less clunky, too. It might do more than the Google Glass, but the extra stuff shouldn't take up too much more space. After all, most networks could support it. There wouldn't be a bandwidth issue, considering the fact that most routers are pushing 100 Mbps now. Running it through the network should only put a few ms delay on the device, but how much would that affect anything? Plus,people could still walk around in their homes. The HoloLense isn't being marketed as something that people would take out in public, nor would anyone take a (probably) $1,000 device somewhere it would get broken (and it would be pretty useless, anyways).
Your comment made me think of this forum post I read recently: https://www.pathofexile.com/forum/view-thread/175363 "Let me begin by saying that absolutely every game that is played online has to deal with desync..." Doesn't answer your question, I know--it's tangential.
I wonder how low of a ping would be required to have it still be immersive. I know that something like 50ms is playable online, but it might be a lot less for real time things, in order not to be disorienting. I've found a site that says people can tell if something is off by 1/300^th of a second, which is about 91.44ms. However, that's just being able to tell if a light flashed for 1/300^th of a second, and they were trained pilots. I'm getting <1 ms pings (Sorry for lack of screenshot) from the router from my laptop, and I'm on WiFi (Granted, I'm in the same room as it and everyone's asleep), but I've seen people get up to 4ms. The only thing that might really affect that is the processing, but it would be no different than having the device do it already. I don't know, someone smarter than me would need to look at the actual numbers, but I think it would work.
its when you move your head and the rendered stuff lags behind it will wobble. Even a tiny amount of latency would be easily visible. It's a bit like how looking at a fluorescent light directly, you can't see it's 60Hz flicker, but if you move your eyes left to right quickly, you can discern it. John Carmack has studied it extensively and I would consider him to be the expert these days on that stuff as he works for oculus.
ISPs don't have control of the speeds between two devices on the same network; only speeds between two networks connected via the internet. A phone and computer on the network can transfer data as fast as the slowest network adapter or router, permitting that nothing else is using the connection. My router can push ~356 Mbps to a device via WiFi, if the device supports it, and I think 1 Gbps through a wired connection, if the device supports it. However, I wonder how much information the HoloLense is processing every second. It shouldn't be an extreme amount, looking at the size of the device.