The problem is not that you'll hit something. The problem is you never will.

We use scientific notation to eliminate orders of magnitude and converse and operate knowledgeably about concepts that are beyond human scale in one direction or another. We do this because we are irrevocably tied to human scale whether we like it or not. It's hard to wrap your head around the forces and energy necessary to get out of the gravity well - but we burned up an office building worth of explosives to get three guys to the moon and back. And yeah - orbit is 90% of the way to anywhere in the universe but you have to get everything up there.

And then you wait.

It's easy to point out that Voyager 1 is past the heliopause but it's just as easy to point out that it took 25 years to make it 125 AU. IF Voyager were steaming to Proxima Centauri it'd get there around 4000 AD. And sure. lightsails, VASIMR rockets, Project Orion and on and on and on. But it took us eight or nine tries before we successfully got a rover on Mars so we're a long goddamn way from needing to worry about deep space objects.

Personally? I rather like it here. And I know it's a long damn way to anywhere else. I don't particularly like the climate of Nevada let alone Mars and Mars is the best second-place contender we know of.