Is this world a dead-end without some huge galaxy-spanning conflict?

You never need to go with a Star Wars conflict if you don't want. I get sick of those myself, though I often find I write them into my worlds anyway. Seems to be a trope I have a hard time breaking out of. :\

The sci-fi backdrop can be utilized well for more personal conflicts. Space can be lonely and daunting, and a perfect thematic metaphor for all kinds of personal internal conflicts.

The trading colony in the mined-out asteroids does leap out at me as an intriguing place. I think with that, if you do want to get semi-big with conflict, there can be political squabbles there, be it about taxes, trade, the future of the colony, or even the attempt of a government trying to convert it to something else (which would cost people their jobs), like a prison colony. I think that area has the most interesting possibilities.

Of course, I do highly recommend working on personal struggles and smaller conflicts. Your description here reminded me of George R.R. Martin's "Meathouse Man", a splatterpunk tale with a grimdark sci-fi setting; it's certainly far different in specifics to what you've described (the tale includes zombies), but I think it gives off a very distinct tone that works well with the setting, so you might want to look into it and see if it's a direction worth pursuing for your world.

As for where to start, if your focus is on the characters, then start with the characters, and build the world around them. Add technology that is necessary for their use, things which they'll come across, and are necessary to further the stories, but don't dwell too much on extraneous bits. Start small, with just that one colony, and expand to the wider universe as it becomes necessary to do so.

Figure out who your characters are, what makes them unique, what generic struggles and obstacles they might face due to the world you're making, and what personal conflicts arise from the characters interactions. Consider quick sketches, and play and experiment with them. Write a couple of pages on what happens when you throw Tom and Mike in a spaceship together with no entertainment but conversation. See what pops out. It will help you build the characters and evolve them, even if the scene you write isn't canon and exists only as an exercise. Perhaps put it up here for review and ask pointed questions about it.