I had assumed he wasn't traveling in time but rattling things from his head. After all, we establish a present but he also tells us about his murder. There is no physical conceit for time travel -- simply that he's not in control of when he is. The first sentence is "Billy Pilgrim has become unstuck in time." Whether anyone else perceives this is immaterial. That's how Vonnegut worked: "and so it goes." Vonnegut had been trying to figure out how to tell the story of the Dresden firebombing. He was there, but its impact got eclipsed by the atomic bombs in Japan. He was only able to unravel the tale by wrapping it in other tales. Being unstuck in time, whatever way it happens, lets us look at a bigger event in a digestible way.
I get why the story was told that way. But I think that something you said suggests that he actually did travel through time. He predicted his own death and it came to pass. Unless he paid someone to do it then it's a strong suggestion that he did actually experience time in the way that the Tralfamidorians did.