I think that's a fair position to take. There are probably a lot of safeguards in place to make sure you don't get massive doses of radiation, but it's very true - your average TSA agent is no genius and probably would have no idea if anything was awry. I think we could do a lot better than the TSA, in terms of security oversight; it'd be nice to have people who are more professional. TSA agents tend to be all over the place.
As far as I know, TSA agents are not required and possibly not allowed to wear radiation exposure badges. Why should we trust that these machines output a safe level of radiation (for passengers or workers) if the company has proven to be untrustworthy?
Regarding that specific issue, I think the fear is that if they let their employees wear dosimeters, then people would think, "Man, if the agents have to wear dosimeters, then there's probably a serious risk of dangerous levels of radiation." So that puts us at "the simplest way not to have problems is to never know if you have problems," i.e., disable the check engine light in your car and nothing's ever wrong with it.