I also often feel uncomfortable with wage equity arguments on gender grounds, and you're absolutely right that there are women parking lot attendants and male library clerks. I think that the differences are often historical, though, and built into the system.
And actually, I think that wage negotiators often do compare wages for similar jobs at different institutions and ask for equity - I know that's been done with college teaching positions, for example. Not that they always get it, but it's a common tactic. Especially when they are public service jobs, funded by the government, so the institution is arbitrarily choosing how to use the funding. It's a bit different in the private sector, where there is competition, but often then wages will be used as a way to get and keep good people.
But I respect your position - no doubt there are lots of others who share it! And unfortunately the longer they are out the more impatient and unsympathetic people will be.