To design a proper survey you only vary the single item you want to investigate. So if you want to see if yellow cars are faster than green cars you take two cars of the same make and model, with the only difference between them being the colour. So you might compare the speed of a yellow 2CV and a green 2CV.

This ensures you are testing for differences caused only by the variable you are interested in (the colour). Doing the same test by comparing a yellow 2CV and a green BMW would clearly not tell you anything about how colour effects the speed of a car.

But this nonsense is exactly the fundamental failure of all surveys I have ever seen into the ‘pay gap’. The studies do not compare only the variable they are trying to assess, but compare two vastly different groups of employees. By simply comparing the pay of men and women in similar jobs they are not testing only for the difference in pay between genders, they are assessing what the difference in pay is between :-

1. A group of workers who usually work longer hours, full-time and wish to build up their careers through continuity of employment and proving their commitment by tolerating personal inconvenience (working away, for example);

and

2. A group of workers who work part-time, often ask for flexible working and flexible holidays, don’t want to work away from home because of child-care commitments, may take months off for paid child-rearing duties

Of course, group 1 is mostly men, and group 2 is mostly women, but this isn’t because of discrimination. This is simply because this is how most people in our society choose to organise themselves. So these studies are comparing the difference in pay between these two groups and not between men and women.

To find out what the real gender pay-gap is (if they really wanted to know) they should compare Men in group 1 above with women in group 1 above, ie women who may not have children but in any case who choose to pursue their careers as most men do. (Of course, they could also compare men in group 2 with women in group 2).

If they did this, they would find out what the true difference is in pay between genders. Sadly, it seems the real gender pay-gap is the last thing most studies are interested in finding out.