If I am critically listening it's important that the setting be right and that I am comfortable with my surroundings.
That comfort level matters for me because in order to easily hear sonic differences I have to be relaxed and not on guard or self-conscious about being judged.
It is part of my nature to freeze up if it feels like I am on trial—never a good thing in school when tests were what determined your place in the educational rankings.
Your mileage may vary.
I think it's valuable for each of us to self-examine our strengths and weakness as listeners and evaluators.
I, for example, am far more accurate when I don't know what's on offer. If a listening session starts out with prejudice it takes time and effort to wash away those preconceptions. Much easier if it's merely A and B from which to decide.
As in most things in life, we do our best work when our comfort level is highest.