Professor Quirinus Quirrell was not an unintelligent man. Maybe allowing Voldemort to take up residence in his head was not the best choice, but at least he had a strong incentive to cover his back.

Something was wrong with James. Something was worse with Albus. Harry can only thank magic that Lily seems to have been spared.