Triple Dog Dare

Albus Potter isn't afraid of anything, and he definitely isn't afraid of ghosts. He's met lots of friendly ones, like the Fat Friar or Nearly Headless Nick. So why is it that when he's finally old enough to join in on the Potter-Weasley cousins last-day-of-summer shenanigans, he's paralyzed with fear? A muggle "haunted house" can't be that scary, right?