The Madman who became a wizard

What if at 'The End of Time' episode, instead of being sucked back into the Time-War, the Master fell through a crack in the universe and ended up on an alternate earth, memories erased, and transformed into a baby? What if he was then adopted by the Potters and named 'Harry Potter? What if he then discovered who he really was? What if the Doctor and Clara followed him soon after?