Damon and Elena did meet first, when a fight between Matt and Elena drove her away from the party. Damon did compel her to forget him, but he didn't know that she was a Gilbert. The Gilbert's never forgot their history in Mystic Falls, and would never go out at night without protecting themselves.