The south coast side agreed to pay more than £2 million in compensation to Swansea to bring the 43-year-old to the Amex, according to the Guardian's Stuart James.
Potter made a name for himself at Swedish outfit Ostersunds FK, leading the budget-conscious side from the fourth tier to the Europa League knockout stages.
Brighton sacked Chris Hughton a day after their final match of the Premier League season, a 4-1 defeat to eventual champions Manchester City.
Swansea, who finished 10th in the Championship, had hoped to keep Potter for the upcoming campaign, reportedly offering him an improved contract that would've made him one of the highest-paid managers in the English second tier. But the lure of the Premier League was enough to convince Potter of the move.
A former full-back for Stoke City and West Bromwich Albion, Potter began his coaching career in Sweden in 2010.
Using unusual techniques - his players famously put on a production of Swan Lake - Potter galvanized a previous unknown in Ostersund. He capped his eight years there with a Swedish Cup in 2016-17 and a 2-1 win at Arsenal during a fairytale run in the 2017-18 Europa League.
He joined Swansea in 2018 and guided them to the quarterfinals of this season's FA Cup.