Elon Musk engaged in a brief affair last fall with the wife of Sergey Brin, prompting the Google co-founder to file for divorce earlier this year and ending the tech billionaires’ long friendship, according to people familiar with the matter.

Their falling out is one of a string of personal issues Mr. Musk has faced even as he juggles business challenges, including manufacturing disruptions at Tesla and a court fight over his desire to withdraw his $44 billion bid for Twitter Inc.