David S. Gottesman, a low-profile investment manager who was an ally of Warren Buffett, died Wednesday at the age of 96, according to his company, First Manhattan Co.

Mr. Gottesman, known to friends as Sandy, was a longtime confidant of Mr. Buffett and served on the board of Berkshire Hathaway Inc. Forbes estimated his fortune at $2.9 billion and said most of it came from his ownership of Class A shares of Berkshire Hathaway.