A. Fred Bultman was the second child and only son of A. Fred and Pauline Bultman. His family was prominent in New Orleans, where his father owned a Catholic funeral company. By the age of thirteen he was interested in art, and worked with Morris Graves, who was a family friend. As a high school junior in 1935 Fritz went to study in Munich for two years, and there boarded with Maria Hofmann, the wife of artist and teacher Hans Hofmann. After returning to the United States he studied with Hofmann in New York City and Provincetown, Massachusetts. Despite his father's wishes that he become an architect, with Hofmann's encouragement he decided instead to continue his study of art. In 1944 he bought a house in Provincetown, and thenceforth Bultman and his wife Jeanne divided their time between Cape Cod and New York City.
His early paintings have been described as "rough and painterly", an amalgam of symbolism and geometry.