Frederick Mercer was born in Birmingham. By the time of the 1871 Census, he was working as a photographer, while still living with his parents and five surviving siblings in Fordrough Street, Birmingham. He had also been taking painting lessons and began exhibiting that year at the Royal Birmingham Society of Artists.

His extensive travels took him to Cornwall, and he is likely to have first visited Polperro in the 1870s.

After his marriage to Elizabeth Alice Tompson (1850-1927) he moved from Birmingham to Staffordshire.