In Marion prior to 1858, Protestants and Catholics were buried alongside each other in the Old Cemetery located on Quarry Street. This cemetery was eventually deemed to be poorly located and too small for the growing need.

In 1857, citizens organized the Marion Cemetery Association and purchased forty-seven acres of land for a new city cemetery, to be used for burials of all denominations. However, the Catholics preferred to all remain together. In 1858 the area located at the southwest corner of Marion Cemetery was purchased by Cincinnati Archbishop John Baptist Purcell for $337.50. This new "Catholic Section" was set aside for the sole purpose of burying Catholics.

In 1892, Joseph and Sarah Johnston offered to sell their parcel of land located directly across the street. Their twenty-two acres were purchased by Archbishop William Henry Elder for $3,500. This land became St. Mary Cemetery, and the people buried in the Catholic Section at Marion Cemetery were reinterred in the new site. Today, the cemetery is governed by the St. Mary Cemetery Board and maintained for the benefit of the parishioners.