Founded in 1798 by United Empire Loyalists, Cobourg grew as an important milling and small manufacturing centre in Upper Canada. Harbour construction and a steady rise in population further spurred prosperity and the demand for a town hall.

The municipal council authorized the decision to build a town hall in 1851, but construction on the building did not begin until December 30, 1856 when Sir Allan MacNab laid the building's cornerstone. By 1860, the town hall was completed at a cost of $110,000 (£22,000). On September 7, 1860, the Prince of Wales officially opened the building, named for his mother Queen Victoria. The building housed a number of functions including a court room, judge's chambers, a concert hall, municipal and county administrative offices, council chambers, law offices and Masonic lodge rooms. As Cobourg's economic fortunes faltered, so did the future of Victoria Hall. Expected revenues from office rentals never reached expectations, maintenance declined and it was only in 1938 that the debt for construction costs was retired. Activists were encouraging its restoration by 1949. The building was renovated and restored through fundraising efforts led by the Society for the Restoration of Victoria Hall. In 1983, it was officially reopened, serving many of its traditional functions.

The three-storey town hall was designed by noted architect Kivas Tully in a classical Palladian-influenced style. Constructed of white brick supplied by the local firm of William and David Burnet, the town hall was also faced on three sides with Cleveland sandstone and embellished with elaborately carved ornamentation. Victoria Hall is distinguished by a multi-bay façade, including 34 pilasters and 96 windows, and is notable for its prominent frontispiece comprising a substantial pedimented portico with Corinthian columns enclosing a speaker's balcony. The roofline is dominated by an octagonal belfry and clock tower that rises up above the Cobourg skyline. The interior of the building includes the courtroom, fashioned after the Old Bailey in London, England, and the concert hall that is distinguished by a decorative ceiling with lozenge design, laurel wreaths, arabesques and floral motifs. Its plastered walls are painted with trompe l'œil panels and pilasters.

Victoria Hall was designated a National Historic Site in 1959. The Ontario Heritage Trust secured a heritage easement to conserve the building in 1979. The property was also designated under the Ontario Heritage Act by the Town of Cobourg in 1982.