Typically, when you hear the phrase “college town,” you think frat houses, wild parties and noisy coeds who keep you up half the night. And while that’s not entirely wrong, that’s not what we’re talking about this time. The nation is full of college towns, many of which are known for epic parties and vibrant campus life. But this time, we’re talking about towns that have a history that eclipses the institutions of higher learning.

Burlington, Vermont

Credit: Sean Pavone / Shutterstock.com

If you’re not a student at the University of Vermont, you’re probably most familiar with the town of Burlington as a popular ski-resort haven during the winter season. The largest town in Vermont is also home to a historic downtown district that’s an architecture lover’s paradise. Enjoy the Victorian and Art Deco buildings.

If that doesn’t tickle your fancy, you might be more impressed that the quirky ice cream brand Ben & Jerry’s got its start out of a repurposed gas station in Burlington. The town is also home to an impressive number of official sites that are listed on the National Register of Historic Places: You can find 28 buildings, three shipwrecks and 17 historic districts scattered across the town.

St. Augustine, Florida

Credit: ESB Professional / Shutterstock.com

It's now home to Flagler College, but did you know that St. Augustine is the oldest city in the Sunshine State? Long before the British were colonizing the New World, the Spaniards were planting their flag by creating settlements on both shores. But what you probably don’t know is that St. Augustine wasn’t Spain’s first attempt to create a colony on what is now the eastern seaboard. The nation had tried, unsuccessfully, to create settlements between 1513 and 1559, only to have the expedition leader killed or the location abandoned.

But St. Augustine was a success and was founded to serve as a deterrent to the French. It also stood out during the Pre-Colonial period as a safe haven for slaves who managed to escape British colonies in the north. In 1693, King Charles II of Spain declared that any slaves who arrived in Florida from British colonies would be given their freedom as long as they converted to Catholicism.

Williamsburg, Virginia

Credit: Stephen B. Goodwin / Shutterstock.com

We round out this list with a college town that’s also well known for its historic angle. If you’ve never been to Williamsburg, one of its biggest tourist attractions is Colonial Williamsburg, where you can “go back in time” and see how the Colonists lived before and during the fight for independence. So, it’s no surprise that Williamsburg deserves to be on this list. The historic town was once the capital of Virginia from 1699 when the colony was formally founded until 1780 when it was replaced by Richmond.

And it continued to be a central locale during the Colonial era and Revolutionary War. The planned city served as a backdrop for the Continental Army leading up to the siege of Yorktown that took place in 1781. As for colleges, Williamsburg is best known for the College of William and Mary where historic American figures from the Colonial Era such as Thomas Jefferson honed their education.