When it comes to the history of people of color in Sag Harbor, the predominant thru-line revolves around the Eastville community, the involvement of Black sailors in the 19th century whaling trade, and, in the early 20th century, the rise of the historic summer subdivisions of Sag Harbor Hills, Azurest and Ninveh Beach (aka SANS).
However, there’s a question that remains unanswered — and often unasked. What about people of color who populated Sag Harbor in earlier times? Long before whaling voyages or summer cottages, there was the port of Sag Harbor, a vital shipping cog in the economic hub of a burgeoning nation. With people and goods coming and going by sea as far back as Sag Harbor’s founding in 1707, and slavery legal in New York until 1827, enslaved individuals surely played an integral role in the economic life of the region — both in terms of the labor required to build and maintain this place and the goods that were shipped out to slaveholding societies in the Caribbean.
But finding evidence of those early residents of color can be challenging — unless you know where to look.
Sag Harbor’s Donnamarie Barnes, a photographer and the curator and archivist of Sylvester Manor Educational Farm on Shelter Island, and East Hampton’s David Rattray, editor of the East Hampton Star, are co-founders of The Plain Sight Project. The mission of the project is to uncover the names and stories of the East End’s enslaved and free Black population from the 1600s to the mid-19th century, locate and preserve burial grounds, living quarters and work sites, and create a template for other communities to follow.
Since the creation of The Plain Sight Project in 2018, Ms. Barnes and Mr. Rattray have focused their efforts largely on areas in and around East Hampton Village. But now, they are now extending their search and hoping to discover the identity of Sag Harbor’s earliest Black inhabitants.
On Sunday, August 8, at 4 p.m., the pair will present “Forgetting to Remember: Sag Harbor’s Role in Slavery and the Path to Reconciliation” as part of “Projections,” a new series of talks at Sag Harbor Cinema. In their research, Ms. Barnes and Mr. Rattray are exploring the area’s relationship to slavery, and through their efforts, hope to identify and tell stories of the individuals who lived, worked and died in the community. The discussion will be moderated by Dr. Jennifer L. Morgan, a part-time Sag Harbor resident, a professor of history at New York University and author of the book “Reckoning With Slavery, Gender, Kinship and Capitalism in the Early Black Atlantic.”
“We assumed there were enslaved people here, but didn’t know any names. We’re starting to look at the history of Sag Harbor from its founding and reading the history books that are popular and available, but there seems to be a lack of information,” Ms. Barnes said in a recent interview, noting that there are descriptions of how the village was built, but what’s missing are details about who did the building.
“It’s a classic passive move,” Dr. Morgan added. “It’s not that they don’t know, it’s that the work of indentured and enslaved labor is never the main part of the story. It’s not understood to be part of what you write down.”
Though her efforts are just beginning, in the archives of John Jermain Memorial Library, Ms. Barnes found the first name of an individual of color in her Sag Harbor research. It came from Presbyterian Church records and was a payment of 30 shillings a year to “Luce — colored woman” to sweep the church meeting house in 1784.
“The records repeat the payments to her to sweep and also to ring the bell through 1796,” Ms. Barnes explained. “As of yet, I don’t know if Luce was enslaved and if so, who owned her, or if she was a free woman of color. But the frequency of the mentions of her name let me know that there would be others and that they were here in Sag Harbor, but until now forgotten.”
Money makes the world go round — it’s true now and was true in colonial America, and like the evidence linking Luce to her work at the church, Mr. Rattray notes that clues about the presence of people of color are found in records of commerce.
“Sag Harbor is an incredibly important fulcrum of economic life because it was a port from really early on. By the 1730s, they’re using it substantially for getting and receiving goods of value,” said Mr. Rattray in a recent phone interview. “Sag Harbor becomes internationally important from being a whaling port and regionally as a manufacturing center, but before that it was tied to slavery in the West Indies through exports and imports.”
Mr. Rattray says that evidence of how Sag Harbor functioned in the 18th century can be seen in the well-documented history of nearby Sylvester Manor, which was established in 1651 as a provisioning plantation for the family’s holdings in Barbados, especially its sugar operations
“We can see that people paid debts in sugar — usually raw — so there’s evidence that shows the whole East End of Long Island was engaged in the greater Atlantic economy. It was driven by sugar and sugar was driven by slavery,” Mr. Rattray said. “Nearly all commerce activity in the colonial era was tied to sugar and the wealth it created.”
Mr. Rattray explains that items that couldn’t be made or obtained in the Caribbean, like barrel staves, shingles and even livestock, were procured from New England or places like the East End. He has found documentation indicating that Sag Harbor’s first U.S. Customs master, Henry Packer Dering (a Sylvester descendent who was appointed to his post by George Washington in the late 1780s), was involved in an effort to sell Gardiners Island goods to plantations in Haiti — specifically, barrels, salted fish and horses.
“If you have something to sell on eastern Long Island in the 1740s, odds are that market is in the Caribbean, of course,” he said.
One intriguing line of inquiry that interests Mr. Rattray involves the grazing pastures of Montauk where cattle and sheep were sent each summer to fatten up.
“There were thousands of heads of cattle and sheep there. We’ve been told the colonialists were self-sufficient, but why did they need this much meat?” he asked. “I hope we can show the connection between agriculture — livestock, flax and wool — and the West Indies with sugar coming into the economy.”
For now, Mr. Rattray’s and Ms. Barnes’ mission is to find records of Sag Harbor’s colonial-era residents who not only made that economy run, but also owned slaves. Because they used a hybrid barter system rather than cash, Mr. Rattray notes that merchants kept meticulous account books of what was traded and the value. It’s likely this will include human property, and as Ms. Barnes points out, Packer was not just customs master, he was also a fourth generation slave holder.
“That’s not an aberration — it was a natural state and it continues for four generations,” said Ms. Barnes who, as part of her research, is compiling a list of manumissions records of the East End — noting when enslaved people were released from slavery and by whom.
“There was a rush of manumissions after the revolution. We have to look at that,” she said. “How is that economy changing? After being in exile during the war, do they have to redo their businesses? Can they not house and clothe them?
“We feel it’s our duty to present the information. What to do with the information is the community’s decision,” Ms. Barnes added. “Now that you know these stories and names, how are you going to pass along this knowledge so it’s never forgotten — so it’s acknowledged as part of the history of our village?”
“There were vast numbers of enslaved here early on. There were more Black people walking the streets in the 1790s than there are today,” Mr. Rattray added. “Why I think Sag Harbor is really important is, it’s a ‘follow the money’ story. If we can demonstrate that Black people contributed vastly to the creation of what we know today, their lives and roles have to be recognized.
“The whole coming year we’ll be digging into Sag Harbor records. I think small communities in the North are the next frontier in the awareness of slavery,” he said. “‘The Plain Sight Project’ approach asks the simple question of who was here? Just by asking the question, you can change people’s perceptions. Asking the question is a step toward reconciliation and making things better today.”
“Forgetting to Remember: Sag Harbor’s Role in Slavery and the Path to Reconciliation” is Sunday, August 8, at 4 p.m., at Sag Harbor Cinema, 90 Main Street, Sag Harbor. Register at sagharborcinema.org.