In the heart of the East End, there is a place that dates back 11 generations — located on an 8,000-acre island that Nathaniel Sylvester and his wife, Grizzell, bought from the Manhansett Indians in 1651.
It would become known as Shelter Island, home to their Sylvester Manor, where they loved and lived off the land, a romance that has spanned centuries.
But there is a parallel story — one that is hushed, and rarely ever talked about.
It paints slavery as alive and well in early American history, but not just in the South. Of all the northern states, New York was home to more slaves than any other, and on Sylvester Manor alone, its more than 20 enslaved people made it one of the largest concentrations in the region — the last slave freed in 1821, just six years before the state banned slavery and 42 years before the Emancipation Proclamation.
“This conversation that we’re having, in the greater context, it’s not about shame,” Sylvester Manor curator and archivist Donnamarie Barnes said. “It’s not about pointing the finger and saying, ‘You did this,’ or ‘You didn’t do that.’ We all agree: slavery was a terrible thing. That’s not the argument. The point is this is a history that has been hidden. Whatever the reasons are, we don’t have time to point fingers or say, ‘Shame on you.’ All we can do now is be open and honest and face it.”
On Sunday afternoon, Barnes will do just that, seated alongside a panel of fellow experts at Bay Street Theater in Sag Harbor for the discussion, “Hidden in Plain Sight: Facing the Enslaved History of the East End,” as part of the fifth annual Black History Month Celebration.
Presented in collaboration with the Eastville Community Historical Society, the talk will explore the history of slavery on the East End — and the omission of that history from the founding narrative of the United States — explained panelist Aileen Novick, site administrator and project manager at Hempsted Houses of Connecticut Landmarks, who will also be joined by Georgette Grier-Key, executive director and curator of the Eastville Community Historical Society, and David Rattray, owner and editor of The East Hampton Starand director of the East Hampton Plain Sight Project.
“Especially up north, people still don’t like to talk about slavery. In general, across the whole country, people are really uncomfortable talking about it — everywhere,” Novick said, noting that many Hempsted Houses visitors are still surprised to hear about slavery in New London.
“The North did a really good job of hiding that history and there’s been such a romanticism of the past that I’m not sure we’d even recognize colonial times if we saw it,” she continued. “This country never really came to any reconciliation of that history and, instead, people buried it all over the country. I think that’s why we see such problems today. There’s never really been a good discussion of ‘What was slavery?’ in these different communities.”
The reason for this silence remains unknown, both women agreed. Speculation could point to embarrassment — and a strong desire to hold onto the narrative that most northerners were righteous abolitionists and southerners were evil plantation owners — but Barnes said she is attempting to flip the script and shine a new light on enslaved members of the community by, instead, looking at their contributions.
“Let’s say we find a record that the blacksmith of East Hampton had two enslaved African boys working in his blacksmith shop,” she said. “Well, in a blacksmith shop, young apprentices learn how to make nails. So let’s say these two young boys learned how to make nails and those are the nails that the people used to build their houses. If we extrapolate and say the two slave boys working in the blacksmith shop made the nails that built the town, isn’t that a different way of talking about the history?
“You make the story come alive and give it three dimensions,” she continued. “It’s not about shame. It’s about, ‘Wow, these people were an integral part of this society from the beginning. We haven’t acknowledged them; we don’t even know their names. But let’s find out.’ And that’s the point of it all.”
The sheer number of enslaved people on the East End remains a mystery — albeit a work in progress — though Barnes said she might never know. A tribute rock on the Sylvester Manor property reads, “Burying Ground of the Colored People of the Manor From 1651,” where an estimated 200 bodies remain there.
“So the question has always been, who are these people? Two hundred people buried, that’s a very large number,” Barnes said. “In 1680, when Nathaniel Sylvester died, he listed 23 enslaved black people on his will. Twenty-three in 1680, versus 200 buried. So who are these people? It’s a big question that I don’t have answers for, but it starts a conversation in a kind of way that you don’t usually have.”
The history lost can be disheartening, Novick said, but with every new discovery, even a new name, researchers across the Northeast are one step closer to a more complete truth.
“Some African American families have a strong feeling about their history, but they feel like no one’s ever let them tell that story. It’s hidden away,” Novick said. “Sharing that history to really get people talking is really important right now, with all these conversations on race. It’s so important to be true to what the history was and, sometimes, we don’t acknowledge the history, and it’s hard to see where people are coming from.”
The panel discussion “Hidden in Plain Sight: Facing the Enslaved History of the East End,” with Georgette Grier-Key, Aileen Novick, David Rattray and Donnamarie Barnes, will be held on Sunday, February 24, from 2 to 4 p.m. at Bay Street Theater, located at 1 Bay Street in Sag Harbor. Advance tickets are $15 and $20 at the door. For more information, call (631) 749-0626 or visit sylvestermanor.org.