By the late 1800s, there existed a class of new millionaires with unprecedented levels of wealth.

Major economic changes brought new sources of wealth through banking, land speculation, canals, railroads and manufacturing. While some older elites invested in these enterprises, many did not keep pace and were soon outdistanced by new entrepreneurs. At Cherry Hill, the family’s fortune was in trouble as early as the 1820s. Such families as the Whitneys, Vanderbilts and Morgans made vast fortunes in railroads, finance and heavy industry. Their way of life stood in direct contrast to the refined existence of the old families. Their New York City mansions and Newport cottages typified their excess and ostentation. Families like the Van Rensselaers would have considered these new millionaires to be vulgar in their taste and ostentatious display of their wealth.

There were no Whitneys, Vanderbilts or Morgans in Albany. Instead, new banking, railroad, and industrial empires were being built in cities such as New York, Pittsburgh, Buffalo and Chicago. The new economy seemed to pass by Albany. This may have seemed an insult to the Van Rensselaers, who had considered themselves to be part of a national elite. Albany’s decline threatened to reduce families like the Van Rensselaers to the status of a merely local elite.

At the same time, the old Albany families and others like them faced challenges from those below them on the social and economic scale. Urban populations changed dramatically after 1890. Unprecedented numbers of Jewish, Catholic and Eastern Orthodox immigrants fled economic, political and religious persecution in Europe. Welcomed by some as a source of cheap labor, they faced discrimination from the older immigrants from England, Germany, Ireland and Scandinavia. Mainly settling in cities, they became important sources of votes for urban political machines, much to the dismay of the elites. And in the 1910s and 1920s, millions of African Americans migrated to northern cities. In Albany, they joined an older black community that traced its ancestry to the Hudson River Valley slaves of the 1700s. Albany’s South End, where Cherry Hill is located, became home to African Americans, Germans, Italians, and Jews from Eastern Europe by the late 19th and early 20th centuries.

By 1876, evidence of industrialization surrounded Cherry Hill. Workers’ houses, along with an icehouse and a brewery, were next door. And just beyond the front gate was the noise and dirt of the Albany & Susquehanna Rail Road and the Olcott Iron Manufacturing Company.

During this time of enormous change, anti-immigrant, antiblack, anti-Catholic, and anti-Semitic sentiments were common among American elites. Threatened by their growing economic and political power, native born, old-stock Americans perceived these new populations as a moral and political threat to America, and unfairly labeled them as lazy, depraved and crude–as unrefined. Such sentiments underlay the movement to restrict immigration, culminating in the National Origins Acts of 1921 and 1924. This legislation established quotas of immigrants from each country, and the quotas were deliberately set to welcome immigrants of some ethnic backgrounds– northern Europeans–and to exclude others–southern and eastern Europeans. It was also during this period that women’s suffrage was on the national agenda. Many women from these old families, politically conservative in their upbringing, were ardent anti-suffragists. Because their political loyalties were tied so closely to their class identity, they would have thought it unacceptable to extend the right to vote to working class and immigrant women.

For families like the Van Rensselaers, the challenges they faced from both the new millionaires and new immigrants were not that different–they were feeling squeezed from two directions. Both groups–the new millionaires from above, and the rising immigrant class from below–took away power and standing that had once belonged to the older American aristocracy. Frightened of becoming powerless and, even worse, irrelevant, these older aristocrats needed to prove they still had some basis for social and cultural authority, something no other group could claim. That something was their colonial lineage. Along with it came the refinement associated with the lives of the colonial aristocracy. Neither money nor numbers could buy real class, they claimed–their history was both beyond challenge and unavailable to newcomers.

This pride in lineage spawned a fascination with tracing one’s ancestry. Older aristocratic families founded hereditary organizations like Daughters of the American Revolution, Colonial Dames of America, The Order of Colonial Lords of Manors in America, United States Daughters of 1812, and Sons of the American Revolution. These patriotic organizations were all exclusive, and to gain membership, one had to be able to trace one’s lineage to service in the wars or earliest Dutch and English settlement in North America. These same families socialized at exclusive clubs. In Albany, these included the Fort Orange Club and the University Club. Nationally, many of these clubs excluded those who did not have the proper lineage, including Catholics, Jews and African Americans. In Albany, the Fort Orange Club excluded African Americans until 1977. While an 1880 founding member of the club was Jewish, there was a long period through the mid-20th century during which no new Jewish members were admitted–until 1967.