Joe Biden declared he “should not have been so cavalier” on Friday after he told a prominent black radio host that African Americans who back President Donald Trump “ain’t black.”

The presumptive Democratic presidential nominee quickly moved to address the fallout from his remark, which was interpreted by some as presuming black Americans would vote for him. In a call with the U.S. Black Chamber of Commerce that was added to his public schedule, Biden said he would never “take the African American community for granted.”

“I shouldn’t have been such a wise guy,” Biden said. “No one should have to vote for any party based on their race or religion or background.”

That was an acknowledgement of the stinging criticism he received in response to his comments, which he made earlier in the day on “The Breakfast Club,” a radio program that is popular in the black community.

The rebukes included allies of Trump’s reelection campaign — anxious to go on the offense after weeks of defending the Republican president’s response to the coronavirus pandemic — and some activists who warned that Biden must still court black voters, even if African Americans overwhelmingly oppose the president.

“None of us can afford for the party or for this campaign to mess this election up, and comments like these are the kinds that frankly either make black voters feel like we’re not really valued and people don’t care if we show up or not,” said Alicia Garza, a Black Lives Matter co-founder and principal of Black Futures Lab.

Near the end of Biden’s appearance on the radio program, host Charlamagne Tha God pressed him on reports that he is considering Minnesota Sen. Amy Klobuchar, who is white, to be his vice presidential running mate. The host told Biden that black voters “saved your political life in the primaries” and “have things they want from you.”

Biden said that “I guarantee you there are multiple black women being considered. Multiple.”

A Biden aide then sought to end the interview, prompting the host to say, “You can’t do that to black media.”

Biden responded, “I do that to black media and white media,” and said his wife needed to use the television studio.

He then added: “If you’ve got a problem figuring out whether you’re for me or for Trump, then you ain’t black.”

Trump’s campaign and his allies immediately seized on Biden’s comments. South Carolina Sen. Tim Scott, a Trump supporter and the Senate’s sole black Republican, said he was “shocked and surprised” by Biden’s remarks.

“I was struck by the condescension and the arrogance in his comments,” Scott said in a conference call arranged by the Trump campaign. “I could not believe my ears that he would stoop so low to tell folks what they should do, how they should think and what it means to be black.”

Charlamagne Tha God later said on CNN, “A black woman running mate is necessary, especially after today.” He added that the question of “what makes somebody black” is a discussion for black people, not for “a white man.”

Trump himself has a history of incendiary rhetoric related to race.

When he launched his presidential campaign in 2015, Trump called many Mexican immigrants “rapists.” Campaigning in 2016, he asked black voters, “What the hell do you have to lose?”

In 2017, he said there are good people on “both sides” of the clash in Charlottesville, Virginia, between white supremacists and anti-racist demonstrators that left one counterprotester dead.

In 2018, during a private White House meeting on immigration, Trump wondered why the United States was admitting so many immigrants from “shithole countries” like African nations. He also blasted four Democratic congresswomen of color, saying they hate America and should “go back” to where they come from, even though all are U.S. citizens and three were born in the U.S.

Black voters helped resurrect Biden’s campaign in this year’s primaries with a second-place finish in the Nevada caucuses and a resounding win in the South Carolina primary after he’d started with embarrassing finishes in overwhelmingly white Iowa and New Hampshire. Sixty-one percent of black voters supported Biden during the primary season, according to AP VoteCast surveys across 17 states that voted in February and March.

Biden is now seeking to maintain his standing with black voters while building the type of multiracial and multigenerational coalition that twice elected Barack Obama, whom he served as vice president. He has already committed to picking a woman as his running mate and is considering several African American contenders who could energize black voters. But Biden is also considering candidates such as Klobuchar, who could appeal to white moderates.

There is little chance of a sudden shift in support for Trump among black voters. A recent Fox News poll shows just 14% of African Americans who are registered to vote have a favorable opinion of Trump, compared with 84% who view him unfavorably.

Seventy-five percent of African American registered voters say they have a favorable view of Biden; 21% hold an unfavorable opinion.

There is a risk, however, of black voters, especially those who are younger, staying home in November, which could complicate Biden’s path to victory in a tight election. The Breakfast Club is a particularly notable venue for Biden’s comments because the program is popular among younger African Americans.

Adrianne Shropshire, executive director of BlackPAC, a national organization that works to mobilize black voters, said many black Americans are loyal Biden supporters. But she said his comments make it harder to attract people who are on the fence about voting.

“The first thing I thought about was to what degree did this just turn off those voters and how much more work the rest of us are going to have to do to convince people that it is worth their time and their efforts,” she said.

Biden’s selection of a running mate, along with his pledge to appoint the first black female Supreme Court justice, could help motivate voters. He’s begun vetting vice presidential contenders, a process he’s said will likely last through July.

Several black women are among those under consideration, including California Sen. Kamala Harris, Georgia voting rights advocate Stacey Abrams, Atlanta Mayor Keisha Lance Bottoms, Florida Rep. Val Demings, Ohio Rep. Marcia Fudge and Susan Rice, Obama’s former U.S. ambassador to the U.N.