As President-elect Donald Trump fills out his Cabinet and chooses his closet advisers ahead of Inauguration Day, many African American leaders are asking why more Black people haven’t been appointed to key positions.

ABC News spoke to more than a half dozen longtime and new African American conservatives and Republicans within Trumpworld, in and outside of Washington.

Last month, the president-elect appointed Scott Turner, who was executive director of the White House Opportunity and Revitalization Council during Trump’s first term, to serve as secretary of the Department of Housing and Urban Development.

One area of frustration for many Black Republicans was speculation that if Trump did nominate an African American to his administration, it would be at HUD, the department that has had the most Black secretary appointments of any.

One Black Southern Republican told ABC News, “Why is every Black person given HUD?” adding that it was “the literal Black job of the administration.”

ABC News reached out to Trump’s transition team for comment on his selections but did not get a reply.

Six African Americans have served as HUD secretaries, including Robert Weaver, the first. His appointment in 1966 by President Lyndon B. Johnson also made him the first African American appointed to a Cabinet-level position.

The first African American woman to serve in a presidential cabinet was Patricia Roberts Harris, who was the first African American Health and Human Services director and later HUD secretary. She served under President Jimmy Carter.

Former HUD Secretary Samuel Pierce was the only Black Cabinet official in the Ronald Reagan administration. Dr. Ben Carson at HUD was Trump’s sole Black cabinet member in his first term.

In mid-November, Carson posted that he was “excited to speak with President Trump about how I will continue to advance the America First agenda, and I am meeting with him in the near future.”

“However, contrary to reports, I will not be serving as the Surgeon General,” he said.

The departments of Treasury and Interior remain the only departments that have never had a Black secretary.

Another key position Black conservatives believe needs to be filled by an African American is assistant to the president, a senior-level role within the White House. Trump previously appointed Omarosa Manigault Newman, a contestant on his “The Apprentice” TV series, as the sole Black assistant to the president.

“I do think if you’re really talking about Black influence inside the White House, does Trump want his only legacy of having the only AP for two terms be Omarosa?” one Black Republican strategist asked.

After Newman’s departure, Ja’Ron Smith served as a special assistant on legislative affairs before rising to deputy assistant to the president for domestic policy. Ashley Bell, a prominent Black Republican, served as a White House adviser on entrepreneurship and a regional administrator for the Small Business Administration. Bell’s former boss is former SBA Administrator Linda McMahon, who Trump announced as his pick for the Department of Education. McMahon also is co-chair of the Trump-Vance transition team.

Some of the former president’s most ardent defenders have grown restless with the lack of Black appointees, as Trump selected several others who would be firsts in those positions. If confirmed, Sen. Marco Rubio would be the first Hispanic secretary of state; Scott Bessent would be the first openly gay Treasury secretary; and Tulsi Gabbard will be the first Asian American and youngest-ever director of national intelligence.

“I can’t tweet that we need more Black conservatives because the left will attack me saying it’s a DEI hire,” the southern Republican said.

However, one Black Republican operative told ABC News it is still very early when it comes to appointments.

“The Republican Party has never really fallen into the category of ‘representation matters.’ Our strength comes from diversity, but that is not our bumper sticker slogan. We’re not going to nominate Black folks for the sake of nominating Black people,” the Republican operative said.

The operative noted that Black Republicans have made strides in leadership across the country. Sen. Tim Scott will chair the National Republican Senatorial Committee; Rep. Byron Donald is speculated as possibly running for governor or U.S. Senate in Florida; and Winsome Sears is seeking to become the first Black female governor in the country in Virginia.

Donalds on CNN last month took aim at President Joe Biden’s focus on diversity within his administration when asked about the lack of diversity among Trump’s nominees, saying “if you look at how the Democrats filled Joe Biden’s cabinet, they wanted to have a piece of every identity. But did they get the job done? Did they actually serve the interest of the American people?”

“What Donald Trump’s election is about, is bringing competency and reality back to D.C. in the White House, regardless of their race, regardless of their religion, regardless of their creed.” the Florida congressman added.

Although some Black leaders inside and out of politics are highly qualified, another barrier that Black conservatives face is added scrutiny because of their dual identity.

Dedrick Asante-Muhammad, president of the Joint Center for Political and Economic Studies, a non-partisan organization that studies diversity in government and congressional staff, said that the Trump administration differs vastly from past Republican administrations.

“Trump isn’t a traditional Republican, he doesn’t use traditional Republican institutions in the same way that past presidential candidates have, so there isn’t a kind of pipeline, long-term relations,” Asante-Muhammad said.

“Being a part of Trumpworld isn’t easy. It is almost a personal blacklist thing in the outside world. So it is risky, in my opinion, to be a part of this organization for Black people that may want to be a part of the cause, but it’s not worth their bottom dollar,” one Black Republican strategist added.

“When Black people are put underneath this microscope of being Trump-affiliated, they look past your color and you are public enemy number one,” a Black Republican operative said.

The Republican strategist said Black Republicans face a different set of expectations — they don’t necessarily thrive in spaces that are massively disruptive, such as the second Trump term.

“I don’t think disruption really leans into our skillset the way things are currently set up, we have to play things a little bit differently. There’s no Black Republican version of Matt Gatez. They don’t last,” the strategist added.

“Those of us who have survived and been there for years and made it work have done so by being steady, consistent, reliable, and trustworthy, ” the strategist said.

The majority of Black Republicans who spoke to ABC News acknowledged that while the optics of Trump being surrounded by mostly white people aren’t ideal, they believe if Trump delivers on a better economy, securing the border, Second Step Act, and judicial reform it could be transformative for the Black community, but they do want the former president to hire more Black conservatives.