The #FreeBritney movement seemingly faced a setback this week when Britney Spears's request to have her dad removed as conservator of her estate was denied. But a legal expert views it a different way.

Attorney Harry Nelson, founder and managing partner of Nelson Hardiman, tells Yahoo Entertainment he believes Judge Penny's rejection is strictly "a matter of procedure." In fact, Nelson thinks the "conservatorship's days are numbered" after Bessemer Trust requested to resign.

The wealth management firm was set to take over as the co-conservator of Britney's estate with Jamie Spears, but is backing out "due to changed circumstances." (Conservator of the estate controls all of the singer's financial decisions while Jodi Montgomery is conservator of the person.) Bessemer Trust stated in papers obtained by the New York Times it was told Britney's conservatorship was voluntary and that she consented to the company acting as co-conservator. Britney made it very clear in her emotional 24-minute address to the court she wants the "abusive" conservatorship to end. The company said it "respects her wishes."

"Bessemer Trust's decision to seek to withdraw as co-conservator with Britney's dad tells us everything we need to know. There is a major 'stink' on this conservatorship, and they want to get out in front of it as damage control for having played a role in an abusive situation," says Nelson. "This conservatorship's days are numbered. Britney will be free sometime soon."

That's not the indication many people got this week when Judge Penny denied Britney's request to have Jamie removed as co-conservator. Last week, the singer compared her father to a sex trafficker and said he "loves" the control he wields over her. However, Nelson believes the ruling isn't indicative of what will soon happen and that the judge "didn't want to rush procedurally."

"I read Judge Penny's denial as strictly a matter of procedure. Judge Penny wants the issue to be presented formally in a petition to terminate where all parties can weigh in before she rules, and — no matter how compelling Britney’s comments were — did not want to deviate from the process," he says. "Every indication is that, upon presentation of the petition, this conservatorship will be terminated."

However, no petition to terminate the conservatorship has been filed.

Nelson says it's "head-scratching" that Britney's court-appointed lawyer, Samuel Ingham III, has yet to officially file papers to terminate the conservatorship after the singer made it clear that was her wish.