Part of the boiler house at the power station collapsed on 23 February during demolition work by Coleman and Company, killing one person, leaving three missing in the wreckage and injuring five others.
In a statement, Coleman and Company said the move was "hugely disappointing".
"We have now reached a stage where we will be handing over the remaining recovery operation to another contractor within the next few weeks. This is of course hugely disappointing as we all wanted to recover our friends and colleagues and return them to their families," it said.
"It has been explained to us that this decision has been made for reasons of safeguarding employee welfare and the preservation of critical evidence ... While the sequence of future recovery operations is not our decision, all stakeholders are agreed that the safety of those involved in the process going forward is of paramount importance."
Coleman and Company will remain as a principal contractor for the wider site and, once the recovery is complete, hopes to resume work to complete the contract.
The company did not announce the replacement contractor, however DemolitionNews "understands that power station demolition specialist Brown and Mason has been consulting with client RWE Npower with regard to a planned blast of the remaining portion of the boiler house".
