If your contract has validity, then the credit note is only part of the solution ad only for past invoice(s). What your customer wants is future proofing against being invoiced for your services under a valid contract with Albany - although honestly I do not know whether the validity of the contract is affected by the Liquidation and/or Administration, and whether any invoicing can in effect continue to take place from those companies under Administration. You will need to seek legal advice in this area - speak to your customer's legal department (this is what I did.) Remember that companies under administration can then reinitiate trading. I doubt this will happen with the Albany set as the name is mud and they would sooner start afresh with a new incorporation.
My client refused to set up a parallel contract too until Albany terminated the current contract. I was lucky enough to get my contract terminated effective on the actual start date as it was only 3 weeks in, and there were no issued invoices. My client was also kind enough to threaten legal action with the premise that I would not come in to work...
Remember, as long as you keep going to work, there is no breach of the Albany/Customer contract, and your customer will need to find grounds to terminate - e.g. terminate your emplyment, or make your position redundant and employ you under another auspice... All time consuming, and labourious.
I wish you all the best!