Anthony Albanese has been dealt another blow ahead of the federal election.