Graham Poll has been named as one of 14 referees who have been sent home from the World Cup.
The English referee has paid the price for his yellow-card blunder in the Australia versus Croatia game when he booked Josip Simunic three times.
Russian Valentin Ivanov, who issued a World Cup record 16 yellow and four red cards in Portugal's fiery game against Holland, is also on his way home.
Argentine Horacio Elizondo will referee England's quarter-final with Portugal.
Poll, 42, had been tipped as a possible World Cup final referee prior to the tournament getting under way.
But a series of errors in Australia's crucial final Group F match ended any such hopes of being given the showpiece final.
Poll's biggest mistake led to him cautioning Croatia's Simunic three times before finally sending him off.
"Graham Poll is an exceptional referee, who will be able to overcome the situation "
But the Tring official also dismissed Australia's Brett Emerton and Croatia's Dario Simic.
And Poll awarded the Socceroos a first-half penalty for handball by Stjepan Tomas but failed to spot an even more blatant handball by the same player after the break.
Mark Viduka also seemed to be bundled to the ground in the box during the first half, while it appeared that Harry Kewell was offside when he scored Australia's second equaliser.
A statement from Fifa referees committee president Angel Maria Villar Llona defended Poll but also conceded he had made a mistake.
The statement read: "Graham Poll is an exceptional referee and a great sportsman, who will be able to overcome the situation thanks to his strong personality and love of the game.
"Thursday evening's 2-2 draw between Croatia and Australia in Stuttgart saw referee Graham Poll make an error.
"The experienced official is disappointed at having committed the error, the first such mistake in his 26-year career.
"The Fifa referees committee also recognised the oversight and the fact that none of the match officials at the stadium picked up on the error.
"In explaining his actions to the committee, Poll said he incorrectly noted down the name of the Australia number three Craig Moore when booking Simunic for the second time and failed to realise his error."
Poll's assistant referees Philip Sharp and Glenn Turner will also not be involved in the latter stages of the tournament in Germany.
Referees for World Cup quarter-finals:
Germany v Argentina (Lubos Michel, Slovakia)
Italy v Ukraine (Frank de Bleeckere, Belgium)
England v Portugal (Horacio Elizondo, Argentina)
Brazil v France (Luis Medina Cantalejo, Spain)
Other retained referees:
Toru Kamikawa (Japan), Coffi Codjia (Benin), Benito Archundia (Mexico), Jorge Larrionda (Uruguay), Mark Shield (Australia), Massimo Busacca (Switzerland), Markus Merk (Germany), Roberto Rosetti (Italy).