Sir Clive Woodward's role as the British Olympic Association's elite performance director has been queried before a parliamentary committee.
Olympic cycling gold medallist Chris Boardman told MPs: "We don't see anything he has to offer cycling that we are not already getting."
UK Athletics chief Ed Warner said: "He has not had any impact in athletics yet and he is a political hot potato.
"We must let sports run themselves and are determined not to be dictated to."
He added: "Ultimately, it has to be about what the performance director of each sport believes in and they must stand and fall by that."
Woodward, who led England to Rugby World Cup success in 2003, is 14 months into his job at the BOA.
"We have a thin layer of athletic talent - it doesn't go very deep"
His mission is to come up with ways to help Team GB finish at least fourth in the medal table at the 2012 Games in London.
The BOA's recruitment of Woodward has attracted controversy as his role was seen by some to be duplicating work already done by UK Sport, the government-backed body which funds elite sport to the tune of more than £100m a year.
Last month, BBC Sport reported that 18 of the 35 Olympic sports believed Woodward had made no impact whatsoever in his first year at the BOA, and many were still unsure as to what his role actually was.
Woodward responded to these criticisms by saying he had spent his first 12 months learning about Olympic sport and working on a model for elite performance based on an amateur golfer.
Earlier this month, the former England and Lions centre unveiled this performance model and announced he was about to start working with his first Olympic athlete, judoka Euan Burton.
UK Sport, with British Judo and the BOA, will assess Burton's progress early next year. If they give Woodward's approach the green light, his scheme will be extended to other Olympic sports that want it.
606: DEBATE
But Warner told members of the Culture, Media and Sport select committee: "My concern for athletics is the risk that we have caught the backlash of two organisations with rival programmes."
The athletics boss rejected a claim by Conservative MP Nigel Evans that the medal haul from the 2004 Games had been "dismal".
Athletics contributed only two of Britain's nine gold medals in Athens.
But Warner conceded that the pool of talent that will make up the team for Beijing - where Great Britain are expected to finish at least eighth to be on target for 2012 - is a shallow one.
He admitted: "We have a thin layer of athletic talent - it doesn't go very deep."
~RS~q~RS~~RS~z~RS~54~RS~)