Rashtriya Janata Dal (RJD) chief and Union Railway Minister has said the Congress too was responsible for the demolition of the Babri mosque as it did nothing to prevent it.