Adrian Duncan: ‘I remember praying for a statue not to move because I didn’t want my world to be destroyed’