Plans for a massive barn conversion in Burnham Thorpe which have sparked a flood of objections have been turned down.