What about the option to demolish a single building 1 time, or like the realm transfer rules of once per 90 days. This would allow people to fix early game mistakes without allowing people to take advantage during might growth events. This could be done by making a request to ATA just like you would for a realm transfer.
The suggestion for destroying buildings has been made several times. I'm sure ATA has it in their list of ideas for future updates. Honestly not sure why something as important as destroying buildings (2nd farm/quarry?) is put on the back burner.
Definitely something our development team has on our list of future tasks for consideration, though we don't have any ETA for this at the current time.