Ask JB how long it can take to build a house.
Having said that, he stated that it was built by GS, therefore, somehow it was built. However no keep , nor portcullis. My credulity can only be stretched so far.
I'm intrigued as to the reason the castle was built at all. What narrative purpose does it serve? Think about it for a second, in a residential area, JB built a medieval castle. Granted that the neighbors are deaf, dumb and blind, given all the mayhem that has occurred at Harry's home. But a castle?