No I think that's reasonable, I thought about something like this too, and the underlying component is made with that in mind. The layout in particular can be saved/loaded from JSON if needed.
The problem indeed is how to create a UI that can be developed relatively easily. Drag and drop is often messy to implement so I'd favour a solution without it but not sure what.
