It's strange how this seems to happen to certain people. Charming can do whatever he'd like apparently because doors will open to Storybrooke and the Enchanted Forest, but only for so long and never to where he wants. He needs to be back aboard the Jolly Roger with his family before something happens to him. "That sounds like I should buy you more than another drink, then. You deserve a pitcher."