Nobody gets everything in this life. You decide your priorities and you make your choices. I'd decided long ago that any cake I had would be eaten.
I start with the story, almost in the old campfire sense, and the story leads to both the characters, which actors should best be cast in this story, and the language. The choice of words, more than anything else, creates the feeling that the story gives off.
Once he became a series character, I made the conscious choice that he would never act like a series character, never wink at the reader, never pull his punches. Better for him, better for me.