I just read a blog post by James Padolsey (Making APIs is hard). He makes a comment that is at the core of how we design APIs in my day job. From the post:
I am sitting here thinking about that end-user-programmer-guy. I guess this holy API should seek to provide exactly what this guy wants. The trick is knowing what he wants.
While I don't agree with the entire blog post (in particular, there's way more to API design than just naming), I do believe James has partially nailed the essence of what makes API design difficult.
At Infragistics, we have a design tenet that states we will optimize for the 80% case while still remaining flexible enough to accommodate the other 20%. What this means is, in 80% of the use cases for what we are designing the developer should be able to use it without consulting product guidance. This requires a lot of research to determine what the 80% case is, but in the end the result is a product that is easy to use most of the time.
That's not to say that we don't care about the other 20%. We just don't design for that first. The basic idea is: the more common it is, the easier it should be. On the flip side: the less common it is, the more difficult it can be.