Here's what people seem to not understand. When you're a coach in college, you are so much more than that. Coaches in college are the General Manager as well. They recruit, schedule, etc. In the NBA, the scheduling is pretty much done for you by the NBA. I think Fred just wants to coach, and let GarPax get his players for him (although I don't think the current roster is a Hoiball type roster).
I just can't blame him for not wanting those extra duties. Although, there are AT LEAST 82 games in a season. Then you have preseason and playoffs. Some coaches even participate in the summer league (others relegate job to assistants). The traveling all over would be fun for a bit, but would get really old and annoying after a while. Reggie Miller explained this on NBATV Open Court on one of the shows.