In the case of Pratt, he just admitted that he just doesn't want to be that fit. Hemsworth did 2 movies that come out before AoU, and is already doing another one that comes after it. Evans had 2 movies after Avengers, and starred, directed, and produced a movie that comes out who knows when. So different situations for different actors. Especially depending on the role they have in a movie.