Many people think that in a large family not everyone gets individual attention. Obviously I can't speak from personal experience on this as there are just four kids in my family, but I know that in my friend Luke's family (he's fourth out of 14), his parents frequently do things with just one or two of the kids. Luke and his brother, Mark (the only other one I know) seem to be quite well-adjusted individuals.
The other issue people have with large families is the myth of overpopulation. The world isn't overpopulated, the people just aren't spread out wisely. Every family in the world could live comfortably in part of the US with a modest house and backyard. I believe I've also heard that China, with its one child per family policy, will not be producing enough people to replace the ones that die.
Have kids, people...it won't kill you. ;)