I think it unequivocally causes psychic harm to people - or makes them sad, anyway, in everyday English. And there's still the point that you can never know which species will turn out to be the lynchpin of a particular ecosystem, which, once removed, could have pretty disastrous consequences for our food supplies.
Also, these species are much more likely to be among the obscure, "random" species that few non-specialists have ever heard of - small plants, fungi, invertebrates, etc. - than the charismatic megafauna (polar bears, tigers and so on) that people actually care about.
But really, to me, it's just self-evidently true that a world with polar bears and tigers in it is a richer world that one without those creatures.