Yes, "society" may be part of what runs a market, but it is not what determines value. The market itself, which is an abstract force made up of more than just "society" and has a dynamic all its own (which is what is fascinating to me about economics) that cannot be reduced to the effects of "society" is what determines value.
The discussion becomes less confusing if one is careful with terminology.
Markets determine
Prices. Prices are different from values.
the rest of society outside markets is interested in prices, because it is assumed -- whether right or wrong is a different matter -- that prices are (weak) indicators of market-external facts, which we may call
values. Examples of such market external facts are (1) notions of optimal resource allocation, (2) socially aggregated individual preference orderings, (3) the average amount of labour that needed to go into the production of a given priced item, (4) scarcity of a good, (5) expected future developments of one of the above, (5) a metaphysical notion of value that is inherent in things ... and i'm sure one can come up iwth more.
Hence prices (which is what markets produce), can be seen as a quantification and linearistation and simplication of some market-external value).
Because those notions of market-external value are not readily observable, market-based pricing mechanism are deemded to be good tools to circumvent the difficulties in observing market-external values.
Markets don't determine prices outside of society, very much on the contrary. But because markets are
large, distributed and self-referential systems, market prices are no longer predicatble to society (and in fact to market participants themselves), at least with precision. This unpredicatbility is often simplified as
market autonomy or
market forces.