« I'd take it | Main | Break It Down »

July 02, 2008


"Such arguments indicate disagreement about facts, not just values."

But values are facts, aren't they? Facts viewed from a certain mental perspective. For example a loaf of bread sitting on the table is just a fact. But when a starving person comes along, and you know that they will die without it, then you (the observer) are able to see the loaf in new light - it is *valuable* to him.

I am willing to agree that, in principle, facts and values can be separated. However, I believe that we often greatly overestimate our ability to separate them. There comes a point, in many discussions, where it is clear that more argument will not resolve the issues. Experiment is required.

Let's take a specific example of a debate in which fact and value are closely intertwined: In IQ testing, it is assumed that there is a single underlying factor, called the g factor, that explains the correlation in IQ test scores. But the argument for g is actually rather weak. If there is no g, then it would seem that we have no basis for reducing intelligence to a single number, such as IQ. In theory, there is a fact of the matter here. In practice, this debate has been going on for decades, with no resolution in sight. In this case, it seems to me that the prolonged arguments are about values, not about facts. What are the facts here? The point of Cosma's post is that we don't really know what the facts are. Our values have biased us so strongly that we have not yet done the experiments that are needed.

"In sum, it can make sense to have differing opinions due on differing values, but not to have arguments on differing values similar to our arguments on ordinary facts. Such arguments indicate disagreement about facts, not just values."

In reality, there is always disagreement about some facts, so if two agents have different values, one can try to deceive the other, presenting the deception in the format of an argument. In this case, there is a "disagreement" about the fact that one agent is deceiving the other rather than leading a thuth-seeking argument, and a different disagreement, irrelevent to this one, is used as a lever to influence other's beliefs in a way that will lead to behavior promotes different values.

Ian, Peter, Vladimir, I don't think I disagree with any of you here.

People's values do change as the result of conversation. They may not change as a result of rational argument, but they can change, and conversation is one of the better ways to fiddle around inside someone's brain.
Other people sharing my values advances the resulting goals. (Excluding "more strawberries for me!' situations.)

More generally, if you think others are mistaken on their values, and that your reasons should help them see their true values, then your dispute is on facts of what are their actual values.

Do you really think that people can be ignorant of their true values? I ask because my One True Moral Value is much easier to argue in such a situation.

Arguing values and rhetoric can be a way of making the other change their values. It's not about sharing facts but about excercising power on the other so that they will advance the cause of your values.

James and an, if you know that your values would change if you were made aware that you were in a certain sort of situation, then your values are really depend on that kind of situation, and an argument persuading you in this manner would be persuading you of the fact that you are in that sort of situation.

Facts and values cannot be separated - 'values' are simply assertions of fact that we don't wish to hold up to analysis and be forced to justify or discard.

What we call 'values' serve purposes, which are themselves established to serve deeper purposes and so on. We can logically examine a set of purposes and determine how well they meet the requirements of the deeper purposes beneath them; we can compare them to other sets and find whether they are better or worse at meeting the goals.

Persuasion, as opposed to browbeating or indoctrination, is only possible if all entities involved share the same deep goals, because the goals are what determine the nature of the evaluation criteria, and only by demonstrating that one strategy is inherently better at meeting those criteria than the current approach can one being be persuaded to adopt that new strategy.

Humans dislike taking a hard look at their motivations, goals, and preferences, which is why they do it so infrequently and so poorly. Bringing them into the light of consciousness temporarily impairs their functioning, and runs the risk of our having to discard them if we find rational problems.

"Ian, Peter, Vladimir, I don't think I disagree with any of you here."

But if you agree that the presence of a starving man can create a new perspective on a loaf of bread that we might fairly call "value," then this is potentially a first step to finding out what objective value might be.

Because maybe not every combination of objects can give rise to this perspective, for example substitute a spinning top for the man and the perspective is gone. And if only certain objects can create the perspective, then only certain objects can be valuable.

This wasn't clear, but my comment was two independent parts:
1. The fact is, people are irrational animals whose values can be influenced in various ways, sometime it is in my interest to influence thooose values, and sometimes conversation is a way to do that.

2. If an agent suspects it is possible that it is wrong about its true values, and it believes 'all utilities are zero', then there is still a nonzero possibility that it has a value it doesn't know about, and a positive expected utility to increasing its ability to figure out what its true values are.
So this is a value that is derivable from Amoralism plus uncertainty.

James, you presume there is already some specific and known computation such that the agent believes that if it returned an output, that output would be one of its values.

What's the basis for your assumption that it's possible to distinguish between facts and values? This is quite a controversial topic in philosophy these days.

Proposition: values are not "facts," in the sense that they are observable entities in the world, the appropriate object of probabilities, etc. But they are still ideally responsive to reasons -- that it, it's the case that an agent who is exercising the optimal cognitive operations will modify his/her values in response to appropriate reasons.

No, but if the agent believes there is a nonzero probability that it might find such a process, then it should look.

If the agent knows that it is fundamentally error-prone, then it should never have a zero probability.

"their true values"
I'm not quite sure what that's supposed to mean.

But they are still ideally responsive to reasons -- that it, it's the case that an agent who is exercising the optimal cognitive operations will modify his/her values in response to appropriate reasons.
That should have observable consequences, making the question of what "true values" are one of empirical fact.

Let's put it this way: any concept that has absolutely no basis in empirical fact at some level is contentless and has no meaning.

James: Then the agent must already know some computation such that if it returns an output, that output is a computation which, if it returns an output, that output is a value.

I can't believe nobody injected Thomas Kuhn into this discussion yet. Any set of values gives a higher weight to certain facts over others - we feel a fact matters, or doesn't. Common argument pattern: yuo state a fact that matters to you, opponent responds with "yes, but..." followed by an unrelated fact that's important to them. Political example: lefties think Hitler is the ultimate indictment of the Right, righties think Cambodia is the ultimate indictment of the Left, and neither side can ever convince the other.

hmm, If I concede that, I may as well retract my previous 'No'. But I'm going to say no again:

It only has to believe that its values might in principle be discoverable. It does not need to have a process.
It doesn't even have to know how to add.

In order to have that value-finding-process making process, the agent would have to construct it. If the process were built into the agent 'at birth', we would abstract it into its primary goal system, if it ever returned.

In order to construct such a process, or to successfully run it, the agent first has to have a goal of figuring out the right answer.

Consider the value-finding-process making process (or the process that makes it). If it searches a finite region of the possible process space (such as all process our agent could execute) and returns null or an error, then we are left where we started. If it searches the entire process space, then our agent will probably die before it returns. And this assumes that it has the cognitive ability to simply run such a process [without error].

In order to serve its possible-true-value the agent has to have the goal of finding its possible true value even if that takes more time, energy, ram, or basic knowledge than it currently has available to it. Even if it thinks it has a magic value finding process, it might need to eat while it runs.

If the agent is a colossally stupid product of evolution it might take millennia of collaboration to even get to the concept of a value-finding-process making process.

So is there anyone who thinks themselves more rational to argue about values than to argue about facts? I thought some might disagree with my main claim here.

Values are not for arguing about- they are for convincing other people that they'll be happier if they share your own--whether or not you actually think they will be. A good friend of mine became the president of the Party of the Right at Yale. She was a very philosophical-abstract sort of person, though I generally disagreed with most of what she said. As irony would have it, she was an atheist, but still touted Christian fundamentalist doctrine whenever she could as a political strategy. She justified this as a morally valid position under the grounds, "Whether or not Christianity is correct, wouldn't the world be a much better place if everyone believed in it, at least in the form I so aggressively defend?" Whether or not anyone wants to take a stab at answering her question, I thought her hypothetical unrealistically absurd to try to even try to accomplish, and thus her argument invalid. If she wanted to say 'if MORE people believed it...' that would be another question, one I would still disagree with, being the good little truth-seeker I am...

So yeah- inflicting one's values on others is generally a selfish activity, though we don't have many other options if we want to write our own commandments...

Values can be dissolved into facts. eg. "Chocolate is the best." Can become "I like chocolate the most." or "Most people prefer chocolate." or whichever objective thing the value actually represents. If you don't know what the value represents, ask why your brain produces the value in the first place. After this you only have to deal with facts.

PK- I think we can safely describe 'values' as 'general societal preferences,' though I might be wrong. The problem with preferences arises when they involve the actions of *other* people, not just yourself. It's fine to abandon the idea of values if you want to sit under a tree and meditate all day like Siddartha, but if you think that *other* people should be good to each other, then you will need to inflict your values upon them. You can tell them the facts, "Tying that child to a flagpole and dumping bleach down his pants was a very hurtful thing, which given our latest research seems like it might have damaged him for life, and generally nasty little boys like you are doing it in order to assert dominance, and truth be told, it seems to work- but society as a whole will be nicer if everyone stopped doing hurtful things like torturing the other boys in the school yard........."

Lets see how far this argument will get you.

Or, "What you did was evil and you are a very bad person for having done it, and Jesus will hate you and you will go to hell and burn in hellfire for all eternity if you don't repent and promise never to do that again. Now go to your room- you are grounded for a week, daddy will be up there shortly with the switch."

Not that I'm advocating that *we* should use the latter style of impressing values upon people. I don't think it's much different than what the Party of the Right was doing. But I am making a point that facts are not always able to resolve disputes/change behaviors...

Unless you are suggesting that the facts will add up to an objective morality... Plato's "the Good"???

Have you read Schneier's take on this or the article he sites?


Robin, humans not only seem to commonly argue about terminal values (things done for themselves and not their consequences), but also seem to occasionally update their professed beliefs about terminal values as the result of such arguments, and even, every now and then, change their actual behavior. Is this a malfunction of rationality? What is the truth that destroys it?

Eliezer, if you could have in principle anticipated that your "terminal" values and behavior would have changed in response to such arguments (and changed to something else in anticipation to other arguments), even if you did not actually bother to so anticipate, then your values seem clearly state-dependent values, where info is relevant to figuring out the values. This fits fine in the standard framework. If you could not anticipate such changes even in principle, but all your other behavior is as if you could, well I'd say you are as above except for a defect in your anticipation abilities.

If you permit "state" to refer to impossible possible worlds, then I think we're essentially in agreement about how to formalize this to the extent it can be formalized. But in what sense is this not "arguing about values"?

Eliezer, yes let's accept impossible worlds as states. It is arguing about "facts" in the sense I defined in the post above, but it may also be "arguing about values" as well.

"So is there anyone who thinks themselves more rational to argue about values than to argue about facts?"
I'm confused by this question. I thought you clarified your agreement with some comments above that arguments about values are arguments about the underlying facts or context on which those values are based.

In other words, arguments about values are arguments about facts. If so, how can they be more or less rational than arguments about facts...which they are anyway.

The semantics of values is intrinsically fact-dependent. If you are uploaded in the virtual world, where each time you eat an apple for breakfast, a child is automatically killed in the real world, and you don't know it, you will make decisions that will lead to killed children. Does it change your values? In what sense it doesn't?

I don't know if I'm having a dumb day, or if this post simply uses words with definitions that are very different than what I assume.

In any case, I cannot even work out the structure of the argument given here, never mind whether I agree with it or not. As far as I can tell, its gibberish.

This has happened in previous posts and I've usually been able to track it down to a set of underlaying assumptions that I simply don't have. That may be the case here as well, but if so its on a far grander scale than any case I've seen before.

The comments to this entry are closed.

Less Wrong (sister site)

May 2009

Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30