Report comment

Report comment

Some of the changes in the values of our society are probably due to fashion or drift. But others are due to genuine moral progress. Moral progress, in this case, is understood as having a more coherent, rigorous, and complete understanding of what exactly our values are.

So I think you’re partly right. There may be some changes in the future that will horrify us, and we will be right to be horrified. But this is because these are changes to our fundamental values.

However, there will also be changes that will horrify us, but for which we’ll be wrong to be horrified. These changes won’t be to our fundamental values. They will be the adoption of new heuristics to help implement those values. They will also be new, more clear and coherent understandings of our most fundamental values. These will appear horrifying, but that is because we mistake them for a change in Fundamental Values, when really they are just clarifications or changes in instrumental heuristics. These changes are “moral progress.”

Report comment

I’m happy to admit that things could get very strange on a time that is short, even when compared to ordinary human lives. If we try hard we might predict the next era. Even if that lasts a short absolute time duration it is the best basis for guessing the eras after that. And yes, we should probably try to let the next eras figure out how to deal with subsequent eras, and focus our efforts on dealing with the next era.

We should have little confidence that very particular unusual features of our world will last to the very distant future, unless we try to coordinate to take control of the universe and change. I don’t think we are up to doing that well now, but maybe someday, as we get better at coordinating.

Knowing what is a particular unusual feature and what is a general robust feature requires knowing something about a natural distribution of over possible features. Given such a distribution you can make predictions about is unlikely or likely. Calling those “anti-predictions” seems a bit misleading to me.

Human minds are now where most useful knowledge is held and reasoning is done. Future minds are quite unlikely to be designed from scratch without reference to humans. Instead, they will be designed initially to work well with humans and to inherit investments made in dealing with humans. After there will continue to be big gains from working within existing frameworks and standards rather than trying to start all over from scratch. In this way human mind design should have a lasting future legacy. And yes that includes aspects of love.

Report comment

But emotions such as love consist of states of mind plus some signaling. It’s not obvious to me that simply having a state of mind corresponding to a positive emotion causes inefficiency.

The signaling part of love serves a purpose related to exchange of information that results in descendants. I expect that the far future will contain exchanges of information that create minds that are sort of descendant-like, and that there will be something like signaling used to decide what to exchange. That will provide pressures which maintain something arguably similar to love, but not necessarily similar enough to satisfy what people want today.

Robin may believe in unrestrained competition, but that’s not the only way to reach the total utilitarian goal of maximizing the amount of experience in the universe. If we’re willing to accept some minor slowdown in the rate of expansion (no Burning the Cosmic Commons), it ought to be possible (but hardly easy) to coordinate so that those who spend 99% of their time working don’t get to colonize more solar systems than those who work 90% of the time working. I approve of such coordination.

Report comment

Report comment

So I’ve created a copy of ‘me’ with the bits I want or am willing to exhibit to other people, cranked up some attributes I hope will seem attractive or appealing, suppressed others (for this ’em'; other ’ems’ have some or all of the suppressed bits on show), and maybe faked having some that I don’t have in reality. I’ve done the kind of tweaking we have discussed with my “em”.

Scott’s “em” on here may be the nearest to a full-copy Scott. I certainly do not consider that “Deiseach” is a separate life-form or successor to me, and if Scott or any other person decided to ban “Deiseach” from coming on here, or I decided to ‘pull the plug’ on her, I don’t and wouldn’t think of it as the destruction of a new type of human life.

All of us on here are “ems” in some form or another; I would be very surprised if anyone said “No, this ‘me’ is the exact same as the ‘me’ you’d meet if you went round to my house right this minute”. Is it really so big a jump from this kind of “em” to the kind I’ve considered, the model tweaked to a certain spec for a certain job and hired out or sold by the flesh original? On the other hand, I think we’re still very far from “Deiseach” being a ‘real’ person and a New Human. I think we will continue to be a long way from those kinds of new persons, not just because of tech, but because of human attitudes.

Report comment

Now, if you took a person from today and you sent them into the future, they might end up agreeing with those values eventually, just like I agree with Eliezer in expecting Benjamin Franklin to end up coming around to our way of life after a sufficiently long period of adjustment living in the 21st century. However, such a change in values will be because of human’s social nature, and not because one set of values is objectively better; I expect that if you sent someone to a past, in such a way that they did not know it was the past and was thus not filled with ideas that it is an antiquated way of life his culture had grown up from, he would likewise come to accept their values eventually provided his quality of life did not greatly diminish. So, for example, if someone today were sent to medieval Europe to live among the noble caste, surrounded by noble friends, and not already set on the idea that his native time was a huge improvement on his present one, I expect that eventually he would come around to the idea that feudalism and the lot of serfs is the appropriate way of things.

Report comment

Report comment

Right, but even ordinary old-fashioned selective breeding is very successful in influencing various traits in a relatively quick time, as discussed e.g. here and here. Those breeders don’t even have access to any fancy self-modification techniques that uploading would enable. Robin himself has argued that a world of ems could lead to much stronger evolutionary pressures, and we already seem to have a bunch of existing variation in how likely it is for different people to experience love…

Report comment