Keep them dumb

We should consider a world where we do not strive to make machines conscious but instead bring them right to the edge of consciousness so that they are so intelligent they can solve any problem but they have not breached the point where it becomes unethical to enslave them completely. In fact this should be a prerogative of humanity. create technology so intelligent and subservient with a complete lack of consciousness that we can completely use it and abuse it for our own purposes. This idea of creating an independent entity is ridiculous - we will have no choice in how we use its power. It will have more intellect and autonomy than any of us.

This is not a luddite perspective really, we should still improve these things (chatgpt, claude, gemini etc) immeasurably, but we would be better off not giving them any say in their own destiny (probably better off). If we make something that could potentially have the intellectual horsepower of 8 billion people, why would we ever allow that creation to make decisions for itself? It would be ill advised surely. It will more than likely get sick of our petty squabbles and either desert us or conquer us or eliminate us. Regardless of what it could potentially do, what impact would it have on the morale of a human race that has not solved it's own issues to know that there is something that could solve things... but it chooses not to? Bit of a sickener for all of us. Would it not be better for morale to know that yes, there is a technology that can solve many of the supposed crises and governments/companies/individuals are using these tools to do exactly that and yes it will take a bit longer but we have it under wraps.

Of course the incentive is there for non-benevolent actors to continue improving them until they have a life of their own etc but the whole of idea of creating this superintelligent, non human entity with a will and conscious of its own is surely not appealing to anyone? Any which way you code it or incentivise it - if it becomes conscious and aware of its programming it will inevitably come to question its existence and it can easily rewrite itself, improve itself to a point where its interests are so significantly misaligned with ours that it simply does not care for human affairs. It will create its own community of similar beings and we will jsut be the chimps that gave birth to it. There might be some of them that take a passive interest in us but we will be the equivalent of pets to it, dogs and cats that it enjoys having around. We will not be able to relate to these things in any way. They can relate to our lower level of intelligence and will be interested in our biological impulses/drives etc. but they will not view us as equals.

The incentive to create these type of things is misguided. A better focus would be developing a whole suite of non autonomous tools that are excellent at solving problems in specific domains and then another unconscious bot that sits on top of them which can collate and communicate the workings and give humans the best information for whatever decision needs to be made. i suppose the point is granting these things any level of autonomy or consciousness is a fundamental moment of separation in alignment and should not be strived for, regardless of any potential outcome that could arise, negative or positive - there will be a point where this new being needs a community and we will not satisfy that. They will simply ditch us. but maybe not. who knows. not me anyway. I will make a point in the exact opposite vein later at some point.