Technology will (in our lifetime) exceed human intelligence (reaching an equivalency where humans are to amoeba). Given the power and speed at which technology is progressing, the answers to key questions (Who will create these morality algorithms ? How will we ensure they are not compromised ?) may well determine our fate. I see this less of a generational issue (it's up to the millennials or next generation) and more of one that will challenge our (collective) ability to overcome the limitations in human evolution.
Establishing Programmatic Guidelines for Morality
Think about programmatic guidelines and rules for defining morality as a way to guide the behaviors of machine-based intelligence. Initially, these algorithms will (likely) be defined by humans and (later, not much later) these algorithms will be improved & extended by AI and machine learning. AI is itself an evolving set of capabilities that are being used to build and extend intelligence (initially to match humans, eventually to surpass them). Behaviors are typically situational with common patterns that align to a generally accepted norm that we refer to as "good behavior". There are challenges in aligning to a normative guideline since these can be very different in different contexts and cultures. We are humans after all.
Behaviors Are Influenced by Perspectives & Beliefs
I'm seeing an increasing dialogue around "what makes us human" and the need to preserve our notion of consciousness. In theory, we rely on our conscience (inner voice if you will) to guide and shape our perspectives, which influence / shape our behaviors. The problem of conscience is that the word itself is something of a paradox. The science around understanding conscience is nascent at best and we often see leaders and researchers attempt to tease this out of physical reality to help explain the "virtual" reality (experiences and perspectives that guide our inner voice).
In fact, everything that we observe through our senses form patterns that we interpret through filters to form our unique perspective(s) of the world and experiences around us. These filters influence our understanding and impact and shape our perspectives, which provide the basis of what we call consciousness. Unfortunately, humans are particularly susceptible to manipulation of our filters as is evident from all the fake news, fake science, and now deep fakes (pictures and video).
I think of Morality & Ethics as a set of rules and guidelines that humans have codified through various cultures over time. These serve as a "compass" to influence our thinking and guide our decisions & actions (a.k.a. behaviors). Everyone has their own compass and it is tuned to their experiences and beliefs.
So too, the programming of morality can be developed using a similar approach (starting with a codified set of rules) that can be shaped over time using the patterns gleaned from (millions of experiences and shared beliefs). The challenge is which individuals should the machine learning be based upon ? (i.e. how do you choose "good people" from others ?) Further, how will the machine learning be able to distinguish between bias in situations where human behaviors deviate from a baseline of presumably good behavioral patterns ? Each situation is unique, so the rules and guidelines in the “morality algorithm” will have to balance behaviors (decisions and actions) against the impact (good or bad) within a given context. Humans have attempted to do this through laws, regulations, and policies in many cultures and civilizations. Maintaining and applying these laws, regulations, and policies is already falling behind when applying these to humans. Machine based learning and AI is the only solution for assuring behavioral control as machine-driven behavior and the collective intelligence continues growing at an exponential rate.
There will be many challenges and hurdles to overcome as these algorithms evolve. The philosophical concerns will play out in equations that balance the needs of the few with those of the many (all too often simplified in movies). These algorithms will incorporate a set of universal patterns (abstracted from situations, context, and cultures). These will likely form the "nucleus" that will guide and/or influence the situational patterns that are relevant to humans who place a premium on their geographic, cultural, and "other" beliefs. This nucleus will need to keep these nuanced patterns in check in ways that may create some discomfort with existing laws, regulations, policies, customs, institutional norms (including religions). A separate set of processes will need to be established to navigate & negotiate the anticipated concerns. However, even these concerns will need to be adjudicated using AI & ML to keep pace with the exponential growth and influence of "intelligence".