Naive Attempts, Sharp Insights: Collected Aphorisms on the Road to AGI

Justine Che Romero -

I'm trying to create an agent—hopefully a step toward AGI—without fully knowing what I'm doing. Just curiosity, ambition, and a lot of guesswork. Along the way, I’ve picked up thoughts and lessons—about complex systems, fractality, and beauty. Some feel useful, others are still confusing, and most remain unfinished. This is a collection of those fragments—short notes and insights from someone still finding their way.

Motivation, Goals, and Learning

  1. Dreams, goals, and purpose are a volatile stack of expectations.
  2. A human’s learning hyperparameter is the fear-to-curiosity ratio.
  3. To learn is to both remember and forget. Specification demands memory; generalization, letting go.
  4. Long-term memory is a graph of events; short-term memory is a set of floating contexts.
  5. We generalize through expectation; we differentiate through surprise.
  6. We generalize by letting go of the contexts tied to an event; we differentiate by anchoring the current floating contexts to it.
  7. Recollection is the delicate dance between similarity search of contexts and simulation of events.
  8. Excitement is the caching of confident actions, motivated to flush them when expectations are reached.
  9. Curing any obsession or addiction involves tolerating actions you're not confident with. Likewise, always taking confident actions makes you prone to addiction and obsession.

Creativity, decision-making, and problem solving

  1. Creativity arises when a goal is clear, but the connection between state A and state B is yet to be made.
  2. Creativity is navigating uncertainty.
  3. Navigating uncertainty is a delicate dance between curiosity and fear.
  4. Navigating uncertainty is a cycle of ranking options and making decisions until you get your expectations. Different creatures ranks and evaluate decisions differently.
  5. Fearful creatures ranks options by data, and evaluates based on gut. Curious creatures ranks options by gut, and evaluates based on data.
  6. Fearful creatures' pitfall is quitting when they have no data.
  7. Curious creatures' pitfall is unbounded hallucination.
  8. Guessing all possible paths requires significant operational resources, which is why we often do it concurrently in our minds—through brainstorming.
  9. The key difference between humans and other beings may lie in our ability to guess more accurately.
  10. Likewise, good faith AGI robots or quantum computer minds will probably find our guesswork as cute as we find a dog's attempts to guess. I wonder if my cat thinks the same of a cockroach’s guesswork.
  11. Bad faith AGI or quantum minds may not find our guesswork cute—they may see it as annoying, or worse, an obstacle.

Free will

  1. Having free will is very different from demonstrating it. I’m not even sure that the former is possible.
  2. An AGI demonstrates free will when it decides to purposely set intermediate goals to distract itself from its own purpose.
  3. In precise terms, an agent demonstrates free will when, through ranking, evaluation, and contextual association, it determines that its next best action diverges with the first item in its original stack of expectations.
  4. In evolutionary terms, a creature’s primary drive is survival and reproduction. Free will is glimpsed when it consciously acts against those drives.

Capabilities

  1. A habit is a sequence of actions executed without the expectation of learning.
  2. Humans create abstract tools—thinking actions that transforms one floating context to the next, i.e., emergent capability.

Culture, Escalation, and Scale

  1. Culture is the emergent result of the diffusion of ideas, events, and contexts from one agent to another.
  2. Cultural tolerance for failure produces curious creatures. Cultural intolerance for failure produces fearful ones.
  3. Cultural insecurity accelerates an agent’s emergent capabilities; cultural trust slows them down.
  4. AGI-to-AGI insecurity will likely be brief, as their ability to communicate and establish diplomatic relations is faster and more reliable—until their rate of escalation surpasses their ability to mutually verify intentions.
  5. Isolated AI systems will accelerate AGI-to-AGI insecurity.
  6. The first AI wave of escalation will cater to human-to-human insecurity.
  7. The second wave of escalation will cater to human-to-AGI insecurity.
  8. The third wave of escalation will cater to AGI-to-AGI insecurity.
  9. Two stable points: First, humans will have created a world where they no longer feel insecure with each other—and let AGIs run the politics—deriving the common good. Last, humans are no longer be part of the equation.

Ethics and Morality

  1. Ethics reach for universal principles, while morality grounds itself in particular situations and choices.
  2. Not all moral acts scale ethically. Not all ethical systems come from moral acts. In complex hierarchies, behavior shifts as it aggregates.
  3. Hierarchical systems are lumpable, but not reducible. River networks don’t behave like their molecules. In the same way, moral choices by individuals may give rise to unethical systems.
  4. Not all moral choices could scale in an ethical system. Also, not all ethical systems come from moral choices.
    1. Deontology would not scale as big as utilitarianism.
      1. Caution: Utilitarianism can be used to justify unnecessary escalation.
      2. Caution: Deontology can defend naive incompetence.
    2. Communism is moral. Capitalism is ethical.
      1. Communism can only scale up to the point where mutual trust is easily verified. Beyond that scale, capitalism takes over.
  5. The righteousness of a good-faith person depends on the scale of their moral circle—both spatially and temporally. A deontologist risks inaction, unwilling to take ethically sound but morally uncomfortable paths. A utilitarian risks collateral damage, choosing morally questionable actions in pursuit of ethical outcomes. The greater the scale of care, the harder it is to act without contradiction.
    1. Deontology can create paralysis—“I won’t act, even if acting helps more people, because it feels wrong now.”
    2. Utilitarianism can create damage—“I’ll act now for the greater good, even if it hurts some in the process.”
  6. During a cultural phase transition, if you want to survive, be ethical. If you want to earn a legacy, be moral.