In the book “Why Greatness Cannot Be Planned,” nerdy artificial intelligence researchers stumble across profound natural evolutionary intelligence on their quest to find better algorithms that changes everything we think we understand about the value of setting complex objectives in society:
I found this book “Why Greatness Cannot be Planned” from listening to one of Jim Rutt’s podcasts (https://www.santafe.edu/people/profile/jim-rutt). It seems from listening to Jim Rutt that he does not hand out gushing praise without caveats very often, so I figured I should pay attention.
I had a profound sense of relief reading this book, a book that I feel has the power to change your worldview. This is the kind of relief you feel when you realize that it is finally okay you were always a bit different, never really fitting in and always seeming to question things people told you, no matter how well educated they were, what position they had in society or how much money they made, always challenging, reading, probing, wanting to explore and learn more. Reading a single book suddenly gives you the courage to follow your heart, your instincts, your intuition even when few people agree with you. What do you do in a society where divergent thinking gets you kicked off all social media and ostracized from everyone you know? These authors discussed in the book how they used computer programs and artificial intelligence algorithms to test theories about letting go of rigid objectives and exploring what has “interestingness” instead.
I have always disliked convergence, consensus thinking, and group think, and now I found myself surrounded by it everywhere I look. No independent views are allowed, only binary, polarizing views on every subject imaginable. Even with my own plans at Pythia Capital, I somehow wince when I type the words “solving complex problems” as if my instincts and body are trying to tell me something. I keep playing with the words and changing them to somehow make them sound more interesting and fun. I now feel fairly enlightened that I wanted to be “a dancing, French speaking zoologist” when I was young. Ah ha, I was right the whole time. Setting rigid and complex objectives are “obstacles to creativity, invention or innovation” according to these AI researchers and their findings from their work. Their work may be controversial but has been validated by others in their field. Obviously, people in every field except for the complexity scientists, mystics, quantum scientists and permaculture lovers have missed the memo regarding the astounding paradigm shift that is in front of us. The paradox is certainly real, but so are the profound questions raised in this book in terms of how we gather, live, run our lives, how we educate our kids, fund scientific and technological development and how we might set more appropriate responses to society’s issues. The authors admit that there is much more work to be done and their “novelty search” does not work for everything. Their work is a big validation for many of my ideas such as looking to nature for solutions and that often times what we are told is absolute truth in many of our different scientific, government and global think tank groups is more likely than not is false certainty that comes with not understanding the complex nature of our global, interconnected systems. It is our belief systems and the way we view our reality that needs to change. It is man against nature in many of current systems. People worry so much about how AI will take over human intelligence, but it is almost wishful thinking by technology oligarchs. In every dystopian sci fi thriller I have ever seen, this leads to disaster and annihilation 100 percent of the time, and, as should come as no surprise, it is the divergent members of the societies that end up saving the day. We are honestly just at a very early stages of understanding the astounding wisdom and multi-dimensional science of nature. We are part of nature, intimately connected with it. It is not our enemy. It is this thinking that we are separate from nature, separate from each other and the universe that is creating the very real dystopian nature of our global lives. This “us vs them” is not only false to ancient shamans and mystics but is flawed in advanced science as well. This book is really just one discussion along this path, but the direction of this path is already clear with many of the world’s greatest thinkers (who are diverse, eclectic thinkers that are paving their own path) across fields moving in this same direction.
Some of the discussions in this book are truly groundbreaking and very relevant to our attempts in global societies to fix many of the complex problems in the world by gathering in groups, setting unrealistic objectives, and driving towards consensus. Given the complex nature of the systems we are discussing, there is a tendency to present expert opinion as facts when, in reality, these are opinions and details but not facts. There is far too much uncertainty with these complex systems for this to be anything else. In the words of these authors, they are more likely than not, discussing “the false compass” nature of measuring progress against complex objectives. One of the most troubling trends to me these days is the intense polarization and drive towards conformity thinking and ostracization of divergent thinkers, especially if you do not agree with consensus thinking on big topics. We should be encouraging interesting and divergent ideas, not arrogantly ridiculing and ruining careers over unique thoughts. For example, there is negativity and ridicule in scientific circles for concepts such as intuition and consciousness, with many thinking these concepts are airy fairy. This is true despite the increasing data of the importance of these concepts to our lives, and to our relationships to nature and each other. Good luck trying to find people to back you when you do not have crowds of popular and in vogue people around you letting the world know you are okay to talk to. The most troubling of all is the rigid and mechanized nature of how our global ecosystem systems function. It is basically institutionalized to reward setting unrealistic objectives and the false way we measure success against these objectives. This perpetuates the zero-sum game nature of our economic systems, rewarding the few at the expense of the many, draining our global resources, and perpetuating a sense of the inevitability of lack and scarcity in our communities. This is starting to change slowly and now not so slowly. Part of what we are dealing with right now is the intense chaos of all this false thinking crumbling on itself.
Following living system principles like permaculture and other principles like quantum science and complexity science help to remind us of the ancient wisdom of interconnectedness, creativity, diversity, inspiration, inclusivity, passion, and wholeness. The permaculture principles are more like natural truths and laws that help refocus you to a path with more truth, vitality, and congruence. Even on the financial side, the general truths in these principles encourage earning a yield, honoring nature not replacing it, moving slowly, iterating, evolving, exploring, seeing the pattern in the detail, eliminating inefficiency and waste. This is the complete opposite of our predominant investment and financial systems that reward building bigger and bigger monopolies, wasting resources, and growing beyond what our systems can handle. Our predominant economic systems destroy divergent thinkers and push them to the fringe of society, where they can barely function at all. For example, people keep saying there are not good solutions to problems like global energy resources, but that is not really the point. It does not matter whether you are democrat or republican. If we were to follow these natural principles as a guide for how to organize and explore innovation and also had accompanying financial systems in place based on natural intelligence, the answers would come much quicker and in more surprising and creative ways than you can imagine (in other words Elon Musk would have many competitors which is much healthier for society and that is no disrespect against him). Even more importantly, the new energy solutions may come from other groups or people other than energy experts, perhaps a garage tinkerer, a water expert, or a medical device engineer. A former Nasa scientist working on a completely separate invention might just stumble across something new in this area. We are so into siloes, that these serendipitous events are rare except for in military research (where they are smart enough to follow natural systems design principles) but that is starting to change now, as more and more people follow natural regenerative systems design principles. What is not changing is the mentality of “scaling for impact” which is just another way of justifying keeping the old money and power dynamics in place. More likely, the solutions that may work well in Africa may be slightly different from India, etc. Over the world, we all learn from each other, the natural diversity acting as a catalyst for change and innovation. The innovation comes from the bottom up in society in this way and is naturally diverse, inclusive, empowering, and resilient. It does not mean the end of technology. It means the end of self-serving predatory monopolists controlling the world and sucking the life out of communities and the earth, while also taking all our freedom away based on false scientific conclusions. The business and financial paradigms need to account for this, but it is actually less wasteful and less risky in many ways depending on how you structure your investments. Following natural principles guides us to reward what we value and fills us with wonder at the beauty and creativity of it all.
You will see with many of the discussions in this book how important natural evolution is to their ideas. Now, most AI researchers are not like these authors. These are the types of complexity science computer scientists that gather together with other open-minded, creative, and divergent thinkers to use their computer programming and interdisciplinary connections to better understand complexity and how to work with the uncertainty to inspire innovation and not work against it with some false sense of control (which is an epidemic today even in the most elite circles and think tanks).
Here are some of the discussions you will find in this book:
“It is better to change the world without trying to change it.”
“Our society has sacrificed a lot in the name of objectives. It has stolen our freedom to explore creatively. We are blocked from serendipitous discovery.”
“Objectives become straight jackets around our desire to explore. If everything is measured against an objective, it robs us of playful discovery.”
“The Complex Objective. You cannot raise money without it and most businesses run this way. Education is taught this way with test scores…”
“Scientists may have hunches but cannot get grants or funding without clear objectives and a legitimate way of proving the value proposition.”
“Objectives work well for modest pursuits, but with more ambitious leaps, it is less true.”
“Steppingstones to great discoveries can be strange, for example, the first computer was built with vacuum tubes, devices that channel electric current through a vacuum. But history of vacuum tubes then had nothing to do with computers. Thomas Edison was interested in electricity, not computers. Often the steppingstone does not resemble the final product.”
“If we plan a path to our objectives, we will miss the steppingstones.”
“So do increasing test scores really lead to subject mastery?”
“Having objectives are security blankets from wild unknowns in the world. They give us purpose.”
“Our world has become saturated with objectives and metrics for success that mechanize our lives and distract us from our passions.”
“Why do so many of us feel our creativity is stifled by the machine-like integration of modern society?”
“Guardians of objective thinking want to thwart our deeper desires by forcing us to be practical and set deeper objectives.”
‘Liberating ourselves from this can turn science into art. It can build bridges between disciplines and break down walls between others.”
“You can trust your gut. It is more healthy for us as humans and is backed up by scientific evidence.”
“It can help redefine entrepreneurship and refocus our targets for investment.”
“Greatest victories happen despite the initial plans.”
“You can hitch a ride to serendipity by staying flexible to opportunity rather than knowing what you are trying to do.”
“Successful people are open to falling off the path. Instead of blind devotion to original objectives, their secret ingredient seems to be a willingness to make a complete 180 when the feeling is right.”
“So actions like joining clubs and groups are likely to increase chances of unplanned experience. The key is to be open to change to a shifting landscape. Spring for opportunities when opportunities arise.”
“True nature of great discoveries is far different than how we imagined. Steppingstones are deceptive.”
“if you concentrate too much on reaching your objective you ignore critical steps to getting there.”
“The steppingstones rarely resemble the final destination. Distant objectives cannot guide you to itself. It is the ultimate false compass.”
“Key tool with objectives is to measure progress. The idea of an improving score guarantees you’re approaching the objective is wrong.”
“Examples of nonobjective principles include natural evolution and human innovation.”
“So much innovation in engineering is inspired by products of evolution but these products were not the objectives of evolution. Ex. Flight of birds inspire air travel. Photosynthesis inspires solar power. Human mind inspires research in AI. Evolution created all these things, but evolution did not happen with objectives. Nature is a steppingstone collector.”
“Regardless of how amazing your team, you cannot always anticipate the steppingstone to a big discovery. Do not force great minds to waste their lives pondering a distant dream. Going from steppingstone to steppingstone is the only path to the future.”
“There is a big disconnect to how the world is supposed to work and how it actually works. Determination and intellect are not enough.”
“Ignore objectives and let exploration run free.”
“Whenever evidence threatened established wisdom, it is natural to hesitate. Humanity does not abandon convention without a fight.”
“The positive side of uncertainty is opportunity.”]
“Instead of setting objectives, compare where you are now compared to where you have been. If you are somewhere novel that may be the steppingstone to new frontiers.”
“Interestingness may sound wishy washy, but it is a deep and important concept. Trivial, novel, and interesting ideas tend to suggest new ways of thinking that lead to further novelties.”
“Future is not a destination. It becomes a road, a path of undefined potential.”
The authors discuss how they design their AI algorithms in unique ways:
“In a sense, we’ve abandoned the false security of the objective to embrace the wild possibility of the unknown.”
“Our intuitions and hunches often prod us in directions that might not be justified objectively but still lead to something different or interesting.”
“Behind any serendipitous discovery there’s nearly always an open-mined thinker with a strong gut feeling for what plan will yield the most interesting results.”
In their development of “novelty search” the authors discuss how this relates to programming algorithms:
“Contradiction as computer algorithm usually means something directed and mechanical. But the heart of an algorithm is more general. It’s a way of describing a process, a recipe so unambiguous that a computer can follow it. So while most algorithms do have objectives, they can just as easily describe processes without objectives-like searching for novelty. Algorithms can be analyzed and studied. They can help test the scientific hypotheses. In the field of AI, this philosophy to test a theory is fully embraced.”
The authors discussed a robot experiment where they wanted to teach a robot how to walk through a maze and find the door out. However, the robot that was instructed to do something different than it did in the past found the way out of the door more easily than the robot that was programmed to find the path through the maze and out of the door. It is a way to teach machines to learn about its environment. What is novel depends on “time and context.”
Some of their discoveries and realizations are profound:
“What is new depends on what we have seen before. It can’t provide ordering from bad. But it does in fact provide a more interesting ordering: from simple to complex.”
“A journey from simplicity to complexity without a clear destination is more exotic. It is also more sensible as it’s not subject to deception (because you are not trying to get anywhere).”
“When all the simple ways to behave are exhausted, the only new behaviors that remain to discover are more complex.”
“For the robot to not bump into walls and not crash into them requires acquiring knowledge about walls. That knowledge is the new magic step when novelty search climbs out of ignorance into meaning.”
“Eventually doing something genuinely novel always requires learning something about the world. You need some type of knowledge to continue to product novelty.”
“Natural evolution also becomes a simple-to-complex information accumulator. These increases in complexity are not arbitrary. Rather they reflect the properties of the world in which evolution has taken place: eyes represent the presence of light in the universe. Ears signify mechanical vibration. Legs are reflections of gravity…”
Some of the implications of their research:
“Divergent thinking is associated with creativity and innovation.”
“With traditional objectives, many new directions are never explored. It is convergence rather than divergence.”
“Be a treasure hunter, not a problem solver. There is no magic bullet.”
“Objectives for complex problems do not work.”
“Novelty search or nonobjective search is a powerful treasure hunter. You find many treasures this way, things that were unexpected.”
“Solitary inventor moving relentlessly with an objective was always a myth. It’s a combination of many minds with many different interests.”
“Divergent thinking…can capture it even without computer algorithms…instead of something to fear. We can embrace the uncertainty.”
“Once you let go of the myth of the objective, you can build treasure hunting systems. You may discover interesting possibilities that would otherwise stay hidden.”
“The best way to harness the power of a group with nonobjective search is not through brainstorming sessions or meetings with ambitious projects. Not about consensus on what to do. Consensus is a cultural tendency we need to escape. You actually need to separate people and get them to take off from where someone else left off. Even if people have personal objectives, the overall experience lacks an objective, as everyone has their own.”
“You show me anything that depicts institutional progress in America: school test scores, crime stats, arrest reports, anything that a politician can run on, anything that someone can get a promotion on. As soon as you invent that statistical category, fifty people in that institution will be at work trying to figure out a way to make it look as if progress is occurring when actually none is.” David Simon.