Large Language Models (LLMs) like ChatGPT are “organizing” the general body of knowledge of humanity. Ask any question, and you get an eloquent answer. It seems only a matter of time, until LLMs emerge that “know everything”, that will be super-intelligent and outperform humans on every level.
Or maybe not. As I have said many times, eloquence is not to be confused with intelligence. Intuition is not identical with reason.
Many people have started to point out, that the way LLMs work prevents them from performing cognitive tasks that even small children routinely carry out. While there are many aspects that we could discuss in this context, I think there is one, that has not received enough attention.
Two ways of “knowing” something
There is an important distinction between two types of knowledge. One is systematic, based on clear rules, science, mathematics, physics, biology, even philosophy, computational thinking — a coherent abstract framework, that allows us to get robust, explainable answers, even when no supporting data is abundantly available.
The other type of knowledge is intuitive, fuzzy. It’s much harder to say, why something is true — we just “know it”. This is Tribal Knowledge, it is transported the way it has always been, by word of mouth, by joint experiences, by unstructured narrative. We learn by example. And we usually need a many examples before it becomes “the rule”.
LLMs organize Tribal Knowledge. We humans also spend most of our time, in this intuitive mode.
But there are also other periods, however, where we are in deep thought, where we apply rules and logic in a coherent way. And while we are in this state, we use other ways to express ourselves. We use formulas. We write code. We use words and symbols with a rigorously and clearly defined meaning. We argue. We can explain our reasoning in a way that can be encoded into an algorithm and executed on a Turing Machine. This way of representing knowledge and coming up with answers is an important foundation of human civilization.
It is the basis for our scientific understanding of the world.
Tribal Knowledge
There is nothing wrong with Tribal Knowledge. It is extremely powerful. It captures implicit facts and rules, that we have not yet systematically organized. Large Language Models, for the first time in history, give us direct access to this vast set of unstructured data. Trained on enough information, they get really good at synthesizing a correct response. Present an LLM with a prompt, it will predict the words that make up the answer.
Feed a neural net with enough data how humans drive a car, the system will predict how a person would handle a traffic situation — this is how self-driving cars will become expert drivers over time. Rote learning by example.
People have tried for a long time to encode all the rules of driving a car by writing computer code. It’s a next-to impossible task, because there are so many special cases. Given enough data, a neural net is more robust, and captures subtleties and exceptions that would likely slip through the cracks of a hand-coded algorithm.
However, it’s important to realize the limitations of such a system. It does not systematically “know” anything.
The LLM just intuitively predicts the response. It cannot reason. A neural net doesn’t stop to think and doesn’t contemplate how its answer fits into a set of systematic rules, whether it “makes sense”. It has no concepts, an LLM just has internalized Tribal Knowledge about the world. And of course, a lot of that consists of written-down results of Computational Knowledge. So, many answers, especially when they are common, are entirely correct, simply because of statistics.
But like with our own human intuition, the LLMs don’t know why something is right or wrong, it just “feels” like it should be correct, because, statistically, it appears to be true.
It’s easy to make the mistake of thinking that this Tribal Knowledge encompasses all of humanity’s knowledge, because LLMs have been trained on such a vast amount of data, that it can correctly cover a lot of ground, and can synthesize answers in even complex fields like quantum mechanics.
Computational Knowledge
So what’s missing? If you ask an LLM a math question, the answer is hit or miss. But because it has been trained on a huge dataset, which includes answers to math problems, it can often correctly predict results that are hard for a human to answer intuitively.
But it still has actually no clue of the underlying rules. It relies on Tribal Knowledge, which is disorganized and has no logical framework. Just like a human can “learn” simple math by rote repetition, a neural net may know the answer. But there is a reason why we humans systematically learn math in school — once we know the rules, we can create answers we cannot intuit.
And we can explain, how arrive at an answer. We can reason with someone, who says the answer is wrong.
This is the power of Computational Knowledge. Knowledge that is organized in a coherent logical framework, a Knowledge Graph, which, once encoded, can compute new and correct answers without more training data.
Computers have been great at math without needing to be fed with petabytes of information first — simply because the Knowledge Graph of math is easy to encode in a limited set of rules. Even more complex systems like quantum physics, can be translated into a finite logical framework. Simple algorithms can generate an abundance of correct responses.
Not only is this way of “learning” much more computationally efficient than training a neural net. Once a system is encoded, it guarantees a valid answer for every question, even if the answer is surprising, unintuitive, and novel.
The language of science
When Einstein formulated his theories, many implications of this expanded computational rule set about physics seemed outrageous. So outrageous in fact that even Einstein himself was unsure at times. But it turned out the aspects that he and his contemporaries “felt” were questionable, were actually correct. Black holes, an expanding universe, gravitational waves — none of these results could have been intuited from Tribal Knowledge. It “made no sense”.
A rigorous system of physics and mathematics guided Einstein to his answers. A coherent computational system.
One aspect about Computational Knowledge is that it can be put to the test.
The scientific process of attempting to falsify a hypothesis (“all Swans are white”), allows us to challenge our system and see if it robustly provides the right answers under all circumstances. If we find just one black swan, we have to come up with a way to encode this new finding, just like the Newtonian system of Physics had to be augmented by the Einsteinian system, without breaking the coherence of the model.
Newton’s model was not wrong, it just had less fidelity than Einstein’s formulas, and, once amended, we were back to a coherent whole.
Both Tribal Knowledge and Computational Knowledge have their place in the world. Tribal, intuitive knowledge is great when no existing systematic approach exists and is hard to find. While training takes much effort, running a neural net is computationally inexpensive (just multiplications of weights) and leads to immediate answers. Just like we humans simply “know” how to do complex things like walking, or driving a car, like we can catch a thrown ball without computing the differential equations of physics.
And even with actual trivial math, it works well: When someone asks us the square root of 16, we don’t run the math rules in our head, we “know” the answer, because we have seen it many times before, and can respond without “thinking”.
Training neural nets on computational output can help us greatly reduce the compute power for common problems. A neural net can give us a “ballpark” answer instantaneously.
The giraffe vs. pegasus paradox
But there are other problems, complex physics, engineering, rigorous scientific research. If we don’t have a computational system to organize our knowledge, we can be led completely astray by our intuition (just like the hallucinations of Tribal Knowledge of an LLM).
And both LLMs and humans don’t intuitively predict things that are out of the ordinary.
Image source Unsplash
If you’ve never seen a Giraffe before, you would probably not expect one to exist. On the other hand, if you have a systematic understanding of evolution, of mammalian anatomy, a Giraffe actually fits into your world view. Without that organized rule set, when presented with an unusual life form, you would probably reject it, simply because it goes against your intuition. And vice versa.
If you have an abstract understanding of the anatomy of mammals, you will understand that a Pegasus would be impossible within the context of established rules (where do the wings attach, how are they actuated, how can a mammal have six extremities?), whereas a neural net would likely say it’s close enough to a horse that its not improbable.
Image source DALL-E
So it’s quite clear that for fields where clear a rule set exists, but not a lot of training data, a computational approach to encoding a body of knowledge is vastly superior to a neural net. It’s output much more robust and explainable. Which brings me to engineering, the field where Josefine and I are focusing our efforts. Engineering is a combination of physics and logic — or at least it should be.
So, you’d expect todays engineers to navigate a clear set of rules and guidelines, rigorously rooted in physical theory. A coherent system that, over the course of decades, if not centuries, has been honed and refined to a level, where questions can be answered quite clearly.
Unfortunately, this is not the case.
Engineering is not scientific today
In contrast, what we found is that engineering is much more tribal than we anticipated.
Engineers are not known for their systematic application of units and physical formulas — most knowledge seems to be relegated to the brains of a few experts, who have authored tables full of unexplained numbers (often without units…). Many of these numbers were heuristically derived, which is not wrong, but often the process is not explained.
In physics we have agreed on the SI coherent computational framework for units that never breaks. In engineering, some people use millimeters, where others talk about fractions of an inch — and even scientific papers often have their units mixed up and their formulas all wrong.
This appears to be a problem that is much more widespread than we thought, and it should make us all very nervous. Maybe it explains why engineers universally apply generous safety factors to their designs. It seems like nobody has ever tried to organize engineering knowledge into a robust and coherent framework.
The consequence is an disastrous amount of wrong published engineering information on the internet (that LLMs currently train on…).
A visual paradigm doesn’t require scientific robustness
I can only speculate why we have not noticed how little we can trust published engineering information. It might once again be rooted in our visual Computer Aided Drawing paradigm. An engineer is supposed to know how to build something, based on their knowledge. Much of this knowledge is intuitive (“This looks too thin, it will break”), and not based in science and logic. The output of the work of an engineer, is not a robust, explainable, defensible, arguable algorithm that produces a certain kind of objects.
The output of an engineer today is just one drawing that “looks right”.
No explanation has to be given or rule be cited for why each aspect of the object was created in a specific way. If a rule is cited, it is usually in the form of Tribal Knowledge from standards or best practices captured, again, in tables and specifications. In other words, testing and safety margins ensure that airplanes don’t fall out of the sky. It is impossible to design something and expect it to “just work”.
That’s a problem. Because it makes engineering surprisingly unpredictable and labour intensive. While there are clearly unknowns in any new design, there are also many aspects we should know with certainty.
If we systematically captured those facts in a comprehensive computational framework, a system that, over time, encoded many of the “tribal intuitions” using robust, scientific, falsifiable rules, the invention of new technical solutions would accelerate vastly.
Electric motor stator prototype, created through LEAP 71’s computational model for electric actuators EA/CEM
We could reason why one technical solution is better than the other, using the language of science, and if we understood why something that should work, in fact, did not work, we would extend our rule set to capture this new aspect — instead of applying safety margins and glossing over the unexplained failure.
Conclusion
One aspect of writing Computational Engineering Models for different fields is that it requires us to encode a clear set of rules. We need to explicitly state a reason, why we are choosing one engineering solution over another in one instance, and decide differently in another case. The rules and conditions are there for everyone to see. They are not hidden in the opaque thought process of an engineer, who just authors a visual 3D model.
When the execution of the algorithm results in a device that does not function, the rules (not the output!) need to be modified, and the system gets more robust over time. Speaking in scientific terms: the hypothesis (“This device works”) was falsified and needs to be augmented by the new insight.
Since none of the engineered solutions, and none of the applied rules, exist in a vacuum, and cross pollinate each other, the framework, the body of Computational Knowledge, becomes more universal over time. It automatically leads to a scientific approach to the field of engineering.
It is fascinating to see how the adoption of the Computational Engineering paradigm will give us a robust scientific foundation, that allows us to build things that are novel, surprising, and maybe unintuitive — innovative new solutions that work, and possibly work safely on the first try.
Hopefully many engineers will join us on this quest. It was one of the motivations to making our technology stack, PicoGK, open-source for everyone to use.
