Hi! I'm Tim Tyler - and this is a video relating to the issue of which
problems are most likely to drive the development of superintelligent
Firstly, an introduction to the concept of a "driving problem".
Driving problems are problems that are used to foster the development
of a desirable innovation.
The driving problem is not necessarily the same as the main application
which the final invention is used for.
To illustrate with some examples:
nuclear power was produced as a consequence of the development
effort that resulted in the nuclear bomb;
hydropower was developed by the ancient Greeks as a means of
grinding grains into flour in order to produce bread;
hot air balloons were originally developed for military
In some cases, an innovation only has one application - in which case
the driving problem is obvious.
Also, not every innovation has associated driving problems.
Some things are discovered by accident. For example, the
discoveries of penicillin, crisps, fireworks, saccharin, LSD and LEDs
were largely the result of chance.
The driving problem for superintelligence
This leads us to the question of what will be the driving problem
responsible for the development of superintelligent machines.
Here we will consider three clues associated with the origin of
Follow the money
The first clue relating to the issue involves the common idea of
"following the money".
The development of superintelligent machines will probably involve
a considerable quantity of research money.
Also, researchers with access to super-computers have about a 10 year
head start on researchers working with desktops - in terms of when
the required hardware becomes available.
This approach points the finger at governments, large companies and
Within governments, supercomputer power is concentrated mostly in the
intelligence agencies - such as the NSA in America.
These organisations also have some of the smartest folk on the planet
working for them - and face extremely-challenging problems.
Among companies, server-side computing giants have some of the biggest
computers. Search companies have an obvious need for superintelligent
machines - and are among the most likely companies to develop them.
These companies also have a lot of money and large research budgets.
Another possibility is that the development of machine intelligence
will be driven by investment management organisations who trade on the
Both of these areas currently use enormous computers running highly
The second clue relating to the problem considers the issue of
build-test cycle timing.
One desirable property of the driving problem is that it should have
a short build-test cycle - to facilitate rapid development.
Any proposed intelligent machine needs to operate using sensors,
processsing units and actuators.
However, using mechanical sensors, and actuators in the real
world would probably be slow and inefficient.
Fortunately, there is no good reason why the sensors, and actuators of
an intelligent machine need to extend very far into the "real" world.
Virtual worlds can be highly complex and interesting - and they can
also be made to operate rapidly.
So, for example, a proposed intelligent machine could extend sensors
and actuators onto the internet, and then perform operations at
However, even this approach seems rather likely to be bottlenecked
by network bandwidth issues. Why not extend sensors, and actuators
into an entirely synthetic universe, and dispense with the network
An example of this approach involves getting intelligent agents to
play the game of go. The universe of the game of go is rich and
complex, and apparently requires a deep and sophisticated intelligence
to successfully navigate it. However, it is entirely self-contained -
and can thus machine players be made to operate at very high speeds
entirely within the computer.
The core problem
The third and final clue relates to the consideration of the
core problem for a prospectively superintelligent machine.
A machine will not be superintelligent until it is better at
designing machines than humans are.
As with other automation tasks, eliminating humans from the loop
typically allows operation to be sped up considerably. So, it seems
likely that the development of intelligent machines will accelerate
when their development is perfomed primarily by other intelligent
machines. So, the problem of developing a superintelligence will be
mostly complete - once a designer of intelligent machines that is
itself an intelligent machine has been created.
Since this is - in some respects - the core problem, perhaps resources
and attention are best devoted directly to it. The problem is broadly
similar to that faced by high-level language designers: given a
high-level specification of the problem to be solved, produce
circuitry - or other machine code - which then solves the problem.
From this perspective, tasks such as playing go - and playing the stockmarket -
are seen as mostly distractions from the real underlying problem.
These clues pull in conflicting directions:
Playing the stockmarket potentially generates cash. A go-playing
program can be sold for something - while developing computer
programming language compilers typically produces something which is
extremely challenging to sell.
Naturally, cash is important - it can be used to buy more and better
hardware, and to pay the wages of researchers, programmers and
inventors. Unfortunately, the most financially-attractive paths seem
to be those that lead most indirectly towards the goal.
However, my expectation is that the strategy that produces
superintelligent machines will heed each of these three clues - with
financial considerations strongly influencing the most likely origin
of the agents, while the other issues influencing the details of how
the development takes place.
To return to the issue of which type of organisation is most likely to
host the first superintelligence - can much more be said about the
likely origin of such intelligent machines - besides that it is most
likely to happen in a government, corporation or organisation with
powerful computers and considerable research funds?
I think it is hard to say very much more on the topic at this stage
with much certainty. However, to my eyes, the NSA is well placed in
some respects - but I have some difficulty in imagining their budget
proposals for constructing a superintelligent agent. Also, if such an
organisation constructed a superintelligence, they might do their best
to keep it chained up in their underground basements for as long as
DARPA funds might contribute to the development of superintelligent
machines - but most of DARPA's projects seem like half-hearted side
dishes to me. Their "
Deep Green" battle computer is probably the nearest thing on their
menu to a superintelligent machine. DARPA don't show many signs of
trying hard to me.
The government claims a
long history of funding research into intelligent machines. They
did effectively invent ARPANET - the precursor to the
internet. I'm sure the other funds they have put into the field have
also helped. However, my perception is that the government's
main influence has been via the funding of academic research. I
am inclined to doubt whether that will be enough. It is difficult for
me to imagine the scenario in which a superintelligent machine arises
within a government department. It seems to me that most development
in the IT industry has effectively been done by companies.
So: I currently think that scenarios in which superintelligent
machines first arise within companies are the most realistic. Search
oracles seem to me to be the most likely suspect. Stockmarket traders
are probably the next in line, and perhaps after that conglomerate
companies - global giants with fingers in various different pies.
The company scenario
Though is seems probably to me, the "company" scenario is a worrying
one. What probably happens next is that the associated company will rapidly
expand until it pushes up against the limits set by the monopoly and mergers
commission. Then it will expand "horizontally", into whatever other
market areas it can find - until it runs into the limits there. Then
it will consider its options.
At that point, it will probably have mastered molecular nanotechnology,
and will have the ability to create autonomous living systems. Its
options are thus likely to be fairly diverse. Then, much will depend
on precisely what the intentions and values of its makers are.