Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

Nvidia
on
Monday
announced
a
new
generation
of
artificial
intelligence
chips
and
software
for
running
artificial
intelligence
models.
The
announcement,
made
during
Nvidia’s
developer’s
conference
in
San
Jose,
comes
as
the
chipmaker
seeks
to
solidify
its
position
as
the
go-to
supplier
for
AI
companies.

Nvidia’s
share
price
is
up
five-fold
and
total
sales
have
more
than
tripled
since
OpenAI’s
ChatGPT
kicked
off
the
AI
boom
in
late
2022.
Nvidia’s
high-end
server
GPUs
are
essential
for
training
and
deploying
large
AI
models.
Companies
like


Microsoft
and


Meta
have
spent
billions
of
dollars
buying
the
chips.

The
new
generation
of
AI
graphics
processors
is
named
Blackwell.
The
first
Blackwell
chip
is
called
the
GB200
and
will
ship
later
this
year.
Nvidia
is
enticing
its
customers
with
more
powerful
chips
to
spur
new
orders.
Companies
and
software
makers,
for
example,
are
still
scrambling
to
get
their
hands
on
the
current
generation
of
“Hopper”
H100s
and
similar
chips.

“Hopper
is
fantastic,
but
we
need
bigger
GPUs,”
Nvidia
CEO
Jensen
Huang
said
on
Monday
at
the
company’s
developer
conference
in
California.

Nvidia
shares
fell
more
than
1%
in
extended
trading
on
Monday.

The
company
also
introduced
revenue-generating
software
called
NIM
that
will
make
it
easier
to
deploy
AI,
giving
customers
another
reason
to
stick
with
Nvidia
chips
over

a
rising
field
of
competitors.

Nvidia
executives
say
that
the
company
is
becoming
less
of
a
mercenary
chip
provider
and
more
of
a
platform
provider,
like
Microsoft
or
Apple,
on
which
other
companies
can
build
software.

“Blackwell’s
not
a
chip,
it’s
the
name
of
a
platform,”
Huang
said.

“The
sellable
commercial
product
was
the
GPU
and
the
software
was
all
to
help
people
use
the
GPU
in
different
ways,”
said
Nvidia
enterprise
VP
Manuvir
Das
in
an
interview.
“Of
course,
we
still
do
that.
But
what’s
really
changed
is,
we
really
have
a
commercial
software
business
now.”

Das
said
Nvidia’s
new
software
will
make
it
easier
to
run
programs
on
any
of
Nvidia’s
GPUs,
even
older
ones
that
might
be
better
suited
for
deploying
but
not
building
AI.

“If
you’re
a
developer,
you’ve
got
an
interesting
model
you
want
people
to
adopt,
if
you
put
it
in
a
NIM,
we’ll
make
sure
that
it’s
runnable
on
all
our
GPUs,
so
you
reach
a
lot
of
people,”
Das
said.

Every
two
years
Nvidia
updates
its
GPU
architecture,
unlocking
a
big
jump
in
performance.
Many
of
the
AI
models
released
over
the
past
year
were
trained
on
the
company’s
Hopper
architecture
—
used
by
chips
such
as
the
H100
—
which
was
announced
in
2022.

Nvidia
says
Blackwell-based
processors,
like
the
GB200,
offer
a
huge
performance
upgrade
for
AI
companies,
with
20
petaflops
in
AI
performance
versus
4
petaflops
for
the
H100.
The
additional
processing
power
will
enable
AI
companies
to
train
bigger
and
more
intricate
models,
Nvidia
said.

The
chip
includes
what
Nvidia
calls
a
“transformer
engine
specifically
built
to
run
transformers-based
AI,
one
of
the
core
technologies
underpinning
ChatGPT.

The
Blackwell
GPU
is
large
and
combines
two
separately
manufactured
dies
into
one
chip
manufactured
by


TSMC.
It
will
also
be
available
as
an
entire
server
called
the
GB200
NVLink
2,
combining
72
Blackwell
GPUs
and
other
Nvidia
parts
designed
to
train
AI
models.

Amazon,


Google,


Microsoft,
and


Oracle
will
sell
access
to
the
GB200
through
cloud
services.
The
GB200
pairs
two
B200
Blackwell
GPUs
with
one
Arm-based
Grace
CPU.
Nvidia
said
Amazon
Web
Services
would
build
a
server
cluster
with
20,000
GB200
chips.

Nvidia
said
that
the
system
can
deploy
a
27-trillion-parameter
model.
That’s
much
larger
than
even
the
biggest
models,
such
as
GPT-4,
which
reportedly
has
1.7
trillion
parameters.
Many
artificial
intelligence
researchers
believe
bigger
models
with
more
parameters
and
data

could
unlock
new
capabilities.

Nvidia
didn’t
provide
a
cost
for
the
new
GB200
or
the
systems
it’s
used
in.
Nvidia’s
Hopper-based
H100
costs
between
$25,000
and
$40,000
per
chip,
with
whole
systems
that
cost
as
much
as
$200,000,
according
to
analyst
estimates.

Nvidia
also
announced
it’s
adding
a
new
product
named
NIM,
which
stands
for
Nvidia
Inference
Microservice,
to
its
Nvidia
enterprise
software
subscription.

NIM
makes
it
easier
to
use
older
Nvidia
GPUs
for
inference,
or
the
process
of
running
AI
software,
and
will
allow
companies
to
continue
to
use
the
hundreds
of
millions
of
Nvidia
GPUs
they
already
own.
Inference
requires
less
computational
power
than
the
initial
training
of
a
new
AI
model.
NIM
enables
companies
that
want
to
run
their
own
AI
models,
instead
of
buying
access
to
AI
results
as
a
service
from
companies
like
OpenAI.

The
strategy
is
to
get
customers
who
buy
Nvidia-based
servers
to
sign
up
for
Nvidia
enterprise,
which
costs
$4,500
per
GPU
per
year
for
a
license.

Nvidia
will
work
with
AI
companies
like
Microsoft
or
Hugging
Face
to
ensure
their
AI
models
are
tuned
to
run
on
all
compatible
Nvidia
chips.
Then,
using
a
NIM,
developers
can
efficiently
run
the
model
on
their
own
servers
or
cloud-based
Nvidia
servers
without
a
lengthy
configuration
process.

“In
my
code,
where
I
was
calling
into
OpenAI,
I
will
replace
one
line
of
code
to
point
it
to
this
NIM
that
I
got
from
Nvidia
instead,”
Das
said.

Nvidia
says
the
software
will
also
help
AI
run
on
GPU-equipped
laptops,
instead
of
on
servers
in
the
cloud.