The whole city was shaken by the blast, which began with a fire at the port which exploded into a mushroom cloud.
President Michel Aoun said 2,750 tonnes of ammonium nitrate had been stored unsafely in a warehouse for six years.
He scheduled an urgent cabinet meeting for Wednesday, and said a two-week state of emergency should be declared.
The country will observe an official period of mourning for three days from Wednesday.
President Aoun also announced that the government would release 100 billion lira (£50.5m; $66m) of emergency funds.
"What we are witnessing is a huge catastrophe," the head of Lebanon's Red Cross George Kettani told local media. "There are victims and casualties everywhere."
Rescue workers are continuing to search through rubble and the death toll is expected to rise.