A military officer said he had left the historic emblems in a rucksack in his car, while on his way to the airport.
The bag was later found by police in a local church after a tip-off.
The gold medal is studded with emeralds and dates to the country's foundation in 1825, when it was presented to national hero Simón Bolívar.
Quoted by AFP news agency, former Bolivian President Carlos Mesa compared the crime to "the theft of the crown of Queen Elizabeth II of England".
President Evo Morales was due to wear the emblems at the annual celebration of the armed forces on Wednesday but appeared without them.
The officer in question had made his way to El Alto airport, west of the capital La Paz, in order to fly to Cochabamba to give them to Mr Morales.
However, the officer missed his flight and the items disappeared while he waited for his rescheduled departure.
When he returned to his car, he found the rucksack was gone.
The guard has been arrested and an investigation is under way, the country's defence ministry says.
Containing 22-carat gold and almost 200 years old, the medal is usually kept with the sash in the vault at the Bank of Bolivia.
The items are only taken out for ceremonial use.