A company source told AFP that the missing aircraft was a McDonnell Douglas DC-9 and that some 110 people of various nationalities are listed as being on board the flight.
The source said contact with the aircraft was lost while it was still in Malian airspace approaching the border with Algeria.
Despite international military intervention still under way, the situation remains unstable in northern Mali, which was seized by jihadist groups for several months in 2012.
On July 17, the Bamako government and armed groups from northern Mali launched tough talks in Algiers aimed at securing an elusive peace deal, and with parts of the country still mired in conflict.
"The plane was not far from the Algerian frontier when the crew was asked to make a detour because of poor visibility and to prevent the risk of collision with another aircraft on the Algiers-Bamako route," the Air Algerie source said.
"Contact was lost after the change of course."
The airline announced that the plane had gone missing in a brief statement carried by national news agency APS.
"Air navigation services have lost contact with an Air Algerie plane Thursday flying from Ouagadougou to Algiers, 50 minutes after takeoff," the statement said.
It added that the company initiated an "emergency plan" in the search for flight AH5017, which flies the four-hour passenger route four times a week.
One of Algeria's worst air disasters occurred in February this year, when a C-130 military aircraft carrying 78 people crashed in poor weather in the mountainous northeast, killing more than 70 people.
The plane was flying from the desert garrison town of Tamanrasset in Algeria's deep south to Constantine, 320 kilometres (200 miles) east of Algiers.
Tamanrasset was the site of the country's worst ever civilian air disaster, in March 2003.
In that accident, all but one of 103 people on board were killed when an Air Algerie passenger plane crashed on takeoff after one of its engines caught fire.
The sole survivor, a young Algerian soldier, was critically injured.
In December 2012, two Algerian military jets on a routine training mission collided in mid-air near Tlemcen in the northwest, killing both pilots.
A month earlier, a twin-turboprop CASA C-295 military transport aircraft, which was carrying a cargo of paper for the printing of banknotes in Algeria, crashed in southern France.
The five soldiers and one central bank representative on board were all killed.