Bangladesh’s finance minister downplayed the impact of last week’s factory-building collapse on his country’s garment industry, saying  he didn’t think it was “really serious” hours after the 500th body was pulled from the debris.

Finance Minister Abul Maal Abdul Muhith spoke as the government cracked down on those it blamed for the disaster in the Dhaka suburb of Savar. It suspended Savar’s mayor and arrested an engineer who had called for the building’s evacuation last week, but was also accused of helping the owner add three illegal floors to the eight-story structure. The building owner was arrested earlier.

The government appears to be attempting to fend off accusations that it is in part to blame for the tragedy because of weak oversight of the building’s construction.

The official death toll from the April 24 collapse reached 512 Friday and was expected to climb, making it likely the deadliest garment-factory accident in world history.

At the site of the collapse, workers carefully used cranes to remove the concrete rubble and continue the slow task of recovering bodies.

The official number of missing has been 149 since Wednesday, though unofficial estimates are higher.