Two men reportedly were caught smashing the ceramic tiles that cover the tomb's stone late on Wednesday night.
The arrested haredi Orthodox man, 30, told police that he came to the tomb in order to pray for a mate. He said that a second man, who fled the scene before he could be nabbed by police, told him that the only way for his prayers to be answered is if he prayed directly to the stones.
The two men then began cracking and removing the tiles, which are hundreds of years old.
The tomb recently was renovated including restoring the original structure and repairing the tiles.
Scholars doubt that King David is actually buried at the site.