Those rats you just scared went up and freaked out the prize elephants! They're now outside, rampaging around the palace!
― Bentley, after Sly released the elephants[src]
Sly needed to steal six gems off of Rajan's prize elephants.
Bentley needed Sly to retrieve six gems from the headdresses of two of Rajan's prize elephants. They would be used to create a saw durable enough to cut the Clockwerk wings from the statue in the ballroom.
To get the elephants out of their pen, Sly crawled underneath a hole beneath the steps to find an alternate route. As he crawled further up, he frightened two rats, causing them to go up and freak out the elephants. They barged through the door and began rampaging around the palace grounds. Sly crawled back out and spots one elephant, noting how spooked he is.
The next objective was getting on top of both elephants and getting the headdress jewels off of them. However, the jewels were firmly embedded, so Sly needed to whack at them with his cane to knock them loose one-by-one. After Sly accomplished the task, the elephants returned to their pen.