The temple is named after the mythological figure Hidimba, a Rakshasa (demon) from the epic Mahabharata.
According to legend, Hidimba resided in the forest near the fort, and his sister, Hidimbi, married Bhima, one of the Pandava brothers.
It is believed that Hidimba meditated at this site and eventually became a devotee of Lord Shiva. The temple is dedicated to him as Hidimbeshwara, a form of Lord Shiva.
Special rituals and festivals are celebrated on auspicious days dedicated to Lord Shiva, such as Maha Shivaratri, when numerous pilgrims visit the temple to offer their prayers.