The White City has it all
Arequipa is a compact city, with great hotels and hostels near the historic center of the city.
You can find everything from nice hotels to cheap hostels near the center. Another neighborhood to consider is Yanama, which is more modern.
This is just across Puente Grau from the Santa Catalina Monastery.