The artisans excavated these caves out of the vertical cliff above the Waghora riverbank in the Ajanta Hills.

Ajanta caves were created by scooping into the vertical surface of the rocky hill, much like a single-ended, short tunnel. What astonishes me is the sophistication and aesthetic zeal with which these caves were made. Everything from the pillars to sculptures to arches to the stupa has been created out of sheer carving work. Nothing is added except lime plaster for making the painting.

Location selection

The real estate for rock cut architecture is the vertical face of a cliff or hill. Not the flatland used in the traditional construction.
The cliff here was formed out of gigantic and almost monolithic rock mountains formed out of volcanic eruption. The horseshoe shaped Ajanta hill had everything ideal for rock cut architecture - plenty of vertical surface with deep rocky formation.

Techniques

The outline for the cave design is marked first on the vertical cliff face. Then the excavation is started from the top, that is, from the ceiling level downward. This is in contrast with the regular construction sequence, where the work starts from the base.

Once the ceiling portion is finished, the excavation moves downward progressively, finishing the walls and finally the floor.
So in an unfinished cave, you'll see the ceiling complete, and the floor is yet to be excavated.

Unfinished caves

The unfinished caves (caves 5, 24, and 29) provide excellent evidence of the techniques and methodology employed in rock excavation. These give a good idea of the various stages and methods by which the excavation was done.

From the tool marks on the walls, archaeologists could even deduce the types of chisels and pickaxes they used to construct, or rather carve, the Ajanta caves.

In the unfinished Ajanta Caves 3 and 5, you can see the method of excavation. A grid of deep alleys was cut on the excavation floor using pickaxes. These blocks were then removed one by one for further chipping operations.

Once the ceiling, walls, and floor were done, the excavation continued on the walls to create chambers and cells.

The only portions that are left untouched are parts of the cave design, like the pillars, stupas, or portions that would later be carved into sculptural and architectural elements. In other words, unwanted rock is removed to create the space that is finally left behind in the cave.

Once the roughly finished cave was ready, the artisans started their work on carving the details.

Carvings

While scooping out the bulk of the rock, the most dexterous work is carning the finer details and the final touch. By nature, the rock in Ajanta is porous, unlike marble, granite, or soapstone used in other parts of Indian architecture.

One mistake or an extra chipway, there is no way to make an amendment!

Finally...

In short, in this architecture they just taken out what's not needed!

Where did all the tons of excavated debris go? Deep down into the Waghora River valley. The torrential monsoons washed it away. So, in Ajanta, you’ll not see heaps of debris created as a result of the excavation.