Ceramic On-Demand Extrusion (CODE) is an extrusion-based additive manufacturing process recently developed for fabricating dense, functional ceramic components. This paper presents a further development of this process and focuses on fabricating 3 mol% yttria-stabilized zirconia (3YSZ) components that cannot be fabricated without using support structures. The 3YSZ paste is deposited through the main nozzle, and a polycaprolactone (PCL) pellet feedstock is melted and deposited through an auxiliary nozzle to build support structures. After a green part is printed and dried, the support structures are removed by heating the part to ~70 °C to melt the PCL. The part is then sintered at 1550 o C to achieve near theoretical density. The maximum angle of overhanging feature that can be fabricated without support was determined to be 60°. Sample parts were fabricated and evaluated to demonstrate the effectiveness of the PCL support material and CODE’s capability to fabricate geometrically complex parts.