Both hold their shape when dry
When insulating the domes of earth ovens, I have used a mixture of 6 parts perlite to 1 part clay slip, and I have also used a mixture of 6 parts perlite to 1 part Portland cement. The less clay or cement you use the better, because they both reduce the insulation value of the perlite, but unless you take extraordinary measures, you need some sort of binder to hold the perlite in place.
Only the perlite with cement holds its shape when wet
An important thing to consider is moisture. Perlite bound with cement holds together when wet, but perlite bound with clay slip does not.