A light highly-prefabricated panel over a heavy handmade wall, a contradiction that works.
A heavy concrete panel will imply having mass on the outside of the thermal insulation, as those solutions are commonly insulated from the inside. If the objective is having this thermal inertia in the inner side of the insulation, the solution is good: a heavy interior wall insulated over its outer face and a low-weight panel so as not to overload the façade where it is not necessary.