Two dimensional materials are usually envisioned as flat, truly 2D layers. However out-of-plane corrugations are inevitably present in these materials. In this manuscript, we show that graphene flakes encapsulated between insulating crystals (hBN, WSe2), although having large mobilities, surprisingly contain out-of-plane corrugations. The height fluctuations of these corrugations are revealed using weak localization measurements in the presence of a static in-plane magnetic field. Due to the random out-of-plane corrugations, the in-plane magnetic field results in a random out-of-plane component to the local graphene plane, which leads to a substantial decrease of the phase coherence time. Atomic force microscope measurements also confirm a long range height modulation present in these crystals. Our results suggest that phase coherent transport experiments relying on purely in-plane magnetic fields in van der Waals heterostructures have to be taken with serious care.