We report a thermally activated metastability in a GaAs double quantum dot exhibiting real-time charge switching in diamond shaped regions of the charge stability diagram. Accidental charge traps and sensor backaction are excluded as the origin of the switching. We present an extension of the canonical double dot theory based on an intrinsic, thermal electron exchange process through the reservoirs, giving excellent agreement with the experiment. The electron spin is randomized by the exchange process, thus facilitating fast, gate-controlled spin initialization. At the same time, this process sets an intrinsic upper limit to the spin relaxation time.