Work, moments of work, and heat flux are studied for the generic case of a strongly driven two-level system immersed in a bosonic heat bath in domains of parameter space where perturbative treatments fail. This includes in particular the interplay between non-Markovian dynamics and moderate to strong external driving. Exact data are compared with predictions from weak-coupling approaches. Further, the role of system-bath correlations in the initial thermal state and their impact on the heat flux are addressed. The relevance of these results for current experimental activities on solid-state devices is discussed.