Observations using the 7 mm receiver system on the Australia Telescope Compact Array have revealed large reservoirs of molecular gas in two high-redshift radio galaxies: HATLAS J090426.9+015448 (z = 2.37) and HATLAS J140930.4+003803 (z = 2.04). Optically the targets are very faint, and spectroscopy classifies them as narrow-line radio galaxies. In addition to harbouring an active galactic nucleus the targets share many characteristics of sub-mm galaxies. Far-infrared data from Herschel-ATLAS suggest high levels of dust (>10^9 M_solar) and a correspondingly large amount of obscured star formation (~1000 M_solar / yr). The molecular gas is traced via the J = 1-0 transition of 12CO, its luminosity implying total H_2 masses of (1.7 +/- 0.3) x 10^11 and (9.5 +/- 2.4) x 10^10 (alpha_CO/0.8) M_solar in HATLAS J090426.9+015448 and HATLAS J140930.4+003803 respectively. Both galaxies exhibit molecular line emission over a broad (~1000 km/s) velocity range, and feature double-peaked profiles. We interpret this as evidence of either a large rotating disk or an on-going merger. Gas depletion timescales are ~100 Myr. The 1.4 GHz radio luminosities of our targets place them close to the break in the luminosity function. As such they represent `typical' z > 2 radio sources, responsible for the bulk of the energy emitted at radio wavelengths from accretion-powered sources at high redshift, and yet they rank amongst the most massive systems in terms of molecular gas and dust content. We also detect 115 GHz rest-frame continuum emission, indicating a very steep high-radio-frequency spectrum, possibly classifying the targets as compact steep spectrum objects.