Long gamma-ray bursts (GRBs), among the most energetic events in the Universe, are explosions of massive and short-lived stars, so they pinpoint locations of recent star formation. However, several GRB host galaxies have recently been found to be deficient in molecular gas (H2), believed to be the fuel of star formation. Moreover, optical spectroscopy of GRB afterglows implies that the molecular phase constitutes only a small fraction of the gas along the GRB line-of-sight. Here we report the first ever 21 cm line observations of GRB host galaxies, using the Australia Telescope Compact Array, implying high levels of atomic hydrogen (HI), which suggests that the connection between atomic gas and star formation is stronger than previously thought, with star formation being potentially directly fuelled by atomic gas (or with very efficient HI-to-H2 conversion and rapid exhaustion of molecular gas), as has been theoretically shown to be possible. This can happen in low metallicity gas near the onset of star formation, because cooling of gas (necessary for star formation) is faster than the HI-to-H2 conversion. Indeed, large atomic gas reservoirs, together with low molecular gas masses, stellar and dust masses are consistent with GRB hosts being preferentially galaxies which have very recently started a star formation episode after accreting metal-poor gas from the intergalactic medium. This provides a natural route for forming GRBs in low-metallicity environments. The gas inflow scenario is also consistent with the existence of the companion HI object with no optical counterpart ~19 kpc from the GRB 060505 host, and with the fact that the HI centroids of the GRB 980425 and 060505 hosts do not coincide with optical centres of these galaxies, but are located close to the GRB positions.