On 2015 March 23, the Very Energetic Radiation Imaging Telescope Array System (VERITAS) responded to a Swift-Burst Alert Telescope (BAT) detection of a gamma-ray burst, with observations beginning 270 s after the onset of BAT emission, and only 135 s after the main BAT emission peak. No statistically significant signal is detected above 140 GeV. The VERITAS upper limit on the fluence in a 40-minute integration corresponds to about 1% of the prompt fluence. Our limit is particularly significant because the very-high-energy (VHE) observation started only similar to 2 minutes after the prompt emission peaked, and Fermi-Large Area Telescope observations of numerous other bursts have revealed that the high-energy emission is typically delayed relative to the prompt radiation and lasts significantly longer. Also, the proximity of GRB 150323A (z = 0.593) limits the attenuation by the extragalactic background light to similar to 50% at 100-200 GeV. We conclude that GRB 150323A had an intrinsically very weak high-energy afterglow, or that the GeV spectrum had a turnover below similar to 100 GeV. If the GRB exploded into the stellar wind of a massive progenitor, the VHE non-detection constrains the wind density parameter to be A greater than or similar to 3 x 10(11) g . cm(-1), consistent with a standard Wolf-Rayet progenitor. Alternatively, the VHE emission from the blast wave would be weak in a very tenuous medium such as the interstellar medium, which therefore cannot be ruled out as the environment of GRB 150323A.