DSGE models are a prominent tool for forecasting at central banks and the competitive forecasting performance of these models relative to alternatives--including official forecasts--has been documented. When evaluating DSGE models on an absolute basis, however, we find that the benchmark estimated medium scale DSGE model forecasts inflation and GDP growth very poorly, although statistical and judgmental forecasts forecast as poorly. Our finding is the DSGE model analogue of the literature documenting the recent poor performance of macroeconomic forecasts relative to simple naive forecasts since the onset of the Great Moderation. While this finding is broadly consistent with the DSGE model we employ--ie, the model itself implies that under strong monetary policy especially inflation deviations should be unpredictable--a "wrong" model may also have the same implication. We therefore argue that forecasting ability during the Great Moderation is not a good metric to judge the usefulness of model forecasts.