That is effectively what a USA Today article implied when it told readers that:

"U.S. [population] growth is the envy of most developed nations."

The article implied that countries with stagnant or declining populations will experience economic hardship as a result. In fact, the impact of productivity growth in raising living standards is an order of magnitude greater than whatever drag demographics, in the form of rising dependency ratios, might be in slowing the growth of living standards. Furthermore, lower populations may directly improve living standards by reducing congestion and pollution and increasing the ratio of capital to labor. This is especially likely to be the case in the densely populated countries of Europe and Japan.