The percentage of American workers ages 25 to 54 declined this year, and that’s not a good sign for the economy, experts say.