What Does It Mean to 'Blow the Man Down' in the Song?

Have you ever worked together with friends to move something really heavy, like a big toy box? Maybe you all said, “One, two, three, lift” to time you ...