The phrase all fours refers to a poker hand in a game of five-card draw poker played with a single, standard, four-suit deck of cards. The probability that you will have the winning hand is exceptionally high if you have an all fours hand because it means that all five cards that you are holding are fours. Obviously, five of a kind beats two, three and four of a kind, so this is truly a charmed hand, indeed.
Of course, holding an all fours hand is also exceptionally likely to get you maimed or killed if real money is being gambled on the game. When that happens—i.e., when, after flashing an all fours hand, you find yourself on the floor beaten or dead (or, rather, someone else finds you dead because one can rarely find one’s self in that state) and motionless (which is very common during death) you are said to be “down on all fours.”