Two spots in the Baseball at the 2004 Summer Olympics were available to baseball teams of the Pan American Baseball Confederation. To decide which teams would qualify, a tournament was held in 2003 in Panama, from 30 October to 10 November in the Estadio Nacional de Panamá and the Estadio Juan Demóstenes Arosemena in Panama City and the Estadio José Antonio Remón Cantera in Aguadulce. The tournament was won by Cuba, with Canada also qualifying for the Olympics. Notably, the United States team did not qualify, after losing to Mexico in the quarterfinals. This elicited shock in the American media as the USA was the previous gold medal winner, and was the first country to play baseball and is the country where baseball has the largest following.

Withdrawals

Although the tournament was originally scheduled to include 13 teams, four withdrew before their first game: Venezuela, the Dominican Republic, Aruba and the Bahamas. As the Bahamas were a last-minute withdrawal, the tournament structure was left unbalanced, with one pool larger than the other.

Pool Play

Pool A

Pool B

The game between the and was delayed due to weather. Before it could be played it became irrelevant to the standings, and was therefore cancelled.

Elimination