Pancreatic organogenesis is promoted or restricted by different signaling pathways. In amniotes, inhibition of hedgehog (Hh) activity in the early embryonic endoderm is a prerequisite for pancreatic specification. However, in zebrafish, loss of Hh signaling leads to a severe reduction of β-cells, leading to some ambiguity as to the role of Hh during pancreas development and whether its function has completely diverged between species. Here, we have employed genetic and pharmacological manipulations to temporally delineate the role of Hh in zebrafish endocrine pancreas development and investigate its relationship with the Bmp and retinoic acid (RA) signaling pathways. We found that Hh is required at the start of gastrulation for the medial migration and differentiation of pdx1-expressing pancreatic progenitors at later stages. This early positive role of Hh promotes β-cell lineage differentiation by restricting the repressive effects of Bmp. Inhibition of Bmp signaling in the early gastrula leads to increased β-cell numbers and partially rescued β-cell formation in Hh-deficient embryos. By the end of gastrulation, Hh switches to a negative role by antagonizing RA-mediated specification of the endocrine pancreas, but continues to promote differentiation of exocrine progenitors. We show that RA downregulates the Hh signaling components ptc1 and smo in endodermal explants, indicating a possible molecular mechanism for blocking axial mesoderm-derived Hh ligands from the prepancreatic endoderm during the specification stage. These results identify multiple sequential roles for Hh in pancreas development and highlight an unexpected antagonistic relationship between Hh and other signaling pathways to control pancreatic specification and differentiation.