Duke students can take physics courses at nearby UNC Chapel Hill, which substantially increases the variety of possible physics courses. Travel back and forth between Duke and UNC is made convenient by the free Robertson Scholarship bus
that runs about every half hour between Duke and UNC.
You can take up to one course per semester at UNC-CH (or at any other of the local universities like NC State although UNC is by far the easiest to get to) as long as Duke does not offer an equivalent course. UNC's Physics Department especially has advanced undergraduate courses in condensed matter physics and astrophysics that are currently not offered at Duke.
Please see the UNC physics course offerings. Check out these courses and then sit down with the Director of Undergraduate Studies to see if one of them can fit into your educational plan.