The Albanian has been deemed surplus to requirements and the Blues are aiming to offload him for around £30m this month.
Ipswich recently expressed an interest in negotiating a transfer deal, but Broja has made it clear that he does not wish to join them.
The Tractor Boys are firm favourites to finish in the bottom three and Broja does not seem keen on fighting a relegation battle.
Chelsea will now need to find other suitors in the transfer market. Monaco were interested in him earlier this summer, but are no longer keen.
The Blues may have to reduce their valuation to around £20m in order to invite more offers on the table before the deadline.
Aside from Broja, Trevoh Chalobah, David Datro Fofana and Romelu Lukaku are expected to be sold in the near future.
Chelsea are desperate for funds to sign a marquee striker.
