Local media have speculated that further poll blows could lead to a change of party leadership.
For months, public support for Ishiba, who took office in October, has been at rock bottom, thanks in part to rising inflation and soaring rice costs on his watch.
Polls this month showed a slight uptick in support, thanks in part to new policies to tackle the rice price issue.
Polling stations will close at 8.00pm (1100 GMT).
A record 295 candidates were running in Sunday’s election, the highest since 1997, including 99 women candidates, the highest ever, which officials have hailed as a “delightful trend”.
Japan remained at 118th place out of 148 countries in the World Economic Forum’s Global Gender Gap Index ranking this year, as women’s participation in politics was particularly low.
